Instructables

USB Biofeedback Game Controller

FeaturedContest Winner
Picture of USB Biofeedback Game Controller
DSC01501.JPG
DSC01451.JPG
Building upon our last tutorial, which taught you how to build a muscle sensor, we've decided to give you a project that really showcases the power of our muscle (EMG) sensors. Advancer Technologies’ “USB Biofeedback Game Controller” harnesses the power of electromyography to allow players to directly control video games with their muscles. At the core of this plug-in-play controller is the powerful yet low-cost Arduino UNO microcontroller, a favorite among hobbyist and students, acting as a HID keyboard interface. Integrated with the Arduino UNO are four of our Muscle (EMG) Sensors, allowing four muscles to act independently or in combination with each other to control over four buttons. In our setup, we elected to use a six button setup with the left forearm controlling the B button (RUN/ATTACK), the right forearm controlling the A button (JUMP), the left bicep controlling the LEFT button, the right bicep controlling the RIGHT button, and combinations for UP and DOWN.

This tutorial will teach you to build a USB Biofeedback Game Controller. Use it to play any computer game (that uses keyboard inputs) using your muscles as the controller.




Check out our previous Instructable to learn how to build your own Muscle Sensor

or

Drop by our webstore to purchase muscle sensors, kits, cables and electrodes
Muscle Sensor Kit (now also on SparkFun)
Muscle Sensor Electrodes

Note: This sensor is not intended for use in the diagnosis of disease or other conditions, or in the cure, mitigation treatment, or prevention of disease, in a man or other animals.

About Advancer Technologies
Advancer Technologies is a company devoted to developing innovative game-changing biomedical and biomechanical technologies and applied sciences. Additionally, Advancer Technologies promotes all forms of interest and learning into biomedical technologies. To help cultivate and educate future great minds and concepts in the field, they frequently post informative instructions on some of their technologies. For more information, please visit www.AdvancerTechnologies.com .
 
Remove these adsRemove these ads by Signing Up

Step 1: Materials

Picture of Materials
Basics
1 x Arduino Uno R2 (needs the atmega8u2 USB chip which is only available on newer Arduino MCUs)
1 x Arduino Project Enclosure
1 x USB cable for your Arduino
4 x Advancer Technologies Muscle Sensor 
1 x +/-5V Power Supply
1 x 12V Power Supply (Wall wart) 
1 x Muscle Sensor Electrodes

Misc.
Jumper cables or solid core wire
3 x 1" length pieces of 24-30 AWG wire (1/8" stripped off each end)
3 x 2" length pieces of 24-30 AWG wire (1/8" stripped off each end)
3 x 3" length pieces of 24-30 AWG wire (1/8" stripped off each end)
3 x 4" length pieces of 24-30 AWG wire (1/8" stripped off each end)
3 x 3 position Female Receptacle 
1 x 3 position Female Housing
3 x Crimps for Female Housing
Plastic board (ABS)
1 pack x 3/8" 4-40 Screws (91772A108)
2 packs x 4-40 Nuts

Tools
Soldering Iron
Scissors or Tin Snips
Crimping tool
Screw driver
Drill with 1/8" bit
Sandpaper
Dremel tool or other cutting tool

Software
Arduino
Processing

Step 2: Constructing the plastic insert

Picture of Constructing the plastic insert
First we will need to create a insert to attach the Power Supply board and three of the Muscle Sensors. This insert will fit inside the project enclosure and rest on top of the Arduino UNO and the last muscle sensor.

If you're using relatively thin plastic board material, you can probably use a set of heavy duty scissors or tin snips to cut out the insert.

The dimensions (in inches) are shown in the drawing above and are approximate. You might need to do some sanding for a better fit.

After you construct the insert, try sliding it into the bottom of the enclosure and putting the top on. If it doesn't fit properly, get the sand paper out and sand down the parts that are interfering. 

Step 3: Preparing the bottom of the project enclosure

Next we'll prepare the Arduino project enclosure. 

First, trim down the two middle supports on the bottom of the enclosure. Leave enough height so that they will still go through your Arduino mounting holes but not go past the Arduino headers. (Pic 1)

Next, place one of the muscle sensors in the bottom left of the enclosure by sliding the bottom left support through the muscle sensor's mounting hole (the one near the female headers). Straighten the sensor by rotating it around the support such that its sides are parallel with the enclosures. Gently and such that you don't jostle the sensor board, slide your plastic insert onto the enclosure and hold it above the sensor. Now, look down the rectangular cutout on your insert and verify the three male headers of your sensor are directly below. If not, remove the insert, adjust the sensor position and repeat until it is in position. Once it is in position, mark the location of the muscle sensor's second mounting pin, remove the sensor, and drill the hole. 

Note: If you don't get the hole in exactly the right position, it's OK you can just widen the cutout more to compensate.

Step 4: Preparing the top of the project enclosure

Next, you'll need to modify the top portion of the enclosure as well. Using a Dremel tool or some other cutting tool, completely cut off the reinforcement wall near the front of the enclosure (see picture). Additionally, completely cut off the two middle supports that complement the supports you cut on the bottom portion of the enclosure. These both need to be removed to give us some more head room for the muscle sensors and plastic insert.

Step 5: Soldering the power supply connectors

We now need to create a way to connect the Power Supply board to each of the sensors. Grab your power supply board, the four sets of wire lengths, and the female housing and receptacles.

On the Power Supply board, there is a grid of through holes in a 3 col x 4 row pattern. Solder the three 3" wires to the holes in row 1, solder the three 2" wires to the holes in row 2, solder the three 1" wires to row 3, and solder the 4" wires to the holes in row 4.

Next, grab your 3 female receptacles and bend all the solder tabs so that they're at a 90 degree angle. For the 1"-3" wire sets, solder the female receptacles (with the ports facing downward) such that the wire soldered to the +5V column is soldered to the right solder tab, the GND wire is soldered to the middle solder tab, and the -5V wire is soldered to the left solder tab. For reinforcement, dab a glob of hot glue on the joints.

Using a crimping tool (you can also choose to solder instead of crimping), crimp the crimp tips to each of the 4" wires with the tabs pointing upward. Now insert each of the wires' crimp tips into the female housing, again following the +5V right, GND middle, -5V left order.

Your power supply board should be ready to go. double check each solder and crimp to make sure they'll good and solid.

Step 6: Mounting the muscle sensors and power supply board

Now it is time to mount your sensors using the 4-40 screws and nuts.

Grab three of your sensors and the power supply board and put a screw into each mounting hole. Thread a nut on each screw but leave it slightly loose. These nuts will be used as standoffs. 

Next, place each sensor's mounting screw into each corresponding mounting hole. Remember to keep the muscle sensors' female headers on the same side as the shelf's offset. Thread another nut onto each screw until they are nice and tight. Tighten more with a screw driver and a wrench, holding the second nut secure.

Next, follow this same process for the power supply board. Make sure the power jack is facing the front side of the shelf.

For the last muscle sensor, insert a screw up through the bottom of the enclosure and secure with a nut. Then slide the muscle sensor down the support column and insert the screw into the sensor's mounting hole and secure with a nut.

Check to make sure you can fit the female housing through the gap between the sensors and down into the rectangular cutout and onto the male headers of the sensor below. If the gap is too tight, remove one of the top sensors and sand one of its mounting holes on the shelf until the gap is wide enough.


For the remainder of these instructions we're going to refer to the sensors as follows:
Sensor 1 = Closest to Power Supply
Sensor 2 = Middle
Sensor 3 = End
Sensor 4 = Bottom

Step 7: Connecting the power supply to the muscle sensors

Picture of Connecting the power supply to the muscle sensors
By this time, your muscle sensors and power supply should be mounted and your Arduino should be in the project enclosure. Now you should be ready to start connecting everything together and setup your electrodes. 

First you'll want to give all your sensors power. Insert the plastic insert into the project enclosure and then connect each female receptacle and housing to the corresponding sensors. (Hint... the 1" wire goes to the sensor closest to the power supply, the 2" wire goes to the middle sensor on the insert, the 3" wire to the last sensor on the insert, and the 4" wire goes through the rectangular cutout to the sensor on bottom).

MAKE SURE THE CORRECT VOLTAGES ARE CONNECTED TO THE CORRESPONDING SENSOR PINS (+Vs = +5V, GND = GND, -Vs = -5V). IF NOT, YOU CAN POTENTIALLY BURN OUT YOUR SENSOR BOARD.

DOUBLE CHECK.... BETTER SAFE THAN SORRY!

Step 8: Decide which muscles are going to control which button(s)

Picture of Decide which muscles are going to control which button(s)
Now, you're going to have to decide which muscles/muscle combos you want to use for your button(s) layout. We've opted to go with the following layout. If you want to go with a different layout, you'll have to make some modifications to the Arduino code we'll get to later on.

Function buttons:
Button A --> Right forearm muscle
Button B --> Left forearm muscle
Start Button --> Not muscle controlled
Select Button --> Not muscle controlled

Directonal pad:
Right  --> Right bicep muscle
Left --> Left bicep muscle
Up --> All four muscles (combo)
Down --> Left and right bicep muscles (combo)

Step 9: Attaching the electrodes to your muscles

Next, you'll need to attach the electrodes to your muscles. The image above illustrates where to place the electrodes for our button layout. You can use any muscle you want (bigger muscles have stronger signals and surface muscles are easier to detect) but for our example we've gone with the ones shown.

Check out this video for a quick overview of how to connect the electrodes to your bicep...

Step 10: Connecting the electrodes to the muscle sensor(s)

Picture of Connecting the electrodes to the muscle sensor(s)
Now, we'll attach the electrodes to each muscle sensor using the EMG cables.

Sensor 1
Dark green- M.Mid Pin
Light green - M.End Pin

Sensor 2
Dark pink- M.Mid Pin
Light pink - M.End Pin

Sensor 3
Dark red - M.Mid Pin
Light red - M.End Pin

Sensor 4
Dark blue- M.Mid Pin
Light blue - M.End Pin

Each muscle sensor needs a reference electrode to be connected to the Ref pin. For our example, we are going to place them on the back of the forearm near the elbow. The reference electrodes should be placed on electrically neutral areas of the body such as the bony area by the elbow.

Step 11: Connecting the muscle sensor(s) to the Arduino

Picture of Connecting the muscle sensor(s) to the Arduino
Now we're getting close. By this step, your muscle electrodes and power supply should be hooked up to your muscle sensors. We just have to connect the sensors to the Arduino.

Using the solid core wire/jumper wire, connect the Vout pin of each muscle sensor to the analog pins on your Arduino.

Sensor 1 Vout --> Analog pin 0
Sensor 2 Vout --> Analog pin 1
Sensor 3 Vout --> Analog pin 2
Sensor 4 Vout --> Analog pin 3

Then connect the GND pin of muscle sensor 1 to the GND pin on your Arduino and the GND pin of muscle sensor 2 to the Analog pin 3 of your Arduino. That last part will keep the remaining two analog pins from floating along with your sensor 4's output.

Step 12: Upload visualization sketch to your Arduino

Picture of Upload visualization sketch to your Arduino
DSC01449.JPG
By this point, your USB Biofeedback Game Controller should be completely assembled and hooked up. Before we go any further, we'll need to test out your muscle sensors to make sure everything is setup properly. 

To test your setup, we're going to use the same visualization software as we used in our muscle sensor demo video. If you haven't already, download and install both the Arduino and Processing software.

Next, download and unzip the visualization program files: Arduino, Processing. Open the Arduino sketch and upload it to your UNO. If you're new to Arduino, check out the information on Arduino's website to help you on this step. Once uploaded, plug in the wall adapter to the barrel jack to power up your muscle sensors.

Next, open the Processing sketch and hit the Run button in the top left. Another window will pop up with 6 colored bars on it. Flex your right forearm, left forearm, right bicep, and left bicep and watch the bars move accordingly. If while at rest, your sensors value is below or above 150 then adjust the gain of your sensor using a screwdriver and turning the poteniometer on the corresponding sensor so that they are around 150 at rest. 

As you flex each muscle, the bars should move past the 300 mark. This is important since the software to follow uses the 300 mark to trigger a "button press". You can adjust the gain of your sensor and this threshold level in the code to fine tune your desired button press sensitivity.

Once everything appears to be in working order, close out Processing and you're ready to hack your Arduino into a keyboard.


Step 13: Upload keyboard sketch to your Arduino

Picture of Upload keyboard sketch to your Arduino
Now, we'll work on turning your Arduino UNO into a HID keyboard.

First, Download the zipped Arduino project from our Google Docs page.

Then unzip, compile and upload the sketch to your Arduino.

This is the code that, if desired, you would modify the threshold value in to adjust the "button" sensitivity.

Step 14: Hack your Ardunio to load new atmega8u2 firmware

To play video games on your computer with the Arduino, we're going to turn your Arduino into a HID USB keyboard to interface with your computer; to do this you'll need to upload new firmware.

For Mac:
Follow the instructions on this very well put together blog...
Arduino Keyboard HID Part 1
Arduino Keyboard HID Part 2
Latest Hex Files for Uno and Mega 2560

For Windows:
Download the latest hex files
Then follow the instructions on how to upload the hex files to your board using free software called Flip
Upgrading the Arduino Uno 8U2 using Flip

For Linux:
Download the latest hex files
Then follow the instructions on how to upload the hex files to your board
Building Arduino Atmega8u2 firmware on Linux

Note: You'll need to reflash your Arduino with the original firmware to upload new sketches again

Once your Arduino can be connected to your computer and recognized as a HID device, it should be ready to go. Disconnect it and slide it down into your project enclosure.

Step 15: Playing Super Mario Bros 3


akatzbreaker10 months ago
Hi!! FOR ANYONE WHO HAS AN ERROR LIKE THIS:

java.lang.ArrayIndexOutOfBoundsException: 1

just replace the:

String V3 = Serial.list()[1];

to:
String V3 = Serial.list()[0];


Just found out!
wan xi1 year ago
hello,,im having problem with Processing sketch.when i hit the Run button the window don't pop up.It show error..it said java.lang.ArrayIndexOutOfBoundsException: 1...WARNING: RXTX Version mismatch
Jar version = RXTX-2.2pre1
native lib Version = RXTX-2.2pre2
Exception in thread "main" java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: 1
at processing.core.PApplet.runSketch(Unknown Source)
at processing.core.PApplet.main(Unknown Source)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 1
at sketch_apr11a.(sketch_apr11a.java:55)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at java.lang.Class.newInstance0(Class.java:355)
at java.lang.Class.newInstance(Class.java:308)
... 2 more

...can help me.I am new with this software....
wan xi1 year ago
hello,,im having problem with Processing sketch.when i hit the Run button the window don't pop up.It show error..it said java.lang.ArrayIndexOutOfBoundsException: 1......can help me.I am new with this software
Capturea.JPGCapturea.JPG
lovetronic1 year ago
when i try to run the code in processing software, it replies me "serial does not run in 64-bit mode". i'm still new in this software. please help me.
elzurdo862 years ago
Hi , I got it to funciton but only until the third phase, the fourth one is still not working. I will post my advance later today to show you what im doing to see if you can give me a hand, also I was trying to upload the NES EMG files but it does not let me it shows as a pde and it has problems with a boolean type of variable. It seems like it is a processing sketch not an Arduino one. Also I was checking the steps to updgrade the firmware. Is it mandatory to do it? by doing that would I be able to upload the .pde file to the arduino?

I will wait for your reply. Thanks
Gundanium (author)  elzurdo862 years ago
Hi,

What version of Arduino's software are you using? This sketch won't work for the new release, v1.0. They moved some header files around. If I find time, I'll work on updating the sketches.

.pde was used by previous arduino files as well as processing files.

I'm not sure I understand what you are talking about in regards to the firmware.
I-m using the Arduino UNO R3 which is different than the one in the link you posted. I haven't tried to update the firmware in this one since the pictures in the link does not match with the one that I have.
Gundanium (author)  elzurdo862 years ago
Are you talking about the Arduino hardware firmware? You shouldn't have to update the hardware firmware.

The Arduino development software has changed since I posted these instructions. The hardware firmware should be the same.
So basically I just have to update the sketch for the Arduino and I should be good to go?. How do I set up the keyboard to play the nes game? . Is a special game? or can I download any NES emulator?
mahfatouh2 years ago
i downloaded the ardiuno code and when i am about to upload the code to the mcu in the verification , it gives me errors and the code cannot be upoaded. please help
elzurdo862 years ago
Hello, I'm trying to get this project going, but I'm not able to get anything from the sensors, I made this one with the arduino UNO, and I noticed that the one you have was perhaps a duemilanove? , I tried to compile and Arduino displayed the message, BYTE keyword no longer supported , instead use Serial.write. I changed all of the commands and the program didn't show any mistakes but after uploading everything to Arduino nothing happened. Can you give me a hand with this?
Gundanium (author)  elzurdo862 years ago
I've updated the Arduino and Processing code used to visualize the sensor output with the new Arduino code base. Let me know if you have any problems with it.
Gundanium (author)  elzurdo862 years ago
Nope the one I used was an UNO. They've updated the code syntax since this instructable was posted. From my quick google search, it looks like you were on the right track with replacing all .print with .write.

http://www.engblaze.com/changes-in-the-arduino-1-0-release/

I'll update the code when i can find some time.

You might want to look into grabbing an oscilloscope to debug your circuits. This is the first tool I always use prior to hooking things up to a MCU.
Yes That is what I'm thinking. I will look for one and then post my results, hopefully I solved the problem . I'm pretty far away to buy another one :p
I had anorhter question , would it matter if I'm only using 1 EMG sensor for the processing sketch?

Gundanium (author)  elzurdo862 years ago
No it shouldn't matter. You might want to ground the unused analog pins on your Arduino tho. Their values will float otherwise.
ALEXANDER_F2 years ago
Wonderful idear ! But I beg one thing ,I can not downioad the zipped Arduino project in the step 13 ,is there any lovely one can mail me the zipped Arduino project ? My E-mail address : yinshi438788349@126.com .Thank you very much.
epicnoobpwn2 years ago
Or you could grow big muscles, wrap arm straps around your biceps and flex to press switch. That would be a lot easier. I'm just saying.
elzurdo862 years ago
Great!, I'm actually starting to study Biofeedback as well, I just have a question about the noise that the circuits may get, is that a lot? or the circuit is designed to leave that aside.
Greetings from Mexico
ARIrish2 years ago
Finally I have a real reason to want to build up my weak and feeble arms...
13thvulture2 years ago
that is pretty cool, but does it actually provide some kind of workout, or is it just for fun?
Cool :-)
NES Kinect! Seems like my arms would get tired after hours of Mario Bros. action.
asocial2 years ago
great