Speech-controlled Game Boy Advance

This is a project I worked on for my electronics class at Pomona College. Thanks to Professor Dwight Whitaker and Tony Grigsby for their help and guidance throughout this project, and credit to Jonathan Wong for the idea for this project!

Also, here's a list of Instructables I looked at for ideas/inspiration while working on this project:

https://www.instructables.com/id/Speech-Recognition...

https://www.instructables.com/id/Wireless-Audio-Str...

https://www.instructables.com/id/Speech-Recognition...

https://www.instructables.com/id/How-to-modify-your...

How it works

The basic idea of this speech-controlled Game Boy Advance is that you can say the name of a button (left, A, start, etc.) and have the GBA respond as if that button had been pressed. Here’s a quick rundown of how I’ve set it up to work:

1. You say a word into a small microphone (let’s assume you say “start”), and this signal is sent from the microphone to the computer through the Arduino.

2. The speech recognition software BitVoicer sees that “start” is a word it’s supposed to respond to and sends the Arduino the string “start”.

3. The Arduino receives the string and sets the voltage of one digital output pin to HIGH and the rest to LOW. The pin set to HIGH is connected to a relay that is in turn connected to two metal pads on the GBA circuit board that correspond to the start button.

4. Since the pin is set to HIGH the relay switches states, making the two metal pads electrically connected. This electrical connection is what happens when you usually press GBA buttons, so the GBA responds as if the start button was pressed.

I have this currently set up for all buttons except L, R, and select. I chose not to do it for these buttons because they aren’t used quite as often as the other buttons and I wanted to first focus on getting the essential buttons to work. The direction buttons are set to stay on HIGH until you say a new command (equivalent to holding down the button) while the A, B, and start buttons are set to stay on HIGH for 200 ms before switching to LOW (equivalent to pressing the button once). I’ve also programmed a “stop” command that sets all the pins to LOW to stop all ongoing commands.

Issues

Although the speech control works fairly well, there are a few issues to be aware of. For one, there’s a delay of about one or two seconds between when you say a command and when the GBA responds to it. So don’t expect to be playing any games where timing’s important, unless you happen to be really good at thinking ahead and saying commands early! Another issue is that BitVoicer sometimes fails to recognize a command, but this doesn't happen too often and when it does you usually only need to repeat yourself once or twice. I’ve particularly had trouble with it thinking I said “b” when I was saying “a”. Also, menu navigation can sometimes be troublesome with the current setup: sometimes you’ll scroll through menus really fast since the direction buttons are held down, while at other times you’ll have to alternate between a direction button and the stop command (i.e. “down, stop, down, stop,...”) if you want to keep going in one direction on a menu. Whether you run into these problems or not depends on how the game you’re playing deals with menu navigation. To fix this last problem I’m thinking of eventually adding some code that’ll allow for two commands for each direction button, one that’ll hold it down and one that’ll press it once.

Despite these issues I thought this project turned out fairly well and I really enjoyed working on it. I hope you do too!

Step 1: What You'll Need

Here’s a list of what I used for this project:

• Arduino Uno
  
• Game Boy Advance
• Computer with BitVoicer speech recognition software (You can download BitVoicer online, after which you’ll need to buy an activation code for US $4.50. This can all be done on the BitSophia site.)
• 7 relays (I used 3 OMRON G5V-1 relays and 4 Magnecraft W171DIP-7 relays. I used two different kinds of relays because I didn’t have seven of either. It shouldn't matter what kinds you use as long as you have seven, though.)
• 2 8-pin female headers
• 2 6-pin female headers
• Adafruit electret microphone amplifier (here’s the one I used)
• 4 IC sockets (I used these for just the Magnecraft relays because I couldn't get my hands on any or the OMRON relays)
• Perfboard (At least 20 x 28 holes; I opted for using a perfboard rather than a printed circuit board due to time constraints. Feel free to put everything on a PCB if that’s more to your liking.)

And here’s a few other things you’ll find useful:

• Wire
• Solder and a nice soldering iron
• Tri-wing screwdriver (for opening up the Game Boy Advance)

Step 2: Setting Up the Circuit Elements on the Perfboard

Note: The next few steps require soldering, so get your soldering iron out! If you don’t have too much experience soldering, then it might be helpful to look at some Instructables on soldering for advice before diving in.

We’ll start by setting up the circuit elements we’ll need on the perfboard. The diagram above shows what we want to have set up when we’re done with this step. Look at this diagram to figure out where each component should be placed. If you hover over the boxes on the diagram you can see notes that’ll help you with setting everything up. Be sure you have the headers at the same relative positions as in the diagram. These headers will plug into the pins on the Arduino, so they have to line up with the Arduino pins (actually they won’t perfectly line up due to spacing of the Arduino pins, but the positioning of the headers in the diagram will give the best possible fit)!

I’d recommend soldering in this order: first IC sockets and headers, then relays. Be careful when soldering in relays. Too much heat can damage them! Then place any relays that you aren't soldering into their IC sockets, as the wiring we’ll be doing soon might make it difficult to put in these relays at a later time. Be careful about the orientation of the relays. The wiring that’s coming up is dependent upon the relays being oriented as shown in the diagram, so it’ll be easiest if you have your relays oriented the same way.

Step 3: Wiring Up the Perfboard

Now we want to put in most of the wiring on the perfboard as well as the microphone. This wiring is shown on the diagram above (once again, hover over the diagram to see the notes). Keep in mind that this wiring is based on the pin configuration of the relays I used. If you’re using different relays then you’ll need to look up the data sheets for those and adjust your wire placement accordingly.

The red wires (EXCEPT for the one towards the bottom of the board) have one end soldered next to the input pin of a relay and the other end plugged into a header. The orange wires have one end soldered next to the ground pin of a relay and the other end on the same row as one end from each of the other orange wires. These eight orange wire ends are soldered together in a row along with one end of the white wire. This white wire then plugs into a header (which will go to a ground pin on the Arduino). Once you have these red and orange wires soldered to the board, also solder them to the pin that they’re right next to.

Now let’s look at the wires that interact with the microphone (around the lower left of the board). The leftmost brown wire goes from the microphone output to the board, where it’s soldered to the yellow wire right below it. This yellow wire then goes to a header pin that will go to the A0 pin on the Arduino. The middle brown wire goes from the microphone GND to the board, where it’s soldered to the orange wire that ends up going to ground. The rightmost brown wire goes from VCC on the microphone to the board, where it’s soldered to the short yellow wire. The other end of this yellow wire is soldered to the ochre and red wires next to it. The ochre wire has its other end at a header that’ll go to AREF on the Arduino. The red wire has its other end at a header that’ll go to the 3.3 V pin on the Arduino. This 3.3 V will be the reference voltage for the microphone.

In terms of connecting the brown wires to the microphone, I currently just have mine bent over into place (see picture). This seems to be working fairly well for me so far. My brown wires are long enough that they hold the microphone above the other wires on the board, which makes it easier to talk into the mic.

Step 4: Opening Up the Game Boy Advance

You now want to connect the perfboard to the GBA, but you can’t do that if you don’t have access to the GBA’s circuit board! To get access to the circuit board you’ll need to open up the GBA. I found this guide really helpful while doing this. We don’t want to completely dismantle the GBA, though, so don’t go past step 9 of this guide. Be careful when handling the inner components of the GBA. Don’t pull the ribbon wire too hard and try to avoid touching the circuit board as much as you can.

After opening up the GBA you’ll have a bunch of screws and plastic pieces like the L and R buttons that you've removed. I haven’t put most these back on the GBA in any way and I’ve just put them all in a plastic bag for safekeeping. As for the direction pad, start/select pad, and A and B buttons, I just keep those upside down where they would normally go as a rudimentary means of preventing dust from getting into the GBA (as seen in the first picture in this Instructable).

Step 5: Connecting Wires to the Relays

Now that the Game Boy Advance is open, you should be able to see that the front of the circuit board has regions marked with the names of the different buttons. These regions are right under where the buttons would normally have been. We want to connect each relay to one of these regions with two wires. In this step we’ll go over how to connect the wires to the relays and in the next step we’ll see how these wires should be connected to the regions on the circuit board. I recommend using wires that are flexible and long, as this comes in handy when trying to connect the wires to the GBA circuit board.

The diagram above shows how I have these wires arranged for my setup. Note that I've omitted the wires and the mic that we put in earlier for clarity. The wires are placed next to two pins of the relay that are electrically connected when the relay is switched on, but are not electrically connected when the relay is in its normal off state. If you’re using relays with a different pin configuration than mine, then be sure to adjust your wire placement so that the wires will have this behavior.

Once you've soldered these wires into the board, don’t forget to also solder each soldered end to the relay pin that’s right next to it.

Step 6: Connecting Wires to the Game Boy Advance

Before you begin connecting wires from the relay to the Game Boy Advance circuit board, here’s a list of which relay I connected to which button. I’ve given the relative position of the relays (leftmost, top left, etc.) based on the orientation of the perfboard in the previous step’s perfboard diagram.

Leftmost OMRON relay – start

Top left Magnecraft relay – left

Bottom left Magnecraft relay – down

Top right Magnecraft relay – up

Bottom right Magnecraft relay – right

Top right OMRON relay – A

Bottom right OMRON relay – B

If the relays aren’t connected to the buttons as given above, then you’ll need to either change which red wire on the perfboard goes to which header or you’ll need to change the Arduino code. Otherwise when you say a button name a different button will be pressed.

Now that we know which relay’s wires should go where, take a look at the regions on the circuit board that are marked with button names (you can see this on the above picture). You’ll see that each region consists of some number of small metal pads, with two pads for the direction buttons, three for the start button, and four for the A and B buttons.

Once you’ve found these metal pads, take the wires you just connected to the relays in step 5 and trim the loose end of each. Leave only enough exposed wire to fit on one of the metal pads. For the direction buttons there are two metal pads, and you’ll want to solder one of your two relay wires to each of the pads. For the start button there are two small metal pads and one larger one in between the smaller two. You want to solder one relay wire to either one of the small pads and the other wire to the larger pad. For the A and B buttons there are four pads, and you want to solder the relay wires such that they’re either on adjacent pads or on the pads furthest away from each other. I chose to use the pads furthest away from each other so that the wires would be further apart and less clustered. For all of these connections it doesn’t matter which of the two relay wires goes to which of the pads as long as they’re the pads I just discussed.

Soldering advice

In terms of how to go about the soldering, what I did was I first put a bit of solder on the pad I wanted to solder to, letting it form a small puddle. Then I placed the wire on top of the (now solid) puddle and put the soldering iron tip over it, waiting until the puddle had melted enough to let the wire sink into it. Be very careful when soldering onto the circuit board. Don’t have the soldering iron set to too high of a temperature. The board can be damaged if it’s exposed to too much heat for too long.

Also, make sure that the solder doesn’t form an electrical connection between any two pads. This will result in the GBA thinking that the corresponding button is always being held down. If you do accidentally do this, then use a solder sucker, solder wick, or some other tool for desoldering and remove the solder that’s connecting the pads.

Zip tying

Once all the wires have been soldered to the circuit board pads, you should zip tie the wires on either side of the GBA to the plastic GBA case. I zip tied the start and direction wires to one side and the A and B wires to the other. There’s a hole on either side of the case that you can pass the zip tie through, and I’ve included a picture of the back of my GBA to show where these holes are. By zip tying the wires to these holes there will be less tension on the wire-pad connections as the wires get moved about, which we want so that the wires don’t ever get pulled off of the circuit board and potentially take the pads off the board with them (which would be really bad). Before passing the zip ties through the holes you might need to cut a little of the width of the zip ties off. I had to do this for one zip tie because one of the holes was too small for it to fit through normally.

Once you’ve finished this step you’re all done with the circuitry! Now all that’s left is configuring BitVoicer and programming the Arduino.

Step 7: Setting Up BitVoicer

Now we’ll move on to setting up BitVoicer. After opening up the program go to File -> Preferences. The settings I’m using are shown in the image above. I’ve also attached the Voice Schema I used for this project; just download that and open it with BitVoicer and you should be good. If you have any questions I'd suggest first going to Help -> BitVoicer Manual, as I found that the manual answered a lot of the questions that I had. In fact my Arduino code is just a modified version of the code found in section 5.5.2 of the BitVoicer Manual.

Step 8: Programming the Arduino

I’ve attached the code I’m using for my Arduino. I've set it up so that when BitVoicer hears a word that's in the Voice Schema, it sends the button name to the Arduino as a string. The Arduino then sets the digital output pin corresponding to that button to HIGH and all the others to LOW. For the direction buttons the pin is briefly set to LOW and then set to HIGH until a new command is sent (effectively holding down the button), while for A, B, and start, the pin is set to HIGH for 200 ms (similar to pressing the button once). I also have a “stop” command that just sets everything to LOW.

Something to be aware of is that you HAVE to have the line

bitVoicer.setAnalogReference(BV_EXTERNAL);

in your setup. This is because we’re using a reference voltage for the microphone. If you don’t have this line, then the calls to other BitVoicer functions later on can damage your Arduino.

In order to have this code work you’ll need to find the folder called “BitVoicer11” (located at C:\Program Files\BitSophia\BitVoicer\Library for me) and copy it over into the libraries subfolder of the Arduino IDE installation folder (located at C:\Program Files (x86)\Arduino\libraries for me). These folder locations will vary depending on where you have BitVoicer and the Arduino IDE installed. You can find more information on doing this in section 5.4 of the BitVoicer Manual.

Step 9: Putting Everything Together

You’re almost done! Now it’s just a matter of putting things together. Plug the perfboard shield into your Arduino and connect your Arduino to your computer. After uploading the Arduino code, go to BitVoicer and press the “Start” button. Text should appear in the Activity area on the right side of the BitVoicer window. Now just turn on your Game Boy Advance and speak into the mic and you should be all set!

If you have any questions or comments on this project, feel free to message me or write them below. Also if there’s any way I could make this guide clearer, please let me know about that, too! Suggestions for improvement are always welcome!