NESBot: Arduino Powered Robot Beating Super Mario Bros for the NES

This guide will take you through the steps to build an NES playing robot. 

You will need:

An Arduino Duemilanove (other boards will probably work, but you will have to adjust the steps for your device)
Working NES Console
Super Mario Bros. (Note: this must be only the single game, not the two- or three-in-one cartridge)
A controller you can cut the cord from
Two LEDs (two different colors are recommended but not required)
2x ~390 ohm resistors
3x 1.6k ohm resistors
3x 3.3k ohm resistors
4021 (16 pin Shift Register. Note: the 74xx series will not work)
SD Card slot (more on this later)
A pushbutton (not required, but recommended)
A switch (not required, but recommended)
A soldering iron (you'll probably need one)
Multimeter
Hookup wire

Software needed:
Arduino Interface, available from http://arduino.cc/ 
Processing, available from http://processing.org/
FCEUX, available from http://fceux.com/ (you probably want the windows binary unless you know what you're doing)

Step 1: Background Information

This project is based off the efforts of the people at TASVideos . For a full description of what they do, take a look at their Welcome Page .

Every approved submission (called "movies", even though they contain no audio or video data) on their website is simply a series of recorded button presses that in theory can be played back on the actual console for which they are intended. In most cases however the emulator differs too greatly from the actual consoles to make this possible, but in the case of the NES the difference is significantly smaller.

The game choice also makes a big difference as to whether the button presses can be played back or not. If the game relies on any uninitialized memory for randomness, or if it is heavily based on console timing, it may not work. In the case of Super Mario Bros however, as long as the button presses start play back at the right time, the movie will play back correctly.

These button presses are based on frames on the console. Almost every time the console redraws the screen (~60 times a second), the controller is polled for input. However, there are times when the screen is redrawn, but the controller is not polled. These are called lag frames. Due to the difference in the way the emulator handles these lag frames, any movie must be modified to run correctly on the console. This is handled later on. 

To turn "in theory" into reality, we will start by modifying the controller. 

Step 2: Modifying the Controller

We need to cut the cable on the controller and make it possible to plug it into the schematic on the next step, but we also need to identify the wires inside to make sure they are plugged in correctly. 

If you're lucky (like I was) you can open your controller and the 5 wires will be labeled inside. 

If you're not as lucky you'll need to do a bit more work. If you opened your controller to check for labels, keep it open, and get your multimeter. Probe each wire inside the controller and on the console end of the wire. Use this picture  to find the GND, +5, Data Out, Clock, and Latch wires. Note that the picture is of the socket, so you'll need to mirror it when looking at your cable. 

Once you know which wires are which, cut the controller away from the cable, strip the cable. You'll need a way to plug these wires into your breadboard. Since these wires are usually small and delicate, I recommend soldering a piece of hookup wire to each. You can also wire each to a line of pin headers like I did. I also modified my controller to have pin sockets which allow me to plug the cord back in and still use the controller.

Once these are able to be plugged into a breadboard, you are ready to wire the rest of the system.

Step 3: Wiring the Schematic

Wire the schematic as shown. I recommend doing this on a breadboard. And remember this schematic is based on the Arduino Duemilanove when looking at the pin numbers on the left.

For the SD card, I used a microSD card with a converter that made it into a full sized SD card. I soldered pins onto the converter making it possible to plug it into a breadboard. However you do it, make sure you take note of the pin numbers on the card and wire them correctly.

The pushbutton and switch are not required. If you don't have any handy just use hook up wires that you connect to ground when you want to "push" them, and unplug them when you're done. We will be using the pullups built into the arduino, so none are needed externally.

Note that the grounds from both the arduino and the NES are tied together. Also that  the 4021 IC gets power from the NES, not the arduino.

If you are modifying this to work on another device, the NES LATCH signal MUST go into an interrupt input. Also the SD card must be connected connected to the SPI interface. For more information that, read here, namely the "Physical Connections" section, and substitute your correct pin numbers. Also note that I left pins 0 and 1 on the arduino disconnected. These are for TX/RD to the computer, and will need to be disconnected every time when programming the device, so I worked around them completely. 

Step 4: Setting Up FCEUX

FCEUX is available here.

You will need to find the Super Mario Bros rom for the NES to be used in the emulator. It is up to the reader to locate said rom by their own means.

You will also need the movie we will be playing back which will beat the game. That can be found on the NES movie list  at TASVideos . Look for "NES Super Mario Bros (JPN/USA PRG0)", but feel free to try out any movie that plays Super Mario Bros.  At the time of writing this article, this movie, made by HappyLee, is currently the fastest Super Mario Bros completion the site has. 

Which ever movie you choose, download the .zip file containing the .fm2 file that is the movie.

You will next need the Lua code which will modify the movie for playback on the console. This code will remove any lag frames from the movie file, and the button data in the same directory as the .fm2 file. Download this code and save it somewhere convenient. 

Open the emulator and load the rom into the emulator, and press the pause key on your keyboard to pause the emulator. Open a new Lua window (File - Lua - New Lua Script Window), and browse to the lua script you downloaded. Press run.

Then load the movie you downloaded. Go to File - Movie - Play Movie. From the drop down list pick Browse..., and navigate to the movie you downloaded. Press the pause key again to unpause the emulator, and let the video finish playing. If you'd like, you can speed up playback by pressing + or holding the tab key for turbo. 

When the movie is finished, pause the emulator again, and stop the Lua script. 

You should now have another file in the same directory as the .fm2 file. We will need this later. 

We will next load the code onto the arduino.

Step 5: Programming the Arduino

The Arduino IDE is available here.

Note: if you are using a different arduino, make sure to modify the pin numbers in the code!

Download the code, uncompress it, and save it on your computer. Load the arduino interface, and open the .pde file. Make sure the interface is configured for your device, then plug in your arduino and download the code onto it. 

When the upload is complete, you can confirm it is working if you leave the SD LOAD SWITCH unconnected and the READY light is on. Also, if in this state you press the GO BTN the status light should turn on, and it is now waiting for the console. But we first need to load the movie onto the SD card.

To do that, we will need Processing.

Step 6: Using Processing

Processing available here.

Download the code and save it to a convenient place.  

Load up processing. Open the code file, and you will be prompted to make a directory for this code, say yes. In the location where you saved this file you will now find a directory with the same name. Open this and inside make a directory named Data. Copy the generated file from step 4 into this new directory.

Modify the filename at the top of the code to match your file name. 

Next, we need to load the movie onto the SD card. 

Step 7: Uploading the Movie

Switch the SD LOAD SWITCH so that the pin is tied to ground, and reset the arduino. The device should restart, but the READY light should not turn on. The device is ready to have a movie loaded.

Run the processing code. The arduino should restart again, and then in the processing window you should see how many bytes have been uploaded. The number of bytes is same as the length of the movie in frames, minus the number of lag frames. 

When the upload is finished, you can stop the processing code. Switch the SD LOAD SWITCH back, and the ready light should turn on.

The movie is ready to be played! 

Step 8: Playing the Movie

Plug the controller cable into the console and load the game. You might want to turn on the console to make sure the game runs ok before starting the bot. 

When you get the game to load fine, turn off the console, press the GO BTN, and then turn back on the console. If the game doesn't start playing on its own, there was a problem with the timing. Turn off the console, restart the arduino and try again. For best results, quickly and completely press the power button to prevent bouncing which may mess with the timing. 

When the game is being played, the STATUS light will blink. After the movie has finished playing this light will stop blinking.

Step 9: Congratulations!

If everything worked correctly, the bot should have just beaten Super Mario Bros! Congratulations! You can now try playing other movies, or even try making your own. You can also try playing other games, but keep in mind most will not work. With some modification I was able to get a Super Mario Bros 3 video to play back partially. I had no luck with some of the other games I have in my collection.

Enjoy your bot! If you manage to get other games to work fully let me know!