NESBot: Arduino Powered Robot Beating Super Mario Bros for the NES

77,111

63

69

Introduction: NESBot: Arduino Powered Robot Beating Super Mario Bros for the NES

This guide will take you through the steps to build an NES playing robot. 


You will need:

An Arduino Duemilanove (other boards will probably work, but you will have to adjust the steps for your device)
Working NES Console
Super Mario Bros. (Note: this must be only the single game, not the two- or three-in-one cartridge)
A controller you can cut the cord from
Two LEDs (two different colors are recommended but not required)
2x ~390 ohm resistors
3x 1.6k ohm resistors
3x 3.3k ohm resistors
4021 (16 pin Shift Register. Note: the 74xx series will not work)
SD Card slot (more on this later)
A pushbutton (not required, but recommended)
A switch (not required, but recommended)
A soldering iron (you'll probably need one)
Multimeter
Hookup wire

Software needed:
Arduino Interface, available from http://arduino.cc/ 
Processing, available from http://processing.org/
FCEUX, available from http://fceux.com/ (you probably want the windows binary unless you know what you're doing)

Step 1: Background Information

This project is based off the efforts of the people at TASVideos . For a full description of what they do, take a look at their Welcome Page .

Every approved submission (called "movies", even though they contain no audio or video data) on their website is simply a series of recorded button presses that in theory can be played back on the actual console for which they are intended. In most cases however the emulator differs too greatly from the actual consoles to make this possible, but in the case of the NES the difference is significantly smaller.

The game choice also makes a big difference as to whether the button presses can be played back or not. If the game relies on any uninitialized memory for randomness, or if it is heavily based on console timing, it may not work. In the case of Super Mario Bros however, as long as the button presses start play back at the right time, the movie will play back correctly.

These button presses are based on frames on the console. Almost every time the console redraws the screen (~60 times a second), the controller is polled for input. However, there are times when the screen is redrawn, but the controller is not polled. These are called lag frames. Due to the difference in the way the emulator handles these lag frames, any movie must be modified to run correctly on the console. This is handled later on. 

To turn "in theory" into reality, we will start by modifying the controller. 

Step 2: Modifying the Controller

We need to cut the cable on the controller and make it possible to plug it into the schematic on the next step, but we also need to identify the wires inside to make sure they are plugged in correctly. 

If you're lucky (like I was) you can open your controller and the 5 wires will be labeled inside. 

If you're not as lucky you'll need to do a bit more work. If you opened your controller to check for labels, keep it open, and get your multimeter. Probe each wire inside the controller and on the console end of the wire. Use this picture  to find the GND, +5, Data Out, Clock, and Latch wires. Note that the picture is of the socket, so you'll need to mirror it when looking at your cable. 

Once you know which wires are which, cut the controller away from the cable, strip the cable. You'll need a way to plug these wires into your breadboard. Since these wires are usually small and delicate, I recommend soldering a piece of hookup wire to each. You can also wire each to a line of pin headers like I did. I also modified my controller to have pin sockets which allow me to plug the cord back in and still use the controller.

Once these are able to be plugged into a breadboard, you are ready to wire the rest of the system.

Step 3: Wiring the Schematic

Wire the schematic as shown. I recommend doing this on a breadboard. And remember this schematic is based on the Arduino Duemilanove when looking at the pin numbers on the left.

For the SD card, I used a microSD card with a converter that made it into a full sized SD card. I soldered pins onto the converter making it possible to plug it into a breadboard. However you do it, make sure you take note of the pin numbers on the card and wire them correctly.

The pushbutton and switch are not required. If you don't have any handy just use hook up wires that you connect to ground when you want to "push" them, and unplug them when you're done. We will be using the pullups built into the arduino, so none are needed externally.

Note that the grounds from both the arduino and the NES are tied together. Also that  the 4021 IC gets power from the NES, not the arduino.

If you are modifying this to work on another device, the NES LATCH signal MUST go into an interrupt input. Also the SD card must be connected connected to the SPI interface. For more information that, read here, namely the "Physical Connections" section, and substitute your correct pin numbers. Also note that I left pins 0 and 1 on the arduino disconnected. These are for TX/RD to the computer, and will need to be disconnected every time when programming the device, so I worked around them completely. 

Step 4: Setting Up FCEUX

FCEUX is available here.

You will need to find the Super Mario Bros rom for the NES to be used in the emulator. It is up to the reader to locate said rom by their own means.

You will also need the movie we will be playing back which will beat the game. That can be found on the NES movie list  at TASVideos . Look for "NES Super Mario Bros (JPN/USA PRG0)", but feel free to try out any movie that plays Super Mario Bros.  At the time of writing this article, this movie, made by HappyLee, is currently the fastest Super Mario Bros completion the site has. 

Which ever movie you choose, download the .zip file containing the .fm2 file that is the movie.

You will next need the Lua code which will modify the movie for playback on the console. This code will remove any lag frames from the movie file, and the button data in the same directory as the .fm2 file. Download this code and save it somewhere convenient. 

Open the emulator and load the rom into the emulator, and press the pause key on your keyboard to pause the emulator. Open a new Lua window (File - Lua - New Lua Script Window), and browse to the lua script you downloaded. Press run.

Then load the movie you downloaded. Go to File - Movie - Play Movie. From the drop down list pick Browse..., and navigate to the movie you downloaded. Press the pause key again to unpause the emulator, and let the video finish playing. If you'd like, you can speed up playback by pressing + or holding the tab key for turbo. 

When the movie is finished, pause the emulator again, and stop the Lua script. 

You should now have another file in the same directory as the .fm2 file. We will need this later. 

We will next load the code onto the arduino.

Step 5: Programming the Arduino

The Arduino IDE is available here.

Note: if you are using a different arduino, make sure to modify the pin numbers in the code!

Download the code, uncompress it, and save it on your computer. Load the arduino interface, and open the .pde file. Make sure the interface is configured for your device, then plug in your arduino and download the code onto it. 

When the upload is complete, you can confirm it is working if you leave the SD LOAD SWITCH unconnected and the READY light is on. Also, if in this state you press the GO BTN the status light should turn on, and it is now waiting for the console. But we first need to load the movie onto the SD card.

To do that, we will need Processing.

Step 6: Using Processing

Processing available here.

Download the code and save it to a convenient place.  

Load up processing. Open the code file, and you will be prompted to make a directory for this code, say yes. In the location where you saved this file you will now find a directory with the same name. Open this and inside make a directory named Data. Copy the generated file from step 4 into this new directory.

Modify the filename at the top of the code to match your file name. 

Next, we need to load the movie onto the SD card. 

Step 7: Uploading the Movie

Switch the SD LOAD SWITCH so that the pin is tied to ground, and reset the arduino. The device should restart, but the READY light should not turn on. The device is ready to have a movie loaded.

Run the processing code. The arduino should restart again, and then in the processing window you should see how many bytes have been uploaded. The number of bytes is same as the length of the movie in frames, minus the number of lag frames. 

When the upload is finished, you can stop the processing code. Switch the SD LOAD SWITCH back, and the ready light should turn on.

The movie is ready to be played! 

Step 8: Playing the Movie

Plug the controller cable into the console and load the game. You might want to turn on the console to make sure the game runs ok before starting the bot. 

When you get the game to load fine, turn off the console, press the GO BTN, and then turn back on the console. If the game doesn't start playing on its own, there was a problem with the timing. Turn off the console, restart the arduino and try again. For best results, quickly and completely press the power button to prevent bouncing which may mess with the timing. 

When the game is being played, the STATUS light will blink. After the movie has finished playing this light will stop blinking.

Step 9: Congratulations!

If everything worked correctly, the bot should have just beaten Super Mario Bros! Congratulations! You can now try playing other movies, or even try making your own. You can also try playing other games, but keep in mind most will not work. With some modification I was able to get a Super Mario Bros 3 video to play back partially. I had no luck with some of the other games I have in my collection.

Enjoy your bot! If you manage to get other games to work fully let me know!

Microcontroller Contest

Finalist in the
Microcontroller Contest

Be the First to Share

    Recommendations

    • Battery Powered Contest

      Battery Powered Contest
    • Plywood Challenge

      Plywood Challenge
    • Plastic Contest

      Plastic Contest

    69 Discussions

    2
    pikaboy70_
    pikaboy70_

    Tip 4 months ago

    Before posting your comments accusing the TAS of being fake, remember the game was made in 1985 and isn't perfect. There are many enemies that have a hitbox smaller than their sprite (i.e. Piranha Plant, Fire Bar, Blooper). The mechanic to push Mario out of a wall can be abused resulting in 1-1 FPG, 1-2 Pipe Clip, 4-1 FPG, 4-2 Wall Clip, 8-1 FPG, 8-3 FPG. Mario can run across 1 block gaps with enough speed, this is consistent through many 2D Mario games, the most recent 2D Mario game (NSMBUDX) still has this feature. Mario can walljump (Shigeru Miyamoto found out about the glitch and said "I guess it's a feature") if he hits the pixel between to blocks and jumps on the frame he "lands" on the block. All glitches seen can be done on a real console (as demonstrated in the video), some can be done by humans, others are too precise for humans but still work. Bisbuth's WR Explanation video on YouTube also talks about this stuff.

    0
    9a8k0007
    9a8k0007

    Reply 2 days ago

    What file format is saved in sdcard?
    BIN OR TXT?

    1
    hayzFish
    hayzFish

    Reply 7 weeks ago

    Thank you so much for clarifying that. Some people need to research before commenting "dis is fake"

    0
    fadeddeath
    fadeddeath

    Reply 9 years ago on Introduction

    He would also have died in the under water scene at 5:05 5:10 5:11 5:12 5:14 and 5:16 but that's not the biggest tell that it is fake. Since when does an NES system start up that quickly. . .

    0
    PaulMakesThings
    PaulMakesThings

    Reply 9 years ago on Introduction

    Also Mario doesn't move forward, at least not nearly that far, while facing backwards.

    0
    pikaboy70_
    pikaboy70_

    Reply 4 months ago

    Mario accelrates faster in the direction opposite of which he is facing.

    1
    Antzy Carmasaic
    Antzy Carmasaic

    Reply 9 years ago on Introduction

    nah, its not fake. Check out the site he mentions. It's really a sequence of key-presses created after using a very complex emulator which allows us to rewind time and remove our mistakes(kinda like braid, i guess)...

    0
    gossumx
    gossumx

    9 years ago on Introduction

    Why don't the Piranha flowers hurt him?

    0
    pikaboy70_
    pikaboy70_

    Reply 4 months ago

    The plants don't hurt him because Mario's hitbox never touches the plan'ts hitbox.

    0
    patrick p f
    patrick p f

    Reply 9 years ago on Introduction

    its a cool skill to have in super mario. because of how mario jumps, if you time it right, you can just make it over a piranha. i can do it without even thinking

    0
    Flyinseamnky
    Flyinseamnky

    Reply 9 years ago on Introduction

    no man he touched them. he did not make it over them.

    0
    skitchmusic
    skitchmusic

    Reply 9 years ago on Introduction

    Yes, but the way collision detection works in Super Mario Bros, that specific type of collision does not result in death.

    It's a known glitch in the game.

    0
    lioneyexp
    lioneyexp

    9 years ago on Introduction

    Next Instructible from him should be "How to heavily mod Super Mario Bros. so that you can phase through walls, disregard Piranha Plants, and ignore virtually any other threat in the game."

    0
    pikaboy70_
    pikaboy70_

    Reply 4 months ago

    The only thing modded in this is the controller. Nothing about the game or console is modified.

    0
    skitchmusic
    skitchmusic

    Reply 9 years ago on Introduction

    The game was not modded, and none of the techniques demonstrated involved modding the game.

    These are all exploits that existed in the original game, but are generally difficult/impossible to generate under normal circumstances.

    0
    saintneko
    saintneko

    9 years ago on Introduction

    Sorry, this is faked.

    At 4:50, notice how the Piranha Plant disappears when Mario jumps up onto the tube. It doesn't retract, it just disappears as he lands on it.

    0
    pikaboy70_
    pikaboy70_

    Reply 4 months ago

    The piranha plant disappears because of a mistake when developing the game. The reset trigger for the room was put in too early so the plant disappears early.

    1
    skitchmusic
    skitchmusic

    Reply 9 years ago on Introduction

    No, it's not faked. Anything that appears to be an oddity are side effects of glitches within the game itself.

    0
    ThisIsFake1
    ThisIsFake1

    9 years ago on Introduction

    So you are still saying this is not fake? And that it's your code playing it? So in 4-2, your robot just happens to use a glitch? You just happen to be jumping in the exact interval to use a glitch??