Instructables

NESBot: Arduino Powered Robot beating Super Mario Bros for the NES

FeaturedContest Winner
Picture of NESBot: Arduino Powered Robot beating Super Mario Bros for the NES
This guide will take you through the steps to build an NES playing robot. 


You will need:

An Arduino Duemilanove (other boards will probably work, but you will have to adjust the steps for your device)
Working NES Console
Super Mario Bros. (Note: this must be only the single game, not the two- or three-in-one cartridge)
A controller you can cut the cord from
Two LEDs (two different colors are recommended but not required)
2x ~390 ohm resistors
3x 1.6k ohm resistors
3x 3.3k ohm resistors
4021 (16 pin Shift Register. Note: the 74xx series will not work)
SD Card slot (more on this later)
A pushbutton (not required, but recommended)
A switch (not required, but recommended)
A soldering iron (you'll probably need one)
Multimeter
Hookup wire

Software needed:
Arduino Interface, available from http://arduino.cc/ 
Processing, available from http://processing.org/
FCEUX, available from http://fceux.com/ (you probably want the windows binary unless you know what you're doing)

Step 1: Background information

This project is based off the efforts of the people at TASVideos . For a full description of what they do, take a look at their Welcome Page .

Every approved submission (called "movies", even though they contain no audio or video data) on their website is simply a series of recorded button presses that in theory can be played back on the actual console for which they are intended. In most cases however the emulator differs too greatly from the actual consoles to make this possible, but in the case of the NES the difference is significantly smaller.

The game choice also makes a big difference as to whether the button presses can be played back or not. If the game relies on any uninitialized memory for randomness, or if it is heavily based on console timing, it may not work. In the case of Super Mario Bros however, as long as the button presses start play back at the right time, the movie will play back correctly.

These button presses are based on frames on the console. Almost every time the console redraws the screen (~60 times a second), the controller is polled for input. However, there are times when the screen is redrawn, but the controller is not polled. These are called lag frames. Due to the difference in the way the emulator handles these lag frames, any movie must be modified to run correctly on the console. This is handled later on. 

To turn "in theory" into reality, we will start by modifying the controller. 

Step 2: Modifying the Controller

Picture of Modifying the Controller
DSCN0220.JPG
DSCN0219.JPG
We need to cut the cable on the controller and make it possible to plug it into the schematic on the next step, but we also need to identify the wires inside to make sure they are plugged in correctly. 

If you're lucky (like I was) you can open your controller and the 5 wires will be labeled inside. 

If you're not as lucky you'll need to do a bit more work. If you opened your controller to check for labels, keep it open, and get your multimeter. Probe each wire inside the controller and on the console end of the wire. Use this picture  to find the GND, +5, Data Out, Clock, and Latch wires. Note that the picture is of the socket, so you'll need to mirror it when looking at your cable. 

Once you know which wires are which, cut the controller away from the cable, strip the cable. You'll need a way to plug these wires into your breadboard. Since these wires are usually small and delicate, I recommend soldering a piece of hookup wire to each. You can also wire each to a line of pin headers like I did. I also modified my controller to have pin sockets which allow me to plug the cord back in and still use the controller.

Once these are able to be plugged into a breadboard, you are ready to wire the rest of the system.

Step 3: Wiring the schematic

Picture of Wiring the schematic
DSCN0223.JPG
Wire the schematic as shown. I recommend doing this on a breadboard. And remember this schematic is based on the Arduino Duemilanove when looking at the pin numbers on the left.

For the SD card, I used a microSD card with a converter that made it into a full sized SD card. I soldered pins onto the converter making it possible to plug it into a breadboard. However you do it, make sure you take note of the pin numbers on the card and wire them correctly.

The pushbutton and switch are not required. If you don't have any handy just use hook up wires that you connect to ground when you want to "push" them, and unplug them when you're done. We will be using the pullups built into the arduino, so none are needed externally.

Note that the grounds from both the arduino and the NES are tied together. Also that  the 4021 IC gets power from the NES, not the arduino.

If you are modifying this to work on another device, the NES LATCH signal MUST go into an interrupt input. Also the SD card must be connected connected to the SPI interface. For more information that, read here, namely the "Physical Connections" section, and substitute your correct pin numbers. Also note that I left pins 0 and 1 on the arduino disconnected. These are for TX/RD to the computer, and will need to be disconnected every time when programming the device, so I worked around them completely. 

Step 4: Setting up FCEUX

Picture of Setting up FCEUX
FCEUX is available here.

You will need to find the Super Mario Bros rom for the NES to be used in the emulator. It is up to the reader to locate said rom by their own means.

You will also need the movie we will be playing back which will beat the game. That can be found on the NES movie list  at TASVideos . Look for "NES Super Mario Bros (JPN/USA PRG0)", but feel free to try out any movie that plays Super Mario Bros.  At the time of writing this article, this movie, made by HappyLee, is currently the fastest Super Mario Bros completion the site has. 

Which ever movie you choose, download the .zip file containing the .fm2 file that is the movie.

You will next need the Lua code which will modify the movie for playback on the console. This code will remove any lag frames from the movie file, and the button data in the same directory as the .fm2 file. Download this code and save it somewhere convenient. 

Open the emulator and load the rom into the emulator, and press the pause key on your keyboard to pause the emulator. Open a new Lua window (File - Lua - New Lua Script Window), and browse to the lua script you downloaded. Press run.

Then load the movie you downloaded. Go to File - Movie - Play Movie. From the drop down list pick Browse..., and navigate to the movie you downloaded. Press the pause key again to unpause the emulator, and let the video finish playing. If you'd like, you can speed up playback by pressing + or holding the tab key for turbo. 

When the movie is finished, pause the emulator again, and stop the Lua script. 

You should now have another file in the same directory as the .fm2 file. We will need this later. 

We will next load the code onto the arduino.

Step 5: Programming the Arduino

Picture of Programming the Arduino
The Arduino IDE is available here.

Note: if you are using a different arduino, make sure to modify the pin numbers in the code!

Download the code, uncompress it, and save it on your computer. Load the arduino interface, and open the .pde file. Make sure the interface is configured for your device, then plug in your arduino and download the code onto it. 

When the upload is complete, you can confirm it is working if you leave the SD LOAD SWITCH unconnected and the READY light is on. Also, if in this state you press the GO BTN the status light should turn on, and it is now waiting for the console. But we first need to load the movie onto the SD card.

To do that, we will need Processing.

Step 6: Using Processing

Picture of Using Processing
Processing available here.

Download the code and save it to a convenient place.  

Load up processing. Open the code file, and you will be prompted to make a directory for this code, say yes. In the location where you saved this file you will now find a directory with the same name. Open this and inside make a directory named Data. Copy the generated file from step 4 into this new directory.

Modify the filename at the top of the code to match your file name. 

Next, we need to load the movie onto the SD card. 

Step 7: Uploading the movie

Switch the SD LOAD SWITCH so that the pin is tied to ground, and reset the arduino. The device should restart, but the READY light should not turn on. The device is ready to have a movie loaded.

Run the processing code. The arduino should restart again, and then in the processing window you should see how many bytes have been uploaded. The number of bytes is same as the length of the movie in frames, minus the number of lag frames. 

When the upload is finished, you can stop the processing code. Switch the SD LOAD SWITCH back, and the ready light should turn on.

The movie is ready to be played! 

Step 8: Playing the movie

Picture of Playing the movie
Plug the controller cable into the console and load the game. You might want to turn on the console to make sure the game runs ok before starting the bot. 

When you get the game to load fine, turn off the console, press the GO BTN, and then turn back on the console. If the game doesn't start playing on its own, there was a problem with the timing. Turn off the console, restart the arduino and try again. For best results, quickly and completely press the power button to prevent bouncing which may mess with the timing. 

When the game is being played, the STATUS light will blink. After the movie has finished playing this light will stop blinking.

Step 9: Congratulations!

If everything worked correctly, the bot should have just beaten Super Mario Bros! Congratulations! You can now try playing other movies, or even try making your own. You can also try playing other games, but keep in mind most will not work. With some modification I was able to get a Super Mario Bros 3 video to play back partially. I had no luck with some of the other games I have in my collection.

Enjoy your bot! If you manage to get other games to work fully let me know!
 
Remove these adsRemove these ads by Signing Up
1-40 of 55Next »
I tried to create a NESBot shield on DipTrace. I got this (see the picture below).

 Is it all OK?
NESBot Shield.BMP
pjgat09 (author)  GenesisMaster1 year ago
Hard to say without checking out all those traces by hand. Cool to see it in shield form!

This instructable is kind of stale now. I really need to update it, but haven't had time. I have since removed the need for processing, and instead the movie files are loaded as files directly onto the SD card. I can try to find all the actual changes if needed, but I think this instructable might still work.

If you decide to get this printed, I'd love to see the result!
I tried to improve the shield which now looks like this (see picture below), with a built-in SD slot and a more convenient place for the switches and the LEDs. I also tied the VCC pins of the NES and the 4021 IC to the 5V pin of the Arduino. Was it a good idea?
NESBot.BMP
Just got an NES today. Totally doing this!
almost absolutely impossible for human..awesome!
I don't understand your schematic very well. What are all these "GND"? Are they somehow linked? And these tracks linked by green dots?
pjgat09 (author)  GenesisMaster1 year ago
GND stands for Ground. The arduino and the NES have seperate grounds. In my schematic on the right I link them together (ARD GND and NES GND) to make just GND (with the symbol). Then yes, all the GND symbols are tied together and would be plugged into the GND port on the arduino.

The green lines are connections. The green dots are just to clarify that when multiple wires meet they are connected. Otherwise they are not connected and are just crossed wires.
Hi Im trying to do this same thing but without the arduino, I was thinking of using an EEPROM, some counters, and the shift register. But I dont understand whats in the txt file the lua creates. Is it the exact buttons frame by frame?
techboy4112 years ago
why is the converter is solderder
GASSYPOOTS2 years ago
can you fix it to use the duckhunt/mario one? or possibly a DSI game
Kingpin3k2 years ago
Why does the 7421 not work and is it possible to drop the ShiftRegister if you are using the shiftout function of the arduino?
pjgat09 (author)  Kingpin3k2 years ago
The 7421 is a dual 4-input AND gate, so I'm not sure how that would be useful. The closest 7400 series chip I found was the 74165. I actually started by using that chip, but if I recall correctly the clock input is inverted compared to the 4021. I tried using a NOT gate to fix that, but it was too slow and I lost sync very easily.

I switched to using the 4021 after my order for them came in since it is the chip the controller actually uses. I don't think using the shiftout will work on the arduino due to the speed required. The latch pulse arrives every so often, slow enough for the arduino to handle, but after it arrives the 4021 is clocked 7-8 times extremely quickly, faster than the arduino can likely handle.
now try that without cheats.
There were no cheats used in this run. It's a Tool-Assisted Run, but there are no external cheats modifying the game's actual code to make it run this way.
Well this whole thing is still an external cheat lol.
right!
Sgt. Cookie3 years ago
The world is going to hell in a handbasket and you created SkyNet. Thank you.



On a serious note:

Cool, all you need now is one that can play brawl and you will be champion of the world, at brawl.
ironorr843 years ago
For all you "too young to know the original Mario game", Mario can't "wall jump" like he does @ 3:54. I call BS on this. It's either fake or the game was altered, which doesn't merit a WIN in my books. I say FAIL...
OMG thanks for that link, it was a hole new dimension of SMB, I've no idea you can do that on the game. Before I've read the post I thought the game was cheated, but I think it is real, but I can't stop thinking on the SMB personalized games I've see on youtb. Awesome
He can so. I havent done it but playing with my friend he did a single wall jump. I am 34 BTW.
ThisIsFake13 years ago
So you are still saying this is not fake? And that it's your code playing it? So in 4-2, your robot just happens to use a glitch? You just happen to be jumping in the exact interval to use a glitch??
It is just playing back button presses, no robot is figuring out what to do.
kellykel3 years ago
There are so many noobs here. This is real. Those are just glitches that are hard to do unless you are TASing. Talk about something you know about.
saintneko3 years ago
Sorry, this is faked.

At 4:50, notice how the Piranha Plant disappears when Mario jumps up onto the tube. It doesn't retract, it just disappears as he lands on it.
No, it's not faked. Anything that appears to be an oddity are side effects of glitches within the game itself.
lioneyexp3 years ago
Next Instructible from him should be "How to heavily mod Super Mario Bros. so that you can phase through walls, disregard Piranha Plants, and ignore virtually any other threat in the game."
The game was not modded, and none of the techniques demonstrated involved modding the game.

These are all exploits that existed in the original game, but are generally difficult/impossible to generate under normal circumstances.
gossumx3 years ago
Why don't the Piranha flowers hurt him?
its a cool skill to have in super mario. because of how mario jumps, if you time it right, you can just make it over a piranha. i can do it without even thinking
no man he touched them. he did not make it over them.
Yes, but the way collision detection works in Super Mario Bros, that specific type of collision does not result in death.

It's a known glitch in the game.
popscott33 years ago
WWWOOOOHHHH!!!!
lioneyexp3 years ago
Mario can wall jump?? 3:55

did you just jump off of lava? O_o 4:49
touched the flame rope? 5:11. 5:14
touched the squid? 5:12, 5:15
Yeah I noticed that none of the plants hurt him, that he was running across gaps, and could run through walls that as far as I know of you could not do before. Like the first warp zone. Usually you have to use the moving platforms to get on top of the bricks and run across.
For the jumping off lava, there's a platform down there that's difficult to see in the real game. Almost everyone spent way too long trying to figure that out unless you spot it early on.

As for apparent collisions with squid / ropes / piranha plants / etc, the collision detection algorithms aren't perfect in this game (remember, this was made in 1985...) so if you time it right you can seemingly pass by the edges of something.

Running across gaps works fine as long as there's only a single block gap on a panel (ie, several in world 8, etc.)

And you can definitely slide through bricks - the trick in 1-2 is actually a method to get to the so-called minus world (-1) that is a never ending water world.

It looks weird, and part of the oddity with this is the 'perfect" computer controlled player, but every one of these tricks can be done by a human with the right timing. I've done all of these at one time or another save for the jumping out of the large gap - now I'm really tempted to try it...
i know the minus world trick, but how come it didnt send the computer there? this whole thing is fishy
Because if you go far enough to the right, it treats it as a normal warp zone rather than the minus world / level 5 warp zone.
there was a platform in the lava.

the flame rope and squid, i have no explanations for.
Nice project, but the instructable needs some work. Mostly, you fail to list several components and never really mention them. For example, wouldn't I need an SD card? Also, you never list the part you used to connect the Micro SD adapter to the breadboard. Normally, this is not a huge deal, but is definitely aggravating to someone who has not used an Adruino controller before or has little experience in wiring SD cards. There definitely needs to be some clarity around the use of the SD card.

Other than those details, it seems to read pretty well for a novice audience.
1-40 of 55Next »