A Lesson in Using a Raspberry Pi to Remote Operate an Arduino Based Robot

The following is a step-by-step lesson to build a remote-operated robot using a Raspberry Pi and Arduino. I will be using a "Star Wars Classic - 18" (31" scale) R2-D2;" but, there is no reason why you can't modify these instructions to fit whatever robot you want. And it is with that in mind that I will be writing these instructions, so as to make them broad and general. In short, my focus isn't going to be building R2 so much as building a robot in general; and, in such a way as to make it easy for the reader to customize the project to suit their needs.

Notes:
Just a heads up, I will be assuming that you're at least familiar with programming, so some of my test code won't be using a bunch of comments. And that you have your Pi setup to run headless. If your Pi isn't and you're not sure how to set it up, I suggest checking out some of these tutorials first:

• https://www.instructables.com/id/Intro-to-the-Head...
• https://learn.sparkfun.com/tutorials/headless-rasp...
• https://blog.adafruit.com/2017/03/17/headless-rasp...

If you have any questions... or any suggests, feel free to contact me! We're all here to learn and teach!

Also, you can find all of the code on my GitHub page( https://github.com/MattTucker22689/R2D2 ), which I will be linking back to throughout the Instructable. Please keep in mind though, as it is Github, I can not promise that the current version posted will always work 100%. It is likely to be buggy at times.

One final note, you will likely find you need to download libraries for the Arduino IDE. If you don't know how to do so, please see: https://www.arduino.cc/en/Guide/Libraries

Step 1: Parts!

Before we start we're going to need a few parts. So let's start there!

BOM(Bill of Materials):

1. Chassis(again I used Jakks 2015 R2D2 toy before they put the robotics/electronics into them-https://www.amazon.com/Star-Wars-Classic-scale-R2... go to eBay to find them cheaper ;P)
2. Raspberry Pi
3. Micro SD x2
4. Roll of Electrical Wire
5. Arduino(I'm using a Mega 2560)
6. Motor x2( https://www.pololu.com/product/1120 )
7. Wheel x2( https://www.pololu.com/product/199 )
8. Motor Drive Shield( https://www.amazon.com/gp/product/B010C7U99M/ref=... )
9. WiFi dongle ( https://www.amazon.com/TP-Link-TL-WN725N-wireless... )
10. 7.2v Battery ( https://www.amazon.com/7-2V-3300mAh-NiMH-Battery-... )
11. 3.2v battery( https://www.amazon.com/gp/product/B00J2QET64/ref=... )

12. PowerBoost Charger( https://www.amazon.com/gp/product/B00PY2YTVU/ref=o... )

    
    From here the parts are optional; but, I did incorporate them into my project, so I'll be mentioning them.

13. Servo
14. Ultrasonic Range Sensor(I recommend you incorporate one of these into your project or some other method of preventing your bot from crashing into things- https://www.radioshack.com/products/radioshack-ul... )
15. Sensor Shield( https://www.sainsmart.com/products/sensor-shield-... )
16. Camera for the Pi(I had to buy a Logitech camera)
17. MicroSD reader( https://www.amazon.com/gp/product/B01APDJ1TM/ref=... )
18. Speaker (I pulled one out of a set of speakers I bought at Goodwill... Goodwill is GREAT for parts)
19. LM386 Audio Amplifier( https://www.amazon.com/gp/product/B01FDD3FYQ/ref=o... )
20. Shrink Tube
21. Power switch(I salvaged mine from an old remote control truck)
22. LEDs x4
23. Resistors(based on LEDs- https://www.kitronik.co.uk/blog/led-resistor-valu... )
24. Jumper Wires(M-M and M-F)
25. Perf board(s)

Tools:

1. Screwdrivers(phillips and flat)
2. Sodering Iron
3. 3D Printer (suggested)
4. Lighter

NOW LET'S GET STARTED!

Note:

Every step that I had to write unique code for or had to design a mount for, I will provide a .zip file containing the corresponding files. That said, every mount is specific to my case, so you'll likely need to modify it.

Step 2: Getting Things Going(Yes, That Was a Pun)

To start, let's first focus on making the bot move. To do that, we're going to need:

1. Arduino
2. Motor Shield
3. Motors x2
4. Wheels x2
5. Roll of wire
6. Sodering Iron
7. Soder
8. Screwdriver
   -Optional-
9. Shrink tube

Let's start by measuring out and cutting twice the length of wire that our motors will PROBABLY need to connect to the motor shield. Do this for both motors.

Now, one at a time solder the wires to the leads on the motors.

Stack the motor shield onto your Arduino and connect your motors accordingly. For my bot I connected my motors to terminals 2 and 3.

Now let's bust out some test code and make sure our motors are running in the right directions.

https://github.com/MattTucker22689/R2D2/blob/maste...

If your motors are running in opposite directions, just pick one and flip the wires.

Note, you don't HAVE to flip your wires or adjust the code to your case just yet. But it is something that will eventually need to be addressed.

Now, let's mount our boards and motors to the chassis and the wheels to the motors.

Note:

Because I am using the motor shield that I am; I'm also using an old Adafruits Motor library( https://learn.adafruit.com/adafruit-motor-shield/d...).

Attached is a zip copy of the code and the motor mounts I designed for R2's feet.

Step 3: Ultrasonic Sensor(Optional, But Recommended)

So for part 2, we need to make sure our bot can "see!" We don't want it running into walls, potentially breaking it. To do this we're going to need:

1. Our previous boards
2. Sensor Shield(makes it easier to access a some of the pins)
3. Ultrasonic Sensor
4. Jump Wires(Female to Female/FF)

To do this let's add our sensor shield to the two others. This will likely require you pull the first two apart and placing the sensor shield between them. Now, pick a pin that we won't be using for anything else. For me, this was pin A4. Using the jumper wires, connect the signal pin "SIG" of the sensor to A4, then "VCC" to the voltage source pin, and "GND" to the ground. In my case on the board, A4 is one of the most exterior blocks of pins, and the order of the pins are "A4, V, G" moving from the interior of the board out. That said, ALWAYS check your board's schematics!

Now let's mount our new monstrosity of boards and the sensor to our chassis and start coding!

https://github.com/MattTucker22689/R2D2/blob/maste...

Test the code a couple of times with a book, make sure it gives you the right value back. If it's off, go to "sensor_loop()" and tweak the value as needed. Remeber, k_new=k_old*(d_actual/d_sensor). So, k_new=(1/148)*(d_actual/d_sensor)

Cool, the ultrasonic is working. Now what? Now let's combine our test code for the motor and the sensor to make sure they're working together.

https://github.com/MattTucker22689/R2D2/blob/maste...

Step 4: MicroSD(Optional)

Since there are a number of reasons we might want our bot to use a MicroSD(i.g. hold the results of previous events/data collection; or, in my case, to store .mp3's on for it to later play.), let's get that set up.

For this you'll need:

1. MicroSD board
2. A MicroSD
3. Jumper wires(F to M)
4. You may also need to solder your pins to the MicroSD board

First, select a pin that you wish to have the SD be read from. I used pin 53.

Connect(SD Reader to Arduino):

• GRND to GRND
• 5v to 5v
• CLK to pin 52
• D0 to pin 50
• D1 to pin 51
• CD to pin 53

Now for the code, this time let's keep it simple, let's use the stock example to check that everything is connected correctly. I used CardInfo in the SD example directory. Whichever example you decide to use, make sure the code is using the pin your card is supposed to be read from. For example, mine read

const int chipSelect = 53;

To secure the MicroSD reader, I found a free mounting hole on my chassis. That said, it would have been just as easy to use one of the mounting holes on the motor shields.

Step 5: Speaker(Optional)

So, we have the bot moving, avoiding objects, and now reading microSDs... Let's have it make noise! Why? Because maybe you want it to alert you to an event. Or, maybe you want to make a mobile media device. In my case, because I wanted it to be more like R2.

For this you'll need:

1. Amplifier
2. Speaker

To do this, we're going to leave our previous setup as it is, with the exception of our new addition of a speaker amplifier and a speaker. Before we begin, let's take a moment to identify an appropriate pin to control our speaker with. I'm using, pin 46.

Next, we need to wire the amplifier up with the speaker and the Arduino.
Connect(Amplifier to Arduino):

• GND to GND
• VCC to 5v
• In to pin 46(or w/e your source is)

Now, let's bust out that C++ and get our bot talking, preliminarily.

Note: I had 61 files labeled 2.wav-62.wav. You should change this code to reflect your case.

https://github.com/MattTucker22689/R2D2/blob/maste...

Just a friendly word of advice if you do have a bunch of sound files... Don't rename them by hand. Here's some Python code I wrote that will rename them for you!

https://github.com/MattTucker22689/R2D2/blob/maste...

In typical DIY fashion, to fasten down/mount my speaker I had to design and print a special motor mount for my particular set up. I've shared it in the zip file. Feel free to use it, or redesign it as you please.

Step 6: Servo(Optional)

As far as wiring goes... The servo will likely be the easiest part of this project. You just have plug the servo into one of the pin blocks on the motor shield designated for servos. Bam. EASY! That said, due to EVERYTHING running on the Arduino, coding a test sketch won't be quite as easy.

So the problem I ran into, and you may run into it too depending on your bot, a clash in libraries using Timer1. The workaround for this is to use the ServoTimer2 library, instead of the Servo library.

https://github.com/MattTucker22689/R2D2/blob/maste...

Now, mounting the servo... that's a different story. Since every case is going to be different, it's hard for me to tell you how to mount yours. But attached in the .zip file is a copy of the mount I made. feel free to use it or modify as you need. I also provided some instructions on how to mount it specific to R2.

Step 7: LEDs(Optional)

Now, let's add lights!

To do this, let's look at three basic circuit diagrams and see which one we want to use.

1. Single LED-
   • This is the circuit for a single LED
2. Different VIN, Shared Ground-
   • To control multiple LEDs individually this is the circuit to use
3. Shared VIN, Shared Ground-
   • To control multiple LEDs that act in "unison," this is the circuit to use

To find out which resistor to use with your LEDs I suggest checking out this site:
https://www.kitronik.co.uk/blog/led-resistor-value...

Like the MicroSD, I'm not going to give an example of test code here as Arduino already provides one via the stock examples that comes with its IDE.

Step 8: Wiring the Batteries

Wiring the batteries can get a bit tricky if you want to allow for charging and aren't particularly familiar with building circuits. What I did was use a single switch between my batteries and the device. In particular, I used a switch that I salvaged from an old remote control truck, which allowed me to wire both circuits(double poled). The circuit diagrams are provided.

Step 9: Systems Test 1

Before we go ANY furth, let's take a moment to combine ALL of our code to make sure everything is working together. You'll want to do this now so that later you won't have to work your way back to try and figure out what is going on.

https://github.com/MattTucker22689/R2D2/blob/maste...

How will I know if the code and my bot are working though? Well, as I wrote this sketch:

If R2 is 6 inches or closer to an object its head will turn, the LEDs will blink at random delays, it will play a sound, and try to move away. Otherwise, R2 will keep moving forward.

In short, our test code will do almost everything we want... minus remotely control the bot. We do this so that, again later there will hopefully be less coding involved. But, also to make sure that everything is running smoothly.

Step 10: Raspberry Pi Setup

At this point in the project, it's a good idea to have w/e sensors, do-dads, or gadgets that are going to be operated via the Arduino connected, mounted, and tested. From this point on, we don't want to do anything to the Arduino other than modifying the code; which, we'll do via the Pi.

Go ahead and plug the wifi dongle into your Pi and mount it.

To start, let's make sure we have all the Python(I'm using Python 2. If you want to use Python 3 use pip3 instead.) libraries we're going to need. Type in:

pip list

And look for:

• Flask
• pyserial

If you are missing either, run:

pip install Flask
pip install pyserial

Cool, we've got everything installed. But what do they do?

Well...

• Flask
  • Will let your website interact with Python code
• pyserial
  • Will let Python code talk to your Arduino

So is that everything I need to install? It for right now. We'll worry about the rest later.

Step 11: Serial Connection Test

First, we want to write some basic code for our Arduino and Pi to communicate.

Note:

You will see later that I am not using the coms check functions. I did this, and am including this step, because I thought it was a good way to get use to using the different languages in sync to achieve a common goal.

To do this, we need to make sure we have pyserial installed for whichever version of Python we're using. I used Python 2, so I installed it using pip, hence the previous page. But if you want to use Python 3, use pip3 instead.

Here's the test code we want to use for this:

https://github.com/MattTucker22689/R2D2/tree/maste...

To check to make sure everything is copacetic, upload the sketch to your Arduino(leave it connected) and run the python code. You should get the following print out: "Connection has been made."

You may run into one of two problems. The first is reading from the wrong serial connection port. The second is running the Arduino IDE and leaving the Serial Monitor window open. The second problem is an easy fix... Just close the window. That said I have NOOOOOOO idea why this is an issue. So if anyone knows the reason why I'd like to find out. However, to fix that first problem unplug your Arduino, open Terminal, and run

ls /dev/tty*

This will present you with a huge list of ports. Now plug the Arduino back in and run it again. The new terminal in the list is the terminal the Arduino is plugged into. Now that you know the port, replace the old port address with the new one in the Pi.py.

ser = serial.Serial('PORT ADDRESS', 9600)

Now try running the code again.

Step 12: Making the Website

So to make the site let's first consider what we want as an interface for our bot. In my case I wanted,

1. control mechanism
2. live video feed(next step- optional)

First I started with the bare minimum and moved on to creating a layout using style. Then I began populating each part accordingly. Finally, I went back through and spruced the page up with decorations, like a background image and a favicon.

Looking through the pictures you can see the evolution process I used. You can also see the original code I used, for designing the layout.

Here is a copy of my final test HTML code:
https://github.com/MattTucker22689/R2D2/blob/maste...

Note:

If you don't plan on incorporating a live feed, delete the div section named, "LIVE_FEED."

Depending on how you are running your setup, you may want to download apache2. To start, open up Terminal and install Apache by typing:

sudo apt-get install apache2

Then you'll want to give it the rights to edit itself. To do this type:

sudo chown -R pi:www-data /var/www/html/

sudo chmod -R 770 /var/www/html/

Now on your external machine type your Pi's IP into the browser and you should see a page like:

https://www.raspberrypi.org/documentation/remote-a...

Once that's done swap the file /var/www/html/index.html with the one you made. Refresh your browser and you should see your page!

Step 13: Adding a Live Feed(Optional)

To add a live feed to our page, we first need to install "Motion." To do that, open Terminal and do the following:

1. sudo apt-get update
2. sudo apt-get install motion
3. sudo nano /etc/default/motion
   • set Motion daemon to, "yes"
   • save and exit
4. sudo chown motion:motion /var/lib/motion/
5. sudo nano /etc/motion/motion.conf
   • Set "stream_localhost" to "off"
   • save and exit
6. sudo reboot

Once you've done that, you're going to want to replace the IP address in the div section named, "LIVE_FEED," with your own IP. Be sure to keep it in the format: YouIP:8081

Note:

I was not able to get the traditional Pi Camera to work with Motion. So instead I opted to use a Logitech camera. Also, I did not mount my camera in any particular way. I just found a spot and put some screw through it.

Step 14: Controlling the Arduino Via the Website

So, we have our website(mostly), we have the serial connection code working, and we have the Arduino code running... Let's put it together!

First, let's open a new sketch and throw ALL of our code together there. However, in loop() we will need to replace everything in it with a set of if-else or while loops that will be conditionally based on serial inputs from our Python code.

Second, we need to add a set of jQuery instructions that make calls to our Python script, based on the user's input on our HTML page. This is why we needed Flask. Now, depending on how you set everything up, you jQuery script should look something like this:

$( document ).ready(function(){
	$("#ButtonName1").on("mousedown", function() {
		$.get('/action_press1');
     		}).on('mouseup', function() {
     		$.get('/action_stop1');
   	});
	$("#ButtonName2").on("mousedown", function() {
		$.get('/action_press2');
		}).on('mouseup', function() {
		$.get('/action_stop2');
   	});
	$("#ButtonName3").on("mousedown", function() {
		$.get('/action_press3');
		}).on('mouseup', function() {
     		$.get('/action_stop3');
   	});  
....for EVERYTHING that is meant to be used to control the bot
});

I should note, each "/action_" will correspond to a Python def(); where "_press" is what will happen while you're holding it down, and "_stop" is what will happen when you let go. So for example, you might use:

$("#up").on("mousedown", function() {
	$.get('/forward');
     	}).on('mouseup', function() {
     	$.get('/stop');
});

Next, let's open the Python code we used to test the serial comms. Let's delete the call to startup(), then add a series of defs() that will send a single statement to the Arduino. I used the basic "Left," "Right," "Forward," "Backward," and "Auto." At the end of each one of those defs() add the line,

return 'true'

Now, above each of those, we need to write:

@app.route("/'action_name' ")

We also need to add these lines:

app = Flask(__name__)<br>@app.route("/")
def index():
	return render_template('robot.html')

Then at the bottom, we need to add:

if __name__ == "__main__":
	print "Start"
	app.run(host='0.0.0.0',port=5010)

So, when all is said and done your code should look something like R2D2.py in:
https://github.com/MattTucker22689/R2D2/tree/maste...

Now... Let's test it! Upload the new sketch and open up Terminal. In it, type:

python path/to/your/file.py

Now, on your machine, type your pi's IP address, but add ":5010" to the end. It should look like this:

YourPi'sIP:5010

It should pull up your site. And all you have to do is press your buttons.

If you do run into a problem, make sure your file directory is setup like so:

--app.py

--templates/

--index.html

--static/

--"background"

--"icon"

Here's a link to a copy of my code for it:

https://github.com/MattTucker22689/R2D2/tree/maste...

Step 15: Setting Up Your Pi to Be a Router and Making It Direct ALL Traffic to Your Website(Optional)

Why might we want to do this? Well, first and foremost, security. 2ndly, if we didn't, every time we encountered a new WiFi network we would have to crack open our bot just so we could set it up to remotely access and operate it...

That said, the process to do this could itself be an entire instructable. So... I feel it's best for me to point you in the direction of two GREAT tutorials for how to do this.

1. https://circuitdigest.com/microcontroller-project...
2. https://www.instructables.com/id/A-Grey-Hat/

Step 16: Run Your Python Script on Bootup Via Crontab(Optional- But HIGHLY SUGGESTED!)

Congrat's, you're nearly done! But before we call this project done, let's make sure we never have to ssh or vnc in again, unless we want to play with the code... let's make our code run every time we turn it on!

For this, make note of the path for the final version of your Python script. Mine is currently: /home/pi/Desktop/R2D2/Code/R2D2_Code/Tests/WebPyTest/R2D2.py (there's a redundant file in there that I need to nix to make that shorter... *rolls eyes*)

Next, open up Crontab by entering:

crontab -e

Now that you're in Crontab, scroll to the bottom and enter:

@reboot sudo python /path/to/your/file.py

In my case, I wrote:

@reboot sudo python /home/pi/Desktop/R2D2/Code/R2D2_Code/Tests/WebPyTest/R2D2.py

And if you want to keep a log of your outputs, add:

/home/pi/Desktop/pyprog/log.txt

And you're done!

Step 17: Enjoy Your New Robot!

From here it's just a matter of preference. Rewrite the code however you like and make it your own! Some suggestions for projects that this can be modified for:

1. A bot that maps its environment.
   • You could use the ultrasonic and the camera to create a sort of Google map of a place.
2. A personal assistant that follows you.
   • You could add code to the pi that acts as a sort of personal assistant and have the bot follow an RFID tag you keep with you, or you could have it use image recognition(check out Open CV).
3. A mobile music player.

...there is literally an endless list of things you might want to repurpose this Instructable for! And I look forward to seeing some of them!

Step 18: Credits and Resources

For more information here are some sites to check out-

HTML:

• http://jinja.pocoo.org/docs/2.10/templates/
• https://www.w3schools.com/html/default.asp

Flask:

• http://flask.pocoo.org/docs/1.0/
• https://codehandbook.org/python-flask-jquery-ajax-...
• https://pythonprogramming.net/jquery-flask-tutoria...
• http://flask.pocoo.org/docs/1.0/patterns/favicon/
• https://pythonspot.com/flask-web-app-with-python/

Server/Router:

• https://www.instructables.com/id/A-Grey-Hat/
• https://readwrite.com/2014/06/27/raspberry-pi-web-...
• https://www.instructables.com/id/Host-your-website...
• https://raspberrypihq.com/how-to-turn-a-raspberry-...
• https://howtoraspberrypi.com/how-to-install-web-se...

• https://circuitdigest.com/microcontroller-projects...

Serial Coms:

• https://classes.engineering.wustl.edu/ese205/core/...
• http://pyserial.readthedocs.io/en/latest/pyserial.html

Related projects:

• https://circuitdigest.com/microcontroller-projects...

Step 19: Final Words...

My apologies for having to keep sending y'all back to GitHub. For some reason, when I left a step with code in it and later returned; the Instructables site rewrote it with a bunch of HTML embedded that stayed present in the preview. As a result, I thought it was best just to send y'all to Git as a way of ensuring everything was readable.

As I'm sure you saw, if you looked through some of the final code, in the future I plan on incorporating AI into the project using a process similar to the one in the attached flowchart(Process.pdf).

I hope y'all have enjoyed this instructable and found what you were looking for.

If you have any questions, suggestions, or want to show off your adaptation of this feel free to hit me up.

Also, I've attached the most recent .zip copy of my project folder containing EVERYTHING I have in regards to this project. I apologize if some of the code is buggy(WebPyTest). I am still in the midst of building the final version of my bot, which will include an AI and several new functions(lights, sound, head turning) on the web page.