Introduction: Build Your Internet Controlled Video-Streaming Robot With Arduino and Raspberry Pi

I'm @RedPhantom (aka LiquidCrystalDisplay / Itay), a 14 years old student from Israel learning in the Max Shein Junior High School for Advanced Science and Mathematics. I'm making this project for everyone to learn from and share!

You may have thought to yourself: hmm... I'm a geek... And my children want me to make a project with them...
He wanted to build a robot. She wanted to dress it up like a little puppy. It's a good weekends project!

The Raspberry Pi is perfect for every use: today we will explicit the abilities of this micro-computer to make a robot. This robot can:

  • Drive around and be controlled via LAN (WiFi) using any computer connected to the same WiFi network as Raspberry Pi is.
  • Stream video live using the Raspberry Pi Camera Module
  • Send sensor data using Arduino

To see what you need for this nice light project just read the next step (warnings) and after that the Wanted: Components step.

Here is the GitHub repo: GITHUB REPO BY ME

Here is the Project Site: PROJECT SITE BY ME

Step 1: Warning: Be Careful Trying This at Home

CAUTION:

THE AUTHOR OF THIS TUTORIAL ASSUMES YOU HAVE SUFFICIENT KNOWLEDGE ABOUT ELECTRICITY AND THE BASIC OPERATION OF ELECTRICAL EQUIPMENT. IF YOU ARE NOT CAREFUL AND DO NOT FOLLOW THE INSTRUCTIONS IN THIS TUTORIAL YOU MAY: DAMAGE ELECTRONIC EQUIPMENT, BURN YOURSELF OR CAUSE A FIRE.Please be careful and use common sense. If you do not have the knowledge required for this tutorial (soldering, basics of electronics), please perform with an individual who does. Thank you.

THIS INSTRUCTABLE's AUTHOR REMOVES ANY RESPONSIBILITY FROM HIMSELF FOR DAMAGE CAUSED OR LOST PROPERTY OR PHYSICAL DAMAGE. USE COMMON SENSE.

Step 2: Components

All you need for this light week-end project is:

  • A Raspberry Pi
    Any model will do: we will use two USB ports: one for the WiFi adaptor and another for the Arduino.
    I'm using Raspberry Pi 2 Model B
  • A WiFi Adapter for you Raspberry Pi. Occupies 1 Pi USB port. (Connected to Raspberry Pi)
  • A USB-A to USB-B short cable. Occupies 1 Pi USB port. (Connected to Raspberry Pi, used to connect between the Arduino and the Pi)
  • An Arduino
    (Connected to Raspberry Pi)
    Most models are sufficient (Uno/Nano/Leonardo bear minimum requirements) . This little micro-controller will receive the output from our sensors and send signals to our DC motors.
    I'm using Arduino Uno.
  • Sensor(s) (Optional). (Connected to Arduino)
    These will get information from the environment and collect it for us.
  • Dual H-Bridge
    An H-Bridge is used to control the motors, like a big transistor. The Arduino sends pulses (PWM, see final step for explanations) to the H-Bridge which powers the DC motors from an external source. (See LOGICAL VOLTAGE and OPERATIONAL VOLTAGE in the final step (Explanations)).
    I use an L298N-Based dual H-bridge.
    (Connected to Arduino)
  • DC Motors
    Note: They must be the same model ones so that they're speed is the same.
  • Using Servo motors is also an option: Hook up the GND (Ground) wire to the Arduino and the power source. The signal to a PWM-able pin on the micro-controller and the PWR wire to an external power source.
  • A Battery Pack for the Pi and Arduino.
    I recommend a Battery Bank as it comes with a charging circuit and can be charged from any computer or USB phone charger.
    I use a 5V 5000mAh 1A Battery Bank.
    Note: a minimum for 1A is required for the robot to operate normally. Less than that will cause heating and may damage the battery.
    (Connected to H-Bridge)
  • Breadboard Jumpers
    They are very useful for tinkering. They connect all of your components together without soldering - the ultimate solution for prototyping.
  • 6v Battery Pack/ Battery pack for your motors
    As I use 4 AA batteries, you should use a battery pack that suits your DC motors. Note: Unlike other components, DC motors (like LEDs) use all of the current given to them and so you should use regular Alkaline batteries and not rechargeable cells. Just be cautious.
    (Connected to H-Bridge)
  • Platform
    As Raspberry Pi is a great platform to create this robot, we need a physical platform to place all of the components on. You can use whatever you want: wood and aluminum are just to great material.
    NOTE: If you decide to build your robot with a metal or a conducting material, cover it with a layer of transparent plastic / any other non-conducting material as when you put a board on it, the through-hole pins can short and destroy your board.
    I use a plastic base I bought from E-Bay for 12$. There are many to pick from out there. Even ones that come with motors (like mine).
  • Knowledge
    You mostly need simple programming understanding and basic Linux-environment operating skills. These are simple to acquire-- I learned Python and other programming languages via an eBook!

Step 3: Reading Material

Is is recommended that you will go over:

And visit:

The GitHub repo and the Site for this project are available on the first page of this instructable.

Step 4: Power Managment

First, we will have to take a look over our component's power usage. Normally all of them operate on 5V. Note that some sensors and components may require special voltages (like 3.3V).

Raspberry Pi 2 B (Any model will do) : ~500mA
Camera Module: ~250mA
Arduino (Uno): ~150mA
Ultrasonic Distance Sensor: ~50mA

Sum: 950mA. My battery is capable of delivering up to 1A so it's all good. If your setup requires more than 10% what the battery is capable of, consider connecting two in parallel or buying a higher-amperage one.

Important note regarding the H-Bridge: If your motors require more than 6V, then connect the power for the H-Bridge to the 12 In DC Pin and not the the 5V input. In that case, the 5V in acts as 5V Output. See your one's datasheet and/or an Instructable about using an H-Bridge.

Step 5: Connections

Before we heat up the soldering Iron, we need to go over what should be connected to what. I made this simple chart (MS Paint never lets me down) that describes where certain part are located within the robot.

The image is built so you can zoom in and see in full resolution and read the text.

Step 6: Address for the Pi

The Arduino talks with the Pi according to the plan. And the Pi talks to the computer, so how does this all work?

Lets look at our Connection Initiation Sequence:

  1. Raspberry Pi starts
  2. Arduino Starts
  3. Raspberry Pi starts TCP Client. It shoots out its IP address via an LED.
  4. Raspberry Pi starts Serial Communications service and connects to Arduino

Therefore we've established some sort of communication:

Computer <-> Raspberry Pi <-> Arduino

I've used Visual Basic .NET (Microsoft Visual Studio 2013 Community) to write the program that talks with the Raspberry Pi and Python to write the Arduino/Raspberry Pi protocol.

All you need to do to know your Pi IP address, is to connect it to an HDMI screen, log in to the Shell and type the command:

hostname -I

Step 7: The Plan

Now that we've got Pi's IP address we will SSH into it (SSH is Secure Shell - we connect remotely to the Linux shell) and write a file that displays the Server's IP address. The Pi, on startup will do so as well and write the port it's listening to. Here I will only give a few examples from the code but it is available to download from this step and from the GitHub branch I've created. Details on that later.

It works like so:

  1. RPi starts up.
  2. RPi starts the Tcp program on its local IP and a designated port.
  3. RPI starts to stream video
  4. RPI shuts down.

Step 8: Going Physical

Now, we are ready to start physically build the whole thing. If you haven’t read step 1 (warning text and licensing) please do so before proceeding. I'm not responsible for any damage caused. And in case of doubt, this robot must not be used for military purposes unless it's a zombie apocalypse. And even then use common sense.

It is suggested you read the instructables listen in the Reading List.

Download the connection scheme from the "Connections" step.

MOTORS

The motors you've bought probably look like this, and it's OK if they don't: if they have only two wires (black and red at most cases) it should work. Look up their datasheet online to see their operating voltage and current. Feel free to ask questions in the comment section. I always read them.

H-BRIDGE

I've never worked with an H-Bridge before. I googled a bit and found a good instructable explaining the principles of an HB. You can look there too (see Reading List step) and hook yours too. I won't explain much. You can read there and know all you should about this circuit.

LED

This little lightbolb can run from logical voltage just because it requires almost no current, and a voltage of 3V-5V 4mA-18mA. Optional.

ARDUINO

Arduino will get signals and commands via Serial connection from the Raspberry Pi. We use Arduino to control our motors because Raspberry Pi cannot output analog values via the GPIO.

Step 9: Autostarting Raspberry Pi

Every time you will power up Raspberry Pi, you will have to type the username and password. We don't want to do that because sometimes we just can't connect a keyboard to the Pi, so we will follow these steps from this tutorial to auto-start the program that prepares the Pi. If it will get stick in a loop we can always Ctrl+C to interrupt it.

  • sudo crontab -e
  • And then we will enter the command that adds that file to auto-strartup in the cron manager.

We will call the file pibot.sh which will give commands to start all kinds of python scripts to operate the robot. Let's go over it: (We sudo with curtain Python programs to allow the program to access the GPIO)

raspivid -o - -t 0 -hf -w 640 -h 360 -fps 25 | cvlc -vvv stream:///dev/stdin --sout '#rtp{sdp=rtsp://:8554}' :demux=h264

The code that does all of the work on the side of the pi will be called upon_startup.sh.

It is a simple shell script that runs everything.

Step 10: Houeston, We've Had a Problem... DC Motors Aren't the Same Model.

I've already tested the H-Bridge and it works fine, but when I hook the motors I got from the robot platform I ordered online those two motors turn at different speeds and make different noises. I changed the throttle to 100% on the motors. Both of them couldn't run at their maximum capability.

It seems like these are two different motors. One has a bigger torque which is great for this kind of robot but the other just wouldn't move the robot. So it turns in circles.

At this point, what I've got is the serial program on the Arduino works totaly fine but the Tcp server on the PC and the Tcp Client on the Pi are not coded yet. At I need to complete this entry for the competition. What do I do?

  1. First, I triple the voltage for the motors. The datasheet said 3V, 6V didn't move them. It's 9V then. I connected teo batteries in parallel to double the current and the voltage remains the same.
  2. Do I have other motors that fit to the mount on the platform? Maybe I can see If they are similar models.
  3. I can replace to Servos if the chocolate really hit the fan.

School started. I'll have to see what to do.

Note: Why on earth do I write the problems I encounter here? So if you are less experienced and have the same problems too you'll know what to do.

The Solution:

So I've made another test. I've adapted the difference in speed in the Arduino code.

NOTE: the motors may spin in different speeds for you! Change the values in the Arduino sketch.

Step 11: [TCP]: Why Tcp and Not Secure Shell? What Is TCP?

I have two explanations why use Tcp and not SSH for the P.C. - Pi communication.

  1. Firstly, SSH (Secure Shell, see Explanations) is meant to launch commands from a remote computer. Making the Pi respond with information we want is more difficult because our only option to analyse the data is through tough and tedious string processing.
  2. Secondly, we already know how to use the SSH and we want to learn more ways of communication between devices in this tutorial.

TCP, or Transmission Control Protocol, is a core protocol of the Internet Protocol Suite. It originated in the initial network implementation in which it complemented the Internet Protocol (IP). Therefore, the entire suite is commonly referred to as TCP/IP. TCP provides reliable, ordered, and error-checked delivery of a stream of octets between applications running on hosts communicating over an IP network.

(From Wikipedia)

So TCP Pros are:

  • Secure
  • Fast
  • Works anywhere on a network
  • Provides methods to check correct data transmission
  • Flow Control: has protection in case the data sender sends data too fast for the client to register and process.

And the cons are:

  • In TCP you can't broadcast (Send data to all devices on a network) and multicast (same but little different- gives the ability to each device broadcast like a server).

  • Bugs in your program and operating system libraries (that manage themselves the TCP communication, your router does almost nothing except connect the two [or more] devices)

Why not use UDP, you may ask? Well, unlike TCP, UDP doesn't make sure your client gets the data before sending more. Like sending an Email and not knowing if the client receives it. Besides, UDP is less secure. For more info, read this post from Stack Exchange Super User

This article is good and recommended.

Step 12: [TCP]: Lets Make a Client!

The client (Raspberry Pi in our case), that receives the data from the server (Our P.C. in our case) will get data to send to the Pi (Serial commands that will be performed on the Arduino) and receive data back (Sensor Readings and feedback directly from the Arduino. The scheme attached shows the relationship between the three.

The Python Wiki TcpCommunication article shows that it's so simple to make such communication with a few lines of code using the built-in socket module. We will have a program on the PC and another program on the Pi.

We will work with interrupts. Learn more in the Explanations step about them. Read there about buffers too.
Now, we can read the data we have using data=s.recv(BUFFER_SIZE) but it will be how many characters we've defined with empty bites. Can we use interrupts? Another question: will the buffer be empty or will it wait for the server to send more data in which case the server/client will throw a timeout exception?

Lets tackle that one at a time. Before we do so, I've looked up this Wikipedia article that lists used TCP and UDP ports. After a quick look I've decided that this project will communicate on port 12298 because it is not used by the Operating System and local services.

Step 13: Try Our Tcp Comms

In order to see if we can use interrupts, lets make a simple client and a
server using the Python Command Line. I will do so in the following steps:

  1. Start a program that sends a text via Tcp in a loop through a curtain port
  2. Start another program (in parallel) that reads all of the text in a loop and prints that to the screen.

Only segments of the program will be shown. All programs run with Python 3.
All these programs do is to send the serial command from the keyboard of the PC user to the Arduino through the Pi.

  • SBcontrolPC.py - To be run on the PC. Starts a TCP connection on the local address and on the port specified (I use port 12298, see previous step why)
  • SBcontrolPi.py - To be run on the Pi. Reads its buffer every half a second (0.5 seconds). Starts a shell script that manages things such as the video streaming etc.

Check out the script from the GitHub Repo! It works well!

Step 14: And Now... Stream Live!

So we want to "Abuse" the next capability of the Pi: the R-Pi Camera Module. A neat piece of hardware that can record 1080p@30fps or 720p@60fps (!) video and capture 5Mi Pixel images. Impressive for this 2.5 cm squared board (1 inch by 1 inch).

Lets follow the next steps to get everything working. You can always asist the internet and the raspberrypi.org official site.

  • Connect the Ribbon Cable to the Camera Serial Port (the long port, closest to the HDMI port.)
  • The golden connections of the cable should face away from the Ethernet port.
  • Lift the two little handles at the edges of the port.
  • Insert the cable with one hand, and make sure that it sits securely and all the way to the end.
  • Set the plastic bar in place through pushing the little handles in the corners of the connection.

You have connected your Camera Board!

At the next step (WIP) we will run a shell script that streams the video live to your local network where you can watch it everywhere with VLC Player.

Step 15: Explanations

Content from Wikipedia, the free Encyclopedia

Secure Shell, or SSH, is a cryptographic (encrypted) network protocol for initiating text-based shell sessions on remote machines in a secure way.

Pulse-width modulation (PWM), or pulse-duration modulation (PDM), is a modulation technique used to encode a message into a pulsing signal.

  • Operation Voltage VS. Logical Voltage

Logical Voltage is a low-voltage low-current flow used to transmit data between to objects, a Micro-controller and a computer, for example. This voltage is to weak and low to run a device like a motor. To run a motor we use Operational Voltage: a higher currenthigher-voltage flow used to power devices and not transmit data. Raspberry Pi and Arduino both output logical voltage and to control a motor we use a bridging device like an H-Bridge. It checks to see: if there's logical voltage analyze the pulse width and control the motors.

In telecommunication and computer science, serial communication is the process of sending data one bit at a time, sequentially, over a communication channel orcomputer bus. This is in contrast to parallel communication, where several bits are sent as a whole, on a link with several parallel channels.

Using Interrupts is a method to tell a program to wait for a specific condition to call a method in a code. It's like the computer turns on the screen saver and waits for you to move the mouse. Until then it does something else.

First Time Author Contest

Participated in the
First Time Author Contest

Raspberry Pi Contest

Participated in the
Raspberry Pi Contest