Camera Equipped RC Car Using Sixth Sense

Using open source code from 6th sense and webpages, we have created a motor toy car that is controlled using 6th sense technology. A webcam picks up different hand movements from the user who has 4 colored finger gloves, and will send that data over to the car, which will move appropriately. From there the car will live stream a video feed to the Pi's webpage!

~~~~~

This project was made forWashington State University's 24hr Hardware Hackathon. We will be continuing to perfect this instructable and add more details / additions in the future! Please let us know if you have any questions!

Step 1: Gather Materials

General Materials:
- Toy R/C monster truck, large enough to fit a Raspberry Pi or Arduino on.

- Rasbperry Pi (or Arduino)

Pertaining to the Pi:
- Micro SD card (8GB or larger)
- Breadboard

- Jumper Wires
- USB Dongle (wifi adapter)
- Pi-Camera
- Power cord
- Monitor to connect to Pi
- Portable mouse / keyboard

Step 2: Install Raspbian Onto Your New Pi

We recommend using at least an 8 GB Micro USB for installing Raspbian. We went with a 16GB just to be safe. We also used the Raspberry Pi 2 body.

Step 3: Install Open Source Cam Web Interface

Install the web streaming software. We used RPi Cam Web Interface.

Follow the instructions here: http://elinux.org/RPi-Cam-Web-Interface

Make sure to run the install again after you install the software to run the configure options.

You need to "start" the camera and should set it to autorun if you want it to start automatically every time you boot your Pi

Step 4: Configure Open Source Code for 6th Sense Technology

Install the Sixth Sense Tech software

Open Source Code Here: http://www.pranavmistry.com/projects/sixthsense/

If you are using a 64 bit computer, you may need to make some modifications.

We recommend the following YouTube video for assistance: https://code.google.com/p/sixthsense/wiki/Tutoria...

Step 5: Hacking the Toy Car Transmitter and Receiver: Testing

First we need to find a way to turn the physical buttons on the transmitter and make a micro-controller enable these contacts.

Instructions

1: Remove all four buttons.

2: Find what is the positive and negative contact of each of the switches.

3. Solder jumper cables to all of the switches' contacts.

4: If the contacts have been surface mounted then you might want to glue the transmitter board to a prototyping board. Rewire the jumpers to that prototyping board.

5: Once that is done, connect the jumpers to their respective transistors (NPN) as shown in the schematic.

6: Now connect the base of the transistors to their respective pins on the micro-controller.

7: Connect the power cables and double check the circuit.

8: Connect power to the reviving circuit board.

9: Upload the sketch onto the micro controller.

10: The car must start moving according to sketch completely wireless.

(this sketch is for an Arduino microcontroller, we used this for testing and later switched to Python on the Raspberry PI for control)

Arduino Sketch

/*
* Hardware Hackathon

* Test Code 1, this will control the R/C car, this code does not accept any user input

* 14/11/15

*/

int forwardPin = 2;

int backwardPin = 7;

int leftPin = 8;

int rightPin = 9;

void setup(){

pinMode(forwardPin, OUTPUT);

pinMode(backwardPin, OUTPUT);

pinMode(leftPin, OUTPUT);

pinMode(rightPin, OUTPUT);

Serial.begin(9600);

}

void loop(){

// Move Forward

digitalWrite(forwardPin, HIGH);

Serial.println("Move Forward");

delay(750);

digitalWrite(forwardPin, LOW);

// Move Backward

digitalWrite(backwardPin, HIGH);

Serial.println("Move Backward");

delay(750);

digitalWrite(backwardPin, LOW);

// Turn Left

digitalWrite(forwardPin, HIGH);

digitalWrite(leftPin, HIGH);

Serial.println("Turn Left");

delay(750);

digitalWrite(forwardPin, LOW);

digitalWrite(leftPin, LOW);

// Turn Right

digitalWrite(forwardPin, HIGH);

digitalWrite(rightPin, HIGH);

Serial.println("Turn Right"); delay(750);

digitalWrite(forwardPin, LOW);

digitalWrite(rightPin, LOW);

}

Step 6: Assemble Final Code and Compile!

We had our Sixth Sense computer setup as a web server and modified the Sixth Sense program to write a single character to a text file that was on the web server and accessed that file from the Raspberry Pi to read the values as inputs for the car control.

The text file simply contained one of five characters based on the current position of each of the four colored markers:

1, 2, 3, 4, 0

The Python code on the Raspberry Pi (or C code on an Arduino) reads the character from the web file and decodes it into the appropriate action for the car.

Sixth Sense Code can be found at our git repository:

https://github.eecs.wsu.edu/mblaisde/TPAHRC

This is the Python code for the Raspberry Pi that ties everything together.
Paste this into a python file, and run.

#Hardware Hackathon
#Code 1

#14/11/15 #Ground -> pin 20 grey

#5V -> pin 2 #3.3 -> pin 1

#receive the data #!/usr/bin/env python

import urllib.request

print ("Pi Home server running!")

forwardPin = 17 # GPIO pin3 white

backwardPin = 23 # pin 5 purple

leftPin = 27 #pin 13 grey

rightPin = 22 #pin 15 red

from time import sleep from RPi import GPIO import time

# set our mode GPIO.setmode(GPIO.BCM)

def setup():

GPIO.setup(forwardPin, GPIO.OUT)

GPIO.setup(backwardPin, GPIO.OUT)

GPIO.setup(leftPin, GPIO.OUT)

GPIO.setup(rightPin, GPIO.OUT)

def loop():

response=urllib.request.urlopen('http://192.168.1.114/WriteLines.txt')

html=str(response.read())[2:-1]

html = html.split("\\r\\n")[:-1]

print(html)

print(html[0])

if (html[0]== "1"):

# Move Forwards

GPIO.output(forwardPin, True)

time.sleep(.75)

GPIO.output(forwardPin, False)

print("HTML 1")

elif (html[0]== "2"):

# Move Backward

GPIO.output(backwardPin, True)

time.sleep(.75)

GPIO.output(backwardPin, False)

print("HTML 2")

elif (html[0]== "3"):

#// Turn Left

GPIO.output(forwardPin, True)

GPIO.output(leftPin, True)

time.sleep(.75)

GPIO.output(forwardPin, False)

GPIO.output(leftPin, False)

print("HTML 3 left")

elif (html[0]== "4"):

#// Turn Right

GPIO.output(forwardPin, True)

GPIO.output(rightPin, True)

time.sleep(.75)

GPIO.output(forwardPin, False)

GPIO.output(rightPin, False)

print("HTML 4")

def main():

GPIO.setwarnings(False)

setup()

while(1):

loop()

time.sleep(.1)

main()