Using open source code from 6th sense and webpages, we have created a motor toy car that is controlled using 6th sense technology. A webcam picks up different hand movements from the user who has 4 colored finger gloves, and will send that data over to the car, which will move appropriately. From there the car will live stream a video feed to the Pi's webpage!
This project was made forWashington State University's 24hr Hardware Hackathon. We will be continuing to perfect this instructable and add more details / additions in the future! Please let us know if you have any questions!
Teachers! Did you use this instructable in your classroom?
Add a Teacher Note to share how you incorporated it into your lesson.
Step 1: Gather Materials
- Toy R/C monster truck, large enough to fit a Raspberry Pi or Arduino on.
- Rasbperry Pi (or Arduino)
Pertaining to the Pi:
- Micro SD card (8GB or larger)
- Jumper Wires
- USB Dongle (wifi adapter)
- Power cord
- Monitor to connect to Pi
- Portable mouse / keyboard
Step 2: Install Raspbian Onto Your New Pi
We recommend using at least an 8 GB Micro USB for installing Raspbian. We went with a 16GB just to be safe. We also used the Raspberry Pi 2 body.
Step 3: Install Open Source Cam Web Interface
Install the web streaming software. We used RPi Cam Web Interface.
Follow the instructions here: http://elinux.org/RPi-Cam-Web-Interface
Make sure to run the install again after you install the software to run the configure options.
You need to "start" the camera and should set it to autorun if you want it to start automatically every time you boot your Pi
Step 4: Configure Open Source Code for 6th Sense Technology
Install the Sixth Sense Tech software
Open Source Code Here: http://www.pranavmistry.com/projects/sixthsense/
If you are using a 64 bit computer, you may need to make some modifications.
We recommend the following YouTube video for assistance: https://code.google.com/p/sixthsense/wiki/Tutoria...
Step 5: Hacking the Toy Car Transmitter and Receiver: Testing
First we need to find a way to turn the physical buttons on the transmitter and make a micro-controller enable these contacts.
1: Remove all four buttons.
2: Find what is the positive and negative contact of each of the switches.
3. Solder jumper cables to all of the switches' contacts.
4: If the contacts have been surface mounted then you might want to glue the transmitter board to a prototyping board. Rewire the jumpers to that prototyping board.
5: Once that is done, connect the jumpers to their respective transistors (NPN) as shown in the schematic.
6: Now connect the base of the transistors to their respective pins on the micro-controller.
7: Connect the power cables and double check the circuit.
8: Connect power to the reviving circuit board.
9: Upload the sketch onto the micro controller.
10: The car must start moving according to sketch completely wireless.
(this sketch is for an Arduino microcontroller, we used this for testing and later switched to Python on the Raspberry PI for control)
* Hardware Hackathon
* Test Code 1, this will control the R/C car, this code does not accept any user input
int forwardPin = 2;
int backwardPin = 7;
int leftPin = 8;
int rightPin = 9;
// Move Forward
// Move Backward
// Turn Left
// Turn Right
Serial.println("Turn Right"); delay(750);
Step 6: Assemble Final Code and Compile!
We had our Sixth Sense computer setup as a web server and modified the Sixth Sense program to write a single character to a text file that was on the web server and accessed that file from the Raspberry Pi to read the values as inputs for the car control.
The text file simply contained one of five characters based on the current position of each of the four colored markers:
1, 2, 3, 4, 0
The Python code on the Raspberry Pi (or C code on an Arduino) reads the character from the web file and decodes it into the appropriate action for the car.
Sixth Sense Code can be found at our git repository:
This is the Python code for the Raspberry Pi that ties everything together.
Paste this into a python file, and run.
#14/11/15 #Ground -> pin 20 grey
#5V -> pin 2 #3.3 -> pin 1
#receive the data #!/usr/bin/env python
print ("Pi Home server running!")
forwardPin = 17 # GPIO pin3 white
backwardPin = 23 # pin 5 purple
leftPin = 27 #pin 13 grey
rightPin = 22 #pin 15 red
from time import sleep from RPi import GPIO import time
# set our mode GPIO.setmode(GPIO.BCM)
html = html.split("\\r\\n")[:-1]
if (html== "1"):
# Move Forwards
elif (html== "2"):
# Move Backward
elif (html== "3"):
#// Turn Left
print("HTML 3 left")
elif (html== "4"):
#// Turn Right