Introduction: Mikey, the Robot With Vision
Mikey the Robot "sees the frog" using the Pixy camera vision system. Pixy costs about $70. and easily interfaces with an included cable to Arduino. Pixy can be "trained" to recognize up to seven different colors. Pixy returns the x,y coordinates of the center of a trained object, along with the width and height of the object. Pixy is open source and all the hardware and software are available through cmucam.org.
In my setup, Pixy communicates with an Arduino which I call a "Sensor Interface." When the center of the target (frog) is toward the right (>170 out of 319), the red LED illuminates and the robot turns toward the right. When the frog is toward the left (<150 out of 319), the white LED illuminates and the robot turns to the left. When the frog is pretty much in the center (>150 and <170), both LEDs illuminate and the robot moves forward.
Parts:
Pixy Camera---Amazon.com
4WD Arduino Compatible Mobile Platform--Makershed.com
Arduino Motor Shield--Makershed.com
(2) Arduino Uno--Amazon.com
(7) NiMh AA batteries--Amazon.com
Misc: Red led, white led, (2) 220 ohm resistors, SPST toggle switch
Step 1:
Start by soldering wires to the motors on the mobile platform. I used an AA battery to make sure that positive on the red wire and negative on the black would cause each motor to move the platform forward. This needs to be consistent on all motors (which means the red wire will probably be on top of the right side motors and on the bottom of the left side motors).
Step 2:
Secure the lower plate with screws, making sure that the motor wires come through open holes.
Step 3:
Add battery holders. My choice of (7) AA batteries relates to a future recharging home base (which Mikey will locate using his vision--soon).
Step 4:
Add the upper plate, switch, two Arduinos (one with a motor shield on top) and the Pixy Camera.
Files for the 3D printed switch holder can be found here:
http://www.thingiverse.com/thing:331319
Pixy is connected to the Arduino with a simple cable (included with Pixy). More information on connecting Pixy can be found here:
Step 5: Training Pixy
Training Pixy to a color is mainly a question of holding a button (red for the first color, orange second, yellow third, green fourth, cyan fifth, blue sixth, violet seventh) then releasing. Place the object in front of the camera. When the LED is pretty much the color of the object, quickly press and release the button. The LED will blink red a few times if Pixy "bought" the object. That's it.
Step 6: Schematic
This is the schematic.
Step 7: Sensor Software
The sensor software for Arduino is a modified version of their "hello world" sketch.
Download the Arduino library "arduino_pixy-x.y.z.zip" here.
http://cmucam.org/attachments/download/958/arduino...
Bring up the Arduino IDE and import the Pixy library by selecting Sketch➜Import Library in the Arduino IDE, and then browsing to the Pixy.zip file that you just downloaded.
//
// begin license header // // This file is part of Pixy CMUcam5 or "Pixy" for short // // All Pixy source code is provided under the terms of the // GNU General Public License v2 (http://www.gnu.org/licenses/gpl-2.0.html). // Those wishing to use Pixy source code, software and/or // technologies under different licensing terms should contact us at // cmucam@cs.cmu.edu. Such licensing terms are available for // all portions of the Pixy codebase presented here. // // end license header //
#include #include
Pixy pixy;
int stuff=0; int position=150; const int red=5; const int white=9; const int redled=3; const int whiteled=8;
void setup() {
pinMode(red, OUTPUT); pinMode(white, OUTPUT); pinMode(redled, OUTPUT); pinMode(whiteled,OUTPUT); digitalWrite(redled, HIGH);//right test digitalWrite(whiteled,HIGH);//left test delay(2000); digitalWrite(redled,LOW);//right off digitalWrite(whiteled,LOW);//left off digitalWrite(red,LOW); digitalWrite(white,LOW); delay(2000);
Serial.begin(9600); Serial.print("Starting...\n"); }
void loop() { static int i = 0; int j; uint16_t blocks; char buf[32]; blocks = pixy.getBlocks(); if (blocks) { i++; if (i%50==0) { //sprintf(buf, "Detected %d:\n", blocks); // Serial.print(buf); //for (j=0; j170) { digitalWrite(red,HIGH); digitalWrite(redled,HIGH); delay(1000); digitalWrite(red,LOW); digitalWrite(redled,LOW); } if(pixy.blocks[j].x<150){ digitalWrite(white,HIGH); digitalWrite(whiteled,HIGH); delay(1000); digitalWrite(white,LOW); digitalWrite(whiteled,LOW); } if(pixy.blocks[j].x>149 && pixy.blocks[j].x<171) { digitalWrite(white,HIGH); digitalWrite(whiteled,HIGH); digitalWrite(red,HIGH); digitalWrite(redled,HIGH); delay(1000); digitalWrite(white,LOW); digitalWrite(whiteled,LOW); digitalWrite(red,LOW); digitalWrite(redled,LOW); } } } }}
Step 8: Motor Software
This sketch is loaded into the Arduino that has a motor shield on top.
Note that the "jump=1" moves to the second part of the program, where the robot chases the frog. The first part of the program is used to enable the robot to move around without getting stuck (a future enhancement--coming soon).
int val=0;
int val1=0; int valm=0; int valm1=0; int jump=1;
const int pwmA=3; const int pwmB=11; const int brakeA=9; const int brakeB=8; const int dirA=12; const int dirB=13; const int right=5; const int left=7;
void setup() { pinMode(dirA, OUTPUT); pinMode(brakeA, OUTPUT); pinMode(dirB, OUTPUT); pinMode(brakeB, OUTPUT); pinMode(right,INPUT); pinMode(left,INPUT); digitalWrite(dirA, HIGH);//forward A digitalWrite(brakeA,LOW);//release brakeA analogWrite(pwmA,100);//set speed A digitalWrite(dirB,HIGH);//forward B motor digitalWrite(brakeB,LOW);// analogWrite(pwmB,100);//set speed B delay(700);
}
void loop () { if (jump==0) { valm=analogRead(0); valm1=analogRead(1); if(valm>500 or valm1>500) { valm=analogRead(0); valm1=analogRead(1); if(valm>450 or valm1>450) { digitalWrite(brakeA, HIGH); digitalWrite(brakeB, HIGH); digitalWrite(dirA, LOW);//reverse A digitalWrite(brakeA, LOW);// analogWrite(pwmA, 200); digitalWrite(dirB,LOW); digitalWrite(brakeB,LOW); analogWrite(pwmB,200); delay(700);//backup digitalWrite(brakeA,HIGH);//stop one wheel analogWrite(pwmA,0); delay(1000); digitalWrite(brakeB,HIGH);//stop other wheel //start both wheels forward digitalWrite(dirA,HIGH); digitalWrite(brakeA,LOW); analogWrite(pwmA,140); digitalWrite(dirB,HIGH); digitalWrite(brakeB,LOW); analogWrite(pwmB,140); delay(1000); }}} //jump equal one analogWrite(pwmA,0); analogWrite(pwmB,0); val=digitalRead(right); val1=digitalRead(left); if (val==HIGH && val1==HIGH) { analogWrite(pwmA,140); analogWrite(pwmB,140); delay(1000); } if (val==HIGH &&val1==LOW) { analogWrite(pwmA, 140); analogWrite(pwmB,50); delay(1000); } if (val1==HIGH && val==LOW) { analogWrite(pwmB,140); analogWrite(pwmA,50); delay(1000); }
}
Step 9:
I hope to enable Mikey to come out on command, run around and do things, then return (on his own) to recharge.
Having Pixy vision should simplify all these tasks.
I've changed the Pixy part of the software slightly so that time is not wasted reading empty blocks. Also, I've included the Arduino files for both Pixy and the motor shield.

Participated in the
Sensors Contest
51 Comments
5 years ago
The pixy camera light is detecting the colored signature below the camera lens, but the colored LEDs aren't lighting up. I have both LEDS wired to 220 ohm resistors that meet together at a single ground.
Reply 5 years ago
It sounds like the Pixy camera is working, but the Arduino is not interpreting the data. Did you use the "hello_lightdetect0518.ino" file at the end of step #9 for the Arduino connected to the camera--that's the best way to avoid any typing errors.
Reply 5 years ago
When I run the hello_lightdetect0518.ino file, I do not get any serial reading. If I run the native hello_world from the pixy library I get active serial readings. Both programs do not light up the LEDs
Reply 5 years ago
Let's try something else--maybe the signature needs to be set in your Pixy. After the blocks = pixy.getBlocks(); statement, make sure it looks like the code below (also make sure that Pixy is trained to the first color signature for the color you want to track. The "signature" is set for 1 to 7 to determine which trained color is being searched for).
if (blocks)
{
i++;
if (i%5==0){
if(pixy.blocks[0].signature==1) {
if(pixy.blocks[0].x>170)
Reply 5 years ago
I've been trying to isolate the issue, but it's all coming back to the pixy camera not outsourcing block data from the pixy camera. The serial print says "starting..." but doesn't move past the initial starting line in the serial monitor.
Reply 5 years ago
When the ball moves left or right, or even forward I am no getting any of the LED's to light. In Pixymon I can see that ball very clearly, but the outsource from the Pixy to the Arduino show below isn't communicating properly.
Question 5 years ago
Hey there Mike, Ive used both codes in step 7 and step 8 to control my robot. In your motor control it seems that there is a problem. I uploaded step 7 code into the Arduino closest to the pixy camera and uploaded step 8 to the arduino below the motor shield. My pixy camera is "taught" a specific signature as well. Any ideas as to how I can get your design to work? Thankyou for your time.
Answer 5 years ago
Let's ignore the motors for a moment and make sure that the information coming from the Pixy arduino is correct. Do both leds illuminate when the target is straight ahead? Does only one led come on when the target is to the right, then the other to the left?
Once we know that is correct, we can diagnose the situation further.
Question 5 years ago
Can you please show more wiring pictures or explain unshown diagram wiring. I’m attempting to recreate Mikey and I’ve noticed the yellow wire next to the motor shield. Also is this your full code? I’m using all the same components and I’m still not getting proper motor response. Any information would be greatly appreciated!
Answer 5 years ago
If you follow the schematic diagram in step #6, everything should be just fine. The yellow wire next to the motor shield connects Vin on the screw terminals to Vin on the power header--this is only necessary if the "Vin connect" trace on the back side of the motor shield has been cut (mine was cut for use in another project).
When testing, watch the red and white leds on pins 3 and 8 of the "sensor Arduino." If they are both on, the left and right motors should be active. If one led is on, the motors on one side should be active. If no leds are illuminated, the motors will not turn.
Most problems relate to the image that the camera "sees." If a window or lamp is in the camera's image area, the camera will probably be "blinded' by the light and fail to see the target. If the target color (green in my example) can be found on objects other than the target, the Pixy camera will "see" and "identify" those and the system will act confused due to multiple target identification.
6 years ago
hello sir, i have some questions regarding your code. some terminologies you used, i am unable to comprehend the meaning. the terms include, val, val1, valm, valm1, valm3 and the use of ctr. Thanks
7 years ago
hi sir..can u email me the actual coding for this robot? im getting confuse because there are two separate coding which are the code that u attach that needs to be download and the one u copy and paste to your website. Both of the codes i have try before but does not working. I have connect all the wire connection exactly the same as the schematic diagram that you provided. Im really interesting with this robot. I hope u can help me :) this my email
syahir_tombstone28@yahoo.com
7 years ago
Awesome !!!!!!
I have also tried to track a colour with Pixy after reading your post but only in x-direction because i want to use it in my Camera Drolly so that i can shoot a video by tracking a colour of any object. for that i have used PWM controller but because of this speed not varies as the object change its speed. Now i am planning to control with PID so that it moves in desired speed. Everything is ready but only need help to use PID. My email: lohanibiswas@yahoo.com please respond to this mail. So that i can send you all the file.
Hope for your positive response.
Thanks.
7 years ago
Can I use one of pixy library codes and then add the tasks i want the arduino to do when pixy see a certain color (if yes which code should i use)
Reply 7 years ago
Take this code and examine the bold parts. The "if" statements indicate "see color to the right--see color to the left--see color straight ahead." After the "if" question is a { That is the place you put your code to do what you want. In my case, I make pins "high," wait .2 seconds then make them low.
Copy this code exactly and set up led's on pins 3 and 8.
Train the Pixy to a color and the leds should come on to indicate left, right or straight ahead.
When this works, change the code after the "if" to do whatever you want.
#include <SPI.h>
#include<Pixy.h>
Pixy pixy;
int stuff=0;
int position=150;
const int red=5;
const int white=9;
const int redled=3;
const int whiteled=8;
void setup() {
pinMode(red, OUTPUT);
pinMode(white, OUTPUT);
pinMode(redled, OUTPUT);
pinMode(whiteled,OUTPUT);
digitalWrite(red,LOW);
digitalWrite(white,LOW);
digitalWrite(redled, HIGH);//right test
digitalWrite(whiteled,HIGH);//left test
delay(2000);
digitalWrite(redled,LOW);//right off
digitalWrite(whiteled,LOW);//left off
delay(2000);
Serial.begin(9600);
Serial.print("Starting...\n");
}
void loop() {
static int i = 0;
int j;
uint16_t blocks;
char buf[32];
blocks = pixy.getBlocks();
if (blocks) {
i++;
if (i%5==0) {
//sprintf(buf, "Detected %d:\n", blocks);
// Serial.print(buf); //
//for (j=0; j=170;) {
if(pixy.blocks[j].x>190){
digitalWrite(red,HIGH);
digitalWrite(redled,HIGH);
delay(200);
digitalWrite(red,LOW);
digitalWrite(redled,LOW);
}
if(pixy.blocks[j].x<170){
digitalWrite(white,HIGH);
digitalWrite(whiteled,HIGH);
delay(200);
digitalWrite(white,LOW);
digitalWrite(whiteled,LOW);
}
if(pixy.blocks[j].x>169 && pixy.blocks[j].x<191) {
digitalWrite(white,HIGH);
digitalWrite(whiteled,HIGH);
digitalWrite(red,HIGH);
digitalWrite(redled,HIGH);
delay(200);
digitalWrite(white,LOW);
digitalWrite(whiteled,LOW);
digitalWrite(red,LOW);
digitalWrite(redled,LOW);
} } } }
Reply 7 years ago
Hi i tryed using some parts of this code but it still not working I'm not going to use the pixy servo i tryed using the pixy if statement to make the arduino blink when it see one color but it still not working
Reply 7 years ago
It has been a while since I worked on this, but I think you have to be set up to read an array before the if statement will find the correct data.
static int i = 0;
int j;
uint16_t blocks;
char buf[32];
blocks = pixy.getBlocks();
if (blocks) {
i++;
if (i%5==0) {
7 years ago
Hi, I have a question about the Pixy. In the beginning, the pixy used to flash a light whenever it see the signature but today it didn't flash the light so what can the problem be
Reply 7 years ago
Possibly it has been reset or set to another color.
Look here
http://cmucam.org/projects/cmucam5/wiki/Teach_Pixy...
for clear instructions on how to set the colors.
7 years ago
Hi can you tell me if it's possible to make the Arduino generate power to a relay and stop a servo from moving when ever the pixy see an object with a certain color what i mean is it possible to program the Arduino to do some tasks only when the pixy see one color