Picture of Mikey, the Robot with Vision

Mikey the Robot "sees the frog" using the Pixy camera vision system. Pixy costs about $70. and easily interfaces with an included cable to Arduino. Pixy can be "trained" to recognize up to seven different colors. Pixy returns the x,y coordinates of the center of a trained object, along with the width and height of the object. Pixy is open source and all the hardware and software are available through

In my setup, Pixy communicates with an Arduino which I call a "Sensor Interface." When the center of the target (frog) is toward the right (>170 out of 319), the red LED illuminates and the robot turns toward the right. When the frog is toward the left (<150 out of 319), the white LED illuminates and the robot turns to the left. When the frog is pretty much in the center (>150 and <170), both LEDs illuminate and the robot moves forward.



4WD Arduino Compatible Mobile

Arduino Motor

(2) Arduino

(7) NiMh AA

Misc: Red led, white led, (2) 220 ohm resistors, SPST toggle switch

Remove these adsRemove these ads by Signing Up

Step 1:

Picture of

Start by soldering wires to the motors on the mobile platform. I used an AA battery to make sure that positive on the red wire and negative on the black would cause each motor to move the platform forward. This needs to be consistent on all motors (which means the red wire will probably be on top of the right side motors and on the bottom of the left side motors).

Step 2:

Picture of

Secure the lower plate with screws, making sure that the motor wires come through open holes.

Step 3:

Picture of

Add battery holders. My choice of (7) AA batteries relates to a future recharging home base (which Mikey will locate using his vision--soon).

Step 4:

Picture of

Add the upper plate, switch, two Arduinos (one with a motor shield on top) and the Pixy Camera.

Files for the 3D printed switch holder can be found here:

Pixy is connected to the Arduino with a simple cable (included with Pixy). More information on connecting Pixy can be found here:

Step 5: Training Pixy

Picture of Training Pixy

Training Pixy to a color is mainly a question of holding a button (red for the first color, orange second, yellow third, green fourth, cyan fifth, blue sixth, violet seventh) then releasing. Place the object in front of the camera. When the LED is pretty much the color of the object, quickly press and release the button. The LED will blink red a few times if Pixy "bought" the object. That's it.

Step 6: Schematic

Picture of Schematic

This is the schematic.

Step 7: Sensor Software

The sensor software for Arduino is a modified version of their "hello world" sketch.

Download the Arduino library "" here.

Bring up the Arduino IDE and import the Pixy library by selecting Sketch➜Import Library in the Arduino IDE, and then browsing to the file that you just downloaded.

// begin license header // // This file is part of Pixy CMUcam5 or "Pixy" for short // // All Pixy source code is provided under the terms of the // GNU General Public License v2 ( // Those wishing to use Pixy source code, software and/or // technologies under different licensing terms should contact us at // Such licensing terms are available for // all portions of the Pixy codebase presented here. // // end license header //

#include #include

Pixy pixy;

int stuff=0; int position=150; const int red=5; const int white=9; const int redled=3; const int whiteled=8;

void setup() {

pinMode(red, OUTPUT); pinMode(white, OUTPUT); pinMode(redled, OUTPUT); pinMode(whiteled,OUTPUT); digitalWrite(redled, HIGH);//right test digitalWrite(whiteled,HIGH);//left test delay(2000); digitalWrite(redled,LOW);//right off digitalWrite(whiteled,LOW);//left off digitalWrite(red,LOW); digitalWrite(white,LOW); delay(2000);

Serial.begin(9600); Serial.print("Starting...\n"); }

void loop() { static int i = 0; int j; uint16_t blocks; char buf[32]; blocks = pixy.getBlocks(); if (blocks) { i++; if (i%50==0) { //sprintf(buf, "Detected %d:\n", blocks); // Serial.print(buf); //for (j=0; j170) { digitalWrite(red,HIGH); digitalWrite(redled,HIGH); delay(1000); digitalWrite(red,LOW); digitalWrite(redled,LOW); } if(pixy.blocks[j].x<150){ digitalWrite(white,HIGH); digitalWrite(whiteled,HIGH); delay(1000); digitalWrite(white,LOW); digitalWrite(whiteled,LOW); } if(pixy.blocks[j].x>149 && pixy.blocks[j].x<171) { digitalWrite(white,HIGH); digitalWrite(whiteled,HIGH); digitalWrite(red,HIGH); digitalWrite(redled,HIGH); delay(1000); digitalWrite(white,LOW); digitalWrite(whiteled,LOW); digitalWrite(red,LOW); digitalWrite(redled,LOW); } } } }}

Step 8: Motor Software

This sketch is loaded into the Arduino that has a motor shield on top.

Note that the "jump=1" moves to the second part of the program, where the robot chases the frog. The first part of the program is used to enable the robot to move around without getting stuck (a future enhancement--coming soon).

int val=0;
int val1=0; int valm=0; int valm1=0; int jump=1;

const int pwmA=3; const int pwmB=11; const int brakeA=9; const int brakeB=8; const int dirA=12; const int dirB=13; const int right=5; const int left=7;

void setup() { pinMode(dirA, OUTPUT); pinMode(brakeA, OUTPUT); pinMode(dirB, OUTPUT); pinMode(brakeB, OUTPUT); pinMode(right,INPUT); pinMode(left,INPUT); digitalWrite(dirA, HIGH);//forward A digitalWrite(brakeA,LOW);//release brakeA analogWrite(pwmA,100);//set speed A digitalWrite(dirB,HIGH);//forward B motor digitalWrite(brakeB,LOW);// analogWrite(pwmB,100);//set speed B delay(700);


void loop () { if (jump==0) { valm=analogRead(0); valm1=analogRead(1); if(valm>500 or valm1>500) { valm=analogRead(0); valm1=analogRead(1); if(valm>450 or valm1>450) { digitalWrite(brakeA, HIGH); digitalWrite(brakeB, HIGH); digitalWrite(dirA, LOW);//reverse A digitalWrite(brakeA, LOW);// analogWrite(pwmA, 200); digitalWrite(dirB,LOW); digitalWrite(brakeB,LOW); analogWrite(pwmB,200); delay(700);//backup digitalWrite(brakeA,HIGH);//stop one wheel analogWrite(pwmA,0); delay(1000); digitalWrite(brakeB,HIGH);//stop other wheel //start both wheels forward digitalWrite(dirA,HIGH); digitalWrite(brakeA,LOW); analogWrite(pwmA,140); digitalWrite(dirB,HIGH); digitalWrite(brakeB,LOW); analogWrite(pwmB,140); delay(1000); }}} //jump equal one analogWrite(pwmA,0); analogWrite(pwmB,0); val=digitalRead(right); val1=digitalRead(left); if (val==HIGH && val1==HIGH) { analogWrite(pwmA,140); analogWrite(pwmB,140); delay(1000); } if (val==HIGH &&val1==LOW) { analogWrite(pwmA, 140); analogWrite(pwmB,50); delay(1000); } if (val1==HIGH && val==LOW) { analogWrite(pwmB,140); analogWrite(pwmA,50); delay(1000); }


Step 9:

Picture of

I hope to enable Mikey to come out on command, run around and do things, then return (on his own) to recharge.

Having Pixy vision should simplify all these tasks.

I've changed the Pixy part of the software slightly so that time is not wasted reading empty blocks. Also, I've included the Arduino files for both Pixy and the motor shield.

farsadheidari2 months ago

The maximum distance should be?

MikeTheMaker (author)  farsadheidari2 months ago

Using the target frog in my video, about 10 feet (3 meters). With a larger target (and no objects of the same color to cause confusion), the distance can be increased greatly.

jeveda128 months ago


good news I went back to use your old files the ones on the blog, because the new ones are not compatible with my robot, I loaded and now is working but my robot is avoiding the colors instead of chasing, do you know what can be the problem. thank you for your help, I tried to do this other project that is similar of yours but i could not continue,it look difficult, I like yours better. if you want to take a look.

MikeTheMaker (author)  jeveda128 months ago

It sounds like you are getting close--congratulations!

I'm guessing that left and right are swapped somewhere if the robot is avoiding the colors. If the target color is on the right and the robot goes left, that can be fixed without too much trouble. (You could switch the wires going to the left and right motors, or switch the wires going from the Pixy Arduino to the motor Arduino--but I would suggest making the change in software instead). In the Arduino connected to the pixy, there is a line in the code: "const int red=5; const int white=9;" Try changing that to "const int red=9; const int white=5;"

I hope this helps.

I don`t know what is happening,I was able to make it go forward by changing the polarity on the motor, when it detect a color in the middle goes forward but it does not stop, and is hard to make it detect a color in the middle, when detect a color on the left or to the right(this is no problem to detect) turns half way and stays there, thank you for following up

MikeTheMaker (author)  jeveda128 months ago

It sounds like it is turning so much that the camera loses the color. When a color is on the left or right, the robot should only turn slightly. Perhaps if you reduce the pwmA and pwmB numbers in the software, it will slow down and turn less . . .

jeveda129 months ago


you are right, pixy is making it go crazy, when I turn it on does not move, but when pixy detects a color it spins around then reset power and goes around the other way for a second then spins again non stop. I power pixy and arduino with 7volts, the other arduino and shield with 7 volts. thank you

jeveda129 months ago

hello, I almost got it but robot is just pining around, do you now what can be wrong. i loaded one code on one arduino and one on the other as you said in the blog.are you using servos for wheels? I am using dc motor.

MikeTheMaker (author)  jeveda129 months ago

I am using dc motors.

You might try putting a lens cap over the Pixy. If the robot just sits there with the lens cap on, but goes crazy with the lens cap off, then you'll know that confusing signals are coming from Pixy. I put led's on pins 5 and 9 of the pixy so that I could see what commands the Pixy was giving to the motor controller.

jeveda129 months ago

spin aound one way and then arduino reset and spin the other war for one second then sping the other way, looks like arduino is reseting it self

MikeTheMaker (author)  jeveda129 months ago

Two things I can think of that can cause this type of problem:

1. If the supply voltage dips below 7 volts, then the Arduino may reset (supply voltage will dip when the motors come on because the motors use a lot of energy).

2. Motor noise (caused by "sparking" of brushes on dc motor) may reset the Arduino. Try reducing the "analogWrite(pwmA,200)" to something like "analogWrite(pwmA,75)" This will reduce the electrical noise and reduce the drain on the batteries. It will also slow the motors down quite a bit.

jeveda129 months ago


I am tried to load this program in to arduino but i did copy and paste and everything wento into one line and has a bunch of errors, you said the first part is for one arduino. and if I start with the second part on the other arduino it does not work. maybe is missing something, can you please provide with two different codes for each arduino. thank you.

MikeTheMaker (author)  jeveda129 months ago

I'm sorry you are having difficulty with the program. I have added the arduino files (last step of the instructables) along with a video showing improved performance of the Pixy program.

Unfortunately, when I cut and paste Arduino listings into the Instructables text block, all the formatting seems to disappear and that makes the code hard to read or copy.

jeveda129 months ago


I am tried to load this program in to arduino but i did copy and paste and everything wento into one line and has a bunch of errors, you said the first part is for one arduino. and if I start with the second part on the other arduino it does not work. maybe is missing something, can you please provide with two different codes for each arduino. thank you.

jeveda1210 months ago

I didn't know what to do with my pixy but thanks to you it will Come out from the box, is this the final release of your work or there is more, thank you for your ideas, I'm waiting for parts to see if I can build one like yours.

MikeTheMaker (author)  jeveda1210 months ago

I made a homing station:

and I hope to have more software and hardware enhancements within two or three weeks . . . .

rohit74710 months ago

Hi...i am trying to make similar robot using almost the same components. But my query is why did you use two arduinos?? Is it not possible to do in one??


MikeTheMaker (author)  rohit74710 months ago

The short answer--one Arduino should be doable.

I tried using one Arduino, but had problems which I blamed (perhaps incorrectly) on a possible conflict between the motor shield and Pixy. Adding the second Arduino solved my problem, but whether it is because of pin usage or software is something I did not try to resolve. I'm using the motor shield because it allows me to measure motor current. In the independent "run around the room" mode (not yet posted on instructables) I use the "stuck motor draws more current feature" to detect when the robot needs to change directions.

Yes i am facing the same problem. I am using a single arduino and motor shield behaves strangely with pixy.....I have been trying to solve it but in vain.

MikeTheMaker (author)  rohit74710 months ago

I thought Pixy was using SCL and SDA on the Arduino (according to
documentation), but I somehow burned out pin 3 of the Arduino Uno (which
is used for the motor shield). I was trying so many things that I
can't pinpoint what I may have done--I'm just suspicious that there is
pin conflict between the motor shield and the Pixy. Separately, they
each worked fine. Then together, they didn't (and wouldn't work
separately after I had had them together).