Mikey the Robot "sees the frog" using the Pixy camera vision system. Pixy costs about $70. and easily interfaces with an included cable to Arduino. Pixy can be "trained" to recognize up to seven different colors. Pixy returns the x,y coordinates of the center of a trained object, along with the width and height of the object. Pixy is open source and all the hardware and software are available through cmucam.org.

In my setup, Pixy communicates with an Arduino which I call a "Sensor Interface." When the center of the target (frog) is toward the right (>170 out of 319), the red LED illuminates and the robot turns toward the right. When the frog is toward the left (<150 out of 319), the white LED illuminates and the robot turns to the left. When the frog is pretty much in the center (>150 and <170), both LEDs illuminate and the robot moves forward.


Pixy Camera---Amazon.com

4WD Arduino Compatible Mobile Platform--Makershed.com

Arduino Motor Shield--Makershed.com

(2) Arduino Uno--Amazon.com

(7) NiMh AA batteries--Amazon.com

Misc: Red led, white led, (2) 220 ohm resistors, SPST toggle switch

Step 1:

Start by soldering wires to the motors on the mobile platform. I used an AA battery to make sure that positive on the red wire and negative on the black would cause each motor to move the platform forward. This needs to be consistent on all motors (which means the red wire will probably be on top of the right side motors and on the bottom of the left side motors).

Step 2:

Secure the lower plate with screws, making sure that the motor wires come through open holes.

Step 3:

Add battery holders. My choice of (7) AA batteries relates to a future recharging home base (which Mikey will locate using his vision--soon).

Step 4:

Add the upper plate, switch, two Arduinos (one with a motor shield on top) and the Pixy Camera.

Files for the 3D printed switch holder can be found here:


Pixy is connected to the Arduino with a simple cable (included with Pixy). More information on connecting Pixy can be found here:


Step 5: Training Pixy

Training Pixy to a color is mainly a question of holding a button (red for the first color, orange second, yellow third, green fourth, cyan fifth, blue sixth, violet seventh) then releasing. Place the object in front of the camera. When the LED is pretty much the color of the object, quickly press and release the button. The LED will blink red a few times if Pixy "bought" the object. That's it.

Step 6: Schematic

This is the schematic.

Step 7: Sensor Software

The sensor software for Arduino is a modified version of their "hello world" sketch.

Download the Arduino library "arduino_pixy-x.y.z.zip" here.


Bring up the Arduino IDE and import the Pixy library by selecting Sketch➜Import Library in the Arduino IDE, and then browsing to the Pixy.zip file that you just downloaded.

// begin license header // // This file is part of Pixy CMUcam5 or "Pixy" for short // // All Pixy source code is provided under the terms of the // GNU General Public License v2 (http://www.gnu.org/licenses/gpl-2.0.html). // Those wishing to use Pixy source code, software and/or // technologies under different licensing terms should contact us at // cmucam@cs.cmu.edu. Such licensing terms are available for // all portions of the Pixy codebase presented here. // // end license header //

#include #include

Pixy pixy;

int stuff=0; int position=150; const int red=5; const int white=9; const int redled=3; const int whiteled=8;

void setup() {

pinMode(red, OUTPUT); pinMode(white, OUTPUT); pinMode(redled, OUTPUT); pinMode(whiteled,OUTPUT); digitalWrite(redled, HIGH);//right test digitalWrite(whiteled,HIGH);//left test delay(2000); digitalWrite(redled,LOW);//right off digitalWrite(whiteled,LOW);//left off digitalWrite(red,LOW); digitalWrite(white,LOW); delay(2000);

Serial.begin(9600); Serial.print("Starting...\n"); }

void loop() { static int i = 0; int j; uint16_t blocks; char buf[32]; blocks = pixy.getBlocks(); if (blocks) { i++; if (i%50==0) { //sprintf(buf, "Detected %d:\n", blocks); // Serial.print(buf); //for (j=0; j170) { digitalWrite(red,HIGH); digitalWrite(redled,HIGH); delay(1000); digitalWrite(red,LOW); digitalWrite(redled,LOW); } if(pixy.blocks[j].x<150){ digitalWrite(white,HIGH); digitalWrite(whiteled,HIGH); delay(1000); digitalWrite(white,LOW); digitalWrite(whiteled,LOW); } if(pixy.blocks[j].x>149 && pixy.blocks[j].x<171) { digitalWrite(white,HIGH); digitalWrite(whiteled,HIGH); digitalWrite(red,HIGH); digitalWrite(redled,HIGH); delay(1000); digitalWrite(white,LOW); digitalWrite(whiteled,LOW); digitalWrite(red,LOW); digitalWrite(redled,LOW); } } } }}

Step 8: Motor Software

This sketch is loaded into the Arduino that has a motor shield on top.

Note that the "jump=1" moves to the second part of the program, where the robot chases the frog. The first part of the program is used to enable the robot to move around without getting stuck (a future enhancement--coming soon).

int val=0;
int val1=0; int valm=0; int valm1=0; int jump=1;

const int pwmA=3; const int pwmB=11; const int brakeA=9; const int brakeB=8; const int dirA=12; const int dirB=13; const int right=5; const int left=7;

void setup() { pinMode(dirA, OUTPUT); pinMode(brakeA, OUTPUT); pinMode(dirB, OUTPUT); pinMode(brakeB, OUTPUT); pinMode(right,INPUT); pinMode(left,INPUT); digitalWrite(dirA, HIGH);//forward A digitalWrite(brakeA,LOW);//release brakeA analogWrite(pwmA,100);//set speed A digitalWrite(dirB,HIGH);//forward B motor digitalWrite(brakeB,LOW);// analogWrite(pwmB,100);//set speed B delay(700);


void loop () { if (jump==0) { valm=analogRead(0); valm1=analogRead(1); if(valm>500 or valm1>500) { valm=analogRead(0); valm1=analogRead(1); if(valm>450 or valm1>450) { digitalWrite(brakeA, HIGH); digitalWrite(brakeB, HIGH); digitalWrite(dirA, LOW);//reverse A digitalWrite(brakeA, LOW);// analogWrite(pwmA, 200); digitalWrite(dirB,LOW); digitalWrite(brakeB,LOW); analogWrite(pwmB,200); delay(700);//backup digitalWrite(brakeA,HIGH);//stop one wheel analogWrite(pwmA,0); delay(1000); digitalWrite(brakeB,HIGH);//stop other wheel //start both wheels forward digitalWrite(dirA,HIGH); digitalWrite(brakeA,LOW); analogWrite(pwmA,140); digitalWrite(dirB,HIGH); digitalWrite(brakeB,LOW); analogWrite(pwmB,140); delay(1000); }}} //jump equal one analogWrite(pwmA,0); analogWrite(pwmB,0); val=digitalRead(right); val1=digitalRead(left); if (val==HIGH && val1==HIGH) { analogWrite(pwmA,140); analogWrite(pwmB,140); delay(1000); } if (val==HIGH &&val1==LOW) { analogWrite(pwmA, 140); analogWrite(pwmB,50); delay(1000); } if (val1==HIGH && val==LOW) { analogWrite(pwmB,140); analogWrite(pwmA,50); delay(1000); }


Step 9:

I hope to enable Mikey to come out on command, run around and do things, then return (on his own) to recharge.

Having Pixy vision should simplify all these tasks.

I've changed the Pixy part of the software slightly so that time is not wasted reading empty blocks. Also, I've included the Arduino files for both Pixy and the motor shield.

<p>hello sir, i have some questions regarding your code. some terminologies you used, i am unable to comprehend the meaning. the terms include, val, val1, valm, valm1, valm3 and the use of ctr. Thanks </p>
<p>hi sir..can u email me the actual coding for this robot? im getting confuse because there are two separate coding which are the code that u attach that needs to be download and the one u copy and paste to your website. Both of the codes i have try before but does not working. I have connect all the wire connection exactly the same as the schematic diagram that you provided. Im really interesting with this robot. I hope u can help me :) this my email</p><p>syahir_tombstone28@yahoo.com</p>
<p>Awesome !!!!!!</p><p>I have also tried to track a colour with Pixy after reading your post but only in x-direction because i want to use it in my Camera Drolly so that i can shoot a video by tracking a colour of any object. for that i have used PWM controller but because of this speed not varies as the object change its speed. Now i am planning to control with PID so that it moves in desired speed. Everything is ready but only need help to use PID. My email: lohanibiswas@yahoo.com please respond to this mail. So that i can send you all the file.</p><p>Hope for your positive response.</p><p>Thanks.</p>
Can I use one of pixy library codes and then add the tasks i want the arduino to do when pixy see a certain color (if yes which code should i use)
<p>Take this code and examine the <strong>bold</strong> parts. The &quot;if&quot; statements indicate &quot;see color to the right--see color to the left--see color straight ahead.&quot; After the &quot;if&quot; question is a <strong>{ </strong>That is the place you put your code to do what you want. In my case, I make pins &quot;high,&quot; wait .2 seconds then make them low.</p><p>Copy this code exactly and set up led's on pins 3 and 8. </p><p>Train the Pixy to a color and the leds should come on to indicate left, right or straight ahead.</p><p>When this works, change the code after the &quot;if&quot; to do whatever you want.</p><p>#include &lt;SPI.h&gt;<br>#include&lt;Pixy.h&gt;<br>Pixy pixy;<br>int stuff=0;<br> int position=150;<br> const int red=5;<br> const int white=9;<br> const int redled=3; <br>const int whiteled=8;<br>void setup() { <br>pinMode(red, OUTPUT);<br> pinMode(white, OUTPUT); <br>pinMode(redled, OUTPUT);<br> pinMode(whiteled,OUTPUT);<br> digitalWrite(red,LOW);<br> digitalWrite(white,LOW); <br> digitalWrite(redled, HIGH);//right test <br>digitalWrite(whiteled,HIGH);//left test<br> delay(2000); <br>digitalWrite(redled,LOW);//right off <br>digitalWrite(whiteled,LOW);//left off <br><br>delay(2000); <br>Serial.begin(9600);<br> Serial.print(&quot;Starting...\n&quot;);<br> }<br>void loop() {<br> static int i = 0;<br> int j;<br> uint16_t blocks;<br> char buf[32];<br> blocks = pixy.getBlocks();<br> if (blocks) {<br> i++;<br> if (i%5==0) {<br> //sprintf(buf, &quot;Detected %d:\n&quot;, blocks);<br> // Serial.print(buf); //<br>//for (j=0; j=170;) {<br> <strong>if(pixy.blocks[j].x&gt;190)</strong>{ <br>digitalWrite(red,HIGH);<br> digitalWrite(redled,HIGH); <br>delay(200); <br>digitalWrite(red,LOW);<br> digitalWrite(redled,LOW); <br>} <br><strong>if(pixy.blocks[j].x&lt;170)</strong>{ <br>digitalWrite(white,HIGH);<br> digitalWrite(whiteled,HIGH);<br> delay(200); <br>digitalWrite(white,LOW);<br> digitalWrite(whiteled,LOW); <br>} <br><strong>if(pixy.blocks[j].x&gt;169 &amp;&amp; pixy.blocks[j].x&lt;191)</strong> { <br>digitalWrite(white,HIGH);<br> digitalWrite(whiteled,HIGH);<br> digitalWrite(red,HIGH);<br> digitalWrite(redled,HIGH); <br>delay(200); <br>digitalWrite(white,LOW); <br>digitalWrite(whiteled,LOW); <br>digitalWrite(red,LOW);<br> digitalWrite(redled,LOW); <br>} } } }</p>
Hi i tryed using some parts of this code but it still not working I'm not going to use the pixy servo i tryed using the pixy if statement to make the arduino blink when it see one color but it still not working
<p>It has been a while since I worked on this, but I think you have to be set up to read an array before the if statement will find the correct data.</p><p><strong>static int i = 0;<br> int j;<br> uint16_t blocks;<br> char buf[32];<br> blocks = pixy.getBlocks();<br> if (blocks) {<br> i++;<br> if (i%5==0) {</strong></p>
<p>Hi, I have a question about the Pixy. In the beginning, the pixy used to flash a light whenever it see the signature but today it didn't flash the light so what can the problem be</p>
<p>Possibly it has been reset or set to another color.</p><p>Look here </p><p><a href="http://cmucam.org/projects/cmucam5/wiki/Teach_Pixy_an_object">http://cmucam.org/projects/cmucam5/wiki/Teach_Pixy...</a></p><p>for clear instructions on how to set the colors.</p>
Hi can you tell me if it's possible to make the Arduino generate power to a relay and stop a servo from moving when ever the pixy see an object with a certain color what i mean is it possible to program the Arduino to do some tasks only when the pixy see one color
<p>The Arduino can pull a pin high (or low--depending on the software instruction) whenever an object with a certain color is seen. The Arduino pin can control a transistor and the transistor can turn a relay on or off.</p>
How can i write this in arduino ( which code can i use for the pixy and how can i make it control things when it see an object with a certain color ( I'm a biggner with ardiuno but I'm trying to learn because i have a project)
<p>Hello, sir. Can you tell me how you integrated two arduino working together. How did you wire them and what is the code?</p>
<p>In the Arduino used with the Pixy camera, I set up pin number 5 and pin number 9 as outputs, step number 7.</p><p><strong>Pixy pixy;</strong></p><p><strong>int stuff=0; int position=150; const int red=5; const int white=9; const int redled=3; const int whiteled=8;</strong></p><p><strong>void setup() { </strong></p><p><strong>pinMode(red, OUTPUT); pinMode(white, OUTPUT); pinMode(redled, OUTPUT); pinMode(whiteled,OUTPUT); </strong></p><p>Later in the &quot;loop&quot; code, I write those pins high or low depending on where the Pixy camera senses color.</p><p>In the motor arduino code (step number 8), I set pin number 5 as input and pin number 7 as input. On the motor control Arduino, inputs on pin 5 or pin 7 cause the robot motors to come on or off (steer). </p><p>In other words, the Arduino attached to the Pixy camera does nothing but evaluate image data and supply an output based on what it &quot;sees.&quot; The motor control Arduino performs actions based upon the inputs it receives--it doesn't care how the inputs are created.</p><p>In theory, you could do this all with one Arduino, but it's easier to troubleshoot and understand if the tasks are broken down.</p><p>For example, the red and white led lights on the Pixy Arduino tell me whether the target is right, left or ahead. If the robot is steering incorrectly, then the problem must be in the motor control Arduino system.</p>
<p>Ok, got it. Thanks. </p><p>By the way, do you know how to communicate two arduino mega, like using the master mega to tell the slave mega to turn on the led.....</p>
<p>On the slave mega, identify a pin (pin number 7, for example) as an input. In the first part of the &quot;loop&quot; for the slave, do a &quot;digital read&quot; of pin number 7. When pin number 7 is &quot;high,&quot; then turn on the led; otherwise, keep reading pin number 7.</p><p>On the master mega, identify a pin (pin number 5, for example) as an output. When you want the slave to turn on the led, have the master &quot;digitalWrite&quot; a high on pin number 5.</p><p>Connect &quot;gnd&quot; from the master to &quot;gnd&quot; on the slave. Connect pin number 5 on the master to pin number 7 on the slave.</p>
<p>do we need to use pixymon?</p>
<p>Pixymon is useful for &quot;seeing&quot; what the camera &quot;sees.&quot; It is not required to setup or operate the Pixy--it can be used to help you understand the results of Pixy's image processing.</p>
<p>i have a question, can not we use another camera similar to pixy </p>
<p>The Pixy camera board performs the visual recognition and passes that information to an Arduino (or another processor). Although the Pixy is open source, it would take a lot of work to duplicate what the Pixy accomplishes.</p>
<p>Thanks for the reply. Yes i'm sure of this point. Why i want to use anther camera is i have to finish this project as soon as possible. I oredred pixy and i'm still waiting for it</p>
<p>The maximum distance should be?</p>
<p>Using the target frog in my video, about 10 feet (3 meters). With a larger target (and no objects of the same color to cause confusion), the distance can be increased greatly. </p>
<p>hello</p><p>good news I went back to use your old files the ones on the blog, because the new ones are not compatible with my robot, I loaded and now is working but my robot is avoiding the colors instead of chasing, do you know what can be the problem. thank you for your help, I tried to do this other project that is similar of yours but i could not continue,it look difficult, I like yours better. if you want to take a look. http://www.bajdi.com/pixy-cmucam5/</p>
<p>It sounds like you are getting close--congratulations!</p><p>I'm guessing that left and right are swapped somewhere if the robot is avoiding the colors. If the target color is on the right and the robot goes left, that can be fixed without too much trouble. (You could switch the wires going to the left and right motors, or switch the wires going from the Pixy Arduino to the motor Arduino--but I would suggest making the change in software instead). In the Arduino connected to the pixy, there is a line in the code: &quot;const int red=5; const int white=9;&quot; Try changing that to &quot;const int red=9; const int white=5;&quot;</p><p>I hope this helps.</p>
<p>I don`t know what is happening,I was able to make it go forward by changing the polarity on the motor, when it detect a color in the middle goes forward but it does not stop, and is hard to make it detect a color in the middle, when detect a color on the left or to the right(this is no problem to detect) turns half way and stays there, thank you for following up</p>
<p>It sounds like it is turning so much that the camera loses the color. When a color is on the left or right, the robot should only turn slightly. Perhaps if you reduce the pwmA and pwmB numbers in the software, it will slow down and turn less . . .</p>
<p>hello </p><p>you are right, pixy is making it go crazy, when I turn it on does not move, but when pixy detects a color it spins around then reset power and goes around the other way for a second then spins again non stop. I power pixy and arduino with 7volts, the other arduino and shield with 7 volts. thank you</p>
<p>hello, I almost got it but robot is just pining around, do you now what can be wrong. i loaded one code on one arduino and one on the other as you said in the blog.are you using servos for wheels? I am using dc motor.</p>
<p>I am using dc motors. </p><p>You might try putting a lens cap over the Pixy. If the robot just sits there with the lens cap on, but goes crazy with the lens cap off, then you'll know that confusing signals are coming from Pixy. I put led's on pins 5 and 9 of the pixy so that I could see what commands the Pixy was giving to the motor controller.</p>
<p>spin aound one way and then arduino reset and spin the other war for one second then sping the other way, looks like arduino is reseting it self</p>
<p>Two things I can think of that can cause this type of problem:</p><p>1. If the supply voltage dips below 7 volts, then the Arduino may reset (supply voltage will dip when the motors come on because the motors use a lot of energy).</p><p>2. Motor noise (caused by &quot;sparking&quot; of brushes on dc motor) may reset the Arduino. Try reducing the &quot;analogWrite(pwmA,200)&quot; to something like &quot;analogWrite(pwmA,75)&quot; This will reduce the electrical noise and reduce the drain on the batteries. It will also slow the motors down quite a bit. </p>
<p>hello. </p><p>I am tried to load this program in to arduino but i did copy and paste and everything wento into one line and has a bunch of errors, you said the first part is for one arduino. and if I start with the second part on the other arduino it does not work. maybe is missing something, can you please provide with two different codes for each arduino. thank you.</p>
<p>I'm sorry you are having difficulty with the program. I have added the arduino files (last step of the instructables) along with a video showing improved performance of the Pixy program. </p><p>Unfortunately, when I cut and paste Arduino listings into the Instructables text block, all the formatting seems to disappear and that makes the code hard to read or copy.</p>
<p>hello. </p><p>I am tried to load this program in to arduino but i did copy and paste and everything wento into one line and has a bunch of errors, you said the first part is for one arduino. and if I start with the second part on the other arduino it does not work. maybe is missing something, can you please provide with two different codes for each arduino. thank you.</p>
<p>I didn't know what to do with my pixy but thanks to you it will Come out from the box, is this the final release of your work or there is more, thank you for your ideas, I'm waiting for parts to see if I can build one like yours. </p>
<p>I made a homing station:</p><p><a href="https://www.instructables.com/id/Mikey-the-Robot-with-Vision/" rel="nofollow">https://www.instructables.com/id/Mikey-Go-Home/ <br></a></p><p>and I hope to have more software and hardware enhancements within two or three weeks . . . .</p>
<p>Hi...i am trying to make similar robot using almost the same components. But my query is why did you use two arduinos?? Is it not possible to do in one??</p><p>Thanks </p>
<p>The short answer--one Arduino should be doable.</p><p>I tried using one Arduino, but had problems which I blamed (perhaps incorrectly) on a possible conflict between the motor shield and Pixy. Adding the second Arduino solved my problem, but whether it is because of pin usage or software is something I did not try to resolve. I'm using the motor shield because it allows me to measure motor current. In the independent &quot;run around the room&quot; mode (not yet posted on instructables) I use the &quot;stuck motor draws more current feature&quot; to detect when the robot needs to change directions.</p>
<p>Yes i am facing the same problem. I am using a single arduino and motor shield behaves strangely with pixy.....I have been trying to solve it but in vain.</p>
<p>I thought Pixy was using SCL and SDA on the Arduino (according to <br>documentation), but I somehow burned out pin 3 of the Arduino Uno (which <br> is used for the motor shield). I was trying so many things that I <br>can't pinpoint what I may have done--I'm just suspicious that there is <br>pin conflict between the motor shield and the Pixy. Separately, they <br>each worked fine. Then together, they didn't (and wouldn't work <br>separately after I had had them together).</p>

About This Instructable




Bio: I am an author and a maker. My current project is Santa's Shop. I'm working on a science fiction type book--more later. @EngineerRigsby
More by MikeTheMaker:Automated Climbing Bear Supercapacitor Flashlight Another Slinky Machine 
Add instructable to: