Mikey, the Robot With Vision

About: I am an author and a maker. My current project is Santa's Shop. I'm working on a science fiction type book--more later. @EngineerRigsby

Mikey the Robot "sees the frog" using the Pixy camera vision system. Pixy costs about $70. and easily interfaces with an included cable to Arduino. Pixy can be "trained" to recognize up to seven different colors. Pixy returns the x,y coordinates of the center of a trained object, along with the width and height of the object. Pixy is open source and all the hardware and software are available through cmucam.org.

In my setup, Pixy communicates with an Arduino which I call a "Sensor Interface." When the center of the target (frog) is toward the right (>170 out of 319), the red LED illuminates and the robot turns toward the right. When the frog is toward the left (<150 out of 319), the white LED illuminates and the robot turns to the left. When the frog is pretty much in the center (>150 and <170), both LEDs illuminate and the robot moves forward.


Parts:

Pixy Camera---Amazon.com

4WD Arduino Compatible Mobile Platform--Makershed.com

Arduino Motor Shield--Makershed.com

(2) Arduino Uno--Amazon.com

(7) NiMh AA batteries--Amazon.com

Misc: Red led, white led, (2) 220 ohm resistors, SPST toggle switch

Step 1:

Start by soldering wires to the motors on the mobile platform. I used an AA battery to make sure that positive on the red wire and negative on the black would cause each motor to move the platform forward. This needs to be consistent on all motors (which means the red wire will probably be on top of the right side motors and on the bottom of the left side motors).

Step 2:

Secure the lower plate with screws, making sure that the motor wires come through open holes.

Step 3:

Add battery holders. My choice of (7) AA batteries relates to a future recharging home base (which Mikey will locate using his vision--soon).

Step 4:

Add the upper plate, switch, two Arduinos (one with a motor shield on top) and the Pixy Camera.

Files for the 3D printed switch holder can be found here:

http://www.thingiverse.com/thing:331319

Pixy is connected to the Arduino with a simple cable (included with Pixy). More information on connecting Pixy can be found here:

http://cmucam.org/projects/cmucam5/wiki/Hooking_up...

Step 5: Training Pixy

Training Pixy to a color is mainly a question of holding a button (red for the first color, orange second, yellow third, green fourth, cyan fifth, blue sixth, violet seventh) then releasing. Place the object in front of the camera. When the LED is pretty much the color of the object, quickly press and release the button. The LED will blink red a few times if Pixy "bought" the object. That's it.

Step 6: Schematic

This is the schematic.

Step 7: Sensor Software

The sensor software for Arduino is a modified version of their "hello world" sketch.

Download the Arduino library "arduino_pixy-x.y.z.zip" here.

http://cmucam.org/attachments/download/958/arduino...

Bring up the Arduino IDE and import the Pixy library by selecting Sketch➜Import Library in the Arduino IDE, and then browsing to the Pixy.zip file that you just downloaded.

//
// begin license header // // This file is part of Pixy CMUcam5 or "Pixy" for short // // All Pixy source code is provided under the terms of the // GNU General Public License v2 (http://www.gnu.org/licenses/gpl-2.0.html). // Those wishing to use Pixy source code, software and/or // technologies under different licensing terms should contact us at // cmucam@cs.cmu.edu. Such licensing terms are available for // all portions of the Pixy codebase presented here. // // end license header //

#include #include

Pixy pixy;

int stuff=0; int position=150; const int red=5; const int white=9; const int redled=3; const int whiteled=8;

void setup() {

pinMode(red, OUTPUT); pinMode(white, OUTPUT); pinMode(redled, OUTPUT); pinMode(whiteled,OUTPUT); digitalWrite(redled, HIGH);//right test digitalWrite(whiteled,HIGH);//left test delay(2000); digitalWrite(redled,LOW);//right off digitalWrite(whiteled,LOW);//left off digitalWrite(red,LOW); digitalWrite(white,LOW); delay(2000);

Serial.begin(9600); Serial.print("Starting...\n"); }

void loop() { static int i = 0; int j; uint16_t blocks; char buf[32]; blocks = pixy.getBlocks(); if (blocks) { i++; if (i%50==0) { //sprintf(buf, "Detected %d:\n", blocks); // Serial.print(buf); //for (j=0; j170) { digitalWrite(red,HIGH); digitalWrite(redled,HIGH); delay(1000); digitalWrite(red,LOW); digitalWrite(redled,LOW); } if(pixy.blocks[j].x<150){ digitalWrite(white,HIGH); digitalWrite(whiteled,HIGH); delay(1000); digitalWrite(white,LOW); digitalWrite(whiteled,LOW); } if(pixy.blocks[j].x>149 && pixy.blocks[j].x<171) { digitalWrite(white,HIGH); digitalWrite(whiteled,HIGH); digitalWrite(red,HIGH); digitalWrite(redled,HIGH); delay(1000); digitalWrite(white,LOW); digitalWrite(whiteled,LOW); digitalWrite(red,LOW); digitalWrite(redled,LOW); } } } }}

Step 8: Motor Software

This sketch is loaded into the Arduino that has a motor shield on top.

Note that the "jump=1" moves to the second part of the program, where the robot chases the frog. The first part of the program is used to enable the robot to move around without getting stuck (a future enhancement--coming soon).

int val=0;
int val1=0; int valm=0; int valm1=0; int jump=1;

const int pwmA=3; const int pwmB=11; const int brakeA=9; const int brakeB=8; const int dirA=12; const int dirB=13; const int right=5; const int left=7;

void setup() { pinMode(dirA, OUTPUT); pinMode(brakeA, OUTPUT); pinMode(dirB, OUTPUT); pinMode(brakeB, OUTPUT); pinMode(right,INPUT); pinMode(left,INPUT); digitalWrite(dirA, HIGH);//forward A digitalWrite(brakeA,LOW);//release brakeA analogWrite(pwmA,100);//set speed A digitalWrite(dirB,HIGH);//forward B motor digitalWrite(brakeB,LOW);// analogWrite(pwmB,100);//set speed B delay(700);

}

void loop () { if (jump==0) { valm=analogRead(0); valm1=analogRead(1); if(valm>500 or valm1>500) { valm=analogRead(0); valm1=analogRead(1); if(valm>450 or valm1>450) { digitalWrite(brakeA, HIGH); digitalWrite(brakeB, HIGH); digitalWrite(dirA, LOW);//reverse A digitalWrite(brakeA, LOW);// analogWrite(pwmA, 200); digitalWrite(dirB,LOW); digitalWrite(brakeB,LOW); analogWrite(pwmB,200); delay(700);//backup digitalWrite(brakeA,HIGH);//stop one wheel analogWrite(pwmA,0); delay(1000); digitalWrite(brakeB,HIGH);//stop other wheel //start both wheels forward digitalWrite(dirA,HIGH); digitalWrite(brakeA,LOW); analogWrite(pwmA,140); digitalWrite(dirB,HIGH); digitalWrite(brakeB,LOW); analogWrite(pwmB,140); delay(1000); }}} //jump equal one analogWrite(pwmA,0); analogWrite(pwmB,0); val=digitalRead(right); val1=digitalRead(left); if (val==HIGH && val1==HIGH) { analogWrite(pwmA,140); analogWrite(pwmB,140); delay(1000); } if (val==HIGH &&val1==LOW) { analogWrite(pwmA, 140); analogWrite(pwmB,50); delay(1000); } if (val1==HIGH && val==LOW) { analogWrite(pwmB,140); analogWrite(pwmA,50); delay(1000); }

}

Step 9:

I hope to enable Mikey to come out on command, run around and do things, then return (on his own) to recharge.

Having Pixy vision should simplify all these tasks.


I've changed the Pixy part of the software slightly so that time is not wasted reading empty blocks. Also, I've included the Arduino files for both Pixy and the motor shield.

Share

    Recommendations

    • Plastics Contest

      Plastics Contest
    • Optics Contest

      Optics Contest
    • Make it Glow Contest 2018

      Make it Glow Contest 2018

    51 Discussions

    0
    None
    RyanKetterhagen

    7 months ago

    The pixy camera light is detecting the colored signature below the camera lens, but the colored LEDs aren't lighting up. I have both LEDS wired to 220 ohm resistors that meet together at a single ground.

    5 replies
    0
    None
    MikeTheMakerRyanKetterhagen

    Reply 7 months ago

    It sounds like the Pixy camera is working, but the Arduino is not interpreting the data. Did you use the "hello_lightdetect0518.ino" file at the end of step #9 for the Arduino connected to the camera--that's the best way to avoid any typing errors.

    0
    None
    RyanKetterhagenMikeTheMaker

    Reply 7 months ago

    When I run the hello_lightdetect0518.ino file, I do not get any serial reading. If I run the native hello_world from the pixy library I get active serial readings. Both programs do not light up the LEDs

    0
    None
    MikeTheMakerRyanKetterhagen

    Reply 7 months ago

    Let's try something else--maybe the signature needs to be set in your Pixy. After the blocks = pixy.getBlocks(); statement, make sure it looks like the code below (also make sure that Pixy is trained to the first color signature for the color you want to track. The "signature" is set for 1 to 7 to determine which trained color is being searched for).

    if (blocks)

    {

    i++;

    if (i%5==0){

    if(pixy.blocks[0].signature==1) {

    if(pixy.blocks[0].x>170)

    0
    None
    RyanKetterhagenMikeTheMaker

    Reply 7 months ago

    I've been trying to isolate the issue, but it's all coming back to the pixy camera not outsourcing block data from the pixy camera. The serial print says "starting..." but doesn't move past the initial starting line in the serial monitor.

    pixy_question.png
    0
    None
    RyanKetterhagenRyanKetterhagen

    Reply 7 months ago

    When the ball moves left or right, or even forward I am no getting any of the LED's to light. In Pixymon I can see that ball very clearly, but the outsource from the Pixy to the Arduino show below isn't communicating properly.

    image1-2.jpeg
    0
    None
    RyanKetterhagen

    Question 7 months ago

    Hey there Mike, Ive used both codes in step 7 and step 8 to control my robot. In your motor control it seems that there is a problem. I uploaded step 7 code into the Arduino closest to the pixy camera and uploaded step 8 to the arduino below the motor shield. My pixy camera is "taught" a specific signature as well. Any ideas as to how I can get your design to work? Thankyou for your time.

    IMG_1676.jpg
    1 more answer
    1
    None
    MikeTheMakerRyanKetterhagen

    Answer 7 months ago

    Let's ignore the motors for a moment and make sure that the information coming from the Pixy arduino is correct. Do both leds illuminate when the target is straight ahead? Does only one led come on when the target is to the right, then the other to the left?

    Once we know that is correct, we can diagnose the situation further.

    0
    None
    RyanKetterhagen

    Question 7 months ago

    Can you please show more wiring pictures or explain unshown diagram wiring. I’m attempting to recreate Mikey and I’ve noticed the yellow wire next to the motor shield. Also is this your full code? I’m using all the same components and I’m still not getting proper motor response. Any information would be greatly appreciated!

    1 more answer
    0
    None
    MikeTheMakerRyanKetterhagen

    Answer 7 months ago

    If you follow the schematic diagram in step #6, everything should be just fine. The yellow wire next to the motor shield connects Vin on the screw terminals to Vin on the power header--this is only necessary if the "Vin connect" trace on the back side of the motor shield has been cut (mine was cut for use in another project).

    When testing, watch the red and white leds on pins 3 and 8 of the "sensor Arduino." If they are both on, the left and right motors should be active. If one led is on, the motors on one side should be active. If no leds are illuminated, the motors will not turn.

    Most problems relate to the image that the camera "sees." If a window or lamp is in the camera's image area, the camera will probably be "blinded' by the light and fail to see the target. If the target color (green in my example) can be found on objects other than the target, the Pixy camera will "see" and "identify" those and the system will act confused due to multiple target identification.

    0
    None
    ChiamakaU1

    1 year ago

    hello sir, i have some questions regarding your code. some terminologies you used, i am unable to comprehend the meaning. the terms include, val, val1, valm, valm1, valm3 and the use of ctr. Thanks

    0
    None
    SyahirM

    2 years ago

    hi sir..can u email me the actual coding for this robot? im getting confuse because there are two separate coding which are the code that u attach that needs to be download and the one u copy and paste to your website. Both of the codes i have try before but does not working. I have connect all the wire connection exactly the same as the schematic diagram that you provided. Im really interesting with this robot. I hope u can help me :) this my email

    syahir_tombstone28@yahoo.com

    0
    None
    BiswasL

    2 years ago

    Awesome !!!!!!

    I have also tried to track a colour with Pixy after reading your post but only in x-direction because i want to use it in my Camera Drolly so that i can shoot a video by tracking a colour of any object. for that i have used PWM controller but because of this speed not varies as the object change its speed. Now i am planning to control with PID so that it moves in desired speed. Everything is ready but only need help to use PID. My email: lohanibiswas@yahoo.com please respond to this mail. So that i can send you all the file.

    Hope for your positive response.

    Thanks.

    0
    None
    Khualm

    2 years ago

    Can I use one of pixy library codes and then add the tasks i want the arduino to do when pixy see a certain color (if yes which code should i use)

    3 replies
    0
    None
    MikeTheMakerKhualm

    Reply 2 years ago

    Take this code and examine the bold parts. The "if" statements indicate "see color to the right--see color to the left--see color straight ahead." After the "if" question is a { That is the place you put your code to do what you want. In my case, I make pins "high," wait .2 seconds then make them low.

    Copy this code exactly and set up led's on pins 3 and 8.

    Train the Pixy to a color and the leds should come on to indicate left, right or straight ahead.

    When this works, change the code after the "if" to do whatever you want.

    #include <SPI.h>
    #include<Pixy.h>
    Pixy pixy;
    int stuff=0;
    int position=150;
    const int red=5;
    const int white=9;
    const int redled=3;
    const int whiteled=8;
    void setup() {
    pinMode(red, OUTPUT);
    pinMode(white, OUTPUT);
    pinMode(redled, OUTPUT);
    pinMode(whiteled,OUTPUT);
    digitalWrite(red,LOW);
    digitalWrite(white,LOW);
    digitalWrite(redled, HIGH);//right test
    digitalWrite(whiteled,HIGH);//left test
    delay(2000);
    digitalWrite(redled,LOW);//right off
    digitalWrite(whiteled,LOW);//left off

    delay(2000);
    Serial.begin(9600);
    Serial.print("Starting...\n");
    }
    void loop() {
    static int i = 0;
    int j;
    uint16_t blocks;
    char buf[32];
    blocks = pixy.getBlocks();
    if (blocks) {
    i++;
    if (i%5==0) {
    //sprintf(buf, "Detected %d:\n", blocks);
    // Serial.print(buf); //
    //for (j=0; j=170;) {
    if(pixy.blocks[j].x>190){
    digitalWrite(red,HIGH);
    digitalWrite(redled,HIGH);
    delay(200);
    digitalWrite(red,LOW);
    digitalWrite(redled,LOW);
    }
    if(pixy.blocks[j].x<170){
    digitalWrite(white,HIGH);
    digitalWrite(whiteled,HIGH);
    delay(200);
    digitalWrite(white,LOW);
    digitalWrite(whiteled,LOW);
    }
    if(pixy.blocks[j].x>169 && pixy.blocks[j].x<191) {
    digitalWrite(white,HIGH);
    digitalWrite(whiteled,HIGH);
    digitalWrite(red,HIGH);
    digitalWrite(redled,HIGH);
    delay(200);
    digitalWrite(white,LOW);
    digitalWrite(whiteled,LOW);
    digitalWrite(red,LOW);
    digitalWrite(redled,LOW);
    } } } }

    0
    None
    KhualmMikeTheMaker

    Reply 2 years ago

    Hi i tryed using some parts of this code but it still not working I'm not going to use the pixy servo i tryed using the pixy if statement to make the arduino blink when it see one color but it still not working

    0
    None
    MikeTheMakerKhualm

    Reply 2 years ago

    It has been a while since I worked on this, but I think you have to be set up to read an array before the if statement will find the correct data.

    static int i = 0;
    int j;
    uint16_t blocks;
    char buf[32];
    blocks = pixy.getBlocks();
    if (blocks) {
    i++;
    if (i%5==0) {

    0
    None
    Khualm

    2 years ago

    Hi, I have a question about the Pixy. In the beginning, the pixy used to flash a light whenever it see the signature but today it didn't flash the light so what can the problem be

    1 reply
    0
    None
    Khualm

    2 years ago

    Hi can you tell me if it's possible to make the Arduino generate power to a relay and stop a servo from moving when ever the pixy see an object with a certain color what i mean is it possible to program the Arduino to do some tasks only when the pixy see one color