Using a PixyCam (CMUcam5) to Get Distance of a Known Object

This is one of those projects that turns out to be a good thought exercise.

You have a Pixycam, an Arduino uno, and a lcd screen. (Lucky you - you also have a computer, and usb cords to hook the electronics up and program with!). You need to know the distance of "something" from where you are sitting.

Looking around you - you find, a ruler and some colored construction paper - a little glue, and thin card board.

Well you could use the ruler to find the distance - that is true - but where is the fun in that! :-)

Being an electronics kind of person, you think - How can I use the Pixycam to give me a distance!

That's what this instructable is all about. :-)

Ok, so really, "We" (meaning me and members of the Automation Technology Club I attend) have been thinking about doing some distance detection using some pretty simple parts, and trying to keep the cost low. With as little hardware, and parts as we can. Mainly we are wanting some longer ranges for some of our robotic projects.

In terms of hardware, this is pretty simple, in terms of cost - it's not the cheapest option. In terms of setting up the Pixy - well it can get tricky. In terms of is this good for a robotic project (?) Well from my few attempts, I'd say no. But it maybe good for other things. And again, it's good to start thinking. Use this as a thought exercise :-)

Curious about how it works - but don't care about my build? Jump ahead to Step 4.

Let's get started -

Step 1: The Electronics and Miscellaneous Parts

From a hardware stand point, this fairly simple build. What makes it even better the Pixycam was made with the UNO in mind. (Of course you can use it with other micros)

Electronics Hardware:

Arduino Uno (1)

PixyCam (1) http://www.cmucam.org/

Parallax 1602 Serial LCD - https://www.parallax.com/product/27977 (The code has some control codes in it for this display, any serial LCD that just uses standard ascii should work without too many changes to the code). I was going to use a cheap I2C display, but turns out with the pixy library, and the libraries needed for that type display - I ran out of memory - so I went simple. I had the Parallax Serial LCD from a project years ago, I haven't used it much since then. But this seemed like the right project for it! And since it doesn't need a library it doesn't use up valuable space.

Other things you may need (or want):

sisors, Glue, thin cardboard, different color construction paper, something to mount the camera and arduino too.

Some wires for the display, maybe some screws/nuts, screw driver. That is about it.

I used 2.5 inch PVC pipe connectors for mounting - this was a bit of over kill, but worked out great. PVC is pretty easy to cut holes in as well with just some simple tools.

(In fact, I used a screw driver, and a self taping screw to make holes in it - didn't have access to my drill, so needed to go real simple here.)

Software -

Arduino IDE 1.6.9 is what I used. It should work with other versions of the IDE thou.

PixyMon - http://www.cmucam.org/projects/cmucam5/wiki/Pixy_R...

Arduino Pixy Library - http://www.cmucam.org/projects/cmucam5/wiki/Latest...

Setting up the PixyCam was one of the hardest parts, I am going to go over some of the basic stuff, but you really need a good understanding of how this camera works, and the different modes. And how to tweak it for your colors. (I've got a very basic understanding of myself, and I have read the wiki a few times.)

http://www.cmucam.org/projects/cmucam5/wiki

Step 2: Mounting and Wiring.

Based on my experience with, we are going to mount everything first - in some later steps you'll see me holding the camera - That's because mounting was one of the last things I did. But it should have been one of the first.

Pictures are of course of my mounting, yours will most likely be different. Because of how the camera is made I did mount it high. I tried to keep it level thou, and not looking down or up.

Another thing, I only used one screw to mount it - I didn't take into account that what I was mounting to was round, and the Pixy is flat. which causes one of the mounting holes to be slightly off. This wasn't a big deal for me, because I used the rim of the pipe connector as a steady surface.

I also used screws that were too long for the project - again, this is ok - because as you can see I used the extra length of the screw to mount the display.

Hooking the Pixycam to the Arduino is pretty simple, the cable that comes with the camera attaches to the ICSP pins. The connector has a end with no cable, and a end with cable on it. The end with out cable faces the USB port of the Arduino.

The LCD uses 5v, GND, and the RX pin of the LCD is connected to the TX pin of the Arduino.

That's it.

Step 3: Let's Make Our Color Codes

You'll want at least 2 different colors for to make a color code that the Pixycam will read.

We are going to take our construction paper, and cut it down. Folding the paper in half helps keep things roughly the same size.

Your color paper should be roughly the same size for each color - This is a rough guess of distance - so it doesn't have to be perfect.

I just made a couple of rectangle of different colors, I cut extra just because it's fun to act like a kid again, and cut paper LOL.

My thin cardboard came from the insert of a Pepsi 24 pack box, this worked pretty good. It's thin enough to bend to make little legs with, but thick enough to stand on it's own.

I glued my two rectangles to next to each other, in about the center of the card board.

Later I cut the box down even more and glued it to another pipe connector. This may or may not have been a good idea. I think the larger box keep some of the false readings from getting to the camera, but It does look better.

You will want to use your ruler or tape measure and measure, the height and width of your color code.

Your color code is both colors together. I rounded my numbers a little bit, but the closer you get to actual size the more better your distance should be.

It's never going to be dead on perfect distance, but it can be pretty good. Next step I'll explain how this works.

Step 4: How It Works.

Credit where credit is due - My project is based from work done here:

http://www.pyimagesearch.com/2015/01/19/find-dista...

In that project, the creator is using Python and OpenCV.

What I am using here is something called, triangle similarity. (Probably more than you ever wanted to know - http://www.mathopenref.com/similartriangles.html )

The idea is a triangle are similar if they have the same shape but not necessarily the same size.

Think of it like zooming in and out.

Pyimagesearch says: Let's say we have a object with a known width (W). We place the object a known distance (D), and take a picture of it and measure the apparent width in pixels (P). We come up with this formula for perceived focal length (F) of our camera.

F = (P x D) / W

We can now use this formula and will have two known values. (F and W). We can now figure out a unknown distance.

D = (W x F) / P The camera will read a different apparent width in pixels if the camera or the object moves. And a rough distance can be found.

I took this idea one more step and applied it to the height as well. The formula is pretty much the same, substituting width with the height of the object, and substituting the apparent height in pixels for P.

I then take the two unknown distances add them together, and divide them by two.

There are things that can negatively effect these. Background noise, not being able to see the whole object.

Either too much light or not enough light seems to play a big role in how well this works as well.

Setting up the color code.

Step 5: Set Up a Color Code.

This step takes a lot of patients, and a lot of tweaking, and tinkering.

To be completely up front here - I am still learning the ins and outs of setting up the Pixycam myself.

I have a basic understanding to set it up. I did a lot of trail and error before I found what works for me.

For this step we will be using the PixyMon software.

You'll probably want to unhook the Pixycam from the Arduino. Connect the Pixycam and start the PixyMon software.

I think for this part, I'm going to refer to CMUcam website.

Teach Pixy an Object http://www.cmucam.org/projects/cmucam5/wiki/Teach...

Improving Detection Accuracy: http://www.cmucam.org/projects/cmucam5/wiki/Some_T...

And finally, Using Color Codes http://www.cmucam.org/projects/cmucam5/wiki/Using_...

The website is full of helpful information, and it is something that everyone who has a Pixycam should be reading - seriously.

This does take some time, so take your time at it.

Next the Arduino code.....

Step 6: The Arduino Code

I've got a github repository setup for the code.

https://github.com/kd8bxp/find_distance_with_pixyc...

As I said earlier you will need the Pixy cam library.

http://www.cmucam.org/projects/cmucam5/wiki/Hookin...

There are a couple of things to note in the sketch.

#define Calibrate 0 //Change to 1 see above for details.
#define calDistance 24 //in inches 24inches or 2 foot
#define LCD 1 //Display Average Distance to Parallax 1602 serial LCD otherwise Zero will display much more information to serial console and/or LCD if connected

Put your camera about 2 foot from your color code object. (using the ruler to get it as close as you can)

Setting calibrate to 1 will display information on the serial monitor - you'll see a height and a width when the camera sees your object. (these are in pixels I believe)

They may or may not change, so you'll want to give yourself time, and make sure most of the numbers the serial monitor is putting out are the same or very close. (a little trail and error here isn't bad)

You'll want to make a note of your apparent height and width. And at what distance you are (24 inches).

I've already set the calDistance to 24 - if you are using something different this is where you change it.

int calWidth = 40; //Calibrated width reading
int calHeight = 29; //Calibrated height reading
int pixelsWidth;   //read by the camera
int pixelsHeight; //read by the camera
float distanceWidth;   //calculated distance based on the width of the object
float distanceHeight;  //calculated distance based on the height of the object 
float widthOfObject = 3.75; //inches (3.75 inches) real size of your object
float heightOfObject = 2.5; //inches (2.5 inches) real size of your object
int focalLengthWidth;  //calculated focal length for width
int focalLengthHeight; //calculated focal length for height
float avg;
int feet;
int inches;

In this block, you'll see calWidth, and calHeight - this is where you put the recorded numbers at.

you'll also see widthOfObject and heightOfObject - this is the real world inches of the object.

The rest of the variables are used for the maths.

- Back to the first block of code, you'll see #define LCD 1

This tells the sketch you are using a LCD or if you want everything to go to the serial console.

Because the LCD is only a 16x02 display, I wanted it to look nice when you read it. A one here means I am using a LCD, and zero means I am NOT using a LCD.

Not using a LCD more information will be printed to the serial console, so it might be useful.

Once you make a change to any of the variables, you'll need to upload the sketch to the Arduino board again.

That's about it, lastly - the video and a few more photos....