Having a webcam is all well and good, but what can we do with it? Well as it turns out, rather a lot. Computer vision (CV) is a big part of advanced robotics and CV algorithms have become the subject of much interest in recent years. Allowing your robot to “see” its surroundings is one of the most powerful things you can do to allow it to perform complex tasks. One of the most prominent tools in CV development is OpenCV. OpenCV is a set of libraries developed by Intel to aid in image processing tasks. Luckily for us, OpenCV works well on the Phidgets SBC and can be easily installed.

Note that many image processing applications are extremely computationally intense and may be too much for the SBC to handle. This makes it important to test your application as early in the development cycle as you can so you can get a good feeling of how the SBC will cope with the amount of work you are requesting of it. This instructable is going to walk through setting up OpenCV, and then writing a small program that detects red objects in an image.

Step 1: Setting Up OpenCV

In order to use OpenCV on your Phidgets SBC you will first need to install the libraries and development files. To do this we need to configure the SBC a little bit, as seen in the first two images above.

Then connect to the SBC via an SSH terminal. Since our desktop computer is using Windows, we’re using PuTTY. Mac and Linux users can use the openSSH terminal application. If you are uncertain on how to connect to an SBC via SSH, we encourage you to check out our getting started on SBC video, which devotes some time to the topic.
Once we’ve connected to the SBC, we’ll install all of the necessary openCV libraries with:

apt-get update

apt-get upgrade

apt-get install libopencv-core2.3 libopencv-core-dev libopencv-dev libcv2.3 libhighgui2.3 libcvaux2.3 libcv-dev libhighgui-dev libcvaux-dev

If apt-get cannot find these packages that may mean there are newer versions available. By running the following commands, you’ll be able to see if the version in the repository is something other than 2.3:

apt-cache search opencv

In order to make sure OpenCV installed properly, check to see that the files are in the right location.

ls /usr/include/opencv

In particular you are looking for cv.h and highgui.h. It’s possible that a different version of OpenCV may put highgui.h in a separate folder and give it a slightly different name (highgui_somecharacters.h). If you do not see both cv.h and highgui.h in /usr/include/opencv then you may have to search around a bit to find them.

Step 2: Using OpenCV in Your Program

Once you have installed OpenCV you can begin writing your code. Let’s start with an example we already have: detecting red pixels with opencvtest.c

opencvtest.c
Download

Step 3: Compiling

To compile you will need to link the correct libraries. To do this use the pkg-config tool. You may need to install it with apt-get before you can use it.

g++ opencvtest.c `pkg-config --cflags --libs opencv`

If you are not copying and pasting these commands, but rather typing them out manually, then you need to ensure that you use backticks (`) rather than the more common front tick (‘). It may also be necessary to link to the highgui.h separately based on what path it got installed to (likely opencv2 or highgui) if you had to search for it previously. Note that since OpenCV is designed for C++ you will need to use g++ rather than gcc in order to compile. It should have been installed when we enabled the C/C++ development tools.

Step 4: Results

To run the program, we will execute the binary file we just compiled. We do this with:

./a.out 100 75 75

What that means is it will accept any pixel with an RGB value of >99, <76, <76 for the processed image. The program will output two JPG files; original.jpg which is the original image from the webcam, and redDetect.jpg which is the processed version. As you can see above, it did a pretty good job of singling out the red pixels. These numbers could be fine-tuned a bit so that it picks up less of the robot’s outline, but it obviously works in principle.

Getting these two images can be easily done with the file editor functionality of the SBC web configuration tool as seen in the third image.

This is only a simple example of some of the things that be done with computer vision. In a bigger application processing of the images coming from the webcam can be done for a more practical purpose, and in a more real time fashion. For example, one could have a robot that detects red objects and tries to move towards them. The applications of computer vision are limitless, so go ahead and experiment!