HackerBox 0024: Vision Quest

Introduction: HackerBox 0024: Vision Quest

Vision Quest - This month, HackerBox Hackers are experimenting with Computer Vision and Servo Motion Tracking. This Instructable contains information for working with HackerBox #0024, which you can pick up here while supplies last. Also, if you would like to receive a HackerBox like this right in your mailbox each month, please subscribe at HackerBoxes.com and join the revolution!

Topics and Learning Objectives for HackerBox 0024:

  • Experimenting with Computer Vision
  • Setting up OpenCV (Computer Vision)
  • Programming the Arduino Nano from the Arduino IDE
  • Controlling Servo Motors with the Arduino Nano
  • Assembling a Mechanical Pan and Tilt Assembly
  • Controlling Pan and Tilt Motion with a Microcontroller
  • Performing Face Tracking using OpenCV

HackerBoxes is the monthly subscription box service for DIY electronics and computer technology. We are hobbyists, makers, and experimenters. We are the dreamers of dreams. HACK THE PLANET!

Step 1: HackerBox 0024: Box Contents

  • HackerBoxes #0024 Collectable Reference Card
  • Three Bracket Pan and Tilt Assembly
  • Two MG996R Servos with Accessories
  • Two Aluminum Circular Servo Couplers
  • Arduino Nano V3 - 5V, 16MHz, MicroUSB
  • Digital Camera Assembly with USB Cable
  • Three Lenses with Universal Clip Mount
  • Medical Inspection Pen Light
  • Dupont Male/Female Jumpers
  • MicroUSB Cable
  • Exclusive OpenCV Decal
  • Exclusive Dia de Muertos Decal

Some other things that will be helpful:

  • Small wooden board scrap for camera base
  • Soldering iron, solder, and basic soldering tools
  • Computer for running software tools

Most importantly, you will need a sense of adventure, DIY spirit, and hacker curiosity. Hardcore DIY electronics is not a trivial pursuit, and we are not watering it down for you. The goal is progress, not perfection. When you persist and enjoy the adventure, a great deal of satisfaction can be derived from learning new technology and hopefully getting some projects working. We suggest taking each step slowly, minding the details, and never hesitating to ask for help.

FREQUENTLY ASKED QUESTIONS: We like to ask all HackerBox members a really big favor. Please take a few minutes to review the FAQ on the HackerBoxes website prior to contacting support. While we obviously want to help all members as much as necessary, most of our support emails involve simple questions that are very clearly addressed in the FAQ. Thank you for understanding!

Step 2: Computer Vision

Computer vision is an interdisciplinary field that deals with how computers obtain high-level understanding from digital images or videos. From the perspective of engineering, computer vision seeks to automate tasks that the human visual system can do. As a scientific discipline, computer vision is concerned with the theory behind artificial systems that extract information from images. The image data can take many forms, such as video sequences, views from multiple cameras, or multi-dimensional data from a medical scanner. As a technological discipline, computer vision seeks to apply its theories and models for the construction of computer vision systems. Sub-domains of computer vision include scene reconstruction, event detection, video tracking, object recognition, 3D pose estimation, learning, indexing, motion estimation, and image restoration.

It is interesting to note that computer vision may be considered the inverse of computer graphics.

Step 3: Processing and OpenCV

Processing is a flexible software sketchbook and a language for learning how to code within the context of the visual arts. Processing has promoted software literacy within the visual arts and visual literacy within technology. There are tens of thousands of students, artists, designers, researchers, and hobbyists who use Processing for learning and prototyping.

OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library. OpenCV was built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products. The OpenCV library has more than 2500 optimized algorithms, which include a comprehensive set of both classic and state-of-the-art computer vision and machine learning algorithms. These algorithms can be used to detect and recognize faces, identify objects, classify human actions in videos, track camera movements, track moving objects, and so forth.

Install OpenCV within Processing from the File > Examples menu by selecting “Add Examples” and then under the Libraries tab installing both the Video and the OpenCV libraries. Open the LiveCamTest example for basic face tracking. Check out some other OpenCV for Processing examples here.

More Resources:

Getting Started with Computer Vision is a book project providing an easy entry point for creative experimentation with computer vision. It introduces the code and concepts necessary to build computer vision projects.

Programming Computer Vision with Python is an O'Reilly book on PCV, an open source Python module for computer vision.

Learning OpenCV

Computer Vision: Algorithms and Applications

Mastering OpenCV

Stanford Course CS231n Convolutional Neural Networks for Visual Recognition (16 Videos)

Chris Urmson TED Talk How a driverless car sees the road

Step 4: Arduino Nano Microcontroller Platform

We can use any common microntroller platform for controlling servos in our pan and tilt camera mount. The Arduino Nano is a surface-mount, breadboard-friendly, miniaturized Arduino board with integrated USB. It is amazingly full-featured and easy to hack.

Features:

  • Microcontroller: Atmel ATmega328P
  • Voltage: 5V
  • Digital I/O Pins: 14 (6 PWM)
  • Analog Input Pins: 8
  • DC Current per I/O Pin: 40 mA
  • Flash Memory: 32 KB (2KB for bootloader)
  • SRAM: 2 KB
  • EEPROM: 1 KB
  • Clock Speed: 16 MHz
  • Dimensions: 17mm x 43mm

This particular variant of the Arduino Nano is the black Robotdyn design. The interface is by an on-board MicroUSB port that is compatible with the same MicroUSB cables used with many mobile phones and tablets.

Arduino Nanos feature a built-in USB/Serial bridge chip. On this particular variant, the bridge chip is the CH340G. Note that there are various other types of USB/Serial bridge chips used on the various types of Arduino boards. These chips allow you computer's USB port to communicate with the serial interface on the Arduino's processor chip.

A computer's operating system requires a Device Driver to communicate with the USB/Serial chip. The driver allows the IDE to communicate with the Arduino board. The specific device driver that is needed depends upon both the OS version and also the type of USB/Serial chip. For the CH340 USB/Serial chips, there are drivers available for many operating systems (UNIX, Mac OS X, or Windows). The maker of the CH340 supplies those drivers here.

When you first plug the Arduino Nano into a USB port of your computer, the green power light should come on and shortly after the blue LED should start to blink slowly. This happens because the Nano is pre-loaded with the BLINK program, which is running on the brand new Arduino Nano.

Step 5: Arduino Integrated Development Environment (IDE)

If you do not yet have the Arduino IDE installed, you can download it from Arduino.cc

If you would like additional introductory information for working in the Arduino ecosystem, we suggest checking out the instructions for the HackerBoxes Starter Workshop.

Plug the Nano into the MicroUSB cable and the other end of the cable into a USB port on the computer, launch the Arduino IDE software, select the appropriate USB port in the IDE under tools>port (likely a name with "wchusb" in it). Also select "Arduino Nano" in the IDE under tools>board.

Finally, load up a piece of example code:

File->Examples->Basics->Blink

This is actually the code that was preloaded onto the Nano and should be running right now to slowly blink the blue LED. Accordingly, if we load this example code, nothing will change. Instead, let's modify the code a little bit.

Looking closely, you can see that the program turns the LED on, waits 1000 milliseconds (one second), turns the LED off, waits another second, and then does it all again - forever.

Modify the code by changing both of the "delay(1000)" statements to "delay(100)". This modification will cause the LED to blink ten times faster, right?

Let's load the modified code into the Nano by clicking the UPLOAD button (the arrow icon) just above your modified code. Watch below the code for the status info: "compiling" and then "uploading". Eventually, the IDE should indicate "Uploading Complete" and your LED should be blinking faster.

If so, congratulations! You have just hacked your first piece of embedded code.

Once your fast-blink version is loaded and running, why not see if you can you change the code again to cause the LED to blink fast twice and then wait a couple of seconds before repeating? Give it a try! How about some other patterns? Once you succeed at visualizing a desired outcome, coding it, and observing it to work as planned, you have taken an enormous step toward becoming a competent hardware hacker.

Step 6: Servo Motors

Servo motors are generally controlled by a series of repeating electrical pulses where the width of the pulses indicates the position of the servo. The pulse width modulated (PWM) control signal is often generated by a common microcontroller such as an Arduino.

Small hobby servos, such as the MG996R, are connected through a standard three-wire connection: two wires for a DC power supply and one wire for carrying the control pulses. MG996R servos have an operating voltage rage of 4.8-7.2 VDC.

Step 7: Assembling the Pan and Tilt Mechanism

  1. Pull both MG996R servos from their bags and set aside the included accessories for now.
  2. Attach an aluminum, circular servo coupler to each servo. Note that the couplers come in separate bags from the servos. The coupler is a very tight fit. Start by pressing the coupler onto the end of the servo output and then thread a screw into the center hole. Tighten the thread to draw the coupler onto the servo output.
  3. Note that there are three brackets for the pan-tilt assembly - two box-brackets and one U-bracket.
  4. Mount one of the box-brackets onto the aluminum circle for one of the servos. We will call this servo the pan servo. Orient the box-bracket with its center wall against the aluminum circle such that the other two walls of the box bracket face away from the pan servo. Use the center holes on the middle wall of the box bracket. This arrangement should allow the pan servo to spin the attached box-bracket around once it is actuated.
  5. Position the other servo (tilt servo) into the box-bracket that is attached to the aluminum circle of the pan servo. Use at least two nuts and bolts to affix the tilt servo - one on each side.
  6. Holding the U-bracket, insert the brass "bearing" from the inside of the U through one of the large pivot mounting holes.
  7. Place the U-bracket with the bearing onto the tilt servo that is inside the box bracket such that the other large pivot mounting hole (the one without the bearing) aligns with the aluminum circle on the tilt servo.
  8. Use screws to affix the U-bracket onto the aluminum circle on one side of the U-bracket.
  9. On the other side of the U-bracket, tighten a single screw through the bearing and into the small hole in the box-bracket within. This should allow the U-bracket to rotate around the box-bracket later when the tilt servo is actuated.

Step 8: Mounting the Pan and Tilt Assembly

The remaining box-bracket can be screwed down to a small wooden board scrap to serve as a camera base as shown in the image. Finally, the pan servo is mounted within that remaining box-bracket using at least two nuts and bolts to affix the servo to the bracket - one on each side.

Step 9: Wire and Test the Pan and Tilt Assembly

To wire the servos according to the schematic, it is quickest to just cut the original female connectors from the servos and then use some female DuPont jumper ends to get the signal and ground lines attached to the Nano pins.

The Nano does not have enough current on the 5V supply to power the servos from USB, so an additional supply is recommended. This can be anything in the 4.8-7.2 Volt range. For example, four AA batteries (in series) will work nicely. A bench supply or wall-wart is also a good choice.

The simple example Arduino code attached here as PanTiltTest.ino can be used to test control of the two servos from the serial monitor on the Arduino IDE. Set the monitor baud rate to match the 9600bps set in the example code. Entering angle values between 0 and 180 degrees will position the servos accordingly.

Finally, the USB Camera Module (or other sensor) can be mounted to the U-Bracket of the Pan-Tilt Assembly for use in tracking applications.

Step 10: Face Tracking With OpenCV

A machine vision face-tracking system can be implemented by combining subsystems as shown in the block diagram. The SerialServoControl sketch for Arduino can be found in the following Sparkfun tutorial along with a related demonstration using OpenCV, Processing, an Arduino, a USB Camera, and a Pan/Tilt Assembly to track a human face. The demo uses two servos to reposition the camera in order to keep the face centered in the video frame even as the user moves about the room. For example code in C#, check out the GitHub repository for the CamBot video.

Step 11: Hack the Planet

If you have enjoyed this Instrucable and would like to have a box of electronics and computer tech projects like this delivered right to your mailbox each month, please join us by SUBSCRIBING HERE.

Reach out and share your success in the comments below or on the HackerBoxes Facebook Page. Certainly let us know if you have any questions or need some help with anything. Thank you for being part of HackerBoxes. Please keep your suggestions and feedback coming. HackerBoxes are YOUR boxes. Let's make something great!

Share

Recommendations

  • Tiny Home Contest

    Tiny Home Contest
  • Creative Misuse Contest

    Creative Misuse Contest
  • Water Contest

    Water Contest

46 Discussions

0
None
TonyP110

Question 6 days ago

anyone know where I can buy more of these camera's? What they are called? I can't find them anywhere.

1 more answer

You can actually use any camera with a USB output. To find a small camera module like the one in the original HackerBox, search for "laptop camera module" on eBay or aliexpress. Make sure the module comes with the necessary USB cable before you order it.

Just started this project and am now stuck without a power supply for the servos. Is there any recommendation of what I can use? Usually all the components needed for a project are included. Thanks.

It is done! Mucho mucho thanks go to grtyvr. Using his code and adding few modifications to get it all to work on my Mac was all it took. I love it!

I finally figured out how to get LiveCamTest to work with Processing and use the Hackerboxes external cam within the Mac OSX system (MacBook Pro). The sketch was defaulting to the MacBook Pro's FaceTime HD camera. I looked at the parameters for the Capture module and I realized I needed to add two additional arguments to get it to use the external camera:

video = new Capture(this, 640/2, 480/2, "USB2.0 WebCamera", 30 );

The two extra arguments are needed for selecting the camera name and the frame rate.

I found the name of the external cam by using Webcamoid. I picked a frame rate of 30 but 15 works as well.

anyone fix the clip on lenses over the camera?

does anyone have a good stl file to mount the camera?

1 reply

https://www.thingiverse.com/thing:2683961

0
None
TedB8

9 months ago

Has anybody gotten the included webcam to work on OS X? It'd be nice if there were a link to drivers.

Also, what's the suggested way to attach the camera to the servo bracket? All the included screws seem to be either too big or too small.

2 replies

A good app for testing webcams on OSX is WEBCAMOID.

You can attach the camera module to the U-Bracket using a small nut/bolt (probably the "best" choice for several reasons), an adhesive velcro strip (how we did it), double-sided tape, or hot glue.

0
None
NickK8

9 months ago

I have the kit fully assembled, and the servo test works fine. Unfortunately, that's where everything falls apart. First, the USB camera I received doesn't appear to work. When I plug it in, it registers on OSX as "USB 2.0 Web Camera", but no image comes through. PhotoBooth allows the camera to be selected, but then just freezes until I unplug the camera. The tutorial on OpenCV is also over 6 years out of date. None of the software examples work anymore, as the libraries to support them are now gone. I've been wading through various OpenCV forks that are better maintained, but they don't use the same calls as the sample application from SparkFun. SparkFun's response is "the guy who did this project isn't with us anymore. If you figure it out, be sure to let us know!". Anyone have any better luck?

4 replies

I can confirm that WEBCAMOID allows the camera data to be displayed. Unfortunately, that doesn't fix any of the software concerns. Has anyone at Hackerboxes actually completed this build? If so, can you share your code? If not, why not?

Again, if anyone else has better luck, let me know. I am a software dev, so I am plunking away at translating the code, but it is an uphill battle considering how out of date the tutorials and sample code are.

NickK8, I had the same problems as you. I did the following two things and got it working. I suspect the first is not necessary, but just in case it is.

1) Installed GStreamer from https://gstreamer.freedesktop.org/documentation/in... I don't know if that helped. The processing LiveCamTest still did not produce images. I got on to this from someone trying to build this on Ubuntu and installed GStreamer to get the right codex. See

2) Used the following parameters in setup()

void setup()

size(320, 240);

video = new Capture(this, 160, 120, 30);

opencv = new OpenCV(this, 160, 120);

If I were starting from scratch I would do 2 first. The video stream is sensitive to these parameters, which I got from WEBCOMOID. The 30 (fps) can be a 15. Without something there, there is no video displayed. Also the sizes can be messed with some, but at some point the video stops being displayed. Doing "size" parameters the same as "video" parameters doesn't show video. Note that in the sketch they use scale(2) to blow up the video to the frame of size set by "size".

Hope this helps

Took a bit to get the code working for the final project. If you are having trouble, you can find working code at https://bitbucket.org/tcgerlach/facefollower

0
None
JonW27

9 months ago

I finally got around to putting the servos together today, pretty cool! I'm going to use it in a robot I'm building. Now I need to get into OpenCV. For a video of this project running the Arduino IDE on a Raspberry Pi (and a few other projects), please see this: https://www.facebook.com/jwagnertech

0
None
koz12

9 months ago

I just want to make a point since I haven’t seen it expressed in the tutorial or in the comments, with using dual power supplies with servos you should link grounds from the 2 separate power supplies. It’s not a good practice to power servos with a 9v battery. The 9v won’t last long. You can use it with the arduino but it is recommended that you use 4 AA batteries for both supplies.

I don't understand what the webcam is plugged into. Is the webcam connected to the Arduino or is it on a host computer and the Arduino only operates the tilt pan. Does the arduion function independent of being plugged into the computer?