Introduction: HackerBox 0024: Vision Quest

Picture of HackerBox 0024: Vision Quest

Vision Quest - This month, HackerBox Hackers are experimenting with Computer Vision and Servo Motion Tracking. This Instructable contains information for working with HackerBox #0024, which you can pick up here while supplies last. Also, if you would like to receive a HackerBox like this right in your mailbox each month, please subscribe at HackerBoxes.com and join the revolution!

Topics and Learning Objectives for HackerBox 0024:

  • Experimenting with Computer Vision
  • Setting up OpenCV (Computer Vision)
  • Programming the Arduino Nano from the Arduino IDE
  • Controlling Servo Motors with the Arduino Nano
  • Assembling a Mechanical Pan and Tilt Assembly
  • Controlling Pan and Tilt Motion with a Microcontroller
  • Performing Face Tracking using OpenCV

HackerBoxes is the monthly subscription box service for DIY electronics and computer technology. We are hobbyists, makers, and experimenters. We are the dreamers of dreams. HACK THE PLANET!

Step 1: HackerBox 0024: Box Contents

Picture of HackerBox 0024: Box Contents
  • HackerBoxes #0024 Collectable Reference Card
  • Three Bracket Pan and Tilt Assembly
  • Two MG996R Servos with Accessories
  • Two Aluminum Circular Servo Couplers
  • Arduino Nano V3 - 5V, 16MHz, MicroUSB
  • Digital Camera Assembly with USB Cable
  • Three Lenses with Universal Clip Mount
  • Medical Inspection Pen Light
  • Dupont Male/Female Jumpers
  • MicroUSB Cable
  • Exclusive OpenCV Decal
  • Exclusive Dia de Muertos Decal

Some other things that will be helpful:

  • Small wooden board scrap for camera base
  • Soldering iron, solder, and basic soldering tools
  • Computer for running software tools

Most importantly, you will need a sense of adventure, DIY spirit, and hacker curiosity. Hardcore DIY electronics is not a trivial pursuit, and we are not watering it down for you. The goal is progress, not perfection. When you persist and enjoy the adventure, a great deal of satisfaction can be derived from learning new technology and hopefully getting some projects working. We suggest taking each step slowly, minding the details, and never hesitating to ask for help.

FREQUENTLY ASKED QUESTIONS: We like to ask all HackerBox members a really big favor. Please take a few minutes to review the FAQ on the HackerBoxes website prior to contacting support. While we obviously want to help all members as much as necessary, most of our support emails involve simple questions that are very clearly addressed in the FAQ. Thank you for understanding!

Step 2: Computer Vision

Picture of Computer Vision

Computer vision is an interdisciplinary field that deals with how computers obtain high-level understanding from digital images or videos. From the perspective of engineering, computer vision seeks to automate tasks that the human visual system can do. As a scientific discipline, computer vision is concerned with the theory behind artificial systems that extract information from images. The image data can take many forms, such as video sequences, views from multiple cameras, or multi-dimensional data from a medical scanner. As a technological discipline, computer vision seeks to apply its theories and models for the construction of computer vision systems. Sub-domains of computer vision include scene reconstruction, event detection, video tracking, object recognition, 3D pose estimation, learning, indexing, motion estimation, and image restoration.

It is interesting to note that computer vision may be considered the inverse of computer graphics.

Step 3: Processing and OpenCV

Picture of Processing and OpenCV

Processing is a flexible software sketchbook and a language for learning how to code within the context of the visual arts. Processing has promoted software literacy within the visual arts and visual literacy within technology. There are tens of thousands of students, artists, designers, researchers, and hobbyists who use Processing for learning and prototyping.

OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library. OpenCV was built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products. The OpenCV library has more than 2500 optimized algorithms, which include a comprehensive set of both classic and state-of-the-art computer vision and machine learning algorithms. These algorithms can be used to detect and recognize faces, identify objects, classify human actions in videos, track camera movements, track moving objects, and so forth.

Install OpenCV within Processing from the File > Examples menu by selecting “Add Examples” and then under the Libraries tab installing both the Video and the OpenCV libraries. Open the LiveCamTest example for basic face tracking. Check out some other OpenCV for Processing examples here.

More Resources:

Getting Started with Computer Vision is a book project providing an easy entry point for creative experimentation with computer vision. It introduces the code and concepts necessary to build computer vision projects.

Programming Computer Vision with Python is an O'Reilly book on PCV, an open source Python module for computer vision.

Learning OpenCV

Computer Vision: Algorithms and Applications

Mastering OpenCV

Stanford Course CS231n Convolutional Neural Networks for Visual Recognition (16 Videos)

Chris Urmson TED Talk How a driverless car sees the road

Step 4: Arduino Nano Microcontroller Platform

Picture of Arduino Nano Microcontroller Platform

We can use any common microntroller platform for controlling servos in our pan and tilt camera mount. The Arduino Nano is a surface-mount, breadboard-friendly, miniaturized Arduino board with integrated USB. It is amazingly full-featured and easy to hack.

Features:

  • Microcontroller: Atmel ATmega328P
  • Voltage: 5V
  • Digital I/O Pins: 14 (6 PWM)
  • Analog Input Pins: 8
  • DC Current per I/O Pin: 40 mA
  • Flash Memory: 32 KB (2KB for bootloader)
  • SRAM: 2 KB
  • EEPROM: 1 KB
  • Clock Speed: 16 MHz
  • Dimensions: 17mm x 43mm

This particular variant of the Arduino Nano is the black Robotdyn design. The interface is by an on-board MicroUSB port that is compatible with the same MicroUSB cables used with many mobile phones and tablets.

Arduino Nanos feature a built-in USB/Serial bridge chip. On this particular variant, the bridge chip is the CH340G. Note that there are various other types of USB/Serial bridge chips used on the various types of Arduino boards. These chips allow you computer's USB port to communicate with the serial interface on the Arduino's processor chip.

A computer's operating system requires a Device Driver to communicate with the USB/Serial chip. The driver allows the IDE to communicate with the Arduino board. The specific device driver that is needed depends upon both the OS version and also the type of USB/Serial chip. For the CH340 USB/Serial chips, there are drivers available for many operating systems (UNIX, Mac OS X, or Windows). The maker of the CH340 supplies those drivers here.

When you first plug the Arduino Nano into a USB port of your computer, the green power light should come on and shortly after the blue LED should start to blink slowly. This happens because the Nano is pre-loaded with the BLINK program, which is running on the brand new Arduino Nano.

Step 5: Arduino Integrated Development Environment (IDE)

Picture of Arduino Integrated Development Environment (IDE)

If you do not yet have the Arduino IDE installed, you can download it from Arduino.cc

If you would like additional introductory information for working in the Arduino ecosystem, we suggest checking out the instructions for the HackerBoxes Starter Workshop.

Plug the Nano into the MicroUSB cable and the other end of the cable into a USB port on the computer, launch the Arduino IDE software, select the appropriate USB port in the IDE under tools>port (likely a name with "wchusb" in it). Also select "Arduino Nano" in the IDE under tools>board.

Finally, load up a piece of example code:

File->Examples->Basics->Blink

This is actually the code that was preloaded onto the Nano and should be running right now to slowly blink the blue LED. Accordingly, if we load this example code, nothing will change. Instead, let's modify the code a little bit.

Looking closely, you can see that the program turns the LED on, waits 1000 milliseconds (one second), turns the LED off, waits another second, and then does it all again - forever.

Modify the code by changing both of the "delay(1000)" statements to "delay(100)". This modification will cause the LED to blink ten times faster, right?

Let's load the modified code into the Nano by clicking the UPLOAD button (the arrow icon) just above your modified code. Watch below the code for the status info: "compiling" and then "uploading". Eventually, the IDE should indicate "Uploading Complete" and your LED should be blinking faster.

If so, congratulations! You have just hacked your first piece of embedded code.

Once your fast-blink version is loaded and running, why not see if you can you change the code again to cause the LED to blink fast twice and then wait a couple of seconds before repeating? Give it a try! How about some other patterns? Once you succeed at visualizing a desired outcome, coding it, and observing it to work as planned, you have taken an enormous step toward becoming a competent hardware hacker.

Step 6: Servo Motors

Picture of Servo Motors

Servo motors are generally controlled by a series of repeating electrical pulses where the width of the pulses indicates the position of the servo. The pulse width modulated (PWM) control signal is often generated by a common microcontroller such as an Arduino.

Small hobby servos, such as the MG996R, are connected through a standard three-wire connection: two wires for a DC power supply and one wire for carrying the control pulses. MG996R servos have an operating voltage rage of 4.8-7.2 VDC.

Step 7: Assembling the Pan and Tilt Mechanism

Picture of Assembling the Pan and Tilt Mechanism
  1. Pull both MG996R servos from their bags and set aside the included accessories for now.
  2. Attach an aluminum, circular servo coupler to each servo. Note that the couplers come in separate bags from the servos. The coupler is a very tight fit. Start by pressing the coupler onto the end of the servo output and then thread a screw into the center hole. Tighten the thread to draw the coupler onto the servo output.
  3. Note that there are three brackets for the pan-tilt assembly - two box-brackets and one U-bracket.
  4. Mount one of the box-brackets onto the aluminum circle for one of the servos. We will call this servo the pan servo. Orient the box-bracket with its center wall against the aluminum circle such that the other two walls of the box bracket face away from the pan servo. Use the center holes on the middle wall of the box bracket. This arrangement should allow the pan servo to spin the attached box-bracket around once it is actuated.
  5. Position the other servo (tilt servo) into the box-bracket that is attached to the aluminum circle of the pan servo. Use at least two nuts and bolts to affix the tilt servo - one on each side.
  6. Holding the U-bracket, insert the brass "bearing" from the inside of the U through one of the large pivot mounting holes.
  7. Place the U-bracket with the bearing onto the tilt servo that is inside the box bracket such that the other large pivot mounting hole (the one without the bearing) aligns with the aluminum circle on the tilt servo.
  8. Use screws to affix the U-bracket onto the aluminum circle on one side of the U-bracket.
  9. On the other side of the U-bracket, tighten a single screw through the bearing and into the small hole in the box-bracket within. This should allow the U-bracket to rotate around the box-bracket later when the tilt servo is actuated.

Step 8: Mounting the Pan and Tilt Assembly

Picture of Mounting the Pan and Tilt Assembly

The remaining box-bracket can be screwed down to a small wooden board scrap to serve as a camera base as shown in the image. Finally, the pan servo is mounted within that remaining box-bracket using at least two nuts and bolts to affix the servo to the bracket - one on each side.

Step 9: Wire and Test the Pan and Tilt Assembly

Picture of Wire and Test the Pan and Tilt Assembly

To wire the servos according to the schematic, it is quickest to just cut the original female connectors from the servos and then use some female DuPont jumper ends to get the signal and ground lines attached to the Nano pins.

The Nano does not have enough current on the 5V supply to power the servos from USB, so an additional supply is recommended. This can be anything in the 4.8-7.2 Volt range. For example, four AA batteries (in series) will work nicely. A bench supply or wall-wart is also a good choice.

The simple example Arduino code attached here as PanTiltTest.ino can be used to test control of the two servos from the serial monitor on the Arduino IDE. Set the monitor baud rate to match the 9600bps set in the example code. Entering angle values between 0 and 180 degrees will position the servos accordingly.

Finally, the USB Camera Module (or other sensor) can be mounted to the U-Bracket of the Pan-Tilt Assembly for use in tracking applications.

Step 10: Face Tracking With OpenCV

Picture of Face Tracking With OpenCV

A machine vision face-tracking system can be implemented by combining subsystems as shown in the block diagram. The SerialServoControl sketch for Arduino can be found in the following Sparkfun tutorial along with a related demonstration using OpenCV, Processing, an Arduino, a USB Camera, and a Pan/Tilt Assembly to track a human face. The demo uses two servos to reposition the camera in order to keep the face centered in the video frame even as the user moves about the room. For example code in C#, check out the GitHub repository for the CamBot video.

Step 11: Hack the Planet

Picture of Hack the Planet

If you have enjoyed this Instrucable and would like to have a box of electronics and computer tech projects like this delivered right to your mailbox each month, please join us by SUBSCRIBING HERE.

Reach out and share your success in the comments below or on the HackerBoxes Facebook Page. Certainly let us know if you have any questions or need some help with anything. Thank you for being part of HackerBoxes. Please keep your suggestions and feedback coming. HackerBoxes are YOUR boxes. Let's make something great!

Comments

Ctglodek (author)2017-11-10

does anyone have a good stl file to mount the camera?

Gurft (author)Ctglodek2017-12-01

https://www.thingiverse.com/thing:2683961

TedB8 (author)2017-10-31

Has anybody gotten the included webcam to work on OS X? It'd be nice if there were a link to drivers.

Also, what's the suggested way to attach the camera to the servo bracket? All the included screws seem to be either too big or too small.

Douglas Mooney (author)TedB82017-11-26

Yes. See my message to NickKB.

HackerBoxes (author)TedB82017-11-01

A good app for testing webcams on OSX is WEBCAMOID.

You can attach the camera module to the U-Bracket using a small nut/bolt (probably the "best" choice for several reasons), an adhesive velcro strip (how we did it), double-sided tape, or hot glue.

NickK8 (author)2017-10-31

I have the kit fully assembled, and the servo test works fine. Unfortunately, that's where everything falls apart. First, the USB camera I received doesn't appear to work. When I plug it in, it registers on OSX as "USB 2.0 Web Camera", but no image comes through. PhotoBooth allows the camera to be selected, but then just freezes until I unplug the camera. The tutorial on OpenCV is also over 6 years out of date. None of the software examples work anymore, as the libraries to support them are now gone. I've been wading through various OpenCV forks that are better maintained, but they don't use the same calls as the sample application from SparkFun. SparkFun's response is "the guy who did this project isn't with us anymore. If you figure it out, be sure to let us know!". Anyone have any better luck?

HackerBoxes (author)NickK82017-11-01

Try WEBCAMOID.

NickK8 (author)HackerBoxes2017-11-01

I can confirm that WEBCAMOID allows the camera data to be displayed. Unfortunately, that doesn't fix any of the software concerns. Has anyone at Hackerboxes actually completed this build? If so, can you share your code? If not, why not?

Again, if anyone else has better luck, let me know. I am a software dev, so I am plunking away at translating the code, but it is an uphill battle considering how out of date the tutorials and sample code are.

Douglas Mooney (author)NickK82017-11-26

NickK8, I had the same problems as you. I did the following two things and got it working. I suspect the first is not necessary, but just in case it is.

1) Installed GStreamer from https://gstreamer.freedesktop.org/documentation/in... I don't know if that helped. The processing LiveCamTest still did not produce images. I got on to this from someone trying to build this on Ubuntu and installed GStreamer to get the right codex. See

2) Used the following parameters in setup()

void setup()

size(320, 240);

video = new Capture(this, 160, 120, 30);

opencv = new OpenCV(this, 160, 120);

If I were starting from scratch I would do 2 first. The video stream is sensitive to these parameters, which I got from WEBCOMOID. The 30 (fps) can be a 15. Without something there, there is no video displayed. Also the sizes can be messed with some, but at some point the video stops being displayed. Doing "size" parameters the same as "video" parameters doesn't show video. Note that in the sketch they use scale(2) to blow up the video to the frame of size set by "size".

Hope this helps

moldavia (author)NickK82017-11-01

That's discouraging.

tcgerlach (author)2017-11-23

Took a bit to get the code working for the final project. If you are having trouble, you can find working code at https://bitbucket.org/tcgerlach/facefollower

MatthewL223 (author)2017-11-03

I blew up a power supply [OSOYOO lot MB-102 Breadboard 3.3V/5V] last night trying to power the servos. Looking back, 700mA was probably not enough. Anyone have examples of how they powered the higher voltage load? I'm thinking of using a L298N H bridge I have laying around, with the 5v output to power the Arduino, but that has a rube Goldberg feel I don't really like, and it doesn't really even solve my problem directly.

moldavia (author)MatthewL2232017-11-08

Not surprised. Each servo will draw between 500 and 900 mA @ 6V. I think that power supply was rated for 700mA @ 5V. Also, stall current is 2.6A @ 6V. Ideally, it should be powered at 6V, 5A for both servos, but that's a little unrealistic. For normal operation, both servos will draw approximately 11W. A 6V 2A supply should do it, or 5V 2.5A. Here's a datasheet.

http://www.electronicoscaldas.com/datasheet/MG996R_Tower-Pro.pdf

TonyP110 made it! (author)moldavia2017-11-12

Will this work? I'm a new at this. I'm not sure if 300 mA is too much, or too little at 6V.

lhelph (author)TonyP1102017-11-12

Unfortunately it will not provide enough amperage (A)

JonW27 (author)2017-11-10

I finally got around to putting the servos together today, pretty cool! I'm going to use it in a robot I'm building. Now I need to get into OpenCV. For a video of this project running the Arduino IDE on a Raspberry Pi (and a few other projects), please see this: https://www.facebook.com/jwagnertech

koz12 (author)2017-11-07

I just want to make a point since I haven’t seen it expressed in the tutorial or in the comments, with using dual power supplies with servos you should link grounds from the 2 separate power supplies. It’s not a good practice to power servos with a 9v battery. The 9v won’t last long. You can use it with the arduino but it is recommended that you use 4 AA batteries for both supplies.

DuaneB22 (author)2017-11-03

I don't understand what the webcam is plugged into. Is the webcam connected to the Arduino or is it on a host computer and the Arduino only operates the tilt pan. Does the arduion function independent of being plugged into the computer?

JeffG187 (author)2017-11-02

My webcam seems to only return a really crazy, kind of inverted image with scan lines for a moment ( literally 2 or 3 frames. ) Then it just stops returning anything. My built in laptop webcam works just fine, and this was working just fine until a few minutes ago. Anyone experience this?! I've unplugged / replugged it in, restarted the computer, restarted my app, etc. Nothing fixes it.

JeffG187 (author)JeffG1872017-11-03

Ok, I figured it out. The webcam works just fine, but OpenCV has a problem pulling the supported feed info from it. All you have to do is set the width / height / format of the stream by hand to fix it. ( Hint: 1600 x 1200, Fourcc: Y,U,V,2 )

JeffG187 (author)JeffG1872017-11-02

I should note, this webcam was working fine for the last day before this.

Juxpitos (author)2017-11-01

I know it's ugly, but before I wire this up and destroy it can someone with a bit more experience tell me if it will work safely or not, I don't have tons of skill and would rather not destroy this project.

NickK8 (author)Juxpitos2017-11-01

Your brown and orange are swapped. Brown is ground, red is positive, orange is data.

Juxpitos (author)NickK82017-11-02

Thanks Nick, not sure where I was looking for wiring. My other concern is power, can I use a 2x AA battery connector to power just one servo, and another to power the other sharing the ground? I also have a single 9v battery connector but don't think that would power both.

NickK8 (author)Juxpitos2017-11-02

BTW, I only had 2xAA battery boxes lying around. Just take two of them, link the black from one to the red from the other and you have a 4xAA pack

ptrebilcox-ruiz (author)Juxpitos2017-11-02

I just tried it with a single 9v and that powered both.

NickK8 (author)ptrebilcox-ruiz2017-11-02

I believe the servo rating is 4.8-7.2 V. They recommended 4 AA in series (6V). 2AA batteries will only provide 3V, and a 9V battery will provide (unsurprisingly) 9V. So, 2AA batteries will underpower a single servo, and a 9V battery will send too much power. Better to go with 4 1.5V batteries in series.

jevinson (author)2017-10-31

Ok, stupid question here. The Brass bearing, which screw is supposed to be used? The screws that seem to be the right ones are too long and bite in to the back of the Servo.

TedB8 (author)jevinson2017-10-31

I found there aren't actually any threads to screw onto (or maybe I stripped them?). I think the idea is just to use the screw to hack an "axle" for the bushing.

jevinson (author)TedB82017-11-01

thanks all for the feed back. i forgot about the nylock.

moldavia (author)jevinson2017-10-31

Save one of the black screws that come with the aluminum server adapter. That's the one that fits.

RobotFarmer (author)moldavia2017-11-01

I used one of the long silver screws. I put the head side next to the servo with the threads pointed out through the servo bracket. Then I put the brass bearing inside the U shape bracket and the threads sticking through that. I finished it up with a nut.

jevinson (author)moldavia2017-10-31

Tried that already, the black screws are too small. Different bracket different holes (ones that hadn't been messed with). They may catch on the coating but they do not tighten.

MikeM551 (author)jevinson2017-10-31

The silver machine screw and nyloc nut works .

moldavia (author)jevinson2017-10-31

Strange. I didn't have any issues. Just went through the bushing into the bracket.

RobotFarmer (author)2017-11-01

Hello All. If you just want to test to see if the cam is working try going to http://www.testmycam.net You will need to have Adobe Flash Player installed.

Enjoy!

Jbrid867 (author)2017-10-31

any word on mac drivers for the camera? Or am I SOL/just gonna need to use a Pi

TedB8 (author)2017-10-31

Instead of mounting to a board, I used the small tripod from the previous Wi-Fi Hackerbox. I pointed the tripod's mounting post to the side, then used one of the big holes on the "box" bracket. The plastic 1/4" nut from the same Hackerbox attached it.

G'lenH (author)2017-10-31

The screw size for the bearing screw appears to be an M3 x 0.50 (metric) screw. This fit for me!

ThomasL263 (author)2017-10-31

This is where I am going with it..

Racer1TN (author)2017-10-31

I'm thinking this would be fun to put a mask over the camera and have a face that looks at you. Maybe a Guy Fawkes mask? lol. Plenty of halloween masks to try. Add glowing eyes....May be a couple of days b4 i can start...can't wait.

RichW36 (author)2017-10-30

I had one servo couple fit super tight. The other slipped right on. Once you get the horn and the gear to mesh together than just use the screw to drive the horn on to the end of the servo. I can't wait to get to play around with this. . There is going to be something 3D printer i can just feel it.

neobigd (author)2017-10-30

In step 6, part 2, definitely follow instructions on mounting the servo coupler. They are super tight but using a screw to tighten down worked great.

itsonlym3 (author)2017-10-30

can't wait to begin!

About This Instructable

3,813views

11favorites

License:

More by HackerBoxes:HackerBox 0025: Flair WareHackerBox 0024: Vision QuestHackerBox 0023: Digital Airwaves
Add instructable to: