Introduction: Face Tracking Pan-Tilt Camera

Picture of Face Tracking Pan-Tilt Camera

A camera pan-and-tilt gimbal that is capable of tracking faces using computer vision and machine training so it can follow you around as you film or vlog. This way your face is always the centre of the action!

You can also train this gimbal to follow not only human faces but also objects, pets, food, or anything you can imagine by using different training data.

(Sorry for the potato quality of the MacBook camera)

Step 1: Support Me on Patreon

Consider supporting me in Patreon. I'd like to make better quality Instructables and more complex and interesting projects but most of these projects are quite costly and time consuming.

https://www.patreon.com/techmartian

Thank you for your support.

Step 2: Materials

(2) Servos

(1) Arduino Uno

(1) Jumper cables

3D print one

or buy one http://www.ebay.ca/itm/Pan-Tilt-Camera-Servo-Gimb... (includes the servos)

or make one of your own using scrap wood pieces and glue

Step 3: Installing Python

Picture of Installing Python

If you have programmed in Python (2.7) before, you can skip this step.

The language I used for the programs that track the faces and send the coordinates to the Arduino is Python so it must be installed in your computer to be able to run the code.

I personally use PyCharm (download here: https://www.jetbrains.com/pycharm/) as my main python IDE but you can use other IDE's like WingIDE, or just use a text editor and terminal. Most of the time, I just use Sublime Text and Terminal (built-in command line tool for a Mac). But, if you've never programmed or used Python before, I recommend starting with a simple free IDE like WingIDE (download here: https://wingware.com).

My code uses Python 2.7 so download that from here: https://www.python.org/downloads/release/python-2...

You can also use homebrew (this only applies to Mac users) to install python by typing the following line to terminal:

brew install python

Step 4: Installing OpenCV and NumPy

Picture of Installing OpenCV and NumPy

If you already have OpenCV and NumPy installed in your computer you can skip this step.

The libraries I am using are OpenCV and NumPy so both libraries must be installed in your computer. OpenCV is an open-sourced computer vision library used to find faces in still images. I've adapted my code to work on frames of a live video. While, NumPy is a powerful numeric calculations library for Python.

For Windows users install OpenCV by following this guide: http://docs.opencv.org/3.2.0/d5/de5/tutorial_py_s...

For Mac users I often had trouble installing it the way OpenCV recommended so I use homebrew to install it. Follow this set of instructions on how to install homebrew and OpenCV: https://gist.github.com/stilist/cd7fdfabbf9ec5ed8...

Installing OpenCV can be a hit-or-miss sometimes, so test if you have OpenCV using the snippet of code below.

import cv2

If you do not get any error messages when you run it, then you have installed it successfully

To install NumPy, open your command line tool (Terminal for Mac/Linux users and Command Prompt for Windows users) and type the following

pip install numpy

Pip is a package that comes when you install Python in your computer so it works cross-platform.

Similarly, if you want to check if NumPy has been installed successfully, run the following snippet of code below:

import numpy

Step 5: Install ArduinoIDE

Picture of Install ArduinoIDE

If you already have ArduinoIDE installed in your computer you can skip this step.

You can download the ArduinoIDE for the Arduino website: https://www.arduino.cc/en/Main/Software

Setup your your Arduino by connecting it to your computer and selecting the correct board and com port.

Step 6: Wiring the Servos

Picture of Wiring the Servos

The wiring for this project is pretty straight forward. Follow the diagram above.

Step 7: Printing/Assembling the Camera Gimbal

Picture of Printing/Assembling the Camera Gimbal

You can 3d print this camera gimbal designed by Jake King:

http://www.thingiverse.com/thing:71492

Make sure that before you place the servos onto the camera gimbal that you centre it first. You can centre a servo by applying +5V VCC to the red pin and GND to the black pin. The servo will automatically rotate to the centre position.

Step 8: Run the Code

Picture of Run the Code

You can download the code from my Github. I will update this as I continue to improve the code.

https://github.com/raptor16/FacialRecognitionCamer...

First run the main.py using your Python IDE or if you are using terminal like I am, type python main.py. This will launch the video capture mode of your camera.

Then, compile and upload the Arduino sketch.

To exit the video capture mode, go to the screen that is running the video capture -- Python IDLE -- and hit "q" to quit.

Step 9: Calibrating the Camera

Picture of Calibrating the Camera

Once you run the code, you will get two prompts. One is the average distance of the person the track from the camera and the other is the field of view of the camera.

Step 10: More...

Picture of More...

Change the training data, so if you want to film your cat for example, copy and paste the pictures of your cat to the training folder and the camera will be tracking your cat. You can learn more about training the HaarCascade here: http://memememememememe.me/post/training-haar-cas...

Run the code in a Raspberry Pi, which is a pretty much the same steps since I use a Mac and Raspberry Pis are computers that run on Linux. Now, you'll have a more compact and portable camera setup.

If you want to improve the system you can also try playing with the code and controlling Depth and z-axis zoom functionality.

Step 11: Vote for Me!

Picture of Vote for Me!

I've entered this instructables to both the robotics and the photography contest. Please vote for me if you liked it.

Comments

kaneleee made it! (author)2017-05-17

Hello, FlorenceC6.
Good to see you. This is Kane. I am very interested in your "face tracking Pan tilt Camera". I am not used to Python. It is first time for me to use python.
Would you please help me how to solve problem?

I already downloaded your code from Github and then copied your all code files to my pycharm project folder
(C:\Users\kaneleee\PycharmProjects\opencv_test).
After that, I run main.py in pycharm tool. There are error messages like below.
( I did not connect with webcam / arduino , just want to verify running python program. )

--- Error meesages ---
C:\Python27\python.exe C:/Users/kaneleee/PycharmProjects/opencv_test/main.py
Traceback (most recent call last):
File "C:/Users/kaneleee/PycharmProjects/opencv_test/main.py", line 10, in <module>
frameCount = vid2photos()
File "C:\Users\kaneleee\PycharmProjects\opencv_test\vid2photos.py", line 7, in vid2photos
cap.set(cv2.cv.CV_CAP_PROP_FPS, 1)
AttributeError: 'module' object has no attribute 'cv'

Process finished with exit code 1
---------------------------------------------------------------------------------

Would you please help me how to solve the problem ?

Thank you for your help.

From Kane Lee.

TechMartian (author)kaneleee2017-05-17

Hi Kane,

It's saying that the module has no attribute to cv which implies that the OpenCV library has not been installed properly. I uploaded a test.py onto my GitHub for you to run and test if numpy and cv2 is properly installed. Follow the steps in step 3 for installing the opencv library.

If you still have problems let me know.

Best,
Florence

martin2kid (author)TechMartian2017-10-12

Hello Florence,

I'm having same issue like Kane had and I'm also using windows environment(windows 10 home) hoping Kane shares work a around (it appear Kane made it and I'm also new in Python and OpenCV).

I followed all steps per your instruction and OpenCV.org per your link provided( I followed "Installing OpenCV from prebuilt binaries" section, step 1 through 7 and verified output of "3.2.0" as a result of "print cv2.__version__".

When I ran main.py from Python 2.7.13 Shell, I also got very same message in red as followed;

Traceback (most recent call last):
File "C:\Users\marti\Downloads\FacialRecognitionCamera-master\FacialRecognitionCamera-master\main.py", line 10, in <module>frameCount = vid2photos()

File C:\Users\marti\Downloads\FacialRecognitionCamera-master\FacialRecognitionCamera-master\main.py", line 7, in vid2photoscap.set(cv2.cv.CV_CAP_PROP_FPS, 1)

AttributeError: 'module' object has no attribute 'cv'

In addition to that I noticed that:

1: built-in camera light on my Laptop turned on but no popup window for video was opened, and I opened windows Camera program and it shows blue square was tracking my face in photo mode but not in video mode.

2: From Python Shell I typed "q" to stop tracking but I got message

Traceback (most recent call last):

File "<pyshell#1>, line 1, in <module> q

NameError: name 'q' is not defined

3: Whenever Camera program is opened, tracking is continued even without running the Python main.py even after restart. (no way to turn it off)

4: I tried with Webcam (Logitech C-920), same result.

Since I noticed it tracks face movement, I connected Arduino Uno and compiled your SevoControl.ino file and it failed, and I edited "include <Servo.h>" to "#include <Servo.h>" and I was able to compile and upload to Uno.

5: Main.py is running and ServoControl is running and servos are not moving (note servos were tested and runs off separate power source of 5.2 volt and ground wire is shared with Uno powered by 9 volt, I did this way because of direct connection to Uno's pin was not powering servos, I can hear servo motor is humming and maintain a position but are not receiving command from facial position changes or moving accordingly)

I tried to duplicate your setup on my MacBook following your link but OpenCV is not willing to be installed in multiple try and I followed other instruction for Sierra, but installing OpenCV to Mac running Sierra is way beyond complicated for me according to this instruction https://www.pyimagesearch.com/2016/11/28/macos-in...

Please help me,

Best

Martin

martin2kid (author)martin2kid2017-10-16

Hello Florence,

I was hoping to hear from Kane, and no luck so far. and I need your help please.

I thought blue square box was generated by your code, but it was actually from Windows 10 facial recognition process running in the background and confused me at the first place and I'm back to square 1.

Upon running your main.py: it now opens "Frame" popup video-feed in grayscale without any square box. (I changed vid2photos.py line 7 to "cap.set(cv2.CAP_PROP_FPS, 1)" It won't work without making that change.

and I also changed py2arduinopy line 2 to "ser = serial.Serial ('COM5', 9600)--no more error msg when Uno is connected.

But, pixel2degree.py generate "IndexError: list index out of range" followed by numbers [0, [[183], [197]], etc----

Help is appreciated

Martin,

martin2kid (author)kaneleee2017-10-12

Hello Kane,

I'm having exact same problem as you had from my windows 10 machine, and would you share how you solved that error "AttributeError: 'module' object has no attribute 'cv'" please?

Martin Kim

R.B (author)martin2kid2017-10-12

This has to do with the opencv3 library, in which this has been replaced by another function. At this moment cv2.cv.CV_CAP_PROP_FPS is outdated, and is replaced by the number 5, giving cap.set(5, 1). Yet this didn't fix everything with me though, as seen in my reply which I would like to have help with...

martin2kid (author)R.B2017-10-13

In vid2photos.py I changed line 7 to cap.set(cv2.CAP_PROP_FPS, 1) and shows no error and bring up B&W video(frame) feed from Windows built-in camera, but not from usb Webcam (you can not change default camera in windows 10 and I need to figure how to work around this).

Note: I also changed main.py line 9 to if __name__== "__main__":

And I also installed Pyserial, yet servo is not moving.

R.B (author)2017-10-11

Hello, I'm following this tutorial and have found some hiccups.

First off, I fixed a thing in the code because python was giving errors, at the following place:

vid2photos.py - line 7, changed cv2.cv.CV_CAP_PROP_FPS to 5, which I saw in the docs of the newer OpenCV library. The new code therefore is cap.set(5, 1)

Now, if I run python main.py, I get the camera view screen where my camera video is shown. This all seems to work correctly. I also put the Arduino code on the Arduino, which also uploads.

Unfortunately, this is all that happens. I don't see like a square marking my face on the little view and my arduino doesn't do much either. I downloaded the current version of the github repo and am using a Windows PC.

Do you know what I can do about this? Are there steps I missed? Right now I'm at step 8, running python pixel2degree.py returns a few errors (the error at line 44 where == is missing instead of = I already fixed, the current error is that int doesn't have a a get index function, guessing because of (parseList[i])[i]), and before I move on to try and fix that I would like to know what might be going on.

Thanks in advance!

onion2 (author)2017-05-16

voted
And i must try it ?

TechMartian (author)onion22017-05-16

Thank you! Awesome profile pic btw.

vanDrielElectric (author)2017-05-16

Nice I like it, I haven't used openCV before cause I wanted to track humans in general but now I realise that it's actually a neural network haha so I'll put it on the never ending projects to do list. Voted!

Thanks! If you're interested in training datasets you can also consider Clarifai. It's a really simple machine learning computer vision API, but it doesn't have localization yet.

About This Instructable

2,965views

106favorites

License:

Bio: Aerospace Engineer working in Software Development with a passion for Hardware.
More by TechMartian:Flying Captain America's Shield - RC PlaneWide Angle ShotsLED Ping Pong With Shift Register
Add instructable to: