Instructables

Pan / Tilt face tracking with the raspberry pi

Picture of Pan / Tilt face tracking with the raspberry pi
IMG_20130306_234058_423.jpg
With some effort I have found that controlling two servo motors to allow a webcam to pan/tilt while tracking a face in real time using the raspberry pi not as impossible as it may at first seem. With some careful tweaking and code optimization I was able to allow the pi to keep up with two servos while running OpenCV face detection at 320x240 looking for a right profile,left profile, and frontal face and adjusting the servos faster than once per second.
 
Remove these adsRemove these ads by Signing Up

Step 1: Acquire the hardware.

Picture of Acquire the hardware.
Things needed:

A raspberry pi -- Model A will work fine, I have the original Model B which has the same specs as the new Model A (minus network).
A pan/tilt bracket
Two Servos
A GPIO Ribbin Cable
A Pi-Supported Webcam --  I used a Logitech C210

Assuming you already have a raspberry pi and a webcam the additional hardware will run you about $25
JennaSys made it!6 days ago

Nice project! I used your core PiFace.py codebase as a starting point but modified it to use RPIO.PWM instead of ServoBlaster and also to use the RaspiCam instead of a USB camera. I did make one fix where the pan servo was moving the wrong direction when the image was flipped for the reverse profile search:

if lastface==3:

Cface[0]=(w/2)+(320-x-h)

In general for opencv, using an LBP cascade was a little faster for frontal face searches, but HAAR was more reliable for profile searches.

In addition to some other minor tweaks, I also added a video feed window with a face rectangle so I could see what it was looking at.

Did you ever find a more elegant way to handle the frame lag issue than just doing a bunch of successive reads?

PiFace.jpg
tmatar11 days ago

Hello
did you use a bridge to controll those motors ?

jelimoore1 month ago

Hello, how would you go about recording this video?

selvam271933 months ago

hey how do interface the web camera with pi the code send to me please,then which algorithm is best for face detection and tracking also please give the code on for image tracking in python format send to my id selvam27193@gmail.com

selvam271933 months ago

hey how do interface the web camera with pi the code send to me please,then which algorithm is best for face detection and tracking also please give the code on for image tracking in python format send to my id selvam27193@gmail.com

dhaval1236 months ago

I want assembly for fitting servo motor

jangop8 months ago
Hello,

I want to try this one but I can't find the download for the "PiFace.py". Can somebody help me?
Grintor (author)  jangop8 months ago
It's in the zip file on step 6
jangop8 months ago
Hello,

I want to try this one but I can't find the download for the "PiFace.py". Can somebody help me?
JLRTech9 months ago
Skip earlier question.... Resolved. Power fluctuations.
New problem also resolved: Servo 0 was moving opposite to correct tracking direction. Resolution: changed str(_Servo0CP) to str(300 - _Servo0CP) in the two ServoBlaster.write statements. Woohoo! Works like a charm!
JLRTech9 months ago
Face recognition seems to be working great, but servo control seems erratic. When face is seen, and coordinates appear in the stream, the servos move way too far. Could it be that my webcam is running at higher resolution than 320x240? If so, where do I change the cam mode?
Thanks, -Jeff
cdragos george10 months ago
Very good tutorial for beginners that uses Raspberry Pi minicomputer. I add this tutorial in a series of tutorials from where the users can learn how to start working with this device. 
van wilder11 months ago
i too get error on line:
fface = frontalface.detectMultiScale(aframe,1.3,4,(cv2.cv.CV_HAAR_DO_CANNY_PRUNING + cv2.cv.CV_HAAR_FIND_BIGGEST_OBJECT + cv2.cv.CV_HAAR_DO_ROUGH_SEARCH),(60,60))
and rmmod uvcvideo
modprobe uvcvideo nodrop=1 timeout=5000 quirks=0x80
does not do the job.
Could you help?
thanks
u8nc11 months ago
looks similar to something i've always wanted to do:
a tripod head that 'learns' a move temporarily for pan with ease-in and ease-out.
so I set its start point, the end point, the time interval it needs to do the deed.

hooray. one take. maybe one more for good measure.

once done i might break into the iLink connector and get to to control zoom as well.

thing is i have so many more things to do
hotcheezy1 year ago
Hi the program works
I wanted to run the script at start-up so I wrote
python /home/pi/FaceDetection/PiFace/PiFace.py
But it did not work it just output
0,0
0,0
0,0
0,0
0,0
And so on nothing happen even though I stared right into the camera
Wrote it in /etc/rc.local
Grintor (author)  hotcheezy1 year ago
try making it start with LXDE:
http://wiki.lxde.org/en/Autostart
hotcheezy1 year ago
When I run python PiFace.py the outputs were
VIDIOC_QUERYMENU: Invalid argument
VIDIOC_QUERYMENU: Invalid argument
VIDIOC_QUERYMENU: Invalid argument
VIDIOC_QUERYMENU: Invalid argument
VIDIOC_QUERYMENU: Invalid argument
VIDIOC_QUERYMENU: Invalid argument
VIDIOC_QUERYMENU: Invalid argument
0,0
0,0
0,0
0,0
0,0
And it the 0,0 keeps on going and nothing else happens.

Any ideas what's happening?
Grintor (author)  hotcheezy1 year ago
it seems to be running, all the invalid argument stuff is normal. may be something wrong with the video. try to view your webcam feed through another program and see hew the video stream looks.
hotcheezy1 year ago
When I run make install_autostart is says Make: *** No rule to make target 'install_autostart'. Stop'.
any idea what's happening?
Grintor (author)  hotcheezy1 year ago
make sure you are doing 'make install_autostart' not doing 'sudo make install_autostart'
try 'make clean' before you run again.
DomNewell1 year ago
great tutorial but i keep getting an error on this line:
fface = frontalface.detectMultiScale(aframe,1.3,4,(cv2.cv.CV_HAAR_DO_CANNY_PRUNING + cv2.cv.CV_HAAR_FIND_BIGGEST_OBJECT + cv2.cv.CV_HAAR_DO_ROUGH_SEARCH),(60,60))
it says the array type is not supported.
Could you help?
thanks
Hi I had the same problem, you may have forgotten to run these commands

rmmod uvcvideo
modprobe uvcvideo nodrop=1 timeout=5000 quirks=0x80
embumpagi1 year ago
Did you use viola-jones algorithm to tracking the face ?
Grintor (author)  embumpagi1 year ago
No, Haar Cascade.
embumpagi1 year ago
Does it still connection to computer ? I mean it still need connection to PC for image processing or maybe raspberry pi can replacing the PC ?
Grintor (author)  embumpagi1 year ago
No, the pi does all the image processing and controls both servos
Hey Gintor.
This looks project looks awesome I've been trying it out as my first project with the PI but have hit a wall I don't seem to be able to get around, When I run "sudo make install_autostart" to install the servoblaster driver i get the responce "CP: cannot stat '/udev_scripts/servoblaster". I've tried this a root too but i get the same response each time. I've gone through the servoblaster.c file and added the servoblaster and 20-servoblaster.rules file to my /etc/udev directories but still no luck.
I've downloaded the servo blaster directory to my pi and placed it in the folder "/pi/servblaster".
I'm very new to linux and the rapberry pi so I'm guessing it;s something ridiculously simple I've missed. Any Ideas would help.
Go into the Makefile and copy the lines where it gets stuck and execute them manualy in the terminal, i.e cp /udev_scripts/servoblaster...

then delete them from the make file.
about 4 lines gave me issues , not sure why

im stuck on the line " sudo modprobe servoblaster" , it gives me the error: could not insert 'servoblaster' : Exec format error ???

does this have something to do with the kernel version?
Grintor (author)  bluebean1 year ago
I used kernel version 3.6.11+. If the module won't load, servoblaster offers a userspace daemon:
make servod
chmod +x servod
sudo ./servod

You can just start servod before each run
Thanks that worked, will carry on from there.
Grintor (author)  Monkeyking6681 year ago
You know... I think I had a similar problem and I am pretty sure I documented the command wrong. I think sudo throws it off try just running "make install_autostart" instead of "sudo make install_autostart" let me know if that fixes it and I will update the instructable.
Thanks Gintor

Changing the command to "make install_autostart" have solve the issue.

Thanks again
katlanta1 year ago
Hi,.. that's very cool,.. good job...:D

but, i can't acces my camera (c210) on OpenCv

cvCaptureFromCam(CV_CAP_ANY)

and i got warning "Null",.. i'm very confused, what's wrong, and what shall i do..?? please help me,... :(
sorry bad english..
Grintor (author)  katlanta1 year ago
do you have a /dev/video0 device show up when you plug in the cam?
yes, if i plug in my cam on Pi /dev/video0 show up,..
on luvcview my cam properly detected.. but when i use opencv,. i got NULL,..
mathman471 year ago
Nice job, Chris. By coincidence, a pan and tilt kit arrived on my doorstep yesterday. I wondered how I was going to control it. Thanks to you, now I know how.
Grintor (author) 1 year ago
Come on! 411 views and no comments, really?