With some effort I have found that controlling two servo motors to allow a webcam to pan/tilt while tracking a face in real time using the raspberry pi not as impossible as it may at first seem. With some careful tweaking and code optimization I was able to allow the pi to keep up with two servos while running OpenCV face detection at 320x240 looking for a right profile,left profile, and frontal face and adjusting the servos faster than once per second.

Step 1: Acquire the Hardware.

Things needed:

A raspberry pi -- Model A will work fine, I have the original Model B which has the same specs as the new Model A (minus network).
A pan/tilt bracket
Two Servos
A GPIO Ribbin Cable
A Pi-Supported Webcam --  I used a Logitech C210

Assuming you already have a raspberry pi and a webcam the additional hardware will run you about $25
<p>after running the program</p><p>i am getting this error</p><p>Traceback (most recent call last):</p><p> File &quot;/home/pi/Downloads/PiFace[1].py&quot;, line 172, in &lt;module&gt;</p><p> fface = frontalface.detectMultiScale(aframe,1.3,4,(cv2.cv.CV_HAAR_DO_CANNY_PRUNING + cv2.cv.CV_HAAR_FIND_BIGGEST_OBJECT + cv2.cv.CV_HAAR_DO_ROUGH_SEARCH),(60,60))</p><p>error: /build/opencv-ISmtkH/opencv- error: (-206) Unrecognized or unsupported array type in function cvGetMat</p><p>what should i do now??</p>
<p>Hi Grintor. thank you for the nice project. I have a problem. the program runs and detect faces, but the servos doesnt move. i connected them to gpio4 and gpio 17 at raspberry pi A. what can i do?</p>
<p>How would one get this to work with brushless gimbal motors?</p>
Can we use raspberry pi B3 camera module instead of webcam
<p>Hi Grintor - AMAZING WORK- Did you do any further work with this project?</p>
<p>Nice project! I used your core PiFace.py codebase as a starting point but modified it to use RPIO.PWM instead of ServoBlaster and also to use the RaspiCam instead of a USB camera. I did make one fix where the pan servo was moving the wrong direction when the image was flipped for the reverse profile search:</p><p>if lastface==3:</p><p> Cface[0]=(w/2)+(320-x-h)</p><p>In general for opencv, using an LBP cascade was a little faster for frontal face searches, but HAAR was more reliable for profile searches.</p><p>In addition to some other minor tweaks, I also added a video feed window with a face rectangle so I could see what it was looking at.</p><p>Did you ever find a more elegant way to handle the frame lag issue than just doing a bunch of successive reads?</p>
<p>Can you give me some guidance on how you added the video feed window? I am trying to do that, as the camera continuously over shoots the face and constantly corrects. I would like to see if I can determine why it is doing that. I have been trying to use imshow and having no luck. If you could share your code for adding the video display, that would be wonderful. Thanks.</p>
<p>Here are the code snippets I used for displaying the video feed:</p><p>#setup video at startup:</p><p>capture = cv2.VideoCapture(0) <br>capture.set(cv2.cv.CV_CAP_PROP_FRAME_WIDTH, width)<br>capture.set(cv2.cv.CV_CAP_PROP_FRAME_HEIGHT, height)</p><p>cv2.cv.NamedWindow(&quot;video&quot;, cv2.cv.CV_WINDOW_AUTOSIZE)</p><p>#update screen with rectangle around found face after every face search in the while loop</p><p> cv2.cv.Rectangle(cv2.cv.fromarray(aframe), (x,y), (x+w, y+h), cv2.cv.RGB(255, 0, 0), 3, 8, 0)<br> cv2.imshow(&quot;video&quot;, aframe)<br> cv2.waitKey(1)</p>
<p>@JennaSys any chance you could also share the code you used to get the capture from the raspberry pi camera module? I have been strugling for ours trying to figure it out ;)</p>
<p>I've had several people ask for my code updates for this so I went ahead and put it up on github:</p><p><a href="https://github.com/JennaSys/facetrack" rel="nofollow">https://github.com/JennaSys/facetrack</a></p><p>The python code is in facetrack2.py</p><p>I admittedly didn't do a good job of documenting my code changes since I was just messing around with this, so if anyone has questions, let me know and I'll go back and add explanations where necessary.</p><p>Also in case anyone is interested here is the pan/tilt hardware I used for the PiCam:</p><p><a href="http://www.adafruit.com/products/1434" rel="nofollow">http://www.adafruit.com/products/1434 </a> (Pimoroni camera mount)</p><p><a href="http://www.adafruit.com/products/1968" rel="nofollow">http://www.adafruit.com/products/1968 </a> (pan/tilt rig w/o servos) </p><p>or</p><p><a href="http://www.adafruit.com/products/1967" rel="nofollow">http://www.adafruit.com/products/1967 </a> (pan/tilt rig w/servos) </p><p>I also 3D printed an adapter for the camera mount so I wouldn't have to attach it to the pan/tilt rig with rubberbands:</p><p><a href="http://www.thingiverse.com/thing:480559" rel="nofollow">http://www.thingiverse.com/thing:480559</a></p><p>When I get a chance, I'll try and make a video of it in action. Thanks again to Grintor for putting up this instructable!</p>
<p>Hi Jenna. This is great. I am very new to the Pi and Python, I have the Pan/Tilt Mount from Adafruit, a Pi3 and OpenCV installed. I am a bit confused about how to connect the Pan/Tilt to the Pi and haven't been able to find any good info. Can you give me some tips? I realize they need to connect to the GPIO ports, but which wires go to which pins? Do I need a breadboard?</p>
<p>A breadboard definitely makes it a bit easier to make the connections - at least on a temporary basis.</p><p>In the code you'll see the lines:</p>pPan = 23pTilt = 24<br><p>which define the GPIO pins that the pan and tilt servos will connect to for the control signal. (You also have the red/black connections that are used to power the servos as well.)</p><p>That said, I don't know if the RPIO library used in this code will work with the Pi3. I know some work has been done on it, but I don't know if they fixed the compatibility with the newer RasPi boards yet or not.</p>
<p>hi, first of all give you thanks, thats more than i need to achieve my purpose.</p><p>But im having a trouble, when i run your code, the video display appears and inmediately rpi get frozen, it seems like it is too much for my raspberry, im using rpi model b+. Do you know why this happens??</p>
<p>I have had that happen before as well and have 2 possible causes:</p><p>1. Make sure the power to the Raspberry Pi is sufficient and that the servos aren't causing a brownout.</p><p>2. Try running the program outside of any editors (like Idle) using:</p><p>sudo python facetrack2.py</p>
hey Jenna what adjustments should i do when using the picamera?
<p>Thank you for finding that bug. I actually haven't done anything else with this project since I posted this. But, yours looks nicer than mine. If you make a video of it in action, i'll post it in this instructable and link you credit.</p>
How would I record the video and send it to web or the pi sd card storage?
<p>Great instructable. I made it with cam.</p>
Someone should do a step by step video of this
<p>Hello guys,</p><p>I am trying to make this project. But i have an issue, can someone tell me how to connect the servos so that they work? <br>I tried to connect the servos to a GPIO Ribbin atached to Raspberry GPIO but the camera sees my face but doesn't move a bit.</p>
how to show the video from the webcam?
<p>hi!</p><p>wrong, and what shall i do..?? please help me,... :( </p><p>Thanks in advance</p><p>//</p><p>OpenCV Error: Bad flag (parameter or structure field) (Unrecognized or unsupported array type) in cvGetMat, file /build/opencv-ISmtkH/opencv-, line 2482</p><p>Traceback (most recent call last):</p><p> File &quot;PiFace.py&quot;, line 168, in &lt;module&gt;</p><p> fface = frontalface.detectMultiScale(aframe,scaleFactor=1.3,minNeighbors=4,flags=(cv2.cv.CV_HAAR_DO_CANNY_PRUNING + cv2.cv.CV_HAAR_FIND_BIGGEST_OBJECT + cv2.cv.CV_HAAR_DO_ROUGH_SEARCH),minSize=(30,30))</p><p>cv2.error: /build/opencv-ISmtkH/opencv- error: (-206) Unrecognized or unsupported array type in function cvGetMat</p>
<p>Nice job! is there any way to make it with blob detection?</p>
<p>its so cool.. i made it ! </p>
Whether the cam comes to its initial position after the face goes out of its sight??
<p>Traceback (most recent call last):</p><p> File &quot;PiFace.py&quot;, line 14, in &lt;module&gt;</p><p> ServoBlaster = open('/home/pi/Desktop/ServoBlaster', 'w')# ServoBlaster is what we use to control the servo motors</p><p>IOError: [Errno 21] Is a directory: '/home/pi/Desktop/ServoBlaster'</p><p>can u hepl me fix it</p>
<p>It seems like /home/pi/Desktop/ServoBlaster should be a file, and not a directory.</p><p>Python's builtin function &quot;open&quot; opens files for reading from and writing to, so when you specify a directory, the program throws an error.</p><p>PiFace.py uses a file at /dev/servoblaster ... I'm not familiar with controlling servo's, but it seems like /dev/servoblaster is supposed to be a buffer (file) that's responsible for the communication between a the programmed instructions and the actual machine. </p><p>I hope that helps a little and sets you on the right track - but like I said, I know python but nothing about servos, so take what I say with a grain of salt. :)</p>
<p>I was getting all 0's returned to the console for a while. Turns out my glasses were interfering with the face detection! Also helps if you find the optimal distance from the webcam. </p>
<p>i am using code of facetrack2.py ,i got trouble in servo,it is not running along with detection, i connected it to GPIO 23 but still it is not running, servo i am using is 'mg 90s pro'. plz tell me that if there is some thing in the code i have to do? or i am connectiing to wrong pin ? bcz there are two things regarding pins ist is GPIO pin and other is pin #, i connected it to pin# 16, i-e GPIO 23</p>
<p>Hi Chris,</p><p>How easy/hard would it be to adapt this setup to use a joystick to control the movement of this servos? I was looking at something along these lines: <a href="http://letsmakerobots.com/node/33643" rel="nofollow">http://letsmakerobots.com/node/33643 </a> where the camera does not return to the centre position.</p><p>Thanks,</p><p>Chris</p>
<p>i am building a similar project only the camera will be mounted inside a 12&quot; steel ball so I am thinking of sending the coordinate info to an arduino and stepper controllers with step dir to control bigger stepper motors for pan tilt. Awesome project , great job!</p>
<p>Hello <br>did you use a bridge to controll those motors ?</p>
<p>No, they are controlled directly from the GPIO pins on the Pi</p>
<p>hey how do interface the web camera with pi the code send to me please,then which algorithm is best for face detection and tracking also please give the code on for image tracking in python format send to my id selvam27193@gmail.com</p>
<p>The code is in the zip file in the instructable. </p>
<p>Hey Grintor nice tutorial. I'm from the UK and I'm in an engineering scheme placement over here and would like some help with my team's project. We have to construct a rig to stream and control a number of cameras in a test lab at GKN, to a nearby office. We were wondering, is it better to control the servos directly from the GPIO pins, or to use an extension board such as a PiFace Digital? Thanks in advance, Dan.</p>
<p>'better' is subjective. Since raspian is not a real-time operating system (unless you recompile the kernel), the timing will be be more precise when you use an external controller like that, but I haven't seen this translate into any real-world performance impact. Also, the Pi signaling is 3.3v, some servos may require 5v. My advice is to give it a try, if you have problems then use a board.</p>
<p>Hello, how would you go about recording this video?</p>
<p>hey how do interface the web camera with pi the code send to me please,then which algorithm is best for face detection and tracking also please give the code on for image tracking in python format send to my id selvam27193@gmail.com</p>
<p>I want assembly for fitting servo motor </p>
Hello,<br><br>I want to try this one but I can't find the download for the &quot;PiFace.py&quot;. Can somebody help me?
It's in the zip file on step 6
Hello,<br><br>I want to try this one but I can't find the download for the &quot;PiFace.py&quot;. Can somebody help me?
Skip earlier question.... Resolved. Power fluctuations. <br>New problem also resolved: Servo 0 was moving opposite to correct tracking direction. Resolution: changed str(_Servo0CP) to str(300 - _Servo0CP) in the two ServoBlaster.write statements. Woohoo! Works like a charm!
Face recognition seems to be working great, but servo control seems erratic. When face is seen, and coordinates appear in the stream, the servos move way too far. Could it be that my webcam is running at higher resolution than 320x240? If so, where do I change the cam mode? <br>Thanks, -Jeff
Very good tutorial for beginners that uses <a href="http://www.intorobotics.com/18-excellent-tutorials-compilation-start-working-raspberry-pi/" rel="nofollow">Raspberry Pi</a> minicomputer. I add this tutorial in a series of tutorials from where the users can learn how to start working with this device.&nbsp;
i too get error on line: <br>fface = frontalface.detectMultiScale(aframe,1.3,4,(cv2.cv.CV_HAAR_DO_CANNY_PRUNING + cv2.cv.CV_HAAR_FIND_BIGGEST_OBJECT + cv2.cv.CV_HAAR_DO_ROUGH_SEARCH),(60,60)) <br>and rmmod uvcvideo <br>modprobe uvcvideo nodrop=1 timeout=5000 quirks=0x80 <br>does not do the job. <br>Could you help? <br>thanks
looks similar to something i've always wanted to do: <br>a tripod head that 'learns' a move temporarily for pan with ease-in and ease-out. <br>so I set its start point, the end point, the time interval it needs to do the deed. <br> <br>hooray. one take. maybe one more for good measure. <br> <br>once done i might break into the iLink connector and get to to control zoom as well. <br> <br>thing is i have so many more things to do
Hi the program works <br/>I wanted to run the script at start-up so I wrote <br/>python /home/pi/FaceDetection/PiFace/PiFace.py<br/>But it did not work it just output <br/>0,0<br/>0,0<br/>0,0<br/>0,0<br/>0,0<br/>And so on nothing happen even though I stared right into the camera

About This Instructable




More by Grintor:Pan / Tilt face tracking with the raspberry pi 
Add instructable to: