Real Time Face Tracking Robot With Arduino and Matlab

About: Currently ,studying at IIIT-SriCity, (India) and doing computer science (b.tech).

Intro: Real Time Face Tracking Robot With Arduino and Matlab

Real time face tracking refers to the task of locating human faces in a video stream and tracking the detected or recognized faces.Nowadays , there are many real world applications of face detection and other image processing techniques.There are many algorithms like Viola Jones , Kanade-Lucas-Tomasi ,camshift etc. that are quite helpful in face detection.

In our project , we made use of some functions (in Matlab) that together make up the Viola-Jones face detection algorithm.We built a robot that tracks the movement of a face from the video stream using Matlab and Arduino.

Step 1: Components Used

  1. Arduino Uno Microcontroller.
  2. Matlab software(R2012b , installed with arduino I/O support package).

  3. Arduino IDE(1.0.6)

  4. Servo Motors x 2.

  5. Usb Webcam (Logitech 720p).

  6. Jumper cables.

Step 2: Working

Codes could be found attached in this step.

  1. Firstly , code in Matlab detects a face from every frame of the live video stream and inserts a bounding box around the Region of Interest., which is a face in this case(by detecting some haar features present in the human faces).The project code follows the Viola Jones algorithm for face detection.
  2. The set of frames with bounding boxes make up the addition of a bounding box around the face in live video.While adding a bounding box , we also calculate the coordinates of centroid of the bounding box.

  3. These coordinates are sent as a string to the arduino UNO microcontroller., from Matlab and these are processed according to the code written on arduino IDE for the movement of motors.

  4. During processing , the arduino gets the positions of PAN and TILT servo motors (that are attached as shown in the project image).Then , arduino checks if the centroid coordinates lie in the centre region of the screen. We are trying to move the camera in such a way that the centroid lies at the centre of the frame.(The pan and tilt servos are given to the digital pins 9 and 10 respectively...)

  5. For this reason the frame is divided into left and right halves and also top and bottom halves.If the centroid falls in the left half , the camera is panned right and if it falls in the right half , camera is panned left and the same with the top and bottom halves and tilting.

Step 3: References and Team Members

Share

Recommendations

  • Furniture Contest 2018

    Furniture Contest 2018
  • Fix It! Contest

    Fix It! Contest
  • Metalworking Contest

    Metalworking Contest

64 Discussions

0
None
RiddhimanD

3 months ago

Hello,
We have followed the instructions step by step, but in the end only the x axis tracks the face, whereas the y axis just rotates in one direction whenever it recognises the face. Is there any way we can fix this issue?

1 reply
0
None
ErwingLRiddhimanD

Reply 21 days ago

the same problem happens to me :c... I have followed step by step and nothing works with the y axis.. i

0
None
Yokayyo

Question 4 months ago

Hello! Please answer, please.

How does the fopen(arduino) work? How does matlab know which arduino code file to access?

1 more answer
0
None
TroyC61Yokayyo

Answer 3 months ago

Typing in the right serial number and also open the file in Arduino

0
None
GenialG

6 months ago

we are supposed to get an orange image in the video player pls debug this problem

0
None
Jackd155

Question 7 months ago

Hello, I am getting the error
"% Report error if one occurred.

if errorOccurred

if length(jobject) == 1

warning(warnState);

error(message('MATLAB:serial:fopen:opfailed', msg));

else

warning(message('MATLAB:serial:fopen:invalid'));"
How can I fix this?

0
None
sayamegu

7 months ago

I'm use this code. But when I run MATLAB and bbox appear. Servo motors rotate camera away. Make the bbox disappear.

0
None
Jackd155

Question 7 months ago

Hello, do you think I could see the cables and connections clearly? I can't see the cables using the video and images. I am new to arduino

0
None
muhammadusama3607

7 months ago

Greetings. I am using the same code, but servo motor rotate camera in one direction only, once it moved in one direction, i want it to go back as face move to backward position, but it doesnot . how to tackle this

1 reply
0
None
rhishabh_raunak

1 year ago

Nice work buddy...

Is the USB cable attached to arduino be replaced by a battery ? Is it used for power supply only or if not how can I make it laptop free ?

1 reply
0
None
HossamT3

1 year ago

fopen(arduino);

this line gives me error

what do you think?

2 replies
0
None
BhaskarP6Thuta Zaw

Reply 8 months ago

check this line, rename COM6 with your arduino com port,say COM14 etc

0
None
MariaJ62

10 months ago

my matlab got stuck while loading the program whats the problem???

4 replies

i have test camera also i have run voila jones algorithm in it still the program stuck in and no result it only shows busy state

0
None
krishnachaithanya.movva

Reply 1 year ago

Thank you for pointing it out. Now I remember that I too tweaked the variables screenmaxx & screenmaxy to set the motor movements properly though I didn't remember how exactly. Based on what u said, there might be some misunderstandings about the screen resolution ('imaqhwinfo' command gives the details about the imaq. adaptors and the resolution as well). I hope you got the logical idea behind it i.e., whenever the center coordinate x gets > screenmaxx/2 - offset, the motor should move in such a way to compensate that distance from the center. The image is treated as a matrix of 320x240 and thus the values screenmaxx/2 and screenmaxy/2 to represent their center. Offset is used to reduce the shaking of the motors when the face is at the center of the frame.

Coming to the LSB part, I simply let the variables be as they were in the original code that I borrowed from the "FaceRecog. using arduino and Opencv" project on instuctables. It is not the servopositions that I'm sending to arduino but the center coordinates of the bounding box around the detected face in matlab. In matlab, the fprintf() function was called twice to send these x and y coordinates individually and hence there are two lines "LSB = Serial.read()" in the arduino code to receive them. Unlike the code used in Opencv, matlab code sends them using a single line, 'fprintf()'.