FACE TRACKING USING ARDUINO !!!

6,380

12

19

In a previous instructable I shared how you can communicate between Arduino and Python using 'pyserial' module and control a LED. If you haven't seen it check it out here: COMMUNICATION BETWEEN ARDUINO & PYTHON !

And how you can detect colour of an object and track it on screen, Check that out here : COLOUR DETECTION USING OPENCV AND PYTHON.

In this Instructable I will be showing you how to track faces using Arduino & Python and make the Camera follow the face. This may sound difficult but trust me it isn't, All you need is basic knowledge of Arduino and Python.

So lets get started...

Teacher Notes

Teachers! Did you use this instructable in your classroom?
Add a Teacher Note to share how you incorporated it into your lesson.

Step 1: Things You Will Need :

The requirements are minimum . Here I have provided part list of everything you need:

Hardware Requirement :

Software Requirement :

  1. Python 2.7 (Should be installed, Linux OS usually have it pre-installed)
  2. OpenCV (You can download it separately or install using 'pip install' Explained further)
  3. pyserial (Can be installed with pip)
  4. numpy.
  5. Haarcascade.

After every thing is gathered we can move on to the Installation Step...

Step 2: Setting-up Python Environment :

Installing Python:

So first we need Python 2.7 up and running. To do this first download and Install python 2.7.14. To check if it is installed correctly Goto : Windows Search >> Type "IDLE" >> Hit Enter. A Python Shell should pop up.

OR

In search type 'CMD' and hit enter to open Command Prompt. In CMD type >> python and hit enter, Python interface should display .

If you see an error in CMD, Do not panic you probably need to set environment variable. You can follow this tutorialHere to set up Environment Variable.

Installing 'pyserial' , 'OpenCV" and "numpy" in python :

To install these modules we will use use pip install ,

First open CMD and type the following codes:-

>pip install serial
>pip install opencv-python
>pip install numpy

these commands will install the necessary modules. Now we can move to the coding part...

Step 3: Python Script :

Before starting to write code first thing to do is make a new folder as all of the code needs to be stored in same folder. So create a new folder, name it anything you want. and download the 'Haarcascade' from below and paste it in the folder.

Now open notepad and write the script given below , Save it as 'face.py' in the same folder as haarcascade. (You can download the code I have provided the file below) :

#import all the required modules

import numpy as np
import serial
import time
import sys
import cv2

#Setup Communication path for arduino (In place of 'COM5' put the port to which your arduino is connected)
arduino = serial.Serial('COM5', 9600) 
time.sleep(2)
print("Connected to arduino...")

#importing the Haarcascade for face detection
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')

#To capture the video stream from webcam.
cap = cv2.VideoCapture(0)

#Read the captured image, convert it to Gray image and find faces
while 1:
    ret, img = cap.read()
    cv2.resizeWindow('img', 500,500)
    cv2.line(img,(500,250),(0,250),(0,255,0),1)
    cv2.line(img,(250,0),(250,500),(0,255,0),1)
    cv2.circle(img, (250, 250), 5, (255, 255, 255), -1)
    gray  = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    faces = face_cascade.detectMultiScale(gray, 1.3)

#detect the face and make a rectangle around it.
    for (x,y,w,h) in faces:
        cv2.rectangle(img,(x,y),(x+w,y+h),(0,255,0),5)
        roi_gray  = gray[y:y+h, x:x+w]
        roi_color = img[y:y+h, x:x+w]

        arr = {y:y+h, x:x+w}
        print (arr)
        
        print ('X :' +str(x))
        print ('Y :'+str(y))
        print ('x+w :' +str(x+w))
        print ('y+h :' +str(y+h))

# Center of roi (Rectangle)
        xx = int(x+(x+h))/2
        yy = int(y+(y+w))/2
        print (xx)
        print (yy)
        center = (xx,yy)

# sending data to arduino
        print("Center of Rectangle is :", center)
        data = "X{0:d}Y{1:d}Z".format(xx, yy)
        print ("output = '" +data+ "'")
        arduino.write(data)

#Display the stream.
    cv2.imshow('img',img)

#Hit 'Esc' to terminate execution 
    k = cv2.waitKey(30) & 0xff
    if k == 27:
       break

Once this is done , move on to write the code for arduino...

Step 4: Arduino Code :

After the python script is ready we need arduino sketch to control the servo. Refer the code below , paste it in Arduino IDE and save it as 'servo.ino' in the same folder as face.py and haarcascade . upload the code and move on to the next step to make the connections.

(Downloadable file given below)

#include<servo.h>

Servo servoVer; //Vertical Servo
Servo servoHor; //Horizontal Servo

int x;
int y;

int prevX;
int prevY;

void setup()
{
  Serial.begin(9600);
  servoVer.attach(5); //Attach Vertical Servo to Pin 5
  servoHor.attach(6); //Attach Horizontal Servo to Pin 6
  servoVer.write(90);
  servoHor.write(90);
}

void Pos()
{
  if(prevX != x || prevY != y)
  {
    int servoX = map(x, 600, 0, 70, 179);
    int servoY = map(y, 450, 0, 179, 95);

    servoX = min(servoX, 179);
    servoX = max(servoX, 70);
    servoY = min(servoY, 179);
    servoY = max(servoY, 95);
    
    servoHor.write(servoX);
    servoVer.write(servoY);
  }
}

void loop()
{
  if(Serial.available() > 0)
  {
    if(Serial.read() == 'X')
    {
      x = Serial.parseInt();
      if(Serial.read() == 'Y')
      {
        y = Serial.parseInt();
       Pos();
      }
    }
    while(Serial.available() > 0)
    {
      Serial.read();
    }
  }
}

Step 5: Pan-Tilt Mechanism :-

I have used a readily available kit for the Pan-Tilt. If you want you can make one yourself using wood/Plastic or even 3D print one.

The one I used is pretty cheap, and very easy to assemble. Yet if you want instructions on how to do that, you can find it here.

Step 6: Making Connections :

The Circuit is pretty simple. Just attach two servos to arduino.

  1. Vertical to Pin 5
  2. Horizontal to Pin 6
  3. Power to +5V
  4. Ground to GND

Check the circuit diagram for reference.

Step 7: TESTING :

  • After everything is done last thing to do is test if it works. To test first make sure that servos are properly connected to arduino and sketch is uploaded.
  • After sketch is uploaded make sure to close the IDE so the port is free to connect to python.
  • Now open 'face.py' with Python IDLE and press 'F5' to run the code. It will take a few seconds to connect to arduino and then you should be able to see a window streaming the web cam. Now the code will detect your face and the servos will track it track it.
  • The Servo should move as you move the object . Now just attach the camera to the servos so it will move along with servos.

Thank you.

Be the First to Share

    Recommendations

    • Made with Math Contest

      Made with Math Contest
    • Multi-Discipline Contest

      Multi-Discipline Contest
    • Robotics Contest

      Robotics Contest

    19 Discussions

    None
    hariuken00

    Question 27 days ago

    how can i set the full screen , only show part of the camera and my face is in the corner

    1 answer
    None
    WolfxPachariuken00

    Answer 26 days ago

    Try resizing the window using
    img = cv2.resize(img,(xx,xx)
    int the xx set the pixels you want.

    None
    wombat99

    2 months ago

    Hmmmmm... I wondering I could use something like this to track my cats so I can tell when they are going to pounce on me in the middle of the night while I'm sleeping......... 8-)

    1 reply
    None
    WolfxPacwombat99

    Reply 2 months ago

    you can if you find a cascade with cat data set XD

    None
    benwilliams

    2 months ago

    Hi I'm getting this error

    AttributeError: 'module' object has no attribute 'Serial'

    on line 16. I install pyserial instead of serial which enables it to launch however the tracking isnt very accurate. Any suggestions?

    6 replies
    None
    WolfxPacbenwilliams

    Reply 2 months ago

    I know, Most people don't get the perfect results. this depends on a number of factors.
    You can get yours a bit more accurate by playing around with these values.
    int servoX = map(x, 600, 0, 70, 179);

    int servoY = map(y, 450, 0, 179, 95);


    servoX = min(servoX, 179);

    servoX = max(servoX, 70);

    servoY = min(servoY, 179);

    servoY = max(servoY, 95);

    Adjusting it a little you can get better resuls.

    None
    benwilliamsWolfxPac

    Reply 2 months ago

    Thanks, what exactly re each of the values? I think i understand the min and max. are the others to get the starting position etc?

    None
    WolfxPacbenwilliams

    Reply 2 months ago

    Other are the values for map function, it maps the rotation of servo to analog values. in this case values received from the python script. For more information, you can look up the Servo map function on google. It will help you understand better.

    None
    benwilliamsWolfxPac

    Reply 2 months ago

    Ok thanks, what values would i need to change if the camera is overshooting and going slightly further than needed

    None
    WolfxPacbenwilliams

    Reply 2 months ago

    int servoX = map(x, 600, 0, 70, 179);
    int servoY = map(y, 450, 0, 179, 95);

    tinker around with the values "600" and "450" and see how you get the difference

    None
    JMuzz63

    6 months ago on Step 7

    Receiving the following error on line 25:

    " cv2.error: OpenCV(4.0.0) /Users/travis/build/skvark/opencv-python/opencv/modules/highgui/src/window_QT.cpp:592: error: (-27:Null pointer) NULL guiReceiver (please create a window) in function 'cvResizeWindow' "

    Cannot find a solution anywhere. Any suggestions?

    3 replies
    None
    KaliD13JMuzz63

    Reply 3 months ago

    Ok, fixed... line 18 should be:
    cv2.resize(img,(500,500))

    None
    WolfxPacJMuzz63

    Reply 6 months ago

    I couldn't replicate the error, can you run other opencv programs?

    None
    WolfxPacmaniyan

    Answer 6 months ago

    You can but you will have to modify the code accordingly.

    None
    WolfxPacaqilazmi26

    Reply 3 months ago

    If you know Python3 you can modify the code. Check online resources