In this project the Webcam sends video frames to the Visual Studio C++ which contains Open CV library running on our computer.
If Visual Studio C++ program detects the image of the object from the webcam then it calculates the co ordinates of X, Y axis and radius of the object.The co ordinates are sent accordingly to the arduino Mega/UNO via Serial communication between the arduino and visual studio c++. 
After receiving the co-ordinates the servo motors moves in X and Y direction and follows the object.

Here is the video of it :) Sorry for the dull video i was tired at that time :)



1. Servo Motors x 2
2. Webcam ( any webcam) x 1
3. Arduino Mega/Uno x 1
4. External Power Supply 5V x 1
5. BreadBoard x 1
6. Single Core connecting wires


1. Arduino IDE
2. OpenCV 2.3.1
3. Visual Studio C++ 2010 Express SP1


1. Visual Studio C++ Code (attached)
2. Arduino Program Code (attached)

Step 1: Software Installation

1) Download and install the OpenCV -2.3.1. OpenCV contains all the important files needed for the making of the project.


OpenCV installation can cause some problems, So see this video it would help alot. This video shows proper installation of OpenCV.


2) Download and install the Microsoft VisualStudio C++ 2010 Express.


3) Download and install Arduino IDE


Step 2: Attaching Servo Motors AND WebCam

Construction is rather simple...

First Servo Motors:

Project i have made is temporary so i have fixed the servos with Galvanised Sheets, they are easy to cut and use and can be drilled easily plus they are flexible.

You can use various jigs of different material to attach the servos to make them permanent or temporary as suited by you.

Once you have attached the servo motors now its time to attach the Webcam.


At the horn of the second servo controlling the Y direction as shown in the pic.

Connect the webcam using J-Locks.

It surely isn't the most beautiful looking project but it does the work :P

Step 3: Connections


1.Connect Lower Servo motor yellow wire with the PIN No - 9 of the arduino UNO.(controls x axis)
2.Connect Upper Servo motor yellow wire with the PIN No -10 of the arduino UNO(controls y axis)
3.Connect the red wire of the servo motor with 5v coming from arduino UNO as shown in the figure above.
4.Connect the black wire of the servos with the GND(Ground) coming from the arduino UNO as shown in figure above.


Connect the webcam with the usb port of the computer.


Connect the arduino UNO with the usb port of the computer.
Before uploading code in you arduino select proper COM Port and Board accordingly.

 You must change the C++ code to match the PC's com port and baud rate with Arduino UNO/MEGA. Also, the C++ code must be told which USB port the webcam is using.


Arduino UNO

Upload the program in arduino UNO


If everything was done correctly then. 

Start the program and 4 screens would open up...

1. First screen would be for selecting the appropriate HSV values so that u can opt for particular object you want to track for.
2.Second screen would show the original image of your webcam.
3.Third screen would show the processed image of your webcam usually a binary image.
4.Fourth screen would show u the x axis, y axis and radius of that image.

Step 5: Sources

Some of the links i followed and helped me alot.





If you need more help feel free to contact me
email ID: shamsheerverma10@gmail.com

I hope u enjoyed my instructable :P

Thank you :D :D

<p>sir, I am new to all these and I followed your instructions and put the open cv code in visual studio it worked but the output is coming that no serial port is being made and I have also uploded the aurdino code in aurdino but how to link these two ... please help me out to create a serial port... email id is amit.nayal24@gmail.com</p>
<p>hi.. the code worked well for me.. its a great project... i need to know if i can adjust the values of the track bar permanently so that every time i do not need to adjust it and the camera starts to detect the object right when i start running the program...</p>
<p>Hello, this is my first arduino project: sending a file (jpeg, pdf, txt)<br> from an arduino sd card module to android phone using arduino uno and <br>bluetooth module HC-05<br>It's possible with this hardware?<br><br>If it's possible can someone please send me the source code and android app use it to receive the file<br>thank you</p>
<p>can you please give me complete description on my mail?</p><p>my mail id is: atulzalke@yahoo.com</p>
<p>Hi. I have following error. Please help.</p><p>Unhandled exception at 0x766e40f8 in projectbs01.exe: Microsoft C++ exception: cv::Exception at memory location 0x00eff5f0..</p>
<p>請給我的源代碼</p><p>leahjoyce77@yahoo.com.tw </p>
<p>Hello! </p><p>I was wondering if you can tell me how to chose my camera from my USB port. I have Visual Studio 2015 downloaded, along with Arduino and Open CV. </p><p>Please let me know ASAP. Thank you. </p>
Hi, please can you help me? I have a problem - it write me error C2872:'ACCESS_MASK'<br>Please help me! :(
<p>if you wish to use the .NET Serial class, use the naming convention <br>&quot;\\\\.\\COMn&quot; and replace n with a number &gt; 9 to define your com port<br> for COM ports above 9 such as COM10, COM11, etc. </p><p>from link below</p><p><a href="https://www.instructables.com/id/Face-detection-and-tracking-with-Arduino-and-OpenC/" rel="nofollow">https://www.instructables.com/id/Face-detection-and-tracking-with-Arduino-and-OpenC/</a></p>
<p>I want to integrate the object identification into robot arm gripper. The robot arm will move as soon as the object is detected and try to grab it. Any suggestions how to make it?</p>
<p>Serial communication error coming.. from visual studio in this line </p><p>HANDLE = createfile () . This function is not returning a valid value. </p><p>Help me to resolve this..</p>
<p>Hey</p><p>Did you set the COM port correctly in the Visual Studio code ? First find out the COM Port with which the camera is connected after the change the COM Port in the visual studio code accordingly</p><p>Hope it helped. If you require anyhelp feel free to ask.</p>
<p>Hi</p><p>Thanks for your response.</p><p>Yes you are right. I got it. Visual studio will take COM port from 1 to 9 i think. I changed Arduino COM port to 1 and now i that error not coming. But one more difficulty i'm facing with createTrackbar(). Something regarding to pointer variable error coming as below.</p><p>Unhandled exception at 0x0fa745e2 in tracking.exe: 0xC0000005: Access violation reading location 0x0030004e.<br>I have attached the screen shots.</p>
<p>any one who snd me the whole project tutorial step by step i want to make my final year project but i realy dont know about any thing so ples some one help me please</p>
<p>can you please send me visual studio code of this project. Thank you.</p>
<p>Hello,</p><p>I am trying to build this project but my track bars are not working can you please send me the codes for visual studio. my e mail buddhi.charitha@gmail.com</p><p>This is what my track bars look like.</p><p>Please help me</p>
<p>I think your proyect is awesome and i would like to practice but i don't know where i can download the sources. I'll be gratefully if you post the links.</p><p>Thanks a lot.</p>
<p>What sources do you require ? :) </p>
<p>Hi!!</p><p>Visual Studio C++ Code and Arduino Program Code</p><p>Thanks!</p>
<p>Hey can you provide me your email-id? :) i will send you the codes</p>
<p>Hi; would you please send me these code : <br>1.visual studio c++ code </p><p>2.arduino program code </p><p>3.opencv code for color detection</p><p>Thanks a lot :)</p><p>My email address : nafiseh.ghafarnia@gmail.com</p>
<p>Hello</p><p>Have some one the Visual Studio code?</p><p>Can some one share it with me?</p><p>deikiel@gmail.com</p>
Hello sir<br>I'm working on a very similar project and it would be really helpful if u can provide me the codes at rudrasingbal@gmail.com .<br>Thank you
<p>i am new to arduino. i want to make a face detection and tracking system using visual c++. My C++ code is working well but m having problem in transferring data to the ports. can u please help me with this buddy...</p><p>i used face tracking code from</p><p>https://www.instructables.com/id/Face-detection-and-tracking-with-Arduino-and-OpenC/</p>
<p>please send source code</p><p>2012188@iiitdmj.ac.in</p>
<p>please send me source code</p><p>2012188@iiitdmj.ac.in</p>
<p>Hi, Could you please pass me the code? </p><p>Thank you :)</p><p>zeynepsatli@gmail.com</p>
<p>Hey there! Sir your project is great and coincidentally I am working on the same kind of project using Kinect and AVR using Visual Studio C++ 10 and opencv 2.4.9. But for a test run i want to work on it using Webcam. Can you please help me out and provide me the source code. </p>
<p>Could pass me the code? Please.</p><p>e-mail: sid566@yahoo.com.br</p><p>Thank's.</p>
<p>Hey</p><p>Sorry for the late reply. I was really busy these days.</p><p>I will send you the codes.</p><p>Thank You </p>
<p>NOONE GIVES CODES, SOMEONE TEACHES THEM BUT THEY DON'T SHOW OTHERS : NEW STARTED PEOPLE... AND THEY ARE TRYING TO SHOW OFF HERE... I HATE THESE KIND OF PEOPLE REALLY. THIS WEBSITE IS : &quot; https://www.instructables.com &quot; ;so you should do as the website says otherwise get off and don't show your non-sense projects here...</p>
<p>Hey</p><p>By commenting in caps wont help you... Have some respect...</p><p> If you want the codes just ask i will provide you if u want any assistance just ask i will provide you...</p><p> You are not super intelligent guy we all are here to learn... If you don't want to gain knowledge then it's fine but don't criticize people who are trying to learn.</p><p>If you want to learn just ask.</p>
<p>When i run it , it open a console where i have to type 5 numbers from sertest2.cpp and that's all, what am i doing wrong ?</p>
<p>i deleted the sertest2.cpp , ran the program and now i get this :</p><p>&quot;Unhandled exception at 0x776c1d4d in test.exe: Microsoft C++ exception: cv::Exception at memory location 0x005bf978..&quot;</p>
<p>the camera light turns on when i run it, can you help ?</p>
<p>Camera light turns on when you run it :)</p><p>What are the problems that you are getting?</p>
<p>I followed all the steps , made sure of it. When i run it in debug or release it compiles , and then i get </p><p>Unhandled exception at 0x776c1d4d in test.exe: Microsoft C++ exception: cv::Exception at memory location 0x005bf978..</p><p>I have some PDB files that are not loaded from the .dll for the web camera</p><p>and i also saw the following error: assertion failed (dst.data == dst0.data) is unknown function.</p><p>I have loaded the Symbols from Microsoft from options. Double check all the libraries are loaded correctly. Tried a different object detection code and it works, but i need to make this one work to use it on Arduino.</p><p>Ran it on Windows 7 x32, Windows 8 x32 and Windows 8.1 X64 with the proper settings for x64 but i get the same error.</p><p>If i disable the &quot;cvCvtColor(p_imgOriginal, p_imgHSV, CV_BGR2HSV);&quot; line the program compiles and runs, but i can only see the live camera feed the other windows don't display anything.</p><p>Can you help me out ? let me know if i forgot to mention anything, i tried to use all the possible fixes that i could find on google and other OpenCV forums</p>
<p>HANDLE hSerial = CreateFile(L&quot;COM25&quot;, GENERIC_READ | GENERIC_WRITE, 0, 0, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);</p><p>it shows error in this line saying :</p><p>error C2664: 'HANDLE CreateFileA(LPCSTR,DWORD,DWORD,LPSECURITY_ATTRIBUTES,DWORD,DWORD,HANDLE)' : cannot convert argument 1 from 'const wchar_t [6]' to 'LPCSTR'</p><p>i tried chang<em>ing to : L-deleted</em></p><p>HANDLE hSerial = CreateFile(&quot;COM25&quot;, GENERIC_READ | GENERIC_WRITE, 0, 0, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);</p><p>it says:<br>Serial port doesn't exist!<br>Error while setting up serial port!</p><p>and the x,y,r values are printed but my servo doesnt move...</p>
<p>I am really sorry for the late reply. </p><p>Have you checked the baud rate of your arduino microcontroller at Device Manager, Visual Studio C++ and Arduino IDE ?</p>
<p>i solved the problem... ur com port is 3 which is lesser than 9. if the com port is greater than 9 then the com port should be prefixed with &quot;\\\\.\\COM25&quot; like this... and it should be CreateFileW. i was vexed with this error and was trying to solve this error for many weeks.. the declaration should be like this....</p><p>HANDLE hSerial = CreateFileW(L&quot;\\\\.\\COM25&quot;, GENERIC_READ | GENERIC_WRITE, 0, 0, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);</p><p>now my program runs and reads arduino :) </p><p>very happy atlast :)</p>
<p>i had been trying to execute ur project for 2 days.... i had many errors.. i googled and rectified my errors... i was greatly relieved all worked good but i got </p><p>&quot;Serial port doesn't exist!</p><p>Error while setting up serial port!instructables&quot;</p><p>my arduino and servo motors works perfect . i checked by sending through &quot;serial monitor&quot; .</p><p>my only problem is now reading the port through visual studios. im trying to solve tat problem. if any1 knows how to rectify it. kindly share it guys.... </p>
<p>Hey</p><p>First of all if u have connected the arduino to computer. You might be having some COM port.That comport should be written inside the Visual Studio 2010 program that i have attached. You have to write the COM exactly the same as well as the same baud rate then only you can communicate visual Studio program with arduino program.</p>
ya i did tat b4 itself... but it still says com ports doent exist. i tried many solutions. but yet i dint get the o/p..!! im stuck with this smal problem.. im trying this for days... <br>if i use :<br>HANDLE hSerial = CreateFile(L&quot;COM25&quot;, GENERIC_READ | GENERIC_WRITE, 0, 0, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);<br>it says:<br>error C2664: 'HANDLE CreateFileA(LPCSTR,DWORD,DWORD,LPSECURITY_ATTRIBUTES,DWORD,DWORD,HANDLE)' : cannot convert argument 1 from 'const wchar_t [6]' to 'LPCSTR'<br>if i use:<br>HANDLE hSerial = CreateFile(&quot;COM25&quot;, GENERIC_READ | GENERIC_WRITE, 0, 0, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);<br>it says:<br>Serial port doesn't exist!<br>Error while setting up serial port!<br>error: frame is NULL<br>help me :(<br>
<p>Hi, verma I have done installation process but while i debug the program im getting &quot;fatal error C1083: Cannot open include file: 'opencv/cvaux.h': No such file or directory&quot;.</p><p>please reply as quick as possible</p>
<p>Hey you probably haven't added the cvaux library in the visual studio 2010 c++, see the installation guide video. It helps alot .</p>
kk i have done that one, program is succeed but the when i run the program i didnt get the image of original tab and how to connect the connection between computer and arduino
<p>Hi you can only have communication between arduino and computer via serial communication.</p><p>In the visual Studio you have to white the COM port. Com port should be exactly the same that you are using for Arduino. Then only you can have communication</p>
<p>All fine till the VC instructions then sorta &quot;whoaa&quot; looks like a few steps missing here. We aint all super C folks so maybe a bit of explanation to get the code into VC ?</p><p>I started with &quot;new project&quot; but then get 8 different options to choose from ?</p>
<p>you would have to choose win 32 console application :)</p>
<p>Hello, I'm sorry, I'm new to instructables and I see no link to donwload the visual or arduino Code. Could you please assist me? Thank you in advance 8 )</p>

About This Instructable




Add instructable to: