Introduction: OBJECT DETECTION AND TRACKING USING OPENCV, VISUAL STUDIO C++ 2010 AND ARDUINO

INTRODUCTION
 
In this project the Webcam sends video frames to the Visual Studio C++ which contains Open CV library running on our computer.
If Visual Studio C++ program detects the image of the object from the webcam then it calculates the co ordinates of X, Y axis and radius of the object.The co ordinates are sent accordingly to the arduino Mega/UNO via Serial communication between the arduino and visual studio c++. 
After receiving the co-ordinates the servo motors moves in X and Y direction and follows the object.

Here is the video of it :) Sorry for the dull video i was tired at that time :)


MATERIAL REQUIRED:

HARDWARE:

1. Servo Motors x 2
2. Webcam ( any webcam) x 1
3. Arduino Mega/Uno x 1
4. External Power Supply 5V x 1
5. BreadBoard x 1
6. Single Core connecting wires

SOFTWARE:

1. Arduino IDE
2. OpenCV 2.3.1
3. Visual Studio C++ 2010 Express SP1


CODE :

1. Visual Studio C++ Code (attached)
2. Arduino Program Code (attached)


Step 1: Software Installation



1) Download and install the OpenCV -2.3.1. OpenCV contains all the important files needed for the making of the project.

link::http://sourceforge.net/projects/opencvlibrary/files/opencv-win/2.3.1/

OpenCV installation can cause some problems, So see this video it would help alot. This video shows proper installation of OpenCV.

http://www.youtube.com/watch?v=cgo0UitHfp8

2) Download and install the Microsoft VisualStudio C++ 2010 Express.

link::http://www.microsoft.com/visualstudio/en-us/products/2010-editions/visual-cpp-express

3) Download and install Arduino IDE

link::http://arduino.cc/en/Main/Software

Step 2: Attaching Servo Motors AND WebCam

Construction is rather simple...

First Servo Motors:


Project i have made is temporary so i have fixed the servos with Galvanised Sheets, they are easy to cut and use and can be drilled easily plus they are flexible.

You can use various jigs of different material to attach the servos to make them permanent or temporary as suited by you.

Once you have attached the servo motors now its time to attach the Webcam.

Webcam:

At the horn of the second servo controlling the Y direction as shown in the pic.

Connect the webcam using J-Locks.

It surely isn't the most beautiful looking project but it does the work :P



Step 3: Connections

SERVO MOTOR CONNECTION:

1.Connect Lower Servo motor yellow wire with the PIN No - 9 of the arduino UNO.(controls x axis)
2.Connect Upper Servo motor yellow wire with the PIN No -10 of the arduino UNO(controls y axis)
3.Connect the red wire of the servo motor with 5v coming from arduino UNO as shown in the figure above.
4.Connect the black wire of the servos with the GND(Ground) coming from the arduino UNO as shown in figure above.

WEB CAM CONNECTION:

Connect the webcam with the usb port of the computer.

ARDUINO UNO:

Connect the arduino UNO with the usb port of the computer.
Before uploading code in you arduino select proper COM Port and Board accordingly.

 You must change the C++ code to match the PC's com port and baud rate with Arduino UNO/MEGA. Also, the C++ code must be told which USB port the webcam is using.

Step 4: FINAL STEP :D UPLOADING

Arduino UNO

Upload the program in arduino UNO


VISUAL STUDIO C++

If everything was done correctly then. 

Start the program and 4 screens would open up...

1. First screen would be for selecting the appropriate HSV values so that u can opt for particular object you want to track for.
2.Second screen would show the original image of your webcam.
3.Third screen would show the processed image of your webcam usually a binary image.
4.Fourth screen would show u the x axis, y axis and radius of that image.

Step 5: Sources

Some of the links i followed and helped me alot.

https://www.instructables.com/id/Face-detection-and-tracking-with-Arduino-and-OpenC/

http://aleksandarkrstikj.com/tracking-a-ball-and-rotating-camera-with-opencv-and-arduino/

http://www.youtube.com/watch?v=cgo0UitHfp8

http://www.youtube.com/watch?v=bSeFrPrqZ2A

If you need more help feel free to contact me
email ID: shamsheerverma10@gmail.com

I hope u enjoyed my instructable :P

Thank you :D :D




Comments

author
kumar+kumar made it!(author)2017-07-05

I cant find the code:( i can just see that it is attached

author
AmitN33 made it!(author)2017-03-04

sir, I am new to all these and I followed your instructions and put the open cv code in visual studio it worked but the output is coming that no serial port is being made and I have also uploded the aurdino code in aurdino but how to link these two ... please help me out to create a serial port... email id is amit.nayal24@gmail.com

author
anjith made it!(author)2017-01-25

hi.. the code worked well for me.. its a great project... i need to know if i can adjust the values of the track bar permanently so that every time i do not need to adjust it and the camera starts to detect the object right when i start running the program...

author
ZiedB10 made it!(author)2016-10-27

Hello, this is my first arduino project: sending a file (jpeg, pdf, txt)
from an arduino sd card module to android phone using arduino uno and
bluetooth module HC-05
It's possible with this hardware?

If it's possible can someone please send me the source code and android app use it to receive the file
thank you

author
saileshz made it!(author)2016-08-03

can you please give me complete description on my mail?

my mail id is: atulzalke@yahoo.com

author
arash1371 made it!(author)2016-07-16

Hi. I have following error. Please help.

Unhandled exception at 0x766e40f8 in projectbs01.exe: Microsoft C++ exception: cv::Exception at memory location 0x00eff5f0..

author
leahjoyce77 made it!(author)2016-04-07

請給我的源代碼

leahjoyce77@yahoo.com.tw

author
sciencegirl777 made it!(author)2016-02-15

Hello!

I was wondering if you can tell me how to chose my camera from my USB port. I have Visual Studio 2015 downloaded, along with Arduino and Open CV.

Please let me know ASAP. Thank you.

author
TimcoV made it!(author)2015-11-12

Hi, please can you help me? I have a problem - it write me error C2872:'ACCESS_MASK'
Please help me! :(

author
nipon made it!(author)2015-10-20

if you wish to use the .NET Serial class, use the naming convention
"\\\\.\\COMn" and replace n with a number > 9 to define your com port
for COM ports above 9 such as COM10, COM11, etc.

from link below

https://www.instructables.com/id/Face-detection-and-tracking-with-Arduino-and-OpenC/

author
MilapJ made it!(author)2015-08-25

I want to integrate the object identification into robot arm gripper. The robot arm will move as soon as the object is detected and try to grab it. Any suggestions how to make it?

author
Vairamk made it!(author)2015-03-07

Serial communication error coming.. from visual studio in this line

HANDLE = createfile () . This function is not returning a valid value.

Help me to resolve this..

Picture1.png
author
shamsheer+verma made it!(author)2015-03-16

Hey

Did you set the COM port correctly in the Visual Studio code ? First find out the COM Port with which the camera is connected after the change the COM Port in the visual studio code accordingly

Hope it helped. If you require anyhelp feel free to ask.

author
Vairamk made it!(author)2015-03-17

Hi

Thanks for your response.

Yes you are right. I got it. Visual studio will take COM port from 1 to 9 i think. I changed Arduino COM port to 1 and now i that error not coming. But one more difficulty i'm facing with createTrackbar(). Something regarding to pointer variable error coming as below.

Unhandled exception at 0x0fa745e2 in tracking.exe: 0xC0000005: Access violation reading location 0x0030004e.
I have attached the screen shots.

Untitled.pngCapture1.JPG
author
mubashar_mubie made it!(author)2015-07-15

any one who snd me the whole project tutorial step by step i want to make my final year project but i realy dont know about any thing so ples some one help me please

author
bliyanapathirana made it!(author)2015-05-16

can you please send me visual studio code of this project. Thank you.

author
bliyanapathirana made it!(author)2015-05-16

Hello,

I am trying to build this project but my track bars are not working can you please send me the codes for visual studio. my e mail buddhi.charitha@gmail.com

This is what my track bars look like.

Please help me

Capture.PNG
author
Jorge+HugoE made it!(author)2014-10-09

I think your proyect is awesome and i would like to practice but i don't know where i can download the sources. I'll be gratefully if you post the links.

Thanks a lot.

author
shamsheer+verma made it!(author)2014-10-16

What sources do you require ? :)

author
Jorge+HugoE made it!(author)2014-11-03

Hi!!

Visual Studio C++ Code and Arduino Program Code

Thanks!

author
shamsheer+verma made it!(author)2015-02-20

Hey can you provide me your email-id? :) i will send you the codes

author
nafiseh1986 made it!(author)2015-05-02

Hi; would you please send me these code :
1.visual studio c++ code

2.arduino program code

3.opencv code for color detection

Thanks a lot :)

My email address : nafiseh.ghafarnia@gmail.com

author
deikiel made it!(author)2015-04-15

Hello

Have some one the Visual Studio code?

Can some one share it with me?

deikiel@gmail.com

author
rudra.singbal made it!(author)2015-04-09

Hello sir
I'm working on a very similar project and it would be really helpful if u can provide me the codes at rudrasingbal@gmail.com .
Thank you

author
parshav.jain.9 made it!(author)2015-03-20

i am new to arduino. i want to make a face detection and tracking system using visual c++. My C++ code is working well but m having problem in transferring data to the ports. can u please help me with this buddy...

i used face tracking code from

https://www.instructables.com/id/Face-detection-and-tracking-with-Arduino-and-OpenC/

author
rakesh8982 made it!(author)2015-03-12

please send source code

2012188@iiitdmj.ac.in

author
rakesh8982 made it!(author)2015-03-12

please send me source code

2012188@iiitdmj.ac.in

author
ZeynepA made it!(author)2015-03-10

Hi, Could you please pass me the code?

Thank you :)

zeynepsatli@gmail.com

author
FatimaI made it!(author)2015-02-24

Hey there! Sir your project is great and coincidentally I am working on the same kind of project using Kinect and AVR using Visual Studio C++ 10 and opencv 2.4.9. But for a test run i want to work on it using Webcam. Can you please help me out and provide me the source code.

author
sid.steuernagel made it!(author)2015-01-25

Could pass me the code? Please.

e-mail: sid566@yahoo.com.br

Thank's.

author
shamsheer+verma made it!(author)2015-02-20

Hey

Sorry for the late reply. I was really busy these days.

I will send you the codes.

Thank You

author
aykut.sirma1 made it!(author)2015-02-18

NOONE GIVES CODES, SOMEONE TEACHES THEM BUT THEY DON'T SHOW OTHERS : NEW STARTED PEOPLE... AND THEY ARE TRYING TO SHOW OFF HERE... I HATE THESE KIND OF PEOPLE REALLY. THIS WEBSITE IS : " https://www.instructables.com " ;so you should do as the website says otherwise get off and don't show your non-sense projects here...

author
shamsheer+verma made it!(author)2015-02-20

Hey

By commenting in caps wont help you... Have some respect...

If you want the codes just ask i will provide you if u want any assistance just ask i will provide you...

You are not super intelligent guy we all are here to learn... If you don't want to gain knowledge then it's fine but don't criticize people who are trying to learn.

If you want to learn just ask.

author
lazy.alex.948 made it!(author)2014-12-08

When i run it , it open a console where i have to type 5 numbers from sertest2.cpp and that's all, what am i doing wrong ?

EROR.jpg
author
lazy.alex.948 made it!(author)2014-12-08

i deleted the sertest2.cpp , ran the program and now i get this :

"Unhandled exception at 0x776c1d4d in test.exe: Microsoft C++ exception: cv::Exception at memory location 0x005bf978.."

author
lazy.alex.948 made it!(author)2014-12-08

the camera light turns on when i run it, can you help ?

author
shamsheer+verma made it!(author)2014-12-11

Camera light turns on when you run it :)

What are the problems that you are getting?

author
lazy.alex.948 made it!(author)2014-12-11

I followed all the steps , made sure of it. When i run it in debug or release it compiles , and then i get

Unhandled exception at 0x776c1d4d in test.exe: Microsoft C++ exception: cv::Exception at memory location 0x005bf978..

I have some PDB files that are not loaded from the .dll for the web camera

and i also saw the following error: assertion failed (dst.data == dst0.data) is unknown function.

I have loaded the Symbols from Microsoft from options. Double check all the libraries are loaded correctly. Tried a different object detection code and it works, but i need to make this one work to use it on Arduino.

Ran it on Windows 7 x32, Windows 8 x32 and Windows 8.1 X64 with the proper settings for x64 but i get the same error.

If i disable the "cvCvtColor(p_imgOriginal, p_imgHSV, CV_BGR2HSV);" line the program compiles and runs, but i can only see the live camera feed the other windows don't display anything.

Can you help me out ? let me know if i forgot to mention anything, i tried to use all the possible fixes that i could find on google and other OpenCV forums

author
bala.bruce made it!(author)2014-09-26

HANDLE hSerial = CreateFile(L"COM25", GENERIC_READ | GENERIC_WRITE, 0, 0, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);

it shows error in this line saying :

error C2664: 'HANDLE CreateFileA(LPCSTR,DWORD,DWORD,LPSECURITY_ATTRIBUTES,DWORD,DWORD,HANDLE)' : cannot convert argument 1 from 'const wchar_t [6]' to 'LPCSTR'

i tried changing to : L-deleted

HANDLE hSerial = CreateFile("COM25", GENERIC_READ | GENERIC_WRITE, 0, 0, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);

it says:
Serial port doesn't exist!
Error while setting up serial port!

and the x,y,r values are printed but my servo doesnt move...

author
shamsheer+verma made it!(author)2014-10-16

I am really sorry for the late reply.

Have you checked the baud rate of your arduino microcontroller at Device Manager, Visual Studio C++ and Arduino IDE ?

author
bala.bruce made it!(author)2014-10-26

i solved the problem... ur com port is 3 which is lesser than 9. if the com port is greater than 9 then the com port should be prefixed with "\\\\.\\COM25" like this... and it should be CreateFileW. i was vexed with this error and was trying to solve this error for many weeks.. the declaration should be like this....

HANDLE hSerial = CreateFileW(L"\\\\.\\COM25", GENERIC_READ | GENERIC_WRITE, 0, 0, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);

now my program runs and reads arduino :)

very happy atlast :)

author
bala.bruce made it!(author)2014-09-18

i had been trying to execute ur project for 2 days.... i had many errors.. i googled and rectified my errors... i was greatly relieved all worked good but i got

"Serial port doesn't exist!

Error while setting up serial port!instructables"

my arduino and servo motors works perfect . i checked by sending through "serial monitor" .

my only problem is now reading the port through visual studios. im trying to solve tat problem. if any1 knows how to rectify it. kindly share it guys....

author
shamsheer+verma made it!(author)2014-09-19

Hey

First of all if u have connected the arduino to computer. You might be having some COM port.That comport should be written inside the Visual Studio 2010 program that i have attached. You have to write the COM exactly the same as well as the same baud rate then only you can communicate visual Studio program with arduino program.

author
bala.bruce made it!(author)2014-09-19

ya i did tat b4 itself... but it still says com ports doent exist. i tried many solutions. but yet i dint get the o/p..!! im stuck with this smal problem.. im trying this for days...
if i use :
HANDLE hSerial = CreateFile(L"COM25", GENERIC_READ | GENERIC_WRITE, 0, 0, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);
it says:
error C2664: 'HANDLE CreateFileA(LPCSTR,DWORD,DWORD,LPSECURITY_ATTRIBUTES,DWORD,DWORD,HANDLE)' : cannot convert argument 1 from 'const wchar_t [6]' to 'LPCSTR'
if i use:
HANDLE hSerial = CreateFile("COM25", GENERIC_READ | GENERIC_WRITE, 0, 0, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);
it says:
Serial port doesn't exist!
Error while setting up serial port!
error: frame is NULL
help me :(

author
9603666805A made it!(author)2014-09-05

Hi, verma I have done installation process but while i debug the program im getting "fatal error C1083: Cannot open include file: 'opencv/cvaux.h': No such file or directory".

please reply as quick as possible

author
shamsheer+verma made it!(author)2014-09-05

Hey you probably haven't added the cvaux library in the visual studio 2010 c++, see the installation guide video. It helps alot .

author
9603666805A made it!(author)2014-09-06

kk i have done that one, program is succeed but the when i run the program i didnt get the image of original tab and how to connect the connection between computer and arduino

author
shamsheer+verma made it!(author)2014-09-19

Hi you can only have communication between arduino and computer via serial communication.

In the visual Studio you have to white the COM port. Com port should be exactly the same that you are using for Arduino. Then only you can have communication

author
BallscrewBob made it!(author)2014-07-05

All fine till the VC instructions then sorta "whoaa" looks like a few steps missing here. We aint all super C folks so maybe a bit of explanation to get the code into VC ?

I started with "new project" but then get 8 different options to choose from ?

author
shamsheer+verma made it!(author)2014-08-12

you would have to choose win 32 console application :)

About This Instructable

91,610views

215favorites

License:

More by shamsheer verma:OBJECT DETECTION AND TRACKING USING OPENCV, VISUAL STUDIO C++ 2010 AND ARDUINO
Add instructable to: