Introduction: Gesture Recognition Mouse With Matlab Image Processing.

Picture of Gesture Recognition Mouse With Matlab Image Processing.

Ever think that wearables will occupy important position in computer handling ?
Gesture does it all.
Welcome to my new project with gesture recognition mouse.

Here are the project requirements :
1) Colour tokens/markers ( red,blue,green)
2) Webcam.
3) Matlab software.
4) Windows pc.
5) Image acquisition toolbox for Matlab
6) Code for performing gestures virtually.

Step 1: Introduction to the System

Picture of Introduction to the System

1) Introduction To Matlab :
Matlab is a program that allows you to carry out computations in very straightforward manner. It is very useful in creating simulations of neural networks and data visualization .

2) Image Processing Using Matlab :
Matlab and image processing toolbox provide flexible environment to explore design ideas.
They are : 1) Image acquisition toolbox 2) Image processing toolbox.

Step 2: The Webcam

Picture of The Webcam

Webcam is used to track the user's hand gestures using computer vision based techniques.
Webcam constitutes to the important part of our system.

Step 3: Colour Markers

Picture of Colour Markers

Colour markers are the colour tokens used in user's fingers.
Marking the user's fingers with red,green,and blue tokens this helps webcam to recognize the hand gestures from user.

Step 4: Working of Mouse

Picture of Working of Mouse

Basically it works on following principles :
1) Capturing Real time video.
2) Image Flipping.
3) Conversion Of Flipped Image Into Binary Image.
4) Colour Detection.
5) Conversion Of Gray Scale Image Into Binary Scale Image.
6) Finding Centroid Of An Object.

Step 5: Tracking Mouse Pointer

Picture of Tracking Mouse Pointer

Once the coordinate is determined, the mouse driver is accessed and the coordinates are sent to the cursor.
With the help of this coordinates the cursor places itself in the required position. And each time new centroid is determined for new frame the cursor obtains new position creating an effect of tracking .

Step 6: Performing Clicking Actions

Picture of Performing Clicking Actions

Clicking action is based on simultaneous detection of colours.
For moving of cursor the red colour is detected.
For performing left click the red along with blue colour is detected.
For right click the red along with two stips of blue colour is detected.

Step 7: Program Execution.

Picture of Program Execution.

Refer to the image for program flowchart.
Thats it our wearable gesture mouse is ready to bang on .
Hit like if you like this project.

Here is the link to my youtube demonstration video on GRS MOUSE-

Also dont forget to vote for this project in 'Beyond the comfort zone' contest on instructables.

Mayur Yeole


About This Instructable



Bio: Hello Folks! Love to think out of the box and even more to implement .
More by im_MayurYeole:HOW TO MAKE A SIMPLE MUSIC PLAYER APP FOR ANDROID USING ANDROID STUDIO.DIY HAND-MOUSE.Gesture Recognition Mouse With Matlab Image Processing.
Add instructable to: