Gesture Controlled Robotic Arm Using Kinect & Arduino




About: Master of the Possible, an Engineer. An Ardent Tech Enthusiast. October born. A proud ThinkPad user.

This is a Simulink Model to Control a Robotic Arm By Gestures that are captured using Kinect. A Robotic Arm is developed using Servo Motors.

Teacher Notes

Teachers! Did you use this instructable in your classroom?
Add a Teacher Note to share how you incorporated it into your lesson.

Step 1: Things Used in This Project

Hardware components:

  1. Arduino Mega 2560
  2. Microsoft Kinect for Xbox 360
  3. Microsoft Xbox Kinect Adapter
  4. Adafruit Servo Motor
  5. SparkFun USB to Serial Breakout - FT232RL
  6. Gripper
  7. Solderless Breadboard Full Size
  8. Jumper wires (generic)

Software apps and online services:

  1. Arduino IDE
  2. Microsoft Kinect for Windows SDK v1.6
  3. Microsoft Windows SDK 7.1
  4. MATLAB 2013a

Step 2: Story

This is a Simulink Model to Control a Robotic Arm By Gestures that are captured using Kinect Sensor. A Robot Arm is developed using Servo Motors that replicates the Right Arm Shoulder, Elbow and Hand Movements.

My File at Mathworks File Exchange :

A Project By

B.Avinash & J.Karthikeyan

The Zip file contains the Matlab Simulink Support for Kinect and the Simulink Model named "GestureControlledRoboticArm" that WE HAVE CREATED and also the Arduino Code/Wiring Diagram. The Simulink Support for Kinect was downloaded from Mathworks File Exchange and the full credits go to "Takashi Chikamasa" for providing it. We do not own the copyrights or have developed the Simulink Support For Kinect. Simulink

Support for Kinect consists of IMAQ, Image, IR, Depth, Motion and Skeleton blocks to achieve real-time image data acquisition from Kinect devices such as Microsoft Kinect or ASUS Xtion PRO under Simulink simulation environment. It works with 32bit/64bit version of MATLAB R2010b or later in Windows7 32bit/64bit and it requires the following 3rd party software:

[C MEX Compiler] - Microsoft Visual Studio 2010 Express Edition (VC++) - Microsoft Windows SDK 7.1

[To use OpenNI ] - OpenNI - PrimeSense NITE - PrimeSensor Module [To use Microsoft Kinect SDK ] - Microsoft Kinect for Windows SDK v1.6 For more detailed information, please refer to document in the zip file and help of the blocks.

[Optional MATLAB products] - Computer Vision System Toolbox (highly recommended) - DSP System Toolbox (to acquire Kinect Microphone Array data) - Simulink Coder (to generate *.exe file)

Acknowledgements : Kinect Matlab, Simulink Support For Kinect, and Kinect Microsoft Sdk inspired this Project.

Step 3: Custom Parts and Enclosures

Simulink Model

Open the GestureControlledRoboticArm.mdl using Matlab after installing the Simulink Support for Kinect.

Download : GestureControlledRoboticArm.mdl

Matlab Simulink Code & Library Files (Instructions included)

In the .ZIP, you can find the Simulink for Kinect Support developed by Takashi Chikamasa, Mathworks. Inside the "slkinect" folder, you can find the instruction on how to install the library file in Matlab.

Download : Gesture Controlled Robotic Arm Using Matlab Simulink Support For

Step 4: Schematics - ​Arduino - Servo Connection

Arduino - Servo Connection

The data output (angle) from the matlab software is sent to the arduino via TTL device. The connection from the TTL - Arduino & Arduino - Servo motors are as shown in the diagram.

Step 5: Schematics - ​Simulink Model (Gesture Controlled Robotic Arm)

Simulink Model (Gesture Controlled Robotic Arm)

This is the simulink model we have developed to aquire the data from kinect, convert xyz co-ordinates to angle, decrypt the data and send it to Arduino via Serial Communication.

Step 6: Code - Arduino Program to Receive Angle Data From Matlab

Arduino Program to Receive Angle Data from Matlab - Arduino

The angle output from the Matlab Software is encypted. In the Arduino IDE, we have written a rogram to receive the angle data, decrypt it and send it to the servo motors.

Download : Arduino Code

Step 7: Awards and Recognitions

"Gesture Controlled Robotic Arm Using Kinect (Simulink Support For Kinect) and Arduino" has been chosen in the week's ( 29/10/2016) Pick of the Week! - Matlab Simulink Hardware Challenge 2016

Won the 4th place in "MATLAB International Simulink Hardware Challenge 2016"

Step 8: Credits

Step 9: Links

Circuits Contest 2016

Participated in the
Circuits Contest 2016

Be the First to Share


    • Instrument Contest

      Instrument Contest
    • Make it Glow Contest

      Make it Glow Contest
    • STEM Contest

      STEM Contest

    8 Discussions


    1 year ago

    Can you provide me vector equations for all the joints.


    Question 1 year ago on Step 5

    How can i fix this. Please and Thank you.

    Combine.jpgcommend error.pngerror.png

    2 years ago

    First this is an excellent project. I am using kinect xbox 1 sdkv 2.0 and visual studio 2013 desktop app, MATLAB R2016a. When I am running setup_kinectsdk.m file I get the following error.

    >> setup_kinectsdk
    ### Confirming Simulink Support for Kinect installations with Kinect SDK...
    ### Setting MATLAB path to .\Lib and .\Lib\doc_en directories.
    Warning: Name is nonexistent or not a directory: C:\Users\A\Documents\MATLAB\Lib
    > In path (line 109)
    In addpath (line 88)
    In setup_kinectsdk (line 54)
    Warning: Name is nonexistent or not a directory: C:\Users\A\Documents\MATLAB\Lib\doc_en
    > In path (line 109)
    In addpath (line 88)
    In setup_kinectsdk (line 55)
    ### Generating C:\Users\A\Documents\MATLAB\Lib\simulinkfornidinfo.m
    Error using fclose
    Invalid file identifier. Use fopen to generate a valid file identifier.

    Error in setup_kinectsdk (line 68)

    Please help.


    3 years ago

    this project. this website one number projet.
    10 star you deserve

    1 reply