UPDATE JAN 2012: This guide is featured on Microsoft's MSDN Channel9


In this project, I use left and right hand movements in Kinect to control the rotation of two servos tied to Arduino Uno, which in turn moves the left and right arms of a makeshift stickman. The middleware is written in Visual Basic 2010 and is based on the Microsoft Kinect SDK beta 2.

There are plenty of C++ and C# code examples to interface Kinect to other gadgets, but not many Visual Basic examples. So I figured this project could help folks with mostly VB skills to start hacking Kinect with Arduino.


Kinect keeps track of 20 skeleton joints for each player.  To simplify this project, I extract the coordinates of the left hand and right hand joints only. Then I pass those joint coordinates to an Arduino UNO after I do some scaling in Visual Basic from computer screen size to servo angles. 

I modified the VB code in the Microsoft Quickstart samples because it's well documented and comes with good tutorials. So anyone can review these tutorials to learn more about how this VB code works with Kinect.

The changes I made to the sample VB code were mostly to facilitate serial communications with Arduino as well as to map screen coordinates to the servos angles. In this project, I capture the Y (vertical) movement of the HandLeft and HandRight joints. It should be easy to modify my code to add more servos and joints.


Install all the required Microsoft software following the instructions provided by the vendors. Refer to Microsoft's Kinect for Windows SDK Quickstarts http://channel9.msdn.com/Series/KinectSDKQuickstarts Microsoft did a great job with these tutorials.

The biggest challenge is getting the various packages to work together.  When you successfully run the Kinect demo provided by The Microsoft Kinect SDK, you are half way there. 

The second part is wiring the servos to the Arduino and running my code. 

I kept my code as simple as possible so there's little error trapping and no fancy user interface. You can add that per your requirements.  


For this project, I used two TG9e micro servos powered by the Arduino Uno connected to my PC via USB. For bigger servos with more power needs, you may have to power them separately or else the Arduino will act erratically.

I labeled one servo as the Left Hand servo while the other as the Right Hand servo. If you confuse the servos at some point, the fix is often as simple as changing a line of code instead of rewiring.

Left Servo --->  Arduino Digital Pin 9
Right Servo --->  Arduino Digital Pin 11 

For debugging purposes, you may wish to consider testing your build with one servo before adding another.


In addition to the instructions, you can view my step-by-step screen captured stills. Please click the "i" in the upper left corner of each photo to see the full resolution still.

- Download the "ArduinoXYZ.ino" file (part of the attached techbitar.zip file) and load it into your Arduino UNO via the Arduino IDE 1.0.

- Download KinectForWindowsSDKQuickstarts.zip

-  Find and extract the SkeletalTrackingVB folder and place it under the Projects folder for your Visual Basic 2010 Express.

The SkeletalTrackingVB project files you have extracted have old reference files from an older Kinect SDK and Code4Fun toolkit. We need to delete the old references then add the latest ones.

NOTE: When following my videos or screen captured instructions, please keep in mind that I may have extracted the project and support files to different directories than yours.

- From Visual Basic 2010 IDE explorer, double click on MyProject. You will be presented with a project configuration page.

- Click on the References tab.

- Search for Microsoft.Research.Kinect and delete it.

- Search for Code4Fun.Kinect.wpf and delete it.

- Click on Add.

- Click the Browse tab.

- Search for and add the newer versions of those files which you have downloaded part of the latest Microsoft Kinect SDK and Coding4Fun Kinect Toolkit zip files. As of the writing of this guide, the versions for Code4Fun.Kinect.wpf should be and for Microsoft.Research.Kinect

- Download my MainWindow.xaml and MainWindow.xaml.vb (they can be found in the attached techbitar.zip file) which I have attached to this guide and copy them on top of the ones you have extracted under SkeletalTrackingVB\SkeletalTracking folder or cut and paste their content over the extracted files with the same names.

With the Kinect and Arduino connected, you should be able to test the program.

NOTE: The COM port field on the main application window is set to 16. Change this to match the Arduino's COM port on your PC.


The Kinect I am using in this project has its own power supply attached to the USB cable.  I bough this as a stand-alone without the XBOX. Some Kinects don't have an external power supply and get their power from the XBOX. If you have one of those, you will need to buy a Microsoft Kinect power supply which seems overpriced. So before you buy a Kinect/XBOX bundle, check if the bundled Kinect can be powered and operated as a stand-alone.


- Microsoft Kinect
- Arduino Uno
- Micro Servos X 2 (I used TG9e micro servos)
- Breadboards and wires


Arduino IDE 1.0

Microsoft Kinect SDK V1 beta2

Coding4Fun Kinect Toolkit 1.1

Microsoft Visual Basic Express 2010

.NET Framework 4

Microsoft DirectX SDK - June 2010 or later version

Runtime for Microsoft DirectX 9

Microsoft Kinect Code4Fun Quickstart VB code (You will overwrite this with my modified version)


Kinect for Windows SDK Quickstarts

Open Kinect

Kinect Hacks

Kinect Motion Capture

Developer: Hazim Bitar
Email: techbitar {at} gmail {dot} com
<p>Can anyone send me KinectForWindowsSDKQuickstarts.zip and Microsoft Kinect Code4Fun Quickstart VB code my email is w_martin_16_y@hotmail.com</p><p>Thanks!</p>
<p>Link is broken can u send me </p><p><strong>KinectForWindowsSDKQuickstarts.zip</strong></p><p>it my mail kaan@developeradayi.com or kaanulkerin@hotmail.com</p><p>i wait, please send me :(</p>
<p>Link is broken can u send me </p><p><strong>KinectForWindowsSDKQuickstarts.zip</strong></p><p>it my mail kaan@developeradayi.com or kaanulkerin@hotmail.com</p><p>i wait, please send me :(</p>
<p>I have made the Mearm robotic arm http://www.mearm.com/collections/mearm/products/mearm-your-robot-nuka-cola-blue and using arduino to code it to work. I would now like to use my Kinect for windows to control it. Please can you let me know how I can successfully do this please. If you can contact me on shaadk@hotmail.co.uk it would be very much appreciated. Thank you for your cooperation. </p>
<p>Can any one send me the file <strong> KinectForWindowsSDKQuickstarts.zip</strong> my email Id <strong>bboiarjun@gmail.com</strong></p>
<p>can anyone send me the file </p><p>KinectForWindowsSDKquickstarts.zip</p><p>my email id is : hunter.davidson003@gmail.com</p><p>thank you anyone.</p><p>m<br></p>
<p>can anyone send me the file</p><p><a href="http://files.ch9.ms/coding4fun/KinectForWindowsSDKQuickstarts.zip" rel="nofollow">KinectForWindowsSDKQuickstarts.zip</a></p><p>my email ID mahanta.tapas9@gmail.com</p>
You get the kinect for windows SDK quick starts. Zip
Hi,My project title gesture controlled robot using arduino and kinect sensor, I need kinect for windows SDKQuickstarts.zip file plz send to my email Id ssgani28@gmail.com
Hi,My project title gesture controlled robot using arduino and kinect sensor, I need kinect for windows SDKQuickstarts.zip file plz send to my email Id ssgani28@gmail.com
Can anyone send me the file KinectForWindowsSDKQuickstarts.zip my email ID ssgani28@gmail.com
Can any one send me the file KinectForWindowsSDKQuickstarts.zip my email Id ssgani28@gmail.com
<p>can anyone please send me Kinect for Windows SDK Quickstarts my mail is apalash991@gmail.com</p>
<p>can anyone help me by telling how to configure com port for a kinnect?</p>
<p>Hi!<br><br>Can anyone send me the &quot;broken&quot; <a href="http://files.ch9.ms/coding4fun/KinectForWindowsSDKQuickstarts.zip" rel="nofollow">KinectForWindowsSDKQuickstarts.zip</a> file? My email is: adatfalo [at] gmail [dot] com .</p><p>Thanks!</p>
<p>Did you get it?</p>
<p>please also send this file to me at saba.asif84@yahoo.com</p>
<p>Unfortunately not. Pls check you PMs. Thanks</p>
<p><a href="http://files.ch9.ms/coding4fun/KinectForWindowsSDKQuickstarts.zip" rel="nofollow">http://files.ch9.ms/coding4fun/KinectForWindowsSDK...</a></p><p>This download link its broken</p><p>Plz, anyone can upload the code again, or send me a email with the code?</p>
<p>Did you get it?</p>
<p>Ol&aacute;, bom dia!</p><p>Eu n&atilde;o posso fazer o download do link:<br><a href="http://files.ch9.ms/coding4fun/KinectForWindowsSDKQuickstarts.zip" rel="nofollow">http://files.ch9.ms/coding4fun/KinectForWindowsSDKQuickstarts.zip</a></p><p>&Eacute; poss&iacute;vel me enviar por e-mail? </p><p>brunodaldeganlimao@gmail.com </p><p>Obrigado! </p><p>grande trabalho!</p><p></p>
<p>Did you get it?</p>
<p>I need</p><p><a href="http://files.ch9.ms/coding4fun/KinectForWindowsSDKQuickstarts.zip" rel="nofollow">KinectForWindowsSDKQuickstarts.zip</a> file too</p><p>mmmmmms [at] hotmail [dot] de</p>
<p>i need the </p><p><a href="http://files.ch9.ms/coding4fun/KinectForWindowsSDKQuickstarts.zip" rel="nofollow">KinectForWindowsSDKQuickstarts.zip</a> file mmmmmms@hotmail.de</p>
<p>i need the </p><p><a href="http://files.ch9.ms/coding4fun/KinectForWindowsSDKQuickstarts.zip" rel="nofollow">KinectForWindowsSDKQuickstarts.zip</a> file</p>
<p>here book is available to make some basic cool project with arduino + visual basic 6.0</p><p><a href="http://www.amazon.com/Arduino-Visual-Basic-6-0-software/dp/1511667753" rel="nofollow">http://www.amazon.com/Arduino-Visual-Basic-6-0-software/dp/1511667753</a></p>
<p>how can i control servos using raspberry pi....</p>
<p>Hi I am engineering student. I would like to deploy this project could provide me the visual code?</p>
<p>Hopefully, you can send me the Visual Basic code and show me how to let my PC control this remote controlled lawnmower: </p><p>http://www.instructables.com/id/Remote-Controlled-Lawnmower/ </p><p>The above web site has a video of my remote controlled lawnmower. </p><p>Currently, the wheel chair motors are controlled by a Futaba 4YF transmitter. I want to eliminate the Futaba 4YF transmitter. To do this, I want my Windows 7 PC using Visual Basic to control the wheel chair motors with an interface to an Arduino board. The Arduino will send commands to the 2 servos that control each of the 2 wheel chair motors. The Visual Basic program interface needs to allow the following 2 steps: </p><p>Step #1-Learn Mode-Control the mower via the Arduino interface to the wheel chair motors and have the Visual Basic program record the sequence of commands sent to the Arduino. </p><p>Step #2-Playback Mode-Transmit commands recorded on the PC in Step #1 to the mower. </p><p>What is the easiest way to accomplish these 2 steps?</p>
Did u ge the visual basic code? <br><br>Can you give it to me? The link doesn't work.
Sorry, but the approach/steps you outline are way over my capability.<br><br>I'm well versed with the old DOS Basica, but am a novice when it comes to Visual Basic. The remote controlled lawnmower works really well and could have commercial appeal if the 2 servos could be controlled by a PC. Any interest in working with me to let a PC control the lawnmower as previously outlined?
<p>this link is broken): Microsoft Kinect Code4Fun Quickstart VB code (You will overwrite this with my modified version)</p><p><a href="http://files.ch9.ms/coding4fun/KinectForWindowsSDKQuickstarts.zip" rel="nofollow">http://files.ch9.ms/coding4fun/KinectForWindowsSDKQuickstarts.zip</a></p>
<p>will you be so kind if you reupload Microsoft Kinect Code4Fun Quickstart VB code please.? the link is no longer online</p>
Microsoft Kinect Code4Fun Quickstart VB code <br>kevinomaralmendarez@gmail.com
Really thought this project was awesome. I have actually modified the code so that I can operate 4 servos at one time configured as an arm with gripper attachment. I noticed in a comment you posted below about the joint numbers ranging from 1-20 if possible I was wondering if you could list the all the joints that are possible along with the corresponding numbers as I would like to see how my project responds to alternate joint positioning. Again love your work.
Hi, great tutorial but i am having trouble. For some reason the VB does not communicate with the arduino. I only have one com port which is com3 and i have changed it in the coding.
Ola gostaria de saber qual o nome da linguagem que foi utilizado no arduino??
e se teria como explicar melhor sobre os c&oacute;digos tanto no VB como no Arduino??
Techbitar: Would you be able to update the code so it can be used with the Kinect for Windows SDK v1.5 ? I have been trying but the more i &quot;fix&quot; the more i break :P
I promise to so at earliest available opportunity.
hi i've followed all the steps but finally nothing happend the servo does not move ,it seem like the arduino does not recive any thing from the kinect .... can any one help me please ?
You'll need to provide more details. What version of the Kinect SDK are you using?
Quick question; in the arduino code, what does the j and z values represent?
can you post the code segment in question
void loop()<br>{<br>// read the 4-byte byffer sent from VB containing coordinates and joint ID.<br> if (Serial.available() &gt;= 4) {<br> x = Serial.read(); <br> y = Serial.read(); <br> z = Serial.read(); &lt;&lt;&lt;&lt; j = Serial.read(); &lt;&lt;&lt;&lt;<br> if (j == 7) { // HandLeft<br> lservo.write(y); // sets the servo position according to the scaled value <br> delay(10); // waits for the servo to get there <br> }<br><br> if (j == 11) { // HandRight<br> rservo.write(180-y); // sets the servo position according to the scaled value <br> delay(10); // waits for the servo to get there <br> }<br><br> }<br>}<br>
My VB program sends 4 pieces of info to Arduino. X,Y,Z which are the coordinates of tracked body joints relative to the position of the Kinect sensor. J is the joint number. Kinect assigns numbers from 1 to 20 to refer to each one of the joints it tracks. Joint 7 is the left hand and joint 11 is the right hand. These are the only joints I am reading for my paper puppet to move right and left servos. My VB code sends all Kinect joint info to the Arduino. You can change my Arduino code to read other joints to do more elaborate robotics tricks. Hope this helps.
Thanks, I'm planing on making your sick man with both moving arms and legs
It's very doable.
Great tutorial, had some mix-up with the arduino COM I/O but sorted it out changing arduino port to match the program(16).<br> &nbsp;In an instance I also managed to modify the arduino code so that i can control one hand in x and y axis (just typing x instead of 180-y and matching the j-value to 11). THANK YOU!!
Thanks for the feedback and glad it worked for you.

About This Instructable


113 favorites


Bio: Did I unplug the solder iron?
More by techbitar: IR Remote Control Station for Android - TURN THE TV DOWN SensoDuino: Turn Your Android Phone into a Wireless Sensors Hub for Arduino Modify The HC-05 Bluetooth Module Defaults Using AT Commands
Add instructable to: