Introduction: Robotic Hand Controlled by Gesture With Arduino + Leap Motion

Picture of Robotic Hand Controlled by Gesture With Arduino + Leap Motion

Hello everyone,

As promised I have made a full step by step instruction for my university final project!

First of all: sorry for my bad english! If something isn't clear, please leave a comment.

The hand is controlled by Leap Motion + Arduino Uno (Standard Firmata, Nodejs & Johnny Five) + Servo Motors and the hand itself was made with Polymorph. It can recognize each finger and it can move them separately with very good precision and sensibility! This prototype can be controlled remotly too via a normal web server using ajax/json requests. Let me know what you think about it and maybe I will publish a complete tutorial when I will have more time... Thank you!

--- Feedback from NASA ----------------------------------------------------

“ Your hand is very responsive and does a good job of showing how well your gesture control/recognition works. ”

Ron Diftler, Ph.D. Robonaut Project Manager, NASA/Johnson Space Center -------------------------------------------------------------------------------------

Step 1: Parts

Picture of Parts

Parts: (Cost will be about 300$)

  1. x1 Arduino UNO
  2. x1 Leap Motion
  3. 0.5kg of Polymorph
  4. About 3m Fishing Line (0.17mm - Spiderwire)
  5. Some elastic strings
  6. x5 Servos (TowerPro MG956R)
  7. x1 Battery 1600mah and 6V (NiMH Venom VEN-1504)
  8. x1 Small Breadboard
  9. x1 Hot gun
  10. Some wires and tools

Step 2: Make the Hand!

Picture of Make the Hand!
  1. Put all polymorph spheres inside a basin with hot water
  2. Mix up all the spheres in a unique piece of polymorph
  3. Divide the big piece of polymorph in some smaller pieces
  4. In order to make each fingers you have to draw a scheme using your hand (see picture)
  5. Take a piece of polymorph and put it in a microwave oven for like 30s @ max power
  6. It will become transparent and easy to model. Be careful because it is very very HOT
  7. Use hand sketch to adjust the size of the finger and model the piece of polymorph, using some pilers to make all the joints. Joints are the tricky part, they should blend just in one direction!
  8. Make all five fingers and let them cold down
  9. Now you have to make the palm of the hand: put a bigger piece of polymorph in the microwave oven
  10. Model the piece in order to obtain something similar to a hand palm: before it cold down you have to put all fingers inside the warm palm. Be carefull to put all the fingers in the right position because once everything is cold, it would be impossibile to fix possible problems!
  11. Now it's time for some wire loops. We need to make some wire loops on all fingers (on the front, for the fishing line and on the back, for the elastic line) in order to avoid wire intertwine during fingers movements. On each finger you have to make a hole with a hot screwdriver, in between of all joints and on all finger tips (so 4 holes per finger, see pictures)
  12. Make some little balls of polymorph and put them in the microwave oven, press one small ball of hot polymorph inside a hole and model it using a toothpick to make a hole throught it (see picture). Fishing line and the elastic one will pass through all these wire loops!

  13. Make some wire loops on the hand palm too
  14. Finally we have to make the arm part, that will host all servo motors. Take a A4 white paper and draw all servo motors shapes (See picture).
  15. Put a big piece of polymorph in the microwave oven. When it's hot, model it on the drawn schema and use a cutter to make all holes for the servo motors.
  16. When the arm is cold you will have to put all servo motors inside the holes and seal them using some hot polymorph strips. Be carefull to put all servo motors in the right orientation (See picture), two of them will be inverted.
  17. You have to attach the arm to the hand by using a hot gun and some extra hot polymorph
  18. When the hand and the arm are fused together the last step consist of connecting all servo motors with each fingers: you can fuse an extremity of a piece of fishing line at the tip of the finger by using a common lighter whille the other extremity can be simply tied on the servo motor (see picture
  19. It's raccomended to create some wire loops to put on each servos in order to avoid wire interwine while servos will move.
  20. Finally you can put some elastic line on the back of each finger (see picture), make sure that all elastic lines are well strained: this part will give your hand the ability to keep the relaxed position even when the hand is oriented with the palm looking down!

Now you have finished the mechanical part of the project and you are ready to give life to your artificial hand :D

Step 3: Attaching All Servos to the Arduino

Picture of Attaching All Servos to the Arduino

Now that we have our hand and all servos implanted on the arm we must use the little breadboard in order to connect all servos to the Arduino Uno and to the battery pack:

  1. Each servo come with 3 wires, the black one for the ground, the red one for the power supply and the yellow one for the PWM (important!) signal
  2. By following the schema in the picture, you should connect all servos wires in the breadboard
  3. Now it's time to use some wires to connect all grounds and all power supplies (just follow the schema)

    Attention! I didn't pay attention to use right colored wires (because I didn't have all required colors!)

  4. Take a common ground and connect it to the Arduino Uno ground input
  5. Take another common ground and connect it to ground contact of the battery pack
  6. Attention! Don't connect the common power supply to 5V input of the Arduino Uno (as showed in the schema), you have to connect this to the battery pack!
  7. Finally you have to connect all servos yellow wires to a PWM digital input on the Arduino Uno. I have used ports 3, 5, 6, 9 and 10. Attention! Try using these ports because when we will come up with the code, I will assign servos variables to those digital ports!

Now you have the hand, all servos implanted, connected to the Arduino Uno and the battery back.

Step 4: Preparing the Developing Platform and Leap Motion

Now it's time to prepare software platform in order to program the arduino board. Since Leap Motion sends its data throught a Web Socket, we will use Google Node.js as javascript platform (it's based on Google Chrome js engine) so that we can easily read data via the Web Socket. Then, we should install a Node.js module called "johnny-five" that allow us to program arduino board directly with javascript.

NOTE: I did everything with a Macbook Pro so please don't ask me how to solve problems on Windows systems because I can't really help you! Most of problems can be solved by searching on google.

  1. Plug in Leap Motion and install its software. At the end you should be able to see an icon somewhere, that allows to open the AirSpace software and Leap Motion configurations.
  2. Inside Leap Motion configuration you should set "Precision" and make the "Recalibration" tutorial. Please note: Leap Motion surface should be very clean!
  3. Now that Leap Motion is connected, installed and working we have to install Node.js so go to and click on the "Install" button (it's possible that first you need to download and install XCode in order to install nodejs, I already have it installed so I can't be sure). You can check Node.js successful installation by going on Terminal and type "node" and press "enter". If nodejs console show up starting with a ">" then you are ready to go. If you have some problems with the installation, please search on google some tutorial because there are a tons!
  4. Now you have to install some nodejs modules (using npm install syntax). Again, check on google how to install a module it's very easy btw! Needed modules are the following:
    1. gl-matrix: matrix and vector library
    2. johnny-five: arduino library
    3. underscore: javascript support library
    4. ws: websocket library
  5. Install LeapJs SDK for Node.js check this url:

  6. I have installed everything inside a project folder called "Tesi". Inside this folder I have 3 folders:
    1. "lib" containing the "lib" folder of LeapJS: check on github
    2. "node_modules" containing all 4 modules folders
    3. "script" containing my script

Please, try to keep this project folders structure and paths because unpredictable errors may happen if not.

Step 5: Algorithm and Scripts

Picture of Algorithm and Scripts

This is the last step and the most important one. Now that we have our working artificial hand connected to all servos controlled by Arduino Uno and the developing platform is ready-to-use... We really have to make the software in order to:

  1. Retrieve Leap Motion data
  2. Elaborate the data to assign detected objects by Leap Motion to appropriates fingers (servos)
  3. Elaborate the data to obtain appropriates servos angles
  4. Send elaborated data to Arduino Uno in order to move servo motors

Attached to this step there is my full project folder. The script that you should run in Node.js is "/script/app.js" (make sure that leap motion and arduino are connected to your computer, the arduino should be connected to the battery pack and all servos should be attached to the Arduino board too).

Just for a matter of order I have put all core functions inside another file inside "/lib/handToHand.js" that is imported at the very beginning of the "app.js".

I have rewritten all my comments in english so that anyone can understand what's happening and why. If you follow my comments and check the flowchart (see picture) you will understande everything for sure.

If you have any inquiries please leave me a comment!

That's all... Thank you and good luck!

Alessio Sacchetta Cali
Electronic Engineer
University of Roma Tre


sjccrac made it! (author)2017-12-11

Excellent work by @acali! Thank you for providing such a well documented project! I'm happy to provide guidance to anyone that is working on this project and is stuck.

philsau (author)2017-10-18


I tried to run your script and I have an error message :

"C:\Users\psaur\Desktop\leap motion\projet_main_instructable\script>node app.js

It looks like serialport didn't compile properly. This is a common problem and its fix is well documented here

C:\Users\psaur\Desktop\leap motion\projet_main_instructable\node_modules\johnny-five\node_modules\firmata\lib\firmata.js:28

throw "Missing serialport dependency";


Missing serialport dependency"

Could you help me to solve this problem?

I tried to change the folder but still not running!

Thanks a lot.


sjccrac (author)philsau2017-12-11

Did you ever get this running? If not let me know some more information about your setup. i.e. OS, etc. I had originally tried this on windows 10 and it didn't work. I tried again on a mac os and it ended up working. Also you have to make sure that you have the standardfirmata code running on your arduino. Let me know if you have any questions. I'm happy to help if I can.

deleonalexis (author)2017-02-02

Has anyone tried to run this code recently? Just trying to see if anyone can verify that the code still works before trying to interface the leap motion.

sjccrac (author)deleonalexis2017-12-11

I just got it up and running two days ago and it works. You have to make sure the standardfirmata code is running on your arduino. I also took the handtohand code and pasted it into the app.js file. Please let me know if you have any questions and I'll help if I can.

acali (author)2016-10-19

Hello everyone,

Source code was provided as well as you can see in the attached screenshot to this comment. Please consider that the code was working with previous versions of leapjs, nodejs and all the modules!

deleonalexis (author)acali2017-02-02

I have almost everything to get the project started besides the leap motion. Before buying it I just wanted to know if anyone could verify that the program still works with the updated versions of leapjs and modules.


Nettrice (author)2016-12-30

Hi. Where did you store your "Tesi" folder? Mine is in My Users folder and I keep getting this error: "SyntaxError: Invalid regular expression flags"

Thanks in advance!

PatojrC made it! (author)2016-12-06

Hello friend, I write from Ecuador the project is great, but I have the following problem, can you help me? Please

SüleymanK4 (author)2016-11-14

broken link :/ can u send me the details and code

Ilavic (author)2016-10-16

Amazing project , can u please share the code

plz send to

BarbaraM162 (author)2016-10-07

Sir is it possible to give me the code of this project.

Please send me at

Immadfareed (author)2016-09-22

I want to make same can u give me guide how i start making coding

Immadfareed (author)2016-09-22

Its great can u shear the code....

Gyana RanjanB1 (author)2016-09-01

Sir is it possible to give me the code of this project. That will help me a lot

jassem9778 (author)2016-07-26

amazing project sir. can i get the coding for this so that i can understand and learn


chathurangakila (author)2016-07-22

Amazing..sir can i get the code of this.It will be very useful for my project sir...Thank you.. sir..

nomans17 (author)2016-07-11

Hi Alessio!

Can i use leap motion controller to send it's data to arduino wifi shield.

Actually i want to use leap motion to make a robotic hand, which can be operated from another room.Is it possible?

Thanks in advance.

isra.hokage (author)2016-06-18

sir can please share the code of the project.

AungM12 (author)2016-06-09

sir could u please share the code of the project?

bharatk28 (author)2016-05-23

sir can please share the code of the project.

AKalins (author)2016-02-11

Hello Acali,

Could you please share the coding of this

GeepikaG (author)2016-02-08

sir olz help me make this project.. mca final year student

YiXiangZ (author)2016-01-25

Sweet Project. Could you send me the code for this project? Thanks in advance.

ouli0826 (author)2016-01-07

Hi sir, Can I please have the code for this amazing integration?

thank you!

MX26 (author)2015-10-27

can you share the code from the Arduino how the sevor work? thank you

am a sjus student, who working on a final project i have create the hand i need help code it and i would like to see your code see what i did wrong.

email it at

thank you


yatishj (author)2015-08-27

Can you give me the code for the project???

GabrielA7 (author)2015-05-12


CraigA3 (author)2015-05-08

Would anyone be able to help me figure out how to reverse the direction of the servos if the finger moves in the opposite direction of the servo? I got everything else functioning

quinnmiller1997 (author)2015-04-17

Hi! Thank you so much! This tutorial is great!!! I'm just having one problem. I get the program to run and the leap motion and arduino connected, but I get this error after the my hand is calibrated. It is saying that the serial port is not open, but It connects to the arduino and leap motion at the beginning of the program. I'm stumped :/

raxidz (author)2015-02-15

Dear Alessio, i am working on a similar project after seeing it i am very impressed.
i need to start from the basics can you Please help me in the code ?
this is a very basic code that makes an Led glow when a finger OR object is brought into the range of LEAP. the problem is , that i want to make it glow when it detects a specific finger like index finger or thumb etc , i dont want it to glow with any object which in case of using "pointables" does.
so what substitute is there to use instead of pointables in the following code. thank you very much i am so confused here because i have no knowledge of the coding here. thanks

var webSocket = require('ws'),

ws = new webSocket('ws://'),

five = require('johnny-five'),

board = new five.Board({port : '\\\\.\\COM3'}),

led, frame;

board.on('ready', function() {

led = new five.Led(12);

ws.on('message', function(data, flags) {

frame = JSON.parse(data);

if (frame.pointables.length > 0 )




tommylee.neville (author)raxidz2015-02-17

Hey, this isn't going to be your answer sorry but that seems really cool! Do you think you could add me on Skype (Bacon_lover314) or somehow send me the finished product of the glow I would love to do the same :)

tommylee.neville (author)2014-12-22

Where did you get your Servos everywhere I look there $10 each, is that the cheapest I'll get them? I would really like to get some a bit cheaper :/

rohanbansal (author)2014-11-07

It works now. Good!

rohanbansal (author)2014-11-05

Everything in the computer now works fine except for the servos. they do not seem to move an inch!

chuyo09 (author)2014-10-30

Thank you very much, I fix the problem but now when I want to open the app the terminal show me the following error.

acali (author)chuyo092014-10-30

Mmm.. Try to put you open hand above the leap before starting the script. If it gives error again try multiple times. If the error persist maybe you are using updated versions of leapjs, node or modules!

chuyo09 (author)acali2014-10-30

i'm using node v0.10.33 modules, the same as you, the leapjs, i don't have idea, but i'm going to put my open hand,

thank you so much!

chuyo09 (author)chuyo092014-10-31

NOW WORKS ;D but the interface you're using , you open it in your browser or you saved in your computer?

rohanbansal (author)2014-10-28

I'm really sorry to bother but I'm just not getting it. What do I do after I have opened GitBash? What commands do I give? And which is the app.js folder?

acali (author)rohanbansal2014-10-30

Open the console and go to the app.js folder "cd path/to/app.js" and then run "node app.js". The file app.js is inside the provided package so just unarchive it somewhere!

chuyo09 (author)2014-10-30

Hi i have this error, can you help me?

acali (author)chuyo092014-10-30

This is a path problem, double check filename and extension too

rohanbansal (author)2014-10-26

I'm still facing a little problem.....after I installed all modules etc, how do I run app.js. What commands am I supposed to give in node.js to run the script. Please provide me with the finest of details as I have invested a lot into this project.

Thanks a lot!

acali (author)rohanbansal2014-10-26

Hello, to run the script just open the terminal, go to app.js folder and tpye "node app.js". That's all!

rohanbansal (author)acali2014-10-26

You have a really good support. If my project works, all credit would be to your name

rohanbansal (author)2014-10-17

I don't get a few things: where's the Arduino uno code?

How exactly do I install leap.js?

Sorry, I might have a few questions as I am planning to make one myself.


acali (author)rohanbansal2014-10-17

There isn't an arduino code since everything is made in javascript
thanks to JohnnyFive library, on the arduino you just have to load the
StandardFirmata, that's all!

To install LeapJS you should run the proper line of code for Nodejs:

npm install leapjs -v0.6.3

the provided url for more information and please note that the script
is based on old versions of node leapjs and johnnyFive so you could
catch some problems too..!

rohanbansal (author)acali2014-10-18

I just need a how exactly do I run the code. Do I run it on Git Bash or Node.js?

I just don't get the script part. How do I run and install leapjs??

Thanks a lot for help

rohanbansal (author)2014-10-17

I can't fing the "node_modules" folder......neither can I find Tesi. Please help me by providing the the links to all four modules.

Thanks a lot

Help is appreciated

About This Instructable




Bio: Hello! My name is Alessio Sacchetta Cali and I'm an electronic engineer specialized in biomedical devices. I'm also a web developer, I worked ... More »
More by acali:Robotic Hand controlled by Gesture with Arduino + Leap MotionRobotic Hand controlled by Gesture with Arduino + Leap Motion
Add instructable to: