Wireless Robotic Hand Controlled by Gesture and Voice

Basically this was our college project and due to lack of time to submit this project we forgot to take pictures of some steps. We also designed a code using which one can control this robotic hand using gesture and voice at a same time but due to lack of time we were not able to implement it as then we had to make changes in whole circuit and We also designed a code using which you can control this project by using voice only using just Arduino board only then you will not need EasyVR shield which is costly so it is cheap method to make voice control robot . Code is attached below. The aim of this project is to design a mechanical hand and a control glove (for human hand). Mechanical hand will act as a slave by imitating the action of control glove (human hand) which will act as a master. Mechanical hand is designed using servo motors, controller and xbee module. Control glove will be designed using flex sensors, controller and xbee module. The control glove is mounted with flex sensors. The controller reads the voltage change when the ex sensors are bent and send data wirelessly using xbee module to mechanical hand, which will triggers the servos to move according to the gestures performed by hand. This device can help to minimise the requirement of physical presence of a human being in places where human intervention is least required ( Eg : chemical industries, bomb diffusing etc ).

If you find this instructable useful then please vote for me . :)

Step 1: Block Diagram

Step 2: Materials Required

components images in this section are taken from amazon.in and link for those components are provided below:

1. Flex Sensor (x6) - ( https://www.amazon.in/Linwire-Robodo-Flex-Sensor-...

2. Wires for connection

3. Vero Board - (https://www.amazon.in/AnandCircuits-PCB-FR2-Board-...)

4. 10 K ohm Resistor ( x5 )

5. XBEE Module Series 1 ( x2 )

6. XBEE Arduino Shield ( x2 ) - (https://www.amazon.in/Bluetooth-Shield-Wireless-Co...)

7. Arduino Uno board ( x2 ) - (https://www.amazon.in/Bluetooth-Shield-Wireless-Co...)

8. Socks

9. cloth to cover Arm

10. Cardboard

11. servo motors - 5 sg90 servo (fingers) , 1 Mg996r servo (arm) - (https://www.amazon.in/Robodo-Electronics-Tower-Mic...)

12. zip straps - (https://www.amazon.in/4STRON-Antistatic-Heavy-200m...)

13. fishing thread

14. socks

Step 3: Designing of Mechanical Hand

Designing steps

For parts you will need some zip locks, a thread reel (fishing line), 5 ink tubes, 5 servo motors, and an arduino. For structuring of fingers hot glue gun and glue sticks are used.

Step 1: On a piece of paper, mark the distance between each of your fingers as shown in and assign each of them a reference number so you can determine which finger is which later on. Copy each number onto the corresponding ziplock.

Step 2: Take the respective measurements of the joints of each fingers including the pinky finger as per the ideal human hand measurements.

Step 3: Mark the readings on the ziplocks and leave space of 2cm each for bending purpose. Make a dierent markings on the place which needs to be folded, ideally it is 1cm approximately from both the joints.

Step 4: Fold the zip locks at the marked points and press firmly down to ensure that it is a tight fold with help of wood block or other heavy material. Repeat this process for the rest of the zip locks.

Step 5: Now cut ink tubes in 2cm length and position them each on the joint using hot glue. This distance allows the fingers to bend freely.

Step 6: Wrap each of the sections with thread to add shape and volume to the fingers. This will also work to reinforce the structure of each of the digits.

Step 7: Now according to the previous steps make the remaining fingers. Avoid use of excessive hot glue as it adds weight and leads to problem of bending of ngers. Leave the ngers to dry for a minute, we can remove excess hot glue by use of sand papers and relatively we can shape it as per human fingers.

Step 8: Now for the palm, place each finger excluding thumb at a distance of 2 cm so that the fingers get equal individuality to bend and come back to its original position when servo come at 0 position.

Step 9: For the fingers to have equal space place a piece of hot glue stick each of 3cm between each fingers by use of hot glue. For the fixing of ngers tie all the ngers including the hot glue sticks attached between each finger with a thread and apply a thin layer of hot glue for finished effect.

Step 10: Now join the thumb at an angle to the hand from behind , as the fishing line would pull the thumb accordingly it will make a grip same as human hand. Cut the excess ziplock of the thumb as it adds blockage to the bending path.

Step 11: Add the shing line in each finger through the ink tubes. At the top end of the finger tie a knot of the shing line so that it stays in position.

Step 12: The final step is attach each fingers shing line to respective servo motor at its maximum position. Make sure when the servo rotates there is enough fishing line left for fingers to bend. Fix the fishing line to shaft of servo motor by tying knots tightly.

Step 4: Designing of Control Glove

Setting up sensor circuit

The resistors in the used are 10K ohms. The main GND wire, which is connected to all the individual GND wires from the sensors, gets plugged into the arduino's GND. The +5V from the arduino goes to the positive voltage wire, and each blue wire gets plugged into a separate analog input pin. Then we soldered the circuit onto a small veroboard. One that could be easily mounted onto the glove. We were able to solder the wires to the sensors relatively easily also, and used heat shrink to make sure there were no shorts. We then wrapped the area where the wires are connected to the sensors with electrical tape to stabilize the sensors. Near the bottom, where the leads are attached, the sensors are a bit weaker and the tape ensures that they won't bend and won't get damaged.

Sew each sensor tip to the area of each finger just above where each of your fingernails would be. Then, for each sensor, make loose loops around them with thread at both joints in each finger. Once each sensor is in place and slides under the loops of thread nicely. Then we sewed the veroboard onto the wrist part of the glove tightly. After sewing flex sensors the glove and also on the elbow support connect all the wires to the veroboard and arduino uno board. Fix the connections on a cardboard sheet.

Step 5: Software

Steps for establishing wireless communication between two xbees are as follows:

- Download Arduino IDE software and install it on your system.

- Now open arduino software window. This how it will look like.

- Verify this code using compile ( tick mark symbol below le option) option to check for errors and then save it.

- After compiling it select the port ( in Tools option) on which board you want to upload the code.

- Upload this code on arduino UNO board R3. we are uploading this to make arduino a USB port for xbee s1 module conguration.

- Now mount adruino xbee shield on arduino board and then mount xbee module s1 on arduino xbee shield. After this setup switch to USB mode using slide button on arduino xbee shield. Then connect both arduino to laptop usb port using USB and download the X-CTU software.

- Now open the X-CTU window.

- Now click on Add devices or Discover devices to add radio modules to the list.

- After adding devices click on them to change their conguration. NOTE: PAN ID should be same for both coordinator and end point module otherwise they will not communicate. When you assign same PAN ID for both the modules then they can detect each other for communication.

- Now click on Add devices or Discover devices to add radio modules to the list.

- After adding devices click on them to change their conguration. NOTE: PAN ID should be same for both coordinator and end point module otherwise they will not communicate. When you assign same PAN ID for both the modules then they can detect each other for communication.

- Now switch to consoles working mode and click on open the serial connection with the radio module for both devices. After selecting it will appear green in colour.

- Now type any message in coordinator box for example type hi, hello etc. Box in which you are typing text will appear blue.

- Now click on end point box you will see the same message in this box but in red colour which indicate message is received from other device. After successful conguration and communication between coordinator and end point modules they are ready to be used in circuit.

- Steps for final coding wireless communication of transmitter and receiver { Write the code for control glove in arduino software and select the port (here for eg : COM4 is the port for trasnmitter block) which you want to make transmitter block (coordinator). Now upload it on that Arduino UNO board. NOTE: When you upload any code on arduino disconnect (remove) the arduino xbee shield or any connection on it.

{Write the code for mechanical hand in arduino software and select the port (here for eg : COM5 is the port for receiver block) which you want to make receiver block (end point). Now upload it on that arduino UNO board.

Step 6: Flowchart

Step 7: CODE:

We also designed a code using which you can control robotic hand by gesture and voice at a same time but it will need lot of explanation on how to embed it in the project that is why we didn't attached it here. If any body need that code comment below your email id. In this video laptop is used just to supply voltage to both the arduinos as we had battery issues as they drain very fast.