First of all, this project is not finished. We're still trying to improve and fix some problems. Right now, we're only able to translate letters from "A" to "E". The program still have problems while identificating those letters, the values for each of them match sometimes, so we're still trying to fix this.
Many people around the world suffer from speech problems, and can't communicate with others in the same way we do it. To solve that problem, I, Bruno Moraes, and my research partner, Pedro Jorge, developed a Sign Language Translation Glove. Acctually, it only works with Brazillian Sign Language.
The Glove is also able to control a prototype LEGO Wheelchair wirelessly with the movement of your fingers.
For this project, we made use of flexible potentiometers (available at sparkfun) to measure each finger's position. This potentiometer, together with a 10K resistor, form a Voltage Divider that feeds de Arduino ADC. We used an Arduino Mega 2560, but UNO or similars will work just fine.
Bill of Materials:
1 - Glove ( We used a Nike Dry Fit Tour Golf )
1 - Arduino Mega 2560 (http://www.sparkfun.com/products/9949)
1 Arduino UNO ( Optional, used for the wheelchair side control)
5 - Flexible potentiometers (http://www.sparkfun.com/products/8606)
5 - 10K Resistors
2 - 2.2K Resistors (Optional, used for the wheelchair side control)
2 - 2N2222 Transistors (Optional, used for the wheelchair side control)
1 - Breadboard
2 - XBee Series 1 Modules
2 - XBee Explorer Regulated
1 - Soldering Iron
1 - Solder
1 - Stranded wire
Step 1: Attach the Sensors
Sew the five sensors to the glove. Make sure they're all fixed, but not too tight. The sensors should follow finger movement as much as possible.
After, solder 15cm stranded wire pieces to each of the sensor leads.
Remember to reinforce de pin-end of the sensor, this area is susceptible to kinking and eventual failure. Electrical tape ou heatshrink is fine.
Step 2: Voltage Divider Circuit
To interface the sensors with Arduino, we used a Voltage Divider Circuit ( more on http://www.sparkfun.com/tutorials/270).
The sensor, when it's straight, has a resistence of 30K. When bent 90º, 50K. If we just conected the sensor pinends to VCC and to an analog input, the output voltage woudn't be enough to feed the Arduino's ADC. To solve this, we used a Voltage Divider Circuit. With this circuit the higher resistence, higher is the ouput voltage.
The ouput voltage VOUT is given by the expression in picture 3.
See picture 4. R1 is our sensor, and R2 is a 10K resistor. Using 5V from Arduino, when the sensor is straight, VOUT will be 3,75V, and when the sensor is bent 90º, VOUT will be 4,17V.
The Arduino Analog/Digital Converter will assign a number for the input voltage. As it has 10-bit resolution, this number will be between 0 and 1023 (2^10). Actually, as the Voltage Divider output is inversely proportional to the resistance, this ADC output is somewhere between 780~805 when the sensor is straight.
Connect the respective VOUT's to Arduino's Analog Inputs, starting from the thumb, Analog input 5 to 1.
Step 3: Programming
During the research phase of this project, we soon noticed that there are considerable variations between each person's way of making the sign language signs. The software for our glove should, then, be able to adapt for such differences. Our solution was to implement an associative memory. With the associative memory, the user can "train" the glove to recognize the signs. The training is made by making the sign with the glove and telling the software which letter it represents. The software then stores this in a database and, every time the user makes a sign, it looks through the database for the best match. Using this solution, ou prototype is easily adaptable to each particular individual and to different sign languages.
UPDATE: As soon as we fix some bugs, we'll be posting the Arduino Code here.
Step 4: Results
The system actually ended up being pretty accurate, and the associative memory turned out to be a great solution.
Again, we're still working on the system, and we need your help. Anything you think would be a good add-on, improvement or fix.
Step 5: Extension A
Controlling a LEGO Wheelchair.
As an extension to the project, we made a little demo controlling a LEGO Wheelchair, as shown above.
The system work by controlling a 2N2222 transistor and hacking the Mindsotrms Cable, so, our circuit appears to the NXT as a touch sensor. By opening and closing the transistor gate, we're able to input simple data to NXT.
As you can see in the images about, the touch sensor schematic ( image 3 ) is just a SPDT switch. We put a 2N2222 transistor in this place, enabling the Arduino to control the state of the "Touch Sensor".
The circuit is simple ( image 4), so I'll not go on details about assembling.
We made two of them, simulating two sensors. One enables and disables the chair movement, and the other one tells in wich direction the chair must go.
By contraction the fingers 3, 4 and 5, you enable the movement. By moviment finger 2, you control the way the chair moves.
The Arduino code is the following:
[coming soon, we're fixing some bugs]