HandTalk 2 : a Smart Hand Glove Interpreter

This is the modified version of my first project Handtalk.HANDTALK 2 needs to be worn on the hand by the needy and depending on the variation of the movements the device will convert it intelligently into beep (can be voice too) and in a text. This message will be out from the beeper and LCD display. The HANDTALK glove senses the movements through the flex sensors which detects the different patterns of motion. The device can sense carefully each resistance and each movement by the hand. Currently, the device can convert only a few, but depending on the success of this device few more additional features may be added later onto this expressive system.

Technology has always been of great help to the disabled and given them a helping hand to allow them to live a normal and healthy life like others.

Embedded systems are designed to do some specific task, rather than be a general-purpose computer for multiple tasks. “Embedded” reflects the fact that they are an integral part of the system. Some also have real-time performance constraints that must be met, for reason such as safety and usability; others may have low or no performance requirements, allowing the system hardware to be simplified to reduce costs. An embedded system is not always a separate block- very often it is physically built-in to the device it is controlling.

Today, embedded systems are found in cell phones, digital cameras, camcorders, portable video games, calculators, and personal digital assistants, microwave ovens, answering machines, home security systems, washing machines, lighting systems, fax machines, copiers, printers etc.

The system is a way of working, organizing or performing one or many tasks according to a fixed set of rules, program or plan. In other words, an arrangement in which all units assemble and work together according to a program or plan. An embedded system is a system that has software embedded into hardware, which makes a system dedicated for an application (s) or the specific part of an application or product or part of a larger system. It processes a fixed set of pre-programmed instructions to control electromechanical equipment which may be part of an even larger system.

CHARACTERISTICS

a) Embedded systems are application specific & single functioned, the programs are executed repeatedly.

b) Efficiency is of paramount importance for embedded systems. They are optimized for energy, code size, execution time, weight & dimensions, and cost.

c) Embedded systems are typically designed to meet real time constraints; a real time system reacts to stimuli from the controlled object/ operator within the time interval dictated by the environment. For real time systems, right answers arriving too late (or even too early) are wrong.

d) Embedded systems often interact (sense, manipulate & communicate) with external world through sensors and actuators and hence are typically reactive systems; a reactive system is in continual interaction with the environment and executes at a pace determined by that environment.

e) They generally have minimal or no user interface.

Step 1: Components Required

Hardware Requirements:

Arduino Uno/Nano Flex Sensors (2.2 inch) - 4/5

Pulse sensor

GSM Module SIM 900A

Beeper

Resistors - 50 K Ohm,10 K Ohm

LCD Display Glove Capacitors

Connecting Wires

RF Module

Software Requirements : Arduino IDE

Hand Glove -1

Gluegun

Software - Arduino IDE

Step 2: WORKING AND METHODOLOGY

We are working on this project namely Hand talk 2 that translates hand gestures into a particular message. Flex Sensors fixed with the glove pick up the gesture made by the individual and then with the help of the Arduino that analog input is converted to digital for various gestures. For every particular gesture, there is specific digital output and that digital output for the specific gesture is reserved for a specific message. So, when the individual does that particular gesture, the predefined message for that gesture is displayed on the LCD along with a beep (sound) and that specific message is also transmitted in the form of text or call with the help of GSM module to the various registered numbers.

For instance,

If the patient perform a gesture of fist (most often we people form the fist in case of fear, emergency), there would be a message displayed on the LCD: “CRITICAL SITUATION” or “EMERGENCY” along with a beep and that particular message would also be transmitted in the form of text or call to the Doctor’s phone or the Relative’s one.

Not only can this but the particular needy person also use this glove to control the lights, fan, AC or any other devices, yes! We are talking about home automation by gestures i.e. controlling of devices smartly by various gestures.

For instance,

If that particular person performs a gesture of “V” (victory symbol), then the light will be ON, and by performing the same gesture again the light will be OFF. Similarly, we can do the same in the case of AC, Fan or any other devices.

So, basically, we added this feature to make the differently abled to somewhat “completely abled” at least in their own perspective, as they would be able to perform the basic actions without others help.

In addition to this, we have also added the Heartbeat sensor to sense the heartbeat of the person and displaying it on the LCD, so that the nearby person can observe the heart rate easily. In case, the heartbeat is getting too low or too high, the doctor will get the alert too by an SMS and call.

A person not knowing the Sign language and gesture can listen and read via the cell phone or on LCD what the other person is saying in Sign language form or in the gesture.

The main advantage of this design was its simplicity and the cheap components and this project can help to improve greatly the communication barrier between differently abled and the normal one.

Block Diagram of the device is also attached .

Step 3: Circuit and Operation

The above circuit diagram shows the Transmitter section and Receiver section of the prototype.

As per the circuit diagram of project, there are two parts:

1) Transmitter Section

2) Receiver Section

Transmitter Section:

This section is the main part of this project. It consists hand glove (fitted with Flex sensors and Heartbeat module) , GSM module, RF Transmitter and Arduino Uno as a Controller. All the gestures made by the hand glove (with Flex sensor ) is converted into different messages and commands for the operation of different devices.Example , if I want to start a fan then a command is sent to the receiver end by the microcontroller with the help of RF module and the fan is easily controlled by the corresponding gestures. Now in case, I need water (or wheelchair or anything else), all I need to do is to make a corresponding gesture accordingly and that gesture would be converted into the text message and sent to the other person through the GSM module. Now using the Android app we can convert the text messages into voice also.

The Heartbeat module is used for the Heart rate monitoring of the patient.If there is any problem in heart rate then a message is also sent to the doctor or family members also.

In this project currently we are using seven gestures due to the problem of the accuracy but we can add about 100 plus gestures. Some examples of gestures are in shown in images.

Receiver Section:

This section is only for the display the message or the operation of different devices through the gesture. RF Receiver receives the command and this command is catch by the Arduino and Arduino controls the actions of this sections. In this part, we are using fewer devices but you can also interface more devices. In this section LED bulb, motor and beeper are used for showing different action by the different gestures.

LCD (16x2) is used for the displaying the messages or requirement of the patients at different places.We can also place this section in the doctor's chambers also.

Step 4: Setting Up of Flex Sensors

Interface all the sensors or module as given in the previous step (Circuit Diagram)

For the power Supply, you can use 5V adapter for both circuits.

Flex Sensors are in confrigution of variable resistors.

Step 5: Programming

Find Attached Code and Run in Arduino IDE.This code is written in C.

Step 6: Conclusion and Future Work

Future enhancements:

Ø With the help of different gestures commands, different other commands can be added.

Ø For more reliable and low complexity of the circuit microcontroller can be replaced by the Arduino or other Advanced Microcontrollers.

Ø In this project many types of other applications can be added with using the different type of sensors like Heartbeat sensors for heartbeat monitoring and Temperature sensor for body temperature monitoring.

Ø Using the Xbee Module we can create a wireless zone and this device may communicate with the other devices also.

Ø Using the IOT technology we can connect this for live updates of the patient or person like locations, status body conditions etc.

Conclusion:

Ø Sign language is a useful tool to ease the communication between the deaf and the mute community and the normal people. Yet there is a communication barrier between these communities with normal people.

Ø This project is useful for differently abled, speech-impaired and paralyzed patients who cannot speak properly.

Ø This work is done to check the feasibility of recognizing sign language using flex sensor and accelerometer gloves and displaying the data, which proved to be an efficient system.

Ø The main feature of this research work is that the gesture recognizer is a standalone system applicable in daily life and for biomedical purposes.

Thank You,

For any query, you can contact me

[e-mail]

LinkedIn- www.linkedin.com/in/sachin0987