Introduction: Alternative Communication Vest (CoCoA)
The CoCoA Project is a wearable vest connected to the internet that provide tactile symbols of alternative communication to assist people with speech or non-verbal disabilities. The acronym CoCoa comes from the abreviation of the portuguese name: Colete de Comunicação Assistiva.
Speech impairments affect people with autism, aphasia, cerebral palsy, partial or total deafness, among other disorders. In the case of autism, non-verbalization or speech delay is often present. Lack of speech can also be a temporary disability due to some illness.
Impairment of speech, permanent or temporary, needs attention and treatment, because this ability is important for socialization. Alo, speech is a major skill for individuals with another condition or disability express their needs.
In this context, Alternative Communication (AC) assistive resources are used to help speech-impaired people communicate with their families, therapists, and others. Pictograms are one of the most common forms of AC, since an individual can point to symbols fixed at strategic points in the house or use applications that vocalize these symbols, such as Proloquo2Go.
However, physical pictograms can only be used to communicate with others in the same environment, which makes it difficult for an individual to communicate with their caregivers if they have an urgent need and they are in other environments.
The Alternative Communication Vest (CoCoA) is an internet-connected weareable vest that enables the coupling of alternate communication tactile symbols to assist people with speech or non-verbal disabilities to express their needs to caregivers, therapists, teachers and other health professionals, without them always being present in the same environment as the person with disabilities, thus making monitoring easier.
CoCoA will allow the user to include up to six AC symbols allowing the user to perform different combinations. When a button associated with a symbol is pressed two actions happen:
1) A sound corresponding with chosen action is played through a sound speaker attached to the vest. The action in textual format is sent via the internet to any caretaker, therapists or teachers, allowing them to be notified of the person's need if they are not in the same room.
2) To provide feedback to the disabled person, the pressed buttons also flashed an LED to indicate the actions chosen. Thus, a wearable, connected, non-intrusive, portable and user-friendly interface solution is proposed to help speech-impaired people communicate their needs on-site or remotely. The vest does not require physical effort to wear, can be worn in different locations and facilitates assisting the disabled to meet their needs, potentially preventing them from becoming uncomfortable or stressed.
This project was developed by Mauro Pichiliani (pichiliani@gmail.com) and Talita Pagani (talita.cpb@gmail.com)
Step 1: Hardware Components
We used the following hardware components for the CoCoA project:
* 1x DragonBoard 410C card. You can replace this board with a Raspberry Pi or any other small board computer, but keep in mind this is a wearable device.
* 1x NodeMCU ESP 8266 Arduino or other arduino with at least 13 GPIO PWM ports.
* 1x Mifare Model RC522 RFID Tag Reader Module.
* 8x Mifare RFID Tags.
* 8x Push Button Buttons.
* 8x LEDs of various colors.
* 6x 330Ohm Resistors.
* 5x square meters of brown TNT fabric.
* Small amount of cotton to fill in the pads
* 1x meter velcro strap.
* 8x Images of symbols printed on paper
* 1x USB speaker.
* 1x USB sound adapter dongle.
* 4GB miniSD memory card.
* 1x battery pack as power supply (2000 mAPH and 12 V output).
* 20x meters of cable with 2 wires
* General tools (pliers, screwdriver, soldering iron, glue gun, etc.), wires and electrical tape
Step 2: Flow Diagrams
To help understand how the vest works, let's explain its use through three diagrams. Check the figures of this step for reference.
Diagram 1: This flow represents the initial use of the vest describing the steps required to wear it and access the audio files already recorded.
Diagram 2: This diagram shows the steps for changing one of the existing pads (small couchions) so that another symbol/sound is placed on the vest.
Diagram 3: This diagram shows the steps for switching audio associated with an existing pad. This steps were not implemented in vest version 1.0
Step 3: Software
We used the following software in the project:
The project's external dependencies were the arduino library for handling the RFID reader module, the Python twx.botapi library for interacting with Telegram, and the PySerial library for reading/writing data over the serial port with Python. Python libraries can be installed using the pip package manager.
All project source codes have been made available in the following GitHub repository:
Step 4: Wiring the Components
The CoCoA design architecture is based on the DragonBoard 410C board, a NodeMCU 8622 arduino, a RFID card reader, leds, push buttons and a sound speaker. It also integrates with a Telegram bot called ProjectCoCoABot that sends a message with the generated audio each time one of the buttons is pressed. Check the figure in this step for a full view of the complete architecture of the solution.
Start by connecting the RFID reader and LEDs to the arduino and then upload the code in the /CocoaNodeMCUServer folder of this GitHub reposiboty. The connection diagrams of both the LEDs and the RFID reader are shown in the figures illustrating this step.
Next we need to connect the push buttons to the DragonBoard card. There is a diagram and tables in the figures of this step that shows the correspoding ports and wired needed to be connected.
Step 5: Pads Creation
The pads containing the symbols should be created next. There are several symbols and systems for nonverbal communication, but we can used PECS. This system has several symbols that can be printed and placed on the pads.
The pads are small couchions measuring 10x10cm and were filled with cotton. It is important to remember to place one RDID tag inside each pad so that you can identify each of the pad and their respective sound.
We placed the same symbol twice on the front of each pad: one with the image facing upwards and one downwards. This way it is possible for the vest user to observe which symbol was pressed. Be sure to place a velcro on the back of the pad so that you can attach/detach it in the vest
Step 6: Vest Assembly
The next step is the construction of the vest. We used a male childrens vest mold and make cuts on the fabric so that there is a lining. In these links we can find some blueprints of vests you can print
http://marlenemukaimoldeinfantil.com.br/2017/02/0...
http://cuttingecosting.com/Pap%20collect.html
It is important to define the location on the front where the pads will be fitted. In these places we should put velcro parts so that the pads fit correctly. Still in the front we can make the holes to place the the LEDs just above the cushions.
Close to the location of the cushion in the vest we need to fit the LEDs and each of the push buttons. It is important to fit the push buttons so that they are exactly behind the velcro position on the vest. This way as soon as the user presses the center of the cushion he/she will force the "hard" part of the cushion (the rfid tag) and press the push button.
We recommend fixing all the inner parts of the vest (LEDs, resistors, push buttons and wires) using hot glue. The connection of the wires can be made with solder + electrical tape. Another alternative to avoid welding is hot glue or use transparent nail polish.
The next part is build small pockets to store the components of each pad. Also create a pocket on the front of the vest closer to the waist to hold the speaker. It is recommended to put several straps of velcro to give the clothing firmness.
The wires that will connect the LEDs and push buttons of each touch area should be grouped two by two. This way we will have six strips with two cables each. Note that that each cable has two wires: positive and negative. It is important to mark the order of the cables and which component is connected to each cable (led or push button). We recommend wrapping the shoulder straps across two groups of three.
Once the cables and wires are connected we can close the vest by placing the lining. Finally, create a horizontal pocket on the back of the vest to hold the eletronic boards (NodeMCU and DragonBoard), the USB sound connector and the battery that will connect the DragonBoard. A small case is recommed to hold better the boards abd cables.
Step 7: Final Touchs and Testing
Once the vest straps are already attached to the back you must make the connections to the boards. 12 connections for push buttons (6x2) and 12 connections for LEDs are required.
Additional care is required to connect the wires that carry the ground signal (GND) to the LEDs, as all six pins of the LEDs must be connected to the same wire. Similarly, the pins of the ground push buttons (GND) should be connected on the same wire.
Finally, plug the dongle into the USB port and plug the adapter into the speaker, which should be placed in the front pocket of the vest. Plug the USB cable into the NodeMCU and other USB port of the DragonBoard 410c card. Finally, connect the battery pack to the power pin and start the CoCoaServer.py program on the DragonBoard board as sudo (use an SSH plug or connect a monitor + keyboard + mouse directly to the board):
$ sudo python CoCoaServer.py
At each button press a message will appear on the board console, the corresponding audio will be played and the led associated with the pad will be played.
Step 8: References
References of Alternative Communication:
Proloquo2Go symbol vocalization application
Saba M.P., Filippo D., Pereira F.R., by Souza P.L.P. (2011) Hey yaa: A Haptic Warning Wearable to Support Deaf People Communication. In: Vivacqua A.S., Gutwin C., Borges M.R.S. (eds) Collaboration and Technology. CRIWG 2011. Lecture Notes in Computer Science, vol 6969. Springer, Berlin, Heidelberg. DOI 10.1007 / 978-3-642-23801-7_17
Python libraries used:
DrabonBoard 410c Board Information
Arduino RFID Reader Usage Example
Children's vest molds
Participated in the
Assistive Tech Contest
