Nunchuck Text-to-speech Unit. Very Low Cost TVout/Arduino Based

Intro: Nunchuck Text-to-speech Unit. Very Low Cost TVout/Arduino Based

Purpose:
Lowest possible cost text- to-speech unit for the disabled, possibly in developing countries. Stand-alone (i.e. not PC based)

Input device:
Unmodified Wii Nunchuck.
In this example I am using the thumb joystick to navigate the text menu but could also be done by tilting the 'chuck and using the accelerometers (position sensors) inside.
Letters are selected by pressing one of the two buttons on the front of the Nunchuck.

Display:
Connects to any normal TV (PAL or NTSC) via standard yellow "video in" plug.

Computer:
Arduino board - turn on the power (battery) and you are ready to go.

Software:
Based on the MIT "TV-out" system which cleverly lets an Arduino "trick" a TV into displaying a basic picture without using any sort of graphics or video card.
I have modified this software to produce a simple alphabet from which you can select letters and make words with the nunchuck. This has to be done within the limitations of the Arduino as the TVout software uses a lot of the RAM to generate the picture.

Speech conversion:
The selected words are then sent to a serial "text to speech" module connected to a small loudspeaker. This gives the spoken text.

Cost:
Total cost < $100.

Other input options:
- If a wireless nunchuck is used, the user can move with no connecting wires so long as TV is in the room somewhere nearby.
- Currently uses 2 Arduino boards (one decodes the Nunchuck data). If a simple keypad or set of switches were used instead, a single Arduino would be enough to do everything. I am a big fan of the Nunchuck as an input device though.

STEP 1: Create a Connection Between Wii Nunchuck and Arduino

You need to make a socket that you can plug your Nunchuck into. The easiest way is to buy a very cheap "nunchuck extender" cable similar to the one illustrated.
Cut off the socket end with a small length of cable attached and you will find several wires inside.
Only 4 of these wires will be required - one for each of the OUTER four connnections of the total of six within the socket. See next page.

The project does assume some knowledge of Arduino microcontroller system and how you load programs from your PC on to the Arduino boards. If not:
There is a great set of tutorials here:
http://todbot.com/blog/bionicarduino/

Another great set of tutorials.
https://www.instructables.com/id/Arduino-Expermentation-Kit-How-to-get-Started-wi/

STEP 2: Connections Between Socket Pins and Arduino Board

Here is a view of the socket you have cut from the extension cable.
You are going to connect the contacts shown within the socket to the following pins on the Arduino board that you will use to "read" the Nunchuck data:
GND
+5V power
Analog pin 5
Analog pin 4

With a cheap multimeter you can work out which contact in the socket represents which coloured wire in the cable (use the resistance tester on the meter).

It is very easy to make a mistake here so double check. To reiterate, this is a view into the socket you will plug the nunchuck into, NOT a view of the plug on the end of the Nunchuck.

STEP 3: View of Cable Wires Connected to Arduino

This is the wire colour arrangement for the extender cable I used. However, some different makes will have different coloured wires.

STEP 4: Diagram of Nunchuck to Arduino Connections

This illustrates how you will connect the wires from the extension cable socket to the Arduino.

STEP 5: Connect Arduino 1 to Arduino 2

The Arduino that "reads" the Nunchuck data we will call Arduino 1.
The software running on this arduino will send commands to a second arduino -

Arduino2, that is (a) running the TVout software, (b) providing the text display on screen that allows you to select the letters and make the word you want and (c) sends the text strings out to a connected text-to-speech module + loudspeaker.

Arduino 1 is shown here on the right with the nunchuck wires coming into it. The link wires from Arduino 1 to Arduino 2 are shown and Arduino 2 is on the left.
I have put 1K Ohm resistors in each of the link wires apart from the GND link to limit current going between the arduino pins. I am not sure if this is essential but it seemed sensible.

STEP 6: Wiring Arduino 1 to Arduino 1

Connect Digital pins on Arduino 1 to Digital pins on Arduino 2 as follows:

GND to GND
11 to       11
4  to         4
5   to        9
7   to        7

STEP 7: Output to the TV

The output to the TV is simple yellow "video" plug. The cable to this has an inner central wire and an outer metal mesh shield wire around it.

These are connected to the Arduino 2 board as in the diagram:
Pins 8, 9 and GND are used.

The resistors have to be the same as or very close to the values shown.

STEP 8: Text to Speech Converter

This converts written text output from the Arduino 2 into speech.

The one I have used an "EMIC" has been popular with robot builders for some time.

It has 3 wires dealing with the data, 2 power wires and 2 more to a small speaker.

The wiring arrangement is shown on the next page.

STEP 9: Connecting Arduino 2 to Text to Speech Module

Wires are connected from Arduino 2 pins to the "EMIC" uinit I have used as shown here. The speech unit is a "serial to speech" unit and there are several types available. The wiring may vary slightly but the principle will be the same:

The serial link is via the Tx and Rx pins (pins 1 and 0 on Arduino 2), there is a "busy" pin (pin 10) that tells software to wait and not send the serial text to the speech module until it is ready.
The module needs +5V from the Arduino and also has a GND that needs connecting to GND on the Arduino.

Finally, there is a speaker + and speaker - connection on the speech unit that have to be wired to a small speaker.

STEP 10: Connect Text-to-speech Module to a Speaker

Here you can see the EMIC unit and small computer speaker I have connected it to.

The EMIC generates only 300 milliwatts of power so a small speaker is fine. Some others come with a slightly louder amplifier contained on the circuit board.

Alternatives are for example:

Sparkfun SpeakJet
http://www.sparkfun.com/commerce/product_info.php?products_id=9578

VoiceBox (i.e. SpeakJet) shield (here in UK):
http://www.coolcomponents.co.uk/catalog/product_info.php?products_id=508

SpeakJet shield – makes life a little easier:
http://www.droidbuilder.com/home/SpeakJetShield

STEP 11: Download TVout

You need to download TVout from here:
http://code.google.com/p/arduino-tvout/downloads/list

The top (latest version).

Download and extract the files.

You need then to find the file as shown below and put it into your Arduino "libraries" folder (see next page).

STEP 12: Move TV Out Library to Your Arduino Libraries

Move the TV out folder with the various h files etc inside, to your Arduino libraries folder - see photo, it has now been added (bottom right of photo).

STEP 13: Software

There is software to be loaded on to the Arduino1 (which reads the Nunchuck data).

There is software for the Arduino 2 which is a modified form of the TVout software that produces the picture on the screen. It also displays the menu of letters to select with a small cursor that moves under each letter and sends the selected words out to the text-to-speech module.
See video on page one and you will get the idea how this works.

The reason both parts of the software are not all on one arduino is that it will not work and the picture will "crash". I think this is because the TVout software uses a lot of RAM just to create a picture on the TV.

Both Arduino "Sketches" are attached.

NOTE: the modified TVout sketch for Arduino2 is right at limit of the memory. If you for example print just a few more characters on the screen, it will crash. If you want to modify the code, remove anything non essential then build up your new code in very small steps, testing if it works after each small addition.

NOTE: To load a sketch onto Arduino 2 you need to disconnect the Tx and Rx wires from the Arduino that go to the speech module. I could have avoided this by using some other pins to connect to the speech module and using the "software-serial" library - but this would stop the TVout working - I know as I have tried it.

STEP 14: Credits

The code that reads the Nunchuck is based on the initial work here - developed by Chad Phillips and others, see: http://windmeadow.com/node/42
For more info on wiring up a chuck to an Arduino, download Bionic Arduino Class 4 tutorial which is excellent from TodBot blog:
http://todbot.com/blog/2007/11/24/bionic-arduino-class-notes-3-4/



The TVout code that allows an arduino to generate a TV picture, which I have modified for this project is based on the TVout project here:
http://code.google.com/p/arduino-tvout/

Comments

[diy_bloke]

You really need two Arduino's for this? Is that because the program is too big??
Seems a bit of overkill

[diy_bloke]

ah I see, because of memory needs

[litteltoby]

what would you need to change to the code in order to make it work with EMIC2

[shortdiesel]

Very cool. Very cool. I like projects that can benefit people.

If I were up to the challange I would control my Android phone with the Wii remote (or even a device that can scan eye movements) and use the built in text to speech capability to have it on the go. Is anyone aware of pulling something like this off?

[Emiliogiz]

Y'know, it's cheaper to use a wii nunchuck adapter than buy the extension cable. It also doesn't permanently embed the arduino. Both Sparkfun and Solarbotics sell them:
http://www.sparkfun.com/products/9281
http://www.solarbotics.com/products/31040

[WingDings]

Nice! There's some great Engrish on the controller extension pack, too! :)

[rrrmanion]

a few adaption could make this mass (instructable) market:

-small screen on a wrist strap as opposed to big TV.
- speakers mounted on the user, instead of a small loudspeaker by the TV.
-perhaps a less expensive control device than a Nunchuk, because you only need a joystick.

[XenonJohn]

Thanks for the comments.
1) Speaker is currently with the user. When I get time I will use 2 of the really tiny "Nano" Arduino's you can get + Battery + speech unit + speaker all in a little black box with chuck socket hanging out.

2) It would be great to get rid of the TV and use a monocular head mounted display (you can still see what you are doing at the same time). You can buy these but they are serious money - the military use them to beam information to soldiers.
However, I did make one as an Instructable some time ago here:
https://www.instructables.com/id/Glasses-mounted-video-display-to-one-eye-turn-yo/

This would make you completely self contained with respect to text-speech and you would not need any sort of flat screen in front of you.

3) I agree a simple joystick would mean you could do all this with just the one arduino.

4) Predictive texting is beyond my coding ability and not sure if Arduino could handle it. If I add anything to the current code the screen display stops working so I suspect I am already nearing limits. TVout uses a lot of the Arduino. A menu of common words might be a lot more do-able. I will look into that.

5) This originally developed from a PC based version I made, again using a nunchuck: http://www.youtube.com/watch?v=mee7QArmaWE

6) Re: wrist mounted display:
Easiest way would be to make a similar system around a Serial-LCD liquid crystal display. These are available with up to 4 lines of 20 characters. This would avoid using TVout and free up memory for word menus and so on.
You might be able to display alphabet on upper 2 lines and the written text on the lower two. If anyone thinks this would be of practical value I would be happy to do an Instructable.

[rrrmanion]

dude, that would be awesome, and then you could do loads of adaptations an different versions

[kyle brinkerhoff]

hey you know theirs a ps2 keybord input library out their for the arduino , might make your life easyer, also if you use a serial display, you could essentially make an open source version of the speak and spell

[XenonJohn]

Hi,

True but I am thinking of someone more like Stephen Hawking with pretty minimal movement of one hand. With the Nunchuck, with slight code modification, you can use movement to operate it if you cannot work the thumb joystick so it gives you a number of potential input options all at a very low cost.
Below that level of function you are then looking at eye tracking or something along those lines.

For people who have enough function to "text" on a smartphone I have seen something somewhere on the web where a disabled person used an Android phone with software to convert the text to speech - with a toy megaphone plugged into the jack intended for the earpiece of the phone.

[calvin_evird]

Man for a concept idea this is awesome as, works great, and has sooo much capabilities and things that it could potentially do, great job!! :D!!

[thepelton]

Nice idea! If I were making it, I would add a list of the most common 2-3 letter words, such as "The, In, On, For, As, ..." Otherwise Very well done!

[rrrmanion]

if you really wanted to be clever, you could use the same technology used for texting on phones, to recognize what is being typed,