TJBot - Use Your Voice to Control a Light With Watson

23,751

62

44

Published

Introduction: TJBot - Use Your Voice to Control a Light With Watson

About: TJBot is an open source project designed to help you access Watson Services in a fun way. You can laser cut or 3D print him, then use one of his recipes to bring him to life! I can't wait to see what you t...

This instructable guides you through connecting a Raspberry Pi to Watson Speech to Text and change the color of a LED through speech. For example, you may say "Turn the light green" to change the color of the LED to green.

Step 1: Parts

Step 2: Prepare Your Pi

If you have used Raspberry Pi before, install Node.js and the Alsa packages and go to the next step. Otherwise, follow the instructions below to set up your Pi:


Getting Started with Your Pi

Raspberry Pi is similar to a full computer, which means you need a monitor, mouse, and keyboard for it. If you have a TV around, you can connect your Pi to your TV via a HDMI cable. In most of the Pi kits, the SD card is already preloaded with an image of the Raspberry Pi Operating System. You need to put the SD card in the Pi, turn the Pi ON and follow the instructions on screen to complete the installation of the operating system. If you have problems setting up your Pi, you can troubleshoot here.

Here is an easy way to get your Pi connected to wifi.


Install Packages

Open a terminal application on the Pi and execute the following commands to install the latest version of Node.js and npm (Node Package Manager). You need these packages later to run your code.

curl -sL http://ibm.biz/tjbot-bootstrap | sudo sh -

Step 3: Assemble TJBot

Here is a quick video of how to fold the kit. The detailed instructions are available on another instructable: Build TJBot out of Cardboard

Once your TJBot is ready, get ready to connect your LED to the Pi Pins.

Attention

It's very important to make sure you connect the LED to the right pins, otherwise you may damage your LED or the Pi.
ALWAYS start with the GND pin. GND is the longest leg of your LED. The LED has a flat side. The GND is the second leg from the flat side. Again if you are unsure, DO NOT connect the LED to the Pi.




Connect the GND to GND pin on the pi (see the picture). Then connect 'Data in', the 4th leg from the flat side, to the Pi's GPIO pin. Lastly, connect the third leg from the flat side to the 3.3V on the Pi.

Step 4: Download the Sample Code

The sample code is available at github. Download or clone the sample code and execute the following commands from a terminal to install its dependencies.

Here are the instructions for how to clone a repository from github if you have not done that before.

git clone https://github.com/ibmtjbot/tjbot.git
cd tjbot/recipes/speech_to_text
npm install

Step 5: Update Your Credentials

In this step, we help you get API access to the Watson Speech to Text. Let's start with creating a Speech to Text instance on Bluemix (If you don't have a Bluemix account, follow the instructions to create a free trial).

You may leave the default values and select 'Create'.


Almost done! You just need to copy your credentials to the clipboard.
Click on 'Sevice Credentials' on the left menu and copy your credentials into the clipboard.


Now go to the speech_to_text folder and make a copy of the default config file.

cp config.default.js config.js

Open config.js file using your favorite text editor and update it with your username and password.

Step 6: Run the Code

Once done with testing the LED, you can run the main module:

sudo node stt.js

Now talk to your microphone to change the color of the LED.
Say "Turn the light blue" to change the light to blue. You can try other colors: yellow, green, orange, purple, magenta, red, blue, aqua, white). You can either say "Turn the light on" or "Turn the light off"!


Troubleshoot

Is the LED not lighting up? No worries, we can fix it.

You may test your LED setup to make sure the connections are correct and the library is properly installed. When you run the test module, it should turn your LED on.

cd tjbot/bootstrap/tests 
npm install 
sudo node test.led.js

If the LED does not light up, you can try moving the power from 3.3 to 5 volts. If neither the 3.3v or 5v pins work, you will need a 1N4001 diode. The diode is inserted between the power pin of the LED (the shorter of the two middle pins) and the 5v pin on the Raspberry Pi. If you have problems with the setup, please refer to Adafruit's Neopixel on Raspbeery Pi guide to troubleshoot.

The LED library uses the PWM module (GPIO 18) to drive the data line of the LEDs. This conflicts with the built-in audio hardware, which uses the same pin to drive the audio output. Depending on your configuration of Raspbian, the sound drivers may be more aggressive in taking away control of GPIO 18 from other processes. If your LED shows random colors instead of the expected color, use this trick to fix it.

cd ..
sudo cp tjbot-blacklist-snd.conf /etc/modprobe.d/
sudo update-initramfs -u

Reboot and confirm no "snd" modules are running by executing the command "lsmod".

lsmod

Enjoy! Don't forget to share a picture/video of your setup #TJBot! :-)

Step 7: What's Next?

You can add new colors to your color palette in stt.js. TJBot uses a NeoPixel RGB LED, which means it can show any combination of red, green, and blue.

We have hidden a disco party for you. Find the code for disco party in stt.js and uncomment the code. Now you can ask TJBot to show you the disco lights by saying "Let's have a disco party"!

Try implementing your own TJBot party and share it with us #TJBot!

Once ready to move on, try the next recipe to make TJBot respond to emotions using Watson.

3 People Made This Project!

Recommendations

  • Oil Contest

    Oil Contest
  • Woodworking Contest

    Woodworking Contest
  • Clocks Contest

    Clocks Contest
user

We have a be nice policy.
Please be positive and constructive.

Tips

npm disn't install during setup, so use:
sudo apt-get install npm

Questions

44 Comments

i do not understand whats wrong. i get the entire code on the console after error: the speech to text returned an error.

also , i have snd modules displayed after i run snd ...

Hello there! I have a few problems, can you help me?

First off, after you upload your credentials and paste them on to the nano config.js, what do you do?

Second, when I tried to run: sudo node stt.js, it says Syntax error unexpected toke n. I dont know how to fix that.

Thanks!

Hi, everybody!

When I run sudo node stt.js, I receive the info that "TJBot is listening, you may speak now". However, right after the Console gives me the message, the program is closed.

I'm using a USB mic and no Error Message is showed for me.

I already read MauricioM82 comment, but that didn't help either. Is something related to audio configuration too?

Thank You!

Hi.

When I run stt.js program it shows the message:
"TJBot is listening, you may speak now."
but don't give me time to speak and close the program right after showing the message. No errors, nothing, just show the message and close the program.

I am using an USB sound card with both input and output. Both arecord and aplay commands works just fine, I am able to record and listen to the audio I've recorded. I am also able to use the voice control to control bb-8.

Any suggestions?

Thank you.

8 replies

Hi MauricioM82,

You are correct, the issue is definitely related to the app having difficulty setting up audio. For arecord on command line, do you pass any parameters for your app to work correctly ? For example if your sound input might be listed with an unexpected hardware ID or might need several parameters to work.

To find out your hardware device ID run arecord -L ,and then specify that as a parameter when you instantiate the mic in stt.js.

In the sample apps we provide, we use the mic npm library (

https://www.npmjs.com/package/mic) to record audio. You can provide hardware options when you instantiate your mic to record audio e.g

var micInstance = mic({ 'rate': '44100', 'channels': '2', 'debug': false, 'exitO

nSilence': 6, 'device': 'plughw:1,0' });


A related issue (solved) on recording from a bluetooth and usb mic can be found here

https://github.com/ibmtjbot/tjbot/issues/12

Hi Victordibia

First off, thanks for this fun and easy to follow recipe!

I've followed all the steps on the tutorial however i'm getting an error after running the command "sudo node stt.js".

I believe the trouble is derived from raspbian jessie not seeing my usb mic.

The error reads "the speech_to_text service returned an error" and then prints lines of code. It appears right after the console logs "TjBot initializing microphone"

I've ran asplay -l and a usb sound card does not appear. However when i go to the raspberry pi settings > preferences > audio devices> the dropdown shows my usb mic, but when I attempt at making it the default audio, it never saves the change.

I've been stuck on this problem some time, so any help would be greatly appreciated.

I'm running Raspbian 8.0 Jessie.

Thanks for reading and I hope to hear from you soon!

hi ,

I'm facing the same issue

Did you solve it .?

Great!

Thank you for your help. So, I specified the device when instantiating the mic and it worked, it now recognizes my voice.

The problem now is that the led don't change colors (it's green, and always green).

I did the suggestions on the Troubleshooting part of the article but after reboot I see several "snd" modules loaded, like

snd_usb_audio 118800 3
snd_hwdep 5727 1 snd_usb_audio
snd_usbmidi_lib 19801 1 snd_usb_audio
snd_rawmidi 18516 1 snd_usbmidi_lib
snd_seq_device 3621 1 snd_rawmidi
snd_pcm 75762 1 snd_usb_audio
snd_timer 19288 1 snd_pcm

Any suggestions?

Thank you in advance.

Maybe I found the answer.. I may be using a different type of led.

We have the ground pin and one of each other is for one color.

Does the led used on the article has a special name, or some way I can identify it to purchase in a store?

Thank you.

I live in Brazil and it's really hard to find the adafruit version or anything similar around here. And shipping from adafruit site to Brazil is way way way to expensive.

Maybe someone could ship a kit from the site to me via regular mail? :D

Thank you for all your help.

We used a NeoPixel LED. Here is a link to the specs: https://www.adafruit.com/products/1734

Hi,

Thanks for the interesting tutorial and nice recipe!

I have a Common Anode + LED instead of the one used in tutorial so I went ahead and made a quick change to the code to make it work with my LED, if someone wants to try, here is the separated branch in my fork:

https://github.com/dnguyenv/tjbot/tree/common-anod...

Changes:

- Removed rpi-ws281x-native library, use rpio npm lib
- Changed the main tss.js file to work with my LED
- Added my wire diagram (https://github.com/dnguyenv/tjbot/blob/common-anode/recipes/speech_to_text/tts.png)
- Changed README to be relevant to the change

Best - Duy

1 reply

Hi Duy. I see your code for a multi color solution. It is so much easier to understand your code and I think it is closer to what I am trying to do. Can you help me to create an even a simpler code that will set a specified GPIO pin to be an output and then be able to set it HIGH or LOW based on voice commands? If possible, I would like to add this functionality to the Conversation TJbot recipe so that I can have a conversation as well as issue commands to switch GPIO output pins HIGH and LOW.

These instructions are awesome. I just have to follow instructions and they always work. I do want to ask however if someone can anyone help me to modify the TJbot LED Voice control project that controls the PWM GPIO pin? I would like to first understand how to modify the program to simply point to a specified GPIO pin and then change it to an output pin and then be able to set the pin HIGH and LOW in accordance with Speech-to-text voice control. To be honest, I can't figure out where the actual code is located that points to the PWM pin


Then, ideally, the voice control part of this recipe would be added to the conversation recipe so that I can have a conversation with the robot, but when a command is heard, a GPIO pin will be controlled to be set HIGH or LOW.

By way of example, I would say

User: Watson, Hello, how are you?
Watson: I am doing very well. Thank you for asking.
User: Turn on the red LED
Watson will then cause the GPIO 5 to be set HIGH, which will switch a circuit that will route power to a red LED.
User: Watson, thank you for turning the LED on
Watson; Your very welcome.
User: Watson, Turn on the White LED.
Watson will then cause the GPIO 4 to be set HIGH, which will switch a circuit that will route power to a white LED.

1 reply

So for anyone with the same problem, i ended up just resetting the pi back to factory settings and redoing the steps above.

To reset, just type reboot in the console, run it, then hold shift for recovery mode when the pi prompts you to do so.

Hi,

Our light will only stay blue. We tried doing the test and when the light should turn red it stays blue. we have checked that all of our connections are correct, and even tried switching from the 3.3V to 5V and that didn't work either. We also can't get past initializing microphone in the terminal.

1 reply

Having the same issue with the microphone. Did you ever get passed it?

Hi,

I am using raspberry Pi 3 model B with Hardware : BCM2835.

All connections are made as per tutorial. My LED is shining blue all the time.

I installed rpi-ws281x-native, but when i connect to Watson Speech to text, it says connected to TJ bot.

next it gives error

[rpi-ws281x-native] Could not verify raspberry-pi version. If this is wrong and you are running this on a raspberry-pi, please file a bug-report at https://github.com/beyondscreen/node-rpi-ws281x-n...

Any help is welcome in this regard.

Thanks

Divya

1 reply

Hello,

I had the same problem and I fixed it by recompiling the library rpi-ws281x-native

To do it:

go in the catalog with your project

uninstall the old library

npm uninstall rpi-ws281x-native

install sources of the new library

sudo npm install -g node-gyp

git clone --recursive https://github.com/beyondscreen/node-rpi-ws281x-n...

cd node-rpi-ws281x-native/lib

there is a file in this folder ws281x-native.js and you need to edit it and remove the following code

if (raspberryVersion === 0) { console.warn('[rpi-ws281x-native] Could not verify raspberry-pi ' + 'version. If this is wrong and you are running this on a ' + 'raspberry-pi, please file a bug-report at ' + ' https://github.com/beyondscreen/node-rpi-ws281x-n... + 'A non-functional stub of this modules interface will be ' + 'returned.'); return stub;

save the file

cd ..

npm install

node-gyp rebuild

mv node-rpi-ws281x-native node_modules/rpi-ws281x-native

This is more or less what needed to be done :) I hope it helps you to find the right solution for you.