loading

This instructable guides you through connecting a Raspberry Pi to Watson Speech to Text and change the color of a LED through speech. For example, you may say "Turn the light green" to change the color of the LED to green.

Step 1: Parts

Step 2: Prepare Your Pi

If you have used Raspberry Pi before, install Node.js and the Alsa packages and go to the next step. Otherwise, follow the instructions below to set up your Pi:


Getting Started with Your Pi

Raspberry Pi is similar to a full computer, which means you need a monitor, mouse, and keyboard for it. If you have a TV around, you can connect your Pi to your TV via a HDMI cable. In most of the Pi kits, the SD card is already preloaded with an image of the Raspberry Pi Operating System. You need to put the SD card in the Pi, turn the Pi ON and follow the instructions on screen to complete the installation of the operating system. If you have problems setting up your Pi, you can troubleshoot here.

Here is an easy way to get your Pi connected to wifi.


Install Packages

Open a terminal application on the Pi and execute the following commands to install the latest version of Node.js and npm (Node Package Manager). You need these packages later to run your code.

sudo apt-get update

sudo apt-get dist-upgrade

curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash -

sudo apt-get install -y nodejs

sudo apt-get install alsa-base alsa-utils libasound2-dev

Step 3: Assemble TJBot

Here is a quick video of how to fold the kit. The detailed instructions are available on another instructable: Build TJBot out of Cardboard

Once your TJBot is ready, get ready to connect your LED to the Pi Pins.

Attention

It's very important to make sure you connect the LED to the right pins, otherwise you may damage your LED or the Pi.
ALWAYS start with the GND pin. GND is the longest leg of your LED. The LED has a flat side. The GND is the second leg from the flat side. Again if you are unsure, DO NOT connect the LED to the Pi.




Connect the GND to GND pin on the pi (see the picture). Then connect 'Data in', the 4th leg from the flat side, to the Pi's GPIO pin. Lastly, connect the third leg from the flat side to the 3.3V on the Pi.

Step 4: Download the Sample Code

The sample code is available at github. Download or clone the sample code and execute the following commands from a terminal to install its dependencies.

Here are the instructions for how to clone a repository from github if you have not done that before.

git clone https://github.com/ibmtjbot/tjbot.git
cd tjbot/recipes/speech_to_text
npm install

Step 5: Update Your Credentials

In this step, we help you get API access to the Watson Speech to Text. Let's start with creating a Speech to Text instance on Bluemix (If you don't have a Bluemix account, follow the instructions to create a free trial).

You may leave the default values and select 'Create'.


Almost done! You just need to copy your credentials to the clipboard.
Click on 'Sevice Credentials' on the left menu and copy your credentials into the clipboard.


Now go to the speech_to_text folder and make a copy of the default config file.

cp config.default.js config.js

Open config.js file using your favorite text editor and update it with your username and password.

Step 6: Run the Code

Once done with testing the LED, you can run the main module:

sudo node stt.js

Now talk to your microphone to change the color of the LED.
Say "Turn the light blue" to change the light to blue. You can try other colors: yellow, green, orange, purple, magenta, red, blue, aqua, white). You can either say "Turn the light on" or "Turn the light off"!


Troubleshoot

Is the LED not lighting up? No worries, we can fix it.

You may test your LED setup to make sure the connections are correct and the library is properly installed. When you run the test module, it should turn your LED on.

cd tjbot/bootstrap/tests 
npm install 
sudo node test.led.js
If the LED does not light up, you can try moving the power from 3.3 to 5 volts. If neither the 3.3v or 5v pins work, you will need a 1N4001 diode. The diode is inserted between the power pin of the LED (the shorter of the two middle pins) and the 5v pin on the Raspberry Pi. If you have problems with the setup, please refer to Adafruit's Neopixel on Raspbeery Pi guide to troubleshoot.

The LED library uses the PWM module (GPIO 18) to drive the data line of the LEDs. This conflicts with the built-in audio hardware, which uses the same pin to drive the audio output. Depending on your configuration of Raspbian, the sound drivers may be more aggressive in taking away control of GPIO 18 from other processes. If your LED shows random colors instead of the expected color, use this trick to fix it.

sudo cp blacklist-rgb-led.conf /etc/modprobe.d/
sudo update-initramfs -u

Reboot and confirm no "snd" modules are running by executing the command "lsmod".

lsmod

Enjoy! Don't forget to share a picture/video of your setup #TJBot! :-)

Step 7: What's Next?

You can add new colors to your color palette in stt.js. TJBot uses a NeoPixel RGB LED, which means it can show any combination of red, green, and blue.

We have hidden a disco party for you. Find the code for disco party in stt.js and uncomment the code. Now you can ask TJBot to show you the disco lights by saying "Let's have a disco party"!

Try implementing your own TJBot party and share it with us #TJBot!

Once ready to move on, try the next recipe to make TJBot respond to emotions using Watson.

<p>Hi,</p><p>I am using raspberry Pi 3 model B with Hardware : BCM2835.</p><p>All connections are made as per tutorial. My LED is shining blue all the time.</p><p>I installed rpi-ws281x-native, but when i connect to Watson Speech to text, it says connected to TJ bot.</p><p>next it gives error</p><p>[rpi-ws281x-native] Could not verify raspberry-pi version. If this is wrong and you are running this on a raspberry-pi, please file a bug-report at <a href="https://github.com/beyondscreen/node-rpi-ws281x-native/issues" rel="nofollow"> https://github.com/beyondscreen/node-rpi-ws281x-n...</a></p><p>Any help is welcome in this regard.</p><p>Thanks </p><p>Divya </p>
<p>Hello,</p><p>I had the same problem and I fixed it by recompiling the library rpi-ws281x-native</p><p>To do it:</p><p>go in the catalog with your project</p><p>uninstall the old library</p><p>npm uninstall rpi-ws281x-native</p><p>install sources of the new library</p><p>sudo npm install -g node-gyp</p><p>git clone --recursive <a href="https://github.com/beyondscreen/node-rpi-ws281x-native.git" rel="nofollow"> https://github.com/beyondscreen/node-rpi-ws281x-n...</a></p><p>cd node-rpi-ws281x-native/lib</p><p>there is a file in this folder ws281x-native.js and you need to edit it and remove the following code </p> if (raspberryVersion === 0) { console.warn('[rpi-ws281x-native] Could not verify raspberry-pi ' + 'version. If this is wrong and you are running this on a ' + 'raspberry-pi, please file a bug-report at ' + ' https://github.com/beyondscreen/node-rpi-ws281x-n... + 'A non-functional stub of this modules interface will be ' + 'returned.'); return stub;<p>save the file</p><p>cd ..</p><p>npm install</p><p>node-gyp rebuild<br></p><p>mv node-rpi-ws281x-native node_modules/rpi-ws281x-native</p><p>This is more or less what needed to be done :) I hope it helps you to find the right solution for you.</p>
<p>Hi,</p><p>The node-rpi-ws281x-native has provided a fix. Now I am able to run the basic functionality test rainbow.js which is mentioned in <a href="https://github.com/beyondscreen/node-rpi-ws281x-native." rel="nofollow">https://github.com/beyondscreen/node-rpi-ws281x-na...</a></p><p>A directory node-rpi-ws281x-native is installed.</p><p>Issue is when I use the Tjbot example, I am still getting the same error.</p><p>I tried mv the node node-rpi-ws281x-nativeto rpi-ws281x-native.</p><p>But still getting error with Tjbot example</p><p>What change is required to work this with tjbot?</p><p> [rpi-ws281x-native] Could not verify raspberry-pi version. If this is wrong and you are running this on a raspberry-pi, please file a bug-report at <a href="https://gi" rel="nofollow"> https://gi </a> thub.com/beyondscreen/node-rpi-ws281x-native/issues</p><p>Thanks</p><p>Divya</p>
<p>I'm also having the problem that the light is always blue. The console is showing that TJBot is hearing the commands correctly but the light isn't changing. I purchased the light from Adafruit.</p><p>I was successful getting the conversation to work, just not the light.</p>
<p>Please use the Strandtest by adafruit. This will ensure the LED is showing correct colours<br><a href="https://learn.adafruit.com/neopixels-on-raspberry-pi/software" rel="nofollow">https://learn.adafruit.com/neopixels-on-raspberry-pi/software</a></p>
<p>Hi, I got the same problem with the latest Raspbian Jessie (2017-04-10). One month ago, my TJbot worked fine with same hardware with old version Jessie. So, I solved the problem by following;</p><p>1) Install the latest Raspbian Jessie (2017-04-10).</p><p>2) Do NOT execute &quot;sudo apt-get update&quot; and &quot;sudo apt-get dist-upgrade&quot;.</p><p>3) Execute &quot;curl -sL <a href="https://deb.nodesource.com/setup_6.x" rel="nofollow"> https://deb.nodesource.com/setup_6.x </a> | sudo -E bash -<br><br>&quot;, &quot;sudo apt-get install -y nodejs&quot;, and &quot;sudo apt-get install alsa-base alsa-utils libasound2-dev&quot;.</p>
<p>Having the exact same issue.</p>
<p>Hi, </p><p>Our light will only stay blue. We tried doing the test and when the light should turn red it stays blue. we have checked that all of our connections are correct, and even tried switching from the 3.3V to 5V and that didn't work either. We also can't get past initializing microphone in the terminal.</p>
<p>I can't find any directory tjbot/bootstrap/</p><p>for testing the LED.</p><p>Johann</p>
<p>Hi,<br>I finished assembly my TJ and can run all services successfully. Only nasty points are that the Speech to Text is quite never translating correctly. Also in this case here, the LED color wont change. At the shell output the software does it quite fine. But eventhough it tells me the color has been changed... nothing happens.<br>Wiring is exactly as stated here, also the LED type. <br>It's just shining a beatiful blue all the time...<br><br>Any tips&amp;tricks welcome.</p>
<p>I think my USB Mic is not recognized by speech_to_text although it works with arecord/aplay.</p><p>&gt;&gt;verbose: TJBot library version v1.2.1</p><p>&gt;&gt;I understand lots of colors. You can tell me to shine my light a different color by saying 'turn the light red' or 'change the light to green' or 'turn the light off'.</p><p>&gt;&gt;verbose: TJBot initializing microphone</p><p>&gt;&gt;error: the speech_to_text service returned an error. Error: Session timed out, no data received in the last 30 seconds.</p><p>I read below that the mic setting can be modified like this:</p><p>&gt;&gt; var micInstance = mic({ 'rate': '44100', 'channels': '2', 'debug': false, 'exitO</p><p>nSilence': 6, 'device': 'plughw:1,0' });</p><p>My question, which module has this var micInstance? I only find var micParams.</p><p>Thanks..</p>
<p>Well, I'm quite sure now that the mic is recognized. I managed to turn on the mic function debug: true and got this:</p><p>&gt;&gt;Found silence block: 481 of 6</p><p>&gt;&gt;Found silence block: 482 of 6</p><p>&gt;&gt;Found speech block</p><p>&gt;&gt;Found speech block</p><p>&gt;&gt;Found speech block</p><p>&gt;&gt;Found speech block</p><p>&gt;&gt;Found speech block</p><p>&gt;&gt;Found silence block: 1 of 6</p><p>&gt;&gt;Found silence block: 2 of 6</p><p>The timeout suspect is now with speech_to_text call. I have moved the region from US South to Sydney but same result. I tried this </p><p>curl -X POST -u {username}:{password}<br>--header &quot;Content-Type: audio/flac&quot;<br>--header &quot;Transfer-Encoding: chunked&quot;<br>--data-binary @{path}audio-file.flac<br>&quot;https://stream.watsonplatform.net/speech-to-text/api/v1/recognize?continuous=true&quot;</p><p>It responded back fast no timeout so it is not network issues.</p><p>Any hint? </p>
<p>Just completed this recipe and can verify the workaround to blacklist sound drivers is necessary as of now. Wonder if this will be an issue with recipes requiring TJBot to &quot;speak.&quot; </p><p>uname -a reports &quot;4.4.50-v7+ #970 SMP Mon Feb 20 19:18:29 GMT 2017 armv7l&quot;</p><p>Nice recipe. Thanks for creating.</p><p>--John.</p>
<p>I am trying to merge this project with the conversation project. My goal is to have the LED turn green if the name was recognized before passing to conversation and text to speech. This will give the user a notification that the services are working. However, when I run the conversation app as sudo, I no longer get audio through the bluetooth speaker. Since sudo is required to run the LED, this is an issue. Has anyone worked through this before and gotten the audio to work through sudo or the light to work without sudo?</p>
<p>Did you unblock the sound modules? Raspberry Pi has a known issue of PWD conflicting with audio jack. This means you cannot use the PWD pin (used for the LED) and the audio together. To solve this, we suggested on the last step to block all the sound modules https://www.instructables.com/id/Use-Your-Voice-to-Control-a-Light-With-Watson/step7/Run-the-Code/ so if you done that, you need to undo this first and then use an external sound card to get the audio. Here is the sound card we've been using https://learn.adafruit.com/usb-audio-cards-with-a-raspberry-pi/instructions</p>
<p>Hi, </p><p>Thanks for the interesting tutorial and nice recipe! </p><p>I have a Common Anode + LED instead of the one used in tutorial so I went ahead and made a quick change to the code to make it work with my LED, if someone wants to try, here is the separated branch in my fork:</p><p><a href="https://github.com/dnguyenv/tjbot/tree/common-anode/recipes/speech_to_text" rel="nofollow">https://github.com/dnguyenv/tjbot/tree/common-anod...</a></p><p>Changes:</p><p>- Removed rpi-ws281x-native library, use rpio npm lib<br>- Changed the main tss.js file to work with my LED<br>- Added my wire diagram (https://github.com/dnguyenv/tjbot/blob/common-anode/recipes/speech_to_text/tts.png)<br>- Changed README to be relevant to the change</p><p> Best - Duy</p>
<p>Hi.</p><p>When I run stt.js program it shows the message:<br>&quot;TJBot is listening, you may speak now.&quot;<br>but don't give me time to speak and close the program right after showing the message. No errors, nothing, just show the message and close the program.</p><p>I am using an USB sound card with both input and output. Both arecord and aplay commands works just fine, I am able to record and listen to the audio I've recorded. I am also able to use the voice control to control bb-8.</p><p>Any suggestions?</p><p>Thank you.</p>
<p>Hi MauricioM82,<br><br>You are correct, the issue is definitely related to the app having difficulty setting up audio. For arecord on command line, do you pass any parameters for your app to work correctly ? For example if your sound input might be listed with an unexpected hardware ID or might need several parameters to work. <br><br>To find out your hardware device ID run arecord -L ,and then specify that as a parameter when you instantiate the mic in stt.js.<br><br>In the sample apps we provide, we use the mic npm library (</p><p><a>https://www.npmjs.com/package/mic) </a> to record audio. You can provide hardware options when you instantiate your mic to record audio e.g </p><p>var micInstance = mic({ 'rate': '44100', 'channels': '2', 'debug': false, 'exitO<br><br>nSilence': 6, 'device': 'plughw:1,0' });<br><br><br>A related issue (solved) on recording from a bluetooth and usb mic can be found here </p><p><a href="https://github.com/ibmtjbot/tjbot/issues/12">https://github.com/ibmtjbot/tjbot/issues/12</a><br><br></p>
<p>Great!</p><p>Thank you for your help. So, I specified the device when instantiating the mic and it worked, it now recognizes my voice.</p><p>The problem now is that the led don't change colors (it's green, and always green).</p><p>I did the suggestions on the Troubleshooting part of the article but after reboot I see several &quot;snd&quot; modules loaded, like</p><p>snd_usb_audio 118800 3 <br>snd_hwdep 5727 1 snd_usb_audio<br>snd_usbmidi_lib 19801 1 snd_usb_audio<br>snd_rawmidi 18516 1 snd_usbmidi_lib<br>snd_seq_device 3621 1 snd_rawmidi<br>snd_pcm 75762 1 snd_usb_audio<br>snd_timer 19288 1 snd_pcm</p><p>Any suggestions?</p><p>Thank you in advance.</p>
<p>Maybe I found the answer.. I may be using a different type of led.</p><p>We have the ground pin and one of each other is for one color.</p><p>Does the led used on the article has a special name, or some way I can identify it to purchase in a store?</p><p>Thank you.</p>
<p>Great!<br>We use the RGB Neopixel LED based on the ws2812 protocol.</p><p><a href="https://www.adafruit.com/products/1734">https://www.adafruit.com/products/1734</a><br><br>Really look forward to seeing what you build! Pictures, videos ? You can share on twitter #tjbot<br><br>V.</p>
<p>I live in Brazil and it's really hard to find the adafruit version or anything similar around here. And shipping from adafruit site to Brazil is way way way to expensive.</p><p>Maybe someone could ship a kit from the site to me via regular mail? :D</p><p>Thank you for all your help.</p>
<p>We used a NeoPixel LED. Here is a link to the specs: https://www.adafruit.com/products/1734</p>
<p>Can anyone suggest a cheap usb microphone I could buy because my budget isn't big enough for a $70 one</p>
<p>Hi Jamie, I've been using https://www.amazon.com/gp/product/B00IR8R7WQ/ref=oh_aui_detailpage_o01_s00?ie=UTF8&amp;psc=1 and it works well for the price.</p>
<p>can i use raspberry 2 instead of raspberry3</p>
<p>It should work on Raspberry 2. I haven't tested it, but <a href="/member/victordibia">victordibia</a> confirmed he was able to run it on Pi2.</p>
<p>hi, yes right i have a talking robot with Watson Conversation. i fixed the credentials, but still my loudspeaker do not work, i tried it with an apple headset, but also no sound. i used sudo raspi-config and changed the audio settings, but no sound. </p><p>i also tried the code </p><p>sudo amixer cset numid=3 1, but no sound</p><p>Alexander</p>
<p>Alexander, I'm glad your payload quota error is resolved. If you direct the sound to the right channel and still don't get any output sound, I guess the problem is the blacklist. Try the troubleshoot on step 7 and make sure the Pi is not blocking the sound output: https://www.instructables.com/id/Build-a-Talking-Robot-With-Watson-and-Raspberry-Pi/step7/Run-the-Code/ </p>
<p>hi,</p><p>i have the talking tjbot and also getting the error</p><p>TJ is listening, you may speak now.</p><p> ===== An Error has occurred ===== </p><p>You may have exceeded your payload quota.</p><p> [object Object]</p><p> Press &lt;ctrl&gt;+C to exit.</p><p> <br> <br> <br> <br><br><br> i checked all credentials and compared with my bluemix credentials, they are all ok. in conversation i do not have only export.username, export.password, also i have export.conworkspace, but i do not see it in the bluemix credentials! what i have to enter in export.conworkspace.</p><p>greetings</p><p>alexander</p>
<p>Hi Alexander, you are right, the payroll quota error is usually caused by incorrect credentials. </p><p>Which recipe are you trying to run? The &quot;Use your voice to control a light with Watson&quot; doesn't use Watson Conversation. I guess you are referring to another TJBot recipe (&quot;Build a talking robot with Watson Conversation&quot;). If so, please follow the instructions on step 6 to look up your conversation workspace ID. <a href="https://www.instructables.com/id/Build-a-Talking-Robot-With-Watson-and-Raspberry-Pi/step6/Create-a-conversation-flow/"> https://www.instructables.com/id/Build-a-Talking-R...</a> I just added a screenshot of the config file. Hopefully it makes it easier to locate the part. </p><p>Please let me know if this solves your problem! ::robot-face::</p>
<p>Cool stuff!</p>
<p>Help? I get this:</p><p>TJ is listening, you may speak now.</p><p> ===== An Error has occurred ===== </p><p>You may have exceeded your payload quota.</p><p> [object Object]</p><p> Press &lt;ctrl&gt;+C to exit.</p>
<p>Hi MarkK236,<br><br>Can you confirm you have set up your bluemix speech to text credentials correctly in config.js?<br>Usually if your speech-to-text credentials are incorrect (or missing), you will get this error. Please see step 5 for how to add your Bluemix credentials to config.js</p><p><a href="https://www.instructables.com/id/Use-Your-Voice-to-Control-a-Light-With-Watson/step5/Update-your-Bluemix-credentials/" rel="nofollow">https://www.instructables.com/id/Use-Your-Voice-to-...</a><br><br>Please let me know if this helps!<br><br>V.</p>
<p>Got it! I had erroneously tried to cut and paste credentials from a different Watson service that I had created (from the Tone Analyzer demo). That's what I get for not following instructions. :D</p><p>Cool demo! Thank you for your help!</p>
<p>Awesome!</p>
<p>Thanks, Victor. Hi Mark, I just added a screenshot of the config file for you on step #5. https://www.instructables.com/id/Use-Your-Voice-to-Control-a-Light-With-Watson/step5/Update-your-Bluemix-credentials/ Let me know if this solves your problem! ::robot-smile::</p>
That is really cool. I always wanted to have a voice controlled robot.
<p>We cannot wait to see what you build with TJ! </p>

About This Instructable

14,114views

53favorites

Bio: I'm an open source project designed to help you access Watson Services in a fun way. You can laser cut or 3D print me ... More »
More by TJBot:How to Run a TJBot Workshop How to 3D Print a TJBot Build TJBot Out of Cardboard 
Add instructable to: