TJBot - Build a Talking Robot With Watson Conversation

38,024

108

113

Published

Introduction: TJBot - Build a Talking Robot With Watson Conversation

About: TJBot is an open source project designed to help you access Watson Services in a fun way. You can laser cut or 3D print him, then use one of his recipes to bring him to life! I can't wait to see what you t...

This instructable guides you through connecting a Raspberry Pi to Watson conversation services and making a talking robot. You will use (1) Watson Speech to Text to convert your voice to text, (2) Watson Conversation to process the text and calculate a response, and (3) Watson Text to Speech to talk the response back.

Step 1: Parts

Step 2: Prepare Your Pi

If you have used Raspberry Pi before, install Node.js and go to the next step. Otherwise, follow the instructions below to set up your Pi:


Getting Started with Your Pi

Raspberry Pi is similar to a full computer, which means you need a monitor, mouse, and keyboard for it. If you have a TV around, you can connect your Pi to your TV via a HDMI cable. In most of the Pi kits, the SD card is already preloaded with an image of the Raspberry Pi Operating System. You need to put the SD card in the Pi, turn the Pi ON and follow the instructions on screen to complete the installation of the operating system. If you have problems setting up your Pi, you can troubleshoot here.

Here is an easy way to get your Pi connected to wifi.


Install Packages

Open a terminal application on the Pi and execute the following commands to install the latest version of Node.js and npm (Node Package Manager). You need these packages later to run your code.

curl -sL http://ibm.biz/tjbot-bootstrap | sudo sh -

Step 3: Assemble TJBot

Here is a quick video of how to fold the kit. The detailed instructions are available on another instructable: Build TJBot out of Cardboard

Once your TJBot is ready, plug in your USB microphone and the speaker.

Depending on which audio output source you are using with your Pi (HDMI, 3.5mm audio jack, Bluetooth, USB speaker), you may need to set the audio config.

HDMI/ 3.5mm Audio Jack

If you use HDMI or 3.5mm audio jack, you may need to set the audio config. To do this, go to the terminal and open up raspi-config.

 sudo raspi-config  

This will open up the Raspberry Pi configuration screen:

Select "Advanced Options" and press Enter, then select "Audio" and press Enter. Choose the correct channel for the output audio. If you have connected an external speaker to the audio jack, you should select 3.5mm jack.

USB Speaker

If you have a USB audio, you need to update your /usr/share/alsa/alsa.config to set the USB audio as the default device. Begin with running the following command to make sure your USB is connected and listed there.

 lsusb 

Next is to detect the card number of your USB audio.

aplay -l 

Take a note of the card number associated with your USB Audio. Then go to the alsa.config file to set it as default.

sudo nano /usr/share/alsa/alsa.conf

Look for

defaults.ctl.card 0
defaults.pcm.card 0

and update the card number (0 here) to the card number of your USB audio.

Different versions of Raspberry Pi OS may need a different setup. If you have problem with your USB setup, check out this guide to troubleshoot.

Bluetooth Speaker

If you plan to use a Bluetooth speaker, you may need to install the Raspberry Pi Bluetooth software. Here is a good tutorial to set it up: Play audio on a bluetooth speaker with Raspberry Pi 3

Step 4: Download the Sample Code

The source code is available at github. Download or clone the code and execute the following commands from a terminal to install its dependencies..

Here are the instructions for how to clone a repository from github if you have not done that before.


git clone https://github.com/ibmtjbot/tjbot.git
cd tjbot/recipes/conversation
npm install

Step 5: Update Your Credentials

In this step, we help you get API access to three Watson conversation services.

You will use three types of Watson services in this recipe: (1) Watson Speech to Text, (2) Watson Conversation, (3) Watson Text to Speech. You need to copy your credentials for all these services.
Let's start with creating a Watson Conversation instance (If you don't have a Bluemix account, you need to create one).

You can leave the default values and select 'Create'.

Now we need to copy your credentials from Bluemix. Click on 'Sevice Credentials' on the left menu and copy your credentials into clipboard.


Now make a copy of the default config file.

cp config.default.js config.js

Open config.js file using your favorite text editor and update it with your username and password.

The Conversation service needs one more ID to configure. It's called the Workspace ID (exports.conversationWorkspaceId). Don't worry about it for now. In the next step, we will walk you through creating a conversation workspace.

You are done with the Conversation credentials! Next stop is Watson Speech to Text. Follow the same process to create a service and copy your credentials.

You may leave the default values for the service & credential names and select 'Create'.

Now go to 'Sevice Credentials' on the left menu and copy your credentials into clipboard.

You need to update config.js with your Speech to Text credentials.

The last stop is Watson Text to Speech. You need to do the exact same steps you did with the other two services. You may leave the default values for service and credential name and select 'create'.

Copy your credentials and add them to config.js.

Here is the section of config.js that you need to update:

Congradulations! You survived the credentials section! ::robot-smile::

Step 6: Create a Conversation Flow

You need to train your robot with what to say and when to say it. For that, we use Watson Conversation. Open a browser and go to IBM Watson Conversation.

Log in, and in the the top left menu, select the name of your conversation service. Click on 'Import' icon to import the sample conversation.

In the dialog that opens, click 'Choose a file' and select 'workspace-sample.json' from the 'tjbot/recipes/conversation' folder. Make sure 'Everything (Intents, Entities, and Dialog)' is selected.

After the workspace has been imported, click the dotted menu and select 'View details'. Copy your Workspace ID as shown.

Next, update your config.js file with the workspace ID.

Congratulations, your conversation is ready!

Step 7: Run the Code

Now, you are ready to talk to your TJBot!

Open a terminal and execute the following command:

sudo node conversation.js

Watson conversation uses intents to label the purpose of a sentence. For example when you ask TJBot "Please introduce yourself", the intent is to make an introduction. You can add your own new intents in the Conversation editor, but for now, we have started you off with two intents:

  • Introduction. You can say phrases such as "Watson, please introduce yourself", "Watson, who are you", and "Watson, can you introduce yourself"
  • Joke. You can ask "Watson, please tell me a joke" or "Watson, I would like to hear a joke".

If you would like to create your own intents, we recommend following this guide.

An attention word is used so TJBot knows you are talking to him. The default attention word is 'Watson', but you can change it in conversation.js as follows. Update conversation.js and change tjConfig to add the 'robot' section:

// turn on debug logging to the console
var tjConfig = {
    verboseLogging: true,
    robot: {
        name: 'Watson',
        gender: 'male'
    }
};

You can change the 'name' to whatever you would like to call your TJBot. In addition, if you change the gender to 'female', TJBot will use a female voice to speak to you!

Enjoy! Don't forget to share a picture/video of your setup #TJBot! :-)

Troubleshoot

If you see TJBot's response on the terminal but don't hear TJBot talking, there is a good chance that one of these two things has happened: (1) The audio output is directed to a wrong channel (you can fix it from raspi-config), (2) your sound modules are blocked. In that case, go to /etc/modprobe.d/ and remove blacklist-rgb-led.conf

Then run the following command:

sudo update-initramfs -u

Reboot and confirm the "snd" modules are running by executing the command "lsmod". This should solve the problem.

lsmod

3 People Made This Project!

Recommendations

  • Clocks Contest

    Clocks Contest
  • Creative Misuse Contest

    Creative Misuse Contest
  • Oil Contest

    Oil Contest

113 Discussions

0
user
IeuanC

4 months ago

Hello everyone, I was wondering if someone could help me as I can't seem to find an answer on the internet. With the attention word that gets TJBot to realise you are addressing him, is there a way to not use the attention word not at all?

Or maybe, once you use the attention word once, can you not have to say 'watson' before every utterance until you give a command that stops interaction?

Finally finished & is working perfectly!

Few questions if anyone knows & is willing to share out infomation (thanks in advance if you do!, I'm still new to the raspberry pi in general).

1: Is there a way to remove the activation word (e.g default you have to say watson before the question). I would like it set where its always actively listening & will respond when it detects key sentence. I also cant seem to find the file where you can change the voice to male or female & change the activation name.

2: Any way to speed up the results of tjbot giving out responses? (e.g like this video but.. less complicated with a guide on how to do so.) I'm planning on bringing this in the public and would like to give rapid results when listening and saying back conversations to other peoples questions & chit chats in general.

3: Is there any pre-built samples of conversations to use (more or less just to test around with things get abit more knowledge in general).

When I type in sudo node conversation.js I end up getting

module.js:340

throw err;

^

Error: Cannot find module '/home/pi/conversation.js'

at Function.Module._resolveFilename (module.js:338:15)

at Function.Module._load (module.js:280:25)

at Function.Module.runMain (module.js:497:10)

at startup (node.js:119:16)

at node.js:906:3

Any help on the matter would be really appropriated as I'm not an expert on raspberry pi but would love to get this project working.

2 replies

hi, you need to be in the conversation subdirectory. The default subdirectory is /home/pi/Desktop/tjbot/recipes/conversation

Hello

Thanks for the reply (really appreciate it!)

After going to the sub directory /home/pi/Desktop/tjbot/recipes/conversation and then typing in sudo node conversation.js I ended up getting this error

module.js:340
throw err;
^
Error: Cannot find module 'tjbot'
at Function.Module._resolveFilename (module.js:338:15)
at Function.Module._load (module.js:280:25)
at Module.require (module.js:364:17)
at require (module.js:380:17)
at Object. (/home/pi/Desktop/tjbot/recipes/conversation.js:17:13)
at Module._compile (module.js:456:26)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Function.Module.runMain (module.js:497:10)

I followed another comment stating that if this issue happens and you dont see node_modules in the conversation sub directory then do npm install (which I didn't have npm_modules so I did just that) and I got alot of errors upon downloading npm_module, I later tried rebuilding but that didn't work either. Here is a download link to the npm-debug.log I received if you want to take a look.

https://uploadfiles.io/4qg95

(I used uploadfiles.io to upload the npm-debug notepad file).

Also I was in Putty + WINSCP using ssh if that would make a difference.

Thanks again for the assistance!

Cannot find module 'tjbot'

I have tried everything including rebuilding the RPi. Below is what I type to run Conversation.js and the following is the error. Any help here would be great.

I get a similar error when I run stt.js

Error: Cannot find module './config'

It seems like the directory of install is wrong or there is an index file that needs to be configured.

pi@raspberrypi:~/tjbot/recipes/conversation $ sudo node conversation.js

module.js:472

throw err;

^

Error: Cannot find module 'tjbot'

at Function.Module._resolveFilename (module.js:470:15)

at Function.Module._load (module.js:418:25)

at Module.require (module.js:498:17)

at require (internal/module.js:20:19)

at Object.<anonymous> (/home/pi/tjbot/recipes/conversation/conversation.js:17:13)

at Module._compile (module.js:571:32)

at Object.Module._extensions..js (module.js:580:10)

at Module.load (module.js:488:32)

at tryModuleLoad (module.js:447:12)

at Function.Module._load (module.js:439:3)

2 replies

Also first write cd tjbot/recipes/conversation in the terminal window and then run conversation.js
By sudo node conversation.js
// as first you should go into the subdirectory and then run the cide

hi, is there an node_modules subdirectory under the conversation subdirectory? if there isn't, run
npm install
if there is you can try removing the node_modules and rebuilding by running
sudo rm -rf node_modules
npm install

0
user
BENYS7

1 year ago

Hi,

I have also this error but not about npm :( "

Cannot find module 'tjbot'

---------------------------------------------

$ node conversation.js
module.js:471
throw err;
^

Error: Cannot find module 'tjbot'
at Function.Module._resolveFilename (module.js:469:15)
at Function.Module._load (module.js:417:25)
at Module.require (module.js:497:17)
at require (internal/module.js:20:19)
at Object.<anonymous> (......\TJBOT\tjbot-master\recipes\conversation\conversation.js:17:13)
at Module._compile (module.js:570:32)
at Object.Module._extensions..js (module.js:579:10)
at Module.load (module.js:487:32)
at tryModuleLoad (module.js:446:12)
at Function.Module._load (module.js:438:3)

3 replies

I am having the same problem. I have reinstalled and still I can't get it to work. I have configured the config.js and removed the default, but still no joy. I had this working in a previous installation but this one is not working. What does it mean by "Cannot find module 'tjbot'?"

I think this must have something to do with the directory that tjbot is installed. I have tried moving it around, but still not working.

Also first write cd tjbot/recipes/conversation in the terminal window and then run conversation.js
By sudo node conversation.js

check to see if you have a node_modules subdirectory under conversation. If it is there, then try removing and rebuilding.
remove with: sudo rm -rf node_modules
then rebuild with: npm install

hi!

we need to copy our credentials on rasbian itself or need to take sd card an copy in windows?

because on rasbian itself when i am using the web browser and openong my bluemix account and copying the credentials and in the terminal window typing

cp config.default.js config.js

it shows no such file or directory exists

why so?

2 replies

First make a copy in coversation folder of recipes by name config.js of config.default.js, you can also copy paste the folder and rename it, if it is not making a copy by typing in termimal window!! Also after making the copy update credentials

hi, you are probably not in the correct subdirectory. The default subdirectory is /home/pi/Desktop/tjbot/recipes/conversation

Hello Sir/Maam I want to make a simple AI .. the concept is when you ask a question it will answer.. I think 100 simple question is enough can anyone help me Thank you

1 reply

I set everything up as you described and my new buddy is working beautifully. Now I want to go a bit further and have him tell me the time whenever I ask him. Since I have conversation.js running most (read: all) of the time, I'd like to expand this function so that it can do this as well. Any tips on how to do this efficiently for an absolute beginner?