TJBot - Build a Talking Robot With Watson Conversation

43,192

111

126

About: TJBot is an open source paper robot that I designed to help you learn programming Artificial Intelligence in a fun way! You can laser cut or 3D print him, then use one of his recipes to bring him to life! I ...

This instructable guides you through connecting a Raspberry Pi to Watson conversation services and making a talking robot. You will use (1) Watson Speech to Text to convert your voice to text, (2) Watson Conversation to process the text and calculate a response, and (3) Watson Text to Speech to talk the response back.

Step 1: Parts

Step 2: Prepare Your Pi

If you have used Raspberry Pi before, install Node.js and go to the next step. Otherwise, follow the instructions below to set up your Pi:


Getting Started with Your Pi

Raspberry Pi is similar to a full computer, which means you need a monitor, mouse, and keyboard for it. If you have a TV around, you can connect your Pi to your TV via a HDMI cable. In most of the Pi kits, the SD card is already preloaded with an image of the Raspberry Pi Operating System. You need to put the SD card in the Pi, turn the Pi ON and follow the instructions on screen to complete the installation of the operating system. If you have problems setting up your Pi, you can troubleshoot here.

Here is an easy way to get your Pi connected to wifi.


Install Packages

Open a terminal application on the Pi and execute the following commands to install the latest version of Node.js and npm (Node Package Manager). You need these packages later to run your code.

curl -sL http://ibm.biz/tjbot-bootstrap | sudo sh -

Step 3: Assemble TJBot

Here is a quick video of how to fold the kit. The detailed instructions are available on another instructable: Build TJBot out of Cardboard

Once your TJBot is ready, plug in your USB microphone and the speaker.

Depending on which audio output source you are using with your Pi (HDMI, 3.5mm audio jack, Bluetooth, USB speaker), you may need to set the audio config.

HDMI/ 3.5mm Audio Jack

If you use HDMI or 3.5mm audio jack, you may need to set the audio config. To do this, go to the terminal and open up raspi-config.

 sudo raspi-config  

This will open up the Raspberry Pi configuration screen:

Select "Advanced Options" and press Enter, then select "Audio" and press Enter. Choose the correct channel for the output audio. If you have connected an external speaker to the audio jack, you should select 3.5mm jack.

USB Speaker

If you have a USB audio, you need to update your /usr/share/alsa/alsa.config to set the USB audio as the default device. Begin with running the following command to make sure your USB is connected and listed there.

 lsusb 

Next is to detect the card number of your USB audio.

aplay -l 

Take a note of the card number associated with your USB Audio. Then go to the alsa.config file to set it as default.

sudo nano /usr/share/alsa/alsa.conf

Look for

defaults.ctl.card 0
defaults.pcm.card 0

and update the card number (0 here) to the card number of your USB audio.

Different versions of Raspberry Pi OS may need a different setup. If you have problem with your USB setup, check out this guide to troubleshoot.

Bluetooth Speaker

If you plan to use a Bluetooth speaker, you may need to install the Raspberry Pi Bluetooth software. Here is a good tutorial to set it up: Play audio on a bluetooth speaker with Raspberry Pi 3

Step 4: Download the Sample Code

The source code is available at github. Download or clone the code and execute the following commands from a terminal to install its dependencies..

Here are the instructions for how to clone a repository from github if you have not done that before.


git clone https://github.com/ibmtjbot/tjbot.git
cd tjbot/recipes/conversation
npm install

Step 5: Update Your Credentials

In this step, we help you get API access to three Watson conversation services.

You will use three types of Watson services in this recipe: (1) Watson Speech to Text, (2) Watson Conversation, (3) Watson Text to Speech. You need to copy your credentials for all these services.
Let's start with creating a Watson Conversation instance (If you don't have a Bluemix account, you need to create one).

You can leave the default values and select 'Create'.

Now we need to copy your credentials from Bluemix. Click on 'Sevice Credentials' on the left menu and copy your credentials into clipboard.


Now make a copy of the default config file.

cp config.default.js config.js

Open config.js file using your favorite text editor and update it with your username and password.

The Conversation service needs one more ID to configure. It's called the Workspace ID (exports.conversationWorkspaceId). Don't worry about it for now. In the next step, we will walk you through creating a conversation workspace.

You are done with the Conversation credentials! Next stop is Watson Speech to Text. Follow the same process to create a service and copy your credentials.

You may leave the default values for the service & credential names and select 'Create'.

Now go to 'Sevice Credentials' on the left menu and copy your credentials into clipboard.

You need to update config.js with your Speech to Text credentials.

The last stop is Watson Text to Speech. You need to do the exact same steps you did with the other two services. You may leave the default values for service and credential name and select 'create'.

Copy your credentials and add them to config.js.

Here is the section of config.js that you need to update:

Congradulations! You survived the credentials section! ::robot-smile::

Step 6: Create a Conversation Flow

You need to train your robot with what to say and when to say it. For that, we use Watson Conversation. Open a browser and go to IBM Watson Conversation.

Log in, and in the the top left menu, select the name of your conversation service. Click on 'Import' icon to import the sample conversation.

In the dialog that opens, click 'Choose a file' and select 'workspace-sample.json' from the 'tjbot/recipes/conversation' folder. Make sure 'Everything (Intents, Entities, and Dialog)' is selected.

After the workspace has been imported, click the dotted menu and select 'View details'. Copy your Workspace ID as shown.

Next, update your config.js file with the workspace ID.

Congratulations, your conversation is ready!

Step 7: Run the Code

Now, you are ready to talk to your TJBot!

Open a terminal and execute the following command:

sudo node conversation.js

Watson conversation uses intents to label the purpose of a sentence. For example when you ask TJBot "Please introduce yourself", the intent is to make an introduction. You can add your own new intents in the Conversation editor, but for now, we have started you off with two intents:

  • Introduction. You can say phrases such as "Watson, please introduce yourself", "Watson, who are you", and "Watson, can you introduce yourself"
  • Joke. You can ask "Watson, please tell me a joke" or "Watson, I would like to hear a joke".

If you would like to create your own intents, we recommend following this guide.

An attention word is used so TJBot knows you are talking to him. The default attention word is 'Watson', but you can change it in conversation.js as follows. Update conversation.js and change tjConfig to add the 'robot' section:

// turn on debug logging to the console
var tjConfig = {
    verboseLogging: true,
    robot: {
        name: 'Watson',
        gender: 'male'
    }
};

You can change the 'name' to whatever you would like to call your TJBot. In addition, if you change the gender to 'female', TJBot will use a female voice to speak to you!

Enjoy! Don't forget to share a picture/video of your setup #TJBot! :-)

Troubleshoot

If you see TJBot's response on the terminal but don't hear TJBot talking, there is a good chance that one of these two things has happened: (1) The audio output is directed to a wrong channel (you can fix it from raspi-config), (2) your sound modules are blocked. In that case, go to /etc/modprobe.d/ and remove blacklist-rgb-led.conf

Then run the following command:

sudo update-initramfs -u

Reboot and confirm the "snd" modules are running by executing the command "lsmod". This should solve the problem.

lsmod

4 People Made This Project!

Recommendations

  • PCB Contest

    PCB Contest
  • Epilog X Contest

    Epilog X Contest
  • Cardboard Challenge

    Cardboard Challenge

126 Discussions

0
None
Aurovindhya

15 days ago

The sudo node conversation.js ran.. And the output msg was Hello from TJBot!....... Etc..
Then when I gave the voice input.. There was no response and after sometime..this error popped..
Unable to retrieve TTS voices message=socket hang up, stack=Error: socket hang up..
Please reply asap.. And help me..
Thanks in advance..

0
None
JLG20

3 months ago

Hi eveyone. I need help, I installed TJBOT into my Raspberry PI 3 (and node js 9), when execute sudo node conversation.js, I receive the following message:

verbose: TJBot initializing microphone
verbose: TJBot initializing LED
verbose: TJBot initializing servo motor on PIN 7
verbose: TJBot initializing assistant service
verbose: TJBot initializing speech_to_text service
verbose: TJBot initializing text_to_speech service
verbose: TJBot initializing visual_recognition service
/home/pi/tjbot/recipes/conversation/node_modules/tjbot/lib/tjbot.js:542
throw new Error(
^
Error: Missing authentication credentials for tone_analyzer service: username/password or apikey are required.
at TJBot._createServiceAPI (/home/pi/tjbot/recipes/conversation/node_modules/tjbot/lib/tjbot.js:542:19)
at new TJBot (/home/pi/tjbot/recipes/conversation/node_modules/tjbot/lib/tjbot.js:116:18)
at Object.<anonymous> (/home/pi/tjbot/recipes/conversation/conversation.js:40:10)
at Module._compile (internal/modules/cjs/loader.js:654:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:665:10)
at Module.load (internal/modules/cjs/loader.js:566:32)
at tryModuleLoad (internal/modules/cjs/loader.js:506:12)
at Function.Module._load (internal/modules/cjs/loader.js:498:3)
at Function.Module.runMain (internal/modules/cjs/loader.js:695:10)
at startup (internal/bootstrap/node.js:201:19)
at bootstrapNodeJSCore (internal/bootstrap/node.js:516:3)

2 replies
0
None
elvickJLG20

Reply 2 months ago

I have the same error

0
None
ElmarKleikampelvick

Reply 5 weeks ago

Hi JLG20

Edit with a text editor ...~/Desktop/tjbot/recipes/conversation/config.js
You have to enter the credential for Watson Visual Recognition (apikey, and not the url)
Check in Dashboard of IBM Bluemix website what skills you already have, if it´s not there activate Watson Visual Recognition and look for the apikey there
In this last line change api_key to apikey (api_key is faulty here)
If yo´re done it should work.

0
None
thilom1

2 months ago

Hello,
I build my TJBot like instructed .
It seems to work, he did not answear.
I cn tell him something he reconizes the words, but he dont answear at the USB Speaker.
My Config is:
USB - Speaker, USB Mikrophone,
can someone help me?
Thanks Menne
here is the Console-Log:

erbose: TJBot initializing microphone
verbose: TJBot initializing LED
verbose: TJBot initializing servo motor on PIN 7
verbose: TJBot initializing Camera
verbose: TJBot initializing assistant service
verbose: TJBot initializing speech_to_text service
verbose: TJBot initializing text_to_speech service
info: Hello from TJBot! My name is Watson.
verbose: TJBot library version v1.4.0
You can ask me to introduce myself or tell you a joke.
Try saying, "Watson, please introduce yourself" or "Watson, what can you do?"
You can also say, "Watson, tell me a joke!"
verbose: TJBot initializing microphone
info: TJBot heard: what %HESITATION telling a joke
info: TJBot heard: what do you tell me a joke
info: TJBot heard: what can you do
info: TJBot heard: what please introduce your
info: TJBot heard: what
info: TJBot heard: please interviews you also
info: TJBot heard: and what he did leave the joke

1 reply
0
None
ElmarKleikampthilom1

Reply 5 weeks ago

Hi thilom1,

check with aplay -l the parameters of your soundcard (generally it is plughw:1,0)
edit ~/Desktop/tjbot/recipes/conversation/node_modules/tjbot/lib/tjbot.js

look for microphone and edit the parameter there to what aplay -l showed
look for speaker and edit the parameter there to what aplay -l showed

In my settings for the microphone it was 1,0 and the mic worked fine.
But the speaker was set with 0,0 and it didn´t work.
I changed it to 1,0 and everything worked fine.

Good luck

0
None
YuxingW

Question 7 weeks ago

Hi, can someone help me with the credentials for the TJBot. I can't find the 36 digit username and 12 character password. Can someone tell me where it is. I have an attached photo of what my credential page look like on IBM cloud. Thanks!!!!

CREDENTIAL.jpg
1 answer
0
None
ElmarKleikampYuxingW

Answer 5 weeks ago

Hi XuxingW,
you need the apikey and the url.
-The apikey is the first line in your credentials
-The url is the last line in your credentials
Edit with a text editor ...~/Desktop/tjbot/recipes/conversation/config.js
If there is only a config.default.js type
cp config.default.js config.js
Normally you see username and password fields in the file
However something has changed with the parameters, not yet implemented in the config.default.js file:
-Replace password with apikey
-Replace username with url
Remember that password is listed on top, then comes username
So after having changed them, url is on the bootom line
Insert the string from line 1 of your credentials to apikey
Insert the string from the last line of your credentials to url
Use the different credentials for conversation, stt, tts

At the end you also have to enter the credential for Watson Visual Recognition (apikey, and not the url)
Check in Dashboard of IBM Bluemix website what skills you already have, if it´s not there activate Watson Visual Recognition and look for the apikey there
In this last line change api_key to apikey (api_key is faulty here)
If yo´re done it should work.

0
None
thilom1

Question 3 months ago

Hi together,

i have aquestion:

I followed the instructions till Step5.

when i try to update config File for text-to speech i cant find the folder.

pi@botty:~/Desktop/tjbot/recipes $ ls
conversation README.md sentiment_analysis speech_to_text

i have conversation but not text-to-speech.

Which config file do i need to change?

Thanks T

0
None
PaulH496

Question 3 months ago

It would be great to update this tutorial. With the version of software needed and problems in the comments section addressed. This would be incredible helpful.

0
None
BENYS7

1 year ago

Hi,

I have also this error but not about npm :( "

Cannot find module 'tjbot'

---------------------------------------------

$ node conversation.js
module.js:471
throw err;
^

Error: Cannot find module 'tjbot'
at Function.Module._resolveFilename (module.js:469:15)
at Function.Module._load (module.js:417:25)
at Module.require (module.js:497:17)
at require (internal/module.js:20:19)
at Object.<anonymous> (......\TJBOT\tjbot-master\recipes\conversation\conversation.js:17:13)
at Module._compile (module.js:570:32)
at Object.Module._extensions..js (module.js:579:10)
at Module.load (module.js:487:32)
at tryModuleLoad (module.js:446:12)
at Function.Module._load (module.js:438:3)

4 replies
0
None
PaulH496BENYS7

Reply 4 months ago

Did you eventually get running? Having the same problem.

0
None
cmackinBENYS7

Reply 1 year ago

I am having the same problem. I have reinstalled and still I can't get it to work. I have configured the config.js and removed the default, but still no joy. I had this working in a previous installation but this one is not working. What does it mean by "Cannot find module 'tjbot'?"

I think this must have something to do with the directory that tjbot is installed. I have tried moving it around, but still not working.

0
None
sultana123cmackin

Reply 11 months ago

Also first write cd tjbot/recipes/conversation in the terminal window and then run conversation.js
By sudo node conversation.js

0
None
ljmwaughcmackin

Reply 11 months ago

check to see if you have a node_modules subdirectory under conversation. If it is there, then try removing and rebuilding.
remove with: sudo rm -rf node_modules
then rebuild with: npm install

0
None
PaulH496

4 months ago

Bought latest tjbot pi3 and getting this message. Very new to this and would appreciate any help?

pi@Co:~ $ sudo node conversation.js

module.js:549

throw err;

^

Error: Cannot find module '/home/pi/conversation.js'

at Function.Module._resolveFilename (module.js:547:15)

at Function.Module._load (module.js:474:25)

at Function.Module.runMain (module.js:693:10)

at startup (bootstrap_node.js:188:16)

at bootstrap_node.js:609:3

0
None
RS3655

Question 5 months ago on Step 7

Hello,

I have been using the TJBOT program for quite some time now. But when I ran it a day ago, the program terminated with a Websocket connection error. It says that the speech to text service returned an error. Can you please suggest what I should do at the present moment? It would be a great help if you can suggest a remedy.

Regards,

Debashish

Following is the displayed error:

________________________________________________________________________

error: the speech_to_text service returned an error. message=WebSocket connection error, stack=WebSocket connection error: WebSocket connection error

at W3CWebSocket.socket.onerror (/home/pi/tjbot/recipes/conversation/node_modules/watson-developer-cloud/lib/recognize-stream.js:197:23)

at W3CWebSocket._dispatchEvent [as dispatchEvent] (/home/pi/tjbot/recipes/conversation/node_modules/yaeti/lib/EventTarget.js:107:17)

at W3CWebSocket.onConnectFailed (/home/pi/tjbot/recipes/conversation/node_modules/websocket/lib/W3CWebSocket.js:217:14)

at WebSocketClient.<anonymous> (/home/pi/tjbot/recipes/conversation/node_modules/websocket/lib/W3CWebSocket.js:59:25)

at emitOne (events.js:96:13)

at WebSocketClient.emit (events.js:188:7)

at WebSocketClient.failHandshake (/home/pi/tjbot/recipes/conversation/node_modules/websocket/lib/WebSocketClient.js:326:10)

at ClientRequest.<anonymous> (/home/pi/tjbot/recipes/conversation/node_modules/websocket/lib/WebSocketClient.js:265:18)

at emitOne (events.js:96:13)

at ClientRequest.emit (events.js:188:7)

at HTTPParser.parserOnIncomingClient [as onIncoming] (_http_client.js:478:21)

at HTTPParser.parserOnHeadersComplete (_http_common.js:99:23)

at TLSSocket.socketOnData (_http_client.js:367:20)

at emitOne (events.js:96:13)

at TLSSocket.emit (events.js:188:7)

__________________________________________________________________________

0
None
IeuanC

10 months ago

Hello everyone, I was wondering if someone could help me as I can't seem to find an answer on the internet. With the attention word that gets TJBot to realise you are addressing him, is there a way to not use the attention word not at all?

Or maybe, once you use the attention word once, can you not have to say 'watson' before every utterance until you give a command that stops interaction?

0
None
AaronD136

Question 11 months ago on Step 7

Finally finished & is working perfectly!

Few questions if anyone knows & is willing to share out infomation (thanks in advance if you do!, I'm still new to the raspberry pi in general).

1: Is there a way to remove the activation word (e.g default you have to say watson before the question). I would like it set where its always actively listening & will respond when it detects key sentence. I also cant seem to find the file where you can change the voice to male or female & change the activation name.

2: Any way to speed up the results of tjbot giving out responses? (e.g like this video but.. less complicated with a guide on how to do so.) I'm planning on bringing this in the public and would like to give rapid results when listening and saying back conversations to other peoples questions & chit chats in general.

3: Is there any pre-built samples of conversations to use (more or less just to test around with things get abit more knowledge in general).

0
None
AaronD136

1 year ago

When I type in sudo node conversation.js I end up getting

module.js:340

throw err;

^

Error: Cannot find module '/home/pi/conversation.js'

at Function.Module._resolveFilename (module.js:338:15)

at Function.Module._load (module.js:280:25)

at Function.Module.runMain (module.js:497:10)

at startup (node.js:119:16)

at node.js:906:3

Any help on the matter would be really appropriated as I'm not an expert on raspberry pi but would love to get this project working.