TJBot - Build a Talking Robot

52,775

123

139

Introduction: TJBot - Build a Talking Robot

About: TJBot is an open source paper robot that I designed to help you learn programming Artificial Intelligence in a fun way! You can laser cut or 3D print him, then use one of his recipes to bring him to life! I ca…

This instructable guides you through connecting a Raspberry Pi to Watson conversation services and making a talking robot. You will use (1) Watson Speech to Text to convert your voice to text, (2) Watson Assistant to process the text and calculate a response, and (3) Watson Text to Speech to talk the response back.

Step 1: Parts

Step 2: Prepare Your Pi

If you have used Raspberry Pi before, install Node.js and go to the next step. Otherwise, follow the instructions below to set up your Pi:


Getting Started with Your Pi

Raspberry Pi is similar to a full computer, which means you need a monitor, mouse, and keyboard for it. If you have a TV around, you can connect your Pi to your TV via a HDMI cable. In most of the Pi kits, the SD card is already preloaded with an image of the Raspberry Pi Operating System. You need to put the SD card in the Pi, turn the Pi ON and follow the instructions on screen to complete the installation of the operating system. If you have problems setting up your Pi, you can troubleshoot here.


Install Packages

Open a terminal application on the Pi and execute the following commands to install the latest version of Node.js and npm (Node Package Manager). You need these packages later to run your code.

curl -sL http://ibm.biz/tjbot-bootstrap | sudo sh -

Step 3: Assemble TJBot

Here is a quick video of how to fold the kit. The detailed instructions are available on another instructable: Build TJBot out of Cardboard

Once your TJBot is ready, plug in your USB microphone and the speaker.

Depending on which audio output source you are using with your Pi (HDMI, 3.5mm audio jack, Bluetooth, USB speaker), you may need to set the audio config.

HDMI/ 3.5mm Audio Jack

If you use HDMI or 3.5mm audio jack, you may need to set the audio config. To do this, go to the terminal and open up raspi-config.

 sudo raspi-config  

This will open up the Raspberry Pi configuration screen:

Select "Advanced Options" and press Enter, then select "Audio" and press Enter. Choose the correct channel for the output audio. If you have connected an external speaker to the audio jack, you should select 3.5mm jack.

USB Speaker

If you have a USB audio, you need to update your /usr/share/alsa/alsa.config to set the USB audio as the default device. Begin with running the following command to make sure your USB is connected and listed there.

 lsusb 

Next is to detect the card number of your USB audio.

aplay -l 

Take a note of the card number associated with your USB Audio. Then go to the alsa.config file to set it as default.

sudo nano /usr/share/alsa/alsa.conf

Look for

defaults.ctl.card 0
defaults.pcm.card 0

and update the card number (0 here) to the card number of your USB audio.

Different versions of Raspberry Pi OS may need a different setup. If you have problem with your USB setup, check out this guide to troubleshoot.

Bluetooth Speaker

If you plan to use a Bluetooth speaker, you may need to install the Raspberry Pi Bluetooth software. Here is a good tutorial to set it up: Play audio on a bluetooth speaker with Raspberry Pi 3

Step 4: Download the Sample Code

The source code is available at github. Download or clone the code and execute the following commands from a terminal to install its dependencies..

Here are the instructions for how to clone a repository from github if you have not done that before.


git clone https://github.com/ibmtjbot/tjbot.git

cd tjbot/recipes/conversation
npm install

Pro tip: if you get an error for npm install that says npm not found, you should first install npm on your machine.

This is the command line to install npm

sudo apt-get install npm

Step 5: Update Your Credentials

In this step, we help you get API access to three conversation services: (1) Speech to Text, (2) Watson Assistant, (3) Text to Speech. You need to copy your credentials for all these services.

Create instances of the Watson Assistant, Speech to Text, and Text to Speech services and note the authentication credentials.

Import the workspace-sample.json file into the Watson Assistant service and note the workspace ID.

Make a copy the default configuration file and update it with the Watson service credentials and the conversation workspace ID.

$ cp config.default.js config.js
$ nano config.js
<enter your credentials and the conversation workspace ID in the specified places>

Step 6: Run the Code

Now, you are ready to talk to your TJBot!

Open a terminal and execute the following command:

sudo node conversation.js

Watson conversation uses intents to label the purpose of a sentence. For example when you ask TJBot "Please introduce yourself", the intent is to make an introduction. You can add your own new intents in the Conversation editor, but for now, we have started you off with a few intents:

  • Introduction. You can say phrases such as "Watson, please introduce yourself", "Watson, who are you", and "Watson, can you introduce yourself"
  • Joke. You can ask "Watson, please tell me a joke" or "Watson, I would like to hear a joke".

For a complete list, check the content of workspace-sample.json

An attention word is used so TJBot knows you are talking to him. The default attention word is 'Watson', but you can change it in config.js as follows. Update the configuration file to change the robot name in tjConfig section:

// set up TJBot's configuration

exports.tjConfig = {

log: {   level: 'verbose'    },

robot: {   name: 'tee jay bot'  }

};

You can change the 'name' to whatever you would like to call your TJBot. In addition, if you change the gender to 'female', TJBot will use a female voice to speak to you!

Enjoy! Don't forget to share a picture/video of your setup #TJBot! :-)

Troubleshoot

If you see TJBot's response on the terminal but don't hear TJBot talking, there is a good chance that one of these two things has happened: (1) The audio output is directed to a wrong channel (you can fix it from raspi-config), (2) your sound modules are blocked. In that case, go to /etc/modprobe.d/ and remove blacklist-rgb-led.conf

Then run the following command:

sudo update-initramfs -u

Reboot and confirm the "snd" modules are running by executing the command "lsmod". This should solve the problem.

lsmod

5 People Made This Project!

Recommendations

  • Mason Jar Speed Challenge

    Mason Jar Speed Challenge
  • Bikes Challenge

    Bikes Challenge
  • Remix Contest

    Remix Contest

139 Discussions

0
cralvara
cralvara

Question 3 months ago

Hi everyone,
I am using a Raspberry Pi 4 and connected the LED as stated in instructions. I followed all steps but when running the LED test, it gets stucked in "verbose: TJBot initializing LED" and then I got some errors:

pi@TJBot:~/Desktop/tjbot/bootstrap/tests $ sudo node test.led.js
verbose: TJBot initializing LED

Message from syslogd@TJBot at Jun 4 18:05:51 ...
kernel:[ 378.762443] Internal error: Oops: 205 [#1] SMP ARM
Message from syslogd@TJBot at Jun 4 18:05:51 ...
kernel:[ 378.762607] Process kworker/u8:1 (pid: 957, stack limit = 0xfa0b7970)
Message from syslogd@TJBot at Jun 4 18:05:51 ...
kernel:[ 378.762612] Stack: (0xef967e40 to 0xef968000)
Message from syslogd@TJBot at Jun 4 18:05:51 ...
kernel:[ 378.762619] 7e40: ef967e5c ef967e50 c0209948 c020bff4 ef967efc ef967e60 c04377a4 c020992c

The LED is currently in Blue color and it stays the same.
I have made sure that cables are correctly plug in.
What else can I do?

Thanks for help,

0
GhazouaniS
GhazouaniS

1 year ago

Hi, how are you
I'm doing my tjbot, but I have a problem how can change voice from "en-US_MichaelVoice" to en-FR_ReneeVoice thank you for repointing me please !!!

s1.PNG
0
CristianC172
CristianC172

Question 1 year ago

I can't talk with tjbot, I only have the TextToSpeach.
pi@raspberrypi:~/tjbot/recipes/conversation $ sudo node conversation.js
verbose: TJBot initializing microphone
verbose: TJBot initializing LED
verbose: TJBot initializing servo motor on PIN 7
verbose: TJBot initializing assistant service
verbose: TJBot initializing speech_to_text service
verbose: TJBot initializing text_to_speech service
verbose: TJBot initializing visual_recognition service
info: Hello from TJBot! My name is Richi.
verbose: TJBot library version v1.5.1
You can ask me to introduce myself or tell you a joke.
Try saying, "Richi, please introduce yourself" or "Richi, what can you do?"
You can also say, "Richi, tell me a joke!"
verbose: TJBot initializing microphone
pi@raspberrypi:~/tjbot/recipes/conversation $

0
zacaintmyname
zacaintmyname

Answer 1 year ago

is the problem that after verbose: TJBot initializing microphone, the command ends? I am facing the exact same problem!

0
CristianC172
CristianC172

Question 1 year ago

Hello, can I use other microphone? Or maybe I Can not use the USB microphone? if maybe exist a form that i can skip the microphone
Thanks

0
CristianC172
CristianC172

Question 1 year ago on Step 6

Hi, I have other problem:
I can't talk or write to Tjbot, when a execute te command "sudo node conversation.js"
"pi@raspberrypi:~/tjbot/recipes/conversation $ sudo node conversation.js
verbose: TJBot initializing microphone
verbose: TJBot initializing LED
verbose: TJBot initializing servo motor on PIN 7
verbose: TJBot initializing Camera
verbose: TJBot initializing assistant service
verbose: TJBot initializing speech_to_text service
verbose: TJBot initializing text_to_speech service
verbose: TJBot initializing visual_recognition service
info: Hello from TJBot! My name is Watson.
verbose: TJBot library version v1.5.1
You can ask me to introduce myself or tell you a joke.
Try saying, "Watson, please introduce yourself" or "Watson, what can you do?"
You can also say, "Watson, tell me a joke!"
verbose: TJBot initializing microphone"
pi@raspberrypi:~/tjbot/recipes/conversation $

he dies, so, in other words, it comes out

0
CristianC172
CristianC172

1 year ago on Step 5

Hello, my name's Cristian and i need you :c
maybe can you be more explicit with "Where're go the line of code"?
because i have a problem that say:
module.js:549
throw err;
^
Error: Cannot find module 'tjbot'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/home/pi/tjbot/recipes/conversation/conversation.js:17:13)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)

Thanks

0
LSPica
LSPica

Reply 1 year ago

sudo npm install should fix it

0
CristianC172
CristianC172

Reply 1 year ago

¡Hey! Thank you very much, really that, ¡Wow! thank you

"pi@raspberrypi:~/tjbot/recipes/conversation $ sudo npm install
> websocket@1.0.28 install /home/pi/tjbot/recipes/conversation/node_modules/websocket
> (node-gyp rebuild 2> builderror.log) || (exit 0)
sh: 1: cannot create builderror.log: Permission denied
npm notice created a lockfile as package-lock.json. You should commit this file.
added 75 packages from 130 contributors and audited 139 packages in 41.97s
found 0 vulnerabilities
pi@raspberrypi:~/tjbot/recipes/conversation $ sudo node conversation.js
verbose: TJBot initializing microphone
verbose: TJBot initializing LED
verbose: TJBot initializing servo motor on PIN 7
verbose: TJBot initializing Camera
verbose: TJBot initializing assistant service
verbose: TJBot initializing speech_to_text service
verbose: TJBot initializing text_to_speech service
verbose: TJBot initializing visual_recognition service
info: Hello from TJBot! My name is Watson.
verbose: TJBot library version v1.5.1
You can ask me to introduce myself or tell you a joke.
Try saying, "Watson, please introduce yourself" or "Watson, what can you do?"
You can also say, "Watson, tell me a joke!"
verbose: TJBot initializing microphone"

I need now fix the microphone but, not problem, thank you

0
TJBot
TJBot

Reply 1 year ago

Did you run npm install in step 4? That command should install all the modules that you need. Can you confirm that you are able to successfully run npm install in the conversation directory? I'd suggest you delete your current node_modules directory (removes the previous installation) and run npm install again.

0
CristianC172
CristianC172

Reply 1 year ago

HI, me again
I have the same problem...
"pi@raspberrypi:~/tjbot/recipes/conversation $ cp config.default.js config.js
pi@raspberrypi:~/tjbot/recipes/conversation $ npm install
(node:2701) [DEP0022] DeprecationWarning: os.tmpDir() is deprecated. Use os.tmpdir() instead.
pi@raspberrypi:~/tjbot/recipes/conversation $ cp config.default.js config.js
pi@raspberrypi:~/tjbot/recipes/conversation $ nano config.js
pi@raspberrypi:~/tjbot/recipes/conversation $ sudo node conversation.js

verbose: TJBot initializing microphone
verbose: TJBot initializing LED
verbose: TJBot initializing servo motor on PIN 7
verbose: TJBot initializing Camera
verbose: TJBot initializing assistant service
module.js:549
throw err;
^
Error: Cannot find module 'extend'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/home/pi/tjbot/recipes/conversation/node_modules/tjbot/node_modules/watson-developer-cloud/assistant/v1.js:27:14)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)
at TJBot._createServiceAPI (/home/pi/tjbot/recipes/conversation/node_modules/tjbot/lib/tjbot.js:352:27)
at new TJBot (/home/pi/tjbot/recipes/conversation/node_modules/tjbot/lib/tjbot.js:91:18)
at Object.<anonymous> (/home/pi/tjbot/recipes/conversation/conversation.js:36:10)
at Module._compile (module.js:652:30)"

and when I execute "npm install" this's the answer:

(node:2701) [DEP0022] DeprecationWarning: os.tmpDir() is deprecated. Use os.tmpdir() instead.

0
Aurovindhya
Aurovindhya

1 year ago

The sudo node conversation.js ran.. And the output msg was Hello from TJBot!....... Etc..
Then when I gave the voice input.. There was no response and after sometime..this error popped..
Unable to retrieve TTS voices message=socket hang up, stack=Error: socket hang up..
Please reply asap.. And help me..
Thanks in advance..

0
TJBot
TJBot

Reply 1 year ago

Maybe TJBot is hearing an intent that he doesn't understand? Did you say Watson and use one of the intents in the workspace-sample.json to talk to TJBot? I just updated the instructions for that section. Perhaps that solves the problem.

0
rkundra
rkundra

Question 1 year ago on Step 3

what if I use headphones with an inbuilt microphone?

0
JLG20
JLG20

1 year ago

Hi eveyone. I need help, I installed TJBOT into my Raspberry PI 3 (and node js 9), when execute sudo node conversation.js, I receive the following message:

verbose: TJBot initializing microphone
verbose: TJBot initializing LED
verbose: TJBot initializing servo motor on PIN 7
verbose: TJBot initializing assistant service
verbose: TJBot initializing speech_to_text service
verbose: TJBot initializing text_to_speech service
verbose: TJBot initializing visual_recognition service
/home/pi/tjbot/recipes/conversation/node_modules/tjbot/lib/tjbot.js:542
throw new Error(
^
Error: Missing authentication credentials for tone_analyzer service: username/password or apikey are required.
at TJBot._createServiceAPI (/home/pi/tjbot/recipes/conversation/node_modules/tjbot/lib/tjbot.js:542:19)
at new TJBot (/home/pi/tjbot/recipes/conversation/node_modules/tjbot/lib/tjbot.js:116:18)
at Object.<anonymous> (/home/pi/tjbot/recipes/conversation/conversation.js:40:10)
at Module._compile (internal/modules/cjs/loader.js:654:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:665:10)
at Module.load (internal/modules/cjs/loader.js:566:32)
at tryModuleLoad (internal/modules/cjs/loader.js:506:12)
at Function.Module._load (internal/modules/cjs/loader.js:498:3)
at Function.Module.runMain (internal/modules/cjs/loader.js:695:10)
at startup (internal/bootstrap/node.js:201:19)
at bootstrapNodeJSCore (internal/bootstrap/node.js:516:3)

0
elvick
elvick

Reply 1 year ago

I have the same error

0
ElmarKleikamp
ElmarKleikamp

Reply 1 year ago

Hi JLG20

Edit with a text editor ...~/Desktop/tjbot/recipes/conversation/config.js
You have to enter the credential for Watson Visual Recognition (apikey, and not the url)
Check in Dashboard of IBM Bluemix website what skills you already have, if it´s not there activate Watson Visual Recognition and look for the apikey there
In this last line change api_key to apikey (api_key is faulty here)
If yo´re done it should work.

0
thilom1
thilom1

1 year ago

Hello,
I build my TJBot like instructed .
It seems to work, he did not answear.
I cn tell him something he reconizes the words, but he dont answear at the USB Speaker.
My Config is:
USB - Speaker, USB Mikrophone,
can someone help me?
Thanks Menne
here is the Console-Log:

erbose: TJBot initializing microphone
verbose: TJBot initializing LED
verbose: TJBot initializing servo motor on PIN 7
verbose: TJBot initializing Camera
verbose: TJBot initializing assistant service
verbose: TJBot initializing speech_to_text service
verbose: TJBot initializing text_to_speech service
info: Hello from TJBot! My name is Watson.
verbose: TJBot library version v1.4.0
You can ask me to introduce myself or tell you a joke.
Try saying, "Watson, please introduce yourself" or "Watson, what can you do?"
You can also say, "Watson, tell me a joke!"
verbose: TJBot initializing microphone
info: TJBot heard: what %HESITATION telling a joke
info: TJBot heard: what do you tell me a joke
info: TJBot heard: what can you do
info: TJBot heard: what please introduce your
info: TJBot heard: what
info: TJBot heard: please interviews you also
info: TJBot heard: and what he did leave the joke

0
ElmarKleikamp
ElmarKleikamp

Reply 1 year ago

Hi thilom1,

check with aplay -l the parameters of your soundcard (generally it is plughw:1,0)
edit ~/Desktop/tjbot/recipes/conversation/node_modules/tjbot/lib/tjbot.js

look for microphone and edit the parameter there to what aplay -l showed
look for speaker and edit the parameter there to what aplay -l showed

In my settings for the microphone it was 1,0 and the mic worked fine.
But the speaker was set with 0,0 and it didn´t work.
I changed it to 1,0 and everything worked fine.

Good luck

0
YuxingW
YuxingW

Question 1 year ago

Hi, can someone help me with the credentials for the TJBot. I can't find the 36 digit username and 12 character password. Can someone tell me where it is. I have an attached photo of what my credential page look like on IBM cloud. Thanks!!!!

CREDENTIAL.jpg