Raspberri Personal Assistant.
Intro: Raspberri Personal Assistant.
Last summer, in a small store, I found half of a 1950s bakelite Televox intercom . I thought that it would make a nice cast for a project and I bought it as it only cost $9. It sat on a shelf until I bought my first raspberry pi. After I had played with the pi for some days, I was struck by the idea that the intercom would be ideal in combination with the pi, to build a voice controlled device.
Because of the intercom, the idea came that it would be funny to use the intercom for what it was intended: contacting your personal assistant to get information or to give him/her a task. The only difference is that this intercom isn't connected to its other half but to the raspberry pi. And that there is no real person on the other side, but a smart little computer that can do a lot of the same things as that real person.
In this instructable, I'll show you:
Because of the intercom, the idea came that it would be funny to use the intercom for what it was intended: contacting your personal assistant to get information or to give him/her a task. The only difference is that this intercom isn't connected to its other half but to the raspberry pi. And that there is no real person on the other side, but a smart little computer that can do a lot of the same things as that real person.
In this instructable, I'll show you:
- how to connect the pi to the intercom
- how to set up the pi
- how to install the voicecontrole software
- how to add a speech script that will return the output of your own scripts as speech
STEP 1: What Do You Need for This Project?
For this project we need:
- A Raspberry pi model B with Raspian Wheezy installed.
- A USB WIFI adaptor: I used a Delock Nano Stick 150MB/s. It cost about $18
- A USB soundcard: it cost $3.21 at Deal Extreme.
- A small 5V amplifier: I used a PAM8403 board. It cost $5.64 at Deal Extreme.
- A 5V power supply: at least 2A.
- A 5V DPDT relais: DPDT means Double Pole Double Throw. The relais is able to switch two different signals between two different outputs.
- A tiny piece of vero board
- Some wires
- A USB keyboad and mouse + a tv with HDMI or video in connection for installing everything on the Pi
STEP 2: Installing the Wifi.
In this step , I'll show you how to connect to the internet via the raspberry desktop. But before you do anything, update your raspberry. Make sure that your Pi is connected to the internet via a cable.
To update, start-up the raspberry and on the desktop, open the terminal. Enter the following commands in the terminal.
sudo apt-get update
sudo apt-get upgrade
sudo poweroff
To update, start-up the raspberry and on the desktop, open the terminal. Enter the following commands in the terminal.
sudo apt-get update
sudo apt-get upgrade
Once this is done shut down the raspberry:
sudo poweroff
Now insert the USB WIFI adaptor and turn on the power again.
On the desktop, double click the Wifi Config Icon. A window will open.
Click the scan-button which will again open a new window.
In that window, select your SSID by double-clicking it.
A window appears and you should add the password where it says PSK. Then click Add.
You will be back in the first window. There you can see the status of the connection at the IP.
You should now have a Wifi connection.
You can shut the Pi down for now.
On the desktop, double click the Wifi Config Icon. A window will open.
Click the scan-button which will again open a new window.
In that window, select your SSID by double-clicking it.
A window appears and you should add the password where it says PSK. Then click Add.
You will be back in the first window. There you can see the status of the connection at the IP.
You should now have a Wifi connection.
You can shut the Pi down for now.
STEP 3: Setting Up the Hardware.
Inside the intercom, there is not enough space to hide everything, when the whole setup is done with jacks to connect the audio. So to address this problem, I opened the plastic case of the USB sound card and desoldered the connectors. Then I soldered wires to the spots where the connectors were. The connection of these wires will be discussed later in this step.
Most intercoms use the speaker as microphone. So this means that we have to devise a method to connect the same speaker to the output and the microphone input of the USB sound card. There is a push button in the front that was used originally to switch between listen and speak, so that can be of use to us.
It has to be said that I didn't want to change anything on the outside of the intercom because it's such a nice thing in great condition. Drilling holes in it was no option. Otherwise I could simply have added a DPDT switch to switch between input and output. So I had to find the next simplest method.
This is where the 5V DPDT relais comes in. We can use the push button of the intercom to switch it. To do this, we connect 5V to the push button and the second wire of the button goes to one side of the relay coil. The other side of the coil goes directly to ground. So when you press the button, the coil is activated and the relay switches. It is good practice to add a diode in the opposite direction over the 2 pins of the coil.
The center-pins of the DPDT relays are connected to the + and - of the speaker. The NO (Normally Open) pins are connected to the microphone input of the USB soundcard. The NC (Normally Closed) pins need to be connected to the output of the sound-card. But the signal of that output is not loud enough for the speaker so we need to put an amplifier between them.
The Amplifier I used, is quite simple to hook up. Everything is clearly marked on the PCB. Connect the 5V pin to 5V and the GND pin to GND and you have power. Then connect the output and GND from the soundcard to Rin and GND and connect R+ and R- to the NC pins of the relay.
When this is done, everything can be placed into the case. Connect the sound-card to the pi.
It is not a very good idea to try to power everything from the raspberry pi. So I found a 5V 3A wall adaptor to power this project. I did cut of the connector and connected everything inside to it. To do so I had to cut a USB cord. That allowed me to connect to the pi without having to desolder the onboard USB connector.
When everything is hooked up this way, your hardware is set up and the case can be closed.
Most intercoms use the speaker as microphone. So this means that we have to devise a method to connect the same speaker to the output and the microphone input of the USB sound card. There is a push button in the front that was used originally to switch between listen and speak, so that can be of use to us.
It has to be said that I didn't want to change anything on the outside of the intercom because it's such a nice thing in great condition. Drilling holes in it was no option. Otherwise I could simply have added a DPDT switch to switch between input and output. So I had to find the next simplest method.
This is where the 5V DPDT relais comes in. We can use the push button of the intercom to switch it. To do this, we connect 5V to the push button and the second wire of the button goes to one side of the relay coil. The other side of the coil goes directly to ground. So when you press the button, the coil is activated and the relay switches. It is good practice to add a diode in the opposite direction over the 2 pins of the coil.
The center-pins of the DPDT relays are connected to the + and - of the speaker. The NO (Normally Open) pins are connected to the microphone input of the USB soundcard. The NC (Normally Closed) pins need to be connected to the output of the sound-card. But the signal of that output is not loud enough for the speaker so we need to put an amplifier between them.
The Amplifier I used, is quite simple to hook up. Everything is clearly marked on the PCB. Connect the 5V pin to 5V and the GND pin to GND and you have power. Then connect the output and GND from the soundcard to Rin and GND and connect R+ and R- to the NC pins of the relay.
When this is done, everything can be placed into the case. Connect the sound-card to the pi.
It is not a very good idea to try to power everything from the raspberry pi. So I found a 5V 3A wall adaptor to power this project. I did cut of the connector and connected everything inside to it. To do so I had to cut a USB cord. That allowed me to connect to the pi without having to desolder the onboard USB connector.
When everything is hooked up this way, your hardware is set up and the case can be closed.
STEP 4: Setting Up the USB Sound Card.
The raspberry pi needs a little bit of preparation to use a USB sound-card. There is a very good adafruit tutorial for this.
I'll give you the short version here. My sound-card was a CM109 version so the only thing I had to do was to open the alsa config file:
sudo nano /etc/modprobe.d/alsa-base.conf
And change:
options snd-usb-audio index=-2 into:
sudo reboot
I'll give you the short version here. My sound-card was a CM109 version so the only thing I had to do was to open the alsa config file:
sudo nano /etc/modprobe.d/alsa-base.conf
And change:
options snd-usb-audio index=-2
options snd-usb-audio index=0
And then reboot the pi: sudo reboot
STEP 5: Setting Up the Voicecommand Software.
I used Steven Hicksons voice command software. If you are looking for voice command software for your pi, stop looking and use this one. It is by far the easiest to setup and to use. It is also very well documented and it is quite reliable.
You can download it from his website and there are also some videos there that show you how you can set it up.
The voice-command software comes as a part of a whole suite of media related applications for the pi. I only installed the voice-command software, but if you like to use more of them for your project, then feel free to install them.
When the software is installed, you can enter the following command:
voicecommand -s
It will start an automatic setup process that guides you step-by-step through the setup of your device.
When this is done, you can enter:
voicecommand -e
This will open the config file and allow you to set the commando's and the related actions.
It has to be entered in the following format:
command==action
The command system is pretty simple. When you set the command to play, it will only react when you say nothing else but 'play'. When you set it to play music it will only react to 'play music' and not to 'play' or 'play chess'. But when you set it to ~play it will react to any sentence that contains 'play'.
The action can be any executable command.
When you are done editing the config file, save it and everything is ready.
To run the software just enter:
You can download it from his website and there are also some videos there that show you how you can set it up.
The voice-command software comes as a part of a whole suite of media related applications for the pi. I only installed the voice-command software, but if you like to use more of them for your project, then feel free to install them.
When the software is installed, you can enter the following command:
voicecommand -s
When this is done, you can enter:
voicecommand -e
It has to be entered in the following format:
command==action
The action can be any executable command.
When you are done editing the config file, save it and everything is ready.
To run the software just enter:
voicecommand -c
and enjoy your personal assistant. The good lady will now listen to your commands and perform the appropriate task. When she doesn't find a matching task to a command you gave, she will look it up on Google and she will try to give you a satisfying answer. You can finish the project now but in the next steps I will show you how to get more out of it.
STEP 6: Make It Run on Startup.
It is ofcourse nice if the voice-command starts to work on start up. To do this we need to add a little script.
In the home directory (home/pi), open the .config directory
cd .config
then create a new directory:
mkdir autostart
go into that directory:
cd autostart
and in that directory open a new file in nano:
sudo nano voicecommand.desktop
Write the following code in the file:
Save the file and reboot the Pi.
In the home directory (home/pi), open the .config directory
cd .config
then create a new directory:
mkdir autostart
go into that directory:
cd autostart
and in that directory open a new file in nano:
sudo nano voicecommand.desktop
Write the following code in the file:
[Desktop Entry]
Type=Application
Name=Voicecommand
Exec=voicecommand -c
StartupNotify=false
Type=Application
Name=Voicecommand
Exec=voicecommand -c
StartupNotify=false
Save the file and reboot the Pi.
STEP 7: Give Your Own Apps a Voice.
It's all very nice that the device is able to start programs and return info from the internet, but sometimes you want to receive very specific data in a certain way. For instance the weather forecast for your area. One thing you can do(as I did) is to write your own python script that gets the data from the internet, filters out the data you want and returns it in a way that makes sense. The only problem is that those scripts return that data as written text and not as speech. So we need an extra script to do that for us.
I found a very nice script on Oscar Liang's blog. Just like voicecommand, it uses Google speech service to 'translate' the text into speech.
The first one you need is the text2speech.sh script. (I'll copy it here but all credits go to Oscar Liang)
copy it into nano or the text editor on the graphic desktop
#!/bin/bash
INPUT=$*
STRINGNUM=0
ary=($INPUT)
for key in "${!ary[@]}"
do
SHORTTMP[$STRINGNUM]="${SHORTTMP[$STRINGNUM]} ${ary[$key]}"
LENGTH=$(echo ${#SHORTTMP[$STRINGNUM]})
if [[ "$LENGTH" -lt "100" ]]; then
SHORT[$STRINGNUM]=${SHORTTMP[$STRINGNUM]}
else
STRINGNUM=$(($STRINGNUM+1))
SHORTTMP[$STRINGNUM]="${ary[$key]}"
SHORT[$STRINGNUM]="${ary[$key]}"
fi
done
for key in "${!SHORT[@]}"
do
say() { local IFS=+;/usr/bin/mplayer -ao alsa -really-quiet -noconsolecontrols "http://translate.google.com/translate_tts?tl=en&q=${SHORT[$key]}"; }
say $*
done
Once you have saved this script, make it executable with:
This script will be reusable for all applications that need conversion to speech.
The only thing we need now are scripts that run the python script and pump the output into thetext2speech script. We'll need one of these scripts for each python script.
#!/bin/bash
ANSWER=$(python yourpythonscriptname.py)
./text2speech.sh $ANSWER
Save this as whatever-you-like.sh and make it executable.
now you can enter this script into the config file of voicecommand and it will return speech whenever the right command is entered.
I found a very nice script on Oscar Liang's blog. Just like voicecommand, it uses Google speech service to 'translate' the text into speech.
The first one you need is the text2speech.sh script. (I'll copy it here but all credits go to Oscar Liang)
copy it into nano or the text editor on the graphic desktop
#!/bin/bash
INPUT=$*
STRINGNUM=0
ary=($INPUT)
for key in "${!ary[@]}"
do
SHORTTMP[$STRINGNUM]="${SHORTTMP[$STRINGNUM]} ${ary[$key]}"
LENGTH=$(echo ${#SHORTTMP[$STRINGNUM]})
if [[ "$LENGTH" -lt "100" ]]; then
SHORT[$STRINGNUM]=${SHORTTMP[$STRINGNUM]}
else
STRINGNUM=$(($STRINGNUM+1))
SHORTTMP[$STRINGNUM]="${ary[$key]}"
SHORT[$STRINGNUM]="${ary[$key]}"
fi
done
for key in "${!SHORT[@]}"
do
say() { local IFS=+;/usr/bin/mplayer -ao alsa -really-quiet -noconsolecontrols "http://translate.google.com/translate_tts?tl=en&q=${SHORT[$key]}"; }
say $*
done
Once you have saved this script, make it executable with:
chmod +x text2speech.sh
This script will be reusable for all applications that need conversion to speech.
The only thing we need now are scripts that run the python script and pump the output into thetext2speech script. We'll need one of these scripts for each python script.
#!/bin/bash
ANSWER=$(python yourpythonscriptname.py)
./text2speech.sh $ANSWER
Save this as whatever-you-like.sh and make it executable.
now you can enter this script into the config file of voicecommand and it will return speech whenever the right command is entered.
45 Comments
Dravindr 6 years ago
Hi
I really want to do this project but I need to do this using Matlab with simulink on Raspberry pi kit. Can u please tell me whether it's possible?? If not , please suggest me where can I find projects which can be done using Matlab on Raspberry pi kit!!!
Thanks in advance,
Divya
Dravindr 6 years ago
Hi ,, I really want to make this project but I need to do using Matlab with simulink on Raspberry pi kit. Can u please tell me whether we can do it?? If not please please suggest me how can I get or find info about projects which can be done using Matlab with simulink on Raspberry pi .
Dravindr 6 years ago
I wanted to make a project using Matlab with simulink on Raspberry pi.. can this model be done using Matlab ??if not
Please suggest me how can I find projects which can be done using Matlab on Raspberry pi..
thanks in advance
Classified123 6 years ago
HELP: I followed everything step by step, but it doesn't speak or hear me when I talk into it or do the tests (Voicecommand -s). I am using the same sound card as shown and configured it but it doesn't recognize what I say and i can't hear anything.
aleaiyas. 6 years ago
Hello help me on this: I'm trying to handle gpio ports with voice commands with Steven Hicksons but I can not get them to work only stays in: Opening config file ... Running in quiet mode Not querying for answer Running in continuous mode Verifying command as well Keyword duration is 2 and duration is 3 After that nothing happens I do not execute anything I speak and nothing happens my configuration that realizes is this: Light on == gpio write 7 on Light off == gpio write 7 off Thank you very much for your contribution
janw 6 years ago
it sounds like something is missing or wrong in the config file.
SebastianV49 7 years ago
Thank you for this! I have been working on my own project for a while to make something similar and this as very helpful!
janw 7 years ago
YW
DevinM42 7 years ago
Will this project work with Raspberry Pi 3 with Raspian installed? I have a model B.
cgl236 7 years ago
It will, but the software that he recommends is outdated and not kept up very well. I recommend trying to use the Jasper software and documentation. There is a google group that can be found here. https://groups.google.com/forum/?hl=en#!forum/jasp... The downside to this is that you lose the push button functionality. Currently, I am trying to get the push to talk switch to function along with the jasper format. What are your thoughts @janw?
janw 7 years ago
It will indeed work with a raspi3. But time has moved on and as you way, there are other and better was to building this now. The push button has nothing to do with the software. It only switches the speaker between the mic input and the audio output. It is an external relay that makes it work.
DevinM42 7 years ago
So I will be able to use raspian? Or will I have to download Raspian Wheezy?
cgl236 7 years ago
Raspian will work fine. Follow along the google group.
cgl236 7 years ago
Also, there are a multitude of other modules out there for Jasper to run on.
DevinM42 7 years ago
I would like to make this project and am wondering what kind of wires are needed. On the materials list, it says "some wires", this is not specific enough for my needs.
janw 7 years ago
They are ordinary electrical wires that you can find in old computers and stuff. If you have to buy them then buy AWG 24. It is a good standard gauge wire for electronics.
cgl236 7 years ago
Hello,
I know it's been sometime since someone has asked you a question here. I have attempted this build and currently, it's waiting to be turned on. I have a few questions.
1. Are the capacitors needed on the USB soundcard?
2. How can I check to see if my speaker is working and receiving sound?
3. Does burn damage to the soundcard make a difference? It was my first time with a soldering iron and those plastic pieces are so hard to remove.
4. Could I get a look at the XML code? You said that you were going to post it but I havent seen it in the comments. Thank you for replying, when you get a chance.
janw 7 years ago
2. Easy way to check whether your speaker works is to connect your usb soundcard to your pc and play some audio. to see if the input works you can try to record something.
3. depends on what kind of burn damage.If traces are interrupted or components damaged then you card might not work properly anymore. If they are 'cosmetic' burn marks then it'll be fine.
4. I'll lookup the XML parsing script. It's made in Python and uses an XML parsing library. It's not that difficult to make if you are familiar with python.
hacker-cycle 7 years ago
Can i use a USB mic and the built-in 3.5mm output instead of an external soundcard?
ColeN4 8 years ago
Will this work on Raspbian Jessie?