In the past months, I've tested many vocal assistants. I've since come to the conclusion that relying on centralized servers housed by Google and Amazon for simple tasks such as turning on a light or closing my blinds, is, to put it lightly, an incredibly absurd notion. I also live in France. It doesn't make sense that my information pass the Atlantic ocean to flush a toilet at my home in Paris. Privacy implications are equally an evident concern when Amazon or Google has access to all vocal request. Regardless of the old tired argument that "I have nothing to hide" it is no secret that these entities are adept to using personal data for targeted marketing purposes. The services of Google and Amazon are not truly free in the sense of freedom. For simplicity's sake we often forfeit our privacy for these services. They are equally very elegant services, admittedly. But again you have nothing to hide, right?
This is a tutorial to make a simple vocal assistant that could work offline, meaning all the request you do will be processed locally in your home. To do this we will be using Snips, it is a French startup that has the interesting advantage to work offline. To keep things clear and simple we are going to make a very dumb assistant, simple enough to understand the architecture so later make you will make your own more interesting version. So today our assistant will just sum two numbers you will say and playback the answer: You will ask: “how much is 1 plus 2” It will answer: “3”
Affiliation: I’m not affiliated to SNIPS.AI, but I’m making a sound card for raspberry pi that has everything in one RASPIAUDIO.COM it’s a Hat DAC+Speakers+microphone+button and led, you can choose to use my sound card or use your own sound card).
Teachers! Did you use this instructable in your classroom?
Add a Teacher Note to share how you incorporated it into your lesson.
Step 1: What You Need to Know Before Starting
At the end of this tutorial you will have:
- A clear understanding of the architecture of vocal assistant
- You will understand the benefit of an offline assistant
- You will know the vocabulary useful for all assistant
- You will own a working but useless-dumb assistant able to do some additions
- You will have I hope the wish to make a useful or useless assistant but that protects your privacy
What you need:
- Basics understanding of Raspberry PI and Linux command lines
- Time: about 20mn to 1h depending on your skills Internet connection to create the assistant, then later your assistant will be able to work offline!
- Raspberry PI 3 or 3b+ previous version might work too, Zero might be quite slow
- Power supply Keyboard, mouse, and screen, power supply for raspberry
- PC or Mac running on Linux to remotely control the raspberry PI
- Audio Shield with speakers we will use here the Raspiaudio M IC+ , it’s a tiny hat with everything onboard Microphone, DAC, amplifier, speaker, push the button and led. It is also possible to use the raspberry built-in audio(so-so quality), external amplified speakers and an external USB microphone.
Process overview :
We will first create the assistant online on the assistant then we will install it on the Raspberry PI. In this approach we use a mouse, keyboard to setup to the Raspberry, then your computer on Linux to remotely install and configure the assistant reusing a tool called SAM.
Step 2: Register
On your Linux computer go to https://console.snips.ai/ and register, create a new assistant:
Step 3: Setup the Assistant
Give it a name, choose your language and click on create
Step 4: Select the Wake Up Word (also Called Hot Word) “Hey Snips” and Add a Skill
Step 5: Create a Skill
You will see some pre-made skills shared by others a lot of them are not working so at
the end it’s much more efficient and fun to make our own, click on “create new skill”
Step 6: Then Add a Description and Click on Create:
Step 7: Click on Edit Skill:
Step 8: Create a New Intent
For the next screen you will need of a bit of vocabulary:
- “skills” are functions to do, here the ability to sum number1 + number2
- “intents”: are the requests you will say out loud to accomplish this skill, for example, you can have “how much is 1 plus 2” and many others, the more you have the more naturally you will be able to address to your assistant.
- “slots” are variables part in your request here number1 and number2
- Actions: what to do, the actual sum of number1 + number2 then say the result
Step 9: We Will Do a Simple Sum of NumberOne + NumberTwo:
Step 10: Indentify Slots
Identify our 2 variables as slots, specify the type “Number”, and check the button “slot required” it will say this sentence if one of the slots was not heard correctly:
Step 11: Teach Him Where Are the Slots
Now we need to teach the assistant different ways to understand this intent, the more you type in the better it is to address to your assistant naturally, once you have typed the questions you need to identify your slots (variable) by double-clicking on “one” and “two” and selecting the slots.
Once you are done click on “Save”, then come back on the previous screen: “Home>My
Dumb assistant>My calculator”
Step 12: Time for Some Action!
At that point the assistant will understand the question by identifying the number1 and number 2, but what to do with these two numbers. We will write our own snippets, click on Action-Code Snippets
Step 13: Type the Script
Here is the Python script we will cut and paste, There is nothing to save, after this we are done with the assistant! copy/paster from here:
#get the first value and second value and convert it to an integer
A = int(intentMessage.slots.numberOne.first().value)
B = int(intentMessage.slots.numberTwo.first().value)
#Sum A + B
C = A + B
#Convert the result in a string
resul = ‘the answer is’ + str(C) + ‘my dear master’
#the session is the question + answer, it will say the result and close the session
current_session_id = intentMessage.session_id hermes.publish_end_session(current_session_id, resul)
Step 14: Burn Raspbian Stretch Lite on a Fresh SD Card
(If you are not familiar with the process see here
Step 15: Connect Your Raspberry
- Setup your keyboard layout (if not qwery), setup the wifi using localisation options:
- Choose Network Options to set your wifi SSID/password, or just connect an ethernet cable to your router
- Choose Interfacing Options to Enable ssh (as it will be used later)
Step 16: Take Note of Your Raspberry's IP
Check that you are connected to your router type ifconfig and take note of the IP address
Step 17: Install the Raspiaudio MIC+ Sound Card
If you have a MIC+ from RASPIAUDIO.COM plug the card and type:
sudo wget -O mic mic.raspiaudio.com
sudo bash mic
● Reboot, then test :
udo wget -O test test.raspiaudio.com
sudo bash test
● Push on the yellow button you should hear “front left, front right” then a recording will be played indicating that the mic and speakers are working well.
Step 18: Install With the Assistant on the Raspberry From Your Linux PC/MAC
- Install sam on your Linux computer:
sudo npm install -g snips-sam
- One finished log with your credential you have created earlier on snips.ai
- sam login
- Connect to your raspberry pi with Sam:
- sam connect "ip_address_of raspberry"
- sam connect "ip_address_of raspberry"
- Start the installer on the Raspberry Pi using:
- sam init
- You will get: “Installing Snips Platform components. This may take a few minutes... Successfully installed Snips Platform components” ... it will take a while to finish
- To install the assistant type :
- sam install assistant
- Type the following to see the console:
- Sam watch
- Try it! Say:
- “Hey Snips” you hear a beep then
- “how much is 20 plus 22?” then it should reply back “42”
- If it does not work:
- Check the status of SAM using:
- sam status
- Adjust the microphone and speakers gain on the Raspberry Pi:
- Check the status of SAM using: