Introduction: Home Automation Using Raspberry Pi Matrix Voice and Snips
Home automation allows us to control household electrical appliances like light, door, fan, and so on. Home automation not only refers to reduce human efforts but also energy efficiency and time saving.
I modified this project and will add some more features soon
The aim of the project is to implement home automation using Raspberry Pi, Matrix Voice and Snips platform. In this project, Raspberry Pi is connected with Matrix Voice via GPIO pins and all the five lights (such as bedroom, kitchen lights) which we are controlling through voice command are interfaced with expansion GPIO pins of Matrix Voice. We are providing input(voice command) through Matrix voice mic and taking output (response) as light on/off from GPIO of Matrix Voice.
1. Setup Matrix OS on Personal computer and RPi.
2. Setup Matrix core on RPi so that we can install Snips
3. Installed Snips in RPi using client tool on personal computer.
4. Make an app on Snips platform and deploy it in RPi.
5. Make a four scripts and check it.
Step 1: Component Needed
Step 2: How to Connect Matrix Voice With RPi 3
The Matrix Voice is a development board for building sound driven behaviors and interfaces. It is an user-friendly tool for simple to complex Internet of Things (IoT) voice app creation. It comes under two versions
1. Matrix Voice version
2. Matrix Voice ESP32 version
Both versions of these boards run the same on a Raspberry Pi, however, the Matrix Voice ESP32 version has the option to run standalone by programming the ESP32 module.
Here I am using Matrix Voice
Matrix Voice expansion GPIO
DF## pins are equivalent to GPIO## pins.
Raspberry Pi GPIO
Note:- The expansion GPIOs on the Matrix Voice are not connected to the RPi GPIOs. The expansion GPIOs mostly come from the FPGA.
Before installing Matrix OS make sure you have setup Matrix voice. For that you need
- Raspberry Pi 3 Model B+5V
- 2.5A Micro USB Power Supply
- Matrix voice
1. Insert flashed micro SD card into Raspberry Pi
2. Attach Matrix Voice onto Raspberry Pi GPIO pins
3. Power Raspberry Pi with micro USB power supply
Once done next is installing Matrix OS on your personal computer. You can use Windows or macOS or Linux. In this project I am going to use Linux (Ubuntu) to install Matrix OS
Step 3: Installation of Matrix OS on Ubuntu
Before starting installing make sure you have installed NodeJS and npm
1. To install NodeJS, go to NodeJS and download Linux binaries(x64) of version 10.15.1 and then extract it with
tar -xvf node-v10.15.1-linux-x64.tar.xz <br>cd node-v10.15.1-linux-x64 sudo cp -R * /usr/local/ sudo apt-get update sudo apt-get upgrade
and check NodeJS and npm version
node -v npm -v
Now we are ready to install Matrix OS
2. MOS contains a command line interface tool for controlling and managing your Matrix devices. To install it execute the following command in your terminal.
sudo npm install -g matrix-cli
3. Client needed to be configured by registering and then logging into a Matrix Labs account.
To register to the matrix account use command
and then write username and password
For login use command
4. With an account logged into the CLI tool, you can now register your matrix device. This will prompt you to enter a name and description for the device.
matrix register device
After the device is created, a set of unique login credentials will be generated. Save your device id and device secret as you will not be able to see the secret for this device again.
Now we have successfully installed MOS on our machine next step is RPi setup.
Step 4: RPi Setup
Before starting installation make sure you have installed NodeJS and NPM.
tar -xvf node-v10.15.1-linux-armv7l.tar.xz
sudo cp -R * /usr/local/
Then update the npm
sudo npm install npm@latest -g
1. Access the terminal of your Raspberry Pi via SSH or directly access it by connecting a screen, mouse, and keyboard. Then run the following commands to go into your home directory and install MOS on the Raspberry Pi.
cd ~/ curl <a href="https://raw.githubusercontent.com/matrix-io/matrix-creator-quickstart/master/install.sh" rel="nofollow"> https://raw.githubusercontent.com/matrix-io/matri...</a> | sh
After installing it will automatically reboot RPi.
2. Linking Device To Matrix Labs Account
To link your Matrix device to your Matrix Labs account, you need to create a file name.envrc in the home directory of the Raspberry Pi and then paste the device credentials you saved earlier while registering new device.
~/.envrc file look like this
export MATRIX_DEVICE_ID=dc7a2a81bf24 export MATRIX_DEVICE_SECRET=08621018e9d77h15i5n0t4r3alz0f06cd4f7e5544272u
3. Once file is created you need to make the credentials visible to the shell by running the following command. MOS will then be able to read it and properly register your Matrix device.
4. Start MOS with the following command
If you see matrix OS is running you are done to check everything. Go back to your desktop.
Step 5: Check Everything Works
1. Open the terminal on your Desktop and write the following command.
matrix use YOUR_DEVICE_NAME_OR_ID
This selects the Matrix device you want to interact with.
2. Once your device is selected, run these commands to verify that everything has installed properly. We are going to use hello world example Make sure node ~/matrix-os/index.js is active on RPi.
matrix install HelloWorld matrix start HelloWorld
After running the start command you will see rainbow LED sequence.
To stop the app use command.
matrix stop HelloWorld
Now we are done with installation of MOS on RPi. Next step is setup RPi for Snips
Step 6: Matrix Core Installation on RPi
1. Add the Matrix repository and key.
curl <a href="https://apt.matrix.one/doc/apt-key.gpg" rel="nofollow"> https://apt.matrix.one/doc/apt-key.gpg </a> | sudo apt-key add -
echo "deb <a href="https://apt.matrix.one/raspbian" rel="nofollow"> https://apt.matrix.one/raspbian </a> $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/matrixlabs.list
sudo apt-get update
sudo apt-get upgrade
2. Install the the Matrix CORE packages.
sudo apt-get install matrixio-malos
3. To install ZeroMQ
echo "deb <a href="http://download.opensuse.org/repositories/network:/messaging:/zeromq:/release-stable/Debian_9.0/" rel="nofollow"> http://download.opensuse.org/repositories/network...</a> ./" | sudo tee /etc/apt/sources.list.d/zeromq.list
wget <a href="https://download.opensuse.org/repositories/network:/messaging:/zeromq:/release-stable/Debian_9.0/Release.key" rel="nofollow"> https://download.opensuse.org/repositories/networ...</a> -O- | sudo apt-key add
Create a node project folder in the home directory of RPi
cd ~/ mkdir js-matrix-core-app (whatever name you want)
5. Installing npm Packages for ZMQ and Protocol Buffers
To install the ZMQ and MATRIX Protocol Buffers npm packages. Make sure you are in folder (name you give). Here I gave it js-matrix-core-app. This allows you to interact with Matrix Core through Node.js.
npm install zeromq --save npm install matrix-protos --save
Now you are done with Matrix core
Step 7: What Is Snips
Snips is an AI voice platform for connected devices. It is an end-to-end solution with Speech Recognition and Natural Language Understanding, enabling anyone to add a customizable voice assistant to their product. Snips is the only deep-learning based voice platform that runs fully on-device. It can run without the need for a server or the cloud. It offers Privacy by Design.
Step 8: How to Install the Snips Using SAM CLI Tool
For this make sure all the steps mentioned above are working
I. Raspberry Pi
1. To Install the matrix Kernel Modules. Run the following commands in your Raspberry Pi's terminal to add the matrix repository & key and update your repository packages.
curl <a href="https://apt.matrix.one/doc/apt-key.gpg"> https://apt.matrix.one/doc/apt-key.gpg </a> | sudo apt-key add -
echo "deb <a href="https://apt.matrix.one/raspbian"> https://apt.matrix.one/raspbian </a> $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/matrixlabs.list
sudo apt-get update
sudo apt-get upgrade
2. To Install the Matrix Kernel modules. Use the command below
sudo apt install matrixio-kernel-modules
This allows the microphones on your matrix device to register as an ALSA microphone on your Raspberry Pi.
3. Next command will test your microphones by raising your output volume, creating a 5 second long audio recording, and then playing that recording. Make sure you have select audio jack as output for that use command
then blue screen appears, go to advanced option then go to audio and select analog jack.
amixer set PCM 100% && arecord recording.wav -f S16_LE -r 16000 -d 5 && aplay recording.wav
If you are able to hear the audio then you are done.Next we are going to install the SAM tool on desktop
II.Installing the SAM CLI Tool on Personal Computer(Ubuntu)
We are using the SAM CLI tool to create, manage and deploy our Snips assistants, to configure hardware, to view the logs of assistants on the Raspberry Pi, as well as to run your application locally for quick prototyping.
1. Install SAM into your computer with the following command.
sudo npm install -g snips-sam
2. Installing Snips.ai through SAM
We are going to install Snips.ai on our Raspberry Pi, Snips will be installed through the SAM CLI tool from our personal computer.
This command should log all of the Raspberry Pis on your network with their IP address and hostname.
Note:- sam devices not required if you already know your RPi IP address Connect to your Raspberry Pi using command
sam connect ip address of Pi
you'll then be prompted to insert a username and password of RPi.
After you are connected use the init command from your personal computer to install Snips onto your Raspberry Pi.
sudo sam init
III. Configuring Snips.ai on Raspberry Pi
Move on to your Pi with Snips.ai properly installed, the next step is to edit the snips.toml file for configuring the mics. Open terminal and use command to open the snips.toml file
sudo nano /etc/snips.toml
Scroll down to where you see [snips-audio-server] and replace
# mike = "Built-in Microphone"<br>with the following: mike = "MATRIXIO SOUND: - (hw:2,0)"
IV. Testing Snips.ai on personal Computer(Ubuntu)
Through your personal computer's terminal, you can use SAM to test the speaker and microphone of your Raspberry Pi.
1. To Test Raspberry Pi's speaker
sam test speaker
2. To Test the microphones on your Matrix device
sam test microphone
If both commands seem to work, you've properly configured Snips.ai on your Raspberry Pi.
If there is some problem use this command
sam setup audio
Then it will ask whether you are using snips kit or not (I choose no) Then it will ask to select microphone. Here I am using Matrix Voice microphone Then it will ask to select speaker. Here I am using RPi audio jack as speaker.
To test Snips.ai is properly working by installing a demo assistant for the weather or not.
sam install demo
After installing as you see snips assistant is running. Now, test the assistant by saying
"Hey Snips!" "What will the weather be like in Madagascar in two days?"
The response from Snips should be
"You asked for the weather in Madagascar in two days."
Congratulations we have successfully installed Snips on RPi.
Step 9: Voice Control Your Lights With Snips.ai
Making an app on Snips console
1. Creating A Snips Assistant & App
Sign into your Snips.ai and create an assistant. After sign in you page will like this
2. Create a new assistant
give name to an assistant, select language and then create it.Once created you will see page like this
3. Then click on add an app
Then select create a new app and give it a name whatever you want and create it
Once created you will see page like this
4. After that open the app, click on edit app.
You will see page like this. As you see I already created intent (lights). You will see no intent on your page.
5. Create a new intent
After creating you will see page like this.
6. Give training examples
7. Then create a new slots and give it name whatever you want. Here I am using three slots
- switch for on and off state
- room for different rooms such as bedroom, bathroom etc
- device for devices such as light, fan etc
8. Then give slot type name. Here I am building custom slot type.
For room slot
After that add slot values "bathroom", "bedroom" and "kitchen"
For switch slot
After that add slot values "on" and "off".
- For device slot
Give any name whatever you want.
After that add slot values such as "light", "fan" etc
9. Close the app, in order to tell which is slot double click on off and on and select slot name(you provided earlier). Then save it and you can test it by deploying it
Then save it and you can test it in window provided right hand side of the page.
Then deploy it
Step 10: Connect SAM to Your Raspberry Pi on Personal Computer
1. From your computer's terminal, sign in through the Sam CLI Tool
Login using your snips account
2. Connect to your Raspberry Pi. When prompted for a username & password, you can press the enter key to insert the Raspberry Pi credentials.
sam connect YOUR.PI.IP.HERE
3. Copy the cmd given in snips console which deploying an app and paste in your terminal of your computer
Step 11: Creating Assistant on RPi
Assistant will be used to listen and respond to events from your Snips assistant
touch assistant.js relay.js everloop.js func.js
2. Install MQTT to listen in on events from Snips.
npm install mqtt --save
3. Open everloop.js using any editorand paste the code given in program. Code is used to control the LEDs on the MATRIX Creator/Voice.
4. Open relay.js using any editor and paste the code given in program. It is used to turn the relay on and off.
5. Open assistant.js using any editor and paste the code given in program. It combines all the code from everloop.js & relay.js to control the everloop and relay switch through voice. It shows you how to listen to lightState intents and read slots.
var snipsUserName = 'ruchir'; // username of snips account
var lightState = 'hermes/intent/'+snipsUserName+':lightState'; // name of intent you give while creating an intent
Once done now it is time to check.Open your personal computer(Ubuntu) terminal and run
On the other hand, open RPi terminal and go to the directory
node assistant.js // run it
Step 12: Connections
Relay ==> Matrix Voice
- 5V ==> 5V
- GND ==>GND
- Signal1 ==> GPIO0 (DF0)
- Signal2 ==> GPIO1 (DF1)
- Signal3 ==> GPIO2 (DF2)
- Signal4 ==> GPIO3 (DF3)
- Signal5 ==> GPIO5 (DF5)
Step 13: Result
Participated in the