Introduction: Home Automation Using Raspberry Pi Matrix Voice and Snips

By

Introduction

Home automation allows us to control household electrical appliances like light, door, fan, and so on. Home automation not only refers to reduce human efforts but also energy efficiency and time saving.

I modified this project and will add some more features soon

Light control using snips

The aim of the project is to implement home automation using Raspberry Pi, Matrix Voice and Snips platform. In this project, Raspberry Pi is connected with Matrix Voice via GPIO pins and all the five lights (such as bedroom, kitchen lights) which we are controlling through voice command are interfaced with expansion GPIO pins of Matrix Voice. We are providing input(voice command) through Matrix voice mic and taking output (response) as light on/off from GPIO of Matrix Voice.

1. Setup Matrix OS on Personal computer and RPi.

2. Setup Matrix core on RPi so that we can install Snips

3. Installed Snips in RPi using client tool on personal computer.

4. Make an app on Snips platform and deploy it in RPi.

5. Make a four scripts and check it.

Step 1: Component Needed

    1. Matrix Voice ==> $55

      2. Raspberry Pi 3 B+ ==> $35

      3. 5V 2.5A Power Supply Adapter ==> $4.78

      4. 5V Relay single channel ==> $4

      5. 16 GB SD card ==> $7.68

      Total cost of project is around $ 106

      Step 2: How to Connect Matrix Voice With RPi 3

      The Matrix Voice is a development board for building sound driven behaviors and interfaces. It is an user-friendly tool for simple to complex Internet of Things (IoT) voice app creation. It comes under two versions

      1. Matrix Voice version

      2. Matrix Voice ESP32 version

      Both versions of these boards run the same on a Raspberry Pi, however, the Matrix Voice ESP32 version has the option to run standalone by programming the ESP32 module.

      Here I am using Matrix Voice

      Matrix Voice expansion GPIO

      DF## pins are equivalent to GPIO## pins.

      Raspberry Pi GPIO

      Note:- The expansion GPIOs on the Matrix Voice are not connected to the RPi GPIOs. The expansion GPIOs mostly come from the FPGA.

      Before installing Matrix OS make sure you have setup Matrix voice. For that you need

      • Raspberry Pi 3 Model B+5V
      • 2.5A Micro USB Power Supply
      • Matrix voice

      1. Insert flashed micro SD card into Raspberry Pi

      2. Attach Matrix Voice onto Raspberry Pi GPIO pins

      3. Power Raspberry Pi with micro USB power supply

      Once done next is installing Matrix OS on your personal computer. You can use Windows or macOS or Linux. In this project I am going to use Linux (Ubuntu) to install Matrix OS

      Step 3: Installation of Matrix OS on Ubuntu

      Before starting installing make sure you have installed NodeJS and npm

      1. To install NodeJS, go to NodeJS and download Linux binaries(x64) of version 10.15.1 and then extract it with

      tar -xvf node-v10.15.1-linux-x64.tar.xz <br>cd node-v10.15.1-linux-x64
sudo cp -R * /usr/local/ 
sudo apt-get update 
sudo apt-get upgrade

      and check NodeJS and npm version

      node -v

npm -v

      Now we are ready to install Matrix OS


      2. MOS contains a command line interface tool for controlling and managing your Matrix devices. To install it execute the following command in your terminal.

      sudo npm install -g matrix-cli

      3. Client needed to be configured by registering and then logging into a Matrix Labs account.
      To register to the matrix account use command

      matrix register

      and then write username and password

      For login use command

      matrix login

      4. With an account logged into the CLI tool, you can now register your matrix device. This will prompt you to enter a name and description for the device.

      matrix register device

      After the device is created, a set of unique login credentials will be generated. Save your device id and device secret as you will not be able to see the secret for this device again.

      Now we have successfully installed MOS on our machine next step is RPi setup.

      Step 4: RPi Setup

      Before starting installation make sure you have installed NodeJS and NPM.

      wget https://nodejs.org/dist/v10.15.1/node-v10.15.1-linux-armv7l.tar.xz
      tar -xvf node-v10.15.1-linux-armv7l.tar.xz
      cd node-v10.15.1-linux-armv7l 
      sudo cp -R * /usr/local/

      Then update the npm

      sudo npm install npm@latest -g

      1. Access the terminal of your Raspberry Pi via SSH or directly access it by connecting a screen, mouse, and keyboard. Then run the following commands to go into your home directory and install MOS on the Raspberry Pi.

      cd ~/
curl <a href="https://raw.githubusercontent.com/matrix-io/matrix-creator-quickstart/master/install.sh" rel="nofollow"> https://raw.githubusercontent.com/matrix-io/matri...</a> | sh

      After installing it will automatically reboot RPi.

      2. Linking Device To Matrix Labs Account

      To link your Matrix device to your Matrix Labs account, you need to create a file name.envrc in the home directory of the Raspberry Pi and then paste the device credentials you saved earlier while registering new device.

      nano ~/.envrc

      ~/.envrc file look like this

      export MATRIX_DEVICE_ID=dc7a2a81bf24

export MATRIX_DEVICE_SECRET=08621018e9d77h15i5n0t4r3alz0f06cd4f7e5544272u

      3. Once file is created you need to make the credentials visible to the shell by running the following command. MOS will then be able to read it and properly register your Matrix device.

      source ~/.envrc

      4. Start MOS with the following command

      node ~/matrix-os/index.js

      If you see matrix OS is running you are done to check everything. Go back to your desktop.

      Step 5: Check Everything Works

      1. Open the terminal on your Desktop and write the following command.

      matrix use YOUR_DEVICE_NAME_OR_ID

      This selects the Matrix device you want to interact with.

      2. Once your device is selected, run these commands to verify that everything has installed properly. We are going to use hello world example Make sure node ~/matrix-os/index.js is active on RPi.

      matrix install HelloWorld

matrix start HelloWorld

      After running the start command you will see rainbow LED sequence.

      To stop the app use command.

      matrix stop HelloWorld

      Now we are done with installation of MOS on RPi. Next step is setup RPi for Snips

      Step 6: Matrix Core Installation on RPi

      1. Add the Matrix repository and key.

      curl <a href="https://apt.matrix.one/doc/apt-key.gpg" rel="nofollow"> https://apt.matrix.one/doc/apt-key.gpg </a> | sudo apt-key add -
      echo "deb <a href="https://apt.matrix.one/raspbian" rel="nofollow"> https://apt.matrix.one/raspbian </a> $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/matrixlabs.list
      sudo apt-get update
       sudo apt-get upgrade

      2. Install the the Matrix CORE packages.

      sudo apt-get install matrixio-malos


      sudo reboot

      3. To install ZeroMQ

      echo "deb <a href="http://download.opensuse.org/repositories/network:/messaging:/zeromq:/release-stable/Debian_9.0/" rel="nofollow"> http://download.opensuse.org/repositories/network...</a> ./" | sudo tee /etc/apt/sources.list.d/zeromq.list
      wget <a href="https://download.opensuse.org/repositories/network:/messaging:/zeromq:/release-stable/Debian_9.0/Release.key" rel="nofollow"> https://download.opensuse.org/repositories/networ...</a> -O- | sudo apt-key add

      4. JavaScript setup
      Create a node project folder in the home directory of RPi

      cd ~/

mkdir js-matrix-core-app (whatever name you want)
       cd js-matrix-core-app
       npm init

      5. Installing npm Packages for ZMQ and Protocol Buffers
      To install the ZMQ and MATRIX Protocol Buffers npm packages. Make sure you are in folder (name you give). Here I gave it js-matrix-core-app. This allows you to interact with Matrix Core through Node.js.

      npm install zeromq --save

npm install matrix-protos --save

      Now you are done with Matrix core

      Step 7: What Is Snips

      Snips is an AI voice platform for connected devices. It is an end-to-end solution with Speech Recognition and Natural Language Understanding, enabling anyone to add a customizable voice assistant to their product. Snips is the only deep-learning based voice platform that runs fully on-device. It can run without the need for a server or the cloud. It offers Privacy by Design.

      Step 8: How to Install the Snips Using SAM CLI Tool

      For this make sure all the steps mentioned above are working
      I. Raspberry Pi

      1. To Install the matrix Kernel Modules. Run the following commands in your Raspberry Pi's terminal to add the matrix repository & key and update your repository packages.

      curl <a href="https://apt.matrix.one/doc/apt-key.gpg"> https://apt.matrix.one/doc/apt-key.gpg </a> | sudo apt-key add -
      echo "deb <a href="https://apt.matrix.one/raspbian"> https://apt.matrix.one/raspbian </a> $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/matrixlabs.list

      sudo apt-get update 
      sudo apt-get upgrade 
      sudo reboot

      2. To Install the Matrix Kernel modules. Use the command below

      sudo apt install matrixio-kernel-modules

      This allows the microphones on your matrix device to register as an ALSA microphone on your Raspberry Pi.

      sudo reboot

      3. Next command will test your microphones by raising your output volume, creating a 5 second long audio recording, and then playing that recording. Make sure you have select audio jack as output for that use command

      sudo raspi-config

      then blue screen appears, go to advanced option then go to audio and select analog jack.



      amixer set PCM 100% && arecord recording.wav -f S16_LE -r 16000 -d 5 && aplay recording.wav

      If you are able to hear the audio then you are done.Next we are going to install the SAM tool on desktop

      II.Installing the SAM CLI Tool on Personal Computer(Ubuntu)
      We are using the SAM CLI tool to create, manage and deploy our Snips assistants, to configure hardware, to view the logs of assistants on the Raspberry Pi, as well as to run your application locally for quick prototyping.

      1. Install SAM into your computer with the following command.

      sudo npm install -g snips-sam

      2. Installing Snips.ai through SAM
      We are going to install Snips.ai on our Raspberry Pi, Snips will be installed through the SAM CLI tool from our personal computer.

      sam devices

      This command should log all of the Raspberry Pis on your network with their IP address and hostname.
      Note:- sam devices not required if you already know your RPi IP address Connect to your Raspberry Pi using command

      sam connect ip address of Pi

      you'll then be prompted to insert a username and password of RPi.

      After you are connected use the init command from your personal computer to install Snips onto your Raspberry Pi.

      sudo sam init

      III. Configuring Snips.ai on Raspberry Pi
      Move on to your Pi with Snips.ai properly installed, the next step is to edit the snips.toml file for configuring the mics. Open terminal and use command to open the snips.toml file

      sudo nano /etc/snips.toml

      Scroll down to where you see [snips-audio-server] and replace

      # mike = "Built-in Microphone"<br>with the following:
mike = "MATRIXIO SOUND: - (hw:2,0)"

      IV. Testing Snips.ai on personal Computer(Ubuntu)
      Through your personal computer's terminal, you can use SAM to test the speaker and microphone of your Raspberry Pi.

      1. To Test Raspberry Pi's speaker

      sam test speaker

      2. To Test the microphones on your Matrix device

      sam test microphone

      If both commands seem to work, you've properly configured Snips.ai on your Raspberry Pi.
      If there is some problem use this command

      sam setup audio

      Then it will ask whether you are using snips kit or not (I choose no) Then it will ask to select microphone. Here I am using Matrix Voice microphone Then it will ask to select speaker. Here I am using RPi audio jack as speaker.

      To test Snips.ai is properly working by installing a demo assistant for the weather or not.

      sam install demo

      After installing as you see snips assistant is running. Now, test the assistant by saying

      "Hey Snips!" 

"What will the weather be like in Madagascar in two days?"

      The response from Snips should be 

      "You asked for the weather in Madagascar in two days."

      Congratulations we have successfully installed Snips on RPi.

      Step 9: Voice Control Your Lights With Snips.ai

      Making an app on Snips console

      1. Creating A Snips Assistant & App

      Sign into your Snips.ai and create an assistant. After sign in you page will like this

      2. Create a new assistant

      give name to an assistant, select language and then create it.Once created you will see page like this

      3. Then click on add an app

      Then select create a new app and give it a name whatever you want and create it

      Once created you will see page like this

      4. After that open the app, click on edit app.

      You will see page like this. As you see I already created intent (lights). You will see no intent on your page.

      5. Create a new intent

      After creating you will see page like this.

      6. Give training examples

      7. Then create a new slots and give it name whatever you want. Here I am using three slots

      • switch for on and off state
      • room for different rooms such as bedroom, bathroom etc
      • device for devices such as light, fan etc

      8. Then give slot type name. Here I am building custom slot type.

      • For room slot

      After that add slot values "bathroom", "bedroom" and "kitchen"

      • For switch slot

      After that add slot values "on" and "off".

      • For device slot

      Give any name whatever you want.

      After that add slot values such as "light", "fan" etc

      9. Close the app, in order to tell which is slot double click on off and on and select slot name(you provided earlier). Then save it and you can test it by deploying it

      Then save it and you can test it in window provided right hand side of the page.

      Then deploy it

      Step 10: Connect SAM to Your Raspberry Pi on Personal Computer

      1. From your computer's terminal, sign in through the Sam CLI Tool

      sam login

      Login using your snips account

      2. Connect to your Raspberry Pi. When prompted for a username & password, you can press the enter key to insert the Raspberry Pi credentials.

      sam connect YOUR.PI.IP.HERE

      3. Copy the cmd given in snips console which deploying an app and paste in your terminal of your computer

      Step 11: Creating Assistant on RPi

      Assistant will be used to listen and respond to events from your Snips assistant

      1. Go to the directory which we made during JavaScript setup in Matrix core. cd ~/js-matrix-core-app and create 4 new JavaScript files.

       touch assistant.js relay.js everloop.js func.js

      2. Install MQTT to listen in on events from Snips.

      npm install mqtt --save

      3. Open everloop.js using any editorand paste the code given in program. Code is used to control the LEDs on the MATRIX Creator/Voice.

      4. Open relay.js using any editor and paste the code given in program. It is used to turn the relay on and off.

      5. Open assistant.js using any editor and paste the code given in program. It combines all the code from everloop.js & relay.js to control the everloop and relay switch through voice. It shows you how to listen to lightState intents and read slots. 

      var snipsUserName = 'ruchir'; // username of snips account 
      var lightState = 'hermes/intent/'+snipsUserName+':lightState'; // name of intent you give while creating an intent

      Once done now it is time to check.Open your personal computer(Ubuntu) terminal and run

      sam watch

      On the other hand, open RPi terminal and go to the directory 

      cd js-matrix-core-app/
      node assistant.js  // run it

      Step 12: Connections

      Relay ==> Matrix Voice

      • 5V ==> 5V
      • GND ==>GND
      • Signal1 ==> GPIO0 (DF0)
      • Signal2 ==> GPIO1 (DF1)
      • Signal3 ==> GPIO2 (DF2)
      • Signal4 ==> GPIO3 (DF3)
      • Signal5 ==> GPIO5 (DF5)

      Step 13: Result

      Video


      Remix Contest

      Participated in the
      Remix Contest