Google Assistant on Pi for Home Automation and Entertainment

Background:

Last year in May, Google Released their AIY Projects kit and initially not many had access to it. So thats when i started modifying the Google Assistant SDK adding AIY like features. Every project requires a name and so i named it GassistPi (Google Assistant on Pi). Fast forwarding to date, the project has incorporated numerous interesting features and has a community of its own. Git link: https://github.com/shivasiddharth/GassistPi

So what can you do with this project:

Voice control of GPIOs without IFTTT, API.AI, Actions SDK.
Voice control of NodeMCU without IFTTT and MQTT.
Stream internet radio stations.
Voice control of servo connected to RPi.
Safe shutdown RPi.
Stream Music from YouTube.
Track parcels.
Stream RSS Feeds.
Control Kodi.
Streaming music from Google Play Music.
Cast YouTube Videos to Chromecast
Control Chromecast.
Control Sonoff Tasmota Devices/Emulated Wemo

All by voice commands.

Step 1: Getting Started

Download the latest copy of Raspbian desktop image from https://www.raspberrypi.org/downloads/raspbian/ and burn it on to an SD Card.

Connect your peripherals such as mic, speaker, keyboard, mouse, monitor and then boot the Pi.

Open the terminal and execute the following

git clone  https://github.com/shivasiddharth/GassistPi

Step 2: INSTALL AUDIO CONFIG FILES

1. Update OS and Kernel

sudo apt-get update  
sudo apt-get install raspberrypi-kernel

2. Restart Pi

3. Choose the audio configuration according to your setup. The speaker-test command is used to initialize alsa, so please do not skip that. AIY-HAT and CUSTOM-HAT users, please reboot the Pi at places mentioned, else it will lead to audio and taskbar issues.

3.1. USB DAC or USB Sound CARD users,

sudo chmod +x /home/pi/GassistPi/audio-drivers/USB-DAC/scripts/install-usb-dac.sh
sudo /home/pi/GassistPi/audio-drivers/USB-DAC/scripts/install-usb-dac.sh
speaker-test

3.2. AIY-HAT users,

sudo chmod +x /home/pi/GassistPi/audio-drivers/AIY-HAT/scripts/configure-driver.sh
sudo /home/pi/GassistPi/audio-drivers/AIY-HAT/scripts/configure-driver.sh
sudo reboot   
sudo chmod +x /home/pi/GassistPi/audio-drivers/AIY-HAT/scripts/install-alsa-config.sh 
sudo /home/pi/GassistPi/audio-drivers/AIY-HAT/scripts/install-alsa-config.sh  
speaker-test

3.3. USB MIC AND HDMI users,

sudo chmod +x /home/pi/GassistPi/audio-drivers/USB-MIC-HDMI/scripts/install-usb-mic-hdmi.sh
sudo /home/pi/GassistPi/audio-drivers/USB-MIC-HDMI/scripts/install-usb-mic-hdmi.sh
speaker-test

3.4. USB MIC AND AUDIO JACK users,

sudo chmod +x /home/pi/GassistPi/audio-drivers/USB-MIC-JACK/scripts/usb-mic-onboard-jack.sh
sudo /home/pi/GassistPi/audio-drivers/USB-MIC-JACK/scripts/usb-mic-onboard-jack.sh 
speaker-test

3.5. CUSTOM VOICE HAT users,

sudo chmod +x /home/pi/GassistPi/audio-drivers/CUSTOM-VOICE-HAT/scripts/install-i2s.sh
sudo /home/pi/GassistPi/audio-drivers/CUSTOM-VOICE-HAT/scripts/install-i2s.sh
sudo reboot
sudo chmod +x /home/pi/GassistPi/audio-drivers/CUSTOM-VOICE-HAT/scripts/custom-voice-hat.sh
sudo /home/pi/GassistPi/audio-drivers/CUSTOM-VOICE-HAT/scripts/custom-voice-hat.sh
speaker-test

Those Using HDMI/Onboard Jack, make sure to force the audio

sudo raspi-config

Select advanced options, then audio and choose to force audio

Those using any other DACs or HATs install the cards as per the manufacturer's guide and then you can try using the USB-DAC config file after changing the hardware ids

4. Restart Pi

5. Check the speaker using the following command

speaker-test -t wav

Step 3: CONTINUE After SETTING UP AUDIO

1. Download credentials--->.json file (refer to this doc for creating credentials https://developers.google.com/assistant/sdk/devel...

2. Place the .json file in/home/pi directory DO NOT RENAME

3. Use the one-line installer for installing Google Assistant.

4.1 Make the installers Executable

<p>sudo chmod +x ./GassistPi/scripts/gassist-installer.sh</p>

4.2 Execute the installer

<p>sudo ./GassistPi/scripts/gassist-installer.sh</p>

5. Copy the google assistant authentication link from terminal and authorize using your google account

6. Copy the authorization code from browser onto the terminal and press enter

7. After successful authentication, the Google Assistant Demo test will automatically start. At the start, the volume might be low, the assistant volume is independent of the Pi volume, so increase the volume by using "Volume Up" command.

8. After verifying the working of assistant, close and exit the terminal

Step 4: HEADLESS AUTOSTART on BOOT SERVICE SETUP

1. Open the service files in the /home/pi/GassistPi/systemd/ directory and add your project and model ids in the indicated places and save the file.

2. Make the service installer executable

sudo chmod +x /home/pi/GassistPi/scripts/service-installer.sh

3. Run the service installer

sudo /home/pi/GassistPi/scripts/service-installer.sh

4. Enable the services

sudo systemctl enable gassistpi.service

5. Start the service

sudo systemctl start gassistpi.service

Step 5: VOICE CONTROL of GPIOs, SERVO and Pi SHUTDOWN

The default GPIO and shutdown trigger word is **trigger**. It should be used for controlling the GPIOs, servo and for safe shutdown of Pi.

It has been intentionally included to prevent control actions due to false positive commands. If you wish to change the trigger word, you can replace the '**trigger**'in the main.py and assistant.py code with your desired trigger word.

The default keyword for servo motor is servo. For example, the command trigger servo 90 will rotate the servo by 90 degrees.

If you wish to change the keyword, you can replace the 'servo' in the action.py script with your desired keyword for the motor.

For safe shutdown of the pi, command is: trigger shutdown

You can define your own custom actions in the actions.py script.

Step 6: VOICE CONTROL of NodeMCU

There are two ways to control NodeMCU:

Control of NodeMCU running a webserver.
Control of NodeMCU running Sonoff-Tasmota Firmware.

Controlling NodeMCU running webserver:

Download the Arduino IDE code for Nodemcu from here: https://github.com/shivasiddharth/iotwemos/blob/m... Add the wifi credentials, make the desired changes and upload the Arduino code onto the NodeMCU and get the IP address from the serial monitor. Add the NodeMCU's IP address in the actions.py file.

Syntax: "Hey Google, Trigger Turn Devicename On/Off"

For more guidelines follow the first video.

Controlling NodeMCU running Sonoff-Tasmota firmware:

Download the Tasmota firmware from this link

Follow the instructions in the second video to upload the firmware properly.

Pi3 and Pi Zero users, assign the device names and device ip addresses in the main.py script and pushbutton.py script respectively, in the marked locations.

Syntax: "Hey Google, Turn Devicename On/Off"

Advantage of using Sonoff-Tasmota over webserver is that, with Sonoff-Tasmota you can emulate a Wemo switch and control the NodeMCU using Amazon Alexa (Echo Devices) in addition to the GassistPi.

Step 7: CASTING YouTube VDIEOS TO Chromecast and Controlling Chromecast

First, add the IP-Address of your Chromecast in the actions.py script, in the indicated location.

Default command for casting YouTube videos is Play Desired Video on Chromecast, with Chromecast as the trigger word.

Example: Hey Google, Play MasterChef Season 1 Episode 1 on Chromecast casts the MasterChef YouTube Video.

Note: YouTube casting to Chromecast using third party scripts has been blocked, so I have taken a roundabout approach and as a result, you may not find the usual YouTube interface on Chromecast.

Following are the default commands for controlling Chromecast with Chromecast as the trigger word. Pausing:

Hey Google, Pause Chromecast

Resuming:

Hey Google, Resume Chromecast

Stopping:

Hey Google, End Chromecast

Change volume up/down:

Hey Google, Chromecast Volume Up/Down

Step 8: MUSIC STREAMING From YOUTUBE

This makes use of the YouTube Data API v3.

Adding YouTube API and Generating API Key

1. Go to the projects page on your Google Cloud Console-> https://console.cloud.google.com/project

2. Select your project from the list.

3. On the left top corner, click on the hamburger icon or three horizontal stacked lines.

4. Move your mouse pointer over "API and services" and choose "credentials".

5. Click on create credentials and select API Key and choose close. Make a note of the created API Key and enter it in the actions.py script at the indicated location.

6. "From the API and services" option, select library and in the search bar type youtube, select "YouTube Data API v3" API and click on "ENABLE".

7. In the API window, click on "All API Credentials" and in the drop down, make sure to have a tick (check mark) against the API Key that you just generated.

Default keyword for playing music from YouTube without autoplay is Stream.

For example, Stream I got you command will fetch Bebe Rexha's I Got You from YouTube.

Default keyword for playing music from YouTube with autoplay is Autoplay and Stream.

For example, Autoplay and Stream I got you command will play the requested I Got You and after the end of the track will autoplay susequent tracks. The number of autoplay tracks has been limited to a maximum of 10. this can be changed the under the YouTube_Autoplay function in the actions.py script.

Step 9: MUSIC STREAMING From Google Music

The music streaming from Google Music uses [Gmusicapi](https://unofficial-google-music-api.readthedocs.io/en/latest/).

Enter your Google userid and password in the actions.py file in the line

logged_in = api.login('YOUR_GMAIL_ID', 'YOUR_GMAIL_PASSWORD', Mobileclient.FROM_MAC_ADDRESS)

If you are using a two-step authentication or two-factor authentication, generate and use an app specific password.Getting app specific password: Refer to this page on google help -https://support.google.com/accounts/answer/185833...

What you can do:

Play all your songs in loop using the syntax:

"Hey Google, Play all the songs from Google Music"

Play songs added to the user created playlist (does not include: most played playlist, thumsup playlist, etc) using the syntax:

"Hey Google, Play songs from the first playlist in Google Music"

Playlists are sorted by date created, if you have multiple playlists, use a similar syntax replacing first with second, third etc. Also you need to make suitable changes in the main.py (It has been commented in the script to help)

Play songs by a particular artist using the syntax:

"Hey Google, Play songs by artist YOUR_ARTIST_NAME from Google Music"

Play songs from particular album using the syntax:

"Hey Google, Play songs from album YOUR_ALBUM_NAME from Google Music"

What you cannot do at the moment: (some features may be added later):

Change tracks

Shuffle tracks

Repeat tracks

Step 10: CONTROLLING KODI

Adding YouTube API and Generating API Key

Follow the steps already outlined in "MUSIC STREAMING from YOUTUBE" step to add the API key

Enabling HTTP Control on Kodi

The webserver is disabled by default and has to be manually enabled by the user.

1. This can be done in Settings → Services → Control → Allow remote control via HTTP.

2. Set the port number to 8080, username to kodi and password to kodi (username and password should be in lowercase).

Adding YouTube plugin on Kodi

For Kodi to play the YouTube video, you need to add and enable the YouTube Plugin on Kodi.

Command Sytanxes for Kodi Control

Note that "on Kodi" should be used in all the commands.

Step 11: PARCEL TRACKING, RSS FEED PLAYBACK and RADIO STREAMING

PARCEL TRACKING

The default keyword for tracking parcel is parcel.

For example, you can say where is my parcel or track my parcel.

Regsiter for a free account with Aftership at https://www.aftership.com gnereate an API number and add parcels to the tracking list.

The generated API number should be added to the actions.py script at the indicated location. For a better understanding follow the attached youtube video.

RSS FEEDS PLAYBACK

Default keywords for playing RSS feeds is feed or news or quote. Example usage, top tech news will play the top technology news, top world news will play top news related to different countires, top sports news will play the top sports related news and quote of the day will give some quotes.

Do not mix the commands with Play as that has been associated with music streaming from YouTube. numfeeds variable within the feed function in actions.py file is the feed limit. Certain RSS feeds can have upto 60 items and numfeeds variable limits the number of items to stream. The default value has been set to 10, which if you want can change.

RADIO STREAMING

Default keyword for streaming radio is tune into. For example, tune into Radio 2 command will open the corresponding radio stream listed in the actions.py file.

Radio streaming has been enabled for both OK-Google and Custom hotwords/wakewords.

Useful links for obtaining radio streaming links:

http://www.radiosure.com/stations/

http://www.live-radio.net/worldwide.shtml

http://worldradiomap.com/map/