Introduction: Google Assistant on DragonBoard™ 410c

By
About: Just a guy.

This Instructables will teach you to run Google Assistant in the DragonBoard and and setup an smarthome environment with some devices.

Required:

-DragonBoard™ 410c (w/ a fresh install of linaro-alip);

-12V/1A 5.5mm jack power supply;

-CC2531 zigbee module;

-HDMI monitor;

-USB mouse and keyboard;

-OSRAM LIGHTFY™ Tunable White 60 bulb;

-FLC BLE bulb;

-Yale Real Living deadbolt smart lock (w/ zigbee module);

-Host PC (w/ Windows)

Step 1: Google Cloud Log In

In the host PC, go to the Google Cloud page and log in.

Step 2: PubSub - Console

In the left menu hover the mouse over "PubSub" and click on "Topics".

Step 3: PubSub - Create Google Cloud Project

Insert a project name and click on "CREATE".

Step 4: PubSub - Enable API

Click on "Enable API".

Step 5: PubSub - Create Topic:

Click on "Create a topic".

Step 6: PubSub - Topic Name:

Insert a name for the topic and click on "CREATE".

Step 7: PubSub - Create Subscription:

Click on "CREATE SUBSCRIPTION".

Step 8: PubSub - Subscription Setup:

Insert a name to the subscription and fill the rest as the image above shows.

Step 9: PubSub - Required Topics/Subcriptions:

Following the example, create these topics with these subscriptions:

  1. BluDevCtl
    1. DashBluDevSub
    2. BluDevSub
  2. ZigCtl
    1. DashZigSub
    2. ZigCtlSub

Step 10: Actions on Google - Log In:

Go to Actions console page and log in with the same account used before.

Step 11: Actions on Google - Import Project:

Click on "Add/import project".

Step 12: Actions on Google - Select the Project:

Select the previously created project to import and click on "IMPORT PROJECT".

Step 13: Actions on Google - Device Registration:

On project's console click on "Device registration".

Step 14: Actions on Google - Register Model:

Click on "REGISTER MODEL".

Step 15: Actions on Google - Model Setup:

Insert a name for the device, manufacturer and the device type as Speaker and click on "REGISTER MODEL".

Step 16: Actions on Google - Client Secret JSON:

Click on "Download OAuth 2.0 credentials" to download the client secret JSON that will be used later and click on "Next".

Step 17: Actions on Google - Specify Traits:

Only click on "SKIP".

Step 18: Account Activity Control:

Go to the Google Activity control page, log in with the same account as before and ensure the following toggle switches are enabled (blue):

  • Web & App Activity. In addition, be sure to select the Include Chrome history and activity from sites, apps, and devices that use Google services checkbox;
  • Device Information;
  • Voice & Audio Activity.

Step 19: Service Account Credentials - Create Credentials:

Go to Google Cloud Credentials console, log in with the same account as before and click on "Create credentials".

Step 20: Service Account Credentials - Setup:

Choose "New service account" in "Service account", insert a name for it, select "Owner" as the "Role", select JSON as the "Key type" and click on "Create".

Step 21: Service Account Credentials - Download the Credentials:

As will click on "Create", a JSON file will be download. Save as you will need this file ahead.

Step 22: Google Cloud Storage - Create Bucket:

Go to the Google Cloud Storage page, log in and click on "CREATE BUCKET".

Step 23: Google Cloud Storage - Free Trial:

The Storage is a paid service, but Google offers a free trial that has more than needed. To setup the free trial click on "Sign up for a free trial".

Step 24: Google Cloud Storage - Free Trial - Step 1:

Select the country, accept the terms of service and click on “AGREE AND CONTINUE”.

Step 25: Google Cloud Storage - Free Trial - Step 2:

From here you need to enter some institutional information (that vary country to country) and a credit card to be able to access the storage.

Step 26: Google Cloud Storage - Create Bucket W/ Free Trial:

Click again on "CREATE BUCKET".

Step 27: Google Cloud Storage - Bucket Setup:

Inset a name for the bucket and fill the rest as the image above shows.

Step 28: Google Cloud Storage - Files:

Create 3 files with these names:

  • status.txt;
  • statusble.txt;
  • statuszgb.txt.

Step 29: Google Cloud Storage - Upload Files:

Upload the files dragging then to the bucket.

Step 30: Google Cloud Storage - Check the Upload:

Wait until all files be uploaded.

Step 31: Google Cloud Functions - Create Function:

Go to the Google Cloud Functions page and log in. You MUST have followed the instructions for a free trial in Step 22 to be able to create a function.

Step 32: Google Cloud Functions - Function Setup:

Fill the setup as the image above shows. The .zip file available to download below.

Attachments

Step 33: Google Cloud Functions - Inline Editor:

In the Function Console click on the expand icon of the inline editor.

Step 34: Google Cloud Functions - Edit Variable:

Set the bucket variable to the name of the Bucket setted in Step 26 and click on "OK".

Step 35: Google Cloud Functions - Save Changes:

Click on "Deploy" to save the variable change. Wait the funciton to be deployed, and if any error appears, repeat the process again.

Step 36: Dialogflow - Console:

Go to the Dialogflow page and click on "Go to console".

Step 37: Dialogflow - Log In:

Log in using the same Google account as before.

Step 38: Dialogflow - Account Settings:

Choose a country, accept the terms of service and click on "ACCEPT".

Step 39: Dialogflow - Agent Setup:

Select the language, timezone and the Cloud project (Step 3) and click o "CREATE".

Step 40: Dialogflow - Configuration:

In the agent console click on the configuration icon.

Step 41: Dialogflow - Import:

Click on "Export and import".

Step 42: Dialogflow - Restore From ZIP:

Click on "RESTORE FROM ZIP".

Step 43: Dialogflow - Select File:

Click on "SELECT FILE".

Step 44: Dialogflow - Upload ZIP File:

Select the .zip file available to download below, type RESTORE where it says to and click in "RESTORE".

Attachments

Step 45: Dialogflow - Save Changes:

On the agent console click on "SAVE".

Step 46: Dialogflow - Fulfillment:

Click in "Fulfillment" in the left menu.

Step 47: Dialogflow - Cloud Function Console

In another tab go to the console of the previously created function and click on "Trigger".

Step 48: Dialogflow - Cloud Function URL

Copy the URL and paste as the Webhook URL in the Dialogflow fulfillment setup.

Step 49: Dragonboard Setup - Access the DB Terminal:

To perform these steps you need a form to access the terminal of the DragonBoard. In the example, PuTTY was used on Windows to access via serial UART. Do not run any command that the tutorial shows as ROOT! If accessing the terminal via serial UART switch to user linaro as the image shows.

Command:

$ su linaro

Step 50: Dragonboard Setup - Download the Code:

Download the .zip in this step and extract.

Attachments

Step 51: Dragonboard Setup - Connecting Via SFTP:

Now it's time to transfer the credentials generated in the Google Cloud to the Dragonboard. It can be done in various ways. This example will use FileZila on the host machine to send the files to the Dragonboard internal memory.

In "Host" insert the DB local network IP*, the default user and password is "linaro" and the "Port" is 22.

*Run ifconfig in the terminal to discover the local IP.

$ ifconfig

Step 52: Dragonboard Setup - Transfer the Code to the DB:

Transfer the DB_GA folder to the Dragonboard in the /home/linaro/ folder.

Step 53: Dragonboard Setup - Transfering the Credentials:

Now transfer the credentials to the root of the DB_GA folder and rename the "client_secret_XXX.json" to "client_secret.json" and the other to "service_account_key.json".

Step 54: Dragonboard Setup - Installing Dependencies:

  • Run apt update;
$ sudo apt update
  • Install these packages via apt: python-pip, libglib2.0-dev, swig, portaudio19-dev, libpulse-dev; 
$ sudo apt install python-pip libglib2.0-dev swig portaudio19-dev libpulse-dev<br>
  • Go to the “DB” folder in "DB_GA" and run this command to install pip packages;
$ sudo pip install –r requirements.txt
  • Install NodeJS 8.x; 
$ sudo curl -sL https://deb.nodesource.com/setup_8.x | sudo bash - 
$ sudo apt install -y nodejs=8.15.1-1nodesource1<br>
  • Go to the “DB_GA/DB/zbserver” folder and run npm install.
sudo npm install

Step 55: Dragonboard Setup - Credentials Configuration:

Now it's time to connect the HDMI monitor, keyboard and mouse to the DB.

In the DB Desktop enter Ctrl+Alt+T to open an terminal.

Go to the DB_GA folder:

$ cd /home/linaro/DB_GA/

Run this command:

$ sudo google-oauthlib-tool --client-secrets client_secret.json --credentials /root/.config/google-oauthlib-tool/credentials.json --scope https://www.googleapis.com/auth/assistant-sdk-prototype --save

Step 56: Dragonboard Setup - Credential Login:

Copy the URL that the last command outputs and paste into the DB browser and log in with the same Google account used in the entire tutorial.

Step 57: Run the Google Assistant!

Go to the DB_GA/DB/syswatch_dev/ folder:

$ cd /home/linaro/DB_GA/DB/syswatch_dev/

Connect the Zigbee Module to the DB;

Run the watch.py file:

$ sudo python watch.py