ESP8266 Voice Control With Google Assistant and Adafruit IO

This is a simple tutorial showing how to control your ESP8266 based projects using voice commands and the Google Assistant. For the purposes of this tutorial, I'll simply be turning the LED built in to my ESP8266 breakout board on and off, but this could be used to trigger any number of events. If it can be controlled by an Arduino, it can now be controlled by you voice!

Materials:

The only physical materials required are:

An ESP8266 breakout board (I used the ESP-12E from Acrobotics)
A Google Assistant enabled device (Apps available for Android and iPhone, or Google Home)
A computer with access to the Internet

While these are the only physical materials required, this tutorial also requires that you set up accounts on various websites, and install the Arduino IDE.

Adafruit IO - Create a free account.
IFTTT - Create a free ("Maker") account.
Google - The same account you're using the Google Assistant for.

Setup:

Before starting, you should set up the Arduino IDE to program the ESP8266. There are already a ton of tutorials for this, so I won't repeat all the steps here. This is the one I used to set up my IDE.

Step 1: Setting Up Adafruit IO

Adafruit IO is an IOT platform built around the Message Queue Telemetry Transport (MQTT) Protocol. MQTT is a lightweight protocol that allows multiple devices to connect to a shared server, called the MQTT Broker, and subscribe or write to user defined topics. When a device is subscribed to a topic, the broker will send it a notification whenever that topic changes. MQTT is best suited for applications with low data rates, strict power constraints, or slow Internet connections.

In addition to providing the MQTT Broker service, Adafruit IO also allows you to set up dashboards that let you directly manipulate or view the current value of each topic. Since it can be accessed from a web browser, it makes it the ideal hub for monitoring and controlling all of your various IOT projects.

After creating your Adafruit IO account, you should be taken to the homescreen. Select "Feeds" from the left-hand menu. Click the Actions drop-down menu, and create a new feed. I called mine "onoff".

Next, go to Dashboards in the left-hand menu. Click the Actions drop-down menu, and create a new dashboard. I called mine "LightSwitch". Open the new dashboard, and you should be taken to a mostly blank page. Pressing the blue + button will let you add new UI components to the dashboard. For now, all we'll need is a toggle button, which should the first option. When prompted to choose a feed, select the one you just made, and keep the defaults for the rest of the settings.

That's all for now on the Adafruit IO end of things. Next step is connecting your ESP8266 to the MQTT Broker.

Step 2: Connecting the ESP8266

Connecting the ESP8266 to the Adafruit IO system is relatively straightforward. Before you get started with the code, you'll need to install the Adafruit MQTT Client library, which can be found under the Arduino Library Manager (Sketch > Include Library > Library Manager...). Even though this library is produced and maintained by Adafruit, it can be used to connect to any MQTT server.

There are three libraries you'll want to include at the start of your program:

#include <ESP8266WiFi.h>
#include "Adafruit_MQTT.h"<br>#include "Adafruit_MQTT_Client.h"

The first library controls the ESP8266's wifi connections, and the other two control connections to the MQTT server.

You'll also want to #define several strings:

#define WIFI_SSID "<Your Wifi SSID>"
#define WIFI_PASS "<Your WiFi Password>"

#define MQTT_SERV "io.adafruit.com"
#define MQTT_PORT 1883
#define MQTT_NAME "<Your Adafruit IO Username>"
#define MQTT_PASS "<Your Adafruit IO Key>"

These are the settings for connecting to your Adafruit IO Account. Your Adafruit IO Key is a string of characters that can be found by pressing the gold key button on your dashboard, or "View AIO Key" on the left-hand menu of Adafruit IO.

Next, create WiFiClient and Adafruit_MQTT_Client objects as global variables, and instantiate the feed for your light switch:

WiFiClient client;
Adafruit_MQTT_Client mqtt(&client, MQTT_SERV, MQTT_PORT, MQTT_NAME, MQTT_PASS);

Adafruit_MQTT_Subscribe onoff = Adafruit_MQTT_Subscribe(&mqtt, MQTT_NAME "/f/onoff");

Now, in the setup, we'll connect to the WiFi and the MQTT server:

void setup()
{
  Serial.begin(9600);

  //Connect to WiFi
  Serial.print("\n\nConnecting Wifi... ");
  WiFi.begin(WIFI_SSID, WIFI_PASS);
  while (WiFi.status() != WL_CONNECTED)
  {
    delay(500);
  }

  Serial.println("OK!");

  //Subscribe to the onoff topic
  mqtt.subscribe(&onoff);

  pinMode(LED_BUILTIN, OUTPUT);
}

This will connect you to WiFi and the MQTT server, and subscribe to the onoff topic. We also set the led to be an output.

Now, in the main loop, we need to check to see if our subscription has been updated, and act accordingly. We'll also occasionally ping the server to make sure we stay connected.

void loop()
{
  //Connect/Reconnect to MQTT
  MQTT_connect();

  //Read from our subscription queue until we run out, or
  //wait up to 5 seconds for subscription to update
  Adafruit_MQTT_Subscribe * subscription;
  while ((subscription = mqtt.readSubscription(5000)))
  {
    //If we're in here, a subscription updated...
    if (subscription == &onoff)
    {
      //Print the new value to the serial monitor
      Serial.print("onoff: ");
      Serial.println((char*) onoff.lastread);

      //If the new value is  "ON", turn the light on.
      //Otherwise, turn it off.
      if (!strcmp((char*) onoff.lastread, "ON"))
      {
        //active low logic
        digitalWrite(LED_BUILTIN, LOW);
      }
      else
      {
        digitalWrite(LED_BUILTIN, HIGH);
      }
    }
  }

  // ping the server to keep the mqtt connection alive<br>  if (!mqtt.ping())<br>  {<br>    mqtt.disconnect();<br>  }<br>}

Finally, add the MQTT_connect function, which was part of one of the Adafruit_MQTT examples:

/***************************************************
Adafruit MQTT Library ESP8266 Example
Must use ESP8266 Arduino from:  https://github.com/esp8266/Arduino
Works great with Adafruit's Huzzah ESP board & Feather
 ---->   https://github.com/esp8266/Arduino
 ---->   https://github.com/esp8266/Arduino
Adafruit invests time and resources providing this open source code,
please support Adafruit and open-source hardware by purchasing
products from Adafruit!
Written by Tony DiCola for Adafruit Industries.
MIT license, all text above must be included in any redistribution
****************************************************/

void MQTT_connect() 
{
  int8_t ret;
  // Stop if already connected
  if (mqtt.connected())
  {
    return;
  }

  Serial.print("Connecting to MQTT... ");
  uint8_t retries = 3;
  while ((ret = mqtt.connect()) != 0) // connect will return 0 for connected
  { 
    Serial.println(mqtt.connectErrorString(ret));
    Serial.println("Retrying MQTT connection in 5 seconds...");
    mqtt.disconnect();
    delay(5000);  // wait 5 seconds
    retries--;
    if (retries == 0) 
    {
      // basically die and wait for WDT to reset me
      while (1);
    }
  }
  Serial.println("MQTT Connected!");
}

And that should be it! Load it onto your device, start up the serial monitor, and flip the switch in your dashboard a few times to make sure that it's connected! Now, we just need to connect it to our Google Assistant.

mqtt_blink.ino
Download

Step 3: Connecting to Google Assistant Through IFTTT

In this last step, we'll connect our Google Assistant to the Adafruit IO MQTT Broker to allow us to control the lights with voice commands. To do this, we'll use the IFTTT (If This Then That) platform, which allows hundreds of different services to trigger actions in a variety of other services.

After you've set up your account and taken a look around, you'll need to go to the Maker's Platform to start making your own applets. Select "Private" from the left hand menu, then click the blue "New Applet" button. This will take you to the applet editor, where you choose triggers ("If This") and the subsequent actions ("Then That").

For your trigger, choose "Google Assistant" as the service, then select "Say a simple phrase" from the drop down menu of specific triggers. This will bring up a new list of fields to fill in, including variations of the activation phrase, the Google Assistant's response, and the language. For my activation phrases, I chose "Turn the light on," "Turn on the light," and "Switch the light on".

The next section is the filter, which allows your applet to do different things based on either global variables, like time of day, or manipulate user input with a simple Javascript-like language. This is a more advanced feature that I may come back to in a later tutorial, but for now you don't need to worry about it.

The final part of your applet is the Action, what your applet does in response to the Trigger. For the service, choose "Adafruit", and for the specific Action, choose "Send data to Adafruit IO". This will bring up two fields that you need to fill in. The first should be replaced with the name of the Adafruit IO feed you want to send data to, in this case "onoff". The second field is the data to send. For this applet, we'll send "ON", which is the string our ESP8266 is waiting for.

Now all that's left is to add the title and description. Since this is a private applet, they don't really matter that much, but the IOFTTT standard is to have the title be a brief description of what the applet does while the description is a more in depth look at how the applet works.

Once you have that applet finished, create a second one for turning the lights "OFF". You should now see two applets on your IFTTT Platform page. To activate them, go to the My Applets page on the main IFTTT site, click on the applet card, and click set the on-off toggle switch to "On". If you haven't already, IFTTT will ask to connect to your Adafruit IO and Google Assistant accounts. Allow the accounts to be linked, then turn on the second applet as well.

Once both applets are turned on, the setup should be complete!

Step 4: Testing and Troubleshooting

Now all that's left is to test the system. To do this, you'll need a device with the Google Assistant enabled. This is built into the latest versions of the Android operating system, as well as the Google Home series of devices. If you don't have either of these, the Google Allo messaging app, available for Android and iOS, also includes the Google Assistant. Start up the assistant, make sure it's logged into the proper account, and say the activation phrase you used in the previous step. After a 2-5 second delay, the light on your ESP8266 board should switch on or off. Try the other phrase, and make sure it works too.

Congratulations! You've just made a internet-connected, voice controlled light! Hopefully this tutorial has provided enough of a foundation that you can start making your own variety of voice-controlled IOT projects. Have fun!

Troubleshooting

There are a number of places that the connection between your voice and the light can break down. If the light isn't changing when your speak, there are a few things you should check.

Does the light change when you toggle the switch on the Adafruit IO dashboard? If not, your ESP8266 is either not connecting to the server, not subscribing to the feed, or not checking for the correct string values. Check the Serial Monitor output of your ESP8266 device to find out which.

Is the Google Assistant hearing you properly? If you use the Google Allo app, you can see what the Assistant heard, or you can directly type the phrase you want it to interpret.

Is the Google Assistant responding with the correct phrase? If not, then your Google account and your IFTTT account aren't connected. Make sure you're using the same Google accounts for the assistant and IFTTT.

Is the Adafruit IO dashboard not updating when the IFTTT applet triggers? If not, then your Adafruit IO account and your IFTTT account aren't connected. Double check on IFTTT to make sure that your accounts have been linked.