ESP8266 Voice Control With Google Assistant and Adafruit IO

by DqwertyC

87K5954

Intro: ESP8266 Voice Control With Google Assistant and Adafruit IO

This is a simple tutorial showing how to control your ESP8266 based projects using voice commands and the Google Assistant. For the purposes of this tutorial, I'll simply be turning the LED built in to my ESP8266 breakout board on and off, but this could be used to trigger any number of events. If it can be controlled by an Arduino, it can now be controlled by you voice!

Materials:

The only physical materials required are:

  • An ESP8266 breakout board (I used the ESP-12E from Acrobotics)
  • A Google Assistant enabled device (Apps available for Android and iPhone, or Google Home)
  • A computer with access to the Internet

While these are the only physical materials required, this tutorial also requires that you set up accounts on various websites, and install the Arduino IDE.

  • Adafruit IO - Create a free account.
  • IFTTT - Create a free ("Maker") account.
  • Google - The same account you're using the Google Assistant for.

Setup:

Before starting, you should set up the Arduino IDE to program the ESP8266. There are already a ton of tutorials for this, so I won't repeat all the steps here. This is the one I used to set up my IDE.

STEP 1: Setting Up Adafruit IO

Adafruit IO is an IOT platform built around the Message Queue Telemetry Transport (MQTT) Protocol. MQTT is a lightweight protocol that allows multiple devices to connect to a shared server, called the MQTT Broker, and subscribe or write to user defined topics. When a device is subscribed to a topic, the broker will send it a notification whenever that topic changes. MQTT is best suited for applications with low data rates, strict power constraints, or slow Internet connections.

In addition to providing the MQTT Broker service, Adafruit IO also allows you to set up dashboards that let you directly manipulate or view the current value of each topic. Since it can be accessed from a web browser, it makes it the ideal hub for monitoring and controlling all of your various IOT projects.

After creating your Adafruit IO account, you should be taken to the homescreen. Select "Feeds" from the left-hand menu. Click the Actions drop-down menu, and create a new feed. I called mine "onoff".

Next, go to Dashboards in the left-hand menu. Click the Actions drop-down menu, and create a new dashboard. I called mine "LightSwitch". Open the new dashboard, and you should be taken to a mostly blank page. Pressing the blue + button will let you add new UI components to the dashboard. For now, all we'll need is a toggle button, which should the first option. When prompted to choose a feed, select the one you just made, and keep the defaults for the rest of the settings.

That's all for now on the Adafruit IO end of things. Next step is connecting your ESP8266 to the MQTT Broker.

STEP 2: Connecting the ESP8266

Connecting the ESP8266 to the Adafruit IO system is relatively straightforward. Before you get started with the code, you'll need to install the Adafruit MQTT Client library, which can be found under the Arduino Library Manager (Sketch > Include Library > Library Manager...). Even though this library is produced and maintained by Adafruit, it can be used to connect to any MQTT server.

There are three libraries you'll want to include at the start of your program:

#include <ESP8266WiFi.h>
#include "Adafruit_MQTT.h"<br>#include "Adafruit_MQTT_Client.h"

The first library controls the ESP8266's wifi connections, and the other two control connections to the MQTT server.

You'll also want to #define several strings:

#define WIFI_SSID "<Your Wifi SSID>"
#define WIFI_PASS "<Your WiFi Password>"

#define MQTT_SERV "io.adafruit.com"
#define MQTT_PORT 1883
#define MQTT_NAME "<Your Adafruit IO Username>"
#define MQTT_PASS "<Your Adafruit IO Key>"

These are the settings for connecting to your Adafruit IO Account. Your Adafruit IO Key is a string of characters that can be found by pressing the gold key button on your dashboard, or "View AIO Key" on the left-hand menu of Adafruit IO.

Next, create WiFiClient and Adafruit_MQTT_Client objects as global variables, and instantiate the feed for your light switch:

WiFiClient client;
Adafruit_MQTT_Client mqtt(&client, MQTT_SERV, MQTT_PORT, MQTT_NAME, MQTT_PASS);

Adafruit_MQTT_Subscribe onoff = Adafruit_MQTT_Subscribe(&mqtt, MQTT_NAME "/f/onoff");

Now, in the setup, we'll connect to the WiFi and the MQTT server:

void setup()
{
  Serial.begin(9600);

  //Connect to WiFi
  Serial.print("\n\nConnecting Wifi... ");
  WiFi.begin(WIFI_SSID, WIFI_PASS);
  while (WiFi.status() != WL_CONNECTED)
  {
    delay(500);
  }

  Serial.println("OK!");

  //Subscribe to the onoff topic
  mqtt.subscribe(&onoff);

  pinMode(LED_BUILTIN, OUTPUT);
}

This will connect you to WiFi and the MQTT server, and subscribe to the onoff topic. We also set the led to be an output.

Now, in the main loop, we need to check to see if our subscription has been updated, and act accordingly. We'll also occasionally ping the server to make sure we stay connected.

void loop()
{
  //Connect/Reconnect to MQTT
  MQTT_connect();

  //Read from our subscription queue until we run out, or
  //wait up to 5 seconds for subscription to update
  Adafruit_MQTT_Subscribe * subscription;
  while ((subscription = mqtt.readSubscription(5000)))
  {
    //If we're in here, a subscription updated...
    if (subscription == &onoff)
    {
      //Print the new value to the serial monitor
      Serial.print("onoff: ");
      Serial.println((char*) onoff.lastread);

      //If the new value is  "ON", turn the light on.
      //Otherwise, turn it off.
      if (!strcmp((char*) onoff.lastread, "ON"))
      {
        //active low logic
        digitalWrite(LED_BUILTIN, LOW);
      }
      else
      {
        digitalWrite(LED_BUILTIN, HIGH);
      }
    }
  }

  // ping the server to keep the mqtt connection alive<br>  if (!mqtt.ping())<br>  {<br>    mqtt.disconnect();<br>  }<br>}

Finally, add the MQTT_connect function, which was part of one of the Adafruit_MQTT examples:

/***************************************************
Adafruit MQTT Library ESP8266 Example
Must use ESP8266 Arduino from:  https://github.com/esp8266/Arduino
Works great with Adafruit's Huzzah ESP board & Feather
 ---->   https://github.com/esp8266/Arduino
 ---->   https://github.com/esp8266/Arduino
Adafruit invests time and resources providing this open source code,
please support Adafruit and open-source hardware by purchasing
products from Adafruit!
Written by Tony DiCola for Adafruit Industries.
MIT license, all text above must be included in any redistribution
****************************************************/
void MQTT_connect() 
{
  int8_t ret;
  // Stop if already connected
  if (mqtt.connected())
  {
    return;
  }

  Serial.print("Connecting to MQTT... ");
  uint8_t retries = 3;
  while ((ret = mqtt.connect()) != 0) // connect will return 0 for connected
  { 
    Serial.println(mqtt.connectErrorString(ret));
    Serial.println("Retrying MQTT connection in 5 seconds...");
    mqtt.disconnect();
    delay(5000);  // wait 5 seconds
    retries--;
    if (retries == 0) 
    {
      // basically die and wait for WDT to reset me
      while (1);
    }
  }
  Serial.println("MQTT Connected!");
}

And that should be it! Load it onto your device, start up the serial monitor, and flip the switch in your dashboard a few times to make sure that it's connected! Now, we just need to connect it to our Google Assistant.

STEP 3: Connecting to Google Assistant Through IFTTT

In this last step, we'll connect our Google Assistant to the Adafruit IO MQTT Broker to allow us to control the lights with voice commands. To do this, we'll use the IFTTT (If This Then That) platform, which allows hundreds of different services to trigger actions in a variety of other services.

After you've set up your account and taken a look around, you'll need to go to the Maker's Platform to start making your own applets. Select "Private" from the left hand menu, then click the blue "New Applet" button. This will take you to the applet editor, where you choose triggers ("If This") and the subsequent actions ("Then That").

For your trigger, choose "Google Assistant" as the service, then select "Say a simple phrase" from the drop down menu of specific triggers. This will bring up a new list of fields to fill in, including variations of the activation phrase, the Google Assistant's response, and the language. For my activation phrases, I chose "Turn the light on," "Turn on the light," and "Switch the light on".

The next section is the filter, which allows your applet to do different things based on either global variables, like time of day, or manipulate user input with a simple Javascript-like language. This is a more advanced feature that I may come back to in a later tutorial, but for now you don't need to worry about it.

The final part of your applet is the Action, what your applet does in response to the Trigger. For the service, choose "Adafruit", and for the specific Action, choose "Send data to Adafruit IO". This will bring up two fields that you need to fill in. The first should be replaced with the name of the Adafruit IO feed you want to send data to, in this case "onoff". The second field is the data to send. For this applet, we'll send "ON", which is the string our ESP8266 is waiting for.

Now all that's left is to add the title and description. Since this is a private applet, they don't really matter that much, but the IOFTTT standard is to have the title be a brief description of what the applet does while the description is a more in depth look at how the applet works.

Once you have that applet finished, create a second one for turning the lights "OFF". You should now see two applets on your IFTTT Platform page. To activate them, go to the My Applets page on the main IFTTT site, click on the applet card, and click set the on-off toggle switch to "On". If you haven't already, IFTTT will ask to connect to your Adafruit IO and Google Assistant accounts. Allow the accounts to be linked, then turn on the second applet as well.

Once both applets are turned on, the setup should be complete!

STEP 4: Testing and Troubleshooting

Now all that's left is to test the system. To do this, you'll need a device with the Google Assistant enabled. This is built into the latest versions of the Android operating system, as well as the Google Home series of devices. If you don't have either of these, the Google Allo messaging app, available for Android and iOS, also includes the Google Assistant. Start up the assistant, make sure it's logged into the proper account, and say the activation phrase you used in the previous step. After a 2-5 second delay, the light on your ESP8266 board should switch on or off. Try the other phrase, and make sure it works too.

Congratulations! You've just made a internet-connected, voice controlled light! Hopefully this tutorial has provided enough of a foundation that you can start making your own variety of voice-controlled IOT projects. Have fun!

Troubleshooting

There are a number of places that the connection between your voice and the light can break down. If the light isn't changing when your speak, there are a few things you should check.

  • Does the light change when you toggle the switch on the Adafruit IO dashboard? If not, your ESP8266 is either not connecting to the server, not subscribing to the feed, or not checking for the correct string values. Check the Serial Monitor output of your ESP8266 device to find out which.
  • Is the Google Assistant hearing you properly? If you use the Google Allo app, you can see what the Assistant heard, or you can directly type the phrase you want it to interpret.
  • Is the Google Assistant responding with the correct phrase? If not, then your Google account and your IFTTT account aren't connected. Make sure you're using the same Google accounts for the assistant and IFTTT.
  • Is the Adafruit IO dashboard not updating when the IFTTT applet triggers? If not, then your Adafruit IO account and your IFTTT account aren't connected. Double check on IFTTT to make sure that your accounts have been linked.

29 Comments

aszommer 3 years ago

Thanks for this good guidance.

My problem is that it only works if the phone is unlocked. If the phone screen is locked Google assistant understands the command but asks me to unlock the phone first which makes the complete solution meaningless.
Is there any solution for this?

rgvillanueva28 3 years ago

I'm having problem. After around 5 minutes. The device does not subscribe to adafruit IO anymore but serial monitor says that it is connected.

mhdd3 5 years ago

Thank you for the information,,
I will try it

MrReivax2 5 years ago

Google Translator
A fantastic project, the basis of the code served to modify a bit and make a different project
Thank you for this very educational post

SimonasB1 5 years ago

Hello, I can't find library, named AdafruitIO_WiFi, so this code doesn't work to me... Can you help me?

ArindamS16 5 years ago

My activation phrase is 'Wake up, daddy's home'. So I feel like Iron Man when the LED lights up! Thanks a lot for the tutorial. It was clear and concise. One thing though, the feeds in Adafruit are in format 'feeds/onoff' now. Still, after changing that, it works like a charm.

jay_jay 5 years ago

Please help me on this

compiling error massage as below

c:\users\abc\appdata\local\arduino15\packages\esp8266\tools\xtensa-lx106-elf-gcc\1.20.0-26-gb404fb9-2\xtensa-lx106-elf\include\c++\4.8.2\bits\stl_algobase.h:193:5: note: template argument deduction/substitution failed:

C:\Program Files\Arduino\libraries\Adafruit_MQTT_Library-master\Adafruit_MQTT_Client.cpp:84:38: note: deduced conflicting types for parameter 'const _Tp' ('short unsigned int' and 'int')

uint16_t sendlen = min(len, 250);

^

exit status 1

Error compiling for board NodeMCU 1.0 (ESP-12E Module).

GenesisR2 5 years ago

Thanks this was really helpful, the bit about the IFTTT maker platform however turned out to be unnecessary for me I could just use the normal IFTTT new applet menu to set up the applets I needed.

asger.crab 5 years ago

Hello.

Very good instructions but I have a problem that I can't figure out.

When I upload the code from the site to my esp8266 and I check the serial monitor, it only says connecting Wifi... and stops there. It seems like it should write OK! when it has connected to my wifi network but it is stuck. When I speak to my google assistant I see that the trigger tricks at the Adafruit website, but still nothing happens on the esp8266. I have triple checked that the wifi password and SSID have been written correctly. My feed has been named onoff so the code should also adapt to this. I don't know what has happened. can someone help me?

NishantJ8 5 years ago

if it display connecting wifi then it's ok it will work anyway, now check is ok google responding according to your settings in ifttt? if it is then check when you toggle switch in your adafruit dashboard named lightswitch does led changes it's state?

vanditht 5 years ago

Hello,

Basically i had connected a temperature and humidity sensors to the NodeMCU and the appropriate values are displayed on the Adafruit dashboard, but I want the Google Assistant to read-out the values instead of just displaying on the screen.

Can you help me with the issue?

DqwertyC 5 years ago

Unfortunately Google Assistant can't currently read values from MQTT feeds, and there's nothing on IFTTT that has Google Assistant as the "Then That" part of the applet. There may be a way, but it would involve getting deeper into Google Assistant than this article intends.

divaha 5 years ago

Hello,

can you help me with the following error message during compile please ?

Arduino: 1.6.8 (Windows 10), Board:"Generic ESP8266 Module, 80 MHz, ck, 26 MHz, 40MHz, QIO, 512K (no SPIFFS), 2, v2 Lower Memory, Disabled, None, Only Sketch, 115200"

C:\Users\Dirk\Downloads\mqtt_blink\mqtt_blink.ino: In function 'void loop()':

mqtt_blink:44: error: 'MQTT_connect' was not declared in this scope

MQTT_connect();

^

exit status 1

'MQTT_connect' was not declared in this scope

Thank you for the nice project, I hope I did not make a stupid mistake :-(

Dirk

chieldd 6 years ago

Can I also send an integer over the connection to control a rolling shutter?

seasider1960 6 years ago

In my project I was passing free form text strings defined by the $ sign in IFTTT. I struggled for a couple of hours to understand why the code worked when I typed it into the Adafruit IO console but not from the GA -- even though the IO Feed was correctly updating and being passed to the ESP8266. After staring at the Serial Monitor for ages it suddenly clicked that when some strings were originated from the GA the IO was adding a space at the beginning (or at least, the ESP8266 was receiving the string with a space). Also, and more obviously, some strings were being capitalized and not others. Moral: make sure those strings are being tested properly and consider providing alternative tests.

I know the importance of making sure strings are being tested correctly is mentioned in the "Troubleshooting" section but even with that I missed what was going wrong in my case.

StanisławN3 6 years ago

Is it possible to ask google assistant about value? For example for temperature in room,
I connected it with Google Sheets at this moment.
More Comments