Introduction: Expanding Voice Commands for Visually Impaired Interfacing Google Home With Arduino Using Webhooks

There are challenges these days to find remotes to control our home appliances be it our television or setup box or even AC. Smartphones do help us but not completely. “Internet of Things” is a term growing very rapidly, it includes creating a network of physical objects around us and connecting them to the world of Internet. It is designed as an IOT based install-able smart assistive low-cost voice-controlled device as a solution.

This can also be beneficial for visually impaired people. The solution aims to perform all functions which can be controlled by a remote like turning off/on the lights or appliances, controlling TV channels, opening doors, maintaining temperature, surveillance etc. through voice commands and is easily install-able.

The difference is that my project reduces the integration time of ESP8266 with Google Assistant. The latency rate of response between devices reduces as command has to cover only through a single layer of data transfer.

Can check my work on

Youtube Link

Step 1: Process Flow

The system works on the following flow process

Step 2: Resources Required

Things we will need

  1. Hardware
    1. Google Home Mini / Assistant
    2. Wi-Fi Router / Network Router
    3. Arduino UNO
    4. Ai-Thinker ESP8266-01
    5. RGB LED
    6. Battery Pack / Power Supply
  2. Software
    1. Arduino IDE
    2. IFTTT Webhooks

Step 3: Connections and Code

The connections of LED with Arduino and ESP8266

Here is the code for using this method.

Can download it directly from below...

#include <softwareserial.h><SoftwareSerial.h><br>SoftwareSerial esp8266(3, 2); //RX pin = 3, TX pin = 2
int state = 5; //define initial (5 = stand-by)
#define DEBUG true //show messages between ESP8266 and Arduino in serial port<br></softwareserial.h>

The main setup section. Here comments are mentioned inline to make it easy to understand.

void setup() {<br>  esp8266.begin(9600);
  pinMode (10, OUTPUT); // LED attached here sendData("AT+RST\r\n", 2000, DEBUG); //reset module
  sendData("AT+CWMODE=1\r\n", 1000, DEBUG); //set station mode
  sendData("AT+CWJAP=\"WIFI_SSID\",\"WIFI_PASSWORD\"\r\n", 2000, DEBUG); //connect wi-fi network (replace WIFI_SSID by your Wi-Fi router SSID and WIFI_PASSWORD by its password
  sendData("AT+CIPMUX=1\r\n", 1000, DEBUG); //allow multiple connections
  sendData("AT+CIPSERVER=1,PORTNUMBER\r\n", 1000, DEBUG); // start web server on port PORTNUMBER
  sendData("AT+CIPSTO=30\r\n", 1000, DEBUG); // SET TIMEOUT TO 30
This is the loop function which checks for the command
void loop() {<br>  if (esp8266.available())  //verify incoming data
    String msg;
    String i;
    if (esp8266.find("+IPD,")) //if there is a message
      i = esp8266.readStringUntil(',');
      msg = esp8266.readString(); //read whole message
      if (msg.indexOf("on LED") >= 0) digitalWrite(10, HIGH);
      else if (msg.indexOf("off LED") >= 0) digitalWrite(10, LOW);
      String closeCommand = "AT+CIPCLOSE=";
      closeCommand += i; // append connection id
      closeCommand += "\r\n";
      sendData(closeCommand, 1000, DEBUG); // close connection
This is the main function which works in background<br>
//*******************<br>//Auxiliary functions
String sendData(String command, const int timeout, boolean debug)
  String response = "";
  long int time = millis();
  while ( (time + timeout) > millis())
    while (esp8266.available())
      char c =;
      response += c;
  if (debug)
  return response;

Step 4: Connecting With IFTTT Webhooks

  1. Go to IFTTT and create a new Applet or click here
  2. Click on This and search for Google Assistant
  3. Select Say a Phrase with Text Ingredient
  4. Fill the details accordingly
  5. Click Create
  6. Now click on That and search for Webhooks
  7. Select Make a Web request
  8. Fill the details accordingly
  9. Click on Create Action

And you are ready to rock!!!

Step 5: And Its Done.

A small modification made for enhancement...

Changing RGB LED with IR Led

Hope you Liked it.

Enjoy Exploring Arduino!!!

IoT Challenge

Participated in the
IoT Challenge