Expanding Voice Commands for Visually Impaired Interfacing Google Home With Arduino Using Webhooks

There are challenges these days to find remotes to control our home appliances be it our television or setup box or even AC. Smartphones do help us but not completely. “Internet of Things” is a term growing very rapidly, it includes creating a network of physical objects around us and connecting them to the world of Internet. It is designed as an IOT based install-able smart assistive low-cost voice-controlled device as a solution.

This can also be beneficial for visually impaired people. The solution aims to perform all functions which can be controlled by a remote like turning off/on the lights or appliances, controlling TV channels, opening doors, maintaining temperature, surveillance etc. through voice commands and is easily install-able.

The difference is that my project reduces the integration time of ESP8266 with Google Assistant. The latency rate of response between devices reduces as command has to cover only through a single layer of data transfer.

Can check my work on http://www.ijrat.org/downloads/Vol-7/april-2019/PaperID-742019115.pdf

Youtube Link https://www.youtube.com/playlist?list=PLZ2KX-Nxf8ZBOJtkaBng2V9PyHMxujH9z

Step 1: Process Flow

The system works on the following flow process

Step 2: Resources Required

Things we will need

Hardware
1. Google Home Mini / Assistant
2. Wi-Fi Router / Network Router
3. Arduino UNO
4. Ai-Thinker ESP8266-01
5. RGB LED
6. Battery Pack / Power Supply
Software
1. Arduino IDE
2. IFTTT Webhooks

Step 3: Connections and Code

The connections of LED with Arduino and ESP8266

Here is the code for using this method.

Can download it directly from below...

#include <softwareserial.h><SoftwareSerial.h><br>SoftwareSerial esp8266(3, 2); //RX pin = 3, TX pin = 2
int state = 5; //define initial (5 = stand-by)
#define DEBUG true //show messages between ESP8266 and Arduino in serial port<br></softwareserial.h>

The main setup section. Here comments are mentioned inline to make it easy to understand.

void setup() {<br>  esp8266.begin(9600);
  Serial.begin(9600);
  pinMode (10, OUTPUT); // LED attached here sendData("AT+RST\r\n", 2000, DEBUG); //reset module
  delay(1000);
  sendData("AT+CWMODE=1\r\n", 1000, DEBUG); //set station mode
  delay(1000);
  sendData("AT+CWJAP=\"WIFI_SSID\",\"WIFI_PASSWORD\"\r\n", 2000, DEBUG); //connect wi-fi network (replace WIFI_SSID by your Wi-Fi router SSID and WIFI_PASSWORD by its password
  sendData("AT+CIPMUX=1\r\n", 1000, DEBUG); //allow multiple connections
  delay(1000);
  sendData("AT+CIPSERVER=1,PORTNUMBER\r\n", 1000, DEBUG); // start web server on port PORTNUMBER
  delay(1000);
  sendData("AT+CIPSTO=30\r\n", 1000, DEBUG); // SET TIMEOUT TO 30
  delay(500);
}

This is the loop function which checks for the command

void loop() {<br>  if (esp8266.available())  //verify incoming data
  {
    String msg;
    String i;
    if (esp8266.find("+IPD,")) //if there is a message
    {
      i = esp8266.readStringUntil(',');
      msg = esp8266.readString(); //read whole message
      if (msg.indexOf("on LED") >= 0) digitalWrite(10, HIGH);
      else if (msg.indexOf("off LED") >= 0) digitalWrite(10, LOW);
      String closeCommand = "AT+CIPCLOSE=";
      closeCommand += i; // append connection id
      closeCommand += "\r\n";
      sendData(closeCommand, 1000, DEBUG); // close connection
    }
  }
}

This is the main function which works in background<br>

//*******************<br>//Auxiliary functions
//*******************
String sendData(String command, const int timeout, boolean debug)
{
  String response = "";
  esp8266.print(command);
  long int time = millis();
  while ( (time + timeout) > millis())
  {
    while (esp8266.available())
    {
      char c = esp8266.read();
      response += c;
    }
  }
  if (debug)
  {
    Serial.print(response);
  }
  return response;
}