Introduction: Android Voice Commands to Arduino

Picture of Android Voice Commands to Arduino

Speech to text is something really new. I'm going to show you how to use it in android studio and how to control the outputs of an arduino with it.

For example when you say "TV on", the TV switches on etc.

In this tutorial I'll explain how to write a very simple app with 1 button to activate your phone to listen to you.
What you then say to your phone will be send to the arduino.

The code I'm using gives 5 solutions for everything you say, so the right thing is probably one of the five options.
This code works when you choose phrases which don't pronounce the same.

Let's get started!

Step 1: Parts

Picture of Parts

For this tutorial you'll need some parts, these are the parts you'll need:

An Arduino (I got mine from gearbest.com)
An Arduino Ethernet shield (gearbest.com)
A Breadboard (gearbest.com)
Some Jumper wires (gearbest.com)
2 Led's with 220 ohm resistors

An Android phone

You'll also need a (windows) PC with android studio installed.

Step 2: Arduino Sketch

Picture of Arduino Sketch

This is the sketch for the arduino. It looks very similar to the one from my previous instructables. I only changed the "Get request" parameters to check if a "message" was send and if so, if there was a command send with it.

#include <SPI.h>
#include <Ethernet.h>

//Led for TV on pin 22
#define TV_pin 22
//Led for stereo on pin 24
#define stereo_pin 24

// Set the MAC address
byte mac[] = {0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED};

// Set the IP address
IPAddress ip(192, 168, 1, 177);

// Start a server at port 80 (http)
EthernetServer server(80);

void setup() {
  // Open serial communications
  Serial.begin(9600);

  // start the Ethernet connection and the server
  Ethernet.begin(mac, ip);
  server.begin();

  // Pin 22 + 24 output (leds)
  pinMode(TV_pin, OUTPUT);
  pinMode(stereo_pin, OUTPUT);
}


void loop() {
  // Check if client connected
  EthernetClient client = server.available();
  
  if (client) { // If there is a client...
    boolean currentLineIsBlank = true;
    String buffer = ""; // A buffer for the GET request
    
    while (client.connected()) {

      if (client.available()) {
        char c = client.read();// Read the data of the client
        buffer += c; // Store the data in a buffer
        
        if (c == '\n' && currentLineIsBlank) {// if 2x new line ==> Request ended
          // send a standard http response header
          client.println("HTTP/1.1 200 OK");
          client.println("Content-Type: text/html");
          client.println("Connection: close");
          client.println(); // Blank line ==> end response
          break;
        }
        if (c == '\n') { // if New line
          currentLineIsBlank = true;
          buffer = "";  // Clear buffer
        } else if (c == '\r') { // If cariage return...
          //Read in the buffer if there was send "GET /?message=..."
          if(buffer.indexOf("GET /?message=")>=0) {
            // Read the message you said and see if it is a command
            if(buffer.indexOf("tv on")>=0) {
              digitalWrite(TV_pin, HIGH);
            }
            if(buffer.indexOf("stereo on")>=0) {
              digitalWrite(stereo_pin, HIGH);
            }   
            if(buffer.indexOf("tv off")>=0) {
              digitalWrite(TV_pin, LOW);
            }
            if(buffer.indexOf("stereo off")>=0) {
              digitalWrite(stereo_pin, LOW);
            }
          }
        } else {
          currentLineIsBlank = false;
        }
      }
    }
    delay(1);
    client.stop();
  }
}

If you have any questions about this script, please post them in the comment section.

Step 3: Android Layout

Picture of Android Layout

For more info on how to setup a android studio project, see my first instructables. For the android layout, I just made an app with one big imagebutton(mic icon) in the center. When you push this button, your phone starts listening to you.

<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:tools="http://schemas.android.com/tools" android:layout_width="match_parent"
    android:layout_height="match_parent" android:paddingLeft="@dimen/activity_horizontal_margin"
    android:paddingRight="@dimen/activity_horizontal_margin"
    android:paddingTop="@dimen/activity_vertical_margin"
    android:paddingBottom="@dimen/activity_vertical_margin" tools:context=".MainActivity"
    android:orientation="horizontal">

    <LinearLayout
        android:orientation="vertical"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:layout_gravity="center_vertical">

        <ImageButton
            android:layout_width="200dp"
            android:layout_height="200dp"
            android:id="@+id/btn_speak"
            android:layout_gravity="center_horizontal"
            android:src="@drawable/abc_ic_voice_search_api_mtrl_alpha" ></ImageButton>

    </LinearLayout>

</LinearLayout>

Step 4: Android Code

Picture of Android Code

For the android code, you'll need to do several things.

First, just in the mainactivity class, you need to add two things. The first thing is a SpeechRecognizer and the second thing is a TAG for debugging your app.

private SpeechRecognizer sr;
private static final String TAG = "MainActivity";

Now, in the onCreate function, you need to add these four lines. The first two are for the button to do something, the third and fourth are for the speech recognition.

ImageButton speakButton = (ImageButton) findViewById(R.id.btn_speak);
speakButton.setOnClickListener(this);
sr = SpeechRecognizer.createSpeechRecognizer(this);
sr.setRecognitionListener(new listener());

After the onCreate function you need to make a new class for the speech recognition.

class listener implements RecognitionListener
{
    public void onReadyForSpeech(Bundle params)
    {}
    public void onBeginningOfSpeech()
    {}
    public void onRmsChanged(float rmsdB)
    {}
    public void onBufferReceived(byte[] buffer)
    {}
    public void onEndOfSpeech()
    {}
    public void onError(int error)
    {
        if(error != 7) {
            Log.d(TAG,  "error " +  error);
        }
    }
    public void onResults(Bundle results)
    {
        String str = ""; // Create new empty string
        // Get the results from the speech recognizer
        ArrayList data = results.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION);
	// If there is data
        if(data.size() != 0) {
            // Add all the data to a string
            for (int i = 0; i < data.size(); i++) {
                Log.d(TAG, "result " + data.get(i));
                str += data.get(i);
                str += " ";
            }
            // Create a lowercase string
            str = str.toLowerCase();
            // Send the GET request with message
            String message = "message=" + str;
            new Background_get().execute(message);
        }
    }
    public void onPartialResults(Bundle partialResults)
    {
        Log.d(TAG, "onPartialResults");
    }
    public void onEvent(int eventType, Bundle params)
    {
        Log.d(TAG, "onEvent " + eventType);
    }
}

After this class you need to add a onClick function which activates when you press the button.

public void onClick(View v) {
    if (v.getId() == R.id.btn_speak)
    {
        // Activate the speech listener
        Intent intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);
        intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);
        intent.putExtra(RecognizerIntent.EXTRA_CALLING_PACKAGE,"voice.recognition.test");

        intent.putExtra(RecognizerIntent.EXTRA_MAX_RESULTS,5);
        sr.startListening(intent);
    }
}

The only thing we need to add now is the class which handles sending the GET requests.
It's the same class as I used in my previous instructables.

       /*****************************************************/
      /*  This is a background process for connecting      */
     /*   to the arduino server and sending               */
    /*    the GET request with the added data            */
   /*                                                   */
  /*             !! INTERNET Permission !!             */
 /*                                                   */
/*****************************************************/

private class Background_get extends AsyncTask<String, Void, String> {
    @Override
    protected String doInBackground(String... params) {
        try {
            /* Change the IP to the IP you set in the arduino sketch */
            URL url = new URL("http://192.168.1.177/?" + params[0]);
            HttpURLConnection connection = (HttpURLConnection) url.openConnection();

            BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
            StringBuilder result = new StringBuilder();
            String inputLine;
            while ((inputLine = in.readLine()) != null)
                result.append(inputLine).append("\n");

            in.close();
            connection.disconnect();
            return result.toString();

        } catch (IOException e) {
            e.printStackTrace();
        }
        return null;
    }
}

This was the coding for the app. The only thing we need to do now is adding permissions to your app, so it can use the speech and the internet.

Step 5: Adding Permissions

Picture of Adding Permissions

For adding permissions to your app you need to change the manifest file.

You just need to add these 2 lines just before "

<uses-permission android:name="android.permission.RECORD_AUDIO"/>
<uses-permission android:name="android.permission.INTERNET" />

So that's it for the app. Upload your app to your phone and you're good to go!

Step 6: Resume

Picture of Resume

In this tutorial, I showed you how to make an app with voice recognition and how to control your arduino using your voice commands.

I hope you liked it, if so, please vote for it and share it!

If you have recommendations, tips, ideas, questions, etc.
Please post them in the comment section. I'll see what I can for you!

Laurens

Comments

SayaanK (author)2016-02-12

Can i using arduino Wifi shild WRL-09954 Spark fun in this project how to connect this one


Laurens-Wuyts (author)SayaanK2016-02-12

You should be able to use the wifly shield instead of a ethernet shield. But I'm not sure how to connect, because i don't have one myself.

Laurens

mlakhwani (author)2016-02-03

nice project, just few things to confirm:
1) can i use android UNO board?
2) in place of wired connection can i use ESP8266 or any other board to control it over Wi-Fi?

can you please help me with this?

Laurens-Wuyts (author)mlakhwani2016-02-03

Hello and thanks

1) Yes you can use a arduino UNO board. You only need to change the pins, where the led's (relay's, ...) connect.

2) You can most likely do that, but I don't have an ESP8266 (yet :) ).

Laurens

irinak1 made it! (author)2015-10-28

I made it.

But It is not operate. I said the command to android phone after button of app push but message don't sent. What is problem?

Laurens-Wuyts (author)irinak12015-10-28

Does your phone make the sounds when you press the button and when it recognizes the command?

When you connect your phone to your computer, in android studio, you can go to logcat. On the bottom of the window.

When you now run the app and you send the message, maybe you'll see an error (in logcat) . If so, could you please post it here?

Can you also post your project file or apk file, so I can try it myself.

Laurens

irinak1 (author)Laurens-Wuyts2015-10-28

Yes, I heard it when press the button. I'm just test using your app and *.ino.

What does it android OS version? I used 5.0.1

Laurens-Wuyts (author)irinak12015-10-28

Yeah, I'm also using android 5.0.1.

I'll test my app again and see if it still works on android 5.1.1.

Laurens

irinak1 (author)Laurens-Wuyts2015-10-28

I solved the problem, it is me. My android phone is setting the Korean language. And the wrong data sent to board.

I changed the language setting and It's operate.

Thank you very much for your time.^^

Laurens-Wuyts (author)irinak12015-10-28

I'm glad you found the solution!

Laurens

Laurens-Wuyts (author)irinak12015-10-28

I just tested it and it still works, but I thought of something it could be.

1. On windows, press the windows key en search "cmd"

2. In "cmd" type "ipconfig"

3. Find the line which says "Ip-v4 address" (in my case the IP is 192.168.1.130)

4. Look at the first 3 numbers (192 168 1)

5. If they are different, you need to change the ip address in the *.ino file and in the app, matching to your numbers.

6. Try again, and hope it works :D

Laurens

irinak1 (author)2015-09-20

Hi,

I’m irinakim and I’m work in the WIZnet.

Thank you for using our product.

We are collecting so much data using our
product.

And I will post the this project on our
Web site.( http://wiznetmuseum.com)

Can I post your product on our Web site?

Laurens-Wuyts (author)irinak12015-09-20

Yeah, you can do it. It would be super cool!

irinak1 (author)Laurens-Wuyts2015-10-12

I was posted on our website your project.

link: http://wiznetmuseum.com/portfolio-items/android-v...

We developed the WIZwiki-W7500 using the Cortex -M0.

http://wizwiki.net/wiki/doku.php?id=products:wizwi...

Is that Interest in WIZwiki-W7500?

Laurens-Wuyts (author)irinak12015-10-12

The Wizwiki-W7500 looks very interessting!

Laurens

irinak1 (author)2015-10-12

I was posted on our website your project.

link: http://wiznetmuseum.com/portfolio-items/android-v...

We developed the WIZwiki-W7500 using the Cortex -M0.

http://wizwiki.net/wiki/doku.php?id=products:wizwi...

Is that Interest in WIZwiki-W7500?

Chelsea AngelaO (author)2015-09-14

hello sir.. i was just learning in arduino, just part of my study..

I would like to ask a question about about that line:
byte mac[] = {0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED};

was it really constant? or can it be changed? Thank you sir.

You can change it in the program, but you cannot change it while the program is running.

But when you change it, you should watch out that there is no other device on your network with the same MAC address.

Laurens

About This Instructable

3,789views

41favorites

License:

Bio: twitter: @Laurens_Wuyts
More by Laurens-Wuyts:How to Use the Beelink SEA I Media PlayerMinimalistic Led ClockControl Raspberry pi GPIO using an app
Add instructable to: