Instructables

Wireless Audio Streaming and Speech Recognition with Arduino

Picture of Wireless Audio Streaming and Speech Recognition with Arduino
BitVoicerTest3_1_800x600.jpg
BitVoicerTest3_2_800x600.jpg
In this project I connect my Arduino Board to a WiFly module (https://www.sparkfun.com/products/10822) from Roving Networks and stream audio to the PC, so that BitVoicer (http://www.bitsophia.com/BitVoicer.aspx) can perform speech recognition. By using a wireless module, I managed to add speech recognition features to my microcontroller without the need of a physical connection with the PC.

The LEDs do the same basic things they did in my last project (http://www.instructables.com/id/Speech-Recognition-with-Arduino/), but now I eliminated the PC wire connection, so I can finally have a speech activated Arduino anywhere my wireless network reaches.

Here is the YouTube video of the project:



I used a WiFly module from Roving Networks (https://www.sparkfun.com/products/10822) because I read somewhere that it would work with the XBee Shield I had (https://www.sparkfun.com/products/9976). Unfortunately, I found out the hard way that a diode between the Arduino TX and the WiFly RX was blocking the signal. I had to rip out the diode and add a voltage divider between the TX/RX pins of the Arduino and the WiFly module because the RX pin of the WiFly is 3.3V tolerant and the Arduino TX works at 5V. It does not seem to be the best approach, but it worked and I was able to send and receive data from the module. I found out that Sparkfun released a new version of the XBee Shield (https://www.sparkfun.com/products/10854) and that the diode level shifter was replaced with a more robust MOSFET level shifter. This should solve the diode problem.

I believe any of the following wireless modules could be used. You just have to find out which one of them best fits your needs:

- http://arduino.cc/en/Main/ArduinoWiFiShield
- http://www.rovingnetworks.com/products/RN171XV
- https://www.sparkfun.com/pages/xbee_guide (you will need a XBee Explorer Dongle to connect the XBee to your computer: https://www.sparkfun.com/products/9819)
- Many other Arduino WiFi Shields can be found here: http://postscapes.com/arduino-wifi

The setup process of the WiFly module can be tricky, so I’m also adding a step-by-step tutorial on how to do it. In my case, I use a regular access point/router from D-Link to implement my private wireless network. This tutorial should work with any access point out there.

1. Disconnect your computer from any wireless network
2. Set the GPIO9 pin of the WiFly module to high (3.3V) at power up. This enables adhoc mode on the module. I used the 3.3V source from the Arduino board and an ordinary jumper wire. Right after the power up, remove the wire and follow the next steps. Module datasheet: http://dlnmh9ip6v2uc.cloudfront.net/datasheets/Wireless/WiFi/WiFly-RN-XV-DS.pdf
3. On your computer, search for available networks and connect to the WiFly module network (WiFly-GSX-XX). It may take a few seconds before your computer gets an IP address from the module.
4. Download and run a free software called TeraTerm (http://www.rovingnetworks.com/resources/download/86/Teraterm)
5. Enter the following settings and click on OK: IP Address: 169.254.1.1; TCP port: 2000; Service: Telnet; Protocol: IPv4
6. You should see the word “*HELLO*” on the screen.
7. Type $$$ to enter command mode
8. Type scan and hit enter to scan for available access points
9. If your network uses WPA authentication, use the command set wlan phrase   to set the pass phrase. For WEP, set the key using the set wlan key   command. My network is WEP secured, so I had to use the second command. The problem is that the key must be entered ASCII/HEX encoded and it MUST have exactly 26 characters. I found this page that converts text to ASCII/HEX: http://www.string-functions.com/string-hex.aspx.
10. Type set wlan ssid , where XXXX is the SSID of you network (look at the results of step 8)
11. Type save
12. Type join  (look at the results of step 8)
13. The WiFly-GSX-XX will disappear from the windows network list
14. Close Teraterm
15. Reconnect your computer to your local wireless network
16. Turn your Arduino board OFF and back ON
17. Write down the IP number assigned to the WiFly module by your access point (usually, you can find this info on the access point DHCP page)
18. Open Teratem using the new IP address and the other settings from step 5
19. Type $$$ to enter command mode
20. Type set comm close 0 and hit enter
21. Type set comm open 0 and hit enter
22. Type set comm remote 0 and hit enter
23. Type set comm size 1460 and hit enter
24. Type set comm time 1000 and hit enter
25. Type set comm baud 115200 and hit enter (the same baud rate I used in my sketch)
26. Type save and hit enter
27. Type exit and hit enter
28. Close Teraterm

Now your WiFly module should be able to communicate with BitVoicer and your PC. The WiFly user’s manual can be downloaded here: http://dlnmh9ip6v2uc.cloudfront.net/datasheets/Wireless/WiFi/WiFly-RN-UM.pdf

This is the sketch I used to control the LEDs, capture audio and interact with BitVoicer:

#include <BitVoicer11.h>

//Instantiates the BitVoicerSerial class
BitVoicerSerial bvSerial = BitVoicerSerial();

//Stores true if the Audio Streaming Calibration tool
//is running
boolean sampleTest = false;
//Stores the data type retrieved by getData()
byte dataType = 0;
//Sets up the pins and default variables
int pinR = 3;
int pinY = 5;
int pinG = 6;
int lightLevel = 0;

void setup()
{
  //Sets the analog reference to external (AREF pin)
  //WARNING!!! If anything is conected to the AREF pin,
  //this function MUST be called first. Otherwise, it will
  //damage the board.
  bvSerial.setAnalogReference(BV_EXTERNAL);
  //Sets up the microcontroller to perform faster analog reads
  //on the specified pin
  bvSerial.setAudioInput(0);
  //Starts serial communication at 115200 bps
  Serial.begin(115200);
  //Sets up the pinModes
  pinMode(pinR, OUTPUT);
  pinMode(pinY, OUTPUT);
  pinMode(pinG, OUTPUT);
}

void loop()
{
  //Captures audio and sends it to BitVoicer if the Audio
  //Streaming Calibration Tool is running
  if (sampleTest == true)
  {
    //The value passed to the function is the time
    //(in microseconds) that the function has to wait before
    //performing the reading. It is used to achieve about
    //8000 readings per second.
    bvSerial.processAudio(46);
  }
 
  //Captures audio and sends it to BitVoicer if the Speech
  //Recognition Engine is running
  if (bvSerial.engineRunning)
  {
    //The value passed to the function is the time
    //(in microseconds) that the function has to wait before
    //performing the reading. It is used to achieve about
    //8000 readings per second.
    bvSerial.processAudio(46);
  }
}

//This function runs every time serial data is available
//in the serial buffer after a loop
void serialEvent()
{
  //Reads the serial buffer and stores the received data type
  dataType = bvSerial.getData();
 
  //Changes the value of sampleTest if the received data was
  //the start/stop sampling command
  if (dataType == BV_COMMAND)
      sampleTest = bvSerial.cmdData;
 
  //Signals BitVoicer's Speech Recognition Engine to start
  //listening to audio streams after the engineRunning status
  //was received
  if (dataType == BV_STATUS && bvSerial.engineRunning == true)
    bvSerial.startStopListening();
 
  //Checks if the data type is the same as the one in the
  //Voice Schema
  if (dataType == BV_STR)
    setLEDs();
}

//Performs the LED changes according to the value in
//bvSerial.strData
void setLEDs()
{
  if (bvSerial.strData == "wake")
  {
    digitalWrite(pinR, LOW);
    digitalWrite(pinY, LOW);
    digitalWrite(pinG, LOW);
    digitalWrite(pinR, HIGH);
    digitalWrite(pinY, HIGH);
    digitalWrite(pinG, HIGH);
    delay(200);
    digitalWrite(pinR, LOW);
    digitalWrite(pinY, LOW);
    digitalWrite(pinG, LOW);
    delay(200);
    digitalWrite(pinR, HIGH);
    digitalWrite(pinY, HIGH);
    digitalWrite(pinG, HIGH);
    delay(200);
    digitalWrite(pinR, LOW);
    digitalWrite(pinY, LOW);
    digitalWrite(pinG, LOW);
    delay(200);
    digitalWrite(pinR, HIGH);
    digitalWrite(pinY, HIGH);
    digitalWrite(pinG, HIGH);
    delay(200);
    digitalWrite(pinR, LOW);
    digitalWrite(pinY, LOW);
    digitalWrite(pinG, LOW);
    lightLevel = 0;
  }
  else if (bvSerial.strData == "sleep")
  {
    digitalWrite(pinR, LOW);
    digitalWrite(pinY, LOW);
    digitalWrite(pinG, LOW);
    digitalWrite(pinR, HIGH);
    digitalWrite(pinY, HIGH);
    digitalWrite(pinG, HIGH);
    delay(200);
    digitalWrite(pinR, LOW);
    digitalWrite(pinY, LOW);
    digitalWrite(pinG, LOW);
    delay(200);
    digitalWrite(pinR, HIGH);
    digitalWrite(pinY, HIGH);
    digitalWrite(pinG, HIGH);
    delay(200);
    digitalWrite(pinR, LOW);
    digitalWrite(pinY, LOW);
    digitalWrite(pinG, LOW);
    lightLevel = 0;
  }
  else if (bvSerial.strData == "RH")
  {
    digitalWrite(pinR, HIGH);
    lightLevel = 255;
  }
  else if (bvSerial.strData == "RL")
  {
    digitalWrite(pinR, LOW);
    lightLevel = 0;
  }
  else if (bvSerial.strData == "YH")
  {
    digitalWrite(pinY, HIGH);
    lightLevel = 255;
  }
  else if (bvSerial.strData == "YL")
  {
    digitalWrite(pinY, LOW);
    lightLevel = 0;
  }
  else if (bvSerial.strData == "GH")
  {
    digitalWrite(pinG, HIGH);
    lightLevel = 255;
  }
  else if (bvSerial.strData == "GL")
  {
    digitalWrite(pinG, LOW);
    lightLevel = 0;
  }
  else if (bvSerial.strData == "ALLON")
  {
    digitalWrite(pinR, HIGH);
    digitalWrite(pinY, HIGH);
    digitalWrite(pinG, HIGH);
    lightLevel = 255;
  }
  else if (bvSerial.strData == "ALLOFF")
  {
    digitalWrite(pinR, LOW);
    digitalWrite(pinY, LOW);
    digitalWrite(pinG, LOW);
    lightLevel = 0;
  }
  else if (bvSerial.strData == "brighter")
  {
    if (lightLevel < 255)
    {
      lightLevel += 85;
      analogWrite(pinR, lightLevel);
      analogWrite(pinY, lightLevel);
      analogWrite(pinG, lightLevel);
    }
  }
  else if (bvSerial.strData == "darker")
  {
    if (lightLevel > 0)
    {
      lightLevel -= 85;
      analogWrite(pinR, lightLevel);
      analogWrite(pinY, lightLevel);
      analogWrite(pinG, lightLevel);
    }
  }
  else
  {
    bvSerial.startStopListening();
    bvSerial.sendToBV("ERROR:" + bvSerial.strData);
    bvSerial.startStopListening();
  }
}


The BitVocier Voice Schema I used can be downloaded here: http://www.justbuss.xpg.com.br/BitVoicerTest3.zip

If you have any question about this project, please post it here, so anyone else can read the answer.
abevacquadj10 months ago
Does it work stand alone? Without pc?
leandro4b (author)  abevacquadj8 months ago
No. The speech recognition is performed on the PC. You will always need a logical connection to it.
adodman8 months ago
What port and settings do we need to put in BitVoicer? Is this port chnageable and do I define it on the Arduino, the WiFly or BitVoicer?
Thanks
leandro4b (author)  adodman8 months ago
If you are using a WiFly module and it is properly configured, the IP address will be automatically assigned by your access point. The module's default TCP port is 2000.
MichaelGokturk10 months ago
That...is SO cool. Well done and thank you for the instructable! I wanna build now!!!
hykenyde1 year ago
Hi leandro4b,
Firstly thaks for instruction step by step. I and my classmates are developing a project similar to this but we are trying to implement a wifi shield V2 is that you could help us?
leandro4b (author)  hykenyde1 year ago
I'm sorry hykenyde, but I do not know this wifi shield.
ckrmustafa1 year ago
Hi leandro4b,
Firstly thaks for instruction step by step. I made everything that you instructed above and I could be success till getting new IP address form access point. I am new on teraterm so need your help. When I save SSID on teraterm it replied me "AOK". Is this answer right? And another issue is that on step 16. When the Arduino restarted WiFly module run as normal mode or Ad-hoc mode?
Till step 18 there is problem. But at that point I got connetion error. What can be wrong? Thanks...
leandro4b (author)  ckrmustafa1 year ago
Hi ckrmustafa,
There were a few errors on step 9, 10 and 12 which I just fixed. The interface (page) where I entered the tutorial got rid of a few characters. See if that works out now.