Introduction: Diy Smart Home Assistant With Raspberry Pi and ReSpeaker Mic Array

About: Howdy, we are application engineers in Seeed. Sharing projects with maker community is awesome. Hope you like it XD Seeed is the IoT hardware enabler providing services that empower IoT developers to swiftly …

ReSpeaker Mic Array, as the “ear” of Raspberry Pi here, can listen to your speech commands and send them to Raspberry Pi. According to your commands, Raspberry Pi will control Wio Link to do what you want via Wi-Fi.

We have talked the smart home for so many years, but our home is still not so smart enough. And this time, it is really coming, Amazon Echo, Google Home, Apple homekit and so on.

Step 1: What We Need

Raspberry Pi(with network connected) *1

ReSpeaker Mic Array with audio jack *1

SD card with Raspberry Pi image *1

Wio Link with WS2812 LED Strip *1

USB cable *2

PC or Mac

Step 2: Download Our GitHub

First, you should ssh to Raspberry Pi, and download our Github for ReSpeaker Mic Array:

git clone

Step 3: Test the Mic Array

1. Run to control the pixel ring of the Mic Array through USB HID. After running this, ReSpeaker Mic Array will start to blink.

sudo pip install pyusb
sudo python

If you don't want to access USB device with sudo, add a udev .rules file to /etc/udev/rules.d:

echo 'SUBSYSTEM=="usb", MODE="0666"' | sudo tee -a /etc/udev/rules.d/60-usb.rules
sudo udevadm control -R  # then re-plug the usb device 

2. Read 8 channels audio from the Mic Array and estimate sound's DOA. Then the LEDs on Mic Array will show the direction of the sounds.

sudo apt-get install python-numpy    # or pip install 

3. Do VAD and then estimate DOA. Then the LEDs on Mic Array will show the direction of the sounds, too.

sudo pip install webrtcvad

4. Do KWS and then estimate DOA.

Get snowboy work and run python Then you should say “alexa” to Mic Array to wake it up, if you sound is detected, the LEDs will show the direction of the sounds.

git submodule init
git submodule update
cd snowboy/swig/Python
sudo apt-get install python-dev libatlas-base-dev swig           # requiremetns to compile snowboy
echo 'from snowboydetect import *' >     # create__init__.pyfor a python module
cd ../../..                                          # chang to the root directory of the repository
ln -s snowboy/swig/Python snowboydetect

Step 4: Install ReSpeaker Python Library

Install ReSpeaker Python Library to use Bing Speech API

git clone
cd respeaker_python_library 
python install

And you need a Bing key, how to get it?

Step 5: Setup Your Wio Link

Add WS2812 LED Strip to the digital0 port of Wio Link and update the firmware. Then click"view api"and get the access token.

If you are new to Wio Link, please click here for wiki and guide.

Step 6: Add Bing Key and Wio Token to Python Code

Save the code below in your Raspberry Pi, and don't forget to fill in Bing key and Wio token:

from respeaker.bing_speech_api import BingSpeechAPI as Bing
import wave from mic_array import MicArray import Queue from pixel_ring import pixel_ring import sys import numpy as np import collections from snowboydetect import SnowboyDetect import time import json from urllib import urlencode from urllib2 import Request, urlopen, URLError, HTTPError

# write your Wio token here

WIO_TOKEN = "**************"

# write your Bing key here

KEY = "**********" bing = Bing(key=KEY)

RATE = 16000 CHANNELS = 8 KWS_FRAMES = 10 # ms DOA_FRAMES = 800 # ms

detector = SnowboyDetect('snowboy/resources/common.res', 'snowboy/resources/snowboy.umdl') detector.SetAudioGain(1) detector.SetSensitivity('0.5')

# about 5seconds q = Queue.Queue(maxsize=768)

def gen_queue(q): try: data = q.get(timeout=1) while data: yield data data = q.get(timeout=1) except Queue.Empty: pass

def controlLED(onoff=0): try: if onoff == 1: rgb_hex_string = '000080' else: rgb_hex_string = '000000' url = '{}?access_token={}'.format(rgb_hex_string, WIO_TOKEN) request = Request(url, data='') response = urlopen(request) data = result = json.loads(data) if result['result'] == 'ok': return True else: return False except Exception as err: return False

def main(): history = collections.deque(maxlen=int(DOA_FRAMES / KWS_FRAMES)) global q

try: with MicArray(RATE, CHANNELS, RATE * KWS_FRAMES / 1000) as mic: for chunk in mic.read_chunks(): history.append(chunk) # Detect keyword from channel 0 ans = detector.RunDetection(chunk[0::CHANNELS].tostring()) if ans > 0: print("wake up") print("start recording") pixel_ring.arc(12) q.queue.clear() for chunk in mic.read_chunks(): q.put(chunk[0::CHANNELS].tostring()) if q.full(): break print "queue full" pixel_ring.spin() text = bing.recognize(gen_queue(q)) # data can be generator if text: print('{}'.format(text)) if 'turn on' in text: controlLED(1) if 'turn off' in text: controlLED(0)

except KeyboardInterrupt: pass # except ValueError: # pass

if __name__ == '__main__': main()

Step 7: Run the Demo

Save the Step6 python code in, then run it:


Wake it up with saying "snowboy", then say "please turn on the light", or "please turn off the light" to control Wio Link.

Enjoy it!!