Introduction: Diy Smart Home Assistant With Raspberry Pi and ReSpeaker Mic Array

ReSpeaker Mic Array, as the “ear” of Raspberry Pi here, can listen to your speech commands and send them to Raspberry Pi. According to your commands, Raspberry Pi will control Wio Link to do what you want via Wi-Fi.

We have talked the smart home for so many years, but our home is still not so smart enough. And this time, it is really coming, Amazon Echo, Google Home, Apple homekit and so on.

Step 1: What We Need

Raspberry Pi(with network connected) *1

ReSpeaker Mic Array with audio jack *1

SD card with Raspberry Pi image *1

Wio Link with WS2812 LED Strip *1

USB cable *2

PC or Mac

Step 2: Download Our GitHub

First, you should ssh to Raspberry Pi, and download our Github for ReSpeaker Mic Array:

git clone https://github.com/respeaker/mic_array.git

Step 3: Test the Mic Array

1. Run pixel_ring.py to control the pixel ring of the Mic Array through USB HID. After running this, ReSpeaker Mic Array will start to blink.

sudo pip install pyusb
sudo python pixel_ring.py

If you don't want to access USB device with sudo, add a udev .rules file to /etc/udev/rules.d:

echo 'SUBSYSTEM=="usb", MODE="0666"' | sudo tee -a /etc/udev/rules.d/60-usb.rules
sudo udevadm control -R  # then re-plug the usb device 

2. Read 8 channels audio from the Mic Array and estimate sound's DOA. Then the LEDs on Mic Array will show the direction of the sounds.

sudo apt-get install python-numpy    # or pip install numpypythonmic_array.py 

3. Do VAD and then estimate DOA. Then the LEDs on Mic Array will show the direction of the sounds, too.

sudo pip install webrtcvad
python vad_doa.py 

4. Do KWS and then estimate DOA.

Get snowboy work and run python kws_doa.py. Then you should say “alexa” to Mic Array to wake it up, if you sound is detected, the LEDs will show the direction of the sounds.

git submodule init
git submodule update
cd snowboy/swig/Python
sudo apt-get install python-dev libatlas-base-dev swig           # requiremetns to compile snowboy
echo 'from snowboydetect import *' >__init__.py     # create__init__.pyfor a python module
cd ../../..                                          # chang to the root directory of the repository
ln -s snowboy/swig/Python snowboydetect
python kws_doa.py

Step 4: Install ReSpeaker Python Library

Install ReSpeaker Python Library to use Bing Speech API

git clone https://github.com/respeaker/respeaker_python_library.git
cd respeaker_python_library 
python setup.py install

And you need a Bing key, how to get it?

Step 5: Setup Your Wio Link

Add WS2812 LED Strip to the digital0 port of Wio Link and update the firmware. Then click"view api"and get the access token.

If you are new to Wio Link, please click here for wiki and guide.

Step 6: Add Bing Key and Wio Token to Python Code

Save the code below in your Raspberry Pi, and don't forget to fill in Bing key and Wio token:

from respeaker.bing_speech_api import BingSpeechAPI as Bing
import wave from mic_array import MicArray import Queue from pixel_ring import pixel_ring import sys import numpy as np import collections from snowboydetect import SnowboyDetect import time import json from urllib import urlencode from urllib2 import Request, urlopen, URLError, HTTPError

# write your Wio token here

WIO_TOKEN = "**************"

# write your Bing key here

KEY = "**********" bing = Bing(key=KEY)

RATE = 16000 CHANNELS = 8 KWS_FRAMES = 10 # ms DOA_FRAMES = 800 # ms

detector = SnowboyDetect('snowboy/resources/common.res', 'snowboy/resources/snowboy.umdl') detector.SetAudioGain(1) detector.SetSensitivity('0.5')

# about 5seconds q = Queue.Queue(maxsize=768)

def gen_queue(q): try: data = q.get(timeout=1) while data: yield data data = q.get(timeout=1) except Queue.Empty: pass

def controlLED(onoff=0): try: if onoff == 1: rgb_hex_string = '000080' else: rgb_hex_string = '000000' url = 'https://cn.wio.seeed.io/v1/node/GroveLedWs2812D0/clear/4/{}?access_token={}'.format(rgb_hex_string, WIO_TOKEN) request = Request(url, data='') response = urlopen(request) data = response.read() result = json.loads(data) if result['result'] == 'ok': return True else: return False except Exception as err: return False

def main(): history = collections.deque(maxlen=int(DOA_FRAMES / KWS_FRAMES)) global q

try: with MicArray(RATE, CHANNELS, RATE * KWS_FRAMES / 1000) as mic: for chunk in mic.read_chunks(): history.append(chunk) # Detect keyword from channel 0 ans = detector.RunDetection(chunk[0::CHANNELS].tostring()) if ans > 0: print("wake up") print("start recording") pixel_ring.arc(12) q.queue.clear() for chunk in mic.read_chunks(): q.put(chunk[0::CHANNELS].tostring()) if q.full(): break print "queue full" pixel_ring.spin() text = bing.recognize(gen_queue(q)) # data can be generator if text: print('{}'.format(text)) if 'turn on' in text: controlLED(1) if 'turn off' in text: controlLED(0) pixel_ring.off()

except KeyboardInterrupt: pass

pixel_ring.off() # except ValueError: # pass

if __name__ == '__main__': main()

Step 7: Run the Demo

Save the Step6 python code in smart_home.py, then run it:

python smart_home.py

Wake it up with saying "snowboy", then say "please turn on the light", or "please turn off the light" to control Wio Link.

Enjoy it!!

Comments

About This Instructable

833views

13favorites

License:

Bio: Howdy, we are application engineers in Seeedstudio. Sharing projects with maker community is awesome. Hope you like it XD
More by Seeed Studio:Diy Smart Home Assistant With Raspberry Pi and ReSpeaker Mic ArrayBuild Google Assistant on Raspberry Pi With ReSpeaker Mic ArrayDIY a Gesture Keyboard That Can Control the Media Player
Add instructable to: