Speaking Assistant [Intel IoT]

Internet is filled with all kinds of useful information. We are going to build a gadget in this Instructuble using Intel Galileo Gen2 that you can easily use to gather the information and speak out the information using TTS (text to speech synthesis software).

The information available on Internet is aggregated into what is known as RSS feed which stands for Rich Site Summary/Really simple syndication. This web technology is used to update frequently changing information such as News, Stock price, weather etc... Let's say you need to know the stock price of a corporation. There are plenty of web services that update this particular information and each service provider assigns an unique address for the stock price of your favorite corporation when you access this address, as a response you get the current stock price (with some added meta data).

For user input, we are going to use an infrared proximity sensor such as this one to detect the user action and to determine what information the user wants from the Internet.

How does it work?

The main interface between the user and the device is an IR proximity sensor. These sensors get triggered whenever some obstacle is placed in front of the sensor. Various operations are triggered depending on number of times the sensor is triggered (operations are configurable).

Here are the features:

• Reads out the RSS feeds
• Displays number of unread emails in inbox (gmail only)
• Changes LCD back light depending on number of unread emails in inbox
• Reads out temperature
• Reads out emails (sender and subject)
• Control digital pins

Have a look at the attached video to see the above features in actions.

Intel IoT Lniux image will be used for this project. Hence you'll need a micro SD card to load up the Linux. For TTS, espeak will be used.

We will also need an USB sound card which when connected to a speaker, will speak out the information.

So to summarize, following things will be required to complete this Instructable successfully:

• Intel Galileo Gen2
• micro SD card with at least 2 GB capacity
• Grove kit; Intel IoT edition with Serial cable
• IR proximity sensor
• USB sound card

This project is named "Speaking Assistant" for the lack of creativity :p

Step 1: Setting Up the SD Card

First thing to do is get the Intel IoT Linux image onto the SD card. Go to this link and click on "SD-Card Linux Image" and save the image. To get the image onto SD card, follow the applicable instructions

We will have to install espeak package along with it's dependencies for which we need to have access to the Linux console.

P:S: Visit this blog post for more detailed explanation.

Step 2: Accessing the Console

We will have to install espeak package along with it's dependencies for which we need to have access to the Linux console.

Connect the Serial cable supplied with Grove kit.

Insert the SD card and power up the Intel Galileo.

Download putty from here and follow the instructions given in this post to get to the Console.

Step 3: Installing the Required Packages

The packages that we need are on different server than the default one. We need to add additional repository.
Add additional repos by editing the base-feeds.conf file using vi

vi /etc/opkg/base-feeds.conf

hit 'i' to start editing the file and paste the following lines

src/gz all      http://repo.opkg.net/galileo/repo/all
src/gz clanton  http://repo.opkg.net/galileo/repo/clanton 
src/gz i586     http://repo.opkg.net/galileo/repo/i586

Then press 'Esc' key and then ":wq" to save the file and exit.
Install the espeak using following commands: opkg update

opkg install espeak

Insert the USB sound card, connect the audio out to a speaker and enter the following command:

espeak "Hello world"

If every thing went well, you should hear Galileo speak out "Hello world".

Next, we need to install RSS parser library for Python for fetching and reading out the feeds and we will also need the "GMail for Python" library for fetching and reading the mails out. Next step is to install them.

On the console, type following commands

easy_install pip

pip install feedparser   # RSS parser for python


mkdir gmail 

cd gmail 

git clone git://github.com/charlierguo/gmail.git

cd gmail

python setup.py install

cd  # Move to home directory

For the above steps to work, Galileo should be connected to the Internet.

P.S: You can also use methods listed here to transfer files between your computer and the Galileo instead of using the USB drive.

Step 4: Making Hardware Connections

For this step, you'll need following things

1. Grove Kit base shield
2. Grove kit Buzzer
3. Grove kit LCD
4. Grove kit LED (optional)
5. Grove kit temperature sensor
6. IR proximity sensor

Make the following connections:

• Buzzer - D5
• LCD - I2C0
• LED-D4
• Temperature sensor-A1
• IR proximity sensor
  • Vcc/5v to 5V on Galileo header
  • Gnd to any of the GND on Galileo header
  • Output to D7 on Galileo header

You can connect the USB audio card directly to the Galileo USB port but I recommend using an externally powered USB hub to connect the audio card for better performance.

Step 5: Configuring the System

In this step you'll learn to personalize the system.

Download "SpeakingAssistant.zip" and extract the contents of the archive into a folder "SpeakingAssistant".

All of the configuration information is stored in the file "configuration.py" as Python variables. Open "configuration.py" if you want to configure any of the below parameters.

There are five variables that you can change. The first among them is "operations"

"operations" is a list of tuples.

The first tuple in the list, in our case is "RSS" defines what action to take when the IR proximity sensor is triggered once. Similarly, subsequent elements define what action is to be performed when IR proximity sensor is triggered as many times.

Currently following operations are defined:

• RSS
  • You define this operation using "RSS" tag. Triggering this operation will cause the app to fetch the RSS and speak it out
  • First element shall be "RSS" followed by RSS feed address specified in quotes and that is to be followed by content within the feed that you are interested in
  • ex: ("RSS", "http://www.moneycontrol.com/rss/MCtopnews.xml", "title")
• Temperature:
  • Reads out temperature using the Grove temperature sensor connected to one of the analog pins and displays the same on the LCD
  • First element will be "Temperature" followed by the analog in pin. In below example analog pin would be 1
  • ex:("Temperature", 1)
• Mail:
  • Reads out the mail in gmail inbox and displays the sender and subject on LCD
  • First element should be "Mail" followed by a dummy argument (meant to ease parsing)
  • ex: ("Mail", "0")
• GPIO:
  • Controls a GPIO(digital) pin. You can connect a relay to control other devices
  • First element will be "gpio" followed by "on" or "off" indicating the digital pin has to be turned on or off respectively. The last element will be the digital pin 4, in the below example
  • ex: ("gpio", "on", 4)

Next in the configuration file is senders list. This list will be used by the mail thread to fetch only unread emails in inbox from the senders listed in the list. If left empty, all the unread mails will be fetched.

ex: senders = ['[e-mail]',]

"unreadMailCnt" controls the color of LCDs backlight depending on the number of unread mails in the inbox. As the number of unread mails approaches value of "unreadMailCnt", the LCD backlight starts to turn red from green.

ex: unreadMailCnt = 50

"ttsSpeed" let's you control the speed with which the text will be spoken.

ex: ttsSpeed = 150

Use "mailCredentials" to configure the gmail login details.

ex: mailCredentials = {'userName':"[e-mail]", 'pwd':"password"}

Replace '[e-mail]' and 'password' with your account details.

Step 6: Configuring the RSS

While all configuration parameters are easy to configure except for the RSS hence this step. This step is to help you configure the RSS feed.

In this step I'll walk through one example for getting the information that you need from the feed.

Let's say I am interested in knowing the whether of Bengaluru. The first thing to do is google "Bengaluru weather RSS feed" select any of the site and look for the "RSS" logo and right click on it and "copy link location". Paste this link in the configuration file in place of feed parser address.

In place of content of interest, you can try 'title' or 'summary' or 'summary_detail' make a few trails and decide what suits best for you.

With all these information in hand, the config parameter would be some thing like this:

("RSS", "http://in.weather.com/rss-INXX0012", "summary")

Similarly you can construct configuration for news, stock prize etc.

Step 7: Starting the Applicaton

To enable gmail related features on the device, log in to your gmail account from your browser click on "My account"->"sign-in and security" and scroll down till you find option "Allow less secure apps: off" and then toggle it to "on"

Now is the time to get the Python script to be copied onto the Galileo. The folder created from the zip file you downloaded, extracted and modified in step 5 has to be copied onto a USB drive. Connect the USB drive to USB port on Galileo (or alternatively, use methods listed here to transfer the folder).

In Linux console of Intel Galileo, type following command

mount /dev/sda1 /mnt
cp -r /mnt/SpeakingAssistant /home/root/  

cd /home/root/SpeakingAssistant

umount /dev/sda1

Remove USB drive.

Connect the USB audio card to the Galileo (preferably via an externally powered USB hub for better audio performance). Connect speakers to USB card.

You may need to setup the correct time using "date" command. You might have to do this every time you run the application after resetting the Galileo since it lacks battery backed RTC(you can add one if you want to).

Enter following commands in the Galileo console:

date --date="3 JUNE 2015 21:41:00"    # Set the current date time here 

Run the application using following set of commands

cd  /home/root/SpeakingAssistant

python App.py # or do 'chmod +x App.py && ./App.py'

To exit the app any time, press 'ctrl+c'.

You can also configure the application to run automatically using cron.