Introduction: Using Amazon Alexa to Control Ameba Arduino

By

Alexa is a Amazon's cloud voice service that allow developers to voice-control connected Amazon services. An example application is Amazon Echo, which is a voice-control assistant. When users speak to Amazon Echo, it analyzes the received voice and make appropriate response. In this example, we will show how to connect Amazon services (including Amazon Alexa, AWS Lambda, AWS IAM, AWS IoT) and use Alexa to control LED on Ameba Arduino board.

As shown in the above diagram, we can summarize the AWS framework scenario in this example.

  1. When a user says "Turn on the light" to the test device (which supports Amazon Alexa), Alexa analyzes the voice command according to the schema and sample utterances file (provided by user beforehand), and generates intent JSON (JavaScript Object Notation).
  2. Alexa submit the intent JSON to AWS Lambda. According to the intent, Lambda service then sends update message to AWS IoT Shadow service.
  3. AWS IoT Shadow service updates shadow state according to the update message from Lambda service.
    • If Ameba is online and has subscribed to the shadow service, Ameba turns on the LED.
    • Then Lambda generates the text and voice response message and send back to Alexa.
  4. Finally, the user can hear the voice message to indicate the LED has been successfully turned on.

Step 1: Devices

The needed devices are:

  • Ameba Arduino board - This Arduino compatible board is based on Realtek's RTL8195AM chipset which contains built-in wifi so we don't need an extra wifi shield for this project. You can find the Ameba Arduino baord in Amzon: https://www.amazon.com/dp/B01KWXJOGU
  • LED: Any simple two leads through hole LED will do.

We will start to show you how to register to Amazon Alexa, Lambda, and IoT services in the following steps and connect them together to deliver the purpose of this Instructables: Use voice to control IoT device through cloud. The process could be tedious, but you will be satisfied when you experience the final result.

Step 2: Amazon Alexa

Alexa provides two types of services:

  1. Custom Skills: developers can custom design the required interaction model, including conversation flow, analysis keyword, then send intent to AWS Lambda. We will use this custom type due to its flexibility.
  2. Smart Home Skills: Developers can use the smart home models provided by Alexa, and send intent.

Now we'll show you how to register to Alexa. First, go to Alexa webpage https://developer.amazon.com/alexa, click on "Sign in" on top right corner (you do have an Amzon account, don't you?). Click "ALEXA" in the middle of the top bar after you signed in.

Step 3: Alexa - Create Skills 1

When you are in Alexa home page, you can see two squares indicating two of Alexa voice-enabled capabilities. Let's select the first "Alexa Skills Kit" and click "Get Started". This will lead you to skill management page. Click "Add a New Skill" on the upper right corner.

Step 4: Alexa - Create Skills 2

Now we are going to fill in all needed skill information in this page:

  1. Skill type: Choose "Custom Interaction Model"
  2. Name: The name to show to user. Fill in "Control Light of Ameba".
  3. Invocation Name: The name that triggers a session to start. In general, there are two ways to start a session. First, speak complete service content, then Alexa would analyze the voice message and determine which session to activate. Second, user can speak only the invocation name to trigger a specific session. Here we fill in "Ameba Controller". Please note that you have to avoid to use some keywords used by Alexa, this link: https://developer.amazon.com/public/solutions/alex... have detailed invocation name limitation.Click Next.

Click "Next" on lower right corner which will lead us to Interaction Model page where we need to set up the Intent Schema. A skill can have many intents, each is triggered by different condition. An intent can have "slot" field (optional), which enumerates possible values.

  • Custom slot type: developer has to enumerate possible values for the slot.
  • Built-in slot type: Amazon has predefined some common slots, e.g., date, numbers, cities, ……, this kind of slots would have name starts with "AMAZON".

For detailed schema syntax, please refer to this Amazon Website: https://developer.amazon.com/public/solutions/alex...

Please fill in the schema as shown in the above picture (text source also attached). In this customized schema, we have an intent named "ControlLight", with a slot named "LightState" and custom slot type "LIGHT_STATE". Since LIGHT_STATE is a custom slot type, we need to define it, click "Add Slot Type".

Attachments

Step 5: Alexa - Create Skills 3

In this page, we need to fill in the slot type name and enumerate the values.In "Enter Type" field, please type in "LIGHT_STATE" and in "Enter Values" field, fill "on" in line 1 and "off" in line 2 then click "Save". Now we can see our customized slot type name and values are listed in the page.

Next, fill in the "Sample Utterances". This section tells Alexa the trigger sentence of the defined intent. The format begins with intent name followed by a space (or a tab), and followed by a sentence spoken by the user. Please fill in the values as the attached source (sampleUterances.txt). When finished, click "Next" on lower right corner and we'll come to the configuration page. In this page, we need to fill in the connected endpoint with Alexa. That is to tell Alexa where to pass the intent when the intent is triggered. Developers can build a server to receive the intent or use Amazon AWS Lambda service.In this example we use Amazon AWS Lambda. As in the above picture, you'll notice the button "AWS Lambda ARN (Amazon Resource Name)" which we have to register to Amazon Lambda in order to get this ARN. For now, just leave the current page open, and follow next few sections to configure Lambda service.

Attachments

Step 6: AWS Lambda 1

After we've finished the previous Alexa registeration, please open AWS Lambda in a new tab: https://aws.amazon.com/lambda/. Click "Sign in to the Console" on the upper right corner (Please use the same account that you use to login Amazon Alexa). You will enter the Amazon Web Services page. Since currently the service connection of Alexa and Lambda is only provided in us-east-1 region, click upper-right drop-down list to switch region to "US East (N. Virginia)". Then click "Lambda" in the AWS services list. Click "Get Started Now" once you are in AWS Lambda page.

Step 7: AWS Lambda 2

Lambda has predefined many usage scenarios with sample code and default setting. To connect Alexa and Lambda, fill in "Alexa" in the filter. This will limit the pre-defined blueprints to three functions as shown in the above picture. Choose "alexa-skills-kit-color-expert" in the middle. In this Lambda configure triggers page, configure the event that triggers Lambda to start the computation. Click the dot-border square, the predefined service list would appear and we will pick "Alexa Skills Kit" in this case. Click on "Next".

Step 8: AWS Lambda 3

Now we should be in Lambda function config page, we will need to configure the programming function.

  • Name - Used to identify the Lambda function, fill in "ControlLight".
  • Description - Used to describe the usage of the Lambda function. Just leave the default description as is.
  • Runtime - Indicating the programming language used. We will use "Node.js 4.3". You can switch to preferred programming language (Python, Java) here or afterwards.

In the next Lambda function code section, choose "Edit code inline" in the drop-down list and leave the default code as is. We will modify the code later.

Next we will come to the Lambda function handler and role section:

  • Handler - Use the default value, index.handler.
  • Role - Configure the Lambda function to appropriate authority to avoid the Lambda function from accessing prohibited service. The authority management service is provided by AWS IAM. Choose "Create a custom role" in the drop-down list of Role. You will be redirect to AWS IAM in a new browser tab.

Step 9: AWS Lambda 4

In this IAM management page, we need to input authority of role:

  • IAM Role - Leave the default role, "Create a new IAM role"
  • Role Name - Set the nsame to "control_light"

Click "View Policy Document", then click on "Edit". At this time, a popup window with Edit Policy description appears, simply click “OK” (you can also click the "Documentaion" lick to view the execution role description).We can edit the authority of the role. You can see the default code, the role is only authorized with "log" permission. Please modify the code as the attachment (authority.txt). After code modification, click "Allow" on the lower right corner. Now this IAM page will close and bring you back to the previous Lamda management page

Attachments

Step 10: AWS Lambda 5

Now you can see the Role under Lambda function handler and role has changed from "Create a custome role" to "Choose an existing role", and the Existing role filed has shown "control_light" which is just what we created moments ago. Click "Next" on lower right corner.

In this review page, you can review all related settings (which can also be changed afterwards). Click "Create Function" to complete this setup process.

In this Lambda management page, please pay attention to the upper right corner of the ARN string, this is the endpoint of this Lambda function which we will be needed to complete our Alexa configuration.

Step 11: AWS Lambda 6

Before we resume our Lambda configuration, we will need an IoT endpoint. This endpoint can be obtained once you are done registering to AWS IoT service. Please refer to the following Instructables for detail steps for registering to AWS IoT: https://www.instructables.com/id/Arduino-Using-AWS-... As shown in the above picture, the endpoint can be found on the right hand side of AWS IoT page indicated as "REST API endpoint".

Now let's keep moving to complete our AWS Lambda configuration. In the Lambda setting page, click on tab "Code", remove all the default code and replace it with code shown in this link: http://www.amebaiot.com/wp-content/uploads/2016/08...

After you have finished replacing the code, you need to change one line of code. At line 12:

var iotData = new AWS.IotData({endpoint: "a2zweh2b7yb784.iot.us-east-1.amazonaws.com"});

Change the string "a2zweh2b7yb784.iot.us-east-1.amazonaws.com" to your IoT endpoint that you got from your AWS IoT page.

In the code, you can also find the intent handler "ControlLight" and "LightState" slot that we defined in Alexa before. Click "Save" to save the change.

Step 12: AWS Lambda 7

Let's test our Lambda function. Click "Action", and choose "Configure test event" to configure events for testing. We can simulate an event from Alexa and the information of the event. Remove the default code and fill in the content as attached (testEvent.txt). Click "Save and test". Then you can see the execution result and log. In the response message, the "outputSpeech" message "I now know you want to turn on the light" will be spoken by Alexa when the service is deployed.

Attachments

Step 13: Alexa Configuration

Now we will go back to Alexa page to finish our configuartion. Please go to our previous Alexa configuration page and click on "AWS Lamda ARN (Amazon Resource name" button, next select "North America" then fill in the ARN from Lambda configuration. Click "Next"

Now we should be in Alexa test page where we can conduct some basic test. Use Voice Simulator to test how Alexa speaks the received voice data. It uses SSML tags to let Alexa speaks specific voice. Let's input: 'Here is a word spelled out: hi, there', click Listen to test.

We can fill in the sentence spoken by user in Service Simulator to test the response action of Alexa when receiving messages. Here we fill in “Turn on the light”, and click “Ask Control Light of Ameba”. The voice message will be handled by Alexa and send to Lambda function, then the function returns the result. Click "Listen" to hear the result.

Click "Next". Congratulations, we've have finished Alexa configuration and successfully connecting Alexa with Lambda functions. Next is to bring up our Ameba IoT and test with real voice!

Step 14: Ameba Programming

If you have Ameba Arduino board, please update your SDK to v1.1.4. Run Arduino IDE and open file: "File -> Examples-> AmebaMQTTClient -> amazon_awsiot_with_ack". There are some changes that needs to be made to suit your hardware and AWS registration.

  1. AP setup - Change the SSID & password of your local AP.
  2. MQTT server - This will be the REST API endpoint from your AWS IoT page that we mentioned before.
  3. Certificate - Leave the rootCABuff as is since we've applied the correct CA certificate in the sketch. Replace the certificateBuff and privateKeyBuff with the certificate.pem.crt and private.pem.key that you downlaod when you registered to AWS IoT.
  4. LED pin - The default is to connect LED anode to Arduino digital pin 10 with default state to be on. You can change the default as your board fit.

Compile the changed code and download to Ameba board. We are ready to test and control our Ameba with voice.

Step 15: Alexa Test

Before testing Alexa, we need to install Alexa App for our mobile device. Pelase searxh for "Alexa Listens" in your App store, download and install. We can manage the activated skills and status on the Alexa device via Web: http://alexa.amazon.com/spa/index.html#cards. You can also watch Alexa status in your mobile device by installing "Amazon Alexa".

Since the sentences "turn on" and "turn off" are highly relevant to the keywords of Amazon Smart home kit, we will use two voice commands to trigger the ControlLight skill.

  1. User: "Alexa, ask Ameba Controller". This command tells Alexa to search the Invocation Name, "Ameba Controller", therefore when Alexa hears this sentence, it opens a Ameba Controller session.
  2. Alexa responds: "Welcome to the Ameba Controller example. Please tell me next action by saying, turn on the light". Getting this message represent the Intent request is successfully received by the Lambda function.
  3. User: "Turn on the light", Lambda tries to send the command to turn on the light to AWS IoT thing shadow.
  4. Alexa: "I now know you want to turn on the light". The light should be turned on by now. The Lambda function close the session after it sends command to AWS IoT thing shadow.

The Alexa demo video is attached for your reference.