Persist and Visualize AWS IoT Data Using Thingsboard

While some of the AWS IoT and Thingsboard features overlap, you are able to integrate them and leverage best features from both. For example, you can collect data using AWS IoT and then push it to Thingsboard for storage and data visualization on customizable end-user dashboards.

Thingsboard is an open-source server-side platform that allows you to monitor and control IoT devices. It is free for both personal and commercial usage and you can deploy it anywhere. If this is your first experience with the platform I recommend to review what-is-thingsboard page and getting-started guide.

There are two options how one can integrate AWS IoT and Thingsboard.

First option is to write custom lambda function based on Thingsboard APIs. This is quite simple solution, however, it allows only to push data to Thingsboard and does not provide ability to control your devices using Thingsboard widgets.

Second option is to use Thingsboard IoT Gateway, which is an open-source solution that allows you to integrate devices connected to legacy and third-party systems with Thingsboard. We recommend to use second option and will cover basic configuration steps below.

Overview

Thingsboard IoT Gateway is a light-weight service that connects to both AWS IoT MQTT broker and Thingsboard MQTT server and acts as aproxy or API bridge. You are able to configure the Gateway to subscribe to certain AWS IoT topics, convert incoming data to unified format and push it to Thingsboard. This article provides basic configuration steps. You can refer to advanced configuration topic for more details.

Step 1: Prerequisites

We assume you have already installed Thingsboard IoT Gateway and provisioned it within your local or demo Thingsboard instance.

AWS IoT configuration steps

Before configuration of the Thingsboard IoT Gateway we must prepare certificates, policies and copy Rest URL from the AWS IoT console.

Step 2: Custom Endpoint URL of the AWS IoT

Get Custom Endpoint URL of the AWS IoT that we will use later in this guide. This URL is located in the AWS IoT Settings page.

We will refer later to this URL as “$MQTT_ENDPOINT”.

Step 3: AWS IoT Certificates

Download certificates from AWS IoT and copy them beside Thingsboard IoT Gateway in the configuration folder:

Windows: YOUR_INSTALL_DIR/conf
Linux: /etc/tb-gateway/conf

You can put it inside conf folder or create new sub-folder cert for example.

We need to copy private key (2f3b7147dd.private.key in the example), certificate (2f3b7147dd.cert.pem in the example) and root CA certificate that you are able to download from Symantec.

Please copy these three PEM files to configuration folder of Thingsboard IoT Gateway as described above.

Later we will refer to the path where private key PEM file is located as “$PRIVATE_KEY”, certificate as “$CERTIFICATE” and root CA as “$ROOT_CA_CERT”.

Step 4: AWS IoT Policy Configuration

Configure security policy.

In the example above we have allowed any IoT action and for any resources, but you definitely can restrict these values based on your security rules.

Step 5: Enable MQTT Extension in Thingsboard IoT Gateway

Navigate to the gateway configuration folder and edit tb-gateway.yml file. Please change mqtt.enabled property value to true to enable Gateway MQTT extension.

Step 6: MQTT Extension Configuration

Now it’s time to configure Thingsboard IoT Gateway to connect to your AWS IoT broker.

Configuration of the brokers is located in mqtt-config.json file.

You should update it using next values:

{
    "host": "$MQTT_ENDPOINT",
    "port": 8883,
    "ssl": true,
    "retryInterval": 3000,
    "credentials": {
        "type": "cert.PEM",
        "caCert": "$ROOT_CA_CERT",
        "privateKey": "$PRIVATE_KEY",
        "cert": "$CERTIFICATE"
    }
    ...
}

here is sample with real values:

{
    "host": "a2ljyhf3dvidme.iot.us-east-1.amazonaws.com",
    "port": 8883,
    "ssl": true,
    "retryInterval": 3000,
    "credentials": {
        "type": "cert.PEM",
        "caCert": "/etc/tb-gateway/conf/cert/rootCA.pem",
        "privateKey" : "/etc/tb-gateway/conf/cert/privateKey.pem",
        "cert": "/etc/tb-gateway/conf/cert/cert.pem"
    }
    ...
}

Configuration of the broker is done. Now you are ready to start Thingsboard IoT Gateway and publish messages to AWS IoT topics that will be consumed by Thingsboard IoT Gateway and republished to Thingsboard instance.

Step 7: Dry Run

Consider that we have next default configuration of the mapping:

{
    "topicFilter": "sensor/+/temperature",
    "converter": {
        "type": "json",
        "filterExpression": "",
        "deviceNameTopicExpression": "(?<=sensor\/)(.*?)(?=\/temperature)",
        "timeseries": [
            {
                "type": "double",
                "key": "temperature",
                "value": "${$.value}"
            }
        ]
    }
}

To check that everything is configured correctly you are able to use mosquitto_pub tool that is able to publish messages to AWS IoT.

Here is a sample of the command that will publish temperature readings to AWS IoT topic sensor/SN-001/temperature. Thingsboard IoT Gateway will receive this values, create or update device SN-001 inside Thingsboard, and publish telemetry ‘temperature’ using value 73.8

mosquitto_pub --cert ./cert/cert.pem --key ./cert/privateKey.pem --cafile ./cert/rootCA.pem -h a2ljyhf3dvipme.iot.us-east-1.amazonaws.com -p 8883 -t sensor/SN-001/temperature -m '{"value":73.8}'

To validate that data arrived to Thingsboard, please open the administration UI and navigate to Devices->SN-001->Latest Telemetry. See screen-shoot above.

Introduction: Persist and Visualize AWS IoT Data Using Thingsboard