IOT123 - ASSIMILATE SENSOR HUB: ICOS10 CORS WEBCOMPONENTS

The ASSIMILATE SENSOR/ACTOR Slaves embed metadata that is used for the defining visualizations in Crouton. This build is slightly different from previous ones; there are no hardware changes. The firmware now supports hosting custom (richer) editors that can be integrated into the latest build of AssimilateCrouton. More attention will be given to explaining the firmware and the MQTT dashboard in this article.

One of the advantages of serving WebComponents from the device that they control, is that the more advanced control of the device is limitted to the network that device is connected to: your WiFi Access Point. Although once you use an MQTT server with authentication there is a resemblance of protection, on public networks if you leave your Browser momentarily (AssimilateCrouton website) someone could jump in and control your automation devices. This CORS WebComponent feature makes it possible to only have readings (temp, light levels, moisture) shown publicly and command functions (on/off, scheduling) only available from the device network.

On the device, all of the webserver features with Authentication and Hosting in SPIFFS are still supported, but special focus has been made for CORS (Cross Origin Resource Sharing) support for Polymer WebComponents (Crouton uses Polymer 1.4.0).

In AssimilateCrouton (the fork of Crouton used for Assimilate IOT Network) the changes include

• support for a Device card (assim-device) that among other things shows and hides, for a user, individual cards for a device
• info property on all cards that shows a toast of useful contextual information for a card
• support for CORS webcomponents, in this case hosted on the webserver on the device (ESP8266).

Step 1: CROUTON

Croutonis a dashboard that lets you visualize and control your IOT devices with minimal setup. Essentially, it is the easiest dashboard to setup for any IOT hardware enthusiast using only MQTT and JSON.

The ASSIMILATE SLAVES (sensors and actors) have embedded metadata and properties that the master uses to build up the deviceInfo json packet that Crouton uses to build the dashboard. The intermediary between ASSIMILATE NODES and Crouton is a MQTT broker that is websockets friendly: Mosquito is used for the demo.

As the ASSIMILATE MASTER requests properties, it formats the response values in the required format for Crouton updates. The AssimilateCrouton fork adds some features that enable you to decentralize the business rules that run your device i.e. the IOT device does not need any embedded business rules, its just a pipeline for MQTT/I2C communication to the smarter (ATTINY controlled) slave actors and sensors.

Step 2: ASSIMILATE CROUTON

CHANGES TO CROUTON

Changes from the forked version include:

• if an endpoint has a path property defined, the WebComponent for the card will do an HTMLImport for a CORS resource (the webserver on the ESP8266 in this build).
• any resources upstream from (dependencies of) a CORS WebComponent are referenced as if they are served from the Crouton website; when they fail to load an exception handler rejigs the paths and loads if from the website.
• a current local time is displayed top-right, useful for scheduling verification.

POLYMER DEPENDENCIES AND CORS

The leafs of a Polymer dependency tree can be hosted in CORS. Because the root dependencies may be used several times in an app, they cannot be referenced from 2 locations (the website and the device) because the Polymer Module Loader treats them as 2 separate resources and multiple registration errors quickly flounder an application.

For this reason the WebComponent for a card (HTML file in 1.4.0) and the associated CSS file are the only files hosted on the device. The other dependencies are referenced as if the WebComponent is hosted in the "html" folder on the originating website, which makes it easy to develop the WebComponents from that folder until ready to upload to SPIFFS on the ESP8266. AssimilateCrouton will work out how to get the correct files.

DEPLOYMENT

edfungus creator of the original Crouton wrote the source in Pug/Less and had a NPM/Grunt toolchain. I rendered the Pug/Less as HTML/css and just edited/distributed the rendered files. This broke the NPM/Grunt toolchain. Fixing this is covered in the FUTURE section.

You can test the dashboard locally on your DEV box:

• From the commandline in the root folder
• npm start
• the lite-server is spun up for http://localhost:10001

Deploy to a static webserver:

• copy all folders except node_modules
• copy index.html (and possibly web.config)

FUTURE

One of the main goals is to upgrade to Polymer3 and work from the Polymer CLI. Adding advanced editors and framework for IOT developers to develop their own is a high priority. Eventually advanced automated system will be run totally from detached MQTT clients like AssimilateCrouton.

Step 3: DEVICE ASSEMBLY

As there are no hardware changes, here are the links to the relevant info:

• Shell Assembly
• Materials and Tools
• MCU Preparation
• MCU Housing Preparation
• Building the Slaves Low-side Switch/RESET Daughter-board
• Assembling the Major Components

Step 4: FIRMWARE

MAIN CHANGES THIS BUILD

In order for the AssimilateCrouton application to be able to use CORS resources from the device, response headers needed to be configured in a particular way. This was implemented in this release of the firmware (static_server.ino => server_file_read()).

Also the main dependency graph for Polymer needed to be from a single origin. A strategy was used to add an onerror handler (corsLinkOnError) to the SPIFFS CORS files to reload the resources from the AssimilateCrouton website when they are not found on the device.

There are 2 new conventions added to the SPIFFS filesystem for customizing the endpoints that are created in deviceInfo - which AssimilateCrouton uses to create the dashboard cards:

• /config/user_card_base.json Endpoint definition with runtime variables being swapped first: , , . This is typically where the assim-device card will be added. This does not communicate back with the device.
• /config/user_card_#.json Endpoint definition with runtime variables being swapped first: , , . This is typically where the rich editors like assim-weekview card will be added hooked up to the I2C slave (actor/sensor) that relates to #.

THE SKETCH/LIBRARIES

At this stage the project has been packaged as an example for the AssimilateBus Arduino library. This is mainly to make all the necessary files easy to access from the Arduino IDE. The main code artefacts are:

• mqtt_crouton_esp8266_cors_webcomponents.ino - the main entry point.
• assimilate_bus.h/assimilate_bus.cpp - the library that handles the I2C communication with the Slave Sensor/Actors
• VizJson.h/VizJson.cpp - the library that formats/builds any JSON published via MQTT
• config.h/config.cpp - the library that reads/boxes/writes config files on SPIFFS
• static_i2c_callbacks.ino - the I2C callbacks for a property being received and the cycle of slaves requests being complete static_mqtt.ino - the MQTT functions
• static_server.ino - the webserver functions
• static_utility.ino - helper functions

The static INO functions were used (instead of libraries) for a variety of reasons, but mainly so that the Webserver and MQTT functions could play well together.

THE SPIFFS RESOURCES

Detailed explanations of the SPIFFS files can be found here.

• favicon.ico - resource used by Ace Editor
• config
  • device.json - the configuration for the device (Wifi, MQTT...)
  • slave_metas_#.json - generated at runtime for each slave address number (#)
  • user_card_#.json - custom endpoint to be integrated into deviceInfo for each slave address number (#)
  • user_card_base.json - custom endpoint to be integrated into deviceInfo for the device
  • user_meta_#.json - custom metadata override that of the slaves for each slave address number (#)
  • user_props.json - custom property names to override the ones in the metadata of the slaves
• cors
  • card-webcomponent.css - stylesheet for various custom cards
  • card-webcomponent.html - webcomponent for various custom cards
• editor
  • assimilate-logo.png - resource used by Ace Editor
  • edit.htm.gz - gzip of Ace Editor HTML
  • edit.htm.src - original HTML of the Ace Editor
  • favicon-32x32.png - resource used by Ace Editor

UPLOADING THE FIRMWARE

• The code repository can be found here (snapshot).
• A ZIP of the library can be found here (snapshot).
• Instructions for "Importing a ZIP Library" here.
• Once the library is installed you can open the example "mqtt_crouton_esp8266_cors_webcomponents".
• Instructions for setting up Arduino for the Wemos D1 Mini here.
• Dependencies: ArduinoJson, TimeLib, PubSubClient, NeoTimer (see attachments if breaking changes in repositories).

UPLOAD TO SPIFFS

Once the code has been loaded into the Arduino IDE, open device.json in the data/config folder:

• Modify the value of wifi_ssid with your WiFi SSID.
• Modify the value of wifi_key with your WiFi Key.
• Modify the value of mqtt_device_name with your preferred Device Identification (no joining needed).
• Modify the value of mqtt_device_description with your preferred Device Description (in Crouton).
• Save device.json.
• Upload the data files to SPIFFS.

Step 5: DEVICE CARD

The device card (card-type:assim-device) is hosted on the website and it is not necessary to serve it from the device (CORS).

Its default page lists:

• The MQTT topics for reading and writing to the device
• The Access Point the device is connected to
• A link to the SPIFFS file editor hosted on the device using the ACE EDITOR
• An eye icon that reveals the Show/Hide card page.

The Show/Hide card page lists:

• Each card as a separate item
• Bold blue font when showing
• Black normal font when hidden
• An icon depicting the type of card.

The card can be hidden by clicking the hide button on the cards, or clicking on a blue-bold-font item in the list. The cards can be shown by clicking on a black-normal-font item in the list.

Loosely related to this feature is the info toasts. If any of the endpoints in deviceInfo have an info property assigned, an info button will be shown next to the hide button on the card. When clicked the contextual info defined in the endpoint will be "toasted" to the window.

If the device card is not defined, the hide buttons will not be shown on the cards. This is because once hidden there is no way to show them, again.

See ENDPOINT CUSTOMIZATION to detail how the assim-device card can be added via the SPIFFS files on the ESP8266.

Step 6: WEEKVIEW CARD

The weekview card (card-type:assim-weekview) is hosted on the device (cors folder). It is injected into the deviceInfo packet published for AssimilateCrouton, by adding a file config/user_card_#.json to SPIFFS (in this case user_card_9.json).

OVERVIEW

The weekdays are presented as lists of time-slots. The granularity of the time-slot is set with the property "interval_mins" in config/user_card_#.json. It needs to be a fraction of an hour or multiples of an hour e.g. 10, 15, 20, 30, 60, 120, 360. Clicking on a time-slot ensure an on state is commanded for the associated device in that time. If the time-slot is now, a command is sent (published) immediately for the device. Normally the state is checked/published every minute. Selections are saved in LocalStorage, so the times will be reloaded with a browser refresh.

USE CASES

In its current state, the weekview is suitable for devices that can use a Toggle switch to visualize their state i.e. they are either on or off and after being set they remain in that state. Lights, fans, and water-heaters are good candidates.

LIMITATIONS/CAVEATS

• The interval_mins must be one of the values mentioned above
• The weekview does not support momentary actions that are also scheduled, such as turning a tap on briefly (5 seconds) twice a day.

FUTURE

• It is expected that momentary actions will be supported.
• Synchronized storage across devices, for the schedule selections is being considered.

Step 7: ENDPOINT CUSTOMIZATION

As mentioned breifly in FIRMWARE, there are are 2 new conventions added to the SPIFFS filesystem for customizing the endpoints. The JSON files are fragments that get added to the endpoints property in the deviceInfo packet posted to the MQTT broker that becomes the dashboard definition.

The keys of the endpoints are generated in firmware:

• CC_device (Custom Card) for the user_card_base.json
• CC_SLAVE_ENDPOINT NAME for the user_card_#.json (# being the slave address)

As mentioned before, there are variables that get substituted for values at runtime:

• mqtt_device_name
• wifi_ssid
• local_ip

user_card_base.json

An example:

{
  "device_name": "<mqtt_device_name>",
  "card-type": "assim-device",
  "ssid": "<wifi_ssid>",
  "ip_addr": "<local_ip>",
  "endpoints": [
    {
      "title": "Grow Lights",
      "card-type": "crouton-simple-toggle",
      "endpoint": "switch"
    },
    {
      "title": "Planter Lights",
      "card-type": "crouton-assim-weekview",
      "endpoint": "CC_switch"
    }
  ] }

user_card_#.json

An example:

{
  "card-type": "assim-weekview",
  "path": "http://<local_ip>/cors",
  "title": "Planter Lights",
  "info": "Set the lights on or off in 15 minute time slots",
  "interval_mins": 15,
  "values": { "value": "" }
}

Step 8: VIDEOS