Introduction: Home Automation Infrastructure
I started to work on my DIY home automation a few years ago.
Quickly it became obvious that equipments had to be connected to a core system
So step by step I started to design my Home Automation Infrastructure that I want to share
What are right now the main achievements ?
- All equipments are connected either with RF or WIFI
- All equipments are monitored even some managed over Internet
- Equipments actions can be triggered with a scheduler
- One event on one equipment can schedule one action on an other equipement.
- One alert system informs on troubles
- Sensors data are stored for treatments and analysis
- Thanks to standardization it is effortless to add new equipments
- I have total control on my home automation system
In this instructable I will enter in details in this infrastruture.
Some microcontroller softwares can be used as is.
Regarding the HUB, I will be pleased to share the server software with people interested to work on it.
Supplies
One server that operates Linux
Some Arduino or ESP8266 microcontrollers
Some sensors
Step 1: Let's Illustrate With One Example Before Entering in Technic
With this example we can see that it is possible to have a global view of the comfort equipments and to manage them consequently locally or remotely
Step 2: Infrastructure Overview
The core of the infrastructure is the Hub. The Hub is made of a software that runs on a central server.
Sensors and actuators are connected thru microcontrollers.
Microcontrollers communicate with the Hub with a common protocols and some gateways.
Doing so, heterogen equipments can interact with each other.
They are monitored and managed with laptops and smartphones on the home LAN even over Internet.
It is possible to add a new station without any coding.
Step 3: Infrastructure Overview
What are the components of this infrastructure ?
- One architecture
- Some communication standards
- Some software components
- Some hardware components
Step 4: Communication Overview
From top to buttom communication relays on
- UDP/IP over Internet
- UDP/IP using Ethernet and WIFI over the home LAN
- Communication between microcontrollers and upper levels can be:
- direct for those that supports WIFI
- indirect thru a serial/WIFI gateway
- indirect thru a RF433/Ethernet gateway
- Communication between microcontrollers and sensors are specific to the sensors
Step 5: The Software Components: 3 Gateways
Gateways softwares
- Running on ESP8266 with a WIFI connection converts serial link frames to UDP frames and vis et versa
- Running to Arduino with a Ethernet connection converts RF433 frames to UDP frames and vis et versa
- Running to Arduino with a Ethernet connection converts UDP instructions to RF433/Home Easy and Somfy signals
RF stations share the gateways, so there are a maximum of 2 RF gateways for all the RF stations of one home.
Here you can find the ESP8266 WIFI Gateway instructable
Here you can find one use case of the ESP8266 WIFI Gateway
Here you can find an instructable for the RF433 UDP/IP gateway
Step 6: The Software Components: 3 Microcontroller Frameworks
3 frameworks are available that make it easy to develop new connected home stations. It allows to concentrate on data and services without taking care of communication.
2 based on Arduino platforms
- One for radio frequency based communication. It supports unidirectional and bidirectional exchange. Messages go are converted thru RF Gateway between RF and UDP/IP frames.
- One for WIFI based communication. It supports unidirectional and bidirectional exchange. Messages go are converted thru ESP8266 WIFI gateways between RF and UDP/IP frames.
- Use WIFI communication. It supports unidirectional and bidirectional exchange. Messages are directly send to and received from the HUB
Data and services that support by these frameworks are described further in this instructable
Trick: station identifiers are stored in Eeprom. So it is possible to upload the exactly same firmware inside multiple microcontrollers. The HUB will automatically differentiate them.
Step 7: Terminology
Station: one microcontroller with sensors and actuators
Indicator: a station parameter that values is stored in the HUB database. Typically the microcontroller send regularly the indicators values to the HUB. Eventualy parameters values can be set from the HUB to the station
Measurement: value of a sensor to store in the HUB database or in a Google sheet
Station, indicator and measurement are defined in the HUB database and identified with a unique identifier
Alert: an event sent from the station to the HUB that requires specific attention
Register: a microcontroller 8 bits register. It can be send from the station to the HUB and vis et versa. Registers are defined inside the microcontroller identifed with a unique identifier
Command: optionnaly it is easy to add command. A command byte sequence that is specific to the station
The HUB can forward registers and command to dedicated code via an UDP forward
Service: a functionality provided by the HUB to the stations
Action: something we want the station to do. Action are defined in the HUB Database
Step 8: Data Flow
The HUB keeps regularly stations and gateways informed of services availability and theirs UDP/IP addresses. By doing so it is possible move the HUB to an other server without any change on the stations and gateways side.
There are two kinds of data flow
- Binary mode for Arduino stations, Arduino gateways and ESP8266 gateways
- ASCII mode for ESP8266 WIFI station
Step 9: Data Flow: Alerts and Events
Depending on certain conditions a station can decide to send an 'Alert' message to the HUB. The message contains some characteristics as Category, Severity...
The HUB will acts depending on the charactristics. If needed it will schedul one or more tasks.
For instance it can immediatly update a register in an other station, and later send an email.
Tasks can be chained
Step 10: Let's Zoom on the HUB Side
The HUB software runs on Linux platform.
The code is 100% JAVA.
There are 2 kinds of code.
- The "batches" one, standard, mostly running 24/7.
It communicates in real time with all the stations. It continuously stores data in Mysql tables.
It regurarly check for update "indicators" table and if needed informs stations.
Data purge is done once a day according to specifications set in a Mysql table.
- The HMI one running over Tomcat application server
Class and Servlet are specific and developed according to the needs
It shares the Mysql tables with the "batches"
One API allows to send commands to the HUB
It is possible to act with a station by:
updating an indicator target in the Database, that target will be automatically forward to the
updating a microcontroller register
creating an alert on the HUB, that will trigger event
Step 11: Measurments in a Google Sheet
Instead of storing measurement values in a Mysql table, it is possible store them in a Google Sheet.
A new line is added for each new set of values.
Doing so it is possible to get a dynamic google graph.
Step 12: Let's Zoom in the Indicators Dynamics
The above picture illustrates the indicators dynamics
In this example the station with identifier 100 send 3 indicators (1,2 and 3) values to the HUB.
Firstly the message is decoded based on the indicators characteristics table.
indicator 1 is an INT of one byte,
indicator 2 is an INT of two bytes and
indicator 3 is one Byte with a target value of 0x02.
Decoded values are added in the indicators values table with a timestamp.
As indicator 3 value sent by the station is not equal to the target, the BUB will automatically send a request update to the station.
Step 13: Let's Zoom in the Measurements Dynamics
The above picture illustrates the measurements dynamics
In this example the station with identifier 102 send 2 measurments of type 1 values (1 and2) to the HUB.
Firstly the message is decoded based on the measurements characteristics table.
the first measurment of type 0 for the station 102 is 'temperature'
the second measurment of type 0 for the station 100 is 'humidity'
2 records are added to the measurements values table
Measurements are INT16 (signed).
The station can send up to 10 measurements of the same type for a station
Step 14: Let's Zoom in the Registers Dynamics
From station to HUB
Registers provide the easiest way to develop specific code for a station.
When a station send one or more registers to the HUB , there are automatically forwarded to a specific defined UDP/IP address.
Doing so it is easy to add specificities wherever you eed over Internet, just by listening a UDP port.
From HUB to station
The registers mapping table is used to convert a requested action for a station into an registers update command to send to the station.
Actions can come from a scheduled task, a crontab entry or a HMI request.
Update registers is the easiest way to act on a station in real time
Step 15: Alerts & Events Dynamics
A station can send an "Alert" message. Alert message are used to notify that something happened that need human attention and/or that need the HUB to trigger actions. It does not mean something went wrong. For instance the end of a trouble is notified by an "Alert" message with a closed MesgType.
Status, MsgType, Urgency, severity and ResponseType are codified. Category can be defined according to the station need.
All "Alert" messages are stored in the HUB database and can be viewed on a web HMI.
When the HUB receive an "Alert" message it will look in the database for event to trigger depending on the stationId, MsgType and Category. If so the event will be activated. An event provides actions and even schedules an other event.
Step 16: Add a New Station Without Coding
For instance imagine you add already developed a weather station with the id 0xfe01 and want add another one.
To do so you will have:
- Set the address inside the Arduino code to 0xfe02
- Set the MOSI Pin of the new Arduino to ground
- Upload the Arduino code inside the new station the station configuration will be written in eeprom
- Remove the MOSI Pin from ground Restart the Arduino
That's it for the Arduino side
- create the 65026 entry in the station table same as the 65025
- create the 65026 entry in the indicators table same as the 65025
- create the 65026 entry in the measurements table same as the 65025
That's it for the HUB server side