In this tutorial I will describe how to create a secure and reliable wireless plug based on the tiny ESP8266 D1 Wemos mini for controlling appliances in your home.
Cheap 433MHz Remote Controlled Switches have their advantage of being cheap and easy to use, you buy them push 1-2-3-4 ON/OFF on the remote and they just work out-of-the-box but from security point of view they are terrible, only that does not control them who don't want to. Someone can walk up to your home with 4-5 different set of remotes and with a good chance he is now in control of at least some of your wireless plugs. This by itself would not even be an issue if they would not be unreliable. 95% of the time they work but what about the 5% when they don't. That is unacceptable for engineers. Most of these remotes implement repeated resending to increase the success rate (when you push the button on a remote once it sends the ON packet multiple times and hopes for the best). You might want to control some more important components than lights like solenoids or motors with these and since they don't contain a transmitter, don't signal back. There is no way for you to know if the device you sent the command to, turned ON or OFF.
I started to design this plug with barebone Atmega326P due to it's low power consumption and it was just enough for the purpose but when trying to implement wireless encryption with such a tiny MCU I ran into difficulties so it remained a PoC. I will still include my research in a chapter, it might be useful for someone in the future. As a side note here at the start that playing with the mains voltage (110V/230V) is dangerous, if you have just started with electronics you should only build low voltage circuits, this build is only for experienced engineers (build it for your own risk).
Teachers! Did you use this instructable in your classroom?
Add a Teacher Note to share how you incorporated it into your lesson.
Step 1: Harware Design
After not having enough different Home Easy, Intertechno switches in my home I have bought a bunch of secondhand Home Easies again (due to they had a working Arduino Library already) but then I sadly noticed that this series using different encoding or preamble, the Intertechnos (with mechanical rotary selectors) regularly misaligned when they were re-plugged and all of them can be considered unreliable due to the way they work so it was finally time to design my own wireless plug and stop using these commercial :smart: plugs once and for all.
These wireless plugs are very simple devices, they powering their main circuits with a cheap inefficient transformerless power supply (TPS) circuit which steps the mains (230/110V) down to lower voltages (then regulates it AC->DC) what the circuit needs. Then there is a 433Mhz RX module (which looks familiar (same?) to what we can buy for the Arduino's) which goes into the main MCU which processes the data (some more advanced devices can be put into learning mode) and if the incoming signal matches with the ON/OFF signal then they turn the relay ON/OFF.
There are plenty of good tutorials on Instructables about how do they work and how to use them with the Arduino such as: https://www.instructables.com/id/Using-433MHz-Remote-Controlled-Switches-on-Arduino/
I used different models of HE (Home Easy), Intertechno, noname switches in the past what I can sort into 5 categories:
1, Single signal ON/OFF switch with RCswitch library. These devices are the worst, they don't have a specific ON or OFF signal they will go into ON position the first time they are connected to the main then they basically flip flopping, you cannot determine the state. They are not good for anything, avoid them.
2, Wireless ONLY smart plugs with dials/dip switches, they can be only turned ON/OFF with the remote there are 2 dials to adjust the GROUPS/SWITCHES like Group III Switch C. The problem with these dials that even from small fractions (pulling the plug out and replugging it) they can shift out of position, also since there is no manual ON/OFF button on the device if your remote control fails or you just want to switch the thing on right now you are out of luck. Avoid these as well.
3, Wireless plugs with manual ON/OFF (same as the category before but with manual override).
4, Wireless plugs with learning mode switch. In this mode once the programming switch is pushed in for 5-10 seconds the plug enters into learning mode which is indicated by the LED blinking. The devices look for a certain signal set matching the manufacturers predefined codes (eg code of Group II Switch A) once they detect this the blinking stops and the device learned the code (paired).
5, 3+4 combined, these are the best kind of outlets you can get. I never run into any issues with them that they forgot the code and you can easily reprogram them if you have to.
But none of these cheapies will ever be anywhere close as good as this project. There are 2 more major issues with the commercial outlets: one is interference (try to plug 2 smart plugs right next to each other and see what happens or rather what is NOT happening). Since they are using the same bands, encodings you can be unintentionally turning on/off your neighbors devices in a flat. The 433Mhz signal is fairly good with passing through walls and the signal strength is not bad either since the remotes work from mini 12V batteries. This just leads to the second point that they have zero security. If anyone knows about you are using wireless plugs to turn on/off some of your devices all he has to do is go close to your house with 4-5 different remotes made for the major brands and most likely he will be lucky.
With the ESP32 already succeeded the ESP8266 and offers dual cores and other goodies the question is why did I chose the ESP8266 and in particular the Wemos D1 Mini for this build? The answer is: the size. At the time of my original design idea (2018) this seemed to be the smallest board compared to other ESP32 boards and it even had a relay shield. With this shield it stayed within the 25mm height limit of the HE wireless plug, however as you will see eventually the case had to be extended. Regardless that the ESP8266 is under the ESP32 in performance you cannot even compare it in processing power to any Arduino boards, it would be something like 10x better (16Mhz Vs 160Mhz) with a lot more memory as well.
Old and new smartplug design:
In case of my old setup there was a Raspberry PI (which later got swapped out to a PC104 running NetBSD for industrial gradyness :P) -> Arduino -> 433 Mhz transmitter controlling ALL the wireless plugs it also introduced a single point of failure. What if the PI or Arduino (not likely) crashes, then all the home control is down.
This new design is completely decentralized (no blockchains yet :P), the only point of centralization is the wireless access point and if you worry about that you can just easily link multiple APs in a WDS net and the plugs will connect automatically to whichever is on and reachable. The wifi connection is extremely quick, the device connects and reachable in 1-2 seconds after powering on.
Extending the functionality:
As it is a way faster MCU than what the original "smart plug's" had what else can be done with it:
-State detection: as I mentioned this was the main reason why I designed this circuit to always know if the device is ON or OFF, this value is stored constantly in a global variable and also written to the FLASH. This yields complete reliability over the old approach which was send the command X times to the "smartplug" via 433Mhz radio and hope for the best. Here in case of m2m communication the remote machine exactly knows if the device is reachable on https and can resend the command X times if it's not and TCP/IP is guaranteeing reliability and automatic packet resend in case of lost/duplicated packets on wifi.
-Current metering: ACS712 was added for current measurement. Since the AC voltage is constant 230/110V we can determine the power (Watts) or the power consumption during time period (Watt Hours). I have left the PowerAVG calculation in from EOL, it could come handy in situations where you need to do the same thing (create alerts to see if a device is finished the work). Matter of fact this whole code is based on EOL, this project could easily replace it if wifi would go so far as my basement :P
-Keep state over restarts: Fortunately having such a great mini computer built in, this feature is now easily accessible. Device state can be saved in the FLASH: although the write cycle for ESP8266 is limited somewhere around 100K for the FLASH if you consider that the plug will be turned ON and OFF one single time every day and the state will be saved to the FLASH right away that is only 730 writes/year so it should not be an issue for the next 100 years.
-Programmable timer: I added this feature just to demonstrate that even if you add the functionality of a regular wireless plug + a timer plug together they still cannot compete with my project. Of course the ESP8266 can easily do programmed turn ONs and OFFs as well and it will be even more accurate because it sync with NTP every day while a cheap timer plug drifts off with seconds, minutes over time.
If you are still not satisfied with the functionality of the new plug a smaller Multi User Dungeon can be still implemented over TELNET :D
While having all these features is nice, I always implement manual control (at least a single push button to manually be able to turn the device ON or OFF).
Bill of materials
- 1X HLK-PM01 5V/0.6A isolated power supply
- 1X Wemos d1 mini
- 1X Wemos d1 mini relay shield
- 3X 300 ohm resistors
- 1X 10Kohm resistor
- 1X 2K2ohm resistor
- 1X RGB 4pin LED (Common Anode)
- 1X Push button
- 1X ACS712 20A
- 1X Fuse holder
- 1X 10A fuse
Wemos D1 mini hardware
Operating Voltage 3.3V
Digital I/O Pins 11
Analog Input Pins 1(Max input: 3.2V)
Clock Speed 80MHz/160MHz
Flash 4M bytes
Note: This board has one terrible inconsistent pinout, to make it a bit more consistent I have printed and using the Blue Arduino pinout everywhere.
The circuit board exported out in many different formats for your convenience. You can use a lot of online services to manufacture the board for you but I recommend EasyEDA (which was originally used to design the board) because they provide a great service with low prices, they deserve the support.
The first version of the board (shown on some picture), even after careful design was a failure (*surprise, surprise*), although the relay and current meter worked as expected the rest did not, which contributed to non working data pins on the bottom side of the board. Originally the G and B led legs were wired to the RX (3) and TX (1) pins:
//********** CHANGE PIN FUNCTION TO GPIO **********
//GPIO 1 (TX) swap the pin to a
GPIO. pinMode(1, FUNCTION_3);
//GPIO 3 (RX) swap the pin to a
GPIO. pinMode(3, FUNCTION_3);
Sadly did not work and I had to go through lengths on debugging why is a so simple thing like turning on the led does not work after the board was manufactured. Same goes for the SWITCH pin which was originally pin 2. All these were brought around and up to the north side of the board where all seems to work fine. While the relay board connector pin 5, and the red pin 0 was working from the start (strange). Another 2 issues what I was not expecting is that there was an smd flyback diode on the bottom of the relay board, blocking the board to be able to completely pushed down to the motherboard, therefore in the next version I made a cut-out, also the whole board was extended in height because in the original the Wemos D1 mini would just overlap with the cutoff end of the HE smartplug making the extension box insertion more difficult.
Step 2: Case Design
I have designed this board to be fitted into the HE874/HE878 wireless plug/dimmer cases (although no matter how I ordered the components it would not fit because I wanted to keep the middle push button, the LED and the screw hole(s) at the same place). 3D printing cannot compete with hard melted plastic and although this plug is still kinda expensive at the time of writing (15$) you might be able to pickup a bunch second hand/for parts on smaller auction sites for a low price. The HomeEasy series of wireless plugs and dimmers have similar form factors as well so the board can be easily modified to fit into another models case. The board ended up to be longer than what could be fitted into the plugs original case so I have created a 3D printed extension box to complete the case.
If you wonder how did I manage to fit my components in the case perfectly. It' is not due to precisely measuring everything (although I did do some prior measurements with a ruler) but just pulling a simple trick of printing out the board to regular paper, cutting it out, placing the components on top, adjusting and rinse and repeat until it's perfect.
Original board dimensions:
Board length: 47.000 mm
Board width: 47.000 mm
Board height: 25.000 mm
New board dimensions:
Board length: 97.000 mm
Board width: 47.000 mm
Board height: 25.000 mm
3D printed expansion component:
Cuboid length: 50.000 mm
Cuboid width: 48.000 mm
Cuboid height: 25.000 mm
One disadvantage of the HE plugs that they are using the very much disliked T - security screws and of course I didn't have them in any of my sets so what I did was just get a cheap not chromated throwaway screwdriver and cut a V in the middle of it with my diamond dremel head and voila security is now defeated, if you have bunch of regular T or X screws in the same size just throw the originals away don't put them back.
The top extension box is basically a 3 sided cuboid which is glued into the top of the HE plug around the bottom and side plates with strong 2 component glue.
Unfortunately at the time of writing I did not have access to a 3D printer so this small esthetic work will remain for later.
Step 3: Circuit Design
As with other projects I advise against using Transformerless Power Supplies with all of your Arduino/ESP/Raspi projects. Here is a brief explanation of why:
Originally I actually planned to hack the old plugs, leaving the TPS, the button, the relay and the LED in place and de-soldering everything else from the board but I'm glad that I changed my mind about it.
In the circuit I only had one small issue, that the ESP8266 and ESP32 are 3.3V boards, all the pins are also 3.3V rated and NOT 5 volt tolerant however the ACS712 current sensor is: "5.0 V, single supply operation" which means that you don't operate the sensor from 3.3V or not even 4.8V, you operate the thing from precisely 5V otherwise the voltage reference changes and the whole measurement will be incorrect. The solution therefore was to operate the whole circuit from 5V, take the 5V directly to the ACS and the ESP8266 (it has rectifier on board no worries), then put in a small voltage divider to the ACS712 DATA OUTPUT pin which will result a value not exceeding 3V and the AnalogRead mapping 0-1023 will be the similar as what you would get using a regular Arduino with the sensor (in the software section you will see the adjustments I have made including a small correction of reducing the AmpsRMS to get around the same value as my wallplug meter gives me (take it with a grain of salt tho, this is not some thousand dollar current measurement device just another cheapie from China so you can adjust the code for your needs, the hardware does not require modifications). Also stacking up on HLKPM-05 is a good investment since you can just use them in all your Arduino projects.
One common mistake a lot of ESP8266 and ESP32 tutorials about push buttons, leds and interfacing with Arduino modules commit is putting 5V directly to the GPIOs. Just don't! You can consider the 5V pin as the only pin in this circuit where you should be using 5V, it goes into the ESP8266 regulated to 3.3V and from that point use the 3.3V for everything.
The rest of the circuit is straightforward and does not require any explanation. I have doubled the copper both sides and at manufacturing set the thickness of the copper to 2oZ (per side) to be able to handle the max 10A and a 10A MAX rated fuse was also added (not to be cheapskates), I also like to have a spare soft glued safely somewhere in the box for immediate replacement. If an appliance faulty and shorting the mains out, that might break the fuse but will no way ruin the plug.
Just a note, as I managed to blow up some of these standard Arduino relays in the past: regardless that they are rated for 10A these can even withstand max current of 15-16A for shorter time periods, and the ACS 20A so the first thing which will go is the fuse.
The relay shield was meant to be put on top of the Wemos D1 mini, this is not just dangerous since there is a small airgap between the 230V mains and the metal shielding plate of the wifi but in-practical in my build so I have reversed the order and put the relay directly down to my PCB which is soldered into the PCB with DIP female sockets on top of it where the Wemos will slide into. At the relay shield, two out of the three original screw terminals are connected to the board from the 3rd (always connected) the flux was removed and cut out as it would just interfere with the trace going underneath it and we don't need it. On box of the extension board the screw terminal connectors with the high voltage part are hidden, only the USB has a breakout hole.
As you can see this design went from V1 to V3, V3 corrects a lot of (not that obvious) mistakes I have made with V1, on some picture you can see the original board where the connectors were reversed (using MALE bottom sockets) however that did not reach the FEMALE part of the top Wemos D1 board and I had to put in a F-M extender in between which resulted terrible flacky design where I had to force the Wemos with my hand to have connection so I reversed this by using 2.54mm Pitch 10 Pin Single Row Stackable Shield Female Header for Arduino, which are extra long headers and works for me. These are soldered into the main board then soldered again at the relay shield (very easy to do from the side the solder will run down the holes from the legs). The second major issue was a small flyback diode on the bottom of the relay board which was blocking it from laying flat on the motherboard so the V3 features a cutout on that area.
For assembly here is my recommendation:
1, Solder in all the resistors, the switch, the LED
2, Solder in the ACS712 and the Relay Shield
3, You must solder in the cables from the plug at this point as they go from the other side and 1 will be inaccessible from the HLKPM-05 at the middle
4, Solder in the HLKPM-05
5, Solder in the fuse
For soldering you can use different tin lead lines I just used 0.5 mm and it worked great for all parts of the plug. As it might be shown on some pictures I have left the terminal strips on the ACS712 and tried to flow the solder from the other side across. Do NOT do this, the connecting surface area will be a lot less and the device might won't be able to carry 10 AMPs (For the low power part V0/GND/VCC it does not matter soldering from the back will work).
Removing the terminal strips from the ACS712 and the relay can be a major pain if you try it the wrong way. I have found 2 ways:
1, Trying to push a thin screw driver or cutting knife in between the board and the strip and force it out slowly
2, Use your hand to keep moving the terminal strip left and right slowly while applying heat to the pins
Regardless which method you chose, instead of trying to suck the solder off with a de-soldering tool, apply more solder to it. This might sounds strange first but trust me it will work great for heat transfer, as this was put in with some industrial solder and with regular cheapie soldering irons it's a pain to even heat it up, well this is the way to heat it up.
The LED colors show the state of the plug which can be the following:
1, Yellow blinking (not associated with the AP, lost connection to the AP, can't get time data from the NTP server)
2, Red ON (plug turned on manually through the push button)
3, Blue ON (plug turned on remotely through https - computer control)
4, Green ON (plug is in time switched mode - time control)
The states can overwrite each other. For example the plug was turned on manually but there is a time ON event, the color will be Red first then it goes to Green to reflect the last state.
Step 4: Software Design
As this is my first ES8266 project I run into a multitude of issues, but overcoming them was a great experience. If you just want to build the plug you can download the Arduino sketch do minor configurations to fit your needs (wifi network settings, ssl certificate generation, user/pass for auth, nodename etc) then it is enough to read this section and take it as granted. If you are interested in the why read the next Software Issues section.
To provide an out-of-the-box software package I have packed my customized NTP library in a .ZIP, all you will need is the ESP8266 Arduino library what you can download with the board manager.
For IDE use: Arduino 1.8.9 or higher version
For build -as I mentioned it in the code- use:
-Use upload/serial: 115200
-Use CPU/ram: 160Mhz/4M flash
-Use SIPFFS: 1M /4M
-Use erase sketch + wifi
The base code is same as EOL, it's looping constantly and calculating the current/power consumption but with some extras. First of all the logic to determine the type of load was moved out of the code, it will just give back the date, uptime, relay state, current consumption, power consumption and average power consumption beyond turning the relay on or off.
For security I have finally decided to switch from the good old TELNET to HTTPS since the Wemos is more than equipped to handle SSL connections. Therefore the usual xcomm() function is only used to take commands from serial (when debugging). A basic http authentication is also implemented.
will get you a nice user friendly page to turn the device ON or OFF and displays the stats however I mostly use these plugs for Machine 2 Machine communication (eg: one raspi checking the outside sun brightness by measuring the time to charge a small cap, once the threshold is reached (night) it contacts the plug directly to turn ON the lights. For this I have made a simple api:
Where the command will be interpreted and the result is not some web 3.0 fancy but the usual one liner ending with \r\n.
%Node Name_ON/OFF%,%DATETIME%,%UPTIME%,%Current%,%Power%,%PowerAVG% R1_ON,2019-05-14T08:26:32,0000-01-01T01:00:00,1,230,230 R1_OFF,2019-05-14T08:26:32,0000-01-01T01:00:00,0,0,0
This way the requesting machine can do everything from a basic shell script with basic tools like curl and awk.
OK let's move on to securing the plug. This will bring us to creating SSL certificate and setting an username and password for HTTP authentication.
I prefer using self signed certificate (and included the default example here just to make the code compile out of the box but you should change the certificate with your own).
BearSSL has it's limitation of 1024 bit max key length unfortunately so I could not use my own LAN CA which makes 2048 bit keys by default so I had to redo the procedure.
HOW TO GENERATE YOUR OWN CERTIFICATE/KEY PAIR
A sample script, "make-self-signed-cert.sh" is provided in the ESP8266WiFi/examples/WiFiHTTPSServer directory. This script can be modified (replace "your-name-here" with your Organization name). Note that this will be a *self-signed certificate* and will *NOT* be accepted by default by most modern browsers. They'll display something like, "This certificate is from an untrusted source," or "Your connection is not secure," or "Your connection is not private," and the user will have to manully allow the browser to continue by using the "Advanced/Add Exception" (FireFox) or "Advanced/Proceed" (Chrome) link.
I strongly recommend using 1024 bit certificate instead of 512 bit, there is virtually no page load speed gain using a weaker certificate (I talk about this in the next chapter) and you might run into issues with browsers such as:
The server certificate included a public key that was too weak. Error code: SSL_ERROR_WEAK_SERVER_CERT_KEY
There are some guides on the net detailing how to use LetsEncrypt (this only makes sense if you put the plug on the Internet on a public domain) but that might not be such a great idea, you can read more about it in the next section.
Led colors and plug states
The LED colors were described in the hardware section, here I would just make a small note that the plug not just stores down the cause of the latest ON state (manual,time,computer control) but the cause of the latest OFF state as well.
I realize it is a bit of over perfecting it but why not. The original V1 design when you plugged the plug in it read back the relay state which was a simple 0 or 1 and turned the relay on however the internal state was always COMP_ON afterwards.
For this I did a simple trick that instead of storing down 0 or 1, it stores decimals: 1, 2, 3 - ON states / 4, 5, 6 - OFF states.
I have changed all the references in the code.
At init (before even reading the DB) the relay state is 4 which is manual off.
After reading the DB if the state is lower equal to 3 then turn the plug on otherwise off.
The relay_off command does not know or care about states 4, 5, 6 since that is the result, not the cause. It expects either 1, 2, 3 which by the way will all have the same effect turning the relay off it is just for the STATUS variable update so when it is called from the DB readback function it will substract -3 from the value resulting the original 1, 2, 3.
NTP Code & the uptime & the timer
What is NTP and how does it work is explained here: https://lastminuteengineers.com/esp8266-ntp-serve...
In a nutshell from the computers point of view it is just an unsigned long which gets compared with the UNIX epoch (January 1, 1970). The computer keep increasing it in every loop and once in a while it connects to an NTP server to sync the value from there. As we don't want to hammer the NTP server in every 5 minutes with requests the code gets the time from the NTP once a day.
I have rewritten the NTPClient of https://github.com/arduino-libraries/NTPClient
and was planning on removing evil strings because of https://hackingmajenkoblog.wordpress.com/2016/02/...
but I give up on that since the webserver code deeply uses strings in all of it's sublibs, it would be pointless and I don't know if they have the same effects on the ESP8266s memory since it's way more powerful than an Arduino. The original idea was to have a completely String free code but this never happened.
Additional super important function (uptime) was added into the NTP library which is basically starts after the first successfully NTP sync when the current time is determined, the system will set the start date variable as epoch and will be comparing to that.
The UNIX epoch goes from 1970/01/01 for a reason, if it would be going from 0000/01/01 then an UL (Unsigned long) would not be enough to store it. Matter of fact there is a problem with the systems using 32bit INTs to store the NTP long value, it will overflow soon (in 2038 soon enough :P).
- Unsigned Long Max: 4294967295
- Signed INT MAX: 2147483647
- Seconds ~ 1970->2019 1545264000
- Seconds ~ 0000->2019 63671184000
- Seconds in a year: 31536000
So I did the same clever trick as I did in my Coffee Maker project the uptime is calculated by the getFormattedDate() function will subsract - 1970 from the years.
Time switch functionality
10 timer entries can be defined through commands.
The plug can be programmed from serial (DEBUG MODE) with:
TIMER_CLEAR - clears all timing
TIMER_QUERY - queries all entries
The time values will be stored down in a db file on the SPIFFS. This DB file will be read and verified at every boot (the file must contain exactly 10 lines, otherwise the code assumes it to be corrupted and will recreate it with empty entries). If the verification succeeds then as the next step the code will load the values into global variables not to hammer the filesystem anymore.
The format of the DB file is simple:
Where the first column is the event type ( 0 - timer entry disabled, 1 - ON, 2 - OFF). The line number directly corresponds to the variable number (line1 -> t1, line2 -> t2 ...).
To keep things simple since the plug will rarely be programmed, in case of ANY time set commands, the time will be set in the corresponding global variable (t1-t10) first then the whole time DB will be written to the flash, this way all kind of dupes, corruptions, order, record processing etc. can be avoided.
At the start of each void loop() the code will quickly go through only the time entries which are enabled and uses the ntp gethour and get minute functions to get the values and compare it with the global integers to turn things on or off.
The standard way is to set the timers is through the web frontend. For this I have developed a second request processor called handleTimer(), the way of handling GET/POST requests from the forms is quite different from the usual PHP but I made sure to implement proper input validation. This server only accepts exactly 4 arguments and all 4 will be converted into unsigned integers and validated. In case of any errors the code will not proceed with setting the time.
After a time set request was successful, the global variable for that specific time entry will be immediately updated (timer goes active) and then the timer db will be recreated and written out to the flash.
I would note that without modifying the code it is not possible to use this plug even just as a timer without network connection (it would defeat any purpose at the first place) because it MUST get it's time value from the NTP server.
Same goes for the Wifi, if it can't connect to an AP at start, it will still restore the last relay state first (ON/OFF) but then the execution will hang forever until it can connect.
At an earlier version the timer was a BUILD feature, however I spent quite a lot of time on the timer code :P so I have decided to rather make it dynamic. You can turn the whole timer functionality ON/OFF from the webpage any time. The reason why I did this because the timer table generation increases the page load speed and you might don't want that functionality anyway. Turning OFF the timer will leave the DB entries intact. You can also chose to CLEAR the whole time DB with 1 button in which case all entries will be zeroed and the timer will become disabled. By default the timer is disabled.
Step 5: Software Issues
I have provided an all in one package (containing the source code and all the required libs) for Arduino 1.8.x which will make it easy for everyone to program their Wemos however if you willing to put your libraries together use this guide: https://randomnerdtutorials.com/how-to-install-es...
Do NOT install the Adafruit ESP8266 library as it is incompatible with the ESP8266 Community Edition library. If it is installed the right lib what we need will simply never show up in the boards manager no matter what you do (until you remove the Adafruit). I would note that the Adafruit works as well but it lacks SSL, SPIFFS support and lot of other sub libs what we will need in this project.
At the time of writing the current stable version of this library is 2.5.1, do NOT use this version. You must build the latest one from their GIT:
Using git version (basic instructions)
For Linux, it is ~/Arduino by default. cd ~/Arduino cd hardware mkdir esp8266com cd esp8266com git clone <a href="https://github.com/esp8266/Arduino.git"> <a href="https://github.com/esp8266/Arduino.git"> <a href="https://github.com/esp8266/Arduino.git"> <a href="https://github.com/esp8266/Arduino.git" rel="nofollow"> https://github.com/esp8266/Arduino.git </a> </a> </a> </a> esp8266 cd esp8266 git submodule update --init cd tools python get.py
Why NOT to use the 2.5.1 library:
Memory leak in WiFiClientSecure on repeated WiFiServerSecure connections #4302 and **possible** other bugs. I was considering actually attaching the latest GIT version (at the time of writing) to the code but they still have a lot to fix.
If you would use the 2.5.1 crash/resetting the ESP8266 would be so easy as sending a small curl request which does not accept the certificate:
curl <a href="https://<ip"> <a href="https://<ip"> https://IP/ >IP/ </ip></a>>IP/ </ip></a>>IP/ would result: Soft WDT reset >>>stack>>> ctx: cont sp: 3ffffc50 end: 3fffffc0 offset: 01b0 3ffffe00: 00000000 00000000 3ffffe80 00000000 3ffffe10: 00000010 00000000 3ffffe80 40203c4f 3ffffe20: 0000001f 00000000 3ffffe80 40203c4f 3ffffe30: 3ffe89e3 00000000 3fffff58 40206866 3ffffe40: 3ffe89e1 00000000 3fffff58 3ffeef06 3ffffe50: 3ffe89e3 00000000 3ffffe80 40203e65 3ffffe60: 3ffe89e3 3fffff58 3ffffe80 4020309b 3ffffe70: 3fffdad0 3fffff58 3ffef0c8 4020177e 3ffffe80: 4020f040 00000000 00003a98 3ffef0c8 3ffffe90: 00000000 3ffefcd4 00000000 00000000 3ffffea0: 3fff2b5c 3ffefd24 3fff2b5c 00000000 3ffffeb0: 00000000 00000000 00000000 00000000 3ffffec0: 00000000 3fff3a24 3ffefe84 3fff7b74 3ffffed0: 3ffefe9c 00000000 00000000 40202c7f 3ffffee0: 00004145 00000345 00000001 00000000 3ffffef0: 00000000 00000000 00000000 00000000 3fffff00: 00000000 00000000 3ffe0000 40209458 3fffff10: 3ffef1d8 000000b7 00000000 400039a8 3fffff20: 00000000 00000000 00000000 00000000 3fffff30: 3fffda00 00000000 00000000 40203b38 3fffff40: 00000000 00000000 00000000 4020f300 3fffff50: 4100020a 00e77565 00000000 00000000 3fffff60: 00fee760 3ffef198 3ffe853c 3ffef198 3fffff70: 3fffdad0 3ffef140 40206ca4 3fffefa0 3fffff80: 4020723a 000003e8 000003e8 3ffef140 3fffff90: 3fffdad0 00000000 3ffef110 4020183a 3fffffa0: 3fffdad0 00000000 3ffef110 40206d54 3fffffb0: feefeffe feefeffe 3ffe853c 401004d1 <<<stack<<< curl="" -k="" https:="" <ip="">/ curl -k would work.
It is extremely easy to DOS the plug (make it unusable). I have performed some tests:
Machine A (normal client):
for ((;;)); do curl -k https://ip/gpio/0 && sleep 1; done
Here you will normally see 1 response per second on your terminal.
Machine B (attacker):
for ((;;)); do ab -n 1000 -c 100 https://ip/; done
The first scenario I tested was before even SSL just with regular HTTP, the effect was devastating. Once ApacheBench was running with 1000 reqs it completely blocked the other client. With HTTPS the situation got a bit better, some requests get through to the device during the tests.
SSL handshake failed (5). SSL handshake failed (5). SSL handshake failed (5). SSL handshake failed (5). SSL handshake failed (5). SSL handshake failed (5). SSL handshake failed (5). SSL handshake failed (5). SSL handshake failed (5). ^C Server Software: Server Hostname: <ip> Server Port: 443 SSL/TLS Protocol: TLSv1.2,ECDHE-RSA-AES256-GCM-SHA384,1024,256 Document Path: / Document Length: 0 bytes Concurrency Level: 100 Time taken for tests: 112.802 seconds Complete requests: 182 Failed requests: 72 (Connect: 0, Receive: 0, Length: 72, Exceptions: 0) Non-2xx responses: 73 Total transferred: 7519 bytes HTML transferred: 3796 bytes Requests per second: 1.61 [#/sec] (mean) Time per request: 61979.255 [ms] (mean) Time per request: 619.793 [ms] (mean, across all concurrent requests) Transfer rate: 0.07 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 3596 6239.3 0 30187 Processing: 50 13540 24736.0 4562 103046 Waiting: 0 3 3.9 0 21 Total: 688 17136 23521.4 8188 103046 Percentage of the requests served within a certain time (ms) 50% 8188 66% 11962 75% 14440 80% 19480 90% 54749 95% 78327 98% 102940 99% 103043 100% 103046 (longest request) </ip>
It gets worse, if this is not a standard HTTP or HTTPs request just someone telnets on the port guess what that will block the code forever. With automated robots hammering the web 24/7 it's guaranteed that your plug will become unavailable sooner rather than later. For this I could not find a solution (yet) but I'm not planning on putting any of them on public IPs. The problem is with the ESP8266WebServer and ESP8266WebServerSecure libraries that they are blocking single threaded. I will experiment with the asyncwebserver in another project but that has it's own issues as well of handling HTTPs connections so I decided between the two the more important in this projects is the SSL support and not the concurrent access. In my other project it will be just the other way around.
Wifi.mode(WIFI_STA) MUST follow Wifi.begin(ssid,pass) otherwise the ESP will create an access point named ESP_ for unwanted wifi access.
Problem 5: FLASH STORAGE < this is actually NOT a problem
In case you are worried about how many write operation can the flash endure read: https://github.com/pellepl/spiffs/wiki/FAQ#how-lo...
If you consider this TL;DR then:
"Which at one write per second is 355 days. On the other hand, if your update occurs once per minute you can expect a failure in 58 years."
Now go figure how many that will increase when you write to the flash twice a day. Initially I had a build option to disable this but with the addition of the timer db I have disabled it because excluding this feature is pointless.
Problem 6: The mysterious delay() in the middle of the void() loop.
This is something specific to my code as it is not happening with the HelloServerSecure base sketch which does not contain any delay. It is the hundred dollar question why is this delay there necessary. Lowering it results larger and larger packet loss up until the point when it is removed the ESP8266 disconnects from the wifi.
Delay 1000 64 bytes from 192.168.77.15: icmp_seq=27 ttl=255 time=10.9 ms 64 bytes from 192.168.77.15: icmp_seq=29 ttl=255 time=2.37 ms 64 bytes from 192.168.77.15: icmp_seq=31 ttl=255 time=2.54 ms 64 bytes from 192.168.77.15: icmp_seq=33 ttl=255 time=2.21 ms 64 bytes from 192.168.77.15: icmp_seq=35 ttl=255 time=2.55 ms 64 bytes from 192.168.77.15: icmp_seq=37 ttl=255 time=22.3 ms 64 bytes from 192.168.77.15: icmp_seq=39 ttl=255 time=3.49 ms 64 bytes from 192.168.77.15: icmp_seq=41 ttl=255 time=77.2 ms 64 bytes from 192.168.77.15: icmp_seq=43 ttl=255 time=4.48 ms Delay 500 PING 192.168.77.15 (192.168.77.15) 56(84) bytes of data. 64 bytes from 192.168.77.15: icmp_seq=38 ttl=255 time=1093 ms 64 bytes from 192.168.77.15: icmp_seq=49 ttl=255 time=923 ms 64 bytes from 192.168.77.15: icmp_seq=51 ttl=255 time=2.44 ms 64 bytes from 192.168.77.15: icmp_seq=52 ttl=255 time=62.7 ms 64 bytes from 192.168.77.15: icmp_seq=61 ttl=255 time=811 ms 64 bytes from 192.168.77.15: icmp_seq=65 ttl=255 time=2.56 ms 64 bytes from 192.168.77.15: icmp_seq=73 ttl=255 time=674 ms 64 bytes from 192.168.77.15: icmp_seq=78 ttl=255 time=50.2 ms Delay 100 64 bytes from 192.168.77.15: icmp_seq=16 ttl=255 time=110 ms 64 bytes from 192.168.77.15: icmp_seq=19 ttl=255 time=3.00 ms 64 bytes from 192.168.77.15: icmp_seq=22 ttl=255 time=2.98 ms 64 bytes from 192.168.77.15: icmp_seq=25 ttl=255 time=3.29 ms 64 bytes from 192.168.77.15: icmp_seq=27 ttl=255 time=840 ms 64 bytes from 192.168.77.15: icmp_seq=28 ttl=255 time=3.68 ms 64 bytes from 192.168.77.15: icmp_seq=29 ttl=255 time=375 ms 64 bytes from 192.168.77.15: icmp_seq=31 ttl=255 time=3.18 ms 64 bytes from 192.168.77.15: icmp_seq=34 ttl=255 time=3.56 ms 64 bytes from 192.168.77.15: icmp_seq=35 ttl=255 time=348 ms 64 bytes from 192.168.77.15: icmp_seq=37 ttl=255 time=2.20 ms 64 bytes from 192.168.77.15: icmp_seq=40 ttl=255 time=3.98 ms 64 bytes from 192.168.77.15: icmp_seq=43 ttl=255 time=2.58 ms 64 bytes from 192.168.77.15: icmp_seq=46 ttl=255 time=72.5 ms 64 bytes from 192.168.77.15: icmp_seq=49 ttl=255 time=3.24 ms No Delay The plug initially connects to wifi, pingable for couple of seconds then goes offline.
Problem no 7: Packet loss and dupes
Over an 8 hr test period with ping (while running 1 web access emulation loop with curl):
--- ping statistics --- 38137 packets transmitted, 25163 received, +53 duplicates, 34% packet loss, time 38323695ms rtt min/avg/max/mdev = 1.584/83.870/2848.093/176.922 ms, pipe 3
Regardless that TCP will do retransmission, this could be improved. If the plug is left alone (no http requests going in) then the average ping is consistently low with some higher (1k) ping spikes.
Problem 7: Page Load Speed
I have tried everything to lower the page load speed which is hovering around 3-7 sec with the average of 4S, this is the same for the html based page and the text based m2m. Changing the cipher size for SSL between 512 <> 1024 bits does not make much difference.
I have put the relay control to the very beginning of the code to make sure the device reacts as soon as possible and spend the time on generating a response later.
The 4 sec load speed is quite high compared to the base HelloServerSecure:
Hello from esp8266 over HTTPS! real 0m0.538s user 0m0.028s sys 0m0.008s Hello from esp8266 over HTTPS! real 0m0.549s user 0m0.024s sys 0m0.008s Hello from esp8266 over HTTPS! real 0m0.528s user 0m0.016s sys 0m0.008s
Which loads in on half second.
Problem 8: Input/Output error on /dev/ttyUSB0 Errors after a lot of programming
While I was working on this project after certain number of uploads (~70) the ESP8266 board started to become harder and harder to program because it lost connection with some error. Pushing the upload button 5-15 times will get quickly annoying for anyone else too. Sometimes it solved it detaching from the VM and re-attaching the USB port.
Problem 9: server.send() fails to send STRINGS/HTML greater than 6600~ byte long #3205
The original code was written on a way to append everything into a large String, then send it back to the client. For the base structure, just to display amps, watts, on/off switch that was enough but as soon as I added the time entries I run into a brick wall. "If the string you are sending is 6600 bytes long, TCP stack will need another 6600 bytes to store a copy of the string while the transmission is in progress. So if you go below 7kB of free heap, there will be ~400 bytes available in heap at some point, which may not be enough for some TCP or WiFi stack purposes. Also, if the remaining 7kB are fragmented (i.e. not a contiguous chunk of RAM but a bunch of smaller blocks), then the TCP stack will not be able to allocate the temporary storage used for transmitting."
A certainly working solution would've been just to lower the output what I send back to fit in the limits: eg.: you would have 1 single time GET/SET box with no dropdown list just empty raido boxes where you would have to fill in the value, that just looked ugly and non-user friendly to me so I took my time to investigate the root cause and rewrite my whole code which now basically sets the CONTENT LENGTH to unknown pushes out the top of the html page the same way as it did with the server.send then the rest of the data is sent in chunks (as soon as it's generated) with server.sendContent(). This turned out to be a good investment since this way even without the TIMER code I get faster page generation speed.
Without the timer code the page load speed can be as fast as 3 seconds but because I put in so much work in the timer I did not want to make it as a BUILD option. Therefore I put in a simple function to enable/disable the whole timer when it's needed or not needed.
With static IP the WL_CONNECTED status will always be true, even if it can't connect to the Wifi. For this I did not find a definitive solution and did not need to either since I have implemented a secondary check to validate the initial date received from the NTP server. So obviously if it can't connect to the Wifi then the date will always be 1970 so the plug will stuck in initialization state forever, as it should. This is the desired behavior.
Step 6: Arduino Based Wireless Plugs
In this writing I will share my plans for constructing your own Arduino (Atmega 326P) based wireless plugs.
So what components do we need to build an improved wireless smart plug:
1, Plug outlet: these if you want to design with makerbot go ahead I will just use HomeEasy cases. These devices are cheap second hand so you can even sacrifice some fully working units for the project.
2, Atmega 326P + programmer
3, Push button + led
4, HC12 trans-receiver or cheap 433Mhz RX/TX modules (these can interfere with each other, see later)
5, Transformerless power supply parts (this would even work with a closed system which is only programmed by removing the chip, don't ever use TPS with any micros where you plug your laptop into directly)
6, Arduino Promicro/Nano/Leonardo/Mega etc. for control base
7, 17cm copper wire for control base antenna
The big advantage of building your RC switch that it will work exactly as you planned it. You can set your own encryption on the protocol, use any libraries you wish, create your own learning method for the switch.
Learning is definitely the trickiest part of this project so let's start with that. There are multiple ways to implement this as well. Since we have full control over all the commands we don't even need a LEARN button. The control base can simply send a command like R4_RECONFIGURE_Rx where Rx can be the control number of another relay (eg R10) so from this point R4 will act as R10 and does not answer to any R4 based commands anymore until reboot and you can have as many pre-configured Rx definitions as you want. Yep now we hit a brick wall here since there is no EEPROM or other form of storage to keep volatile data we could use the DS1302 https://playground.arduino.cc/Main/DS1302RTC which would be also a double use technology since you could use the chip for it's original function as a timer to use the plug as a timer as well.
As of security you could use commands such as R2_ON_SGVsbG9LaXR0eQo= where the end of the string is the super secret password which if it matches then the relay operates or R2_ON_CODE which gets salted with a secret passphrase on the Arduino's end but none of these methods would stop a knowledgeable determined attacker. Since in both cases if you transmit static data which can be captured and replayed (which is unknown but always the same) that is security through obscurity. Let's say someone knows the type of radio is used to control your plugs and he needs the keyword he could try to brute force it with common strings like ON and it is not likely that he would ever get to R2_ON_SGVsbG9LaXR0eQo= but with listening into the ether this data can be easily captured and replayed. If I go through the trouble of building an own RC plug I want it to be more secure.
Implementing a rolling code based on server<>client time (in this case raspi<>ardu plug) https://forum.arduino.cc/index.php?topic=314845.0... would be a possibility if my project would have a built in RTC, also this would require a time sync between the devices (first time when the Arduino gets booted) and it would lead to issues like clock skew over time especially using cheap components.
Since we have a 2 way communication implementing CHAP (Challange Response Authentication Protocol) is a way to go. For this I have chosen the RC4D project to encrypt the data after the secure communication channel is established.
The idea here is the "sender" first sends a request to the "receiver" saying "Let me in".
The "receiver" then sends back a message saying "Ok, here's your session key".
The "sender" then encrypts the connection credentials using that session key in some way (or encrypts it in some way including that key with the credentials) - ideally with a trapdoor hash algorithm, and sends it to the "receiver".
The "receiver" then does the same operation with the data it expects the "sender" should send, and compares it with what it received.
If they match, then the "receiver" can allow communication from that "client". The "receiver" would then send a new session key to the "sender" which would then be used to encrypt all communication thereafter. That key must be different to the first, and would need to use symmetric encryption.
This all means that:
The authentication key changes every time - there is no chance of "replaying" it The communication key changes with each session, so again there is no chance of replaying messages at a later date, since the key would be wrong.
Yes, there are a lot of overheads with this method, but it's secure.
The authentication methodology is basically the Challenge Handshake Authentication Protocol (server sends challenge, client responds with credentials along with challenge, trapdoor encrypted, server compares with its own encryption and allows or denies). The rest is basic encryption of your choosing using the unique key the remote end of the connection provides.
"In some of my projects I want to use wireless communication between multiple embedded devices. Some applications like smart-home might contain sensitive data or might be susceptible to attackers that might gain control over every device. So I would like to encrypt the messages in a simple but secure way. Currently AES is state of the art but is not simple and comes along with some constraints. Yet it offers a high degree of diffusion. RC4 on the other side is very simple and fast but does not offer the same level of security. For example, there is no diffusion at all. While derived from a stream cipher, the proposed algorithm RC4D is actually a block cipher with blocks of arbitrary length. Compared to AES, the algorithm is up to 10x faster, takes about only 1/4 of flash space and can be applied to plaintext of any length without padding. Compared to RC4 we traded the arbitrary key length and a 2x longer execution time and space consumption for a higher degree of security through diffusion."
From the table I have used:
RC4D kip (opt) 1135 466
The decryption will occur after the CHAP negotiation. I went with the fastest possible implementation simply because I don't want to have more than 3-4 seconds delay between pushing a button in an app and the device to go on. With a commercial wireless plug set (regardless it's shortcomings) the devices turns ON/OFF almost instantaneously (under 1 Sec) but with controlling them through a Phone/webapp > Raspi > Arduino > 433Mhz Transmitter this is already slowed down.
I have separated the encrypted communication process to 2 categories: CHAP negotiation, Encrypted Communication:
CHAP negotiation in theory:
1) The client sends a request to login.
2) The server calls millis(), encrypts it with it's initial key (which is the same in both the server and client), and sends that as the challenge to the client. Although the initial key is hardcoded in both, the data what gets sent over is going to change every time.
3) The client uses it's (embedded, initial) key to decrypt this challenge and send an ACK response back to the server if succeeded using the same key for the encryption
4) The server decrypt the response and it should be identical with the sent challenge -> a secure communication channel has been established.
5) Each time the server sends a new command to the client it is using the latest stored password (session key) to encrypt the packet plus sends along a new key with the command (new session key)
6) Each time a client responds back to the server it is using the last stored session key to decrypt the CURRENT packet and encrypts the response using the NEW session key provided in the packet
CHAP negotiation in detail:
1) The client gets re/booted (replugging the plug) and sends the pairing request for the server: R1_HELO If the pairing process is not completed the client goes into sleep mode for 5 minutes then retries it indefinitely. In this mode it does not accept incoming commands.
2) Server calls millis(), stores this as Session_key and sends it as proposed session key => (for simplicity let's say it is 1111) R1_EHLO_ << This data changes for each session due to the clock source. The millis() value is a long data type. The value will increase once every millisecond from 0 to 4,294,967,295 over a period of 47 days. It is possible to hit it again in 6 weeks, but to get that exact millisecond again is highly improbable.
<br>key = "SuperSecret" < Unknown for the attacker (Initial static hardcoded master key in both the plugs and base station) plaintext = "AUTHSERVERSECRETWORD+1111" < Clock source intermediate = rc4_with_cfb_encrypt(plaintext, key) intermediate = reverse(intermediate) ciphertext = re4_with_cfb_encrypt(intermediate, key)
Here the AUTHSERVERSECRETWORD is necessary for the client to determine after the decryption that this is a valid packet. Even if 1 bit gets damaged during the radio transmission the payload will be gibberish. If we would just check to see if the decrypted value is a valid integer coming from the millis() source that could pass validation leading to the two ends start using other keys (gets out of sync). To avoid this I go back to the KISS (Keep it Simple Stupid) principle, if something does not checks out, abort.
3) Client decrypts the message using the reverse procedure with the initial hardcoded master key
Just a reminder the reason why the millis() clock source is used inside the encrypted packet as a new key because this will cause the whole encrypted payload to change every time. Only the client who has the initial decryption key can decrypt this packet.
key = "SuperSecret" < Unknown for the attacker (Initial static hardcoded master key)
ciphertext = ""
intermediate = re4_with_cfb_decrypt(ciphertext, key)
plaintext = rc4_with_cfb_decrypt(plaintext, key)
plaintext = reverse(intermediate)
Now without the correct decryption key, the decrypted data (plaintext) will always be gibberish. If the decrypted data does not contain the AUTHSERVERSECRETWORD then the client will simply abort the connection and will not message back to the server. If it did contain it, the client will extract the clock source 1111 and use it as:
Session_key = 1111
4) Client sends the password and challenge back to the server encrypted: R1_ACK_
key = "1111" < This is now encrypted with the new session key
plaintext = "AUTHCLIENTSECRETWORD"
intermediate = rc4_with_cfb_encrypt(plaintext, key)
intermediate = reverse(intermediate)
ciphertext = re4_with_cfb_encrypt(intermediate, key)
5) The server still stores the clock source Session_key=1111, it has the hardcoded key. If it receives a packet starting with R1_ACK_ will execute the following procedure:
A, Do I have the session key set? No => abort, Yes => continue
B, Use the last proposed session key as the key to decrypt the data
C, Check if the decrypted data contains the AUTHCLIENTSECRETWORD. No => delete last session key and abort, Yes => keep the session key and send the first command
As you see this CHAP implementation even with the lightest 1.2 seconds encryption times is quite heavy already.
The transmitter and receiver exchanged 3 messages, that is now with taking the radio delays, encryption<>decryptions back and forth, the processing times in account over 6 seconds. If you push that damn button and the thing does not come online right away you will start feeling that security has it's price.
This CHAP negotiation might works in the computer world where the underlying TCP/IP guarantees reliability and packet re-transmission however with this cheap 433Mhz radio set there is no reliability whatsoever, even more likely that you will have major packet loss and even the loss of a single packet during the CHAP negotiation or re-keying will break the encryption so each and every step would require an ACK from both sides and if there is no ACK then either to have hold timer and re-transmit or restart the whole process. So you can even forget about the 6 sec, with all the ACK overheads this would be like 3x higher and since all of these 433Mhz modules are using the same channel, all it would take to render the whole system useless is to have a rogue Arduino running a 5 liner code which just keep sending 'A' characters in a for loop forever with VirtualWire.
This also brings us to the next problem, there is 1 single channel for all making simultaneous transmission difficult. So what you can see at the end on my Halloween video would nowhere near to be even possible. The base station would need to send the command for plug 1, then plug 2, then plug 3... unless you would create a large packet for example:
Which is received by all plugs and they all extract their own control data from it. I don't know what type of limitation does VirtualWire impose on max packet size but you can see this is already more trouble than it worth. When you deal with unencrypted data you can easily create multicast groups or address individual plugs with the flip of couple 1s and 0s.
For this however a workaround would've been using the HC12 trans-receiver which provides over 100 separate channels but this would mean that the base station would have to have 1 HC12 module per plug since this module does not support any kind of "channel hopping". An Arduino Mega has 3+1 hardware serial ports so unless you would use SoftwareSerial you would need 3 Arduino Megas just to be able to control 10 plugs. However to be precise here, the Arduinos are single threaded therefore, although with a very low (mS) delay the 3 megas would execute their tasks sequentially, sending the command with Serial.println() then Serial1.println() and then Serial2.println() so to make it truly parallel you would need 10 Arduinos (any model) connected to a Raspberry PI with an USB HUB + 10 HC12s to control 10 plugs. Although this would solve the simultaneous transmission problem and would even provide more protection against flooding it would not solve the slow negotiation, encryption problem.
It is the main reason why I abandoned this idea and remained only here on paper but let's continue.
The 2 peers now have a secure communication channel, however the key still have to change at each session to prevent replay attacks.
1, Server calls millis()=2222, saves that value as Session_key_new and applies it at the end of the command string.
key = "1111" < Previously agreed key plaintext = "R1_ON+2222" < Command + new proposed Session_key intermediate = rc4_with_cfb_encrypt(plaintext, key) intermediate = reverse(intermediate) ciphertext = re4_with_cfb_encrypt(intermediate, key)
2, The client receives and processes the message it detects that it has the _COMM_ command and assumes an already encrypted string (what it supposed to be able to decrypt).
First it checks if the Session_key is set, if not it aborts right away. If it is it calls the decryption routine.
key = "1111" ciphertext = R1_COMM_ ...
Eventually it's gonna get R1_ON+2222 what if it's a valid command (R1_ON matched) then it proceeds execution, if not then it assumes gibberish and does absolutely nothing, aborts the process.
If it is a valid command, for the fastest processing possible after this step it immediately turns ON or OFF the device.
3, The client will always respond with the proper R1_ON_OK/R1_OFF_OK message in a packet: R1_CACK_
The client sets it's Session_key=2222 and reponds back to the server.
key = "SuperSecret+2222" < Current key plaintext = "R1_ON_OK" intermediate = rc4_with_cfb_encrypt(plaintext, key) intermediate = reverse(intermediate) ciphertext = re4_with_cfb_encrypt(intermediate, key)
It responds back to the server with an R1_ON_OK command.
4) The server sees a message coming in containing _CACK_ it assumes it is a response packet it have a key for in the Session_key_new variable.
It will decrypt the message back to R1_ON_OK and check if it contains a valid command (there is no need for CLIENTSECRETWORLD anymore since R1_ON_OK string can determine if the packet was decrypted successfully). If it doesn't => abort, if it does continue.
If it does then R1_ON_OK will be sent back to the Raspberry PI and it sets it's session_key to: 2222
As you might noticed in this process it is always the Server who is generating the random session keys, the client never generates keys just accepts and reuses them.
The problem with radio transmission is that packets can get lost. When the server sends a command packet to a client, it will wait for an answer for a determined time period. If the answer like R1_ON_OK does not arrive it will try to resend the packet encrypted with the same Session_key (k2) and using the same Session_key_new (k3) as proposed key inside the message. If it does not get a response then it will not update it's keys.
I, Server <> Client goes out of sync. ACK packet lost
Let's assume a scenario where the client got the packet, turned on the relay, set the new session key and sent back an R1_ON_OK but the packet got lost. Since the server did not get back a response it will try to resend the command packet again using it's current Session_key (k2) to encrypt the packet which contains the command and the proposed Session_key_new (k3) once again. However (since no more checks are implemented in the code) the client thinking that everything went fine already switched it's Session_key to the new one therefore when it gets the next command packet it will decrypt it to gibberish and does nothing.
For this solutions could be:
After R1 TURNED ON acknowledged by the server, the server will send another ACK back to the client that it did receive the last packet.
key = "SuperSecret+2222" < Current key plaintext = "REKEYED" ...
It responds back to the client with: R1_FIN_
The client will decrypt the packet using Session_key_new as decryption key and if the content matches REKEYED then and only then will it update it's session key.
In this case if the server had to resend the packet, it can still get decrypted by a valid key.
Client gets restarted (replugging the plug). Executed with a random time delay between 0-2 minutes at startup and executed only once (in case multiple plugs would get plugged back in the same time and the radio channel is the same). R1_HELO, this will clean the previous session_key session_key_new variables if they are present on the server and restart the chap process. An attacker could abuse this by intentionally "depairing" the devices (something like the deauth attack). If it is done while the real R1 is offline all the attacker would see is a response R1_EHLO_, that's it. The server would simply not proceed with any further, however if the real R1 would be plugged in as well it would do the rekeying process.
III, Server gets restarted.
A method on the client side would need to be implemented to re-auth all plugs upon boot time. Otherwise none of the plugs would be in registered ONLINE status, no session keys (k2, k3) would be set and nothing would happen. For pairing a LED blinking on the plug could be implemented.
IV, Flood attacks
Depending on the radio used to the communication whether it is a simple 433Mhz RX/TX unit or a more sophisticated HC12 (which can be set to different channels) even if an attacker does not break the encryption protocol he/she can easily flood this band with junk messages blocking all the plugs to respond. Simple devices like these will not hopp channels automatically such as 802.11 devices if they see the spectrum is overloaded.
Although this Arduino based smart plug remained an idea I have still felt it relevant to document as it might be good enough for someone else's build who can accept a trade off of basic security and attack vectors over an ESP32/ESP8266 secure 802.11 WPA2 plug solution.
To summarize how many steps would it take just to turn a relay on or off with this design:
Android App > C control code raspi (R1_ON) -> Arduino Base Station (Encrypt) -> Single message to plug -> Arduino Client (Decrypt) -> Matched command -> Execute
Step 7: Conclusion and ToDo List
If you fail try it again, again and again. Edison made 1000 unsuccessful attempts at inventing the light bulb, this project had lot less attempts starting with the first idea to use Atmega326P stayed nothing more than theory, then on the new versions there were problems with the layout, pins, webserver it went through another 3 iterations from reasons like the relay shield could not be fitted perfectly on top of the PCB. Overall it was around ~200 reprogramming of the Wemos until I got to the current version. I do have regrets even using shields at the end because they are more hassle than they worth. De-soldering the terminal-strips from both the ACS712 and the relay shield didn't go quite smoothly and sometimes it took part of the PCB with it, however it was necessary otherwise the solder would not run through properly both sides providing double copper area, as I have planned. Overall if I would have to do a V4 that would not use shields for sure.
Resettable FUSE could've been used. The current glass fuse is also blocking the 3rd screw hole on type 2 plug enclosure.
Make a low voltage switch variant of the board. This would have been a modification of the original board design to power small USB appliances directly like LED cubes, raspberrys, arduinos without additional power supply from the main power supply which is always ON anyway. In this variant the 230V mains would be bypassed from IN to OUT without any switching functionality, the plug would still allow chaining any device using that plug without losing 1 valuable socket however it would no longer control it, instead it would have a standard USB connector on the side and it would turn that ON or OFF. I have abandoned this idea, although I have ordered 2 variants of the mini psu:
HLK-PM01 5V, 0.6A, 3W
HLK-5M05 5V, 1A, 5W (this one is also bigger form factor, board would have to be adjusted)
If you think about it this is not that much these days.
The Wemos D1 mini drains on average 150 mA, 250mA tops, add the other components like the ACS712, relay, led ~ 0.5A max. Max currents USBs can provide per specs:
USB 3.0 -> 900mA
USB 2.0 -> 500mA
Sample load: Raspy 3 400% CPU load (stress --cpu 4) 980 mA (5.1 W)
So the PM01 is perfectly enough for the plugs internal operation the 5M05 could give power for an external mini appliance as well like an led cube or raspi zero but it did not worth the hassle to implement this since I also don't know if the PSU has overload protection and shuts off or it just burns out also as you can see in my other project SilverLight that the HLK-5M05 failed to provide stable 0.63A regardless that the claimed current was 1A.
As I described on multiple points the Webserver code is far from perfect so improvements can be done and will be done since the "ESP8266 core for Arduino" library is actively developed.
Timer could be extended to turn ON/OFF even on certain days, months, years. I considered implementing this a waste of time since it's a 1 liner cron job and the plug can easily be turned on/off remotely from any Linux box from crontab with curl.
Timer list could be generated more nicely by using "option selected" but it would require more code generator logic that way.
Develop an Android/IOS app using the m2m api. On phones it is more convenient to use apps than web pages.
Document Over The Air (OTA) sketch updates for the ESP8266 (a possibility to update the plugs software without connecting them one by one to a laptop for reprogramming with the Arduino IDE).
Is the plug ECO friendly compared to the commercial alternatives? I would say it's pretty good in power consumption compared to just how much more it can do. Although I did not have precise meters to measure the consumption other than my cheapo wallplug meter, an Intertecho in idle consumed so low the meter could not measure it, a Home Easy HE877 consumed 0.5W standby (both on or off states), my plug consumes 0.5W standby and 1W relay on and I would say that is a great number which just shows how inefficient those cheap Transformerless Power Supplies are.
Could be said that this project is over complicating the original design. You have a device, push the button and it will turn a device ON or OFF but what's under the hood is a lot more. If you building systems where the highest reliability is expected then the best is to use a centralized relay board to control everything, unfortunately more often than not we don't own the building where we live and rewiring it is not an option so if you need reliability and security this project might be a good trade-off for remote control.
And last but not least, Happy Halloween.
Participated in the
Halloween Contest 2019