Introduction: Powerful Standalone Home Automation System - Pi, Sonoff, ESP8266 and Node-Red

Picture of Powerful Standalone Home Automation System - Pi, Sonoff, ESP8266 and Node-Red

This guide should get you to first base where you can switch a light or an appliance on/off via any device that can connect to your local network, and with a great customisable web interface. The scope for extension/ adding features is vast, including sending emails, twitter messages, reading sensors (e.g. temperature). You can easily set rules e.g. – if the temperature is below 15C at 11pm turn on the electric blanket for 30 mins. The system uses the MQTT protocol with TCP that is considerably more reliable than UDP - that is surprisingly used by some commercial home automation devices. Once the hard work of setting up the system, described here, is done the fun can start. Node Red provides a fantastic user interface that is intuitive, quick and easy to set up, and provides amazing capability.

The outcome is a powerful home automation system that is fully under one’s own control and is entirely on a local network. Many of the systems shown on Instructables work with cloud based servers and hence are a security risk and can require subscription. These are easier to set up but have a higher security risk and potentially higher cost. I should add that one can set up a VPN (virtual private network) to the Pi to enable secure connection to the home network/system if access from outside would be useful (Google Pi OpenVPN for instructions).

This system requires reprogramming Sonoff devices with a great open source system called Sonoff-Tasmoda. This can be done using a special Arduino setup and a USB to serial converter. The Sonoff devices use the ESP8266 Wi-Fi chip so you can also make your own devices using ESP8266 modules or fit them to other items to deliver wireless connectivity.

The biggest task is setting up a Raspberry Pi as the hub. This runs the MQTT broker (or server) that manages the communication with the connected devices. The Pi also runs a system called Node-Red that can intercept messages and data and output commands – such as when to turn on the electric blanket. Node-Red also provides an interface to serve the webpage that you can log on to via smartphone/tablet/PC/Laptop, to see what is going on and enable manual switching of lights and so on.

My aim with this instructable is to give every step in enough detail that a novice can get the system working.

The steps are:

  • Load a Pi with Rasbian Stretch
  • Update and add features to Node-Red
  • Install the Mosquitto MQTT broker
  • Set up the Arduino environment for uploading the Sonoff-Tasmota firmware
  • Re-program a Sonoff switch
  • Set up the Node-Red interface
  • Test it all works.

Hardware you will need:

  • Raspberry Pi and power supply (and keyboard and monitor for initial setting up) (model B used here)
  • Sonoff Switch
  • USB Serial converter
  • Optional – ESP development board such as the NodeMCU.

My experience with the Raspberry Pi is that setting up anything seems to involve considerably more pain than one could ever imagine possible with stuff not working or getting stuck on some trivial step where the explanation could have been better. The forums do not always get advice that works! I found several routes that simply did not work for one reason or another. This cost 3 days! I then wiped everything and started afresh and completed in less than a (long) day. This however used the older Raspian Jessie. After this I found a way to use the current and recently released version (Stretch) and did it all again. This guide enables copy and paste of instructions into the Pi. So you should be done in a few hours. Anyway I hope this gets you faster up the learning curve and with much less pain. Believe me it is worth it for the end result.

Step 1: Load a Pi With Rasbian Stretch

This should be pretty straightforward. Start with an 8Gb or 16Gb SD card. Download the latest version from:

https://www.raspberrypi.org/downloads/

Unzipping the downloaded file gives a folder with the .img file. 7Zip is recommended for Windows (and The Unarchiver Mac). The image has to be burned onto the SD card – but a special program has to be used as the file system is not compatible with Windows. The recommended software is called Etcher and can be downloaded from:

https://etcher.io/

The instructions for Etcher are on their website and could hardly be simpler. Select the image and the drive and click Flash.

Now with our flashed SD card with can get the Pi running.

If you know the IP address your Pi uses or are happy to find it by logging on to your router then you can avoid the need for keyboard and monitor and use SSH immediately. Just add a blank file named SSH to the SD card, insert, connect to Ethernet and power up. Otherwise follow the guide below.

Connect the Pi to internet, screen, mouse and keyboard and connect to power. The first thing we will do is enable SSH so we can do the majority of the setup from the comfort on a PC. It can of course be done direct but it helps a great deal to be able to follow this guide on the same device as one driving the Pi and use copy and paste for most of the instructions. Also in my case my PC workstation is nice and comfortable but not big enough for a pi as well.

There is a guide for a good deal of the work on a YouTube video. This is where I started. You can run the video alongside following these instructions. However when I was running through the process again while writing this I found it easier to just follow the instructions here. There are some significant deviations. There are also some sections where I have advised following the video rather than listing the steps here. I would however recommend looking at it to get an understanding of MQTT, Node-Red and the broad setup process. The video is 38 mins long so make yourself comfortable. The video starts with showing what Node Red can do, and then covers the Pi install and setup, followed by upgrading Node Red and lastly installing Mosquitto. Key timings in case you want to go back to a section:

00:00 Introduction to Video

03:00 Node Red demonstration

14:14 Importing dashboard items into Node Red

21:05 Pi initial setup, including SSH

23:35 Installation of Node Red supporting code

27:00 Introduction to MQTT

29:12 Mosquitto (MQTT) installation (Note only works for Raspian Jessie)

33:00 Node Red examples

I will list the commands in bold italics (rather than using “”). This formatting is ignored when you copy and paste them into the Pi.

The introduction to Node Red will give you an idea of what the system can do and provide a picture of where we will end up.

The YouTube video is at: https://www.youtube.com/watch?v=WxUTYzxIDns

Follow the installation procedure from 21:05 to 23:35. Note the default login is user: pi and password: raspberry. Before restarting find the pi’s IP address. Either click the ‘up+down arrow’ icon in the desktop top bar or enter the following instruction in terminal/terminal window:

sudo ifconfig address

Then note the IP address after: inet addr: . It will be in the form 192.168.x.y

Note this address and restart (enter: sudo shutdown –r now)

The key at this point is that SSH has been enabled and one can set up a link to the Pi from a PC (Mac and Linux include SSH interfaces). A good program to do this is a free download called PuTTY that can also be used as a serial interface and can do Telnet. Putty is available from:

https://www.chiark.greenend.org.uk/~sgtatham/putty...

Download and install PuTTY.

Now, with the Pi restarted, start PuTTY and enter the IP address noted earlier. See example below:

Now click Open

After Login as: enter pi

Then enter your password.

Now follow the instructions here and optionally start the video from 23:35. You can in most cases copy and paste. Highlight the text and use ctrl+C to copy. Then right click in PuTTY to paste. Sometimes the text does not appear immediately so wait a few seconds. If you do enter twice use backspace to delete the second entry. Press enter to execute each instruction.

sudo raspi-update

I got a command not found and ignored this and the following shutdown/restart instruction:

sudo shutdown -r now

sudo apt-get -y update

sudo apt-get -y upgrade

This takes a while.....

sudo apt-get autoremove

sudo apt-get -y update

sudo shutdown -r now

At this point we have the Pi operating system loaded and updated, ready for the next step.

Step 2: Update and Add Features to Node-Red

****************************************************************************************************************************

NODEJS PROBLEM:

We have been hit (1st Nov 17) by an update of Nodejs that is incompatible with npm. If node-v gives v9.0.0 then you have the incompatible version.

The procedure here has now been updated to give the latest working combination of node and npm. (8.9.0 and 5.5.1 respectively)

If node v9.0.0 loaded you can downgrade by:

sudo n 8.9.0

sudo shutdown -r now

Log on after restart and check versions by:

node -v

npm -v

If you want to be thorough now also do:

sudo n rm 9.0.0

This removes the 9.0.0 version data from the install folder.

Now go to Step 3

*****************************************************************************************************************************

The PuTTY connection will have been lost by the restart at the end of the previous step. So close PuTTY and after waiting for the Pi to boot, Log on again as before.

sudo apt-get install npm

Y

sudo npm i -g npm@2.x

Note this takes a while…..

npm -v

- should give 1.4.21

node -v

- should give 4.8.2

sudo npm install -g n

sudo n 8.9.0

sudo shutdown -r now

The PuTTY connection will be lost. So close PuTTY and after waiting for the Pi to boot, Log on again as before.

node -v

- should give v8.9.0

npm -v

- should give 5.5.1

These latter instructions show the installed versions and that our upgrades have been installed.

So at this point we have our pi loaded and updated and with the updates needed for Node Red. It is no harm to do a restart before the next stage.

sudo shutdown -r now

Step 3: Mosquitto MQTT Installation

If you have not already done so it is worth viewing the video introduction to MQTT on the video from 27:00.

This is where we need to take a different path. The procedure outlined in the video only works for the older Jessie version of Raspian. There is a discussion on the mosquitto site but no clear outcome shown and hence will stick to a simpler and safer route.

So log on using PuTTY and enter the following:

sudo apt-get update

sudo apt-get install mosquitto mosquitto-clients

Y

sudo /etc/init.d/mosquitto stop

sudo /etc/init.d/mosquitto start

The latter two instruction stop and start mosquitto and show that our MQTT broker is working.

For a quick test open two more PuTTY sessions and log on to each.

You will know by now that MQTT works by the device the needs data subscribing to a 'topic'. The broker will send any data with the same 'topic'. Then the device wanting to send data/instruction publishes this to the broker using the same 'topic'.

So in one PuTTY session enter:

mosquitto_sub -d -t hello/world

This is an instruction to subscribe to the topic: hello/world.

In the other enter:

mosquitto_pub -d -t hello/world -m "Hello from Terminal window 2!"

This is a publish instruction with the same topic with a message. The text: “Hello from Terminal window 2!” should now appear in the other terminal.

Well done getting this far. We now have the Pi loaded and updated with updates needed for Node-Red and with the mosquito MQTT broker installed and tested. From now on life gets easier a bit more fun. Close the two PuTTY sessions used for the MQTT test.

Step 4: Setting Up the Node Red Interface

First we need to start Node Red. Enter the instruction:

node-red-pi --max-old-space-size=256

Wait for it to get running and you see the text ‘Started flows’.

Now open a browser, I use Chrome, and enter the pi IP address noted earlier followed by :1880 i.e. something like 192.168.0.8:1880

You should now have the Node Red programming page in view as below:

You can now follow the Node Red examples starting at 33:00 or move straight on to a tiny bit of additional setup and loading the first flow that will show the link to MQTT and be ready to drive our switch.

The additional setup needed is to add the dashboard items that enable a web interface.

These are shown on the video at 14:14.

Follow the instructions to load node-red-dashboard.

Now we can have a little play and show MQTT working and being driven by Node-Red. This is not on the video but is key to this project. You can either follow my instructions and/or import the flows from the NRtest1.txt file attached.

First add an input inject node and an output mqtt node and link them together.

Double click the inject node (that initially is labelled timestamp). In the Payload section use the twiddly to change to string and enter: Hello from me. Leave topic blank as we can enter this in the MQTT node. Click Done

Now double click the MQTT node. Click the pencil icon to the right of the Server section. This opens a new dialog. Enter: localhost in the Server section. Click Add. Now, back in the Edit mqtt out node, enter our topic in the Topic section: hello/world . Set QoS to 1. Click Done. Now click Deploy. You should see a green blob and ‘connected’ below the mqtt node.

Now add two more nodes – input mqtt and output debug, and connect these together. Now double click the input mqtt node and enter hello/world in the topic section. The server should already show localhost:1883. Set QoS to 1. Click Done. Then click Deploy and click the debug tab on the right hand pane. Now click the grey square on the left of the ‘Hello from me’ inject node. This sends the text payload to the MQTT broker with the topic hello/world. The broker knows that the mqtt input node has subscribed to the same topic and so forwards the payload. The mqtt input node then sends this to the debug tab (RHS) and the text 'Hello from me' should appear.

This ticks another box as we have Node Red talking to our MQTT broker. Note that Node Red is just a client to the broker – like the Sonoff devices we will connect later. It however enables sophisticated automation and provides a great user interface.

Now we can make a couple of small changes and set up the flows for our Sonoff switch.

So delete the input inject node (click to highlight and press delete key). Now add a switch from the dashboard section and connect it to the mqtt output. Double click switch. Click the pencil on the right of Group. Enter into Name:Light. Then click the pencil to the right of Tab and enter into the Name section: Lounge . Click Add/Update and Add/Update again. Now, back in the Edit switch node, set the on and off Payloads. Use the twidlys to select string and enter ON for the On payload and OFF for the OFF payload. Click Done

Now go to each of the mqtt nodes and change the Topic to cmnd/sonoff/POWER . If copying and pasting, check you have not included a space at the end. This is a different topic and will not work with the Sonoff. Finding a stray space can take an hour or so – believe me! I also went to the dashboard>Theme and selected: Dark. Click Deploy and select the debug tab.

Now open a new browser session in a new window and size it like a mobile phone over the Node Red session. Enter the address : your Pi IP address:1880/ui/#/0 i.e. something like 192.168.0.8:1880/ui/#/0 . You should see a screen with Lounge and Light and switch. Click the switch on and then off. The debug window should show the ON and OFF payloads. Now if you wish also log on via a mobile as well. Note that the switch position is synchronised. It does not matter that our Sonoff is not yet connected. When it is, by subscribing to the Topic, it will pick up the message/payload and act on it.

A last little step is to make Node Red start automatically after the Pi boots up.

Node Red has a guide at: https://nodered.org/docs/hardware/raspberrypi

However the files needed are already loaded so no installation is required.

To then enable Node-RED to run automatically at every boot and upon crashes enter (open PuTTY session):

sudo systemctl enable nodered.service

If you ever need to disable this enter:

sudo systemctl disable nodered.service

Now shutdown the pi with sudo shutdown now and remove power.

This now has our Pi locked and loaded ready for action. We have our PC/mobile phone connecting to Node Red and this talking to our MQTT server. This has been a long haul and worth a big pat on the back. Well done. I found the next, Arduino bit, a good deal easier!

Step 5: Setting Up the Arduino System for Reprogramming Sonoff Devices

All the info to do this is at the Sonoff-Tasmota GitHub. The only part I had difficulty with was the Topic text – but I have cunningly got you to enter this already!

Go to https://github.com/arendst/Sonoff-Tasmota

You will find the setup instructions on the Wiki tab in the Upload tools section: https://github.com/arendst/Sonoff-Tasmota/wiki/Arduino-IDE

Rather than go through every step I will just note key aspects that I felt were important or got stuck on.

The instructions are good but need careful attention to detail. I was tripped up by there being a need for an ESP8266 folder within a folder that was within another folder named ESP8266 and hence missed out two levels.

I followed the recommendation for an entirely separate Arduino setup. I created a new folder called ‘ArduinoSonoff’ that is separate to my existing Arduino folder. The setup is quite advanced and hence keeping it separate is a very good idea. If this is your first Arduino setup make sure you install it a second time in an ‘Arduino’ or other folder for any other Arduino work, including work on ESP8266s.

Download the latest Arduino IDE from https://www.arduino.cc/en/Main/Software . Unzip the file into your new folder.

The instructions include download the Sonoff-Tasmoda system from https://github.com/arendst/Sonoff-Tasmota main page via: Clone or Download>Download ZIP . Unzip the file into your new folder.

Follow the instructions carefully. I didn’t do the Optional: Prepare for OTA upload section. We will leave this for another day.

Now start the Arduino IDE (double click arduino.exe). Load the Sonoff-Tasmota sketch via File>Sketchbook>sonoff . There is no need to do any edits. All the settings are done via a serial connection after the firmware is loaded. These are stored in EEPROM. Hence the firmware can be updated and all the settings retained. This is pretty smart stuff. However you can avoid the need for a couple steps here by going to the user-config.h file and entering your wifi SSID and password and MQTT_HOST (replace ‘domus1’– second reference, with your Pi IP address). Later you may want to enter a MQTT user and password as well. Before downloading check the board settings under Tools. These will need a couple of changes to match the requirements set out in the Wiki. Now click compile (tick icon). It should compile OK. If it does not or if the required board settings are not available then go back and check each step of the setup.

Step 6: Reprogramming a Sonoff Switch

We are now ready to download. At this stage one can either go straight ahead and flash a Sonoff switch or one can first flash an ESP8266 module. I did the latter, partly because my switches had not yet arrived (just arrived as I type this!) but also as a precaution as the flashing of the Sonoff switch is a one way step as the original firmware is not publicly available as far as I am aware. I have a couple of NodeMCU boards. These are easy to connect, having their on-board USB to serial converter. However the nodemcu reset method does not work with this system. So leave the Tools>Reset Method set to “ck”. Do the normal manual flash setup by holding the Flash button (GPIO 0 to ground) while pressing and releasing Reset (Reset to ground). I am not sure if this has a timeout or maybe I didn’t hold GPIO 0 low for long enough but I did need several tries, including doing this while the Arduino IDE is compiling!

If you want to check the responses - the relay output is D6 on NodeMCU boards. On ESP12’s this is GPIO 12. The LED output is D7 (NodeMCU) or GPIO 13 (ESP12’s).

Sonoff switch.

WARNING: I have to say “do not connect to mains under any circumstances where the enclosure is open”. Note that the PCB (at least on the Sonoff Basic (in-line switch) only has ‘single isolation’ distances between the low voltage section and the mains. So one should treat any part of the Sonoff circuit as mains voltage. The Sonoff-Tasmota GitHub does show connection of a temperature and humidity sensor to a Sonoff S20. I would not do this because of isolation concerns. – so if you want to do this get an ESP12 or NodeMCU module and set this up separately with a proper double isolated or earthed power supply.

The Sonoff S20 plug-in switch is a good starting point as it does not need any mains wiring. This can be opened by removing one screw (under the security seal) and prising the case open. The photo below shows where the tags are. Squeezing the case at these points helps.

USB serial converter

My favourite converter is the FTDI version. However this does not have adequate capability to supply the Sonoff 3.3v needs. The FTDI specification says maximum 50ma. The next best alternative is one using the CP2102 chip. This however has a 100ma limit that is still not enough. Clearly many are using this converter direct but there are also reports of loading failing. I would limit the time it is connected as it will warm up under the load. Do this at your own risk. The ideal solution is to have a 3.3v regulator as well, e.g. an AMS1117 3.3. I made up a small PCB to enable this. See Programmer for Sonoff devices.

My sequence for programming is as follows:

Open the Arduino IDE.

Under Tools check the settings are as on the Wiki.

Make any edits required to user_config.h . I set the Wifi SSID and password and MQTT broker address and timezone/daylight savings details.

Click ‘verify’ to check it compiles OK.

Plug the USB serial converter (on its own) to the PC. Note the Port number.

Now disconnect the USB serial lead from the PC and connect it to the Sonoff switch. Check the ground and 3v3 connections are the right way around (ground is connected to the ground plane on the Sonoff PCB).

Hold the programmer so the contacts are secure while also pressing the button.

Now plug the USB lead into the PC, check the port number is correct (in Tools), then click download.

I continue to hold the button throughout the programming because I don’t want to disturb the connections.

When done you should see a screen as below:

The Sonoff needs a couple of bits of information to connect to our system: the local network wifi SSID and password, and the Pi IP address. There are several ways to do this, one of which is to modify the config.ino file as noted earlier. If you have a smartphone you can (after reassembling the switch) press the Sonoff button 4 times fairly quickly to put it into a web-server mode. The LED will flash. I had to try several times to get this to work. Then, on your smartphone look for the new Sonoff network and connect. A webpage will appear where you can set the data required. The Pi IP address goes into hostname. I also changed the second SSID and password to something long and basically unusable.

Alternatively it can be setup via the serial connection just after loading. Open the Arduino Serial Monitor (under Tools).

Commands to enter:

SSId yourWiFiSSID

Password yourWiFiPassword

MqttHost 192.168.x.y (PI IP address)

You can also enter SSId1 and Password1 followed by something long and unusable to effectively disable this.

Now you can box up the Sonoff switch, open Node-Red and the Node-red dashboard and click the switch button and look at the debug tab that should now include the responses from the Sonoff. So we have another major step achieved - our first switch being driven from a PC/smartphone.

Up to now we have not mentioned security. There is the possibility to use encrypted communications. This is quite complex to set up and probably more appropriate where a cloud based broker is being used. There is also the option to add usernames and passwords for all the connected devices and disallowing anonymous users. This is pretty simple to set up. And so now to Security.

Step 7: Security

MQTT allows usernames and passwords for each client. This is easy to setup. It is probably easier to first rename each device and then set their usernames and passwords. This can be done via MQTT commands, and Node-Red is probably the easiest way to send these. First decide on a naming convention. One option is to base names on location and function. Then you will want to record the name(Topic) username and password along with the fallback topic. Note there is also a ‘reset option’ to reset the Sonoff settings to original download (see Wiki Usage>Button Functionality).

Power up the pi and after a few seconds open a browser to Node-Red (IP address:1880).

In Node-Red set up an inject node and link this to an mqtt output and set the mqtt server to localhost . Leave topic, user and password blank as we will set these in the inject node. Also setup an mqtt input node and connect this to a debug node so we can see the responses. Set the mqtt input note to localhost (should already be set) and enter +/+/+ for topic so it catches all traffic.

Enter the following sequence of settings in the inject node.

First check connectivity with

Topic: cmnd/sonoff/Status

Message: 6

click the ‘Inject once on start’. Deploy. We should see debug including 7 lines of data from stat/sonoff/STATUS6

Enter Topic: cmnd/sonoff/Topic andMessage: loungelight. Deploy. This changes the switch name from sonoff to loungelight

The debug tab should show the device restarting with its new name/topic

Topic: cmnd/loungelight/MqttUser

Message: loungelight

Click deploy. A debug should be seen from stat/loungelight/RESULT with {“MtqqUser”:”loungelight”}

Topic: cmnd/loungelight/MqttPassword

Message: loungelightPW (note be more imaginative than this!)

Clear the debug tab and Deploy.

A reply should be seen from stat/loungelight/RESULT with {"MqttPassword":"loungelightPW"}

Now enter NodeRed and NodeRedPW as username and password in the mqtt out node. This is via the Server pencil icon and security tab in the window that opens. This automatically gets copied over to the other MQTT nodes.

Check again with

Topic: cmnd/loungelight/Status and Message: 6 . Deploy.

And that the response is sent.

So at this point we have renamed our Sonof device so it will listen to cmnd/loungelight/…… topics and will log onto the MQTT sever with username loungelight and password loungelightPW. We have also set a username and password for Node-Red.

Next we have to tell the mosquito MQTT server to only accept clients with usernames and to list the usernames and passwords to be accepted.

The procedure is:

  • Stop mosquito
  • Create own config file
  • Edit config file
  • Create password file
  • Add users/passwords.

So log in with a new PuTTY session and run through the following commands:

sudo /etc/init.d/mosquitto stop

cd /etc/mosquitto/conf.d/

sudo nano mosquitto.conf This starts the editor.

Add lines:

allow_anonymous false

password_file /etc/mosquitto/conf.d/passwd

require_certificate false

Save and exit (Ctrl+X), Y, enter .

sudo touch passwd This creates a password file and the following instructions add names and passwords.

sudo mosquitto_passwd -b /etc/mosquitto/conf.d/passwd loungelight loungelightPW

sudo mosquitto_passwd -b /etc/mosquitto/conf.d/passwd NodeRed NodeRedPW

sudo /etc/init.d/mosquitto restart

Note when adding a new device you will have to either enter the user and password via the serial port and add these to the password file or temporarily change the mosquitto config file and comment out (add # at start of line) the lines "allow_anonymous false" and "password_file /etc/mosquitto/conf.d/passwd" and then reset these when the details have been sent to the device and added to the password file as above.

We have changed the sonoff name to loungelight and so update the mqtt output node (connected to the switch) to use the topic cmnd/loungelight/POWER .

Click Deploy and check the mqtt nodes show 'connected'.

Next try the switch button and look for the debug showing the Sonoff switch responding. You will notice that the device shows the change with a topic: stat/loungelight/POWER. So now change the input node that was set to cmnd/sonoff/POWER to stat/loungelight/POWER . We can use this cover a gap in our functionality. The system as initially set up will be synchronised with all logged on users but will not synchronise with switch changes made by pressing the button on the Sonoff switch. So now connect the output of the stat/loungelight/POWER mqtt input node to the switch input (LHS). Now double click the switch and untick the "if msg arrives on input, pass through to output". This brings up some new options - select 'switch icon shows state of the input'. Deploy. So now we have a good feedback loop. The dashboard switch position will always change when the Sonoff switch changes, regardless of where the change was initiated.

So we now have a secure, standalone home automation system up and running and ready for expansion to whatever you want it to do. In the next section I will cover some of my experimentation so far and challenges I plan to tackle.

Step 8: Initial Steps for Expansion

I have another Instructable Home Automation Sonoff-Tasmota Sensors LEDs Development Board that shows some of the further capabilities of the Sonoff-Tasmota firmware:

Temperature and Humidity measurement

Intruder detection (switch input)

IR remote (for TVs etc)

LED strings - both RGB and NeoPixel (individually addressable)

I2C sensors

For the above I am using a ESP12F and a custom PCB. A NodeMCU and breadboard could equally be used. This enables these additional functions without wiring into a Sonoff device, and hence is a much safer approach. With the temperature input I have been able to complete my electric blanket automation:

I have also been exploring Node-Red, including sending emails and making voice alerts. There is also the potential for the system to detect when you are in/out - by pinging your mobile phone IP address. Node-Red can also access weather and news - so one can add information as well as do automation.

There are a few tricks to be learned - but these become dead easy second time around.

Another avenue is adding a display to the pi to show the dashboard. This is 'work in progress' - or in other words I am not too happy. The display I got is difficult to rotate into portrait mode and the Chromium browser is painfully slow. An alternative would be to pick up an old tablet on ebay and use that. I may try with a Pi 2 and see if that delivers enough of an improvement (model B used for this development).

I hope this gets you started and has your imagination buzzing. The potential scope is huge. One could even modify the Sonoff code for other sensors if needed.

Overall I have been amazed at what this system can do. My original objective was to just drive a switch from a smartphone in a reliable manner via a standalone system. I had had visions of needing to manage servers and clients and writing html for a user interface. Where this ended up is way ahead of this, with greater security, excellent reliability, fantastic user interface, drag and drop programming and huge potential for expansion. And all this with much less effort.

Mike

Comments

Mamvcivm made it! (author)2017-11-07

Made this over the weekend - I don't have any Sonoff devices yet so I'm still experimenting with an Adafruit Huzzah. My only difficulties were with the setting up of the Arduino IDE but Mike kindly got me through it. I've never used Node-RED or MQTT before, so this is an excellent introduction to them.

William E.S (author)2017-11-02

Hi, When I get to Step 2: update and add features to node red, the command node -v gives me a command not found error, and the command npm -v gives me no such file or directory error. Any suggestions?

Note I started with a unmodified install of stretch and copied and pasted your commands exactly.

FlyingDrMike (author)William E.S2017-11-03

This looks like they did not install. Did you start from the full version of Stretch or Stretch Lite? Mike

William E.S (author)FlyingDrMike2017-11-03

I started with stretch lite, because I wanted as small a software footprint as possible.

William E.S (author)William E.S2017-11-03

Addendum: nodejs -v runs. Is there supposed to be a symbolic link from node to nodejs?

FlyingDrMike (author)William E.S2017-11-07

I understand that node and nodejs are the same. Have tried following the steps in Step 2, ignoring the responses to the first node -v and npm -v? If you can get npm loaded you can load node. Then you also have to load Node-Red. Mike

FlyingDrMike (author)William E.S2017-11-03

I am sorry but you are in uncharted territory as far as my experience goes. I can only suggest starting from the full Stretch and free up space by unloading the bloatware. Mike

Also note the problem we have with node.js v9.0.0 just out. So until we check a fix for this do not do the:

sudo n latest

or

sudo npm install -g npm node-gyp

Mike

jknight20 (author)2017-11-06

Why are you installing npm separately? Do a standard install and you should get the correct version of npm that matches. If n can't do that, I wouldn't use it.

FlyingDrMike (author)jknight202017-11-07

npm is a convenient way to do the installation. This procedure ends up with more uptodate versions. You might be able to use the versions loaded with Stretch. I have not tried. Sadly compatibility is a potential problem - as we found with the loading of the latest version of node (v9.0.0) that includes the latest version of npm (5.5.1) - that is not compatible/does not match. Hence the revision of the procedure here to load node 8.9.0 that I am comfortable with. If you have an alternative approach please post/publish it. Mike

FlyingDrMike (author)2017-11-04

Part 2 that shows more on the capability of the Sonoff-Tasmota firmware is now out at: https://www.instructables.com/id/Home-Automation-S... Don't forget there is also a load of functionality available from Node-Red, including its ability to drive pins camera etc on the pi.

Mike

Mamvcivm (author)2017-11-02

Hi, I'm working through this but failing when trying to install the Dashboard as per Richard Wenner's video, I get the following error log:-

npm WARN npm npm does not support Node.js v9.0.0

npm WARN npm You should probably upgrade to a newer version of node as we

npm WARN npm can't make any promises that npm will work with this version.

npm WARN npm Supported releases of Node.js are the latest release of 4, 6, 7, 8.

npm WARN npm You can find the latest version at https://nodejs.org/

/usr/local/bin/node[1762]: ../src/node_zlib.cc:430:static void node::{anonymous}::ZCtx::Init(const v8::FunctionCallbackInfo<v8::Value>&): Assertion `args.Length() == 7 && "init(windowBits, level, memLevel, strategy, writeResult, writeCallback," " dictionary)"' failed.

Can you please advise what's wrong?

Thank you

Andrew

FlyingDrMike (author)Mamvcivm2017-11-04

Done. Please see revised Step 2 that includes how to downgrade. Mike

Mamvcivm (author)FlyingDrMike2017-11-04

Marvelous - I'm now working through Step 4 :-)

Thanks v much Mike!

FlyingDrMike (author)Mamvcivm2017-11-04

Progress. I have now revised Step 2 to give the latest working versions of node and npm. Now will have a go at the downgrade. Mike

FlyingDrMike (author)Mamvcivm2017-11-03

This looks like a node/npm problem that will hit all installations as a result of Node.js updating to v9.0.0 a couple of days ago. My versions of node and npm are 8.8.1 and 5.5.1 respectively from less than a week ago. A quick search found: https://github.com/npm/npm/issues/19019#issuecomment-340940842 and a suggestion to try sudo npm install -g npm@5.3.x to downgrade. I have not tried this but will do so on a fresh install as soon as I can. Mike

Mamvcivm (author)FlyingDrMike2017-11-03

Thanks Mike, I'll give this a go and let you know :-)

FlyingDrMike (author)Mamvcivm2017-11-03

I have just got as far as node 9.0.0 being loaded... You are 10 mins ahead of me. Mike

Mamvcivm (author)FlyingDrMike2017-11-03

I get this:-

pi@raspberrypi:~ $ sudo npm install -g npm@5.3.x

npm WARN npm npm does not support Node.js v9.0.0

npm WARN npm You should probably upgrade to a newer version of node as we

npm WARN npm can't make any promises that npm will work with this version.

npm WARN npm Supported releases of Node.js are the latest release of 4, 6, 7, 8.

npm WARN npm You can find the latest version at https://nodejs.org/

/usr/local/bin/node[913]: ../src/node_zlib.cc:430:static void node::{anonymous}::ZCtx::Init(const v8::FunctionCallbackInfo<v8::Value>&): Assertion `args.Length() == 7 && "init(windowBits, level, memLevel, strategy, writeResult, writeCallback," " dictionary)"' failed.

Aborted

FlyingDrMike (author)Mamvcivm2017-11-03

Yes getting same. Now trying to remove node - but it keeps going back to 9.0.0. I may have found the file I need to remove. Will post as soon as I get somewhere. Mike

Mamvcivm (author)FlyingDrMike2017-11-03

Cheers :-)

JeremyH117 (author)2017-11-02

Great write-up! I've considered doing my own home automation but wasn't sure I wanted to tackle it since there was so much to think about. Your Instructable is now bookmarked as a great place to start should I get around to it. At the very least it made me aware of some open source options which look very promising. Thanks for taking the time to write this - great job.

Steinzel (author)2017-11-01

Wow Mike! This is awesome! You lost me a couple of times, but I think I could do it with the hardware right in my hands. I want this working in my house ASAP!

Do you have a mailing list? I want to keep up on what you're doing with this system.

FlyingDrMike (author)Steinzel2017-11-01

Click the Follow button just under FlyingDrMike - next to the nice glider! This Instructable was the slog. The next one concentrates on fun - and automating my electric blanket. Mike

Steinzel (author)FlyingDrMike2017-11-01

Thanks Dr. Mike!

hoekbrwr (author)2017-10-31

For the setup of Raspberry PI you really should have a look at https://tech.scargill.net/home-control-2016/ This is a complete system for the Pi and ESP8266 clients of all sorts. For the setup of the Pi there is a powerful script that does it all!

FlyingDrMike (author)hoekbrwr2017-10-31

Thank you. Yes I have looked at Pete's work. Seeing what he has done was part on my decision to go down the MQTT route. I had problems loading mosquitto on Raspbian Stretch (from another guide) and hence thought it worthwhile to post a route that works. I had seen that Pete had a script but not a guide on how to use it. I like the Sonoff option as I can use cheap existing hardware.

hoekbrwr (author)FlyingDrMike2017-11-01

There is a detailed guide to use Pete's script. It is so simple to use it needs only a few lines to explain how to use it. Probably that's why you overlooked the guide. I have a mix of Theo's Tasmota and Pete's HomeControl in my nodes, they both work with MQTT messages. But OK you past that station now.

peterkn (author)FlyingDrMike2017-10-31

Here’s Peter’s step by step guide: https://tech.scargill.net/the-script-step-by-step/

FlyingDrMike (author)peterkn2017-10-31

Thank you. Will take a look. Mike

ed.darby (author)2017-10-31

Great Instructable there chap, top darts, it's fairly similar to my own set-up, I have a pi running Node-Red and Domoticz, various Sonoff switches, ESP8266's doing odd things like running IR transmitters and Neopixels and the ubiquitous DHT22s, I had some old 433MHz sockets lying around so made an Arduino transceiver that is connected to the Pi and is controlled via Node-Red.

Great write up!

FlyingDrMike (author)ed.darby2017-10-31

Thank you for your comments. I guess you are also using Sonoff-Tasmota firmware. A fantastic piece of work - still being improved at a tremendous rate. I have been playing with LED strings and turning the TV off by IR remote control, voice alerts, email.... I can't believe this stuff is so powerful and so cheap. Mike

ed.darby (author)FlyingDrMike2017-11-01

Hi Mike,

I am actually using the new ESP Easy Mega firmware.. https://www.letscontrolit.com/wiki/index.php/ESPEasyMega

It integrates seamlessly with Domoticz which I can then control from any device. You're right though, it all cheap and great fun!

I also use Tasker for android which has some wonderful plugins that I use, eg MQTT publisher ;)

If you've already seen Pete Scargill's site check out his "Big Timer" for Node-Red - really useful!

Cheers. Ted

CaptClaude (author)2017-10-31

Boom. Just exactly what I have been wanting to do. I've been playing with Node-Red for other purposes (controlling an RPi) and this is right up that alley. Been wondering about N-R on an iPhone but realize now that I was thinking entirely wrong -- N-R is on the RPi and not on the phone (duh). Thanks for pulling all of this together (although it's kinda too bad that we have to do everything for both Mac & PC -- and I am bilingual in that respect) and making it easy.

FlyingDrMike (author)CaptClaude2017-11-01

I hope my write-up gets you to your goal quickly and smoothly. I find this client server stuff takes a bit to get ones mind around. I have to remind myself that Node-Red is just a client to the MQTT server (broker). One could have more than one Node-Red system logging on to the MQTT broker - and they don't need to be on the same machine. This means that you could use Node-Red as an interface between a pi and a home automation system if you wanted functionality that was not provided by the Sonoff-Tasmota firmware. The limit does appear to be ones imagination. Mike

stetho (author)2017-10-31

This is a great write-up but there's one thing that I wanted to "correct". This comment

"The system uses the MQTT protocol that is considerably more reliable than UDP - that is surprisingly used by some commercial devices"

needs some clarification. There's this thing called the OSI Model which describes how network communication works. UDP (User Datagram Protocol) works on Layer 4 - the Transport Layer. UDP is designed to be unreliable and its use isn't unsurprising. You would use it where speed is more important than reliability - for example with SNMP where you'd like to get some information about a machine on your network but you don't want the machine to be tied up answering your SNMP requests.

MQTT is an application and runs on Layer 7 - the cunningly named Application Layer. By default it uses TCP/IP on Layer 4 to send packets of information to other machines. However, some MQTT Server Applications can be configured to use UDP for the reasons outlined above.

To put it another way - MQTT is the package, TCP and UDP are equivalent to Fedex and some guy on a pushbike. They're about how the package gets there, not the package itself. Therefore, in this analogy your comment is about the difference between the package and the courier which aren't interchangeable.

https://en.wikipedia.org/wiki/OSI_model has a good explanation.

Hoping you're reading this as information and not a criticism. That's why I put "correct" in quotes at the beginning!

FlyingDrMike (author)stetho2017-10-31

Thank you for your comments. My reference to devices was meant to be to some home automation devices (switches) where UDP does not seem an appropriate transport as reliability will be more important than speed.

NigelH30 (author)2017-10-31

This is a brilliant write up for beginners Well done. And thanks I use espeasy and node_red and mqtt but node red has big timer by Pete Scargill which has more features for timings. Just install it

FlyingDrMike (author)NigelH302017-10-31

Thank you. Your comment is much appreciated. It makes the slog worthwhile. I did see Pete's BigTimer and will certainly use it when I want a more complex timing requirement. The dusk/dawn feature looks great. I ended using moment for my simple case.

JoeStrout (author)2017-10-31

This looks like a great guide! I am currently working on a very similar system, based on a ZWave-to-MQTT bridge. But I was somewhat dreading getting Mosquitto etc. installed on the RPi, and I hadn't even heard about Node Red, so this is going to be really valuable!

FlyingDrMike (author)JoeStrout2017-10-31

Thank you Joe. Take a look at the video linked for a great introduction to Node-Red. Once you have your MQTT broker set up you can drive the reprogrammed Sonoff devices as well.

peterkn (author)2017-10-31

Very similar to the setup that I’m using, except I also use the Blynk app to provide a much nicer user interface than the one you get with the Node-Red control panel. If you use Peter Scargill’s excellent scrip to configure the Pi you get the Blynk websocket node installed automatically. You also get Peter's amazing Big Timer node, which I use for all of my timing needs.
I’m currently playing with Amazon Alexa integration to Node-Red, along with Ikea’s TRÅDFRI lighting, which seems to be working nicely.

About This Instructable

29,304views

428favorites

License:

Bio: Engineer, retired. Have always had an interest in electronics, often related to radio control. This evolved into a part time business that I still practice ... More »
Add instructable to: