Weather Station: ESP8266 With Deep Sleep, SQL, Graphing by Flask&Plotly

Would that be fun to know the temperature, humidity, or light intensity on your balcony? I know I would. So I made a simple weather station to collect such data. The following sections are the steps I took to build one.

Let's get started!

Step 1: Weather Station With Light, Temperature and Humidity Sensors

When I planned to build a weather station, I was dreaming to have a full-fledged weather station that has wind speed, rain measurement, full-spectrum solar sensor, but turned out, that would not be cheap, and the purchase cost could end up at least $100. I gave up the full options and started to build one with $10, more or less. $10 is the cost of basic components of the weather station as the parts below.

Here are the parts:

1. ESP8266 Wemos brand costs $2.39 pcs on Aliexpress. I would recommend Wemos brand because its EPS8266 is easier to program, update, and have 4MB flash or more.

2. Wemos Charger-Boost Shield costs $1.39 pcs. This is another benefit to use this brand. It has a boost-up board for Lithium battery (nominal voltage = 3.7V) to a 5V for ESP8266. The board also come with charging option with a max charging current = 1M.

*Note: There is a cheaper option for Lithium battery charging/boost up. This one costs $1.77 for 5pcs. However, when I used this board for ESP8266 (either Wemos's or a bare ESP8266), the deep-sleep mode of ESP8266 triggered a reset right after making the ESP8266 in a loop of sleep-reset-sleep, which is very annoying. If you know what was happening, please inbox me.

3. Wemos also has several shields for temperature and humidity but I am going to build from individual components. Photoresistor (or light-dependent resistor -- ldr, cheap), a luminosity sensor such as BH1780 or TSL2561 (about 0.87-0.89c pcs), a temperature sensor such as DS18B20 (75c each), and a humidity and temperature combo such as DHT22 ($2.35 here) or SHT21 ($2.20 here). A total cost for the sensor ~$4.

4. Lithium battery. I salvaged one from a 7.4V Canon Battery which is two 3.7V battery in series or 18650 Lithium battery. Each 18650 costs about $5 apiece. I have a picture show the tear-down the camera battery pack. Be careful though, short-circuiting when cutting through the plastic cover could generate extreme heat, and burn.

5. PCB board, jumper, wire, soldering, your time, maybe some debugging skills.

Let wire components together follows the schematic above.

Then, have a look for the task in the setup loop. It is simply a one-run of tasks and ends by a sleep command.

void setup() {<br>  Serial.begin(115200);
  Serial.println("Starting Node named " + String(SENSORNAME));
  setup_wifi();
  delay(100);
  Wire.begin();
  pinMode(ldrPin, INPUT);
  SHT21.begin();
  if(!tsl.begin()) {
    Serial.print("TSL2561 not found");
    while(1);
  }
  delay(100);
  
  ldr = analogRead(ldrPin);
  
  tsl.enableAutoRange(true);  
  tsl.setIntegrationTime(TSL2561_INTEGRATIONTIME_13MS);  
  delay(100);
  sensors_event_t event;
  tsl.getEvent(&event);
  if (event.light) lux = event.light;
  else Serial.println("Sensor overload");

  h = SHT21.getHumidity();
  t = SHT21.getTemperature();
  tempSensor.setWaitForConversion(false); 
  tempSensor.begin();                     
  delay(100);
  if (tempSensor.getDeviceCount() == 0) {
    Serial.printf("DS18x20 not found on pin %d\n", ds18b20);
    Serial.flush();
    delay(1000);
  }
  delay(100);
  tempSensor.requestTemperatures();
  t18 = tempSensor.getTempCByIndex(0);
  
  Serial.printf("\nLight: %d lux\t", lux);
  Serial.printf("LDR: %d /1024\t", ldr);
  Serial.printf("T: %0.2f *C\t", t);
  Serial.printf("H:%0.2f \t", h);
  Serial.printf("HIC: %0.2f \t", hic);
  delay(100);
  client.setServer(mqtt_server, mqtt_port);
  client.setCallback(callback);
  reconnect();
  delay(100);
  
  ESP.deepSleep(3e8); // 300 millions micro seconds, 300 seconds, 5 minutes;
}

During debugging or setting up, command out the ESP.deepsleep() to have Serial readout continually. As always, the full code to upload to ESP8266 is hosted here ( GitHub).

Remember to put on the jumper between RST and D0/GPIO16 for triggering a wake up after a period of deep-sleep.

Now, time to upload the code using Arduino IDE to the ESP8266.

Step 2: MQTT: a Flexible Medium to Publish and Subscribe Data

First, I'm growing fond of using MQTT to send and receive data across different sensors and clients in my home. That is because the flexibility to send unlimited data categorized by a topic, and unlimited clients to subscribe to one topic from an MQTT broker. Second, I'm not qualified to discuss MQTT in-depth. I got to know MQTT sometimes last year (2017) when following tutorials to set up a weather station and sensors using Node-RED. Anyhow, I will try my best to present to you some info. Another good place to start is Wikipedia.

If you don't have time to read about the theory, and wanted to set up an MQTT broker, I posted another tutorial just to do so. Look up this post, and scroll down to Step 4.

To explain what is Message Queuing Telemetry Transport (MQTT) in my understanding, I prepared a diagram as above. In nutshell, MQTT is an ISO standard, and a product such mosquitto and mosquitto-client, two packages I used build MQTT broker on a Raspberry Pi, have to comply with that standard. The MQTT broker then becomes a medium for publishers to push a message into and subscribers to listen to a target topic.

The combination of Arduino PubSubclient library with ArduinoJson, thanks to its creator knolleary and bblanchon, makes easier for the tinkers and developers for a set of tools from sensors to a target equipment or an end client.

Let move on with create Database and display some data.

Step 3: Save Data to SQL and Display Them on a Web Server

I used sqlite3 to create a database for the web server. Install the sqlite3 in Rapberry Pi by:

sudo apt-get install sqlite3

created a database and a table by typing into the terminal:

sqlite3 weatherstation.db

CREATE TABLE weatherdata (id INT PRIMARY KEY, thetime DATETIME, ldr INT, tls2561 INT, ds18b20 REAL, tsht21 REAL, hsht21 REAL);

.exit //to exit the sqlite command line and return to Linux terminal

To listen to a topic published by the weather station, I used a Paho library with Python:

#! /usr/bin/python3<br># adopted from: <a href="https://github.com/bsmaat/mqtt/blob/master/sql/sqlwriter.py" rel="nofollow"> <a href="https://github.com/bsmaat/mqtt/blob/master/sql/sq...</a">  https://github.com/bsmaat/mqtt/blob/master/sql/sq...>>
# Binh Nguyen, August 04, 2018,
from time import localtime, strftime, sleep
import paho.mqtt.client as mqtt
import sqlite3, json

mqtt_topic = 'balcony/weatherstation'
mqtt_username = "johndoe"
mqtt_password = "password"
dbFile = "/path/to/databse/weatherstation.db"
mqtt_broker_ip = '192.168.1.50'

# The callback for when the client receives a CONNACK response from the server.
def on_connect(client, userdata, flags, rc):
    print("Connected with result code "+str(rc))
    client.subscribe(mqtt_topic)
    
# The callback for when a PUBLISH message is received from the server.
def on_message(client, userdata, msg):
    theTime = strftime("%Y-%m-%d %H:%M:%S", localtime())

    topic = msg.topic
    payload = json.dumps(msg.payload.decode('utf-8'))
    sql_cmd = sql_cmd = """INSERT INTO weatherdata VALUES ({0}, '{1}',\
        {2[ldr]}, {2[tsl2561]},{2[ds18b20]}, {2[tsht21]},{2[hsht21]})""".format(None, time_, payload)
    writeToDB(sql_cmd)
    print(sql_cmd)
    return None

def writeToDb(sql_cmd):
    conn = sqlite3.connect(dbFile)
    cur = conn.cursor()
    cur.execute(sql_command)
    conn.commit()

client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.username_pw_set(username=mqtt_username, password=mqtt_password)
client.connect(mqtt_broker_ip, 1883, 60)
sleep(1)
client.loop_forever()

To display data from use another SQL command to query data from the database such as:

sql_command = """<br>        SELECT * from  weatherdata         ORDER BY thetime DESC         LIMIT 1000;"

This SQL command is included in the app.py that uses Flask framework and Plotty to make a web server and plotting a graph.

The complete code is hosted on the GitHub.

If the ESP8266 cannot read the DS18B20, it assigned a value of -127 as the temperature which skews the relative range of other readable temperatures. I cleaned up those values by set a NULL value to those equals to -127:

sqlite3 weatherstation.db
sqlite3> UPDATE weatherdata SET ds18b20 = NULL WHERE ds18b20 = -127;

To set up an environment for this mini web server, I used the shared libraries on Raspberry Pi. A virtualenv is a better option if the web server is hosted on a powerful computer. Start the web server by:

python3 app.py

Press Control + C to stop the server.

The web server is set to auto-refreshed for every 60 seconds. You can change the interval in index.html file:

<meta http-equiv='refresh' content='60'>

Battery performance:

I did not measure the current between the normal state or sleep state of ESP8266. Many others did so. The first google search turned to this page. The normal state of ESP8266 consumes about 100mA depends on the rate of transmitting and wifi activity. The deep-sleep state needs in the range of micro A, which a thousand times less.

For 5-minute interval between sleeping and waking up, one single Lithium 18650 (2000mAh) could fuel my weather station for 12 days. The same battery only enough for ESP 8266 ran less than a day with a normal working state. The one I took from the camera battery pack (did not know the capacity) was enough to run the weather station with deep sleep for 5-6 days.

Thank you for spending time with me to this end.