IoT Weather Station With RPi and ESP8266

On previous tutorials, we have been playing with NodeMCU, sensors and learning how to capture and log data on ThingSpeak (an Internet of Things (IoT) platform that lets you collect and store sensor data in the cloud and develop IoT applications):

IOT MADE EASY: CAPTURING REMOTE WEATHER DATA: UV AND AIR TEMPERATURE & HUMIDITY

With this new tutorial, we will learn how to do the same but on this time, using a Raspberry Pi to capture data from several different sensors and also exploring different ways of communicating between devices and the web :

Sensors and Cominication type:

DHT22 (Temperature and Humidity) ==> Digital communication
BMP180 (Temperature and Pressure) ==> I2C Protocol
DS18B20 (Temperature) ==> 1-Wire Protocol

The Block diagram shows what we will get at the end with this project:

Step 1: BoM - Bill of Materials

Raspberry Pi V3 - US$ 32.00
DHT22 Temperature and Relative Humidity Sensor - USD 9.95
Resistor 4K7 ohm
DS18B20 Waterproof Temperature Sensor - USD 5.95
Resistor 4K7 ohm
BMP180 Barometric Pressure, Temperature and Altitude Sensor - USD 6.99

Step 2: Installing the Temperature & Humidity Sensor

The first sensor to be installed will be the DHT22 for capturing air temperature and relative humidity data. The ADAFRUIT site provides great information about those sensors. Bellow, some information retrieved from there:

Overview

The low-cost DHT temperature & humidity sensors are very basic and slow but are great for hobbyists who want to do some basic data logging. The DHT sensors are made of two parts, a capacitive humidity sensor, and a thermistor. There is also a very basic chip inside that does some analog to digital conversion and spits out a digital signal with the temperature and humidity. The digital signal is fairly easy to be read using any microcontroller.

DHT22 Main characteristics:

Low cost
3 to 5V power and I/O
2.5mA max current use during conversion (while requesting data)
Good for 0-100% humidity readings with 2-5% accuracy
Good for -40 to 125°C temperature readings ±0.5°C accuracy
No more than 0.5 Hz sampling rate (once every 2 seconds)
Body size 15.1mm x 25mm x 7.7mm
4 pins with 0.1" spacing

Once usually you will use the sensor on distances less than 20m, a 4K7 ohm resistor should be connected between Data and VCC pins. The DHT22 output data pin will be connected to Raspberry GPIO 16. Check the above electrical diagram, connecting the sensor to RPi pins as below:

Pin 1 - Vcc ==> 3.3V
Pin 2 - Data ==> GPIO 16
Pin 3 - Not Connect
Pin 4 - Gnd ==> Gnd

Do not forget to Install the 4K7 ohm resistor between Vcc and Data pins

Once the sensor is connected, we must also install its library on our RPi.

Installing DHT Library:

On your Raspberry, starting on /home, go to /Documents

cd Documents

Create a directory to install the library and move to there:

mkdir DHT22_Sensor
cd DHT22_Sensor

On your browser, go to Adafruit GitHub:

https://github.com/adafruit/Adafruit_Python_DHT

Download the library by clicking the download zip link to the right and unzip the archive on your Raspberry Pi recently created folder. Then go to the directory of the library (subfolder that is automatically created when you unzipped the file), and execute the command:

sudo python3 setup.py install

Open a test program (DHT22_test.py) from my GITHUB

import Adafruit_DHT
DHT22Sensor = Adafruit_DHT.DHT22
DHTpin = 16
humidity, temperature = Adafruit_DHT.read_retry(DHT22Sensor, DHTpin)

if humidity is not None and temperature is not None:
    print('Temp={0:0.1f}*C  Humidity={1:0.1f}%'.format(temperature, humidity))
else:
    print('Failed to get reading. Try again!')

Execute the program with the command:

python3 DHT22_test.py

The below Terminal print screen shows the result.

Step 3: Installing DS18B20 - Temperature Sensor

Sensor Overview:

We will use in this tutorial a waterproofed version of the DS18B20 sensor. It is very useful for remote temperature in wet conditions, for example on a humid soil. The sensor is isolated and can take measurements until 125oC (Adafrut does not recommend to use it over 100oC due to its cable PVC jacket).

The DS18B20 is a digital sensor what makes it good to use even over long distances! These 1-wire digital temperature sensors are fairly precise (±0.5°C over much of the range) and can give up to 12 bits of precision from the onboard digital-to-analog converter. They work great with the NodeMCU using a single digital pin, and you can even connect multiple ones to the same pin, each one has a unique 64-bit ID burned in at the factory to differentiate them.

The sensor works from 3.0 to 5.0V, what means that it can be powered directly from the 3.3V provide by one of the Raspberry pins (1 or 17).

The sensor has 3 wires:

Black: GND
Red: VCC
Yellow: 1-Wire Data

Here, you can find the full data: DS18B20 Datasheet

Sensor Installation:

Follow the above diagram and make the connections:

Vcc ==> 3.3V
Gnd ==> Gnd
Data ==> GPIO 4 (default for library)

Installing the Python Library:

Next, let's install the Python library that will handle the sensor:

sudo pip3 install w1thermsensor

Before running the script to test the sensor, check if the "1-Wire" interface is enabled in your RPi (see above print screen)

Do not forget to restart your RPi, after changeing its configuration

Testing the sensor:

For testing the sensor a simple python script can be used:

import time
from w1thermsensor import W1ThermSensor  
ds18b20Sensor = W1ThermSensor()

while True:		
    temperature = ds18b20Sensor.get_temperature()
    print("The temperature is %s celsius" % temperature)
    time.sleep(1)

Step 4: Installing the BMP180

Sensor Overview:

The BMP180 is the successor of the BMP085, a new generation of high precision digital pressure sensors for consumer applications. The ultra-low power, low voltage electronics of the BMP180 is optimized for use in mobile phones, PDAs, GPS navigation devices and outdoor equipment. With a low altitude noise of merely 0.25m at fast conversion time, the BMP180 offers superior performance. The I2C interface allows for easy system integration with a microcontroller. The BMP180 is based on piezo-resistive technology for EMC robustness, high accuracy, and linearity as well as long-term stability.

The complete BMP datasheet can be found here: BMP180 - Digital Pressure Sensor

Sensor Installation:
Follow the above diagram and make the connections:

Vin ==> 3.3V
GND ==> GND
SCL ==> GPIO 3
SDA ==> GPIO 2

Enabling I2C Interface

Go to RPi Configuration and confirm that I2C interface is enabled. If not, enable it and restart the RPi.

Using the BMP180

If everything has been installed okay, and everything has been connected okay, you are now ready to turn on your Pi and start seeing what the BMP180 is telling you about the world around you.

The first thing to do is to check if the Pi sees your BMP180. Try the following in a terminal window:

sudo i2cdetect -y 1

If the command worked, you should see something similar the above Terminal Printscreen, showing that the BMP180 is on channel '77'.

Installing the BMP180 Library:

Create a directory to install the library:

<p>mkdir BMP180_Sensor<br>cd BMP180_Sensor</p>

On your browser, go to Adafruit GITHub:

https://github.com/adafruit/Adafruit_Python_BMP

Download the library by clicking the download zip link to the right and unzip the archive on your Raspberry Pi created folder. Then go to the subfolder created and execute the following command in the directory of the library:

<p>sudo python3 setup.py install</p>

Open your Python IDE and create a test program and name it, for example BMP180Test.py

<p>import Adafruit_BMP.BMP085 as BMP085<br>sensor = BMP085.BMP085()
print('Temp = {0:0.2f} *C'.format(sensor.read_temperature()))
print('Pressure = {0:0.2f} Pa'.format(sensor.read_pressure()))
print('Altitude = {0:0.2f} m'.format(sensor.read_altitude()))
print('Sealevel Pressure = {0:0.2f} Pa'.format(sensor.read_sealevel_pressure()))</p>

Execute the test program:

<p>python3 BMP180Test.py</p>

The above Terminal print screen shows the result.

Note that that pressure is presented in Pa (Pascals). See next step to better unserstand about this unit.

Step 5: Measuring Weather and Altitude With BMP180

Let's take a time to understand a little bit more about what we will get, with the BMP readings. You can skip this part of the tutorial, or return later.

If you want to know more about Sensor readings, please go to this great tutorial:
https://learn.sparkfun.com/tutorials/bmp180-barome...

The BMP180 was designed to accurately measure atmospheric pressure. Atmospheric pressure varies with both weather and altitude.

What is Atmospheric Pressure?

The definition of atmospheric pressure is a force that the air around you is exerting on everything. The weight of the gasses in the atmosphere creates atmospheric pressure. A common unit of pressure is "pounds per square inch" or psi. We will use here the international notation, that is newtons per square meter, which are called pascals (Pa).

If you took 1 cm wide column of air would weigh about 1 kg

This weight, pressing down on the footprint of that column, creates the atmospheric pressure that we can measure with sensors like the BMP180. Because that cm-wide column of air weighs about 1Kg, it follows that the average sea level pressure is about 101325 pascals, or better, 1013.25 hPa (1 hPa is also known as milibar - mbar). This will drop about 4% for every 300 meters you ascend. The higher you get, the less pressure you’ll see, because the column to the top of the atmosphere is that much shorter and therefore weighs less. This is useful to know, because by measuring the pressure and doing some math, you can determine your altitude.

The air pressure at 3,810 meters is only half of that at sea level.

The BMP180 outputs absolute pressure in pascals (Pa). One pascal is a very small amount of pressure, approximately the amount that a sheet of paper will exert resting on a table. You will more often see measurements in hectopascals (1 hPa = 100 Pa). The library used here provides outputs floating-point values in hPa, which also happens to equal one millibar (mbar).

Here are some conversions to other pressure units:

1 hPa = 100 Pa = 1 mbar = 0.001 bar
1 hPa = 0.75006168 Torr
1 hPa = 0.01450377 psi (pounds per square inch)
1 hPa = 0.02953337 inHg (inches of mercury)
1 hpa = 0.00098692 atm (standard atmospheres)

Temperature Effects

Because temperature affects the density of a gas, and density affects the mass of a gas, and mass affects the pressure (whew), atmospheric pressure will change dramatically with temperature. Pilots know this as “density altitude”, which makes it easier to take off on a cold day than a hot one because the air is denser and has a greater aerodynamic effect. To compensate for temperature, the BMP180 includes a rather good temperature sensor as well as a pressure sensor.

To perform a pressure reading, you first take a temperature reading, then combine that with a raw pressure reading to come up with a final temperature-compensated pressure measurement. (The library makes all of this very easy.)

Measuring Absolute Pressure

If your application requires measuring absolute pressure, all you have to do is get a temperature reading, then perform a pressure reading (see the example sketch for details). The final pressure reading will be in hPa = mbar. If you wish, you can convert this to a different unit using the above conversion factors.

Note that the absolute pressure of the atmosphere will vary with both your altitude and the current weather patterns, both of which are useful things to measure.

Weather Observations

The atmospheric pressure at any given location on earth (or anywhere with an atmosphere) isn’t constant. The complex interaction between the earth’s spin, axis tilt, and many other factors result in moving areas of higher and lower pressure, which in turn cause the variations in weather we see every day. By watching for changes in pressure, you can predict short-term changes in the weather. For example, dropping pressure usually means wet weather or a storm is approaching (a low-pressure system is moving in). Rising pressure usually means that clear weather is approaching (a high-pressure system is moving through). But remember that atmospheric pressure also varies with altitude. The absolute pressure in my house, Lo Barnechea in Chile (altitude 950m) will always be lower than the absolute pressure in San Francisco for example (less than 2 meters, almost sea level). If weather stations just reported their absolute pressure, it would be difficult to directly compare pressure measurements from one location to another (and large-scale weather predictions depend on measurements from as many stations as possible).

To solve this problem, weather stations always remove the effects of altitude from their reported pressure readings by mathematically adding the equivalent fixed pressure to make it appear as if the reading was taken at sea level. When you do this, a higher reading in San Francisco than Lo Barnechea will always be because of weather patterns, and not because of altitude.

To do this, there is a function in the library called sea level(P,A). This takes the absolute pressure (P) in hPa, and the station’s current altitude (A) in meters, and removes the effects of the altitude from the pressure. You can use the output of this function to directly compare your weather readings to other stations around the world.

Determining Altitude

Since pressure varies with altitude, you can use a pressure sensor to measure altitude (with a few caveats). The average pressure of the atmosphere at sea level is 1013.25 hPa (or mbar). This drops off to zero as you climb towards the vacuum of space. Because the curve of this drop-off is well understood, you can compute the altitude difference between two pressure measurements (p and p0) by using a specific equation.

If you use sea level pressure (1013.25 hPa) as the baseline pressure (p0), the output of the equation will be your current altitude above sea level. There’s a function in the library called altitude(P,P0) that lets you get the "calculated altitude".

The above explanation was extracted from BMP 180 Sparkfun tutorial.

Step 6: The Complete HW

Step 7: Sending Data to ThingSpeak

At this point, we learned how to prepare the RPi to capture data from all 3 sensors, printing them on the terminal. Now, it is time to see how to send those data to out IoT platform, the ThingSpeak.

Let's begin!

First, you must have an account at ThinkSpeak.com

Follow the instructions to create a Channel and take note of your Channel ID and Write API Key

Download the Python Script from my GitHub: localData ToTS_v1_EXT.py

Let's comment the code most important parts:

First, let's import the ThingSpeak library, define the WiFi client and define your local Router and Thinkspeak credentials:

import thingspeak

There are several ways to communicate with ThingSpeak, the simplest way would be using the client library for the thingspeak.com API developed by Mikolaj Chwaliz and Keith Ellis.

The library can be download from https://github.com/mchwalisz/thingspeak or be using PIP on terminal:

sudo pip3 install thingspeak

Next, inside the script, update ThingSpeak channel credentials

chId = 9999999 # Enter with your Channel Id
tsKey='ENTER WITH YOUR CHANNEL WRITE KEY'
tsUrl='https://api.thingspeak.com/update'
ts = thingspeak.Channel(chId, tsUrl ,tsKey)

Now, let's Initialize the 3 sensors:

# DS18B20 1-Wire library
from w1thermsensor import W1ThermSensor 
ds18b20Sensor = W1ThermSensor() # By default GPIO 4 is used by library

# DHT22 Library 
import Adafruit_DHT
DHT22Sensor = Adafruit_DHT.DHT22
DHTpin = 16

# BMP180 library
import Adafruit_BMP.BMP085 as BMP085 
bmp180Sensor = BMP085.BMP085()

You should define the real altitude where your Weather Station is located, updating the Global variable "altReal". In my case, my Station is located at 950m above the sea level.

global altReal
altReal = 950

Once enter with the station real altitude as input, we can obtain the absolute pressure, sea level pressure, temperature, and altitude using the function bmp180GetData(altitude):

def bmp180GetData(altitude):	
	temp = bmp180Sensor.read_temperature()
	pres = bmp180Sensor.read_pressure()
	alt =  bmp180Sensor.read_altitude()
	presSeaLevel = pres / pow(1.0 - altitude/44330.0, 5.255) 
	
	temp = round (temp, 1)
	pres = round (pres/100, 2) # absolute pressure in hPa (or mbar)
	alt = round (alt)
	presSeaLevel = round (presSeaLevel/100, 2) # absolute pressure in hPa (or mbar)

	return temp, pres, alt, presSeaLevel

The function getLocalData(), will return all local data that is captured by our station:

def getLocalData():
	global timeString
	global humLab
	global tempExt
	global tempLab
	global presSL
	global altLab
	global presAbs
	
	# Get time of reading
	now = datetime.datetime.now()
	timeString = now.strftime("%Y-%m-%d %H:%M")
	
	# Read Outside Temperature (1 meter distance)
	tempExt = round(ds18b20Sensor.get_temperature(), 1)
	
	tempLab, presAbs, altLab, presSL = bmp180GetData(altReal) 	
	humDHT, tempDHT = Adafruit_DHT.read_retry(DHT22Sensor, DHTpin)
	if humDHT is not None and tempDHT is not None:
		humLab = round (humDHT

Once you have all data captured by above functions, you must send them to ThingSpeak. You will do it using the function sendDataTs():

def sendDataTs():
	data = {
		"field1": tempLab, 
		"field2": tempExt, 
		"field3": humLab, 
		"field4": presSL, 
		"field5": altLab
		}
	ts.update(data)
	print ("[INFO] Data sent for 5 fields: ", tempLab, tempExt, humLab, presSL, altLab)

With your channel data updated, save the script and execute it on your terminal:

sudo Python3 localData_ToTs_v1_EXT.py

About Communication protocols

Note that using the "thingspeak library", the "requests library" is imported, that is an Apache2 Licensed HTTP library, written in Python. The official Request Installation documentation can be found here:

http://docs.python-requests.org/en/latest/user/install/

If necessary, before running your script, you can verify if requests library is installed:

sudo pip3 install requests

Optionally you can use MTTQ as a method to send data to ThingSpeak. MQTT is different from HTTP, once It is specifically designed to be lightweight and intended for embedded devices with low RAM and CPU performance. Also, in most cases, MQTT uses less bandwidth.

Refer to this tutorial: Update a ThingSpeak Channel using MQTT on a Raspberry Pi for more details.

Step 8: Sending Remote Data to ThingSpeak Using ESP8266

For this step, we will use the same HW that was explained in muy tutorial:

IOT MADE EASY: CAPTURING REMOTE WEATHER DATA: UV AND AIR TEMPERATURE & HUMIDITY

The code that we will use here, is basically the same used on that tutorial. Let's comment the code most important parts:

First, let's call the ESP8266 library, define the WiFi client and define your local Router and Thinkspeak credentials:

/* NodeMCU ESP12-E */
#include <ESP8266WiFi.h>
WiFiClient client;
const char* MY_SSID = "ENTER WITH YOUR SSDID";
const char* MY_PWD = "ENTER WITH YOUR PASSWORD";

/* Thinkspeak */
const char* TS_SERVER = "api.thingspeak.com";
String TS_API_KEY ="ENTER WITH YOUR WRITE KEY";

Second, let's include a very important library for IoT projects: SimpleTimer.h:

/* TIMER */
#include <SimpleTimer.h>
SimpleTimer timer;

Third, during setup(), we will initiate serial communication, call the function connectWiFi() and define the timers. Note that the line of code: timer.setInterval(60000L, sendDataTS); will call the function sendDataTS() every 60 seconds, in order to upload data to ThinkSpeak channel.

void setup() 
{
  ...
  Serial.begin(115200);
  delay(10);
  ...
  connectWifi();
  timer.setInterval(60000L, sendDataTS);
  ...
}

At last but not least, during the loop(), the only command needed is to initiate the timer and that's it!

void loop() 
{
  ...
  timer.run(); // Initiates SimpleTimer
}

Below, you can see the two important functions used to handle Thinkspeak communication:

ESP12-E connection with your WiFi network:

/***************************************************
 * Connecting WiFi
 **************************************************/
void connectWifi()
{
  Serial.print("Connecting to "+ *MY_SSID);
  WiFi.begin(MY_SSID, MY_PWD);
  while (WiFi.status() != WL_CONNECTED) 
  {
    delay(1000);
    Serial.print(".");
  }
  Serial.println("");
  Serial.println("WiFi Connected");
  Serial.println("");  
}

ESP12-E sending data to ThinkSpeak:

***************************************************
 * Sending Data to Thinkspeak Channel
 **************************************************/
void sendDataTS(void)
{
   if (client.connect(TS_SERVER, 80)) 
   { 
     String postStr = TS_API_KEY;
     postStr += "&field6=";
     postStr += String(temp);
     postStr += "&field7=";
     postStr += String(hum);
     postStr += "&field8=";
     postStr += String(dataSensorUV);
     postStr += "\r\n\r\n";
   
     client.print("POST /update HTTP/1.1\n");
     client.print("Host: api.thingspeak.com\n");
     client.print("Connection: close\n");
     client.print("X-THINGSPEAKAPIKEY: " + TS_API_KEY + "\n");
     client.print("Content-Type: application/x-www-form-urlencoded\n");
     client.print("Content-Length: ");
     client.print(postStr.length());
     client.print("\n\n");
     client.print(postStr);
     delay(1000); 
   }
   sent++;
   client.stop();
}

The complete code can be found on my GitHub: NodeMCU_UV_DHT_Sensor_OLED_TS_EXT

Once you have the code uploaded to your NodeMCU. Let's connect an external battery and do some measurement under the sun. I put the Remote Station on the roof and start capturing data on ThingSpeak.com as shown in above photos.

Step 9: Final Notes

The main purpose of this tutorial was to show how to connect the Raspberry Pi to ThingSpeak. This is great to capture data and log them on an IoT platform.

Using the opportunity we also sent data to that specific channel, capturing them from a remote station using an ESP8266. This is approach is OK, but not the best one. Because we have an "asynchronous" operation, sometimes, both, RPi and ESP8266 try to log at the same time (or with a small interval) what is refuted by ThingSpeak. The ideal would have the ESP8266 sending data locally to Raspberry Pi and the last one being responsible to handle all data. Doing that, the "Main Station" (Raspberry Pi), could do 3 things: