ESP8266: Parsing JSON

Introduction: ESP8266: Parsing JSON

About: I'm busy building machines.

As promised in my previous instructable, I will be covering more about the ArduinoJson library in detail, in this instructable. JSON (JavaScript Object Notation) is a lightweight data-interchange format that is easy for humans to read and write, and easy for machines to parse and generate. JSON objects are written in key/value pairs and it's a must for keys to be of the string data type while values can be a string, number, object, array, boolean or null. A vast majority of APIs that are now being used will return JSON data when called, and knowing how to parse them will definitely benefit you.

In this instructable, we will be using the ArduinoJson library for the ESP8266 to help us parse JSON data and extract values based on keys. The ArduinoJson library is also capable of serializing JSON, meaning you could generate your own JSON data using data from sensors connected to your ESP8266 or Arduino for example (will be covering more about JSON serialization, in detail, in another instructable). So, let's get started.

Teacher Notes

Teachers! Did you use this instructable in your classroom?
Add a Teacher Note to share how you incorporated it into your lesson.

Step 1: Install the ArduinoJson Library

Before you can use the ArduinoJson library, you have to make sure the library is installed on your computer. To do a quick check, head over to the library manager in the Arduino IDE (Sketch -> Include Library -> Manage Libraries...) and type "ArduinoJson" in the text box, if you see a green coloured "INSTALLED" label beside the name of the library, that means you are all good to go and you can proceed on to the next step. If you don't see the label, click on the box/division once and you will see the "Install" button. Hit install and you are all set for the next step.

Step 2: Performing a GET Request

Before we can start parsing, we need to have the JSON data in the first place and to obtain our data, we perform a GET request. A GET request, as the name suggests, gets the data for us from a particular location using a specific URL. The boilerplate code to perform the GET request can be found below. For this example, we will be performing a GET request using the http://jsonplaceholder.typicode.com/users/1 URL. You can call any API you like.

#include <ESP8266WiFi.h><br>#include <ESP8266HTTPClient.h>
#include <ArduinoJson.h>

// WiFi Parameters
const char* ssid = "YOUR_SSID";
const char* password = "YOUR_PASSWORD";

void setup() {
  Serial.begin(115200);
  WiFi.begin(ssid, password);
 
  while (WiFi.status() != WL_CONNECTED) {
    delay(1000);
    Serial.println("Connecting...");
  }
}

void loop() {
  // Check WiFi Status
  if (WiFi.status() == WL_CONNECTED) {
    HTTPClient http;  //Object of class HTTPClient
    http.begin("http://jsonplaceholder.typicode.com/users/1");
    int httpCode = http.GET();
    //Check the returning code                                                                  
    if (httpCode > 0) {
      // Get the request response payload
      String payload = http.getString();
      // TODO: Parsing
    }
    http.end();   //Close connection
  }
  // Delay
  delay(60000);
}

The data that we are going to parse is contained in the payload variable. We don't actually need this variable when we are parsing our data later on.

Step 3: Using the ArduinoJson Assistant

The developers who developed the ArduinoJson library are so kind that they've even created an Assistant that writes the parser program for us using any JSON data as an input. To use the ArduinoJson assistant, you first need to know how your JSON is formatted and to do that, type in the URL that we used to perform the GET request earlier on into the browser of your choice and hit enter. Copy the JSON and head over to the ArduinoJson Assistant's web page and paste it into the text box below the label named "Input". Then scroll down to take a look at the parsing program generated by the Assistant. Copy the whole program or just a section of it.

Step 4: Completing the Code and the Result

Copying and pasting the parsing program generated by the Assistant into the boilerplate code that we used to perform a GET request earlier on would look like this:

#include <ESP8266WiFi.h>
#include <ESP8266HTTPClient.h>
#include <ArduinoJson.h>

// WiFi Parameters
const char* ssid = "YOUR_SSID";
const char* password = "YOUR_PASSWORD";

void setup() {
  Serial.begin(115200);
  WiFi.begin(ssid, password);
 
  while (WiFi.status() != WL_CONNECTED) {
    delay(1000);
    Serial.println("Connecting...");
  }
}

void loop() {
  // Check WiFi Status
  if (WiFi.status() == WL_CONNECTED) {
    HTTPClient http;  //Object of class HTTPClient
    http.begin("http://jsonplaceholder.typicode.com/users/1");
    int httpCode = http.GET();
    //Check the returning code                                                                  
    if (httpCode > 0) {
      // Parsing
      const size_t bufferSize = JSON_OBJECT_SIZE(2) + JSON_OBJECT_SIZE(3) + JSON_OBJECT_SIZE(5) + JSON_OBJECT_SIZE(8) + 370;
      DynamicJsonBuffer jsonBuffer(bufferSize);
      JsonObject& root = jsonBuffer.parseObject(http.getString());
      // Parameters
      int id = root["id"]; // 1
      const char* name = root["name"]; // "Leanne Graham"
      const char* username = root["username"]; // "Bret"
      const char* email = root["email"]; // "Sincere@april.biz"
      // Output to serial monitor
      Serial.print("Name:");
      Serial.println(name);
      Serial.print("Username:");
      Serial.println(username);
      Serial.print("Email:"); 
      Serial.println(email);
    }
    http.end();   //Close connection
  }
  // Delay
  delay(60000);
}

Since we are only interested in the name, email and username of the user, we just used a section of the parsing program generated by the assistant. You can use the serial monitor to view the output. If you don't see anything, press the reset button on your ESP8266 and *boom* you should see the output there. Note: The last line of code in the code above introduces a delay of 1 minute or 60,000 ms into the loop. This means that the API is only called once every minute. The number of times an API can be called within a specified timeframe varies and you are strongly encouraged to follow the guidelines specified by your API provider.

1 Person Made This Project!

Recommendations

  • Tiny Speed Challenge

    Tiny Speed Challenge
  • Clocks Contest

    Clocks Contest
  • PCB Design Challenge

    PCB Design Challenge

20 Discussions

2
snakeboy69
snakeboy69

11 months ago

No longer compiles :(

sketch_apr18a:159:11: error: DynamicJsonBuffer is a class from ArduinoJson 5. Please see arduinojson.org/upgrade to learn how to upgrade your program to ArduinoJson version 6

DynamicJsonBuffer jsonBuffer(bufferSize);

^

exit status 1
DynamicJsonBuffer is a class from ArduinoJson 5. Please see arduinojson.org/upgrade to learn how to upgrade your program to ArduinoJson version 6

0
subzero69
subzero69

8 months ago

No longer compiles :(

sketch_apr18a:159:11: error: DynamicJsonBuffer is a class from ArduinoJson 5. Please see arduinojson.org/upgrade to learn how to upgrade your program to ArduinoJson version 6

DynamicJsonBuffer jsonBuffer(bufferSize);

^

0
Willem2017
Willem2017

Question 22 days ago

Hello,

Its a very nice tutorial!
The example above (https://www.instructables.com/id/ESP8266-Parsing-JSON/) works fine but with my code below I have the same problem as described by deshmukhprat (under my question)!
It sounds crazy but yesterday it worked and today only empty fields are shown.
I have added a line of code to see the output (=String payload) from the website and the payload data is as expected.
I parse only a part of this data and as you can see: the parsed info is empty, please see the OUTPUT.
Please see also the CODE below.
It is based on ArduinoJson version 6.14.1.
Platform ESP8266 WeMos D1 R1 with Arduino version 1.8.12.
Thanks in advance for any tips or solution!

====================OUTPUT======================
Connecting...
Connecting...
Connecting...
payload: { "liveweer": [{"plaats": "Nieuwenhagen", "temp": "4.5", "gtemp": "1.2", "samenv": "Licht bewolkt", "lv": "78", "windr": "ZZW", "windms": "4", "winds": "3", "windk": "7.8", "windkmh": "14.4", "luchtd": "1006.8", "ldmmhg": "755", "dauwp": "1", "zicht": "45", "verw": "Vanavond buien, in het oosten eerst met onweer en hagel", "sup": "07:13", "sunder": "18:24", "image": "wolkennacht", "d0weer": "halfbewolkt", "d0tmax": "10", "d0tmin": "2", "d0windk": "3", "d0windknp": "8", "d0windms": "4", "d0windkmh": "15", "d0windr": "NO", "d0neerslag": "33", "d0zon": "30", "d1weer": "halfbewolkt", "d1tmax": "9", "d1tmin": "0", "d1windk": "2", "d1windknp": "6", "d1windms": "3", "d1windkmh": "11", "d1windr": "ZW", "d1neerslag": "50", "d1zon": "30", "d2weer": "regen", "d2tmax": "6", "d2tmin": "1", "d2windk": "2", "d2windknp": "6", "d2windms": "3", "d2windkmh": "11", "d2windr": "O", "d2neerslag": "90", "d2zon": "10", "alarm": "0"}]}

Plaats:
Temperatuur:
Samenvatting:
Luchtvochtigheid:
Verwachting:

==============MY CODE================
// Van https://www.instructables.com/id/ESP8266-Parsing-...
// en https://arduinojson.org/v6/assistant/ Voor de Parser code.
//
#include // voor handige seriële output
#include
#include
#include

const size_t capacity = JSON_ARRAY_SIZE(1) + JSON_OBJECT_SIZE(1) + JSON_OBJECT_SIZE(50) + 770;
DynamicJsonDocument doc(capacity);
const char* json = "{\"liveweer\":[{\"plaats\":\"Nieuwenhagen\",\"temp\":\"6.6\",\"gtemp\":\"5.2\",\"samenv\":\"Lichte regen\",\"lv\":\"98\",\"windr\":\"ZZW\",\"windms\":\"2\",\"winds\":\"2\",\"windk\":\"3.9\",\"windkmh\":\"7.2\",\"luchtd\":\" 990.5\",\"ldmmhg\":\"743\",\"dauwp\":\"6\",\"zicht\":\"13\",\"verw\":\"Bewolkt en regenachtig, vanavond geleidelijk droog. Dinsdag enkele buien\",\"sup\":\"07:15\",\"sunder\":\"18:22\",\"image\":\"buien\",\"d0weer\":\"halfbewolkt\",\"d0tmax\":\"9\",\"d0tmin\":\"1\",\"d0windk\":\"3\",\"d0windknp\":\"10\",\"d0windms\":\"5\",\"d0windkmh\":\"19\",\"d0windr\":\"NO\",\"d0neerslag\":\"33\",\"d0zon\":\"44\",\"d1weer\":\"halfbewolkt\",\"d1tmax\":\"8\",\"d1tmin\":\"1\",\"d1windk\":\"3\",\"d1windknp\":\"8\",\"d1windms\":\"4\",\"d1windkmh\":\"15\",\"d1windr\":\"ZW\",\"d1neerslag\":\"50\",\"d1zon\":\"50\",\"d2weer\":\"bewolkt\",\"d2tmax\":\"7\",\"d2tmin\":\"1\",\"d2windk\":\"2\",\"d2windknp\":\"6\",\"d2windms\":\"3\",\"d2windkmh\":\"11\",\"d2windr\":\"W\",\"d2neerslag\":\"30\",\"d2zon\":\"30\",\"alarm\":\"1\",\"alarmtxt\":\"Er zijn momenteel geen waarschuwingen van kracht.\"}]}";
JsonObject liveweer_0 = doc["liveweer"][0];
const char* plaats = liveweer_0["plaats"]; // "Nieuwenhagen"
const char* temp = liveweer_0["temp"]; // "6.6"
const char* gtemp = liveweer_0["gtemp"]; // "5.2"
const char* samenv = liveweer_0["samenv"]; // "Lichte regen"
const char* lv = liveweer_0["lv"]; // "98"
const char* windr = liveweer_0["windr"]; // "ZZW"
const char* windms = liveweer_0["windms"]; // "2"
const char* winds = liveweer_0["winds"]; // "2"
const char* windk = liveweer_0["windk"]; // "3.9"
const char* windkmh = liveweer_0["windkmh"]; // "7.2"
const char* luchtd = liveweer_0["luchtd"]; // " 990.5"
const char* ldmmhg = liveweer_0["ldmmhg"]; // "743"
const char* dauwp = liveweer_0["dauwp"]; // "6"
const char* zicht = liveweer_0["zicht"]; // "13"
const char* verw = liveweer_0["verw"]; // "Bewolkt en regenachtig, vanavond geleidelijk droog. Dinsdag enkele buien"
const char* sup = liveweer_0["sup"]; // "07:15"
const char* sunder = liveweer_0["sunder"]; // "18:22"
const char* image = liveweer_0["image"]; // "buien"
const char* d0weer = liveweer_0["d0weer"]; // "halfbewolkt"
const char* d0tmax = liveweer_0["d0tmax"]; // "9"
const char* d0tmin = liveweer_0["d0tmin"]; // "1"
const char* d0windk = liveweer_0["d0windk"]; // "3"
const char* d0windknp = liveweer_0["d0windknp"]; // "10"
const char* d0windms = liveweer_0["d0windms"]; // "5"
const char* d0windkmh = liveweer_0["d0windkmh"]; // "19"
const char* d0windr = liveweer_0["d0windr"]; // "NO"
const char* d0neerslag = liveweer_0["d0neerslag"]; // "33"
const char* d0zon = liveweer_0["d0zon"]; // "44"
const char* d1weer = liveweer_0["d1weer"]; // "halfbewolkt"
const char* d1tmax = liveweer_0["d1tmax"]; // "8"
const char* d1tmin = liveweer_0["d1tmin"]; // "1"
const char* d1windk = liveweer_0["d1windk"]; // "3"
const char* d1windknp = liveweer_0["d1windknp"]; // "8"
const char* d1windms = liveweer_0["d1windms"]; // "4"
const char* d1windkmh = liveweer_0["d1windkmh"]; // "15"
const char* d1windr = liveweer_0["d1windr"]; // "ZW"
const char* d1neerslag = liveweer_0["d1neerslag"]; // "50"
const char* d1zon = liveweer_0["d1zon"]; // "50"
const char* d2weer = liveweer_0["d2weer"]; // "bewolkt"
const char* d2tmax = liveweer_0["d2tmax"]; // "7"
const char* d2tmin = liveweer_0["d2tmin"]; // "1"
const char* d2windk = liveweer_0["d2windk"]; // "2"
const char* d2windknp = liveweer_0["d2windknp"]; // "6"
const char* d2windms = liveweer_0["d2windms"]; // "3"
const char* d2windkmh = liveweer_0["d2windkmh"]; // "11"
const char* d2windr = liveweer_0["d2windr"]; // "W"
const char* d2neerslag = liveweer_0["d2neerslag"]; // "30"
const char* d2zon = liveweer_0["d2zon"]; // "30"
const char* alarm = liveweer_0["alarm"]; // "1"
const char* alarmtxt = liveweer_0["alarmtxt"]; // "Er zijn momenteel geen waarschuwingen van kracht."

// WiFi Parameters
const char* ssid = "------";
const char* password = "------";
void setup() {
Serial.begin(115200);
WiFi.begin(ssid, password);

while (WiFi.status() != WL_CONNECTED) {
delay(1000);
Serial.println("Connecting...");
}
}
void loop() {
if (WiFi.status() == WL_CONNECTED) { // Check WiFi Status
HTTPClient http; // Object of class HTTPClient
// I deleted my key = ********
// On the Dutch site https://weerlive.nl/delen.php you can get your own key.
http.begin("http://weerlive.nl/api/json-data-10min.php?key=********&locatie=nieuwenhagen");
int httpCode = http.GET();

if (httpCode > 0) { // Check the returning code
String payload = http.getString(); // Get the request response payload

Serial << " payload: " << payload << endl; // TEST.........................

deserializeJson(doc, json);

Serial << "\n Plaats: " << plaats ;
Serial << "\n Temperatuur: " << temp;
Serial << "\n Samenvatting: " << samenv;
Serial << "\n Luchtvochtigheid: " << lv;
Serial << "\n Verwachting: " << verw;

}
http.end(); // Close connection
} // end if
delay(60000); // Weerlive max 300 verzoeken per dag!!
} // end loop()

0
bblanchon
bblanchon

Answer 19 days ago

Your program extracts the values, before calling `deserializeJson()`.
In other words, the `JsonDocument` is empty at the time you extract the values, and that's why all fields are empty.

0
deshmukhprat
deshmukhprat

5 weeks ago

Hello all,
I am getting blank results. Anyone have any clues? I have tried with same code and same website. Below is my output :

Name:
Username:
Email:

0
djalltra
djalltra

7 months ago

hi i am doing a project to read values from my php database the project consist of a main page with 6 on/off button i was able to update my database sucessfully but my problem is how to read each state from database with the nodemcu i used your http get example and i can see the states on the serial monitor how do i use these data to turn on and off led on the board thank you

1
bblanchon
bblanchon

2 years ago

Hi!

Nice work on this tutorial!

Remark that you can avoid the "payload" string by passing "http.getStream()" to "parseObject()".

[delete]

1
DavidR946
DavidR946

Reply 1 year ago

Would be possible adapt the code to use it with https?

0
Leomiracle
Leomiracle

Reply 1 year ago

I am looking fit https top!

0
SuhaimiA1
SuhaimiA1

1 year ago

thank you so much. simpler yet cleaner code than example from arduino.

0
2BECKHAM2
2BECKHAM2

1 year ago

Excellent tutorial Nikhil Raghava. Keep up the good work.

After following your tutorial, now I'm able to get and extract the content successfully from other sites. - Many thanks.

0
amygoodchild
amygoodchild

1 year ago

Thanks for this, super helpful.

I did have to change:
JsonObject& root = jsonBuffer.parseObject(http.getString());
to:
JsonObject& root = jsonBuffer.parseObject(http.getStream());

Not sure why but now it works great.

0
VothneaS
VothneaS

1 year ago

Thank you very much :)

0
RaíL1
RaíL1

Question 1 year ago

Thanks for the article!

Could you please provide us the wiring diagram to connect esp8266 with arduino??

0
UygarP
UygarP

1 year ago

Hey, thanks a lot for the article. I'm trying to apply this to http://www.reddit.com/r/Showerthoughts/top.json (I'm trying to get the titles of posts) and for some reason nothing returns (even though httpCode returns as 301). I pasted my code here: https://paste.ofcode.org/m7MZhnr7B6LW9g74qmQrXU I used ArduinoJson assistant and changed the relevant parts in your code. I tried the suggestions in the "Why Parsing Fails" page of arduinojson.org to no avail. Can it be a memory problem, given that Reddit's JSON is considerably larger than the one in your example? I'm a beginner and still trying to wrap my mind around all of this. Thanks in advance.

0
nikhilraghava
nikhilraghava

Reply 2 years ago

Hi, there seems to be a problem with the server's SSL certificate, it's either too large or they could have blocked off non-browser agents from accessing the API. I tried connecting to the service using HTTPS and the SHA1 fingerprints don't match every time I run it. The certificate size could be to blame. I will try again and will let you know if it works.