Introduction: IoT Charts Using Google Tools: Getting Started

Picture of IoT Charts Using Google Tools:  Getting Started

The internet connected sensor revolution is here

It's getting easier every day to connect your Arduino, Raspberry Pi,ESP8266, Edison, or Photon project to the internet and push sensor measurement data to a cloud database such as ThingSpeak or Sparkfun Data. But, once your data is stored there, how do you best showcase your results to the world?

Levering the free web tools from Google is one answer

This is the first of 2 tutorials that will teach how to chart and share sensor data using free Google tools. By the end of the 2nd tutorial you will know how to rapidly build a professional looking web site that showcases the data produced by your hardware projects

A second goal of these tutorials is to teach how various internet technologies come together to produce a chart visualization

Here is the Instructables roadmap

  • "Getting Started" (this instructable): Learn how to chart sensor data using a Demo Google Sitewith cloud data services ThingSpeak and Sparkfun Data
  • "Build an IoT Website" (coming soon): build your own personal Google website to display custom charts of your sensor data

    Here is an example website showing the potential of the Google tools. Our Hedgehog monitoring websiteis constructed using Google Sites, Google Charts, Google Sheets, and Google Maps. The Dashboard shows nightly and weekly statistics for our athletic pet

    Google Site for Hedgehog Monitoring

    For this tutorial you only need a web browser such as Chrome, Internet Explorer, or Firefox. Best usability with screen width resolutions 1280 pixels or better

    Step 1: ThingSpeak and Sparkfun Data

    Picture of ThingSpeak and Sparkfun Data

    ThingSpeak is a free cloud platform for storing sensor data on a remote server

    Sparkfun Datais another free cloud data service offered by the excellent hobbyist company Sparkfun

    Both of these services store sensor data covering all sorts of interesting applications, from nuclear radiation to beehive monitoring. Most users of these services configure their sensor data storage to be readable by the public. You can access and plot any of this public data. We will cover that in this tutorial

    ThingSpeak calls each bundle of sensor data a Channel while Sparkfun Datacalls it a Stream

    You might already have data on ThinkSpeak or Sparkfun? Great. You can try these Google tools to chart your data



    Step 2: Quick Start - Using the Demo Site With ThingSpeak

    Picture of Quick Start - Using the Demo Site With ThingSpeak

    Let's start by exploring the charting capabilities of Google tools using a demo web site that I designed for this Instructables

    1. Open the IoT Charts Demo website
    2. Open the THINGSPEAK page. The demo site is pre-programmed to show a line chart of temperature coming from a ThingSpeak data Channel of a weather station in Szczecin, Poland
    3. Change the Number of Points setting in the right side panel.Press Update Chart. The timeline changes
    4. Change "field1" to "field2" in the Field box. Press Update Chart. Humidity is plotted
    5. Change "field2" to "field3" in the Field box. Press Update Chart. Atmospheric Pressure is plotted
    6. Change the optional Min Value to set a minimum value on the Y scale. Press Update Chart. Notice the minimum value on the Y axis changes to your new value.Why would you want to do this? Setting a minimum value is particularly useful in situations where your hardware has accidentally generated zeros in the data stream -- I see this a lot. Leaving the Min Value box empty enables auto-scaling
    7. Change the optional Max Value to set a maximum value on the Y scale. Press Update Chart
    8. Enter some dates into Start Date and End Date. Delete the value in Number of Points. Press Update Chart. The timeline changes to the date range you specified

    Step 3: From Sensors to Chart Visualization

    Picture of From Sensors to Chart Visualization

    Let's take a step back for a moment to consider how various internet technologies are working together to produce a chart on the demo web site. The block diagram above and the attached PDF illustrate how sensor data is turned into a chart visualization on your screen

    Here is a step-by-step description of the data flow (numbers below can be found on the block diagram)

    1. sensors are read by the hardware, like Arduino or ESP8266
    2. the hardware pushes the sensor data to a ThingSpeak server via the internet
    3. sensor data is stored on a remote server operated by ThingSpeak
    4. website formatting is stored at a Google server
    5. using an internet browser, like Chrome or Internet Explorer, you click on the THINGSPEAK link on the IoT Charts Demowebsite
    6. sensor data flows from the ThingSpeak server to the browser
    7. web site formatting flows from a Google Sites server to the browser
    8. Google Charts software flows from a Google server to the browser
    9. browser combines 6, 7, and 8 to produce a chart of the sensor data

    Hopefully this helps to demystify the process of rendering a chart on your screen

    This tutorial series primarily focuses on steps 4 to 9. I added a References page with information on how to program the hardware to push sensor data to ThingSpeak and Sparkfun Data (steps 1-3)

    Step 4: Charting ThingSpeak Public Channels

    Picture of Charting ThingSpeak Public Channels

    Many users of ThingSpeak configure their Channels to allow public access. You can use the demo site to chart data from these Channels

    1. Select an interesting Channel from one of the ThingSpeak Public Channels. Some example channels are listed at the bottom of this page
    2. Open one of the Channels by clicking on the title
    3. Find the Channel ID (see photo above)
    4. Enter the Channel ID on the ThingSpeak charting page
    5. Enter a Field. All ThingSpeak sensor measurements are identified by "field1", "field2"... "field8"
    6. Press the Update Chart button. You will see a chart of the sensor data you selected

    Here are some ThingSpeak Channels to try

    Current weather condition for Szczecin, Poland. Temperature, humidity, and pressure are monitored

    Channel ID: 29084
    Fields: field1, field2, field3
    URL: https://thingspeak.com/channels/29084

    Hamstometer is a pedometer for hamsters which measures and tracks their wheel rotations per unit time

    Channel ID: 46669
    Fields: field1, field2, field3
    URL: https://thingspeak.com/channels/46669

    http://tinwhiskers.net/how-i-open-sourced-my-hamsters

    Radiation level in Montreal, Quebec, Canada

    Channel ID: 9892
    Fields: field5
    URL: https://thingspeak.com/channels/9892

    Step 5: Charting Sparkfun Data Public Channels

    Picture of Charting Sparkfun Data Public Channels

    Many Sparkfun Data streams are also configured for public access

    1. Pick a Stream from the listing of Sparkfun Public Streams. Some example streams are listed at the bottom of this page. Click on the Stream Name (see photo above)
    2. There are two things you need from the Stream page (see photo above)
      Public Key (unique identifier for a Stream, shown in the web address)
      Field
      (name of data you want to plot, shown in the column title)
    3. Enter the Public Key and Field into the edit boxes on the Sparkfun charting page of the demo site
    4. Press the Update Chart button. In a short time you will see a plot of the sensor data. You might need to be patient - sometimes the Sparkfun Data site is slow to return data
    5. The chart title can be customized by entering text into the Chart Title box

    Try plotting these Example Sparkfun Streams ...

    A home weather station in Elizabethtown KY USA:

    Public Key: KJ111xY17xsRaKyw27XA
    Fields: dailyrainin, tempf
    URL: https://data.sparkfun.com/streams/KJ111xY17xsRaKyw27XA

    Open Source Beehives Project:

    Public Key: OGw6o7Z7AWsWVEZlEjj6
    Fields: tempc_hive1, tempc_hive2
    URL: https://data.sparkfun.com/streams/OGw6o7Z7AWsWVEZlEjj6

    CO2 Sensor:

    Public Key: ZG0aKVyV3DsQGNY8maO9
    Fields: co2_ppm
    URL: https://data.sparkfun.com/streams/ZG0aKVyV3DsQGNY8maO9

    Step 6: Under the Hood

    Picture of Under the Hood

    This section is written for the technical ninja wanting to know more about the IoT Charts Demo site

    The demo site is a Google Site. The charting features are built using Google Charts and Google Gadgets. A Gadget consists of HTML and JavaScript code wrapped by some "boilerplate" XML code. The gadget is embedded into a web page on the Google Site. The gadget code used on the charting demo site can be downloaded from my GitHub gadget repository. The gadget is called gadget-iotcharts-line-v1-1.xml (version 1-1 created on Oct 20, 2016 to correct a problem with the web site hosting the spinner script)

    You can extend the capabilities of this gadget far beyond what is shown in this tutorial. There is a lot of potential for customization:

    • change chart types. Try a bar chart, a pie chart. The Chart Gallery shows the possibilities
    • change the height and width of the gadget
    • pull data from a different data server like the Nimbits public cloud
    • change colors
    • format for mobile screens
    • report error conditions to user
    • configure the gadget to use tabs for the chart options, rather than a side panel

    If you decide peek under the hood and extend the functionality I would appreciate any technical feedback on my implementation. Thanks !

    Acknowledgements: This tutorial was built using knowledge generously shared by many developers, discovered by hours of Googling and reading StackExchange forums. It is truly a great age, where technical information is shared so openly. In some ways this Instructables is a small way of giving back

    Step 7: Reference: Pushing Your Sensor Data to the Cloud

    Picture of Reference:  Pushing Your Sensor Data to the Cloud

    Here are some references that teach how to program the hardware platforms to push sensor data to a cloud database

    ThingSpeak

    Arduino

    Raspberry Pi

    ESP8266

    Particle Photon

    Intel Edison

    Sparkfun Data

    Arduino

    Raspberry Pi

    ESP8266

    Particle Photon

    Step 8: What's Coming

    Picture of What's Coming

    This tutorial is a first step to bigger and better things. In a soon-to-be-published Instructable I'll show how you can take full advantage of Google Sites. Using a Google Site Template I will show how a customized IoT web site can be built in less than 30 minutes. You will be able to mix sensor charts with other visual Google features, like configurable gadgets and maps

    The end result will be a sensor web site, the way you want it to look. And, no JavaScript or HTML coding will be needed

    Google gives us powerful capabilities to display and share our sensor data. High availability and free

    Comments

    clayton.cowgill (author)2016-07-30

    Hey Mike,

    I'm not sure if this was just a temporary thing or limited to my particular application, but using your iotcharts .xml and the instructable (and your follow-up Google Sites document) I never was able to see a graph from either your sample data or mine.

    After some fumbling around and dropping 'alerts' to see where the code was-- it appears that in my case it was never returning from Line 278: var spinner = new Spinner(spinnerOpts).spin(target);

    Commenting that line out allowed things to work as on the demo site. Just in case anyone else gets stuck on it...

    Thanks for the tutorial!

    RobO43 (author)clayton.cowgill2016-09-08

    I experienced the same thing. The spinner that shows when the plot is loading isn't available, but it does work well without it.

    MikeTeachman (author)RobO432016-10-21

    Hi,

    I just corrected the spinner issue. A new gadget-iotcharts-line-v1-1.xml

    version of the IoT gadget is available in my github repository. thank you for your patience.

    Hi,

    I just corrected the spinner issue. A new gadget-iotcharts-line-v1-1.xml

    version of the IoT gadget is available in my github repository. thank you for your patience.

    RobO43 (author)2016-09-08

    Mike, Great instructions and thanks for sharing your work. I've got my sparkfun data stream tied to your gadget on a google site page and it is plotting the sump water level in my basement. I want to add the ability to count the number of pump cycles per hour or sump cycles per week and estimate the number of gallons pumped out per day/week/month. I'm not sure how to "peak find" in the data table to pull out a pump event. Any ideas on where to go for that?

    MaureenB16 (author)2016-06-29

    Cant´t wait for the bigger tutorial!

    tnickolite (author)2016-04-10

    Mike,

    Thank you so much for providing this tutorial. I have been able to modify the line graph to fit my website. What would I have to change to create the bar graph that you have on your website?

    MikeTeachman (author)tnickolite2016-05-09

    Hi,

    I added a new github repository with the gadget code for the column chart. This should give you an idea on how the column chart works to display the last 7 days of hedgehog running.

    https://github.com/MikeTeachman/gadget-distanceLas...

    The getJSON method pulls data from a spreadsheet on a Google Drive account. The spreadsheet is used as the calculation back end for the site.

    Hope this helps...

    Mike

    tnickolite (author)2016-04-10

    Mike,

    Thank you so much for providing this tutorial. I have been able to modify the line graph to fit my website. What would I have to change to create the bar graph that you have on your website?

    kcajlliw (author)2016-02-22

    Mike,

    I must say your guide and gadget are awesome - you have made something I thought would take a lot of work or would be very expensive as easy as pie!

    I was just wondering if you had any idea how I could plot longitude against latitude on a similar plot - The background satellite map would be cool but not necessary. Could this also be integrated with google charts?

    Many thanks, Jack!

    MikeTeachman (author)kcajlliw2016-03-25

    Hi Jack,

    I experimented with using google map gadgets on our pet hedgehog site -- I created two gadgets that use the google maps API to plot the hedgehog's "run across north america". The maps update automatically and move the hedgehog icon across the map. I posted the gadget code on github.

    https://github.com/MikeTeachman/gadgets-googleMaps

    The gadgets are used on this Google sites page:

    https://sites.google.com/site/sircharlesruns/map

    How it works: the gadget code reads latitude and longitude from a google sheet on a google drive folder using the $.getJSON call. This google sheet only has two cells, one for latitude and the other for longitude -- the calculations for lat and long are done on a separate sheet. I found the parsing easier in the gadget code to have the lat and long isolated on a sheet. Here is a link to the google sheet:

    http://tinyurl.com/zvgfda6

    I hope this is enough description and information to reverse engineer what I built. I would be grateful for any feedback on this work so I can make improvements to the implementation.

    Mike

    ulrikk (author)2016-02-06

    Mike

    I tried ti use the gadget after your intructions but I get a errormessage when I try to use the gadget

    Unsupported feature: org.apache.shindig.common.xml.XmlException: Attribute name "data-pjax-transient" associated with an element type "meta" must be followed by the ' = ' character. At: (46,71)

    How should I proceed?

    The best

    ulrikk (author)ulrikk2016-02-06

    Hello again, it worked when I instead downloaded the zip file and unzipped it. Thanks a lot for a nice instructable.

    Ulrik

    skrallman made it! (author)2015-12-11

    Mike,

    I made a chart following your instruction. It shows up great on my laptop, but I cant see the right side of the chart and the settings on my phone. It is the same when I view your demo site for sparkfun. However, I noticed a link at the bottom of your page to "view as desktop". Through the link I can view your chart on my phone (maybe your labeling should is backwards?). How do I add that feature? BTW, great job.

    MikeTeachman (author)skrallman2015-12-19

    Hi,

    That's great. Thanks for sharing the chart. It looks like an interesting application.

    Sorry that the charts don't work well with smaller screens. At this time the demo chart site is optimized for laptop and desktop screens, like 1024x768 and better. The demo site doesn't render well on the smaller screen resolutions that come with phones and iPads (the right side gets cut off as you see). I'm working on the part 2 of the Instructables - my ambition is to have an option to show the charts on a phone.

    I think the "view as desktop" might be a feature in the mobile browser you are using? - it's not something that I had to add to the site. I'm curious what device and browser you are using. thanks! Mike

    About This Instructable

    18,444views

    69favorites

    License:

    More by MikeTeachman:IoT Charts Using Google Tools:  Getting StartedExceptional Blackberry Jelly
    Add instructable to: