IoT Charts Using Google Tools: Getting Started

Intro: IoT Charts Using Google Tools: Getting Started

The internet connected sensor revolution is here

It's getting easier every day to connect your Arduino, Raspberry Pi,ESP8266, Edison, or Photon project to the internet and push sensor measurement data to a cloud database such as ThingSpeak or Sparkfun Data. But, once your data is stored there, how do you best showcase your results to the world?

Levering the free web tools from Google is one answer

This is the first of 2 tutorials that will teach how to chart and share sensor data using free Google tools. By the end of the 2nd tutorial you will know how to rapidly build a professional looking web site that showcases the data produced by your hardware projects

A second goal of these tutorials is to teach how various internet technologies come together to produce a chart visualization

Here is the Instructables roadmap

• "Getting Started" (this instructable): Learn how to chart sensor data using a Demo Google Sitewith cloud data services ThingSpeak and Sparkfun Data
  

• "Build an IoT Website" (coming soon): build your own personal Google website to display custom charts of your sensor data

Here is an example website showing the potential of the Google tools. Our Hedgehog monitoring websiteis constructed using Google Sites, Google Charts, Google Sheets, and Google Maps. The Dashboard shows nightly and weekly statistics for our athletic pet

Google Site for Hedgehog Monitoring

For this tutorial you only need a web browser such as Chrome, Internet Explorer, or Firefox. Best usability with screen width resolutions 1280 pixels or better

STEP 1: ThingSpeak and Sparkfun Data

ThingSpeak is a free cloud platform for storing sensor data on a remote server

Sparkfun Datais another free cloud data service offered by the excellent hobbyist company Sparkfun

Both of these services store sensor data covering all sorts of interesting applications, from nuclear radiation to beehive monitoring. Most users of these services configure their sensor data storage to be readable by the public. You can access and plot any of this public data. We will cover that in this tutorial

ThingSpeak calls each bundle of sensor data a Channel while Sparkfun Datacalls it a Stream

You might already have data on ThinkSpeak or Sparkfun? Great. You can try these Google tools to chart your data

STEP 2: Quick Start - Using the Demo Site With ThingSpeak

Let's start by exploring the charting capabilities of Google tools using a demo web site that I designed for this Instructables

1. Open the IoT Charts Demo website
2. Open the THINGSPEAK page. The demo site is pre-programmed to show a line chart of temperature coming from a ThingSpeak data Channel of a weather station in Szczecin, Poland
3. Change the Number of Points setting in the right side panel.Press Update Chart. The timeline changes
4. Change "field1" to "field2" in the Field box. Press Update Chart. Humidity is plotted
5. Change "field2" to "field3" in the Field box. Press Update Chart. Atmospheric Pressure is plotted
6. Change the optional Min Value to set a minimum value on the Y scale. Press Update Chart. Notice the minimum value on the Y axis changes to your new value.Why would you want to do this? Setting a minimum value is particularly useful in situations where your hardware has accidentally generated zeros in the data stream -- I see this a lot. Leaving the Min Value box empty enables auto-scaling
7. Change the optional Max Value to set a maximum value on the Y scale. Press Update Chart
8. Enter some dates into Start Date and End Date. Delete the value in Number of Points. Press Update Chart. The timeline changes to the date range you specified

STEP 3: From Sensors to Chart Visualization

Let's take a step back for a moment to consider how various internet technologies are working together to produce a chart on the demo web site. The block diagram above and the attached PDF illustrate how sensor data is turned into a chart visualization on your screen

Here is a step-by-step description of the data flow (numbers below can be found on the block diagram)

1. sensors are read by the hardware, like Arduino or ESP8266
2. the hardware pushes the sensor data to a ThingSpeak server via the internet
3. sensor data is stored on a remote server operated by ThingSpeak
4. website formatting is stored at a Google server
5. using an internet browser, like Chrome or Internet Explorer, you click on the THINGSPEAK link on the IoT Charts Demowebsite
6. sensor data flows from the ThingSpeak server to the browser
7. web site formatting flows from a Google Sites server to the browser
8. Google Charts software flows from a Google server to the browser
9. browser combines 6, 7, and 8 to produce a chart of the sensor data

Hopefully this helps to demystify the process of rendering a chart on your screen

This tutorial series primarily focuses on steps 4 to 9. I added a References page with information on how to program the hardware to push sensor data to ThingSpeak and Sparkfun Data (steps 1-3)

STEP 4: Charting ThingSpeak Public Channels

Many users of ThingSpeak configure their Channels to allow public access. You can use the demo site to chart data from these Channels

1. Select an interesting Channel from one of the ThingSpeak Public Channels. Some example channels are listed at the bottom of this page
   
2. Open one of the Channels by clicking on the title
3. Find the Channel ID (see photo above)
4. Enter the Channel ID on the ThingSpeak charting page
5. Enter a Field. All ThingSpeak sensor measurements are identified by "field1", "field2"... "field8"
6. Press the Update Chart button. You will see a chart of the sensor data you selected

Here are some ThingSpeak Channels to try

Current weather condition for Szczecin, Poland. Temperature, humidity, and pressure are monitored

Channel ID: 29084
Fields: field1, field2, field3
URL: https://thingspeak.com/channels/29084

Hamstometer is a pedometer for hamsters which measures and tracks their wheel rotations per unit time

Channel ID: 46669
Fields: field1, field2, field3
URL: https://thingspeak.com/channels/46669

http://tinwhiskers.net/how-i-open-sourced-my-hamsters

Radiation level in Montreal, Quebec, Canada

Channel ID: 9892
Fields: field5
URL: https://thingspeak.com/channels/9892

STEP 5: Charting Sparkfun Data Public Channels

Many Sparkfun Data streams are also configured for public access

1. Pick a Stream from the listing of Sparkfun Public Streams. Some example streams are listed at the bottom of this page. Click on the Stream Name (see photo above)
2. There are two things you need from the Stream page (see photo above)
   Public Key (unique identifier for a Stream, shown in the web address)
   Field (name of data you want to plot, shown in the column title)
3. Enter the Public Key and Field into the edit boxes on the Sparkfun charting page of the demo site
4. Press the Update Chart button. In a short time you will see a plot of the sensor data. You might need to be patient - sometimes the Sparkfun Data site is slow to return data
5. The chart title can be customized by entering text into the Chart Title box

Try plotting these Example Sparkfun Streams ...

A home weather station in Elizabethtown KY USA:

Public Key: KJ111xY17xsRaKyw27XA
Fields: dailyrainin, tempf
URL: https://data.sparkfun.com/streams/KJ111xY17xsRaKyw27XA

Open Source Beehives Project:

Public Key: OGw6o7Z7AWsWVEZlEjj6
Fields: tempc_hive1, tempc_hive2
URL: https://data.sparkfun.com/streams/OGw6o7Z7AWsWVEZlEjj6

CO2 Sensor:

Public Key: ZG0aKVyV3DsQGNY8maO9
Fields: co2_ppm
URL: https://data.sparkfun.com/streams/ZG0aKVyV3DsQGNY8maO9

STEP 6: Under the Hood

This section is written for the technical ninja wanting to know more about the IoT Charts Demo site

The demo site is a Google Site. The charting features are built using Google Charts and Google Gadgets. A Gadget consists of HTML and JavaScript code wrapped by some "boilerplate" XML code. The gadget is embedded into a web page on the Google Site. The gadget code used on the charting demo site can be downloaded from my GitHub gadget repository. The gadget is called gadget-iotcharts-line-v1-1.xml (version 1-1 created on Oct 20, 2016 to correct a problem with the web site hosting the spinner script)

You can extend the capabilities of this gadget far beyond what is shown in this tutorial. There is a lot of potential for customization:

• change chart types. Try a bar chart, a pie chart. The Chart Gallery shows the possibilities
• change the height and width of the gadget
• pull data from a different data server like the Nimbits public cloud
• change colors
• format for mobile screens
• report error conditions to user
• configure the gadget to use tabs for the chart options, rather than a side panel

If you decide peek under the hood and extend the functionality I would appreciate any technical feedback on my implementation. Thanks !

Acknowledgements: This tutorial was built using knowledge generously shared by many developers, discovered by hours of Googling and reading StackExchange forums. It is truly a great age, where technical information is shared so openly. In some ways this Instructables is a small way of giving back

STEP 7: Reference: Pushing Your Sensor Data to the Cloud

Here are some references that teach how to program the hardware platforms to push sensor data to a cloud database

ThingSpeak

Arduino

Raspberry Pi

ESP8266

Particle Photon

Intel Edison

Sparkfun Data

Arduino

Raspberry Pi

ESP8266

Particle Photon

STEP 8: What's Coming

This tutorial is a first step to bigger and better things. In a soon-to-be-published Instructable I'll show how you can take full advantage of Google Sites. Using a Google Site Template I will show how a customized IoT web site can be built in less than 30 minutes. You will be able to mix sensor charts with other visual Google features, like configurable gadgets and maps

The end result will be a sensor web site, the way you want it to look. And, no JavaScript or HTML coding will be needed

Google gives us powerful capabilities to display and share our sensor data. High availability and free

Comments

[AnthonyI28]

Nice and informative. Where can I find the google follow up instructions?

[MikeTeachman]

Hi. The Part 2 of this Instructables is still on my "To-Do" list. Sorry. I don't have a schedule for starting this work.

[clayton.cowgill]

Hey Mike,

I'm not sure if this was just a temporary thing or limited to my particular application, but using your iotcharts .xml and the instructable (and your follow-up Google Sites document) I never was able to see a graph from either your sample data or mine.

After some fumbling around and dropping 'alerts' to see where the code was-- it appears that in my case it was never returning from Line 278: var spinner = new Spinner(spinnerOpts).spin(target);

Commenting that line out allowed things to work as on the demo site. Just in case anyone else gets stuck on it...

Thanks for the tutorial!

[RobO43]

I experienced the same thing. The spinner that shows when the plot is loading isn't available, but it does work well without it.

[MikeTeachman]

Hi,

I just corrected the spinner issue. A new gadget-iotcharts-line-v1-1.xml

version of the IoT gadget is available in my github repository. thank you for your patience.

[MikeTeachman]

Hi,

I just corrected the spinner issue. A new gadget-iotcharts-line-v1-1.xml

version of the IoT gadget is available in my github repository. thank you for your patience.

[RobO43]

Mike, Great instructions and thanks for sharing your work. I've got my sparkfun data stream tied to your gadget on a google site page and it is plotting the sump water level in my basement. I want to add the ability to count the number of pump cycles per hour or sump cycles per week and estimate the number of gallons pumped out per day/week/month. I'm not sure how to "peak find" in the data table to pull out a pump event. Any ideas on where to go for that?

[MaureenB16]

Cant´t wait for the bigger tutorial!

[tnickolite]

Mike,

Thank you so much for providing this tutorial. I have been able to modify the line graph to fit my website. What would I have to change to create the bar graph that you have on your website?

[MikeTeachman]

Hi,

I added a new github repository with the gadget code for the column chart. This should give you an idea on how the column chart works to display the last 7 days of hedgehog running.

https://github.com/MikeTeachman/gadget-distanceLas...

The getJSON method pulls data from a spreadsheet on a Google Drive account. The spreadsheet is used as the calculation back end for the site.

Hope this helps...

Mike

[tnickolite]

Mike,

Thank you so much for providing this tutorial. I have been able to modify the line graph to fit my website. What would I have to change to create the bar graph that you have on your website?

[kcajlliw]

Mike,

I must say your guide and gadget are awesome - you have made something I thought would take a lot of work or would be very expensive as easy as pie!

I was just wondering if you had any idea how I could plot longitude against latitude on a similar plot - The background satellite map would be cool but not necessary. Could this also be integrated with google charts?

Many thanks, Jack!

[MikeTeachman]

Hi Jack,

I experimented with using google map gadgets on our pet hedgehog site -- I created two gadgets that use the google maps API to plot the hedgehog's "run across north america". The maps update automatically and move the hedgehog icon across the map. I posted the gadget code on github.

https://github.com/MikeTeachman/gadgets-googleMaps

The gadgets are used on this Google sites page:

https://sites.google.com/site/sircharlesruns/map

How it works: the gadget code reads latitude and longitude from a google sheet on a google drive folder using the $.getJSON call. This google sheet only has two cells, one for latitude and the other for longitude -- the calculations for lat and long are done on a separate sheet. I found the parsing easier in the gadget code to have the lat and long isolated on a sheet. Here is a link to the google sheet:

http://tinyurl.com/zvgfda6

I hope this is enough description and information to reverse engineer what I built. I would be grateful for any feedback on this work so I can make improvements to the implementation.

Mike

[ulrikk]

Mike

I tried ti use the gadget after your intructions but I get a errormessage when I try to use the gadget

Unsupported feature: org.apache.shindig.common.xml.XmlException: Attribute name "data-pjax-transient" associated with an element type "meta" must be followed by the ' = ' character. At: (46,71)

How should I proceed?

The best

[ulrikk]

Hello again, it worked when I instead downloaded the zip file and unzipped it. Thanks a lot for a nice instructable.

Ulrik