Visualize Twitter Hashtag Activity With Your Spark Core

Intro: Visualize Twitter Hashtag Activity With Your Spark Core

With its built-in WiFi capability and automatic access to the Spark Cloud, the Spark Core is a great choice for visualizing data from the web. In this guide, we'll utilize the Spark Core with the Spark Button shield (an add-on circuit board) to visualize hashtag data from Twitter in real time. As with any Spark Core development project, you'll need access to an Internet-connected WiFi network. The computer parts of this guide are written for Mac OS X users, but the steps can be followed in a similar fashion on a Windows or Linux machine.

STEP 1: Set Up Your Spark Core

Set up your Core by following the steps on Spark's Getting Started page. Note that I (and a few others I've talked to) have had only minimal success with Spark's automatic WiFi setup, which uses a smartphone and some Wi-Fi protocol hackery to transfer WiFi credentials to the Core, but there's no reason not to give it a shot. What has been working reliably for me is to use a serial monitor (such as the one built into the Arduino IDE) or Spark's CLI tool (requires Node.JS).

Move on to the next step when your Core is connected to WiFi and "claimed" to your Spark.io account.

STEP 2: Load Your Code

Now that your Core is online and ready to go, let's download a program to it. In the Spark Build IDE, click on the "Libraries" icon on the left side (looks like a bookmark) and select SPARKBUTTON, then in the resulting list select "4_GOOD_COMBINATION.CPP". This will load up some example code for controlling your Spark Button shield. Finally, click "Use this example." Since the time this example was written, Spark.io has changed some of their internals and you'll need to make a little change before the example will compile. Change the first line from:

#include "SparkButton.h"
to

#include "SparkButton/SparkButton.h"

Now you can click the Lightning bolt icon to flash this code to your Core. When the light on the Core stops blinking and returns to breathing cyan, you should be able to press the buttons on your Spark Button shield and make different lights light up. Take a look at the example code and see how it works.

When you're done tinkering, copy the code from this page into the IDE and flash it to your device, then move onto the next step.

STEP 3: Control Your Core Over the Web

The Core comes with some cool and useful abilities that are enabled by its persistent WiFi connection, such as the ability to be reprogrammed over the air, as we saw in the last step. Now, we're going to take advantage of the Core's connection to the Spark Cloud to send it remote control messages over WiFi. At first, we'll key in the remote control commands manually, but eventually we'll use a program for this task. To control the Core over the air, you'll need two pieces of information: your Core's ID and Access Token. To get both pieces of this information, you'll need to launch the online development environment at https://www.spark.io/build/ if it isn't already open.

• Get your Core's ID by clicking on the "Cores" icon (the compass-looking one in the bottom left) and clicking on the disclosure triangle to the right of the name of your Core. The Core's device ID, a long alphanumeric string, should be displayed.
• Get your Access Token by clicking on "Settings" (the gear, right below the "Cores" icon). It's also a long alphanumeric string.

Copy and paste both of these pieces of information into an text file or your favorite note-taking application for now (make sure you label which number is which).

To receive data over the web, your Core makes use of a popular architecture style called a RESTful API, which uses the HTTP standard to ensure that the sender and receiver understand each other. Without worrying about the details, this is good news for our project, since HTTP is simple, stable, and its popularity means that it will be easy to find pre-written software libraries to help us test and write our final software.

We're going to use a tool that ships with your Mac called cURL to test out communications with your Core. cURL is a tool for executing HTTP commands and seeing their results (incidentally, this is also what your web browser's purpose is). Open the Terminal application on your computer located in /Applications/Utilities/Terminal.app and enter the following command. Replace YOURDEVICEIDHERE and YOURACCESSTOKENHERE with (you guessed it) the device ID and access token you obtained.

curl https://api.spark.io/v1/devices/YOURDEVICEIDHERE/led -d access_token=YOURACCESSTOKENHERE

If all goes well, your Core should briefly light up! If nothing happens, check the following:

• Did you copy the cURL command exactly? Make sure there are no extra or missing spaces
• Is your Core breathing Cyan? (meaning that it's online and connected to the Spark Cloud)

STEP 4: Get Developer Access to Twitter

Let's put the Core aside for a moment and lay the groundwork for the input of our project: a realtime feed from Twitter displaying Tweets with a hashtag of our choice. First, you'll need to sign up for a regular ol' Twitter account if you don't already have one. Once you've done that, head on over to the Application Management section of Twitter's website for developers and create a new app. The information that you fill in here doesn't matter too much, since no one will see it. You can leave the Callback URL field blank.

After you get the "Your application has been created" success message, scroll down on that application detail page to the Application Settings section, then locate and click on the link that says "manage keys and access tokens". Scroll down and click on "Generate my access token." You'll need to save four values from this page. I recommend you just paste them into a text document for now, but add a little label so you know which is which. You'll need Consumer Key (API Key), Consumer Secret (API Secret), Access Token, and Access Token Secret.

STEP 5: Bring It All Together

Now we have programmatic access to Twitter and programmatic access to your Core, so let's glue them together! Rather than dealing with implementing the OAuth authentication scheme ourselves, we'll rely on a PHP library called phirehose to deal with the annoying work of authenticating with Twitter and filtering our Tweets for us. Download phirehose from Github to your Desktop folder by entering the following command into Terminal and pressing enter:

cd ~/Desktop && git clone https://github.com/fennb/phirehose.git

Now switch to the Finder, go to your Desktop, and open the phirehose folder that was just added. Inside phirehose should be a folder called example. Open the file called filter-oauth.php using a plain text editor (a nice free, plain-text editor for the Mac that I like is Fraise, but you can use the built-in TextEdit as well).

We'll be making a few changes. First, you'll need to add some lines of code. Copy and paste from this pastebin link and paste everything after the right curly brace (}) that directly follows the line that starts with print. (See the screenshot to confirm that you're pasting in the right place.)

http://pastebin.com/YdVBSfu3

After you paste in this code, you'll need to change two instances of YOURDEVICEIDHERE and YOURACCESSTOKENHERE to your actual Spark Core device ID and access token.

Next, you'll need to update the four empty strings at the bottom of the page by filling in your actual Twitter Consumer Key, Twitter Consumer Secret, Twitter OAuth Token, and Twitter OAuth Secret.

Finally, the second to last line of the file with the hashtag or hashtags that you want to track. In my screenshot, I'm just tracking the hashtag lazysunday (no need to put the # symbol here).

Now, save the file and switch back to Terminal. Type the following command to run the program with the PHP interpreter:

php ~/Desktop/phirehose/example/filter-oauth.php

Now your program will keep a connection open to Twitter's servers for a constant feed of any Tweets containing your specified hashtags. When a Tweet is received, your program will turn around and send your Core the "light up" message. Check it out!

Note that if you want to try out different hashtags or modify the PHP program in any way, you'll need to stop it by typing ctrl-C, editing and resaving the file in your text editor, and then re-running the program. A good trick is to use the up arrow to see past commands to save you having to retype run program command. (i.e. ctrl-C, up arrow, enter will stop and restart the program).

STEP 6: Next Steps

Think about how else you might expand on this concept. For example:

• Track multiple hashtags, each with a different color
• Quantify how popular an individual hashtag is in realtime by lighting up one dot each time a new tweet is detected with that hashtag and allowing each dot to stay lit for a second
• Move your PHP script to the cloud, so you don't need your computer on, running the program, and connected to the Internet for the project to function. You could use a cheap VPS provider like Amazon EC2.
• Make a bigger or more interesting light up display than just the NeoPixels

Thanks for reading and have fun!

Comments

[inetwebguy]

Can I use this code with just wiring up some LED strip lights and not use the button shield? I love the concept. (not much of a programmer) but I'm willing to learn.

[seamster]

This is really cool!