Introduction: Twitter Mood Light - the World's Mood in a Box


How's the world feeling right now? This box tells you.

Powered by: an Arduino, a WiFly wireless module, an RGB LED, Twitter.com and a 9v battery.

I’m a news junkie. I want to know everything that is going on in the world as soon as it happens. I want to wake up and know immediately if something big has happened overnight.

However, I’m an extraordinarily busy man; I don’t have time to read news feeds; reading that headline that I already knew about or don’t care about is time that I’m never getting back!

No. What I need is some way to be constantly in touch with the world's events as they unfold, alerted when something big happens, and to be made aware of it all faster than awareness itself!

...A way to get a glimpse of the collective human consciousness as an extension of my own.  Something that I don't have to continually check or poll, but instead, like a part of my body, it will tell me when it's feeling pain or generally in need of my attention ...leaving me time to get on with other things.

And so, I present: The World Mood in a Box!

The Arduino connects directly to any wireless network via the WiFly module, continually searches Twitter for tweets with emotional content, collates the tweets for each emotion, does some math, and then fades the color of the LED to reflect the current World Mood; Red for Anger, Yellow for Happy, Pink for Love, White for Fear, Green for Envy, Orange for Surprise, and Blue for Sadness.

If an unexpectedly high number of tweets of a particular emotion are found, then the LED will flash to alert us to the possibility of a world event that has caused this unusually strong emotional reaction.

For example, a world disaster and it may flash Blue or Red (sadness or anger), if the strong favourite loses a big football game it may fade to Orange (surprise), …and If it flashes White, the collective human mind is feeling extreme fear, and it's probably best to go hide in a cupboard and sit it out, waiting for sunnier skies and a return to Yellow or Pink.  ...OK, I'm not that busy.

Step 1: How It Works

An Arduino connects directly (no computer required!) to any wireless network via the WiFly module, repeatedly searches Twitter for tweets with emotional content (aka sentiment extraction or tapping into the moodosphere), collates the tweets for each emotion, analyzes the data, and fades the color of an LED to reflect the current World Mood:

  • Red for Anger
  • Yellow for Happy
  • Pink for Love
  • White for Fear
  • Green for Envy
  • Orange for Surprise
  • Blue for Sadness
Example search terms to find tweets that may express surprise:
  • "wow"
  • "can't believe"
  • "unbelievable"
  • "O_o"
If an unexpectedly high number of tweets of a particular emotion are found, then the LED will flash to alert anyone nearby to the possibility of a big world event that has caused this unusually strong emotional reaction.

Example signals:
  • A world disaster and it may flash Blue or Red indicating it best to check a news site to see why everyone is so sad and/or angry.
  • If the strong favourite loses a big football game, it may flash Orange to express the surprise at this unlikely event.
  • If there is a heat wave in London it might turn Yellow to reflect how much happier people now are.
  • If it flashes White, the collective human consciousness is feeling extreme fear and something terrifyingly bad is probably about to happen. Time to hide and/or panic.
Uses
  • You could put it on your desk to get an early warning of something big happening somewhere in the world
  • A literal 'mood light' at a party or a game whereby you guess what colour it will change to next and for what reason
  • A world mood barometer perhaps next to your bed to decide if it is best to hit snooze until it's less angry outside
  • A gauge of public sentiment to help you decide when to sell all your stocks and shares, and head to the hills.
  • In a foyer or waiting area or other public space for people to look at and contemplate.
  • Set it to connect to any wireless network and carry it around in the streets, stopping strangers to explain to them that you have managed to capture the world's mood and have it locked in this here box.

Step 2: All You Need Is...

I ordered most of the electronics from Sparkfun, and picked up the rest from the local Radioshack. The acrylic I got from a local plastic shop(!) - they cut it and drilled a hole free of charge.

Materials
The Acrylic Box
  • 1 x (5" x 5" x 0.25") - the top
  • 4 * (4.75" x 4.75" x 0.25") - the 4 walls
  • 1 x (4.5" x 4.5" x 0.25") - the base
  • 1 x (4.5" x 4.5" x 0.125") - the mirror with a 6mm hole drilled in the middle
  • 4 x (4.25 x 1" x 0.25") - the 4 inside walls
  • Acrylic solvent cement
  • Sand paper (to help diffuse the light)
Tools
  • Soldering iron
  • A computer
  • Arduino development environment
  • A wireless network (802.11b/g)
  • Pliers
  • Wire stripper
Useful links

The Arduino development tools can be downloaded from here:
www.arduino.cc/en/Main/Software

and Arduino tutorials start here:
http://arduino.cc/en/Guide/HomePage

Arduino / WiFly:
arduino.cc/en/Reference/HomePage
http://www.arduino.cc/en/Tutorial/SPIEEPROM
http://www.lammertbies.nl/comm/info/serial-uart.html
http://www.tinyclr.com/downloads/Shield/FEZ_Shields_WiFly.cs
http://www.sparkfun.com/commerce/tutorial_info.php?tutorials_id=158
http://www.sparkfun.com/datasheets/Components/SMD/sc16is750.pdf
http://www.sparkfun.com/datasheets/Wireless/WiFi/WiFlyGSX-um.pdf
http://www.sparkfun.com/datasheets/Wireless/WiFi/rn-131G-ds.pdf

http://www.societyofrobots.com/microcontroller_uart.shtml

Related:
nlp.stanford.edu/courses/cs224n/2009/fp/22.pdf
www.webservius.com/corp/docs/tweetfeel_sentiment.htm
i8news.uterm.org/mood/twitter-mood-reader/
community.openamplify.com/content/docs.aspx/
www.instructables.com/id/The-Twittering-Office-Chair/
http://www.tweetfeel.com


Step 3: Connect the Arduino and WiFly to a Computer

Sparkfun have a decent tutorial on how to do this:

www.sparkfun.com/commerce/tutorial_info.php

Firstly, the Wifly breakout board needs to be stacked on top of the arduino and the RX, TX, Vin, Gnd, pin 10, pin 11, pin 12 and pin 13 needed to be connected. I used breakaway headers and soldered the required pins.

Connect to a computer using an A to B USB cable.

Download the Arduino software from here:
arduino.cc/en/Main/Software

Check that you can compile and upload a sample program by following the instructions here:
(remember to set the board and COM ports correctly)
arduino.cc/en/Guide/HomePage

Step 4: Connecting the LED

Only some pins provide 8-bit PWM (Pulse-width modulation)
This gives 256 steps of control from full off (0) to full on (255) for each of the Red, Green and Blue channels of the LED.

PWM pins on the Arduino are 3,5,6,9,10,11. (see www.arduino.cc/en/Main/ArduinoBoardDuemilanove)

I used 3, 5 and 6.
I used the pliers to bend the legs of the LED, and mounted it on the circuit board. Each resistor is then mounted next to each of the RGB legs, and the wires are twisted together. Then I added the 4 connecting wires and twisted them. Finally, I soldered all the connections. 

Note: The pictures illustrate using the same resistor for each colour channel, but I should have used the resistance levels in the data sheet:

180 Ohm for Red
100 Ohm for Green
100 Ohm for Blue

Also note, I covered the back with insulating tape to stop any shorts when putting it all into the box.
Also, from the datasheet, "the Sensor inputs SENS0-7 are extremely sensitive to over voltage. Under no conditions should these pins be driven above 1.2VDC. Placing any voltage above
this will permanently damage the radio module and render it useless."


wiring.org.co/learning/basics/rgbled.html
www.sparkfun.com/datasheets/Components/YSL-R596CR3G4B5C-C10.pdf

Step 5: Choosing Good Search Terms


Twitter allows you to search for recent tweets that contain particular words or phrases.

You can search for tweets that contain any of a list of phrases by using the "+OR+" conjunction.

For example, here is a search request that might find tweets that express Fear:

GET /search.json?q="i'm+so+scared"+OR+"i'm+really+scared"+OR+"i'm+terrified"+OR+"i'm+really+afraid"+OR+"so+scared+i"&rpp=30&result_type=recent

I spent a long time finding good search phrases.

The search phrases needed to produce tweets that:
  1. very often express the desired emotion.
  2. very rarely express the opposite emotion or no emotion.

Many search phrases that I thought would work, turned out to not work that well when I searched with them.

Smileys have been used with some success to extract whether the sentence is positive or negative, but I didn't find them useful for extracting anything more.

The trouble with smileys is that a smile can mean so many things ;D

It is often used, it seems, as a kind of qualifier for the whole sentence; since people have to compress their thoughts into 140 characters, the meaning can become ambiguous.

The smiley often then acts as a qualifier that:
  • 'this is a friendly comment'
  • 'don't take this the wrong  way'
  • 'i am saying hello/goodbye with a smile'
  • 'this is almost a joke'
  • 'I know I'm being cheeky'
  • 'I don't really mean this'
Phrases using adverbs seemed to produce better results.
"so scared" or "really scared" is better than just "scared" which returns bad results: for example, "not scared".

Phrases in the first person seemed to produce better results.
Some search phrases give tweets that suggest the author feels the emotion: for example, "i really hate...", often sounds like they really are full of hate or angry, whereas other phrases containing the word "hate" give tweets that do not seem to express much emotion, like "why do you hate..."

Hyperbole is your best friend, ever:
Using phrases with hyperbole produced good results. Tweets with "I'm terrified" or "I'm petrified" in them were generally more fearful sounding than "I'm scared"

Regardless, the approach is still naive, but statistically, from my tests, it does seem to work well.

While testing the code, I did at one point get the horribly ominous "Flashing White" that signifies the world is feeling intense fear, but since I was still testing it all, I did not hide under the table straight away, but instead, threw caution to the winds, and went on to Twitter to see what people were suddenly so fearful about.

The recent tweets containing the Fear search string (see top of page) were largely relating to a large thunderstorm that had just started somewhere near Florida.

If you're interested, here are some of those tweets: 
  • "Ahhh Thunder I'm so scared of Thunder !!!!! Help some 1"
  • "I'm so scared of lightning now. Like I just ran home praying "
  • "On our way to Narcosses at @Disney world's Grand Floridian hotel and there's a tropical storm right now. I'm terrified! ..."
  • "I'm in my bathroom til the rain stops. I'm terrified of lightning and thunder..."
  • "I'm terrified of thunder storms *hides in corner*"
  • "I'm terrified of Thunder :("
  • "If only I was wit my becky during this thunderstorm cause I'm really scared cause of a bad experience"
So... it works! ...Well, it needs the numbers tweaking to ignore the world's "tantrums", the short-lived fits of emotional outburst, and be more concerned with larger changes that signify bigger news.

Step 6: Download the Code

The attached WorldMood.zip contains 4 subdirectories (or "libraries") and the Arduino sketch WorldMood.pde

The four libraries need to be copied into the Arduino library directory and then they can be imported as shown.

WorldMood/WorldMood.pde (see below) should be opened in the Arduino development environment.

You then need to correct the "[your network]" and "[your network password]" fields. eg.

#define network ("mynetwork")
#define password ("mypassword")

Then the sketch (and libraries) should be compiled and uploaded to the Arduino board.

see arduino.cc/en/Hacking/LibraryTutorial

The next 5 programming steps just give an overview of each of the components and include the most noteworthy parts of the source code...

**** Update ****

If you have a newer board then you may need to change this

struct SPI_UART_cfg SPI_Uart_config = {0x50,0x00,0x03,0x10};

to this:
struct SPI_UART_cfg SPI_Uart_config = {0x60,0x00,0x03,0x10};


See here for more info:
http://forum.sparkfun.com/viewtopic.php?f=13&t=21846&sid=24282242d4256db0c7b7e814d7ca6952&start=15

http://www.sparkfun.com/commerce/product_info.php?products_id=9367

***** End Update ****

 
 
// LED setup - only some pins provide 8-bit PWM (Pulse-width modulation)
// output with the analogWrite() function.
// http://www.arduino.cc/en/Main/ArduinoBoardDuemilanove
// PWM: 3,5,6,9,10,11
#defineredPin    (3)
#definegreenPin (5)
#definebluePin   (6)
 
// delay in ms between fade updates
// max fade time = 255 * 15 = 3.825s
#definefadeDelay (15)
 
// Wifi setup
#definenetwork ([your network])
#definepassword ([your network password])
#defineremoteServer ("twitter.com")
 
constchar* moodNames[NUM_MOOD_TYPES] = {
 "love",
 "joy",
 "surprise",
 "anger",
 "envy",
 "sadness",
 "fear",
};
 
constchar* moodIntensityNames[NUM_MOOD_INTENSITY] = {
 "mild",
 "considerable",
 "extreme",
};
 
// the long term ratios between tweets with emotional content
// as discovered by using the below search terms over a period of time.
floattempramentRatios[NUM_MOOD_TYPES] = {
 0.13f,
 0.15f,
 0.20f,
 0.14f,
 0.16f,
 0.12f,
 0.10f,
};
 
// these numbers can be tweaked to get the system to be more or less reactive
// to be more or less susceptible to noise or short term emotional blips, like sport results
// or bigger events, like world disasters
#define emotionSmoothingFactor (0.1f)
#define moodSmoothingFactor (0.05f)
#define moderateMoodThreshold (2.0f)
#define extremeMoodThreshold (4.0f)
 
// save battery, put the wifly to sleep for this long between searches (in ms)
#defineSLEEP_TIME_BETWEEN_SEARCHES (1000 * 5)
 
// Store search strings in flash (program) memory instead of SRAM.
// http://www.arduino.cc/en/Reference/PROGMEM
// edit TWEETS_PER_PAGE if changing the rpp value
prog_charstring_0[] PROGMEM = "GET /search.json?q=\"i+love+you\"+OR+\"i+love+her\"+OR+\"i+love+him\"+OR+\"all+my+love\"+OR+\"i'm+in+love\"+OR+\"i+really+love\"&rpp=30&result_type=recent";
prog_charstring_1[] PROGMEM = "GET /search.json?q=\"happiest\"+OR+\"so+happy\"+OR+\"so+excited\"+OR+\"i'm+happy\"+OR+\"woot\"+OR+\"w00t\"&rpp=30&result_type=recent";
prog_charstring_2[] PROGMEM = "GET /search.json?q=\"wow\"+OR+\"O_o\"+OR+\"can't+believe\"+OR+\"wtf\"+OR+\"unbelievable\"&rpp=30&result_type=recent";
prog_charstring_3[] PROGMEM = "GET /search.json?q=\"i+hate\"+OR+\"really+angry\"+OR+\"i+am+mad\"+OR+\"really+hate\"+OR+\"so+angry\"&rpp=30&result_type=recent";
prog_charstring_4[] PROGMEM = "GET /search.json?q=\"i+wish+i\"+OR+\"i'm+envious\"+OR+ \"i'm+jealous\"+OR+\"i+want+to+be\"+OR+\"why+can't+i\"+&rpp=30&result_type=recent";
prog_charstring_5[] PROGMEM = "GET /search.json?q=\"i'm+so+sad\"+OR+\"i'm+heartbroken\"+OR+\"i'm+so+upset\"+OR+\"i'm+depressed\"+OR+\"i+can't+stop+crying\"&rpp=30&result_type=recent";
prog_charstring_6[] PROGMEM = "GET /search.json?q=\"i'm+so+scared\"+OR+\"i'm+really+scared\"+OR+\"i'm+terrified\"+OR+\"i'm+really+afraid\"+OR+\"so+scared+i\"&rpp=30&result_type=recent";
 
// be sure to change this if you edit the rpp value above
#defineTWEETS_PER_PAGE (30)
 
PROGMEMconstchar *searchStrings[] =        
{  
 string_0,
 string_1,
 string_2,
 string_3,
 string_4,
 string_5,
 string_6,
};
 
voidsetup()
{
 Serial.begin(9600);
 delay(100);
}
 
voidloop()
{
 // create and initialise the subsystems 
 WiFlywifly(network, password, SLEEP_TIME_BETWEEN_SEARCHES, Serial);
 WorldMoodworldMood(Serial, emotionSmoothingFactor, moodSmoothingFactor, moderateMoodThreshold, extremeMoodThreshold, tempramentRatios);
 LEDled(Serial, redPin, greenPin, bluePin, fadeDelay);
 TwitterParsertwitterSearchParser(Serial, TWEETS_PER_PAGE);
 
 wifly.Reset();
 
 charsearchString[160];
 
 while (true)
 {
    for (inti = 0; i < NUM_MOOD_TYPES; i++)
    {
      twitterSearchParser.Reset();
 
      // read in new search string to SRAM from flash memory
      strcpy_P(searchString, (char*)pgm_read_word(&(searchStrings[i])));
 
      boolok = false;
      intretries = 0;
 
      // some recovery code if the web request fails
      while (!ok)
      {
        ok = wifly.HttpWebRequest(remoteServer, searchString, &twitterSearchParser);
 
        if (!ok)
        {
          Serial.println("HttpWebRequest failed");
 
          retries++;
          if (retries > 3)
          {
            wifly.Reset();
            retries = 0;
          }
        }
      }
 
      floattweetsPerMinute = twitterSearchParser.GetTweetsPerMinute();
 
      // debug code
      Serial.println("");
      Serial.print(moodNames[i]);
      Serial.print(": tweets per min = ");
      Serial.println(tweetsPerMinute);
 
      worldMood.RegisterTweets(i, tweetsPerMinute);
    }
 
    MOOD_TYPEnewMood = worldMood.ComputeCurrentMood();
    MOOD_INTENSITYnewMoodIntensity = worldMood.ComputeCurrentMoodIntensity();
 
    Serial.print("The Mood of the World is ... ");
    Serial.print(moodIntensityNames[(int)newMoodIntensity]);
    Serial.print(" ");
    Serial.println(moodNames[(int)newMood]);
 
    led.SetColor((int)newMood, (int)newMoodIntensity);
 
    // save the battery
    wifly.Sleep();
 
    // wait until it is time for the next update
    delay(SLEEP_TIME_BETWEEN_SEARCHES);
 
    Serial.println("");
 }
}
 

Step 7: Programming Step 1: SPI UART

The WiFly Shield equips your Arduino with the ability to connect to 802.11b/g wireless networks.
The featured components of the shield are:
  • a Roving Network's RN-131G wireless module
  • SC16IS750 SPI-to-UART chip. 
Serial Peripheral Interface Bus (or SPI) is a "four wire" serial bus capable of high rates of data transmission. A serial bus allows data to be sent serially (synchronously), i.e. one bit at a time, rather than in parallel (asynchronous)

The Universal asynchronous receiver/transmitter (or UART) is a type of asynchronous receiver/transmitter, a piece of computer hardware that translates data between parallel and serial forms.
  1. The PC communicates over UART with the Arduino through pins RX and TX
  2. The Arduino communicates over SPI with the SPI-UART chip on the WiFly shield (SC16IS750 SPI-to-UART chip) though pins 10-13 (CS, MOSI, MISO, SCLK respectively)
  3. The RN-131G wireless module accesses network and send/receive serial data over UART.
The SPI-to-UART bridge is used to allow for faster transmission speed and to free up the Arduino's UART.

The code below is based on a number of sources, but primarily from this tutorial over at sparkfun:
www.sparkfun.com/commerce/tutorial_info.php 
WiFly Wireless Talking SpeakJet Server


/* Test if the SPI<->UART bridge has been set up correctly by writing a test
   character via SPI and reading it back.
   returns true if success
*/
boolWiFly::TestSPI_UART_Bridge()
{
 // Perform read/write test to check if SPI<->UART bridge is working
 
 // write a character to the scratchpad register.
 WriteByteToRegister(SPR, 0x55);
 
 chardata = ReadCharFromWiFly(SPR);
 
 if(data == 0x55)
 {
    returntrue;
 }
 else
 {
    m_printer->println("Failed to init SPI<->UART chip");
    returnfalse;
 }
}
 
/* A series of register writes to initialize the SC16IS750 SPI-UART bridge chip
   see http://www.tinyclr.com/downloads/Shield/FEZ_Shields_WiFly.cs
*/
voidWiFly::SPI_UART_Init(void)
{
 WriteByteToRegister(LCR,0x80); // 0x80 to program baudrate
 WriteByteToRegister(DLL,SPI_Uart_config.DivL); //0x50 = 9600 with Xtal = 12.288MHz
 WriteByteToRegister(DLM,SPI_Uart_config.DivM);
 
 WriteByteToRegister(LCR, 0xBF); // access EFR register
 WriteByteToRegister(EFR, SPI_Uart_config.Flow); // enable enhanced registers
 WriteByteToRegister(LCR, SPI_Uart_config.DataFormat); // 8 data bit, 1 stop bit, no parity
 WriteByteToRegister(FCR, 0x06); // reset TXFIFO, reset RXFIFO, non FIFO mode
 WriteByteToRegister(FCR, 0x01); // enable FIFO mode
}


Step 8: Programming Step 2: Connecting to a Wireless Network

Again, this is largely based on the sparkfun tutorial, but I've removed the delays with "waits for response". This speeds things up and is easier to error check.

www.sparkfun.com/commerce/tutorial_info.php

/*
 Send the correct commands to connect to a wireless network using the parameters used on construction
*/
voidWiFly::AutoConnect()
{
 delay(DEFAULT_TIME_TO_READY);
 
 FlushRX();
 
 // Enter command mode
 EnterCommandMode();
 
 // Reboot to get device into known state
 WriteToWiFlyCR("reboot");
 WaitUntilReceived("*Reboot*");
 WaitUntilReceived("*READY*");
 
 FlushRX();
 
 // Enter command mode
 EnterCommandMode();
 
 // turn off auto joining
 WriteToWiFlyCR("set wlan join 0");
 WaitUntilReceived(AOK, ERR);
 
 // Set authentication level to
 WriteToWiFly("set w a ");
 WriteToWiFlyCR(auth_level);
 WaitUntilReceived(AOK, ERR);
 
 // Set authentication phrase to
 WriteToWiFly("set w p ");
 WriteToWiFlyCR(m_password);
 WaitUntilReceived(AOK, ERR);
 
 // Set localport to
 WriteToWiFly("set i l ");
 WriteToWiFlyCR(port_listen);
 WaitUntilReceived(AOK, ERR);
 
 // Deactivate remote connection automatic message
 WriteToWiFlyCR("set comm remote 0");
 WaitUntilReceived(AOK, ERR);
 
 // Join wireless network
 WriteToWiFly("join ");
 WriteToWiFlyCR(m_network); 
 delay(DEFAULT_TIME_TO_JOIN);
 
 boolok = WaitUntilReceived("IP=");
 delay(DEFAULT_TIME_TO_WAIT);
 
 FlushRX();
 
 if(ok == false)
 {
    m_printer->print("Failed to associate with '");
    m_printer->print(m_network);
    m_printer->println("'\n\rRetrying...");
    FlushRX();
    AutoConnect();
 }
 else
 {
    m_printer->println("Associated!");
    ExitCommandMode();
 }
 
 // TODO save this configuration
}
 
 
/*
 Enter command mode by sending: $$$
 Characters are passed until this exact sequence is seen. If any bytes are seen before these chars, or
 after these chars, in a 1 second window, command mode will not be entered and these bytes will be passed
 on to other side.
*/
voidWiFly::EnterCommandMode()
{
 FlushRX();
 delay(1000); // wait 1s as instructed above
 m_printer->println("Entering command mode.");
 WriteToWiFly("$$$");
 WaitUntilReceived("CMD");
}
 
 
/*
 exit command mode
 send the "exit" command and await the confirmation result "EXIT"
*/
voidWiFly::ExitCommandMode()
{
 WriteToWiFlyCR("exit");
 WaitUntilReceived("EXIT");
}


Step 9: Programming Step 3: Searching Twitter With TCP/IP Port 80

Http is just TCP/IP on port 80

for example:

"Open www.google.com 80"

will open a Http connection to www.google.com.

Twitter actually requires more of the Http protocol than google.

For example, the "Host" field is often required in case there's more than one
domain name mapped to the server's IP address so it can tell which
website you actually want.

Twitter also requires a final linefeed and carriage return ("\r\n")

"GET /\n"
"Host: server\r\n"
"\r\n"

I use search.json rather than search.atom to give results in non-html format, and more easily parsed. (see apiwiki.twitter.com/Twitter-API-Documentation)

 
 
/*
 Parameters: The server to telnet into, the get command that needs to be sent, a custom HtmlParser that
 is called every time a character is received. The parser is responsible for processing the HTML
 that is returned.
*/
boolWiFly::HttpWebRequest(constchar* server, constchar* getCommand, HtmlParser* parser)
{
 m_printer->println(getCommand);
 FlushRX();
 FlushRX();
 
 // Enter command mode
 EnterCommandMode();
 FlushRX();
 
 // open a TCP connection, port 80 for HTTP
 WriteToWiFly("open ");
 WriteToWiFly(server);
 WriteToWiFlyCR(" 80");
 
 boolopenOK = WaitUntilReceived(COMM_OPEN);
 
 if (openOK == false)
 {
    m_printer->println("open port failed!");
    delay(1000);
    WriteToWiFlyCR("close");
    WaitUntilReceived(COMM_CLOSE);
 
    ExitCommandMode();
    returnfalse;
 }
 
 // eg. "GET /search.json?q=foo HTTP/1.1\r\n"
 WriteToWiFlyCRLF(getCommand);
 
 // eg. "Host: search.twitter.com\r\n"
 WriteToWiFly("Host: ");
 WriteToWiFlyCRLF(server);
 
 // "\r\n"
 WriteToWiFlyCRLF("");
 
 // now wait for the response
 
 inttimeOut = 0;
 boolok = false;
 
 while(timeOut < 5000)// timeout after 5 seconds
  {
    if((ReadCharFromWiFly(LSR) & 0x01))
    {
      charincoming_data = ReadCharFromWiFly(RHR);
      m_printer->print(incoming_data,BYTE);
 
      booldone = parser->Parse(incoming_data);
      if (done)
      {
        ok = true;
        break;
      }
 
      timeOut = 0; //reset the timeout
    }
    else
    {
      delay(1);
      timeOut++;
    }
 }
 
 FlushRX();
 
 // disconnect TCP connection.
 WriteToWiFlyCR("close");
 WaitUntilReceived(COMM_CLOSE);
 
 ExitCommandMode();
 
 returnok;
}

Step 10: Programming Step 4: RGB LED

A simple library for setting the colour of an RGB LED. The library will fade between the colours as the world mood changes, and will flash if it is a significant change in mood.


*** update ***

If you find the colours look wrong, try removing the "255 -" from the analogWrite calls.
Thanks to shobley for finding this.
More info at http://www.stephenhobley.com/blog/2010/06/11/arduino-world-mood-light-using-twitter-and-wishield/

*** end update ***

/*
 The led is initially set to be currentColorID and over time will fade
 to desiredColorID with a time delay, fadeDelay, measured in ms, between
 each step. No effort is made to scale the step size for each rgb
 channel so each may not complete at the same time.
*/
voidLED::FadeTo(intdesiredColorID)
{
      // check for valid colorID
 if (desiredColorID >= NUM_COLORS ||
      desiredColorID < 0)
    {
      //logger.log("invalid Color id")
      return;
    }
   
 // get a local copy of the colors
 ColorcurrentColor;
 currentColor.r = Colors[m_currentColorID].r;
 currentColor.g = Colors[m_currentColorID].g;
 currentColor.b = Colors[m_currentColorID].b;
 
 ColordesiredColor;
 desiredColor.r = Colors[desiredColorID].r;
 desiredColor.g = Colors[desiredColorID].g;
 desiredColor.b = Colors[desiredColorID].b;
 
 booldone = false;
 
 while (!done)
 {
    // move each of r,g,b a step closer to the desiredColor value
   
    if (currentColor.r < desiredColor.r)
    {
      currentColor.r++;
    }
    elseif (currentColor.r > desiredColor.r)
    {
      currentColor.r--;
    }
   
    if (currentColor.g < desiredColor.g)
    {
      currentColor.g++;
    }
    elseif (currentColor.g > desiredColor.g)
    {
      currentColor.g--;
    }
   
    if (currentColor.b < desiredColor.b)
    {
      currentColor.b++;
    }
    elseif (currentColor.b > desiredColor.b)
    {
      currentColor.b--;
    }
 
    // write the new rgb values to the correct pins
    analogWrite(m_redPin, 255 - currentColor.r);  
    analogWrite(m_greenPin, 255 - currentColor.g);
    analogWrite(m_bluePin, 255 - currentColor.b);  
   
    // hold at this color for this many ms
    delay(m_fadeDelay);
   
    // done when we have reach desiredColor 
    done = (currentColor.r == desiredColor.r &&
            currentColor.g == desiredColor.g &&
            currentColor.b == desiredColor.b);
           
 } // while (!done)
 
 m_currentColorID = desiredColorID;
}
 
 

Step 11: Programming 5: Computing the World Mood

The mood light should be responsive enough to reflect what has just happened in the world, but it must not be so overly sensitive as to be susceptible to noise, and also not be too sluggish to be late in informing you of a big world event.  

The important thing is to carefully normalize and smooth the data, and to adjust the thresholds to give the right level of responsiveness and alarm. (i.e. it should flash when a headline news story
 happens and not when a TV show starts, GMT)
 
Emotion, mood, and temperament

Firstly, the "world's emotion" is calculated by searching twitter for tweets with each of the 7 mood types (love, joy, surprise, anger, fear, envy, sad) .

A measure of "tweets per minute" is used to calculate the current emotion. A higher number of tweets per minute suggests more people are currently feeling that emotion.

Emotions are volatile, so these short-lived emotional states are smoothed over time by using a "fast exponential moving average"
(see en.wikipedia.org/wiki/Moving_average#Exponential_moving_average)

This gives us ratios for the different moods.

Each mood ratio is then compared to a base line, a "slow exponential moving average", that I call the "world temperament".

The mood that has deviated furthest from its baseline temperament value is considered to be the current world mood.

The deviation is measured as a percentage, so, for example, if fear changes from accounting for 5% of tweets to 10% then this is more significant than joy changing from 40% to 45% (They are both a +5% in additive terms, but fear increased by 100% in multiplicative terms.)

Finally, the world temperament values are tweaked slightly in light of this new result. This gives the system a self adjusting property so that the world temperament can very slowly change over time.

These values in WorldMood.pde are used to adjust how sensitive the system is to information.

  • Do you want it to pick up when people are happy about a sport result or scared about the weather?
  • Or would you prefer to only track big events like natural disasters or terrorist attacks?
adjust accordingly...

#define emotionSmoothingFactor (0.1f)
#define moodSmoothingFactor (0.05f)
#define moderateMoodThreshold (2.0f)
#define extremeMoodThreshold (4.0f)

 
MOOD_TYPEWorldMood::ComputeCurrentMood()
{
 // find the current ratios
 floatsum = 0;
 for (inti = 0; i < NUM_MOOD_TYPES; i++)
 {
    sum += m_worldMoodCounts[i];
 }
 
 if (sum < 1e-4f)
 {
#ifdefDEBUG
    m_printer->print("unexpected total m_worldMoodCounts");
#endif// ifdef DEBUG
 
    returnm_worldMood;
 }
 
 for (inti = 0; i < NUM_MOOD_TYPES; i++)
 {
    m_worldMoodRatios[i] = m_worldMoodCounts[i] / sum;
 }
 
 // find the ratio that has increased by the most, as a proportion of its moving average.
 // So that, for example, an increase from 5% to 10% is more significant than an increase from 50% to 55%.
 
 floatmaxIncrease = -1.0f;
 
 for (inti = 0; i < NUM_MOOD_TYPES; i++)
 {
    floatdifference = m_worldMoodRatios[i] - m_worldTemperamentRatios[i];
 
    if (m_worldTemperamentRatios[i] < 1e-4f)
    {
#ifdefDEBUG
      m_printer->print("unexpected m_worldTemperamentRatios");
#endif// ifdef DEBUG
      continue;
    }
 
    difference /= m_worldTemperamentRatios[i];
 
    if (difference > maxIncrease)
    {
      maxIncrease = difference;
      m_worldMood = (MOOD_TYPE)i; // this is now the most dominant mood of the world!
    }
 }
 
 // update the world temperament, as an exponential moving average of the mood.
 // this allows the baseline ratios, i.e. world temperament, to change slowly over time.
 // this means, in affect, that the 2nd derivative of the world mood wrt time is part of the current mood calcuation.
 // and so, after a major anger-inducing event, we can see when people start to become less angry.
 sum = 0;
 for (inti = 0; i < NUM_MOOD_TYPES; i++)
 {
    if (m_worldTemperamentRatios[i] <= 0)
    {
#ifdefDEBUG
      m_printer->print("m_worldTemperamentRatios should be initialised at construction");
#endif// #ifdef DEBUG
 
      m_worldTemperamentRatios[i] = m_worldMoodRatios[i];
    }
    else
    {
      constfloata = m_moodSmoothingFactor;
      m_worldTemperamentRatios[i] = (m_worldTemperamentRatios[i] * (1.0f - a)) + (m_worldMoodRatios[i] * a);
    }
 
    sum += m_worldTemperamentRatios[i];
 }
 
 if (sum < 1e-4f)
 {
#ifdefDEBUG
    m_printer->print("unexpected total m_worldTemperamentRatios total");
#endif// #ifdef DEBUG
    returnm_worldMood;
 }
 
 // and finally, renormalise, to keep the sum of the moving average ratios as 1.0f
 for (inti = 0; i < NUM_MOOD_TYPES; i++)
 {
    m_worldTemperamentRatios[i] *= 1.0f / sum;
 
#ifdefDEBUG
    m_printer->print("temperament ratio: ");
    m_printer->println(m_worldTemperamentRatios[i]);
#endif
   
 }
 
#ifdefDEBUG
 // debug code - check sum is 1.
 sum = 0;
 for (inti = 0; i < NUM_MOOD_TYPES; i++)
 {
    sum += m_worldTemperamentRatios[i];
 }
 
 if (sum > 1.0f + 1e-4f || sum < 1.0f - 1e-4f)
 {
    m_printer->println("unexpected renormalise result");
 }
#endif// #ifdef DEBUG
 
 returnm_worldMood;
}
 
 

Step 12: Building the Box

Build an acrylic box ala this Instructable:

www.instructables.com/id/LED-Cube-Night-Light/

Step 13: Enjoy!

Some possible extensions include:
  • Making it multilingual and not just English speaking places. 
  • Perhaps just associating with a keyword, for example every tweet must contain the word "Obama", then you could gauge public opinion on just that subject.
  • Location specific. Perhaps you just care about your town or country. Twitter allows you to use the geocoding to do this.
  • Make it tweet what the world mood is so as to complete the circle
  • Ability to connect to it from a computer to see what keywords people are so emotive about.

I am very interested to hear any comments, corrections or questions. Please do contact me, if you so wish.

Comments

author
jmccauley2 made it! (author)2012-08-03

The world has changed a bit since this program was first posted.  I finally have my mood light working, but it wasn't always easy.

First, make sure you buy an Arduino Duemilanove.  Yes, there are newer devices out there.  No, you do not want them.  They are all supposed to use the same code, but this code is sensitive to the timing on the Duemilanove.  Use another board and you'll be fighting issues all the way.  After over a year of this, I just switched and I'm glad I did. 

Second, the Twitter world moves much faster than it did.  30 tweets of the original terms go by in less than a second.  Since everything is computed in tweets per minute, this obviously won't do.   Everything registers off the scale. 

You'll have to substitute this code in TwitterParser.cpp to compensate. 

#ifdef DEBUG
    m_printer->println("assuming tweet was from yesterday");
#endif // #ifdef DEBUG
  }

  long seconds = time1 - time2;
  return seconds / 60;
}


with

#ifdef DEBUG
    m_printer->println("assuming tweet was from yesterday");
#endif // #ifdef DEBUG
  }

  long seconds = time1 - time2;
  return seconds;
}


I also had to modify my LED.cpp file to make sure colors showed up properly.  In particular, I had to adjust the array values in the beginning to accurately show colors with my LED -- apparently my red isn't as bright as the blue and green and so full value on all three skews the colors.   I added the following text to the WorldMood.cpe to make it easy to test light colors -- right after the device starts up, it triggers the light to the values specified.  Value range goes from 0 to 255. 

          //a way to quickly test colors and connections
          analogWrite(redPin, 255);
          analogWrite(greenPin, 35);
          analogWrite(bluePin, 0);


It goes immediately after this line:

         LED led(Serial, redPin, greenPin, bluePin, fadeDelay);
 

I am still adjusting my search criteria and making minor adjustments, but this solves most of the "out of the box" issues I've seen people experience. 

author
jamesbrosuk made it! (author)jamesbrosuk2015-02-09

Excellent update/summary interesting that the tweets per second needs to be adjusted, i wonder if it would work on other social media platforms?

thanks for putting the time in to update.

author
RandomMatrix made it! (author)RandomMatrix2012-09-27

Yes, Arduino, Twitter and the World's Mood have changed a lot since May 2010. I"m afraid this is not an active project of mine, and I rely on great comments like this to keep this 'ible current.

author
jmccauley2 made it! (author)jmccauley22014-04-17

Hey, the new API requires OATH and this project no longer works. It's a little beyond my ability to fix it, but would you like to work with me to provide updated code? This seems to be a fairly popular project and it's really cool when it works.

author
jmccauley2 made it! (author)jmccauley22013-06-30

Also, as of today, the project no longer works.

Twitter has changed its API and now the query returns errors.

author
rachitmehrotra made it! (author)rachitmehrotra2014-04-17

Is there a way i can still make it ?

would be a great help if you can sugest a way to do this ?

author
jmccauley2 made it! (author)jmccauley22014-04-17

Unfortunately, no. I've not been able to get it to interface with the new API.

author
Kali Jennifer made it! (author)2017-04-07

I love this, and would love this next to my bed! One thing though. To keep cost down, how hard would it be to program the Arduino to read the information straight from the computer via USB connection

author
MichaelM1003 made it! (author)2016-11-21

Sir do you have any codes for the modern twitter mood light?

author
Jonathanrjpereira made it! (author)2016-01-29

Really Cool Instructable. I featured this Instructable in one of my collections: https://www.instructables.com/id/8-Twittering-Thingamajigs/

author
certainsounds made it! (author)2015-11-17

Please keep us updated, good luck!

author
NdegwaE made it! (author)2015-11-17

Can this still be made or has there been updates to Twitter that will make this accomplishable? Should we use the last change you made ?

author
RegineT1 made it! (author)2014-10-13

Is there anyone out there who has an idea of making code for aduinos? I'm having a hard time doing my project. I have no idea of what code to use. My project is about a gh-718c mini PIR motion sensor detecting my arm or hand. If the motion sensor detects that my hand is low, the light or LED will dim, and if it detects my hand on a high position, the LED will bright up. Please please. Help please. Thanks for the reply!

author
RegineT1 made it! (author)RegineT12014-10-13

By the way, I'm using an arduino uno. Thankyou. Please reply.

author
EndrikE made it! (author)EndrikE2015-10-24

You'll have more luck with a raspberry pi

author
bscoville made it! (author)2015-10-22

You're awesome, thanks for doing this!

author
dmeeker made it! (author)2012-12-18

I am not even sure where to start with this comment!  Bottom line, I got this running on newer hardware with a newer IDE.

Long story short: I started working on this using the 1.01 Arduino IDE, an Arduino Uno and an LED "ball" which I found referenced here:
http://hackaday.com/2012/09/08/hacking-a-floating-rgb-led-decorative-ball/

I liked the look of that ball, and thought to myself "what can I do with it that is cool". I bought one, and then decided to see where I could take this instructable.

Like many of you, I ran into all sorts of errors when trying to load up the sketch and libraries to make it work.

After hacking away at it for about 8 hours over the last week, I've gotten it compiling and running on the new version of Arduino, with an Uno and not the Duemilanove. Once I get things a bit cleaner, and I document what changes I actually made,

I will post the full code somewhere and add it to this comment thread.

The quick overview:

1) Wired up my LED. I am using the LED array that came inside the ball. I also wired up a Sparkfun RGB LED Breakout to see if it would work across both. (https://www.sparkfun.com/products/10111). It does. I had to make the code change referenced in the comments and on this blog to make the colors match best with the Sparkfun LED kit. (http://www.stephenhobley.com/blog/2010/06/11/arduino-world-mood-light-using-twitter-and-wishield/)

2) In each of the libraries that were included, I had to update the code to replace: #include "WProgram.h" with #include

3) The timing and error handling are still pretty screwed up, and not where they should be. I am going to work on that next. The code will execute, and throw some errors, but in true arduino fashion, if you let it run, it will power through things and get to the next step.

4) Pay attention to the comments in "step 6", download the code: I HAD to do this. If you have a newer board then you may need to change this struct SPI_UART_cfg SPI_Uart_config = {0x50,0x00,0x03,0x10}; to this: struct SPI_UART_cfg SPI_Uart_config = {0x60,0x00,0x03,0x10};

5) Once I did those things, I got it to compile and pushed to the Arduino.

It would go out and attempt to make a request to twitter, but I'd get a 404 from twitter. This caused me to scratch my head for a day, contemplate rewriting the entire WIFLY library and then it hit me...

The http request had something wrong with it. In the worldmood sketch, you must look at the lines where you define the search request and make a small change, appending HTTP 1.0 to the end of each.

It WILL NOT WORK UNLESS YOU DO THIS.

These lines:

prog_char string_1[] PROGMEM = "GET /search.json?q=\"happiest\"+OR+\"so+happy\"+OR+\"so+excited\"+OR+\"i'm+happy\"+OR+\"woot\"+OR+\"w00t\"&rpp=30&result_type=recent";

Should become:

prog_char string_1[] PROGMEM = "GET /search.json?q=\"happiest\"+OR+\"so+happy\"+OR+\"so+excited\"+OR+\"i'm+happy\"+OR+\"woot\"+OR+\"w00t\"&rpp=30&result_type=recent HTTP/1.0";

I think that is about it, but I need to go back and look at everything to ensure that there weren't other items necessary.

The one item I haven't addressed yet is that when left as-is, the twitter search for "envy" still fails. I can't understand why, but I haven't taken much of a look at it yet.

To get past this, I changed the search string from the original to this:

prog_char string_4[] PROGMEM = "GET /search.json?q=\"i+wish+i\"+&rpp=60&result_type=recent HTTP/1.0";

I basically got rid of the search and just put in "I wish I" for the time being.

So, there you have it. If anyone wants to continue to improve, the army of "modern" Arduino hackers can start from this modified code that indeed works. (yay!)

https://www.dropbox.com/s/ewrbhqphya64dmc/World_Mood_Nowlab.zip

Enjoy, and please share comments as you make progress.

Dave

author
mikeinfinity made it! (author)mikeinfinity2013-09-05

Hey! I tried going to the dropbox link you gave, but the file is missing. Do you still have it up somewhere? Thanks in advance!

author
wdsparrow made it! (author)wdsparrow2015-08-20

I would love this code also. I prefer the "modern" arduino.

author
GillH1 made it! (author)2015-06-11

Great looking project, any chance it will get updated to work with the current Arduino IDE?

author
Isaac Kondoh made it! (author)2015-05-01

We tried replicating this on the Arduino Uno, and all of the code and libraries have been correctly added to the IDE, but we are currently getting the following error:

The libraries are correctly added as stated above. Any help is appreciated thanks.

Capture.PNG
author
davidbarcomb made it! (author)2014-11-27

This is great. I have to try doing this

author
RobertGoldring made it! (author)2014-08-20

excellent idea, i think the search terms could be 'work in progress' but other than that a really cool idea.

author
dmagryta124 made it! (author)2014-01-23

How can I make this without using the WiFly module?

author
jelimoore made it! (author)jelimoore2014-04-19

I second you, too.

author
rachitmehrotra made it! (author)2014-04-17

So can this still be done ?

If yes , do i need to buy the same hardware?

author
laptopdude90 made it! (author)2013-12-06

What would be better is to have a "mood server" that calculates the mood, and have the Arduino connect to it.

Since all the heavy work runs on the server, the mood could be updated much faster than the Arduino, and it would save battery life.

I'd write the server software myself, except I suck at interfacing with Twitter... :P

author
bkang1212 made it! (author)2013-11-30

hi so i downloaded the code but it doesn't work with processing or arduino because i have updated versions. Is there an updated version of the code in .pde and .ino format

author
luckyvictor made it! (author)2013-09-26

It is a very nice project however, you constantly search for news, does it consume a lot of battery?

Can you actually make it passively receive the data in a periodical manner? or wait for a device or app to send a 'mood' to it whenever this device receives a twitt? I know it is always possible, but what could be the implementation please? do you have any idea or example?

As I would like to make one that ideally be waiting and be triggered so to consume the minimum power

author
facepalm made it! (author)2013-07-31

When running this code I got errors saying "WProgram.h" did not exist. The solution is to replace any instance of

#include "WProgram.h"

with

#if ARDUINO >= 100
#include "Arduino.h"
#else
#include "WProgram.h"
#endif

(source: http://makezine.com/2011/12/01/arduino-1-0-is-out-heres-what-you-need-to-know/)

author
SaranshMESON made it! (author)2013-04-21

can we make this using a wifly module rn xv wifly shield

author
damianest made it! (author)2013-02-05

Being fairly new to Arduino and EE / Physical Computing, I am not at the stage where I feel comfortable making my own Circuit Boards. Would anyone mind posting how to wire this via breadboard?

author
invisiblelight386 made it! (author)2012-12-10

I would love to make one of these... however $70-$100 for the WiFly shield is way too much... any other ideas that would substitute?

author
pabhilash made it! (author)pabhilash2013-02-04

I'm in the same dilemma :( Wifly shields are expensive where i come from. I suppose an Ethernet shield would help?

author
pabhilash made it! (author)2013-02-04

So, i have a question. How could i accomplish this project with an Ethernet shield instead of the Wifly shield? (I ask because the Wifly shield is rare and expensive where i come from..)

author
pwnag3 made it! (author)2012-12-10

a wonderful idea

author
praveenkm made it! (author)2011-12-01

Hi,

I compiled this code and getting me lot of errors..
plz help me!!!

Untitled.png
author
h00k made it! (author)h00k2012-11-11

This looks like a fantastic idea.  I'll probably adapt the base and use wood, and maybe allow for USB power instead of the 9V battery as well.  Also, perhaps an on-off switch on the back may be something I'll do.

How long are you experiencing the 9V battery last?

Thanks for this, it's going to get me working with acrylic, something I haven't done yet.

author
guitarfreak1513 made it! (author)guitarfreak15132012-06-30

did you ever get the problem fixed? i had the same problem

author
jmccauley2 made it! (author)2012-09-27

Something else to keep in mind, the Twitter API is about to change. When it does, it appears that after 5 March 2013, this code will no longer work and will require an update.

If I figure out how to make it work again, I'll post the code here. If not..... I guess I'll hope someone else figures it out.


https://dev.twitter.com/blog/changes-coming-to-twitter-api

author
samirsky made it! (author)2012-08-12

The Olympics stole your idea!
http://www.metro.co.uk/olympics/907250-twitters-olympic-mood-beamed-to-world-from-edf-energy-london-eye

I was still excited to see the concept on the international stage.

author
RandomMatrix made it! (author)RandomMatrix2012-09-27

Ha! Not the first nor the last, i suspect :D

author
citronella made it! (author)2012-06-27

Has anyone tried to make this with processing?
Or perhaps with an ethernet shield? I'm really interested!

author
dava_2 made it! (author)2012-05-07

Has anyone tried to make it work over USB cable, without Wifi?

author
Lachee made it! (author)2012-05-06

is it possible to use a usb with a processing script?

author
ilpug made it! (author)2012-01-31

I have nowhere near the experience to make one of these things, is it possible that I could buy one from someone, and if so, for how much?

author
lance_r made it! (author)2010-05-17

Much appreciated I tried to take it on myself but got hung up on html parsing haha need to brush up on my c++

author
samirsky made it! (author)samirsky2012-01-20

check out:
http://www.mathworks.com/matlabcentral/trendy/plots/644

It only updates once/day, but might be interesting to monitor over time.

author
samirsky made it! (author)2012-01-20

Created a plot to track trends (1/day) based on this great project:
http://www.mathworks.com/matlabcentral/trendy/plots/644

author
madmazelle made it! (author)2011-04-08

Hey ! nice project, very inspiring. btw wanna ask (i'm new with this) is it possible to use ARDUINO ETHERNET SHIELD instead of wifly shield?

will the script be different?

About This Instructable

265,913views

735favorites

More by RandomMatrix:Twitter Mood Light - The World's Mood in a Box
Add instructable to: