TCP/IP stack and base network functions
Microchip's 'free' TCP/IP stack provides all the network functions we need to exist on a home network and grab data from Twitter. The stack is open source and free-as-in-beer, but the Microchip license prohibits distribution. Due to licensing issues, we're only putting our public domain source code in the project Google Code SVN, learn how to download and compile the source here.
The stack has a Dynamic Host Configuration Protocol client that automatically configures the network settings using the DHCP server on your local network. The #twatch requires a DHCP server, but the vast majority of networks and routers have this enabled. The IP address, mask, gateway, and first DNS server are displayed on the LCD screen until valid Twitter data is available.
The stack also includes Microchip's announce server. When the IP address is acquired by DHCP, the #twatch announces its IP address with a broadcast packet to all computers on the local network. Use the MCHPDetect.exe utility in the project archive to view these packets.
Finally, we included a ping (IMCP) server. Use any ping client to check if the #twatch is alive on the network.
Twitter TCP client
The Twitter trend following program is a simple TCP client, similar to a web browser, that pulls data from web servers. Twitter's API will give us data in a variety of formats. We used the light-weight JSON format because it's easiest for the low-power PIC chip to decode, check out JSONView if you use Firefox.
After the #twatch automatically configures network settings, the Twitter TCP client takes control and grabs the current trending topics. It searches through the this JSON datafeed and looks for the "name" tag. Up to 10 trending topics are copied into a 225byte buffer. A separate array stores the end position of each topic in the buffer so we can retrieve the topics in the next step.
Next, the #twatch searches Twitter for 2 tweets for each topic. It appends each topic to the end of the Twitter JSON search url, special characters like spaces and punctuation are URL encoded.
The TCP client parses through the search results and looks for tweets that follow the "text" tag. Tweets have multiple layers of encoding. We decode HTML reserved characters like ampersand (&) and quotes (") because the LCD screen can display them. We remove UTF8 international characters because the HD44780 LCD doesn't have them in its character set.
The parsed, decoded tweets are stored in a 2100byte buffer, an additional array marks the beginning and end of each tweet in the buffer. RAM space was a big problem on the 18F67J60 chip, it only has about 4000bytes total, but the 2100byte buffer seems big enough to handle 20 average-sized tweets. We took special care to protect against out of memory problems, and we tested the client under reduced RAM conditions to ensure that it fails gracefully during errors.
Twitter is well known for its occasional down-time. If the #twatch can't connect to Twitter, it displays a connection error message and the retries twice. If it can't connect after three tries, it waits five minutes before trying again. This gives Twitter a chance to fix their problems without being hammered by #twatch queries.
The #twatch grabs fresh trend and tweet feeds every five minutes. Twitter puts a limit on the number of queries a client may make, so be careful about refreshing more often. Twitter allows 150 trending topic updates per hour, and "significantly more" search queries.
Network LCD backpack mode TCP server
The #twatch can also display system status information from programs like LCD Smartie. The #twatch has a TCP server on port 1337 that accepts Matrix Orbital formatted commands. This also provides control over the LCD backlight. We'll show you how to redirect LCD Smartie from a COM port to the #twatch TCP server in part two of our #twatch article.