TapTunes - an Tap-based Interface to ITunes

As part of my Human-Computer Interaction Term Project, I whipped up a tiny program that would allow users to control iTunes by tapping various patterns on a piezoelectric sensor.  The foundation of this project was the KnockSensor tutorial provided on the Arduino website.  This tutorial was then extended to meet the requirements needed by the system.

My system is comprised of the following components:
• 1 Arduino (the one used is an Arduino Mega 2560)
• 1 Piezoelectric Sensor
• 1 Mega-ohm Resistor
• Breadboard containing 3 LEDs
• 1 USB A-to-B Cable
• 1 Laptop running Visual Studios 2010 and iTunes

The 1Mega-Ohm resistor is present to prevent the piezoelectric sensor from damaging the input port of the Arduino.  The LEDs provide user feedback.  The orange LED blinks whenever the user taps the sensor, thereby showing that the system is actively recording the tap.  The red LED blinks when the user taps a pattern that is not recognized.  Likewise, the green LED blinks when a user’s pattern of taps is accepted by the system.
The system both utilizes and is powered by a USB cable connecting the Arduino and the laptop.

SOFTWARE (ARDUINO)

The software that runs on the Arduino is a watered-down version of C++, inspired by the programming language Processing.  It was written using the standard Arduino IDE which is available online.

The program is a relatively simple one.  The system contains an array of stored tap patterns which can be matched.  The system spins in a loop, waiting for the first tap to be detected.  Once this occurs, the system marks the current time, and waits for the next tap.  For each consecutive tap, the system stores the time interval between it and the previous tap.   This process continues until either the maximum number of taps is exceeded, or no tap has been detected for one second.  The taps are then normalized.  This allows for taps performed at different tempos to be matched to the same pattern.  Normalization is performed by dividing each stored interval by the longest interval and multiplying by 100, thereby transforming each interval to an integer between 1 and 100.

Tap patterns have three major criteria to meet if they are considered to match a stored pattern:

1. Both patterns must contain the same number of taps
2. An user interval cannot deviate by more than 25% from the corresponding stored interval
3. The average of all intervals cannot deviate by more than 15% from the average of the stored intervals.

If all these conditions are met, the system accepts the pattern.  The system then transmits the index of that pattern as a single byte over the serial connection provided by the USB cable to a listening program running on the laptop (in our case, a C# program interfaced with iTunes).  Once the byte has been transmitted, the system returns to waiting for a new user tap pattern.

The tap patterns currently implemented are:

• A single tap
• Two taps
• Three uniform taps
• Four uniform taps
• One slow tap followed by two fast taps
• Two fast taps followed by one slow tap
• The “Shave and a Haircut, Two Bits” pattern

This list of patterns is easily customizable and extendable once one becomes familiar with how patterns are represented.

SOFTWARE (LAPTOP)

I chose to write the laptop-side application in C# because I have had previous experiencing writing custom interfaces with Arduinos in C#.  This could easily be implemented in other languages, as long as the language supports serial communication over COM ports.

The C# program uses Microsoft’s XNA framework which, while normally used for developing games, provides a nice, loop-based program flow suitable for interfacing with Arduinos and displaying a visual GUI.

The program begins opens a serial connection to the Arduino and creating an interface to iTunes.  Once these are established, the program continually polls the serial connection for bytes.  If there are bytes to be read, the program reads each as an integer and executes the corresponding command.  For example, if the byte contained the number 1, then the program toggles whether iTunes is playing or paused.  Since a byte contains 32 values, a total of 32 individual commands can be issued from the Arduino to the C# program.  In this basic demo, only six commands are actually implemented.

The received numbers, commands, and associated patterns are as follows:

0. Toggle visualizations – “Shave and a Haircut, Two Bits”
1. Toggle Play/Pause – One tap
2. Next Song – Two taps
3. Previous Song – Three uniform taps
4. Fast-Forward – One slow tap followed by two fast taps
5. Rewind – Two fast taps followed by one slow tap
6. Toggle shuffle – Four uniform taps

C# interfaces with iTunes through its COM API.  My program also displays a basic GUI that provides information on iTunes’s state, the position within the current song, and the current track’s name, artist, and album.