Introduction: X-Track - Wireless Music Visualization and Tracker
X-Track is a prototype wireless device which connects you to the music, by providing entertainment with its bright beat-synched LEDs and tracking your moves so that you can later learn how much you enjoyed the night, and which parts were especially moving you.
Remote wireless visualizations fully synced to the beat:
* Supporting dynamic control of song playback.
** You can adjust tempo and the visual will always be on the correct beat, and you can even scratch !!
* DJ software automatically analyses song bpm (beats per minute), aligns the beats with the song, and send the high-resolution beat phase messages to external control application.
* The external application on PC broadcasts beat time information to all gadgets in the scene.
Live crowd tracking:
* By making use of the additional sensors (accelerometer), each gadget sends crowd information back to the main PC for logging and any other purposes.
Small form-factor:
* The gadget prototype comes in a small package (11x9x3cm). Great for hand-held interaction or wearing around your neck. Future makes can come in smaller form factors to have a less constraint on your fun.
Support for multiple remote gadgets:
* You can have a a high number of synced displays in a single room, and you can gather sensor data from all (the number of devices is limited by the bandwidth of radio communication. Each device sends sensor readings back to the main computer. The practical limit for efficient data acquisition is not yet studied.)
This instructable was made as part of the final project requirement in the CS graduate course "Tangible Interactive Computing" at the University of Maryland, College Park taught by Professor Jon Froehlich. The course focused on exploring the materiality of interactive computing and, in the words of Hiroshii Ishii, sought to "seamlessly couple the dual worlds of bits and atoms." Please see http://cmsc838f-f12.wikispaces.com/ for more details.
Step 1: Hardware & Wiring Info
Note: All images appear in order of description below.
Arduino FIO
* http://arduino.cc/en/Main/ArduinoBoardFio
* https://www.sparkfun.com/products/10116 ($25)
Microcontroller: ATmega328P
Clock Speed: 8 MHz
Operating Voltage: 3.3V
Input Voltage: 3.35 -12 V
Pro's:
* Includes an Xbee socket already (saves $10)
* Polymer Lithium Ion Batteries from sparkfun provide 3.3V, no step-up component (3.7 to 5V) needed (saves $6)
* Includes battery charger using USB port (saves 10-15$)
Con's:
You'll need a 3.v to 5.v stepper for the display (costs 5$)
No direct programming interface available. You need to use FTDI cables/breakout boards.
The longest dimension is slightly bigger than the LED matrix
---------------------------------------------------------------------------------------------------------
LED Matrix - Serial Interface - Red/Green/Blue (The Visuals)
* https://www.sparkfun.com/products/760 ($59.95)
* Communicates through SPI serial input protocol
* Runs on 5V (if you don't raise up the voltage from 3.3 to 5V, you get very pale colors, and an unstable refresh rate)
* Current: 120mA (typical) 275mA (max)
* Dimension: 2.38x2.38"x0.78''
Wiring: (Using SPI Interface)
* VCC(5V)<->5V
* GND<->GND
* MOSI<->PIN11
* CS<->PIN10
* SCLK<->PIN13
---------------------------------------------------------------------------------------------------------
Triple Axis Accelerometer Breakout - ADXL335
* https://www.sparkfun.com/products/9269
* "Low noise and power consumption (320uA)"
* Sensing range: +/-3g.
* Power in between 1.8-3.6V
* Bandwidth: 50Hz
Wiring:
* 5V/GND : as usual
* X<->Analog Pin 0, Y<->Analog Pin 1, Z<->Analog Pin 2
---------------------------------------------------------------------------------------------------------
XBee 1mW Trace Antenna Series 1 (x2) (Wireless Communication)
* https://www.sparkfun.com/products/11215
* Point-to-point (PTP) & point-to-multipoint (PTM) radio running the [[http://en.wikipedia.org/wiki/IEEE_802.15.4|IEEE 802.15.4]] protocol
* This is a simple, cheaper XBee module with enough range, easy to control (than Series 2), allows point-to-point and broadcast communication.
* I chose trace antenna since the amount of space used was a basic concern, and trace antenna is sufficient.
* The radio receiver should work inside the plastic box. "The radios do not have any issue radiating through plastic cases or housings, and so the antennas can be completely enclosed in those types of situations." Info
Wiring:
* Just place it in the XBee socket of FIO!
---------------------------------------------------------------------------------------------------------
Vibration Motor
* https://www.sparkfun.com/products/8449
* Vibrates on the beat, or controlled manually in real-time!
* 2-3.6V operating range.
* Current: 75mA (note that fio provides a max of 40mA per pin)
* In the current prototype, it sticks inside the case.
---------------------------------------------------------------------------------------------------------
FTDI Basic Breakout - 3.3V
* https://www.sparkfun.com/products/9873 ($14.95)
* For programming the micro-controller using USB connection
* Works on 3.3V
* "Uses SMD 6-pin header on the bottom"
* "This board will auto reset any Arduino board that has the reset pin brought out to a 6-pin connector"
* "This board has TX and RX LEDs that allow you to actually see serial traffic on the LEDs to verify if the board is working"
* Additional USB cable required (A<->mini B) .
Wiring:
* Only necessary when updating the program on FIO, not part of the remote system. Currently using 6-pin male headers.
---------------------------------------------------------------------------------------------------------
XBee Explorer Dongle
* https://www.sparkfun.com/products/9819 ($24.95)
* Allows you to plug-in XBee hardware to your computer. The computer will see the device through a COM channel.
* Use X-CTU software to configure your XBee. More details are presented in further steps
---------------------------------------------------------------------------------------------------------
Polymer Lithium Ion Battery - 2000mAh
* https://www.sparkfun.com/products/8483 ($16.95)
* Very slim, light weight
* 3.7V at 2000mAh
* Battery includes built-in protection against over voltage, over current, and minimum voltage.
Note: Expected power consumption of proposed design:
* 270mA (max) LED Matrix
* 50mA XBee unit
* 30mA (max) arduino
* Total: 350mA (worst case), 2000mAh should be enough for about 6hrs of unplugged operation!
---------------------------------------------------------------------------------------------------------
Project Case
* https://www.sparkfun.com/products/8601 ($9.95)
* Size: 3.95x3.16 (inch) internal (about 10cmx8cm) / slightly larger on external
* Height: 0.75'' internal, 1'' external
---------------------------------------------------------------------------------------------------------
AKAI LPD8 Pad MIDI Controller [[ Optional ]]
* http://www.akaipro.com/lpd8
* It is used both to control DJ software AND the X-Trak PC Control Program. You can use any MIDI mapping and customization for your own setup.
The controls are mapped to Preset 4, which by default send MIDI messages on Channel 4.
Button Index Layout:
5 6 7 8
1 2 3 4
Program Change messages: 1-8 => Select the corresponding display type.
Pad messages:
1 => Vibration on beat (toggle on/off)
5 => Force-Vibration (momentary on/off)
---------------------------------------------------------------------------------------------------------
Other hardware requirements
Wires, soldering material, mini breadboard (https://www.sparkfun.com/products/7916)
Step 2: Soldering Arduino FIO
Step 3: Getting the Pieces Together - Cutting & Drilling
Step 4: Boxing the Pieces Together
Step 5: X-Bee Configuration
In this project, XBee configuration is static, and based on Series-1 devices.
You can Use X-CTU software to update the configuration of X-Bee's. You can download the free software from this link. The configuration is only required once. You need to select one of your X-Bee's as coordinator, and the rest (as many as you'd like) needs to be configured as end devices. The coordinator will be attached to the PC, and the rest will be attached to Arduino FIOs.
First, you may want to adjust the baudrate of XBee's. Each XBee should have the same baudrate, and your Arduino code and PC code should also match this value! The testing configuration uses 57600kbps, and gives a lot of room in the channel for additional data. Remember that the default configuration is different, and you need to set correct baud rate to communicate to X-Bee's, and when you change this option, you need to change the software's serial port baudrate to re-connect with the X-Bee module. So, in addition to the configuration below, you need to set BD to 6 (57600) for all the devices.
COORDINATOR (PC)
AP=2 // Put XBee in API mode
CE=1 // Make this radio the Coordinator
MY=1234 // Set the address of this radio to any arbitrary two byte value
ID=3332 // Set the PAN ID to a two byte arbitrary value. Each XBee in the network must have this same value.
CH=0C // Both radios must have the same Channel and PAN ID to communicate. Chose any channel no appropriate for your case.
END DEVICE (Arduino)
AP=2 Put XBee in API mode (escape control bytes)
CE=0 Make this radio the end device
MY=5678 (or any other unique ID for each of the Arduino's)
ID=3332
CH=0C
Note: The image attached is NOT a direct representation of the settings listed above. Please follow the written instructions.
Note2: While not an X-Bee configuration option, there was one additional step done to make sure the bandwidth is used efficiently: By disabling ACK messages. Basically, we don't need to know if the displays receive the beat phase messages. Lost packets are not important since new phase info is continuously sent. So, TxRequest16 sent by the PC includes TxRequestBase.Option.DISABLE_ACK, and the frame no is set to 0. The same includes to Arduino implementation, which are described later in this instructable. Also, note that this requires that messages are sent in asynch mode. It fits the send&forget mode of communication of this live visualization and tracking software.
Step 6: DJ Software (Traktor) Setup
You need to make adjustments in the DJ software to be able to send high resolution beat phase information.
The configuration steps below are for Traktor DJ software (http://www.native-instruments.com/#/en/products/dj/traktor/). Other software might require different configurations, or may not provide such beat phase information over MIDI channels.
1) Open up configurations panel.
2) Select Configuration Manager on Left Menu
3) Create new device : MIDI Generic, rename this device if you'd like to.
4) Set the output port to your virtual MIDI output port (the port that your MIDI application will read the beat-phase messages).
5) Add new assignment:
Device mapping: You can choose any Midi channel / control code you'd like. In this example, I'm using Ch02.CC.024 to communicate beat phase information.
Modifier Condition: None
Controller range: -0.5000 0.50000
Midi Range: 0 32
You can adjust midi range to your taste, but I find 32 to give enough resolution. The tracking is extended over 4 beats by the MIDI software,so that actual range of data is 256 (a single byte for 4 beats).
Step 7: Arduino Software
Language: Arduino
XBee library: http://code.google.com/p/xbee-arduino
Most of the code is for generating different types of patterns and animations for the display (clearing pixels, drawing lines,circles, patterns, etc)
You can download the Arduino code from here. It's only a single file. You can use the official Arduino software to compile & upload the code to the hardware.
Notes about upload process:
* You need to detach X-Bee module when you are making a wired connection to Arduino. That means, every time you need to upload the code, you need to unplug/plug a component in your system.
* This can be cumbersome, and you may want to use wireless program upload to X-Bee. Some instructions are below, but it requires additional configuration of your X-Bee devices, which may conflict with the configuration listed earlier. I couldn't get the samples running in my first try, but it might be possible, and it can save your from plug/unplug cycle for each update to Arduino code.
Wireless Programming of Arduino FIO:
When XBee socket is attached, you cannot program Arduino boards over USB (since they use the same serial channel) As a temporary fix, I first "loosely" connected the XBee to the board, so that it is easy to take it out when new program is uploaded.
There are two nice tutorials that show how to make wireless programming enabled for FIO
* http://arduino.cc/en/Main/ArduinoBoardFioProgramming ()
* http://www.sparkfun.com/tutorials/122
Additional resources:
http://answers.oreilly.com/topic/2458-how-to-connect-an-arduino-to-an-xbee-radio/
http://bildr.org/2011/04/arduino-xbee-wireless/
Note: The attached image is just for display purposes. It does not include the related source code. To learn how to program your Ardunio, please follow other tutorials.
Step 8: PC Software
Language: Java
XBee library: http://code.google.com/p/xbee-api/
MIDI Library: http://docs.oracle.com/javase/1.4.2/docs/api/javax/sound/midi/package-summary.html
You can access the PC software here. The folder includes eclipse project configuration as well, but you can use lean Java compilers to build & run the software.
The video in this step shows how the custom (console) application can retrieve information from the music software. The beat-phase information is printed for debugging and demonstration purposes in this video. Notice how scratching patterns can be easily tracked, and then send to all remote devices.
Here are the basics of the implementation:
* TraktorReceiver : Listens to messages on the MIDI port Traktor sends its output messages. It basically only listens to BeatPhase messages for now.
* LPD8Receiver : Listens to messages from the MIDI port that your Midi controller is attached to (in my case, it is Akai LPD8). It currently can change the visualization type, and vibration setting (auto on the beat/ manual with a button press)
* AutoMidiTask : An implementation of TimerTask, which can be used to generate beat-phase messages even if Traktor software is not running, thus no messages are received over MIDI channels from a music software. You can use it for debug purposes.
* ControlPC : Includes the main method, which opens MIDI devices, creates messages, and waits for any keyboard input for manual control of X-Bee messages.
* XBeeHelper : Controls access to XBee hardware. Opens up the serial communication to XBee device, creates a log file to start recording acceleration messages that were received, and broadcasts messages sent by other (MIDI) listeners in the software.
* DumpReceiver and MidiCommon : These are part of jsresources.org implementations, copyrighted by Florian Bomers and Matthias Pfisterer. These are helper methods. DumpReceiver can be used to print out the MIDI messages as they are received and used to debug the MIDI system.
Additional Resources:
http://code.google.com/p/xbee-api/wiki/DevelopersGuide
http://www.jsresources.org/examples/MidiInDump.html
