The Carnegie Mellon HCI gang is at it again, this time taking on the serious assignment of making a thing for PLAY. The Royal Play that is. The thing that makes life worth living.

This time, we've combined our nerd-ness to bring some of your favorite movie sound effects to life. The " Augmented Hyper-Reality Glove" uses flex and tilt sensors to send signals to a Wave Shield from Adafruit Industries. The Wave Shield lets you trigger uncompressed sound files directly with Arduino, eliminating the need to call files from your computer.

When you punch with the glove you get a "punch" sound, when you make a pistol shape, a "gunfire" sound, etc. The possibilities are limited only to your imagination, and your world will be augmented by the sounds you normally only hear in your head or make yourself (legitimizing those embarrassing moments when other people hear you doing it).

Team: Becca Chen, Zack Jacobson-Weaver, Jimmy Krahe & Spencer Sugarman from Activating Objects with Eric Paulos and developed at CodeLab.

Let's get started...

Step 1: What You'll Need

Tools: Soldering Equipment, Wire Cutters, Scissors, Needle and Thread, Arduino board (here's a link for everything you need to use it), Lady Ada Wave Shield for Arduino, Solderless Breadboard, Hot Glue Gun, SD Card Formatter with one SD Card, your favorite Mac or PC.

Materials: One Glove, Solder (and some skills, the Wave Shield is a marathon soldering session), Conductive Thread, Electrical Tape, Anti-Static Sheeting, Regular Sewing Thread and a Needle (and a thimble for you sensitive types), Aluminum or Copper Foil Tape, Electrical Wire ( there are two kinds used here: Solid and Stranded. You'll see why later), Velcro and Velcro Straps, Hot Glue Sticks.

Step 2: Homemade (cheap!) Flex Sensor

Flex sensors like the one's you may get individually can be expensive. My friend Tobias found this simple recipe for a DIY flex sensor that yield great (maybe superior) results for a small fraction of the price.

Here's all it takes: Vinyl Electrical Tape (any pliable tape will work); Conductive Thread; Anti-Static Plastic Sheeting; Scissors.

Basically you just make a 'sandwich' of the materials in this order from the bottom up, and like any good sandwich, it can be ordered to size: ELECTRICAL TAPE (sticky-side-up), CONDUCTIVE THREAD (sticking out one end, but not the other), ANTI-STATIC PLASTIC (just covering the thread inside the edges of the tape, not too wide). Now think of the ELECTRICAL TAPE like half of a bun. Make the other half of the 'bun' the same way with one exception: the CONDUCTIVE THREAD should now stick out the opposite direction. Make sure the ANTI-STATIC LETTUCE covers all of thread inside the edges of the tape, but not the whole width of the tape.

Last step: place the two 'buns' directly on top of each other, sticky sides coming together in the middle. Here, you'll see it's important to leave some tape edges in the width. Your 'sandwich' now looks like one vinyl strip with a little tunnel running just under the surface and a short piece of thread sticking out each end.

We made enough, and at a sensible (ha ha) length, for each finger and the thumb.

Step 3: El Glovado

The glove assembly is pretty straight forward. The flex sensors can be sewn with the NORMAL thread and needle. You can use the ALUMINUM FOIL TAPE to connect the lead ends to wires** long enough to reach from each lead end to the back part of your wrist, about 6-7 inches.

You can just cut the tape with regular scissors. It's super thin so you don't have think of it as cutting metal. But you should, because cutting metal is awesome. Additionally we took the time to cut some different colors of electrical tape to color code the wires. For simpletons like me it helps keep things sorted out and, trust me, it's going to get a little hairy when you start wiring.

**This is good place to use STRANDED WIRE (last image). You'll find it is good and flexible which in this case helps follow the motion of your fingers AND not pull wires out of the breadboard.

Step 4: Mercury the Courier...

Let's talk TILT SENSORS. They are not all equal. We happened to find some mercury switches which WE DON"T RECOMMEND! They work, yes. But they're little glass tubes with a drop of mercury (a known poison) inside them! There are TILT SWITCHes available that work the same way and don't have arsenic, lead or mercury in them.

Take your non-deadly tilt switch and solder some 4-5 inch wire leads on them. Set them asside for a bit. Time to get electrified...

Step 5: Arduino, Meet Wave Shield.

"Hello Wave Shield. My my do you have a nice volume knob"...."Oh! Oh, thank you Mr Italian smooth talk....."

Sorry, I digress.

Here's what we're not going to show you: THREE HOURS OF SOLDERING! What's great about Lady Ada and her glorious products is that they're inexpensive and they come with really good instructions! Plus, the Lady, keeps the cost down by not assembling the boards and in turn, YOU learn how to solder like a robot! What sucks is, the path to soldering like a robot is akin the walk Dante took with Virgil.

Don't be skerd! Soldering is a great skill!

The next thing to do is follow the "USE IT" tutorial on the WAVE SHIELD to learn how to...uh....use it. This involves using your SD CARD FORMATTER AND CARD, getting the files on to the card, and getting the CODE! Our code is posted below. THIS IS ASKING A LOT OF A NEW ARDUINO USER!!! Don't be discouraged if it doesn't go smoothly. It didn't for us either.

CODE: Below here, copy and paste into your Arduino IDE.

#include "FatReader.h"
#include "SdReader.h"
#include
#include "WaveUtil.h"
#include "WaveHC.h"

SdReader card; // This object holds the information for the card
FatVolume vol; // This holds the information for the partition on the card
FatReader root; // This holds the information for the filesystem on the card
FatReader f; // This holds the information for the file we're playing
WaveHC wave; // This is the only wave (audio) object, since we will only play one at a time
uint8_t dirLevel; // indent level for file/dir names (for prettyprinting)
dir_t dirBuf; // buffer for directory reads

void setUsUpTheCard(void) {
pinMode(2, OUTPUT);
pinMode(3, OUTPUT);
pinMode(4, OUTPUT);
pinMode(5, OUTPUT);

//if (!card.init(true)) { //play with 4 MHz spi if 8MHz isn't working for you
if (!card.init()) { //play with 8 MHz spi (default faster!)
putstring_nl("Card init. failed!"); // Something went wrong, lets print out why
sdErrorCheck();
while(1); // then 'halt' - do nothing!
}
// Now we will look for a FAT partition!
uint8_t part;
for (part = 0; part < 5; part++) { // we have up to 5 slots to look in
if (vol.init(card, part))
break; // we found one, lets bail
}
if (part == 5) { // if we ended up not finding one :(
putstring_nl("No valid FAT partition!");
sdErrorCheck(); // Something went wrong, lets print out why
while(1); // then 'halt' - do nothing!
}
// Try to open the root directory
if (!root.openRoot(vol)) {
putstring_nl("Can't open root dir!"); // Something went wrong,
while(1); // then 'halt' - do nothing!
}
// Whew! We got past the tough parts.
putstring_nl("Files found:");
dirLevel = 0;
// Print out all of the files in all the directories.
lsR(root);
}

/*
* list recursively - possible stack overflow if subdirectories too nested
*/
void lsR(FatReader &d)
{
int8_t r; // indicates the level of recursion

while ((r = d.readDir(dirBuf)) > 0) { // read the next file in the directory
// skip subdirs . and ..
if (dirBuf.name[0] == '.')
continue;

for (uint8_t i = 0; i < dirLevel; i++)
Serial.print(' '); // this is for prettyprinting, put spaces in front
printName(dirBuf); // print the name of the file we just found
Serial.println(); // and a new line

if (DIR_IS_SUBDIR(dirBuf)) { // we will recurse on any direcory
FatReader s; // make a new directory object to hold information
dirLevel += 2; // indent 2 spaces for future prints
if (s.open(vol, dirBuf))
lsR(s); // list all the files in this directory now!
dirLevel -=2; // remove the extra indentation
}
}
sdErrorCheck(); // are we doign OK?
}

/*
* print dir_t name field. The output is 8.3 format, so like SOUND.WAV or FILENAME.DAT
*/
void printName(dir_t &dir)
{
for (uint8_t i = 0; i < 11; i++) { // 8.3 format has 8+3 = 11 letters in it
if (dir.name[i] == ' ')
continue; // dont print any spaces in the name
if (i == 8)
Serial.print('.'); // after the 8th letter, place a dot
Serial.print(dir.name[i]); // print the n'th digit
}
if (DIR_IS_SUBDIR(dir))
Serial.print('/'); // directories get a / at the end
}

// Plays a full file from beginning to end with no pause.
void playcomplete(char *name) {
// call our helper to find and play this name
playfile(name);
while (wave.isplaying) {
// do nothing while its playing
}
// now its done playing
}

void playfile(char *name) {
// see if the wave object is currently doing something
if (wave.isplaying) {// already playing something, so stop it!
wave.stop(); // stop it
}
// look in the root directory and open the file
if (!f.open(root, name)) {
putstring("Couldn't open file "); Serial.print(name); return;
}
// OK read the file and turn it into a wave object
if (!wave.create(f)) {
putstring_nl("Not a valid WAV"); return;
}

// ok time to play! start playback
wave.play();
}

void sdErrorCheck(void)
{
if (!card.errorCode()) return;
putstring(" SD I/O error: ");
Serial.print(card.errorCode(), HEX);
putstring(", ");
Serial.println(card.errorData(), HEX);
while(1);
}

/*************************************
* THIS IS THE GOOD STUFF ************
*************************************/

int tilt_port = 13; // digital port the tilt sensor is on
int tilt_original; // holds the beginning position of the tilt sensor
// for faux accelerometer action

void setup() {
Serial.begin(9600);

pinMode(tilt_port, INPUT);
int tilt_original = digitalRead(tilt_port);

setUsUpTheCard(); // initialize the Wave Shield
}

int thumb = 0;
int pointer = 0;
int middle = 0;
int ring = 0;
int pinky = 0;
int tilt_waiting = 0;
int tilt_inmotion = 0;

int curled = 0;
int curled_previous = 0;

void loop() {
int thumb = analogRead(A4);
int pointer = analogRead(A3);
int middle = analogRead(A2);
int ring = analogRead(A1);
int pinky = analogRead(A0);

curled = 0;

/*
* the thumb seems to be b0rked
* randomly (?) sticks at 0
* so let's ignore it
*/
if( thumb > 120 ) {
curled += 1;
}
if( pointer > 150 ) {
curled += 10;
}
/*
* the middle finger is seemingly always on
* so let's ignore it
*/
if( middle > 130 ) {
curled += 100;
}
if( ring > 170 ) {
curled += 1000;
}
if( pinky > 120 ) {
curled += 10000;
}

if( curled >= 11100 ){
tilt_check( "PUNCH", "PUNCH.WAV");
} else {
tilt_check( "GUN", "GUN.WAV");
}
/*
if( curled == 0 ){
// karate CHOP!
tilt_check( "CHOP", "CHOP.WAV" );
} else if( curled == 1000 + 10000 ){
// GUN
tilt_check( "GUN", "GUN.WAV" );
} else if( curled == 9999 ){
// SPIIIIIDEY-MAN
tilt_check( "SPIDEY", "SPIDEY.WAV" );
} else if( curled == 10 + 1000 + 10000 ){
// FIST
tilt_check( "PUNCH", "PUNCH.WAV" );
}
*/
Serial.print("thumb: "); Serial.println(thumb);
Serial.print("pointer: "); Serial.println(pointer);
Serial.print("middle: "); Serial.println(middle);
Serial.print("ring: "); Serial.println(ring);
Serial.print("pinky: "); Serial.println(pinky);
Serial.print("curled: "); Serial.println(curled);
Serial.println("");
delay(15);
}

void reset_check( int reset = 0 ){
if( reset || curled != curled_previous ){
tilt_waiting = 0;
tilt_inmotion = 0;
curled_previous = curled;
}
}

void tilt_check( String message, char *filename ){
reset_check();
int tilt = digitalRead(tilt_port);
Serial.println(tilt);
if( !tilt_waiting ){
tilt_waiting = 1;
tilt_original = tilt;
} else {
if( tilt != tilt_original ){
tilt_inmotion = 1;
}
if( tilt_inmotion && tilt == tilt_original ){
Serial.println(message);
Serial.println(filename);
playcomplete(filename);
reset_check(1);
}
}
}

Step 6: Get Wired!

Now comes the complex task of wiring. Complex because there are a bunch!!! of wires. We are working on a diagram inside of program called FRITZING to help clarify how we did it. Please stand by...

FRITZING allows amateurs like use to not only diagram board layouts and schematics, but actually print your own circuit board!!!

The next thing you can do is use velcro or hot glue to attach the breadboard to the back of the glove.

Step 7: Come Together!

Time to get things together.

This is a place where you can use the other kind of wire, solid. Solid, brotha! You can see it was used to make tidy jumps across the breadboard. This reduces what already promises to be a rat's nest of wire. The solid wire is also better for pushing into breadboard holes.

The other thing you might notice is that the ARDUINO and WAVE SHIELD are encased in some awesomeness we haven't described.
( It's also running on a 9v battery and has a speaker soldered to the WAVE SHIELD!) The awesomeness is a custom holder, designed in a CAD software, and made using a 3D printer. It wasn't mentioned earlier because it's a bit of a luxury to have a 3D printer.

(The CAD file is below but in it's native format i.e. 3dm or Rhinoceros. If you need IGES or vrml or whatever, please let us know)

PowerPack.3dm.zip
Download

Step 8: Punch It, Chewy!

Alright! You've got it. We don't care if you tape the stuff to your arm, velcro it, glue it. How that works is up to your creativity. We've used some velcro straps for our custom holder.

The input from the flex sensors, combined with input from the tilt sensors will (should) call sound files from the SD CARD and pump them through the speaker bringing (in our case) some movie sound effects into the real world.

Enjoy your new Hyper-Reality!