loading
The Carnegie Mellon HCI gang is at it again, this time taking on the serious assignment of making a thing for PLAY. The Royal Play that is. The thing that makes life worth living.

This time, we've combined our nerd-ness to bring some of your favorite movie sound effects to life. The " Augmented Hyper-Reality Glove" uses flex and tilt sensors to send signals to a Wave Shield from Adafruit Industries. The Wave Shield lets you trigger uncompressed sound files directly with Arduino, eliminating the need to call files from your computer.

When you punch with the glove you get a "punch" sound, when you make a pistol shape, a "gunfire" sound, etc. The possibilities are limited only to your imagination, and your world will be augmented by the sounds you normally only hear in your head or make yourself (legitimizing those embarrassing moments when other people hear you doing it).

Team: Becca Chen, Zack Jacobson-Weaver, Jimmy Krahe & Spencer Sugarman from Activating Objects with Eric Paulos and developed at CodeLab.

Let's get started...

Step 1: What You'll Need

Tools: Soldering Equipment, Wire Cutters, Scissors, Needle and Thread, Arduino board (here's a link for everything you need to use it), Lady Ada Wave Shield for Arduino, Solderless Breadboard, Hot Glue Gun, SD Card Formatter with one SD Card, your favorite Mac or PC.

Materials: One Glove, Solder (and some skills, the Wave Shield is a marathon soldering session), Conductive Thread, Electrical Tape, Anti-Static Sheeting, Regular Sewing Thread and a Needle (and a thimble for you sensitive types), Aluminum or Copper Foil Tape, Electrical Wire ( there are two kinds used here: Solid and Stranded. You'll see why later), Velcro and Velcro Straps, Hot Glue Sticks.

Step 2: Homemade (cheap!) Flex Sensor

Flex sensors like the one's you may get individually can be expensive.  My friend Tobias found this simple recipe for a DIY flex sensor that yield great (maybe superior) results for a small fraction of the price.

Here's all it takes: Vinyl Electrical Tape (any pliable tape will work); Conductive Thread; Anti-Static Plastic Sheeting; Scissors.

Basically you just make a 'sandwich' of the materials in this order from the bottom up, and like any good sandwich, it can be ordered to size: ELECTRICAL TAPE (sticky-side-up), CONDUCTIVE THREAD (sticking out one end, but not the other), ANTI-STATIC PLASTIC (just covering the thread inside the edges of the tape, not too wide). Now think of the ELECTRICAL TAPE like half of a bun. Make the other half of the 'bun' the same way with one exception: the CONDUCTIVE THREAD should now stick out the opposite direction. Make sure the ANTI-STATIC LETTUCE covers all of thread inside the edges of the tape, but not the whole width of the tape.

Last step: place the two 'buns' directly on top of each other, sticky sides coming together in the middle. Here, you'll see it's important to leave some tape edges in the width. Your 'sandwich' now looks like one vinyl strip with a little tunnel running just under the surface and a short piece of thread sticking out each end.

We made enough, and at a sensible (ha ha) length, for each finger and the thumb.

Step 3: El Glovado

The glove assembly is pretty straight forward. The flex sensors can be sewn with the NORMAL thread and needle. You can use the ALUMINUM FOIL TAPE to connect the lead ends to wires** long enough to reach from each lead end to the back part of your wrist, about 6-7 inches.

You can just cut the tape with regular scissors. It's super thin so you don't have think of it as cutting metal. But you should, because cutting metal is awesome. Additionally we took the time to cut some different colors of electrical tape to color code the wires. For simpletons like me it helps keep things sorted out and, trust me, it's going to get a little hairy when you start wiring.

**This is good place to use STRANDED WIRE (last image). You'll find it is good and flexible which in this case helps follow the motion of your fingers AND not pull wires out of the breadboard.

Step 4: Mercury the Courier...

Let's talk TILT SENSORS.  They are not all equal.  We happened to find some mercury switches which WE DON"T RECOMMEND!  They work, yes.  But they're little glass tubes with a drop of mercury (a known poison) inside them!  There are TILT SWITCHes available that work the same way and don't have arsenic, lead or mercury in them.

Take your non-deadly tilt switch and solder some 4-5 inch wire leads on them.  Set them asside for a bit.  Time to get electrified...

Step 5: Arduino, Meet Wave Shield.

"Hello Wave Shield. My my do you have a nice volume knob"...."Oh! Oh, thank you Mr Italian smooth talk....."

Sorry, I digress.

Here's what we're not going to show you: THREE HOURS OF SOLDERING! What's great about Lady Ada and her glorious products is that they're inexpensive and they come with really good instructions! Plus, the Lady, keeps the cost down by not assembling the boards and in turn, YOU learn how to solder like a robot! What sucks is, the path to soldering like a robot is akin the walk Dante took with Virgil.

Don't be skerd! Soldering is a great skill!

The next thing to do is follow the "USE IT" tutorial on the WAVE SHIELD to learn how to...uh....use it. This involves using your SD CARD FORMATTER AND CARD, getting the files on to the card, and getting the CODE! Our code is posted below. THIS IS ASKING A LOT OF A NEW ARDUINO USER!!! Don't be discouraged if it doesn't go smoothly. It didn't for us either.

CODE: Below here, copy and paste into your Arduino IDE.

#include "FatReader.h"
#include "SdReader.h"
#include
#include "WaveUtil.h"
#include "WaveHC.h"

SdReader card; // This object holds the information for the card
FatVolume vol; // This holds the information for the partition on the card
FatReader root; // This holds the information for the filesystem on the card
FatReader f; // This holds the information for the file we're playing
WaveHC wave; // This is the only wave (audio) object, since we will only play one at a time
uint8_t dirLevel; // indent level for file/dir names (for prettyprinting)
dir_t dirBuf; // buffer for directory reads

void setUsUpTheCard(void) {
pinMode(2, OUTPUT);
pinMode(3, OUTPUT);
pinMode(4, OUTPUT);
pinMode(5, OUTPUT);

//if (!card.init(true)) { //play with 4 MHz spi if 8MHz isn't working for you
if (!card.init()) { //play with 8 MHz spi (default faster!)
putstring_nl("Card init. failed!"); // Something went wrong, lets print out why
sdErrorCheck();
while(1); // then 'halt' - do nothing!
}
// Now we will look for a FAT partition!
uint8_t part;
for (part = 0; part < 5; part++) { // we have up to 5 slots to look in
if (vol.init(card, part))
break; // we found one, lets bail
}
if (part == 5) { // if we ended up not finding one :(
putstring_nl("No valid FAT partition!");
sdErrorCheck(); // Something went wrong, lets print out why
while(1); // then 'halt' - do nothing!
}
// Try to open the root directory
if (!root.openRoot(vol)) {
putstring_nl("Can't open root dir!"); // Something went wrong,
while(1); // then 'halt' - do nothing!
}
// Whew! We got past the tough parts.
putstring_nl("Files found:");
dirLevel = 0;
// Print out all of the files in all the directories.
lsR(root);
}

/*
* list recursively - possible stack overflow if subdirectories too nested
*/
void lsR(FatReader &d)
{
int8_t r; // indicates the level of recursion

while ((r = d.readDir(dirBuf)) > 0) { // read the next file in the directory
// skip subdirs . and ..
if (dirBuf.name[0] == '.')
continue;

for (uint8_t i = 0; i < dirLevel; i++)
Serial.print(' '); // this is for prettyprinting, put spaces in front
printName(dirBuf); // print the name of the file we just found
Serial.println(); // and a new line

if (DIR_IS_SUBDIR(dirBuf)) { // we will recurse on any direcory
FatReader s; // make a new directory object to hold information
dirLevel += 2; // indent 2 spaces for future prints
if (s.open(vol, dirBuf))
lsR(s); // list all the files in this directory now!
dirLevel -=2; // remove the extra indentation
}
}
sdErrorCheck(); // are we doign OK?
}

/*
* print dir_t name field. The output is 8.3 format, so like SOUND.WAV or FILENAME.DAT
*/
void printName(dir_t &dir)
{
for (uint8_t i = 0; i < 11; i++) { // 8.3 format has 8+3 = 11 letters in it
if (dir.name[i] == ' ')
continue; // dont print any spaces in the name
if (i == 8)
Serial.print('.'); // after the 8th letter, place a dot
Serial.print(dir.name[i]); // print the n'th digit
}
if (DIR_IS_SUBDIR(dir))
Serial.print('/'); // directories get a / at the end
}

// Plays a full file from beginning to end with no pause.
void playcomplete(char *name) {
// call our helper to find and play this name
playfile(name);
while (wave.isplaying) {
// do nothing while its playing
}
// now its done playing
}

void playfile(char *name) {
// see if the wave object is currently doing something
if (wave.isplaying) {// already playing something, so stop it!
wave.stop(); // stop it
}
// look in the root directory and open the file
if (!f.open(root, name)) {
putstring("Couldn't open file "); Serial.print(name); return;
}
// OK read the file and turn it into a wave object
if (!wave.create(f)) {
putstring_nl("Not a valid WAV"); return;
}

// ok time to play! start playback
wave.play();
}

void sdErrorCheck(void)
{
if (!card.errorCode()) return;
putstring(" SD I/O error: ");
Serial.print(card.errorCode(), HEX);
putstring(", ");
Serial.println(card.errorData(), HEX);
while(1);
}

/*************************************
* THIS IS THE GOOD STUFF ************
*************************************/

int tilt_port = 13; // digital port the tilt sensor is on
int tilt_original; // holds the beginning position of the tilt sensor
// for faux accelerometer action

void setup() {
Serial.begin(9600);

pinMode(tilt_port, INPUT);
int tilt_original = digitalRead(tilt_port);

setUsUpTheCard(); // initialize the Wave Shield
}

int thumb = 0;
int pointer = 0;
int middle = 0;
int ring = 0;
int pinky = 0;
int tilt_waiting = 0;
int tilt_inmotion = 0;

int curled = 0;
int curled_previous = 0;

void loop() {
int thumb = analogRead(A4);
int pointer = analogRead(A3);
int middle = analogRead(A2);
int ring = analogRead(A1);
int pinky = analogRead(A0);

curled = 0;

/*
* the thumb seems to be b0rked
* randomly (?) sticks at 0
* so let's ignore it
*/
if( thumb > 120 ) {
curled += 1;
}
if( pointer > 150 ) {
curled += 10;
}
/*
* the middle finger is seemingly always on
* so let's ignore it
*/
if( middle > 130 ) {
curled += 100;
}
if( ring > 170 ) {
curled += 1000;
}
if( pinky > 120 ) {
curled += 10000;
}

if( curled >= 11100 ){
tilt_check( "PUNCH", "PUNCH.WAV");
} else {
tilt_check( "GUN", "GUN.WAV");
}
/*
if( curled == 0 ){
// karate CHOP!
tilt_check( "CHOP", "CHOP.WAV" );
} else if( curled == 1000 + 10000 ){
// GUN
tilt_check( "GUN", "GUN.WAV" );
} else if( curled == 9999 ){
// SPIIIIIDEY-MAN
tilt_check( "SPIDEY", "SPIDEY.WAV" );
} else if( curled == 10 + 1000 + 10000 ){
// FIST
tilt_check( "PUNCH", "PUNCH.WAV" );
}
*/
Serial.print("thumb: "); Serial.println(thumb);
Serial.print("pointer: "); Serial.println(pointer);
Serial.print("middle: "); Serial.println(middle);
Serial.print("ring: "); Serial.println(ring);
Serial.print("pinky: "); Serial.println(pinky);
Serial.print("curled: "); Serial.println(curled);
Serial.println("");
delay(15);
}

void reset_check( int reset = 0 ){
if( reset || curled != curled_previous ){
tilt_waiting = 0;
tilt_inmotion = 0;
curled_previous = curled;
}
}

void tilt_check( String message, char *filename ){
reset_check();
int tilt = digitalRead(tilt_port);
Serial.println(tilt);
if( !tilt_waiting ){
tilt_waiting = 1;
tilt_original = tilt;
} else {
if( tilt != tilt_original ){
tilt_inmotion = 1;
}
if( tilt_inmotion && tilt == tilt_original ){
Serial.println(message);
Serial.println(filename);
playcomplete(filename);
reset_check(1);
}
}
}

Step 6: Get Wired!

Now comes the complex task of wiring. Complex because there are a bunch!!! of wires. We are working on a diagram inside of program called FRITZING to help clarify how we did it. Please stand by...

FRITZING allows amateurs like use to not only diagram board layouts and schematics, but actually print your own circuit board!!!

The next thing you can do is use velcro or hot glue to attach the breadboard to the back of the glove.

Step 7: Come Together!

Time to get things together.

This is a place where you can use the other kind of wire, solid. Solid, brotha! You can see it was used to make tidy jumps across the breadboard. This reduces what already promises to be a rat's nest of wire. The solid wire is also better for pushing into breadboard holes.

The other thing you might notice is that the ARDUINO and WAVE SHIELD are encased in some awesomeness we haven't described.
( It's also running on a 9v battery and has a speaker soldered to the WAVE SHIELD!) The awesomeness is a custom holder, designed in a CAD software, and made using a 3D printer. It wasn't mentioned earlier because it's a bit of a luxury to have a 3D printer.

(The CAD file is below but in it's native format i.e. 3dm or Rhinoceros.  If you need IGES or vrml or whatever, please let us know)

Step 8: Punch It, Chewy!

Alright! You've got it. We don't care if you tape the stuff to your arm, velcro it, glue it. How that works is up to your creativity. We've used some velcro straps for our custom holder.

The input from the flex sensors, combined with input from the tilt sensors will (should) call sound files from the SD CARD and pump them through the speaker bringing (in our case) some movie sound effects into the real world.

Enjoy your new Hyper-Reality!


Cool
Do you know some material to replace Velostat?
this is awesome guys.
I made these as you described, but they don't seem to work at all :(
And I'm talking about the flexsensors here.
Hmm. Maybe I can help you debug. Something simple is causing a problem. I say this not because I don't believe you. Only because we've used these same sensors at my lab for the past 4 months and if they're made and wired correctly, they're great. Can you tell me how you made them? And how are you reading them? Through Arduino? And how are they wired into the circuit?
Well after a few hours and scouting the internet for other methods, I got some working. However, I ended up not using the anti-static bags at all - I think it is because none of mine are actually conductive. I used conductive foam that ICs get jammed into during transport. I also found that electrical tape does not hold it together well enough and there are contact gaps, so I used duct tape. <br>And since I was getting really jumpy readings with conductive thread (I suspect that a single strand can get disconnected at random times), I used multistranded copper wire. <br><br>Sooo I guess I kind of changed everything, but thanks anyways!
Awesome. That's the spirit. What are you making?
Also a glove, but planning to control my robot with it. Kind of feel like pretending I have superpowers or something by waving my hands around and making gestures :D I'm also planning to add accelerometers to it.
Cool. I'll be watching for it. <br><br>I'm currently setting up a processing program to control the screen with invisible forces (or photo resistor values). Superpowers, indeed.<br><br>Good luck.
could you hook this up to a USB cable, and use this on a video-game?
You could! It would be a fairly complex hack but (I'm guessing here) you could probably:<br><br>Break down your video game controller and literally find out, of all the wires going from the controller to the console, (this is assuming you're using a wired controller. Do they even make those anymore?) which one is assigned to what button, lever motion, tilt etc. You would have to do this by providing power to the controller and grounding it as well. It would be akin to measuring analog input on Arduino. <br><br>If you look at my only other Instructable, you can see how we used analog input to turn stuff on and off. That's also what the glove does. That also what a WIRED video game controller does. Otherwise it is a digital input which, for most purposes, works (outputs) the same way.<br><br>Then comes the real challenge. You have to &quot;map&quot; the output of the glove to the output of the controller. So, for instance, lets say you have a &quot;B&quot; button on your controller. You could hard wire your index finger to the B wire of the controller and it could then send signals (switch on/ switch off) to the game console. Etc, etc. etc.<br><br>What I'm not explaining here is exactly what it means to sort out these wires, power them, and &quot;map&quot; one controller (the one that came with your game system) to a new controller (like the glove). For me that would be tricky.<br><br>But can it be done? I say, anything CAN be done!<br><br>To quote Ra's al Ghul, Batman's Ninja coach, &quot;You're training is nothing! You're WILL is everything!&quot;<br><br>
Kind of looks like the gunslinger from tf2.<br>http://wiki.teamfortress.com/wiki/Gunslinger
Hi. Cool project :D (even if extremly nerdy :P)<br>I would like to build some of these flexsensors for another project, What kind of output do these give? How precise are they?
They give an analogue input which will Serial.read a resistance value. It actually depends on the individual piece. We've seen ranges from 0-300 or even 0-700.<br><br>These also work as pressure sensors so try it both ways.<br><br>Cheers,
You should consider redoing this with a smaller protoboard, maybe like one of these commercial products Sparkfun sells:<br><br>https://www.sparkfun.com/products/9102<br>https://www.sparkfun.com/products/9101<br><br>They are circular LilyPad protoboards.<br><br>and use conductive thread to tie the sensors into the wave shield.
Agreed. This is first prototype level. The full documentation comes as the hybrid offspring of LOVE FOR OPEN SOURCE(yay!) + CLASS REQUIREMENT(booo). So in a given time frame we struggle to get a working prototype coupled with the academic baggage (which is obviously truly good).<br><br>Regardless, you're point is well-taken. Clean it up! <br><br>Thank you for the ideas and links.
Great project! What are the total costs of the components in this project?
Hard to say as some of the stuff was repurposed from other worlds. To guestimate (I have now become my father) I would say,<br><br>Glove: free - $35 (any glove would work)<br>Wave Shield $25+/-<br>Flex Sensors $10<br>Arduino and Other Electronics $30 (???)<br>Misc $10<br><br>It realistically costs $100 plus assuming you have nothing to start with. Now that you ask, I see that's pretty high.<br><br>The price point on a commercial product should be less than $35 a PAIR!<br><br>You also don't need gloves that cost $35. Ours did, but it's a high quality (Youngstown) glove that kinda looks cool to start with.<br><br>There are lots of places, I suppose, to trim the fat.<br>
Dude, its awesome!
Please enjoy the new Promo Video from Jimmy &quot;Scorsese&quot; Krahe!
Wow this is very cool... I will definitely have to see about trying to build this.
Cool - is there a video of it in action?<br><br>(You know, even it hadn't worked, this still looks <em>way</em> cool!)
Done and done, Kiteman. We didn't have all the bugs worked out as you'll see in the video (step 8). I'm sure your's will be better!
That's great - love the smile as well.
Yes! Kiteman, we do have a short video demo and another video promo to post. Unfortunately, YouTube must have a monster backlog so we're waiting for that to upload. I'll let you know when it's up.<br>Cheers,

About This Instructable

33,353views

196favorites

More by ZJ-Weaver:Augmented Hyper-Reality Glove The Idiot Proof Dishwasher 
Add instructable to: