Augmented Hyper-Reality Glove




Introduction: Augmented Hyper-Reality Glove

The Carnegie Mellon HCI gang is at it again, this time taking on the serious assignment of making a thing for PLAY. The Royal Play that is. The thing that makes life worth living.

This time, we've combined our nerd-ness to bring some of your favorite movie sound effects to life. The " Augmented Hyper-Reality Glove" uses flex and tilt sensors to send signals to a Wave Shield from Adafruit Industries. The Wave Shield lets you trigger uncompressed sound files directly with Arduino, eliminating the need to call files from your computer.

When you punch with the glove you get a "punch" sound, when you make a pistol shape, a "gunfire" sound, etc. The possibilities are limited only to your imagination, and your world will be augmented by the sounds you normally only hear in your head or make yourself (legitimizing those embarrassing moments when other people hear you doing it).

Team: Becca Chen, Zack Jacobson-Weaver, Jimmy Krahe & Spencer Sugarman from Activating Objects with Eric Paulos and developed at CodeLab.

Let's get started...

Teacher Notes

Teachers! Did you use this instructable in your classroom?
Add a Teacher Note to share how you incorporated it into your lesson.

Step 1: What You'll Need

Tools: Soldering Equipment, Wire Cutters, Scissors, Needle and Thread, Arduino board (here's a link for everything you need to use it), Lady Ada Wave Shield for Arduino, Solderless Breadboard, Hot Glue Gun, SD Card Formatter with one SD Card, your favorite Mac or PC.

Materials: One Glove, Solder (and some skills, the Wave Shield is a marathon soldering session), Conductive Thread, Electrical Tape, Anti-Static Sheeting, Regular Sewing Thread and a Needle (and a thimble for you sensitive types), Aluminum or Copper Foil Tape, Electrical Wire ( there are two kinds used here: Solid and Stranded. You'll see why later), Velcro and Velcro Straps, Hot Glue Sticks.

Step 2: Homemade (cheap!) Flex Sensor

Flex sensors like the one's you may get individually can be expensive.  My friend Tobias found this simple recipe for a DIY flex sensor that yield great (maybe superior) results for a small fraction of the price.

Here's all it takes: Vinyl Electrical Tape (any pliable tape will work); Conductive Thread; Anti-Static Plastic Sheeting; Scissors.

Basically you just make a 'sandwich' of the materials in this order from the bottom up, and like any good sandwich, it can be ordered to size: ELECTRICAL TAPE (sticky-side-up), CONDUCTIVE THREAD (sticking out one end, but not the other), ANTI-STATIC PLASTIC (just covering the thread inside the edges of the tape, not too wide). Now think of the ELECTRICAL TAPE like half of a bun. Make the other half of the 'bun' the same way with one exception: the CONDUCTIVE THREAD should now stick out the opposite direction. Make sure the ANTI-STATIC LETTUCE covers all of thread inside the edges of the tape, but not the whole width of the tape.

Last step: place the two 'buns' directly on top of each other, sticky sides coming together in the middle. Here, you'll see it's important to leave some tape edges in the width. Your 'sandwich' now looks like one vinyl strip with a little tunnel running just under the surface and a short piece of thread sticking out each end.

We made enough, and at a sensible (ha ha) length, for each finger and the thumb.

Step 3: El Glovado

The glove assembly is pretty straight forward. The flex sensors can be sewn with the NORMAL thread and needle. You can use the ALUMINUM FOIL TAPE to connect the lead ends to wires** long enough to reach from each lead end to the back part of your wrist, about 6-7 inches.

You can just cut the tape with regular scissors. It's super thin so you don't have think of it as cutting metal. But you should, because cutting metal is awesome. Additionally we took the time to cut some different colors of electrical tape to color code the wires. For simpletons like me it helps keep things sorted out and, trust me, it's going to get a little hairy when you start wiring.

**This is good place to use STRANDED WIRE (last image). You'll find it is good and flexible which in this case helps follow the motion of your fingers AND not pull wires out of the breadboard.

Step 4: Mercury the Courier...

Let's talk TILT SENSORS.  They are not all equal.  We happened to find some mercury switches which WE DON"T RECOMMEND!  They work, yes.  But they're little glass tubes with a drop of mercury (a known poison) inside them!  There are TILT SWITCHes available that work the same way and don't have arsenic, lead or mercury in them.

Take your non-deadly tilt switch and solder some 4-5 inch wire leads on them.  Set them asside for a bit.  Time to get electrified...

Step 5: Arduino, Meet Wave Shield.

"Hello Wave Shield. My my do you have a nice volume knob"...."Oh! Oh, thank you Mr Italian smooth talk....."

Sorry, I digress.

Here's what we're not going to show you: THREE HOURS OF SOLDERING! What's great about Lady Ada and her glorious products is that they're inexpensive and they come with really good instructions! Plus, the Lady, keeps the cost down by not assembling the boards and in turn, YOU learn how to solder like a robot! What sucks is, the path to soldering like a robot is akin the walk Dante took with Virgil.

Don't be skerd! Soldering is a great skill!

The next thing to do is follow the "USE IT" tutorial on the WAVE SHIELD to learn how to...uh....use it. This involves using your SD CARD FORMATTER AND CARD, getting the files on to the card, and getting the CODE! Our code is posted below. THIS IS ASKING A LOT OF A NEW ARDUINO USER!!! Don't be discouraged if it doesn't go smoothly. It didn't for us either.

CODE: Below here, copy and paste into your Arduino IDE.

#include "FatReader.h"
#include "SdReader.h"
#include "WaveUtil.h"
#include "WaveHC.h"

SdReader card; // This object holds the information for the card
FatVolume vol; // This holds the information for the partition on the card
FatReader root; // This holds the information for the filesystem on the card
FatReader f; // This holds the information for the file we're playing
WaveHC wave; // This is the only wave (audio) object, since we will only play one at a time
uint8_t dirLevel; // indent level for file/dir names (for prettyprinting)
dir_t dirBuf; // buffer for directory reads

void setUsUpTheCard(void) {
pinMode(2, OUTPUT);
pinMode(3, OUTPUT);
pinMode(4, OUTPUT);
pinMode(5, OUTPUT);

//if (!card.init(true)) { //play with 4 MHz spi if 8MHz isn't working for you
if (!card.init()) { //play with 8 MHz spi (default faster!)
putstring_nl("Card init. failed!"); // Something went wrong, lets print out why
while(1); // then 'halt' - do nothing!
// Now we will look for a FAT partition!
uint8_t part;
for (part = 0; part < 5; part++) { // we have up to 5 slots to look in
if (vol.init(card, part))
break; // we found one, lets bail
if (part == 5) { // if we ended up not finding one :(
putstring_nl("No valid FAT partition!");
sdErrorCheck(); // Something went wrong, lets print out why
while(1); // then 'halt' - do nothing!
// Try to open the root directory
if (!root.openRoot(vol)) {
putstring_nl("Can't open root dir!"); // Something went wrong,
while(1); // then 'halt' - do nothing!
// Whew! We got past the tough parts.
putstring_nl("Files found:");
dirLevel = 0;
// Print out all of the files in all the directories.

* list recursively - possible stack overflow if subdirectories too nested
void lsR(FatReader &d)
int8_t r; // indicates the level of recursion

while ((r = d.readDir(dirBuf)) > 0) { // read the next file in the directory
// skip subdirs . and ..
if ([0] == '.')

for (uint8_t i = 0; i < dirLevel; i++)
Serial.print(' '); // this is for prettyprinting, put spaces in front
printName(dirBuf); // print the name of the file we just found
Serial.println(); // and a new line

if (DIR_IS_SUBDIR(dirBuf)) { // we will recurse on any direcory
FatReader s; // make a new directory object to hold information
dirLevel += 2; // indent 2 spaces for future prints
if (, dirBuf))
lsR(s); // list all the files in this directory now!
dirLevel -=2; // remove the extra indentation
sdErrorCheck(); // are we doign OK?

* print dir_t name field. The output is 8.3 format, so like SOUND.WAV or FILENAME.DAT
void printName(dir_t &dir)
for (uint8_t i = 0; i < 11; i++) { // 8.3 format has 8+3 = 11 letters in it
if ([i] == ' ')
continue; // dont print any spaces in the name
if (i == 8)
Serial.print('.'); // after the 8th letter, place a dot
Serial.print([i]); // print the n'th digit
if (DIR_IS_SUBDIR(dir))
Serial.print('/'); // directories get a / at the end

// Plays a full file from beginning to end with no pause.
void playcomplete(char *name) {
// call our helper to find and play this name
while (wave.isplaying) {
// do nothing while its playing
// now its done playing

void playfile(char *name) {
// see if the wave object is currently doing something
if (wave.isplaying) {// already playing something, so stop it!
wave.stop(); // stop it
// look in the root directory and open the file
if (!, name)) {
putstring("Couldn't open file "); Serial.print(name); return;
// OK read the file and turn it into a wave object
if (!wave.create(f)) {
putstring_nl("Not a valid WAV"); return;

// ok time to play! start playback;

void sdErrorCheck(void)
if (!card.errorCode()) return;
putstring(" SD I/O error: ");
Serial.print(card.errorCode(), HEX);
putstring(", ");
Serial.println(card.errorData(), HEX);

* THIS IS THE GOOD STUFF ************

int tilt_port = 13; // digital port the tilt sensor is on
int tilt_original; // holds the beginning position of the tilt sensor
// for faux accelerometer action

void setup() {

pinMode(tilt_port, INPUT);
int tilt_original = digitalRead(tilt_port);

setUsUpTheCard(); // initialize the Wave Shield

int thumb = 0;
int pointer = 0;
int middle = 0;
int ring = 0;
int pinky = 0;
int tilt_waiting = 0;
int tilt_inmotion = 0;

int curled = 0;
int curled_previous = 0;

void loop() {
int thumb = analogRead(A4);
int pointer = analogRead(A3);
int middle = analogRead(A2);
int ring = analogRead(A1);
int pinky = analogRead(A0);

curled = 0;

* the thumb seems to be b0rked
* randomly (?) sticks at 0
* so let's ignore it
if( thumb > 120 ) {
curled += 1;
if( pointer > 150 ) {
curled += 10;
* the middle finger is seemingly always on
* so let's ignore it
if( middle > 130 ) {
curled += 100;
if( ring > 170 ) {
curled += 1000;
if( pinky > 120 ) {
curled += 10000;

if( curled >= 11100 ){
tilt_check( "PUNCH", "PUNCH.WAV");
} else {
tilt_check( "GUN", "GUN.WAV");
if( curled == 0 ){
// karate CHOP!
tilt_check( "CHOP", "CHOP.WAV" );
} else if( curled == 1000 + 10000 ){
// GUN
tilt_check( "GUN", "GUN.WAV" );
} else if( curled == 9999 ){
tilt_check( "SPIDEY", "SPIDEY.WAV" );
} else if( curled == 10 + 1000 + 10000 ){
tilt_check( "PUNCH", "PUNCH.WAV" );
Serial.print("thumb: "); Serial.println(thumb);
Serial.print("pointer: "); Serial.println(pointer);
Serial.print("middle: "); Serial.println(middle);
Serial.print("ring: "); Serial.println(ring);
Serial.print("pinky: "); Serial.println(pinky);
Serial.print("curled: "); Serial.println(curled);

void reset_check( int reset = 0 ){
if( reset || curled != curled_previous ){
tilt_waiting = 0;
tilt_inmotion = 0;
curled_previous = curled;

void tilt_check( String message, char *filename ){
int tilt = digitalRead(tilt_port);
if( !tilt_waiting ){
tilt_waiting = 1;
tilt_original = tilt;
} else {
if( tilt != tilt_original ){
tilt_inmotion = 1;
if( tilt_inmotion && tilt == tilt_original ){

Step 6: Get Wired!

Now comes the complex task of wiring. Complex because there are a bunch!!! of wires. We are working on a diagram inside of program called FRITZING to help clarify how we did it. Please stand by...

FRITZING allows amateurs like use to not only diagram board layouts and schematics, but actually print your own circuit board!!!

The next thing you can do is use velcro or hot glue to attach the breadboard to the back of the glove.

Step 7: Come Together!

Time to get things together.

This is a place where you can use the other kind of wire, solid. Solid, brotha! You can see it was used to make tidy jumps across the breadboard. This reduces what already promises to be a rat's nest of wire. The solid wire is also better for pushing into breadboard holes.

The other thing you might notice is that the ARDUINO and WAVE SHIELD are encased in some awesomeness we haven't described.
( It's also running on a 9v battery and has a speaker soldered to the WAVE SHIELD!) The awesomeness is a custom holder, designed in a CAD software, and made using a 3D printer. It wasn't mentioned earlier because it's a bit of a luxury to have a 3D printer.

(The CAD file is below but in it's native format i.e. 3dm or Rhinoceros.  If you need IGES or vrml or whatever, please let us know)

Step 8: Punch It, Chewy!

Alright! You've got it. We don't care if you tape the stuff to your arm, velcro it, glue it. How that works is up to your creativity. We've used some velcro straps for our custom holder.

The input from the flex sensors, combined with input from the tilt sensors will (should) call sound files from the SD CARD and pump them through the speaker bringing (in our case) some movie sound effects into the real world.

Enjoy your new Hyper-Reality!

Be the First to Share


    • Toys and Games Challenge

      Toys and Games Challenge
    • Backyard Contest

      Backyard Contest
    • Silly Hats Speed Challenge

      Silly Hats Speed Challenge

    27 Discussions


    8 years ago on Step 2

    I made these as you described, but they don't seem to work at all :(


    Reply 8 years ago on Introduction

    And I'm talking about the flexsensors here.


    Reply 8 years ago on Introduction

    Hmm. Maybe I can help you debug. Something simple is causing a problem. I say this not because I don't believe you. Only because we've used these same sensors at my lab for the past 4 months and if they're made and wired correctly, they're great. Can you tell me how you made them? And how are you reading them? Through Arduino? And how are they wired into the circuit?


    Reply 8 years ago on Introduction

    Well after a few hours and scouting the internet for other methods, I got some working. However, I ended up not using the anti-static bags at all - I think it is because none of mine are actually conductive. I used conductive foam that ICs get jammed into during transport. I also found that electrical tape does not hold it together well enough and there are contact gaps, so I used duct tape.
    And since I was getting really jumpy readings with conductive thread (I suspect that a single strand can get disconnected at random times), I used multistranded copper wire.

    Sooo I guess I kind of changed everything, but thanks anyways!


    Reply 8 years ago on Introduction

    Awesome. That's the spirit. What are you making?


    Reply 8 years ago on Introduction

    Also a glove, but planning to control my robot with it. Kind of feel like pretending I have superpowers or something by waving my hands around and making gestures :D I'm also planning to add accelerometers to it.


    Reply 8 years ago on Introduction

    Cool. I'll be watching for it.

    I'm currently setting up a processing program to control the screen with invisible forces (or photo resistor values). Superpowers, indeed.

    Good luck.


    8 years ago on Introduction

    could you hook this up to a USB cable, and use this on a video-game?


    Reply 8 years ago on Introduction

    You could! It would be a fairly complex hack but (I'm guessing here) you could probably:

    Break down your video game controller and literally find out, of all the wires going from the controller to the console, (this is assuming you're using a wired controller. Do they even make those anymore?) which one is assigned to what button, lever motion, tilt etc. You would have to do this by providing power to the controller and grounding it as well. It would be akin to measuring analog input on Arduino.

    If you look at my only other Instructable, you can see how we used analog input to turn stuff on and off. That's also what the glove does. That also what a WIRED video game controller does. Otherwise it is a digital input which, for most purposes, works (outputs) the same way.

    Then comes the real challenge. You have to "map" the output of the glove to the output of the controller. So, for instance, lets say you have a "B" button on your controller. You could hard wire your index finger to the B wire of the controller and it could then send signals (switch on/ switch off) to the game console. Etc, etc. etc.

    What I'm not explaining here is exactly what it means to sort out these wires, power them, and "map" one controller (the one that came with your game system) to a new controller (like the glove). For me that would be tricky.

    But can it be done? I say, anything CAN be done!

    To quote Ra's al Ghul, Batman's Ninja coach, "You're training is nothing! You're WILL is everything!"


    8 years ago on Introduction

    Kind of looks like the gunslinger from tf2.


    8 years ago on Step 2

    Hi. Cool project :D (even if extremly nerdy :P)
    I would like to build some of these flexsensors for another project, What kind of output do these give? How precise are they?


    Reply 8 years ago on Step 2

    They give an analogue input which will a resistance value. It actually depends on the individual piece. We've seen ranges from 0-300 or even 0-700.

    These also work as pressure sensors so try it both ways.



    8 years ago on Step 7

    Hi Bruno, check out this website. These gloves are crazy:

    They are VERY expensive but VERY VERY durable. I paid about $35 US for these but they're my second pair and the first pair made it through about 2 1/2 years of mountain biking (i.e. me catching my fall as I'm sliding down dirt and gravel).

    Plus they look cool!


    8 years ago on Introduction

    You should consider redoing this with a smaller protoboard, maybe like one of these commercial products Sparkfun sells:

    They are circular LilyPad protoboards.

    and use conductive thread to tie the sensors into the wave shield.


    Reply 8 years ago on Introduction

    Agreed. This is first prototype level. The full documentation comes as the hybrid offspring of LOVE FOR OPEN SOURCE(yay!) + CLASS REQUIREMENT(booo). So in a given time frame we struggle to get a working prototype coupled with the academic baggage (which is obviously truly good).

    Regardless, you're point is well-taken. Clean it up!

    Thank you for the ideas and links.


    8 years ago on Introduction

    Great project! What are the total costs of the components in this project?


    Reply 8 years ago on Introduction

    Hard to say as some of the stuff was repurposed from other worlds. To guestimate (I have now become my father) I would say,

    Glove: free - $35 (any glove would work)
    Wave Shield $25+/-
    Flex Sensors $10
    Arduino and Other Electronics $30 (???)
    Misc $10

    It realistically costs $100 plus assuming you have nothing to start with. Now that you ask, I see that's pretty high.

    The price point on a commercial product should be less than $35 a PAIR!

    You also don't need gloves that cost $35. Ours did, but it's a high quality (Youngstown) glove that kinda looks cool to start with.

    There are lots of places, I suppose, to trim the fat.