loading
Picture of Puppeteer Motion-Capture Costume
Exploring the use of affordable, off-the-shelf materials and basic techniques to create wireless motion capture wearables.

The idea behind Puppeteer is to create accessible wearable technology solutions for motion-capture, aiming to create as much of the technology from scratch, collecting and sharing this knowledge through DIY instructions. The name Puppeteer comes from the concept of being able to puppeteer or control. In this case, the motion of the body wearing the costume controls whatever data is relevant to the performance or project.
The fabrication of the suit is a handmade procedure, which is not intended for mass-production, but rather for small projects lead by individuals with enthusiasm for making things themselves, sewing, gluing, soldering, programming and bug fixing.

The Puppeteer project is continuously being developed by Mika Satomi and Hannah Perner-Wilson http://www.KOBAKANT.at, and as such is constantly being expanded and refined. We welcome any feedback and input from interested individuals, groups and companies.
 
Remove these adsRemove these ads by Signing Up

Step 1: ABOUT THIS INSTRUCTABLE

Picture of ABOUT THIS INSTRUCTABLE
This Instructable goes into as much detail as we think makes sense. When following the steps you must be prepared to solve some of the problems such as designing a pattern, deciding on sensor placement, planning your circuit by yourself.
This Instructable explains the techniques we applied to bring things together and create a motion-capture wearable. The aim here is not to recreate our Puppeteer costume but to make your own. And hopefully find solutions to existing problems!

PREREQUISITES
- Sewing
- Soldering
- Pattern making
- Basic understanding of the Microcontrollers and the Arduinio, multiplexers, circuits and components
- Basic understanding of code and communicating with serial devices

MATERIALS
- Conductive thread from http://www.sparkfun.com
also see http://cnmat.berkeley.edu/resource/conductive_thread
- Neoprene from http://www.sedochemicals.com
- Stretch conductive fabric from http://www.lessemf.com
also see http://cnmat.berkeley.edu/resource/stretch_conductive_fabric
- Fusible interfacing from local fabric store or
also see http://www.shoppellon.com
- Velostat by 3M from www.lessemf.com
also see http://cnmat.berkeley.edu/resource/velostat_resistive_plastic
- Machine poppers/snaps from local fabric store
- Regular sewing thread from local fabric store
- Amounts of stretch fabrics you will need to make the costume from
- Perfboard with copper line pattern from http://www.allelectronics.com
- Wire
- Shrink tubing
- Arduino LilyPad from http://www.sparkfun.com
- Two XBee modules - one for the suit and one communicating with it
- Two 3.7V Lipo batteries and chargers

TOOLS:
- Pens and lots of paper
- Fabric pen that disappears over time
- Ruler, soft measuring tape
- Fabric scissors and paper scissors
- Sewing machine that can do the stretchy stitch
- Iron
- Sewing needles
- Popper/snap machine - handheld or hammer and simple version
- Pliers for undoing poppers
- Soldering iron and solder
- Helping hands
- Wire clippers
- Wire strippers

Step 2: SUMMARY

Picture of SUMMARY
The principle behind the Puppeteer motion capture costume is to explore the use of fabrication methods that use very basic materials and tools and yet still produce useful solutions for motion capture. We have so far focused on capturing motion of the human body, but the techniques we have developed could also be applied to stuffed toys, animals and other objects.

The puppeteer suit has been developed as an interface for various projects, adhering to requirements that were not purely motion capture oriented. The main focus throughout these projects has not been to collect precise data on the position of every limb and joint of the wearer, but rather to collect motion data such as joint angle and velocity from specific areas of the body. This data, as well as being translated into various outcome, can also be intentionally manipulated by the wearer, allowing them to interact with the system in a meaningful way.

So far there have been three completed versions of Puppeteer motion-capture costumes that have been used in the following performances:
1) Ein Kleines Puppenspiel, 2) Language Game at LEMUR and 3) Perfect Human
More information about all these performances can be found on http://www.KOBAKANT.at and in the next step.

Step 3: PUPPETEER PROJECTS

Picture of PUPPETEER PROJECTS
2867050454_2f743a4b90.jpg
2822705223_5f7f58e088.jpg
1) Ein Kleines Puppenspiel
In this performance the performer, Ivana Kalc http://myspace.com/ivanakalc wearing the costume was moving her body in order to puppeteer a representation of herself in a digital world. This performance was called Ein Kleines Puppenspiel http://puppenspiel.moviesandbox.net/ and was conceived by Friedrich Kirschner, who conceived and created the narrative, using a computer game engine Unreal Tournament, in which characters, cameras and events are all triggered by human actors acting in the real world through various devices, playing out the story, a real time puppet play.
After producing the first working Puppeteer costume for this project, we decided to continue developing the Puppeteer costume. We started to explore new ways of exploiting, manipulating and designing conductive and resistive textiles to sense bodily motion, as well as creating soft and stretchy conductive traces. We wanted to be able to produce comfortable and washable costumes that, unless intended, did not limit the wearer's natural scope of motion.


2) Language Game at LEMUR
The next Puppeteer project was developed during a one-month residency at LEMUR http://lemurbots.org/ in Brooklyn, NY. LEMUR has a set up of musical robots that can be triggered by Midi-signal. Our plan was to have a performer, wearing a motion capture costume, trigger various musical robots with her movements, creating a sound-scape to which in turn would influence her movements. We spent 4 intense weeks refining this concept, as well as designing and fabricating the next Puppeteer suit. We worked with a local ballroom dancer, Micaela Schedlbauer, who was interested in the project. Michaela had a very set dance routine and it was interesting to work with her, though unfortunately the time-frame did not really leave space for enough practice to break the routine in a way that gave her enough confidence to listen to the system, and to react to what was going on.


3) Perfect Human Performance
The third Puppeteer costume was created for a performance inspired by a 1967 short film by Joergen Leth titled The Perfect Human and by Lars von Trier's 2003 The Five Obstructions. The performance took the text from the original film and introduced the obstructions of performance and non-linear narration. The original text was cut up into blocks and the performer was presented with a game like situation in which certain words and segments of sentences were mapped to the movement of certain body parts. The performer had to first figure out what movements triggered what fragments before she could try to speak the full sentence. Once a sentence was intentionally spoken correctly the performer would move on to the next level, the next sentence and a new mapping.
This Puppeteer project was first developed for the Interface Cultures exhibition during the 2008 Ars Electronica in Linz, Austria, the performer was Ivana Kalc. Plans for future performances of the Perfect Human Performance intend collaboration with local performers in various locations. The only setback is that the performers must fit the existing Perfect Human Puppeteer suit, though slight alterations are possible.

Step 4: BEND SENSORS

All three Puppeteer suits to date are based on the principle that pressure sensors also react to bend. This is true since pressure is exerted thorough the fact that materials pressure against each other when bent because the outer layer must stretch and presses against the inner layer, which has excess material that needs to go somewhere.

Shortly prior to the first Puppeteer project we had developed the technique of making pressure sensors sandwiched between two layers of neoprene. The reason we chose to use neoprene was because we had been using it to make fabric buttons for another project http://www.massage-me.at Neoprene has some nice qualities for making pressure sensors since it creates a natural force feedback as well as spreading the pressure evenly.

Fabric Bend Sensor
The first fabric bend sensors we fabricated from two layers of 1.5mm neoprene, which has polyester jersey adhered to each side. The fact that fabric was mounted on either side of the actually neoprene made it possible to fuse a piece of conductive fabric to either piece of neoprene using iron-on interfacing, a heat glue common in quilting and sewing. In between the two pieces of conductive fabric we sandwiched a piece of Velostat, which is basically the black plastic that sensitive electronic components or circuits are packaged in to protect them from electrical charges or statically charging themselves. Velostat is carbon-impregnated polyolefin and is a bad conductor but also has the quality to reduce its resistance under pressure.
All these layers are kept in place by lightly stitching around the edges of the neoprene.
In the first Puppeteer suit we had twelve such sensors places around the body. Although the sensors are stable, they do such a lot of power, even when they are not being bent, since there is enough initial pressure on the sensor so that there is a constant flow of electricity that increases even more when bent.

Conductive Thread Bend Sensor
There is an Instructable that covers this Bend sensor >> http://www.instructables.com/id/Fabric_bend_sensor/

One change that was made in the next Puppeteer suit was to reduce the conductive surfaces on either side of the Velostat. So instead of fusing conductive fabric, we stitches a few diagonal stitches using conductive thread on either side of the neoprene. This reduced the initial conductivity a lot so that some of the sensors that were not mounted to tightly against the body, made no electrical connection unless pressured/bent.
Also, circuit wise instead of power all sensors and measuring the changes in power passing through them we used a multiplexer to individually power and measure each sensor individually in super fast succession, so seemingly seamlessly, all at once. This, along with the conductive thread alternative lowered the energy consumption to a good level.

I am also selling these handmade fabric bend sensors via Etsy. Although it is much cheaper to make your own, purchasing one will help me support my prototyping and development costs >>
http://www.etsy.com/shop.php?user_id=5178109

Step 5: OTHER SENSING IDEAS

Picture of OTHER SENSING IDEAS
Even though we have been exploring new materials, combinations and designs that produce pressure, bend and stretch sensitive sensors we have not yet implemented these results into a full Puppeteer suit. The following experiments offer some interesting starting points for further development, though we have failed to implement them in a full suit because of various reasons such as not being able to reproduce the results with enough consistency. Even though none of these experiments have yet been implemented in a Puppeteer suit, they are still part of the Puppeteer project and they will hopefully make their way into future versions.

Carbon Paint
One technique that produces some lovely results is the application of carbon paint to fabric. Depending on the thickness of the paint and the underlying fabric, one achieves different results. If the paint starts to crack too much, then when stretched the conductivity sinks, since there is simply just less physical connection. But if things come out just right and there is not too much cracking then the conductivity increases when stretched and we believe this is due to the fact that the carbon particles are being compressed though the physicality of the stretching and thus improving the conductive connection.

Conductive Textiles
The Berkley based company Eeonyx http://www.eeonyx.com/ produces some very interesting conductive textiles. They have the capability to coat almost any material/textile in a secret polymer solution, which can be regulated to produce any desired level of electrical conductivity/resistance. When they coat stretchy knits and materials such as Lycra and nylon, they practically produce stretch sensors. This is something we have just started to explore

Conductive Rubbers
There are lots of different conductive rubbers on the market and we have tested a few of them:
- Conductive rubber bands sold by All-Spec http://www.all-spec.com/1/viewitem/1862C/ALLSPEC/prodinfo/i=rss, also see: http://cnmat.berkeley.edu/resource/conductive_black_rubber_band
- Zoflex conductive rubber http://www.rfmicrolink.com/Conductive_rubber.html, also see: http://cnmat.berkeley.edu/resource/zoflex_conductive_rubber
- Conductive rubber ropes from Images Scientific Instruments http://www.imagesco.com/sensors/stretch-sensor.html

The problem we have had with using these rubbers as stretch sensors is that they are either very tough and take a lot of pull to really stretch and get a nice change in resistance, and as such would be hard to integrate in a motion capture suit, unless we make it super tight (or come up with other solutions). The other problem we encounter is that even though the change in resistance is measurable it bounces around a bit. It is often slow to react and then bounces to a peak before dropping down to a certain point, even though the rubber has only been stretches to a certain degree and held in the same position. These rubbers all react very nicely to pressure, when sandwiched between two conductive layers. They make great pressure sensors, replacing the layers of neoprene in that they offer their own natural force feedback and pressure distribution.

Stretching piece of Zoflex video >>

Step 6: SOFT CIRCUIT

The soft circuit that connects all the sensors to the power source, microchip and wireless communication device are a very prominent design aspect of the suit. Since for reasons of convenience and necessity they run visibly across the outside of the suit. Rather than seeing this as a problem we view it as one of the main design aspects.
Our (so far) preferred method of creating these conductive traces, which cover distances up to a max of 1.5 - 2 meters (when reaching all the way to the toes), is to work with thing strips of stretch conductive fabric. This method allows us to create stretchy fabric, comfortable, washable conductive connections that have a resistance of about 100Ohm/meter. Other options would be to use conductive thread, but since we work with stretchy materials and the stretchy stitch uses up a lot more thread that I directly sewn connection, we would reach much higher resistance using thread. Also, to be able to sew with a sewing machine, the full layout of the conductive traces on the suit would have to be clear before the suit is assembled and this has never been the case, since there are so many unknowns before the dancer first puts on the finished suit, and we find it much easier to be able to make alterations when the suit is worn and we can see exactly how it fits and where best to place the sensors.
After we have fused interfacing to a large piece of stretch conductive fabric from LessEMF (www.lessemf.com), we cut it into 5mm wide strips and are then able to fuse these to the suit with the heat from an iron. Once the traces are in place we then isolate them by covering them with a good layer of Aleene's Flexible Stretchable Fabric Glue (http://www.duncancrafts.com/). We have found this to be the best solution so far. It does not influence the conductivity of the trace, nor does it much impede on the stretchiness of the fabric. It dries within one day and after 10 days of curing time it can even be quite safely washed with lukewarm water and a mild detergent. The first puppeteer version did not use conductive fabric but we disguised the fragility of the thin wires, by feeding them through fabric tubes. The second Puppeteer version we used only the stretchy fabric glue to isolate the traces and then some baby powder on top to remove their initial stickiness, even once dry they like to stick to each other. For the last Puppeteer version we used the stretchy fabric glues to isolate as well as to mount strips of the same white fabric, so that they were less prominent, but still part of the texture and design of the suit.

In the last Puppeteer version we decided to make the sensor detachable so that we could exchange them and also it because quite a convenient way to attach them to the stretchy conductive circuit. We attached machine poppers to both sides of the sensors as well as to the ends of the conductive traces. We fabricated the sensors in three different lengths:
- Long for the knees, underarms and elbows
- Medium for the hips and shoulders
- Short for the wrists
- Customized shapes for the heels and toes

Step 7: HARD CIRCUIT

Certain parts of the circuit we simply did not want to risk washing or exposing to a lot of wear and tear. These components include: the power source batteries, the Microcontroller Arduino LilyPad (www.cs.colorado.edu/~buechley/diy/diy_lilypad_arduino.html), the XBee wireless module, the Multiplexers, as well as various small components such as voltage regulators, capacitors, resistors and wire connections. In the case of the first Puppeteer suit all of these components were anyway not affixed to the costume itself, because there was a wired connection. In the later costumes we chose to integrate these components in part of the costume that could be removed from the rest of the suit so that the sensors and conductive traces could be washed separately. For the second Puppeteer version we stored these components inside a little backpack type accessory that came out on top of the rest of the costume like a hood and plugged into the ends of the conductive traces coming from the sensors via some handmade lugs that we were could sew to. These we made from headers, wire and Shapelock http://www.shapelock.com
In the third Puppeteer version we integrated the components in the collar of the suit and the collar came right down the front of the suit, reaching the tops of the thighs and plugging into the ends of the conductive traces via metal poppers. In both cases we had access to the circuit by opening a zipper in the little backpack or the collar.

Step 8: WIRELESS COMMUNICATION

Picture of WIRELESS COMMUNICATION
2379927843_9d16d3b724.jpg
2071754428_b4e8ee7707.jpg
The design of the first Puppeteer costume was intentionally connected through a physical wiring, which the performer herself could disconnect, by unplugging herself. These were early stages and at first we used two Arduino USB boards to read the data from all 12 sensors, since the Arduino comes with 6 analog inputs. An improved version of this scaled down to one Arduino, using four of its six analog inputs directly and one of them in combination with an eight channel multiplexer.

The next Puppeteer suit became wireless, giving the dancer freedom to move around with no physical connection to the computer reading the sensor data. The only restrictions were those of the range of the Bluetooth module. We worked with the Sparkfun BlueSMiRF Gold module in connection with the Arduino LilyPad. The Bluetooth connection, though it worked, was very flaky so that for the third Puppeteer version we happily started working with the XBee, which has proved to be a great solution to all our wireless problems.

Step 9: CONCLUSIONS

Picture of CONCLUSIONS
These sensors are easy to make, though time consuming and tedious. They have shown to deliver stable data, which we are able to work. They have not exhibited any drastic changes over time, even after washing with lukewarm water and a mild detergent they continue to react to pressure in the same range. When worn against the body during performance the sensor's range does tend to shift, not necessarily in size but in range location. Weather this is due to wrinkles in the fabric that contains them, or sweat or something that goes on inside the sensor we do not know.

Step 10: NEXT STEPS

The next steps include sensor development independently from creating the next Puppeteer suit.
Other ideas to explore include adding a kind of array of sensors to one costume, the idea being that if we have enough sensors we would be able to track poses and motion sequences that were unique because all the sensors could only ever be in a certain state if a certain pose or motion were being made.

Since the costume is tailored to fit a certain individual body size, the outcome of these projects has been specifically performance based. We are interested in expanding the design of the costume and creating things that can be worn and experienced by all. Here the planning of a modular system might make sense.

So far the Puppeteer sensors focus mainly on the bend of the joints of the human body, but we are interested in detecting other, subtler motions.

Aspects such as comfort, washability, durability and even sustainability (which is something we have not yet included) are things that we would like to continue to address in future.
bravo09234 months ago

I'm thinking about building a MechWarrior Mech for my next costume scaled at 7 foot tall. I'm trying to find something to sync the leg movement with the sound FX of the Mech walking. Any suggestions?

kooljo2 years ago
would this work well for human controlled humanoids?
KDraz2 years ago
I notice that you don't mention resistors anywhere in regards to the circuit. Were no resistors needed in this whole project?
mahsa arian2 years ago
hi. can i give your email? please
tinker2344 years ago
wow hey could use this for a project
flaresam6 years ago
dude, this could totaly be used in a VR video game
hey use this http://unity3d.com/unity/ im going to use it thttp://www.youtube.com/watch?v=5-X-Ebh1kYAhis week need help here
Could something like this be used to capture movement for film special effects, similar to the technique used in Avatar and parts of Final Fantasy VII: Advent Children?
 I've been waiting for a project like this. Thank you.
Seeing this reminded me of http://www.sonalog.com/. Bearing in mind your second project, do you think your project could possibly be hooked up to this system's software (It appears to be free to download, but I'm not on windows, so I don't know if it needs registration). I realise that the Gypsy Midi controller is £900 of techy goodness, but I'm sure there's a lot of people out there who would be interested in controlling lights and visuals and music through a motion capture suit.
Basically, I suppose what I'm really asking is, does the Arduino software have facility for converting to MIDI signals? Or, would a programme like glovepie be a better idea, in a sort of software daisy chain?
Sorry, just seen your comment about glovepie. Still, is it possible to use something like Bome MIDI translator and MIDI Yoke to eventually control lights and suchlike?
rustlabs6 years ago
awesome instructable, nice work!
masterpj6 years ago
oh almost forgot. 5 stars.
i recommend this one and thx for putting time in making this instructable :D
we need more people like you =D
Plusea (author)  masterpj6 years ago
hi, and thanks so much for these comments. i'm going to look into the glovepie program, it sounds super appropriate for what we are interested in. we do smooth the input from the software side to some degree, but since we have been using the data to trigger events, we have not had need to smooth the data further for output purposes.
a collaboration would be wonderful. maybe you can tell us a bit about who you are and your background. you'll find my contact details on my website >> http://www.plusea.at/
masterpj Plusea6 years ago
sure i will see what i can do.
oh and if you need more smoothing =D
glovepie can apply smoothing too.
and the programming used in glovepie is very easy to handle.
and yes i will look at your site for contact and such :D
masterpj6 years ago
plusea one word:
AMAZING!
thats brilliant i really want such suit.
but it looks complicated though.
and a question...
How much did it cost you?
and oh maybe i got an idea to port this to controll (might)
look for the program (glovepie) its an program to easly assign any input to a specific control or device so you can easly port movement to games :D
or other sort.
and another question: isn't it possible to let the computer apply smoothing to the captured motion? so that it wait for the second slight movement and then interpretates the movement (at high speed so still realtime =D )
with interpretate i mean smooth the first frame of your previous position
to the position you are now. :S

and oh, since i got a lot of time on my hands XD
can i contribute to your project in any way? >3 =)
hurtzmyhead6 years ago
What software do you use to record the movements
Plusea (author)  hurtzmyhead6 years ago
a patch written in max/msp
mskogly6 years ago
Perhaps you could add a tilt-sensitive sensor, like the one in a macbook or iphone, that tells you software how many degrees it is. If you place on of those quite low, perhaps just below the navel, you could get a reading of whether the dancer is bending her body, lying down perhaps, etc. I don't know how accurate such sensors are, but worth checking out if you haven't already. Perhaps you could even control the angle of the arms on a 3d model more accurately using this. If the sensor is at the wrist, and you know the length of each part of the arm, you could perhaps calculate pretty accurately the position of the hand, at least if you combine it with the amount of bend from other sensors. Just an idea. I thought the first video with the 3dmodel was rather stiff, didn't seem to respond to much of what the danser was doing.
MagWeb6 years ago
awesome work! Yesterday, I played around animating the bodyscans I did with DAVID-laserscanner - thought of motion capture - found this great instructable. This will be a next project. See you
Renagade6 years ago
brilliant! question though: could you use this as to direct a (already made and fabricated) 3d model of a person, such as to create animations for a video game?
Plusea (author)  Renagade6 years ago
totally!
and in fact the very first version of this motion-capture concept that we made was made for exactly that. a real time puppet play called Ein Kleines Puppenspiel, where the performer was puppeteering a 3d model of herself that was made using the Ink Scanner. the puppet play was set up in the Unreal Tournament game engine by Friedrich Kirschner. also see step 3 for description.
Renagade Plusea6 years ago
srry, but another question: does it record only simple movements? i checked out the "next steps" and it says only at joints, so does that mean only like walking and simple hand movements and leg movements?
Plusea (author)  Renagade6 years ago
the sensors are sensitive to pressure and thus work as bend sensors because pressure is applied through bend. stretch can also be measure if it applies enough pressure to the sensor. the trick is to work with the fit of the clothing and the placement of the sensors on the body. some very subtle movements can be recorded like the shrugging of shoulders, if the sensors are placed correctly... but they are not the best solution of fine movements, you are right they work best for movement of the limbs.
Renagade Plusea6 years ago
oh ok thanx, because my bro cant animate just create characters, so i help out. And now you are part of that process thanx!
Renagade Plusea6 years ago
THANKS! cuz me and my brother are designing a video game, and we can't go through the hassle of animating every frame, I mean if we can do this might as well just record motion and strap it onto the model right _
LinkVanyali6 years ago
This is really fantastic. Good job. 5/5 +fav
This is really cool. I'm surprised that it only got these two comments so far.