3D Motion Capture

Ever wanted to sketch something in 3D? Well, now you can break free of the confines of 2D paper for the price of two webcams.

I made this project as a present for some former professors of mine. The original intention was to track a dancer's movements. Since I'm not so great at dancing, I used it to sketch.

Feel free to modify/hack/otherwise dismember the code I've posted.

What you'll need:

~ A computer running some flavor of Windows (all the software has been tested on VIsta and XP)

~ 2 webcams (I found mine on eBay for about $23)

~ A green LED (must be of the diffuse variety, or covered in translucent nail polish)

~ The capture applet I coded in Processing.

~ WinVDIG 1.0.1 (NOT the latest version) This will hook up Processing to your webcams.

~ Some way of keeping your cameras at 90 degrees to each other. You could use two friends, half a box with holes cut in it, or the adjustable rig I'll provide instruction for in the next step.

~ If you want to generate a sculpture from your path, you'll also need Rhino 4 SR3 (which is free to try), a 3D modeling program, and its script editor Monkey along with the sculpture generator script I wrote.

~ If you then want to print the sculpture, you'll need access to a 3D printer. If you don't happen to have one just lying around, I'd suggesting trying to get into the Shapeways beta.

You can find a .zip of the applet (with source code) and the RhinoScript files attached to this step.

Step 1 Making the Rig

You can use any method to keep your cameras at 90 degrees. I designed this rig to be portable and easily adjustable to different sizes. However, if you're looking to create something for sketching next to your computer, I'd suggest making three faces of a cube out of cardboard and cutting holes in two of then for the cameras.

Materials:
~ 2 drawer runners. These must be of the kind that don't come apart when not actually wedged inside a drawer. I got a pair of 18" ones at Home Depot for $14.
~ 1 large hinge. The holes in this hinge need to line up with at least two of the holes in the drawer runners.
~ 4 bolts, 4 nuts, and 8 washers. To connect the hinge and drawer runners. The bolts should be fairly short (<3/4") and the washers should be as wide as you can find them.
~ A wrench. To tighten the bolts.
~ A C-clamp. To (eventually) attach the rig to a table or other horizontal surface.

Open one of the drawer runners. Line up one side of the hinge with the holes in the wide end of the drawer runner. Bolt the two together, making sure to place a washer on either side. Repeat with the other drawer runner and other side of the hinge.

Congrats, you're all done with this step.

Comments

[vitoboy]

Hey, first of all congratulations on this great instructable. With your permission I am going to use it for my project to get my industrial engineer degree. I need it to study the movement of an industrial process. For that, I would like to know what are the intervals between measurements? Are they defined by the frames per second you set on WinVDIG 1.0.1? I would like to know also how can I make a second tracker for another color of led (my programming skills are very poor). Thank you very much and congrats again for this applet.

[jftesser]

It'd be great if you used this in your project! I'd love to see the results. The sampling frequency is determined by the frame rate of the applet, i.e. how fast your cpu is. You could add code to modify this in the applet, or you could simply discard data points in the resultant CSV file. And, yes, a second color can be added pretty easily. There seems to be some interest in multiple LED tracking. Send me a message and we can talk about how to implement it. Good luck on your project! Jenna

[vitoboy]

Hello Jenna, thank you for your answer. Well at the moment I just tried the applet on a PC that is not that powerful. I am planning on using a better PC for the project but I think I need to buy a new one. How can I write a code to get 24 measurements per second for example? Or should I just change the frames per second on WinVDIG 1.0.1? For my project I am going to use the applet on different applications. I am going to study first the movements on both hands on a worker, lets say we put a green led on a hand and a red led on the other (then with the measurements on a matrix I can calculate speed and acceleration to get an estimate on how much work it s done), How can I write the other tracker? I would like to change the color of the tracker too to follow an Infrared led (so I won’t have to turn off all the lights) it appears white on the screen if you use a filter (I just learned you can make an IR filter out of a diskette). I want to calculate how much the worker moves on the workshop. I really, really appreciate your help.

[vaultshadow]

does it work with blender?

[jftesser]

You have to use Rhino (in windows) to generate the sculpture, but almost any 3D program will work to render the result.

[mtownsend]

I don't think I'm entirely grasping how to do this. Do you just draw the entire shape in midair, doing all the dimensions? If I were to, say, create a... I don't know.... let's say a dinosaur. If I were to make a dinosaur would I have to do all of the curves on the side view and then draw how wide each part would be? More or less, would it be like carving out of a block of wood where you draw on the top and the side to get the basic dimensions?

[jftesser]

The cameras will capture your motion in all dimensions. It works exactly the same as drawing a line on a piece of paper, except in 3 dimensions.

[explnme]

hi ! Source : Rhinoscript Error: Type mismatch in parameter. Array required. Line : 158 Char : 1 Code : 0 Does it make sense for you?

[lioncour]

Hello, very nice Instructable, and nice code. I got a problem, when i run the the script in monkey i get 20 meshes, most of them are round and when i render i only get like this, (look picture) I have tried to draw 10 different "sculptures" with the lights but still the meshes look very much alike. Any idea what i am doing wrong?

[explnme]

got the same problem... have you solved it? it first perform my drawing and then make an other, a spheric one ... plz help...

[jftesser]

I've never had this happen myself, so I don't know how helpful I can be. You can try the things I suggested in my other reply (try a new rhino file, pay attention to the path preview). Also, I have noticed that using a small scale number in rhinoscript results in more bulgy sphere-things, so you could try just using a bigger number. If you can't solve the problem, I'd be happy to take a look at your points text file and your rhino file. You can send them to [e-mail].

[explnme]

it doesn 't come from the text file caus' i succeed in seeing it in autocad (http://management.cadalyst.com/cadman/AutoCAD/Import-3D-points-from-Excel/ArticleStandard/Article/detail/101138) without the spheres... It looks like if it perfomed spheres (a lot) in rhino after having performed my path. Could it comes from your script? or from my rhino version rhino 4.0? is it a sort of demo trick of the abbilities of rhino, you can switch off ?(it s the first time i use this software)

[jftesser]

As far as I can tell, the spheres are an artifact of the surfaces trying to fit a path that is too small. Try using a larger scale factor and they should disappear.

[explnme]

Here it is, at the end of the script ! in yellow : the sphere thing for the first image, and for the second image path +little circles (selected via "select curves only" command) in the centre of the sphere things in purple. in what scale do you execute your script? what s the dimension u use? could be helpfull maybe...

[jftesser]

What scale works best actually depends on a lot of factors...what unit system you're in, if you chose small or large objects, how close the twists and turns are in your sculpture. To be safe, I'd suggest increasing by a factor of 10. You can always scale down the result and reexport it at the size you really if you plan on 3D printing.

[explnme]

if you mean increase the dimension by increasing by a factor of ten, well it doesn t change anything from 1 to 1000 (when asking what s the aproximate maximum dimension) in whatever unit system i choose. i'm afraid i'm lost. i can see the drawing until asking dimension and then its "pipe failed" ad libitum in the dialog box and big spherics things...in fact i never get the stl file... and i ve been trying all the night and day long all scales...

[jftesser]

Can you email me your points file? [e-mail]

[explnme]

sorry i ve been sleeping a while, here's the coordinates:

http://www.2shared.com/file/5172021/ef9485aa/path_data_1_y2008m3d24h6m3s18.html

[jftesser]

I ran the file you sent and it worked fine for me. My settings were small objects centimeters and a max dimension of 50. Here's a link to the results:

http://dl.getdropbox.com/u/90032/path_data_1_y2008m3d24m3s18.zip

[explnme]

ok, first thanks for the time you spend with my problem. Looking closely to the process, i think i have something going wrong beetween the radius and my mouse. It's like for each part of the curve when your script design the circles around, if it took my pointer(where ever it is, even out of the rhino windows) to choose the radius of the circle or the point where the two circles cross each other, does it makes sens for you ?

[jftesser]

Honestly? Not really. Unless you were holding down your left mouse button the entire time that shouldn't have happened. Also, the script will run much faster if you minimize Rhino while it's running (the geometry won't have to be redrawn each time a segment is added).

[explnme]

i'm not holding the left button of course neither the right one. If i minimize Rhino, the script never end, i tried once, 8 hours and no response, the script ends if i do not minimze but with huges radius circles. So it makes big spherical things... I wonder why, if you do not know . well, i m going to search again and again, and i will tell u when i ll find the answear. Thanks again, and congrats for your nice instructable.

[jftesser]

You need to input the maximum dimension first, but then the script really should run through in about 15 seconds. I really don't know why that's not working.

[explnme]

hi, I didn t succeed yet to run your script completely, but i found that if i run the script from the main window using "run active script"(and not from the script editor) well, i avoid big sphericals things and interaction with the mouse. the sculpture begin to draw but then i have an error message saying: Source : Rhinoscript Error: Type mismatch in parameter. Array required. Line : 158 Char : 1 Code : 0 Does it make sense for you?

[jftesser]

Hmmm. Have you run the script in the same file each time? You might want to try a fresh file for each run. Also, do the previews of your paths look significantly different in the applet? If they don't, it's possible that your paths are all very similar. If you can't figure out what's going on, feel free to e-mail me a copy of the path data or rhino file and I'll take a look at it. Thanks for trying my instructable! Jenna [e-mail]

[explnme]

got the same problem... have you solved it? it first perform my drawing and then make an other, a spheric one ... plz help...

[totoleheros]

Do the webcams must be the same model? i tried with two webcam, twice and i cannot have the second screen... so no path... thanks in advance for answearing.

[jftesser]

They don't, but it's a good idea if they have similar specs (fps, resolution, etc). The applet I wrote should automatically cycle through *any* camera or imaging devices attached to your computer, so it's important to be sure that you have exactly two webcams attached!

[totoleheros]

I tried with :
- 2 logitech : quickcam zoom and quickcam messenger there s only the "z capture field"working with quickzoom.
- a quickam messenger and a hercules "dualpix hd webcam" there's only the "z capture field" working with hercules
-quickam messenger work if it s plug with playstation eye but only in the z capture field.
-but with playstation eye : image frozen if it s plug alone and no "x"image like in the other examples
of course i did try the display path button and i recorded too but it makes coordinates without x ( x=0 all the record long..)
Hope you can help me. thx in advance.
urs sincerly.

[totoleheros]

(removed by author or community request)

[phedhex]

O man - these are cool! I'll have to take a look at this project when I get some free time. Bookmarked!

[leahbuechley]

fantastic instructable! thanks for sharing!

[smiling_gandalf]

seriously awesome dude! im gonna try to incorporate this into my A2 project

[tammasus]

Hey! Excellent project!. Can I capture motion and animate a model with it? like they do in movies? just a thought.

[mwwdesign]

What renderer did you use for the 3D renders? ...and what lighting rig? They look great!

[jftesser]

I used Vray with simple GI lighting. The material is a single color with high reflection.

[mwwdesign]

Thanks, I've been using Flamingo for years, but think it's time to get into Vray...

[jhchawk]

Hey...I was just thinking about trying to combine this project with one I've just been thinking about....trying to make an accurate model of my head (haha). Do you think if I used a green light refracted (refracted? idk if that's the right verb) through, for example, a wine glass, to make a line instead of a dot of light, and then rotate my head with the green line on it, your software could run it? hit me up at [e-mail], I'm kind of a noob at code, but i know my way around 3d modeling a bit.

[Culturedropout]

You need to take a look at "David" http://david-laserscanner.com/ to do something like that. I've played with it a bit (although I was running it inside VMware since I don't run Windoze natively) and it worked pretty well.

[lordhazzard]

woah, we actually have been doing this, but with a guy and 20 ping pong balls. now this really only works with one point in space.if you have 2, and the camera loses site of one, of they cross, u have to do a whole lot of crazy math. Also with this set up, u cant do more than one point because of perspective. but looks cool.