3D Motion Capture

Intro: 3D Motion Capture

Ever wanted to sketch something in 3D? Well, now you can break free of the confines of 2D paper for the price of two webcams.

I made this project as a present for some former professors of mine. The original intention was to track a dancer's movements. Since I'm not so great at dancing, I used it to sketch.

Feel free to modify/hack/otherwise dismember the code I've posted.

What you'll need:

~ A computer running some flavor of Windows (all the software has been tested on VIsta and XP)

~ 2 webcams (I found mine on eBay for about $23)

~ A green LED (must be of the diffuse variety, or covered in translucent nail polish)

~ The capture applet I coded in Processing.

~ WinVDIG 1.0.1 (NOT the latest version) This will hook up Processing to your webcams.

~ Some way of keeping your cameras at 90 degrees to each other. You could use two friends, half a box with holes cut in it, or the adjustable rig I'll provide instruction for in the next step.

~ If you want to generate a sculpture from your path, you'll also need Rhino 4 SR3 (which is free to try), a 3D modeling program, and its script editor Monkey along with the sculpture generator script I wrote.

~ If you then want to print the sculpture, you'll need access to a 3D printer. If you don't happen to have one just lying around, I'd suggesting trying to get into the Shapeways beta.

You can find a .zip of the applet (with source code) and the RhinoScript files attached to this step.

STEP 1: Making the Rig

You can use any method to keep your cameras at 90 degrees. I designed this rig to be portable and easily adjustable to different sizes. However, if you're looking to create something for sketching next to your computer, I'd suggest making three faces of a cube out of cardboard and cutting holes in two of then for the cameras.

Materials:
~ 2 drawer runners. These must be of the kind that don't come apart when not actually wedged inside a drawer. I got a pair of 18" ones at Home Depot for $14.
~ 1 large hinge. The holes in this hinge need to line up with at least two of the holes in the drawer runners.
~ 4 bolts, 4 nuts, and 8 washers. To connect the hinge and drawer runners. The bolts should be fairly short (<3/4") and the washers should be as wide as you can find them.
~ A wrench. To tighten the bolts.
~ A C-clamp. To (eventually) attach the rig to a table or other horizontal surface.

Open one of the drawer runners. Line up one side of the hinge with the holes in the wide end of the drawer runner. Bolt the two together, making sure to place a washer on either side. Repeat with the other drawer runner and other side of the hinge.

Congrats, you're all done with this step.

STEP 2: Setup

Before you can begin sketching you'll need to install some software and setup the rig. Here's how:

Step (1)

To begin, install WinVDIG. Its a video digitizer component for Quicktime that will let the Processing video library access the webcams. The version is 1.0.1. The newest version of WinVDIG is 1.0.5, but it doesnt work with the newer versions of Quicktime. Go figure.

Step (2)

Then plug in the cameras and let Windows find the drivers.

Step (3)

While Windows is searching, and assuming you're using the rig detailed in the last step, open the camera rig and clamp it to the corner of a table. Be sure not to tighten the clamp too much or the drawer runners wont open easily.

Step (4)

Slide out the ends of the drawer runners a few inches so that you can attach the cameras to them. Open the clips on the bottoms of the cameras and press them over the top edges of the drawer runners.

Step (5)

Point the cameras directly ahead. Extend the drawer runners so that the cameras can point at the place the dancer will be standing.





Now you're ready to start the applet.

STEP 3: Using the Applet

Now you're ready to start making sketches.

Step (1)

Unzip the Capture Applet to a local drive, then open the exe inside the windows.application folder. After a few seconds a window should open that looks something like the image below. Now, either turn on your LED (if you decided to go with a pre-made flashlight) or slide a battery in between the two leads. If you're going with option two, make sure the voltage of the battery falls within the LED's acceptable range.

Step (2)

Position you light so that it can be seen by both cameras. A small green circle will show you the position that the camera sees as the greenest spot on each video feed. I've defined the greenness of a pixel based on its RGB value like this:

Greenness = G/4 + (G-R) + (G-B)

Which favors bright green pixels, making a semi-dark room without many green things in it an ideal environment.

The positions of the 0.5% greenest are averaged to find position of the light in each video feed. Both coordinates from the XY camera feed and one from the Z camera feed are used to construct a 3D point.

Step (3)

When you're ready, press the Start Capture button. A green line will appear in the Path Display box. That is the XY plane projection of your path. It should give you an idea of how much of your frame you're using, but (if you're drawing instead of attaching the LED to a dancer) you shouldn't use it as a drawing guide. Instead, think about the shape you're drawing in space. You'll get a much better result.

Step (4)

Press Stop Capture when you're done. If you're happy with the path you've draw, press Export. You'll notice that the export button has a number after it. That number will increase each time you make a new path and choose to export it. The number you see at the time of exporting will be the name of the exported text file. To keep all names unique, the date and time at which they are exported is also included in the name. The text files will write to the same folder as the exe and will be named like this:

path_data_1_y2008m8d19h10m2s25.txt

Step (5)

You can record and export as many paths as you want. When you're finished, make sure to quit by clicking on the small X in the corner of the applet. That will clear the cameras from Quicktime.

Now you're ready to make the sculpture geometry.

STEP 4: Generating the Sculpture

You can now generate a 3D model (and 3D printer file) using Rhino and the script I've written. I've tried to write these instructions for someone with no experience using Rhino/3D modeling software, so even if you've never modeled anything before you should be able to follow them. If you have any trouble feel free to message me and I'll try to get back to you ASAP.

Step (1)

Open Rhino.

Step (2)

Look in the menus on the top for Monkey (Rhino's script editor).

Step (3)

If you can't find it, that means you'll have to run the Monkey installer. Close Rhino, run the installer, and re-open Rhino. You should see Monkey in the menu bar.

Step (4)

Before running the script, you'll need to set up the units for the file. Type Units into the command line. Choose your unit of preference from the drop down menu.

Step (5)

Now, type Monkey into the command line. You should see the text Monkey is initializing... and then a window should appear. This is the script editor.

Step (6)

Find the script file (sculpturegeneratorv3.rvb) and drag it into the tab strip near the top of the script editor window.

Step (7)

Click the Run Script button.

Step (8)

A window will pop up asking for the point text file that was exported from the applet. You'll find it in the same folder as the exe of the applet.

Step (9)

You'll be prompted in the command line to enter the maximum dimension. The dimension will be in the units system of the file, which you set earlier.

Step (10)

The sculpture will begin to draw. To speed things, minimize the Rhino window. If Rhino doesn't need to redraw each time its geometry is updated, it runs much faster.

Step (11)

When the script is done running, the Monkey window will pop up. You can them restore the main Rhino window.

Step (12)

If you look in the point text file's/applet's directory you'll now see an .stl file with the same name as the text file. This file should be ready for printing by most 3D printers.

Step (13)

And that's it! You can import more paths, or save your Rhino file and use it for renderings or modify by hand.





You can send out the stl files you've just made to be 3D printed. They should be acceptable for almost any kind of 3D printer.

STEP 5: Gallery

Here are some renderings of the sculptures I created. The last two images are of the printed sculpture.

I hope you have fun with this project! If you decide to make your own, I'd love to see pics/renderings of what you come up with.

Thanks for visiting my instructable!

Comments

[QpL]

hello, this project is really attractive :D I'm using the logitech C270 webcam, some how when I click on the exe file in the application folder, nothing happen QwQ
Does anyone know how should I fix it or where can I find the relative support? thanks a lot!

[AtomicDragon]

Is it possible to use this system to do full body motion data for character animations, for 3D film studios?

[mykiscool]

Can you make a maya version of this also, since that is the industry standard software.

[vitoboy]

Hey, first of all congratulations on this great instructable. With your permission I am going to use it for my project to get my industrial engineer degree. I need it to study the movement of an industrial process. For that, I would like to know what are the intervals between measurements? Are they defined by the frames per second you set on WinVDIG 1.0.1? I would like to know also how can I make a second tracker for another color of led (my programming skills are very poor). Thank you very much and congrats again for this applet.

[jftesser]

It'd be great if you used this in your project! I'd love to see the results. The sampling frequency is determined by the frame rate of the applet, i.e. how fast your cpu is. You could add code to modify this in the applet, or you could simply discard data points in the resultant CSV file. And, yes, a second color can be added pretty easily. There seems to be some interest in multiple LED tracking. Send me a message and we can talk about how to implement it. Good luck on your project! Jenna

[vitoboy]

Hello Jenna, thank you for your answer. Well at the moment I just tried the applet on a PC that is not that powerful. I am planning on using a better PC for the project but I think I need to buy a new one. How can I write a code to get 24 measurements per second for example? Or should I just change the frames per second on WinVDIG 1.0.1? For my project I am going to use the applet on different applications. I am going to study first the movements on both hands on a worker, lets say we put a green led on a hand and a red led on the other (then with the measurements on a matrix I can calculate speed and acceleration to get an estimate on how much work it s done), How can I write the other tracker? I would like to change the color of the tracker too to follow an Infrared led (so I won’t have to turn off all the lights) it appears white on the screen if you use a filter (I just learned you can make an IR filter out of a diskette). I want to calculate how much the worker moves on the workshop. I really, really appreciate your help.

[vaultshadow]

does it work with blender?

[jftesser]

You have to use Rhino (in windows) to generate the sculpture, but almost any 3D program will work to render the result.

[mtownsend]

I don't think I'm entirely grasping how to do this. Do you just draw the entire shape in midair, doing all the dimensions? If I were to, say, create a... I don't know.... let's say a dinosaur. If I were to make a dinosaur would I have to do all of the curves on the side view and then draw how wide each part would be? More or less, would it be like carving out of a block of wood where you draw on the top and the side to get the basic dimensions?

[jftesser]

The cameras will capture your motion in all dimensions. It works exactly the same as drawing a line on a piece of paper, except in 3 dimensions.

[explnme]

hi ! Source : Rhinoscript Error: Type mismatch in parameter. Array required. Line : 158 Char : 1 Code : 0 Does it make sense for you?

[lioncour]

Hello, very nice Instructable, and nice code. I got a problem, when i run the the script in monkey i get 20 meshes, most of them are round and when i render i only get like this, (look picture) I have tried to draw 10 different "sculptures" with the lights but still the meshes look very much alike. Any idea what i am doing wrong?

[explnme]

got the same problem... have you solved it? it first perform my drawing and then make an other, a spheric one ... plz help...

[jftesser]

I've never had this happen myself, so I don't know how helpful I can be. You can try the things I suggested in my other reply (try a new rhino file, pay attention to the path preview). Also, I have noticed that using a small scale number in rhinoscript results in more bulgy sphere-things, so you could try just using a bigger number. If you can't solve the problem, I'd be happy to take a look at your points text file and your rhino file. You can send them to [e-mail].

[explnme]

it doesn 't come from the text file caus' i succeed in seeing it in autocad (http://management.cadalyst.com/cadman/AutoCAD/Import-3D-points-from-Excel/ArticleStandard/Article/detail/101138) without the spheres... It looks like if it perfomed spheres (a lot) in rhino after having performed my path. Could it comes from your script? or from my rhino version rhino 4.0? is it a sort of demo trick of the abbilities of rhino, you can switch off ?(it s the first time i use this software)

[explnme]

Here it is, at the end of the script ! in yellow : the sphere thing for the first image, and for the second image path +little circles (selected via "select curves only" command) in the centre of the sphere things in purple. in what scale do you execute your script? what s the dimension u use? could be helpfull maybe...

[jftesser]

What scale works best actually depends on a lot of factors...what unit system you're in, if you chose small or large objects, how close the twists and turns are in your sculpture. To be safe, I'd suggest increasing by a factor of 10. You can always scale down the result and reexport it at the size you really if you plan on 3D printing.

[explnme]

if you mean increase the dimension by increasing by a factor of ten, well it doesn t change anything from 1 to 1000 (when asking what s the aproximate maximum dimension) in whatever unit system i choose. i'm afraid i'm lost. i can see the drawing until asking dimension and then its "pipe failed" ad libitum in the dialog box and big spherics things...in fact i never get the stl file... and i ve been trying all the night and day long all scales...

[jftesser]

Can you email me your points file? [e-mail]

[explnme]

sorry i ve been sleeping a while, here's the coordinates:

http://www.2shared.com/file/5172021/ef9485aa/path_data_1_y2008m3d24h6m3s18.html