3D Motion Capture





Introduction: 3D Motion Capture

Ever wanted to sketch something in 3D? Well, now you can break free of the confines of 2D paper for the price of two webcams.

I made this project as a present for some former professors of mine. The original intention was to track a dancer's movements. Since I'm not so great at dancing, I used it to sketch.

Feel free to modify/hack/otherwise dismember the code I've posted.

What you'll need:

~ A computer running some flavor of Windows (all the software has been tested on VIsta and XP)

~ 2 webcams (I found mine on eBay for about $23)

~ A green LED (must be of the diffuse variety, or covered in translucent nail polish)

~ The capture applet I coded in Processing.

~ WinVDIG 1.0.1 (NOT the latest version) This will hook up Processing to your webcams.

~ Some way of keeping your cameras at 90 degrees to each other. You could use two friends, half a box with holes cut in it, or the adjustable rig I'll provide instruction for in the next step.

~ If you want to generate a sculpture from your path, you'll also need Rhino 4 SR3 (which is free to try), a 3D modeling program, and its script editor Monkey along with the sculpture generator script I wrote.

~ If you then want to print the sculpture, you'll need access to a 3D printer. If you don't happen to have one just lying around, I'd suggesting trying to get into the Shapeways beta.

You can find a .zip of the applet (with source code) and the RhinoScript files attached to this step.

Step 1: Making the Rig

You can use any method to keep your cameras at 90 degrees. I designed this rig to be portable and easily adjustable to different sizes. However, if you're looking to create something for sketching next to your computer, I'd suggest making three faces of a cube out of cardboard and cutting holes in two of then for the cameras.

~ 2 drawer runners. These must be of the kind that don't come apart when not actually wedged inside a drawer. I got a pair of 18" ones at Home Depot for $14.
~ 1 large hinge. The holes in this hinge need to line up with at least two of the holes in the drawer runners.
~ 4 bolts, 4 nuts, and 8 washers. To connect the hinge and drawer runners. The bolts should be fairly short (<3/4") and the washers should be as wide as you can find them.
~ A wrench. To tighten the bolts.
~ A C-clamp. To (eventually) attach the rig to a table or other horizontal surface.

Open one of the drawer runners. Line up one side of the hinge with the holes in the wide end of the drawer runner. Bolt the two together, making sure to place a washer on either side. Repeat with the other drawer runner and other side of the hinge.

Congrats, you're all done with this step.

Step 2: Setup

Before you can begin sketching you'll need to install some software and setup the rig. Here's how:

Step (1)

To begin, install WinVDIG. Its a video digitizer component for Quicktime that will let the Processing video library access the webcams. The version is 1.0.1. The newest version of WinVDIG is 1.0.5, but it doesnt work with the newer versions of Quicktime. Go figure.

Step (2)

Then plug in the cameras and let Windows find the drivers.

Step (3)

While Windows is searching, and assuming you're using the rig detailed in the last step, open the camera rig and clamp it to the corner of a table. Be sure not to tighten the clamp too much or the drawer runners wont open easily.

Step (4)

Slide out the ends of the drawer runners a few inches so that you can attach the cameras to them. Open the clips on the bottoms of the cameras and press them over the top edges of the drawer runners.

Step (5)

Point the cameras directly ahead. Extend the drawer runners so that the cameras can point at the place the dancer will be standing.

Now you're ready to start the applet.

Step 3: Using the Applet

Now you're ready to start making sketches.

Step (1)

Unzip the Capture Applet to a local drive, then open the exe inside the windows.application folder. After a few seconds a window should open that looks something like the image below. Now, either turn on your LED (if you decided to go with a pre-made flashlight) or slide a battery in between the two leads. If you're going with option two, make sure the voltage of the battery falls within the LED's acceptable range.

Step (2)

Position you light so that it can be seen by both cameras. A small green circle will show you the position that the camera sees as the greenest spot on each video feed. I've defined the greenness of a pixel based on its RGB value like this:

Greenness = G/4 + (G-R) + (G-B)

Which favors bright green pixels, making a semi-dark room without many green things in it an ideal environment.

The positions of the 0.5% greenest are averaged to find position of the light in each video feed. Both coordinates from the XY camera feed and one from the Z camera feed are used to construct a 3D point.

Step (3)

When you're ready, press the Start Capture button. A green line will appear in the Path Display box. That is the XY plane projection of your path. It should give you an idea of how much of your frame you're using, but (if you're drawing instead of attaching the LED to a dancer) you shouldn't use it as a drawing guide. Instead, think about the shape you're drawing in space. You'll get a much better result.

Step (4)

Press Stop Capture when you're done. If you're happy with the path you've draw, press Export. You'll notice that the export button has a number after it. That number will increase each time you make a new path and choose to export it. The number you see at the time of exporting will be the name of the exported text file. To keep all names unique, the date and time at which they are exported is also included in the name. The text files will write to the same folder as the exe and will be named like this:


Step (5)

You can record and export as many paths as you want. When you're finished, make sure to quit by clicking on the small X in the corner of the applet. That will clear the cameras from Quicktime.

Now you're ready to make the sculpture geometry.

Step 4: Generating the Sculpture

You can now generate a 3D model (and 3D printer file) using Rhino and the script I've written. I've tried to write these instructions for someone with no experience using Rhino/3D modeling software, so even if you've never modeled anything before you should be able to follow them. If you have any trouble feel free to message me and I'll try to get back to you ASAP.

Step (1)

Open Rhino.

Step (2)

Look in the menus on the top for Monkey (Rhino's script editor).

Step (3)

If you can't find it, that means you'll have to run the Monkey installer. Close Rhino, run the installer, and re-open Rhino. You should see Monkey in the menu bar.

Step (4)

Before running the script, you'll need to set up the units for the file. Type Units into the command line. Choose your unit of preference from the drop down menu.

Step (5)

Now, type Monkey into the command line. You should see the text Monkey is initializing... and then a window should appear. This is the script editor.

Step (6)

Find the script file (sculpturegeneratorv3.rvb) and drag it into the tab strip near the top of the script editor window.

Step (7)

Click the Run Script button.

Step (8)

A window will pop up asking for the point text file that was exported from the applet. You'll find it in the same folder as the exe of the applet.

Step (9)

You'll be prompted in the command line to enter the maximum dimension. The dimension will be in the units system of the file, which you set earlier.

Step (10)

The sculpture will begin to draw. To speed things, minimize the Rhino window. If Rhino doesn't need to redraw each time its geometry is updated, it runs much faster.

Step (11)

When the script is done running, the Monkey window will pop up. You can them restore the main Rhino window.

Step (12)

If you look in the point text file's/applet's directory you'll now see an .stl file with the same name as the text file. This file should be ready for printing by most 3D printers.

Step (13)

And that's it! You can import more paths, or save your Rhino file and use it for renderings or modify by hand.

You can send out the stl files you've just made to be 3D printed. They should be acceptable for almost any kind of 3D printer.

Step 5: Gallery

Here are some renderings of the sculptures I created. The last two images are of the printed sculpture.

I hope you have fun with this project! If you decide to make your own, I'd love to see pics/renderings of what you come up with.

Thanks for visiting my instructable!



    • Pocket-Sized Contest

      Pocket-Sized Contest
    • Pro Tips Challenge

      Pro Tips Challenge
    • Science of Cooking

      Science of Cooking

    We have a be nice policy.
    Please be positive and constructive.




    Is it possible to use this system to do full body motion data for character animations, for 3D film studios?

    Can you make a maya version of this also, since that is the industry standard software.

    Hey, first of all congratulations on this great instructable. With your permission I am going to use it for my project to get my industrial engineer degree. I need it to study the movement of an industrial process. For that, I would like to know what are the intervals between measurements? Are they defined by the frames per second you set on WinVDIG 1.0.1? I would like to know also how can I make a second tracker for another color of led (my programming skills are very poor). Thank you very much and congrats again for this applet.

    It'd be great if you used this in your project! I'd love to see the results. The sampling frequency is determined by the frame rate of the applet, i.e. how fast your cpu is. You could add code to modify this in the applet, or you could simply discard data points in the resultant CSV file. And, yes, a second color can be added pretty easily. There seems to be some interest in multiple LED tracking. Send me a message and we can talk about how to implement it. Good luck on your project! Jenna

    Hello Jenna, thank you for your answer. Well at the moment I just tried the applet on a PC that is not that powerful. I am planning on using a better PC for the project but I think I need to buy a new one. How can I write a code to get 24 measurements per second for example? Or should I just change the frames per second on WinVDIG 1.0.1? For my project I am going to use the applet on different applications. I am going to study first the movements on both hands on a worker, lets say we put a green led on a hand and a red led on the other (then with the measurements on a matrix I can calculate speed and acceleration to get an estimate on how much work it s done), How can I write the other tracker? I would like to change the color of the tracker too to follow an Infrared led (so I won’t have to turn off all the lights) it appears white on the screen if you use a filter (I just learned you can make an IR filter out of a diskette). I want to calculate how much the worker moves on the workshop. I really, really appreciate your help.

    You have to use Rhino (in windows) to generate the sculpture, but almost any 3D program will work to render the result.

    I don't think I'm entirely grasping how to do this. Do you just draw the entire shape in midair, doing all the dimensions? If I were to, say, create a... I don't know.... let's say a dinosaur. If I were to make a dinosaur would I have to do all of the curves on the side view and then draw how wide each part would be? More or less, would it be like carving out of a block of wood where you draw on the top and the side to get the basic dimensions?

    The cameras will capture your motion in all dimensions. It works exactly the same as drawing a line on a piece of paper, except in 3 dimensions.

    hi ! Source : Rhinoscript Error: Type mismatch in parameter. Array required. Line : 158 Char : 1 Code : 0 Does it make sense for you?