How to Build a Multi-Touch Surface

Intro: How to Build a Multi-Touch Surface

Multi-touch gestures are the latest and greatest way to interact with your computer.  However, commercial systems are not widely available and are expensive.  Fortunately, you can do it yourself better and for less cost.

Principle components (Click links for product info):
$35 PS3 Eye Camera (Recommended but any decent webcam will work)
$20 IR bandpass filter
$200 5-meter dimmable IR LED Strip
$1000 Any recent/decent computer (PC/MAC)
$1000 Any projector >2000 lumens (NEC NP410W)
$40 Front surface mirror
$80 Abrasion resistant acrylic sheet (23" x 35" x 0.236")
$130 Rear projection acrylic sheet (23" x 35" x 0.118")
$145 EndLighten acrylic sheet (23" x 35" x 0.394")
Total: ~$2500

Software (Click links for downloads):
$free CCV (Used to turn images into blobs)
$free CL-Eye Platform Driver (For PS3 Eye camera)
$free Multi-Touch Vista (Optional Multi-Touch TUIO driver for Windows integration)
$? Windows 7 (Recommended)

Written by Justin Klaassen, Duke University B.S.E. 2012
Sponsored by the Duke Library

STEP 1: What Is a Multi-Touch Surface?

 A Multi-Touch surface is essentially a giant touch screen that can process multiple touches at once.  And in our case, a virtually unlimited amount of touches.  By allowing multiple touches, gestures can be performed on surface.  These gestures are typically simple things like pinching and pulling on an image to zoom in and out.  By supporting gestures like these interfacing with a computer becomes much more intuitive and fun.  In a public place, a Multi-Touch surface can be a real attention grabber.

STEP 2: How Does It Work?

 There are a variety of different technologies that can be used to create a Multi-Touch surface.  All or most surfaces rely on optical based technologies that work using infrared light (IR).  The technology used in this guide is direct surface illumination (DSI) and is explained by the comic below.  The basis for using DSI over other similar technologies is that it allows for a protective abrasion resistant layer to be on top, protecting the delicate under layers from the users touches.  The order of acrylic layers on the surface is as follows:

1) Touch surface (Abrasion resistant acrylic)
2) Diffuser/Projection layer (Rear projection acrylic)
3) EndLighten acrylic

STEP 3: Build a Computer

 You do not need to build a computer specifically for this project.  In fact, any recent computer with a decent processor and graphics card should work well.  However, if you plan on putting the surface in an enclosure when you are done, then ensure the computer has adequate cooling and minimal thermal dissipation.  Another important factor is the size of the computer, since you want to make sure the computer does not interfere with the camera or projection systems.

Example components used in our rig:
Intel Core i5-660 Clarkdale 3.33 GHz
ASUS P7H55D-M EVO Micro-ATX Motherboard
A-DATA 4GB (2 x 2GB) PC3 12800 Memory
XFX HD-567X-YNFC Radeon HD 5670

STEP 4: Modify the Camera

 In order to pick up infrared light the PS3 Eye camera will need to be opened up and modified.  A PS3 Eye camera is used since it is a) cheap and b) supports  a high resolution (640x480) at 60 fps.  This speed and resolution are perfect for a multi-touch surface since they provide the detail needed to pick up small blobs and the speed to make gestures appear smooth.

In order for the PS3 Eye camera to work, the IR filter needs to be removed and an IR bandpass filter needs to be put in its place.  The IR bandpass filter ensures that only the IR light with the utilized wavelength makes it into the system.  Therefore it is important that the IR bandpass filter's centering wavelength matches that of the IR LEDs present in the IR strip used for illuminating the surface.  A video guide created by Peau Productions for modifying the PS3 Eye camera can be found below.

STEP 5: Prepare the Acrylic

 The most important acrylic layer is the EndLighten acrylic.  This layer is the one responsible for redirecting the IR light from its sides out the top and bottom surfaces of the acrylic.  Therefore it is necessary to polish the edges of this layer in order to maximize the amount of IR light that can be injected into it.

If it is possible, I recommend laser cutting the EndLighten layer since that will automatically polish the edge to perfection.  If a laser cutter is not an option, then automotive sandpaper will do the trick.  Start with 600 grit (lower if the saw cut is rough) and sand each edge until they are perfectly smooth.  Finish each of the sides with 1000 grit.  Finally, polish the edges with a plastic polish (Brasso will work too).  Notice, the edges will still appear milky at this point, this is normal for EndLighten acrylic.

You do not need to polish the other two layers, and I actually recommend that they remain rough that way no IR light can shine into them.  The rear projection layer should be placed directly on top of the EndLighten layer.  This layer will act as a diffuser for the projector, that way the projector does not simply shine directly though the surface at the user.  The abrasion resistant layer should be placed on top of the rear projection layer.  This layer is important since it protects the other more expensive layers and is the layer that the actual user will be touching.

STEP 6: Attach IR Strip

The IR strip lighting should be wrapped around the edges of the EndLighten acrylic layer.  You should cut the IR strip once you have gone around, since it tends to get pretty hot when left in a coil at the end.  If you are just building a test setup, tape will work fine to adhere the IR strip to the acrylic layer.  Otherwise, I recommend purchasing/building a u-channel frame and adhering the IR strip to the frame.  That way your acrylic layers will be permanently aligned and in one simple unit that you can drop into your enclosure.

STEP 7: Alignment, Alignment, Alignment

The surface at this point is almost complete.  All that remains to do is to setup the projector and camera and align them so that they cover the same area and are not distorted.

First place the projector under the surface, with a slight angle of elevation so it is not projecting onto anything directly in-front / below it.  Next setup the front surface mirror so that it reflects the projector onto the acrylic layers above.  A front surface mirror is a necessity since an ordinary mirror will distort the image reducing the clarity of the image projected onto acrylic surface.  At this point the image will probably look more like a trapezoid, adjust the keystone  on the projector until a perfect rectangle is achieved.  You also may need to move the projector forwards or backwards, or the acrylic layers up and down in order to achieve the image size you desire.

Lastly, position the camera so it faces directly upwards towards the acrylic layers.  The camera should be placed directly in front of the projector under the center of the surface so it can pick up the entire desired area.

STEP 8: Calibration and Testing

 Now that you have everything aligned its time to test out the setup.  Start by connecting the projector and PS3 Eye camera to the computer and connecting everything to power.  Once the computer has started up, install the CL-Eye Platform Driver and CCV.

CCV is an open-source program that processes the IR image picked up by the camera and turns them into blobs (TUIO events).  These events can then be used by programs to allow the user to interact.  Once CCV is up and running, click on the use camera option to begin receiving input from your PS3 Eye camera.  Now experiment by touching the surface and watching to see what blobs are picked up in the Tracked view.  Play with the "Image Threshold" and other settings until the desired blobs are achieved.

Once that is complete, complete the "Calibration" mode in order to map pixels on the projected image to points picked up by the camera.

A complete guide to calibrating and testing your surface can be found here.

STEP 9: Package It

 At this point you should now have a fully working Multi-Touch surface.  If you are simply building a development environment then you can probably stop here.  But if you are planning on deploying the surface to a production environment, then you will need to have some sort of enclosure.  The enclosure can be designed however you want, but you must ensure that you have the proper angles and space to project the image and to pick up the entire surface on the camera.  Additionally, proper ventilation is essential since the projector and computer can get quite hot in an enclosed space.

I will add additional steps with detailed plans of my enclosure once I have finished building it.

Good luck!

Comments

[joshiajinkya90]

Can I use laptop motherboard instead of desktop cpu motherboard

[edwin.franco]

Congrats!. Does the EndLighten acrylic with DSI the recognition of fiducial objects? Is your table able to have this feature?. Regards!

[RongieC]

Quick question I have a mac I can do this screen and use it in my mac?

[The Life Hacker]

Same here.

[LuisD113]

[e-mail]

[TomL81]

I made something simular a few years back using a cardboard box, a glass picture frame, masking tape and a webcam.

[murtaza.hasan.7]

i had built a multi touch table brother but i could nt adjust it touch in a proper way .. it is touching symentaniously many areas at a time .. i through out the projector rays with the help of a mirrior .. is it wrong ?

[DCFIGHTER]

nice build brother
but one following question to this
u said the price and cost for the following Multi-Touch surface would be 2500, can you please suggest the lowest price that i could spend on such a build because firstly i am 17 and secondly i am trying to save as much as i can
Thanks for the detailed instructions also

[tinker234]

thanks i loved this article but i had some qusteins how can i set up the software i tried before but i never could get it to work where can i buy acrilec cheaply and with the eprojecter whjat do you do

[frankencois]

so with the projector, if i were to be using a short throw projector, i would not need a mirror to reflect the image onto the screen, am i right?

[jugglinggoalie]

Can you tell me approximately how far the camera needs to be away from the top?

[servant74]

Depends on your camera and lens.

I suggest just set up the camera, point it to a wall with an area marked that is the size of the screen desired marked on the wall. Once you can see all edges of the desired screen clearly, you are the 'right' distance away.

Now measure the distance from the camera to the screen. This is the total distance the camera needs to be.

Use the same way to set up the distance for the projector. It needs to project a clear image over all the usable parts of the display screen. Again measure the total distance from the lens to the screen.

Now you have the right distance for both the projector and camera away from the screen (or top).

[lafields86]

I too would like to know the suggested distance the motion camera needs to be from the screen as well. Also is there an easy way to add a second camera or are there any advantages in using the Microsoft Kinect camera?

[lrubin28]

Cheapest place for Front Surface Mirror is BostonCraftWorks.com. They charge $39 for a real glass FSM sheet, 16 inches x 20 inches. They sell it for high quality kaleidoscopes although people also use it for things like cameras, copy machines etc. They also sell smaller sheets of acrylic front surface mirror - its about 12" x 12" and it only costs about ten bucks - the only issue for optics is that the acrylic is bendable, so it needs something firm underneath, or you should probably just buy the glass version. The glass version also has a very thin layer of a protectant on it so that it can handle very light cleaning. Normally you want to keep FSM in a sealed environment so it doesn't pick up dust.

[lfields1]

I have been looking at the idea of trying to use a mirror to reduce the distance the camera needs to be from the screen. Have you tried this? I can't seem to find anyone that has. I would just do it myself but I am waiting on a computer that can handle multi touch. I have all the components except that.

[maewert]

I'm considering building this and I happen to have a broken rear projection TV that has a good front-surface mirror. I've seen other designs on the net that used mirrors to get a larger distance but have not done so myself. If you can get a front-surface mirror, then it should work.

[jck112]

I have not tried using a mirror, but your best bet is to use a super wide angle lens - such as one of these: http://peauproductions.com/store/index.php?main_page=index&cPath=26_4

[sifihog]

Is it possible to wire separate LEDs instead of buying a LED reel?

[aismail6]

some one please respond? how manny layers of arcylic do i have to use ?

[fredbana]

three see step 5