Intro: How to Build a Multi-Touch Surface
Multi-touch gestures are the latest and greatest way to interact with your computer. However, commercial systems are not widely available and are expensive. Fortunately, you can do it yourself better and for less cost.
Principle components (Click links for product info):
$35 PS3 Eye Camera (Recommended but any decent webcam will work)
$20 IR bandpass filter
$200 5-meter dimmable IR LED Strip
$1000 Any recent/decent computer (PC/MAC)
$1000 Any projector >2000 lumens (NEC NP410W)
$40 Front surface mirror
$80 Abrasion resistant acrylic sheet (23" x 35" x 0.236")
$130 Rear projection acrylic sheet (23" x 35" x 0.118")
$145 EndLighten acrylic sheet (23" x 35" x 0.394")
Software (Click links for downloads):
$free CCV (Used to turn images into blobs)
$free CL-Eye Platform Driver (For PS3 Eye camera)
$free Multi-Touch Vista (Optional Multi-Touch TUIO driver for Windows integration)
$? Windows 7 (Recommended)
Written by Justin Klaassen, Duke University B.S.E. 2012
Sponsored by the Duke Library
Step 1: What Is a Multi-Touch Surface?
A Multi-Touch surface is essentially a giant touch screen that can process multiple touches at once. And in our case, a virtually unlimited amount of touches. By allowing multiple touches, gestures can be performed on surface. These gestures are typically simple things like pinching and pulling on an image to zoom in and out. By supporting gestures like these interfacing with a computer becomes much more intuitive and fun. In a public place, a Multi-Touch surface can be a real attention grabber.
Step 2: How Does It Work?
There are a variety of different technologies that can be used to create a Multi-Touch surface. All or most surfaces rely on optical based technologies that work using infrared light (IR). The technology used in this guide is direct surface illumination (DSI) and is explained by the comic below. The basis for using DSI over other similar technologies is that it allows for a protective abrasion resistant layer to be on top, protecting the delicate under layers from the users touches. The order of acrylic layers on the surface is as follows:
1) Touch surface (Abrasion resistant acrylic)
2) Diffuser/Projection layer (Rear projection acrylic)
3) EndLighten acrylic
Step 3: Build a Computer
You do not need to build a computer specifically for this project. In fact, any recent computer with a decent processor and graphics card should work well. However, if you plan on putting the surface in an enclosure when you are done, then ensure the computer has adequate cooling and minimal thermal dissipation. Another important factor is the size of the computer, since you want to make sure the computer does not interfere with the camera or projection systems.
Example components used in our rig:
Intel Core i5-660 Clarkdale 3.33 GHz
ASUS P7H55D-M EVO Micro-ATX Motherboard
A-DATA 4GB (2 x 2GB) PC3 12800 Memory
XFX HD-567X-YNFC Radeon HD 5670
Step 4: Modify the Camera
In order to pick up infrared light the PS3 Eye camera will need to be opened up and modified. A PS3 Eye camera is used since it is a) cheap and b) supports a high resolution (640x480) at 60 fps. This speed and resolution are perfect for a multi-touch surface since they provide the detail needed to pick up small blobs and the speed to make gestures appear smooth.
In order for the PS3 Eye camera to work, the IR filter needs to be removed and an IR bandpass filter needs to be put in its place. The IR bandpass filter ensures that only the IR light with the utilized wavelength makes it into the system. Therefore it is important that the IR bandpass filter's centering wavelength matches that of the IR LEDs present in the IR strip used for illuminating the surface. A video guide created by Peau Productions for modifying the PS3 Eye camera can be found below.
Step 5: Prepare the Acrylic
The most important acrylic layer is the EndLighten acrylic. This layer is the one responsible for redirecting the IR light from its sides out the top and bottom surfaces of the acrylic. Therefore it is necessary to polish the edges of this layer in order to maximize the amount of IR light that can be injected into it.
If it is possible, I recommend laser cutting the EndLighten layer since that will automatically polish the edge to perfection. If a laser cutter is not an option, then automotive sandpaper will do the trick. Start with 600 grit (lower if the saw cut is rough) and sand each edge until they are perfectly smooth. Finish each of the sides with 1000 grit. Finally, polish the edges with a plastic polish (Brasso will work too). Notice, the edges will still appear milky at this point, this is normal for EndLighten acrylic.
You do not need to polish the other two layers, and I actually recommend that they remain rough that way no IR light can shine into them. The rear projection layer should be placed directly on top of the EndLighten layer. This layer will act as a diffuser for the projector, that way the projector does not simply shine directly though the surface at the user. The abrasion resistant layer should be placed on top of the rear projection layer. This layer is important since it protects the other more expensive layers and is the layer that the actual user will be touching.
Step 6: Attach IR Strip
The IR strip lighting should be wrapped around the edges of the EndLighten acrylic layer. You should cut the IR strip once you have gone around, since it tends to get pretty hot when left in a coil at the end. If you are just building a test setup, tape will work fine to adhere the IR strip to the acrylic layer. Otherwise, I recommend purchasing/building a u-channel frame and adhering the IR strip to the frame. That way your acrylic layers will be permanently aligned and in one simple unit that you can drop into your enclosure.
Step 7: Alignment, Alignment, Alignment
The surface at this point is almost complete. All that remains to do is to setup the projector and camera and align them so that they cover the same area and are not distorted.
First place the projector under the surface, with a slight angle of elevation so it is not projecting onto anything directly in-front / below it. Next setup the front surface mirror so that it reflects the projector onto the acrylic layers above. A front surface mirror is a necessity since an ordinary mirror will distort the image reducing the clarity of the image projected onto acrylic surface. At this point the image will probably look more like a trapezoid, adjust the keystone on the projector until a perfect rectangle is achieved. You also may need to move the projector forwards or backwards, or the acrylic layers up and down in order to achieve the image size you desire.
Lastly, position the camera so it faces directly upwards towards the acrylic layers. The camera should be placed directly in front of the projector under the center of the surface so it can pick up the entire desired area.
Step 8: Calibration and Testing
Now that you have everything aligned its time to test out the setup. Start by connecting the projector and PS3 Eye camera to the computer and connecting everything to power. Once the computer has started up, install the CL-Eye Platform Driver and CCV.
CCV is an open-source program that processes the IR image picked up by the camera and turns them into blobs (TUIO events). These events can then be used by programs to allow the user to interact. Once CCV is up and running, click on the use camera option to begin receiving input from your PS3 Eye camera. Now experiment by touching the surface and watching to see what blobs are picked up in the Tracked view. Play with the "Image Threshold" and other settings until the desired blobs are achieved.
Once that is complete, complete the "Calibration" mode in order to map pixels on the projected image to points picked up by the camera.
A complete guide to calibrating and testing your surface can be found here.
Step 9: Package It
At this point you should now have a fully working Multi-Touch surface. If you are simply building a development environment then you can probably stop here. But if you are planning on deploying the surface to a production environment, then you will need to have some sort of enclosure. The enclosure can be designed however you want, but you must ensure that you have the proper angles and space to project the image and to pick up the entire surface on the camera. Additionally, proper ventilation is essential since the projector and computer can get quite hot in an enclosed space.
I will add additional steps with detailed plans of my enclosure once I have finished building it.