Picture of Interactive Multitouch Display
Between the Apple iPhone and Microsoft's interactive table, multi-touch displays are all the rage. This instructable will show you how to turn your lcd projector into an interactive multi-touch display table using a few cheap components readily available from the hardware store.

Here is a video of my display in action:

Step 1: Theory of operation

This multitouch display screen design is based on the description in Jeff Han's paper,

Han, J. Y. 2005. Low-Cost Multi-Touch Sensing through Frustrated Total Internal Reflection. In Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology

The figure below comes from his web site.

An acrylic panel is edge lit with infrared leds. When your finger comes in contact with the acrylic, it scatters infrared light out the back where it is visible via infrared camera. As long as nothing is touching the acrylic, very little of the light escapes, instead just reflecting around inside. Image processing takes care of detecting tips of fingers and relaying their location to application software. Since the camera "reads" the whole display in parallel, it is easy to detect multiple fingertips at once, even those belonging to multiple users. All this sensing goes on in the infrared spectrum, leaving us free to utilize the visible spectrum to display interactive software.

Since most hobbyists can't afford multiple projectors (i don't even own one, just borrowed it from dr.eel), my design uses a ceiling mount that swivels so that the projector can be used either in standard mode (say for watching movies) or can be aimed downwards, bouncing off a reflector and onto the multitouch display screen.

The screen itself can be constructed from hardware store materials and hand tools. Excluding the projector and modified webcam (commodity items these days), the only thing complicated is the software. Halfway through this project, I was happy to discover that there is thriving DIY community which has already undertaken the task of writing the image processing code and several cool open source demos which can be found here:


« Previous41-80 of 330Next »
turkey tek (author)  JamCat5 years ago
wow jamcat, many thanks for the useful advice and insights!
yosaulox4 years ago
man what software you used¡?
zeeebus5 years ago
Awesome Job!  This may be an idiotic question, but....I want to use my table mostly as a whiteboard for writing and brainstorming--hopefully converting into typed text too, if possible.  Would the side palm of my hand resting on the screen while I write interfere with the writing recognition?
Hossein zeeebus4 years ago
Dear Zeeebus
i know what i wanted to ask is out of line but believe me i wouldn't ask if this was not my last chance .
I really really need to have the PDF version of this multi-touch screen here coz i took it as my major project and i have to build it in a week time and to be honest i don't have the money to be a pro member and since i'm a student i don't have any visa card as well.
I was wondering to ask if possible can you downloaded it for me and send it to my mail i would mean a lot to me .
THANK you so much
your hand resting on the side of the screed would create IR light that the wii remote would see, and yes this would mess up writing recognition software, but you could put a clause in the software that ignores any blobs bigger than a certain size - for a palm is much bigger than a finger, or any blobs that are stationary (to a certain degree) or something like that, which would fix things up and make it possible.
let me know how you go! (or are going)
i think so
Neoferatus4 years ago
Interesting!! Thanks alot!!
UniBot5 years ago
Well if you use the 3v output from your standard ATX power supply, you can hook up in parallel N modules of 2 LED's in series (MODULES in parallel, LEDS in series), being theese in a total power consumption of 2.8v... 3v will do, since (I think) that's within the LED's electric tolerance. Then you hook up (as i said) N modules, with this I mean until you complete the average total amperage of a normal ATX power supply (commonly 19amps), with that you can plug as many LED's as you want (as long as theyre within your power supply capabilities).
Without any resistors? Can you draw here a schematic? Thx
Im sorry for the delay, but heres a quick schematic. The thick lines are the 3v power output from the ATX power supply, (positive and negative respectively). the rectangles are (supposedly) LEDs hooked up in series of two LEDs drawing a total voltage of 3 volts (YAY! thats exactly the same voltage than the ATX! xD), thats what we'll call a "module". If we assume that each LED draws 50 Ma (milliamperes), then each "module" will draw that amount from the power supply, if you hook up more modules to the power output of the ATX power suply in PARALLEL you can keep up until you fill the 19,000 Ma (19Amps [depending on your power supply capacity]) of total capacity... that's a HELL LOTTA LEDS MAN!

PS: im sorry for the lame paint-made schematic.
PPS:I know, I made a short story long. Who cares? xD
Ok ok.... thx
Hypernoob6 years ago
did you drill through the acrylic? or are there gaps provided by the spacer blocks so you are actually drilling through an empty space.
turkey tek (author)  Hypernoob6 years ago
i didn't drill thru the acrylic. it is just floated inside the frame.
Im wondering how this could be made into a large desktop for working with cad/photoshop and other programs.
Also, I noticed you were having just a tad difficulty playing with those picture files. Is that a problem with the web cam used?
Please contact me on this. Id like to know more. about making my own.
part of it could be the webcam used. With some webcams there is actually a IR diffuser inside of it over the lens and they can sometimes mess with the IR sensing. Another problem could be the paper that Turkey Tek uses. Normal paper may be too thick and not let quite enough light through. The last problem could be the number of LED's that he has installed. More would obviously be better but require a hell of a lot more soldering and what he already did probly sucked! lol
a nice substitute for the paper is anything like drafting vellum or trace paper or the like...
UniBot Rjessick4 years ago
above this comment I posted a solution on the LED amount problem by eliminating the need of resistors, as for the diffuser issue, the besto soution I know is to put vegetal paper (white / translucent) under the acryllic, it provides the best visibility for the projector's image and touch capability (that last part, im not very sure):
-good visibility
-easy installation, it sticks to acryllic thanks to static electricity

-limited size availability. maximum size I know of: mercury format (110 x 77 cm)
-critical installation. if theres an air bubble between the acryllic and the paper, the IR refraction of the finger will turn too blurred and inaccurate.
-environment. the vegetal paper is EXTREMELY sensitive to changes in temerature and moisture, too much moisture... and the paper bends on itself, causing air bubbles where we dont want them.
jakethink5 years ago
Who is this song by? Is that a Nina Simone sample? Sounds like her
No its not nina simone its... PEACE ORCHESTRA! :D the track's name is Who am I?... The first time i heard it was on The Animatrix. then i got the OST and i found out about the song.
DeadlyDad5 years ago
Hi.  I had an idea that should work quite well with multitouch: control elements that can be placed anywhere on the screen.  The basic concept is to use QR code 'rubber stamps' on the bottoms of the controls to differentiate between them.  A digital button would have a plunger beside the code, an analog button would have a rubber ball plunger (i.e. the harder you press, the more the ball is squished, the larger the area seen by the camera), a slider or dial would have a brush.  You could even have a joystick that is connected to a spring-loaded plunger.  What makes using QR codes so ideal is that you can have over 4K alphanumeric or 2K binary data, which is plenty to describe a particular control's ID, function, and the position(s) of that control's input(s), or even hold a custom routine.

Does that sound feasible, or am I totally off my nut?
 That actually sounds rather feasible - you'd probably have to write your own recognition/filtering for it, but that wouldn't be relatively bad. Heck, with a 4x4 grid you can have what, 2^16 different combinations?

For the slider style, you could simply have 3 points - two statically spaced to orient the slider, and one for the slide position. Of course you could also just make a slide-type image that pops up and use your finger to control it.
Exactly, though the QR code block is the static position needed. Its contents can include the descriptions of areas in which to expect other points and how to interpret them.
Just had an idea on how to increase the resolution of the camera: Instead of having the camera look directly at the display, have it look at a mirror that is attached to a motor, spinning, and sitting almost flat, such that each frame will be at a slightly different angle, but 4 guide points in the edges of the display are included in each frame of video. On the computer side of things, a 'Super-Resolution' routine (check out the references at the bottom of the page) then picks up the frames, does it's magic, and Voila! Drastically increased resolution. (Yes, I realize that it isn't a drag-n-drop solution, but there are lots of references and papers on it, so it is definitely doable.)
 With all the diffusers etc, is it still possible to make out objects placed on the screen? I'd really like to build an interactive screen that supports both detecting finger touches and trackmate tags (trackmate.media.mit.edu), but it seems like this design will have too much diffusion between the screen and the camera for that.
I have bought a usb picoprojector. I intend to build up my multi-touch display using this cheaper projector (~US$250) instead of my last failed projector.  

It can project upto 40" at 1 meter distance away. Thus, I believe I don't need to have a mirror as reflector

However, I wonder whether it is workable while its resolution is just only VGA resolution.

Any comment and suggestion ?
sorry for the late reply, you've probably already built your screen :P
as far as I understand, the resolution of the projector wouldn't have any effect. The program would work out all the things in the resolution of the wii remote infra-red camera (1024 x 768) and it will simply be projected in whatever resolution your projector can do. the blobs of light made by your fingers don't have a resolution.
freerunnin14 years ago
thats it then im going to buy a projector and make 1 of these, 3d scanner, multi touch pc.. sounds gd! :P 3d scanner: http://www.david-laserscanner.com/?section=Downloads&file=david_laserscanner_setup tutorial: http://www.dailymotion.com/video/xc6wsd_how-to-make-a-3d-scanner_school :D
Can it runs on linux?
Everything runs on linux ^_^ haha, but seriously, im sure it can..
Thx, soon or later i'll try.
quitit5 years ago
Has anyone used this kind of display for doing work other than just visual effects. Maybe like using the set up as a oversized touch pad for working with Autocad and things like that?
danny98935 years ago
Excuse me but i have a question about the diffuser...What does it actuaiiy do?
The acrylic is clear, without the diffuser, the image from the projector will pass right through, the diffuser is acting like a screen.
 yargggggg.... this shit be awesome 
 I hope you realize you've built my childhood dream...
imagi5 years ago
Why do you have to reflect the projector image via mirror?
Arbitror imagi5 years ago
Because you want rear projection, not back...
imagi5 years ago
Does it work if I use IR LED strips?
Jetpac8 years ago
This my freind is incredibly cool. would it be possible instead of a projectore to do wthis with like a nomal LCD with the screen on to of it (for those who have no projector?
I'm going to guess no, due to the nature of LCDs.
« Previous41-80 of 330Next »