Introduction: The EyeWriter
Well, my friend Tony's eyes write graffiti and they are wide open.
Behold the latest ocular assault weapon from the Graffiti Research Lab, openFrameworks, The Fat Lab and The Ebeling Group: The EyeWriter. It is a low-cost eye-tracking apparatus + custom software that allows graffiti writers and artists with paralysis resulting from Amyotrophic lateral sclerosis to draw using only their eyes.
This instruction set details how to make your own solderless eyetracker for only $50 dollars using a hacked PS3 Eye and a cheap pair of sunglasses.
"Art is a tool of empowerment and social change, and I consider myself blessed to be able to create and and use my work to promote health reform, bring awareness about ALS and help others"
-- Tony Quan, aka Tempt One
For more information on the story behind the EyeWriter, TemptOne and more, check out http://eyewriter.org.
For more info on ALS click here.
For more words directly from Tempt One click here.
Programmers, engineers, ALS patients/family-members, medical professionals and projection artists! If you're interested in becoming part of the EyeWriter development team contact us here.
The EyeWriter Development Team is:
TemptOne
Graffiti Research Lab
openFrameworks
The Ebeling Group
FAT Lab
Zach Lieberman
James Powderly
Evan Roth
Chris Sugrue
Theo Watson
Keith Pasko
LM4K
Eleanor Dunk
Jamie Wilkinson
Greg Leuch
With special thanks to the Not Impossible Foundation and Sven Travis from Parsons Communication, Design + Technology
Step 1: EyeWriter System Requirements
The goal of the hardware component of the EyeWriter project is to make the most simple and inexpensive eye-tracking head-set possible to use with the "EyeWriter" software suite. Obviously, there are numerous ways to make eye-tracking hardware. Many of these designs, especially those produced for academic research projects (Open Eyes), have already been published openly on the internet. There are also commercial products available -- costing in the range of ~$20,000 US or more-- that are specifically designed to enable people with ALS to communicate using their eyes. We are not in the business of re-inventing these systems. This project is an attempt to address a gap in the development of low-end eyetracking systems, ie to make a super-cheap, eye-tracker that could be made by almost anyone, almost anywhere. Our "EyeWriter" system has several specific design limitations that were meant to emphasize low-cost and ease of construction over other aspects of performance, robustness and appearance. The specific parts and tools you use to build your own "EyeWriter" will depend on your ability, location, financial resources and creative je ne sais quoi (surely you can do better than us -- we're American thugs), but do allow the following design specification to help you to seamlessly connect your hardware to the EyeWriter software and to explain the ethos of the project.
Our functional design specifications are as follows:
1. The EyeWriter should be as inexpensive as possible
2. The fabrication and assembly of the system should require only common hand tools
3. Whenever possible components and parts should be available for purchase locally versus online
4. The camera should produce 640 x 480 NTSC video
5. The camera should be sensitive to near-field IR light
6. The camera should not auto-iris (or auto-iris should be disabled in the camera's driver).
7. IR LEDs should be used to illuminate the pupil
Beyond that its up to you... this instruction set details a solderless variation of the EyeWriter that uses a hacked PS3 Eye and a pair of stunnas we bought on Venice Beach and suggests other possible EyeWriter configurations.
Step 2: Tool and Parts List
The following part and materials list details the components and tools we used to make a solderless EyeWriter:
Parts:
1x IR sensitive Camera (without auto-iris)
-- PS3 Eye. $39.95 US (using this camera system removes the need for an additional video capture card)
1x camera-lens mount
-- you can use the lens mount that comes with the PS3, but it is glue together and difficult to separate
-- Lens holder, M12x0.5, 15.5, centered $6.00
this is the cheap one, but it requires some modification to match PS3 through-hole footprint
-- more expensive $20.00
1x cheap glasses
-- really any cheap sunglasses will do... or try these. ~$5.00
aluminum wire
-- 9-gauge wire $7.99
3x alligator clips
-- Radio Shack $7.00
a pack of wire-ties
-- Radio Shack $2.49
2x IR LEDs
-- Radio Shack $1.99
tape
1x 8mm camera lens
-- Fixed IRIS Lens Set for Webcams and Security/CCTV Cameras (6-Lens Pack) $14.91
IR wratten
-- infrared filter gel $26.99
Cheaper DIY version of IR filter include cutting a piece of film out of a floppy disk or using unexposed and developed photographic film
battery holder
-- Radio Shack AAA holder $1.99
Tools:
small screw driver set
-- 7ps set $14.95
scissors
--steal them from a kindergarten
some other optional parts and tools if you want to get fancy:
electrical junction connectors
screws
drill
soldering iron, solder, flux,
shrink tube
tap and tap handle
perforated circuit board
dremal
a video capture card (if not using a PS3 Eye)
-- we have successfully used the Pinnacle Dazzle DVD recorder (it does require a serial code on the mac)
Step 3: Get Some Stunnas!
We got this model on Venice Beach for $4 US. You can get yours at the truck stop, thrift store or Louis Vuitton.
Other models include:
black hipster frames
Kanye Shades
Step 4: Make the Camera Arm
The camera arm needs to hold the camera rigidly in front of one eye, but also be flexible, positionable and easy to manufacture.
The best material we have found in terms of rigidity, flexibility, machinability, cost and weight is 9-gauge aluminum wire. This type of wire is often used as a support structure inside clay and plaster sculptures and can be found at art supply, hardware, craft and DIY stores. It often has a plastic coating over the aluminum, which is helpful in our case in order to electrically isolate the camera arm from the camera circuit board. If the wire you use is not insulated, you can wrap it with a few strips of electrical tape.
To machine aluminum wire you can use a pair of tin snips or simply bend the wire back and forth repeatedly until it breaks.
1) Break off a long piece of wire 20 to 30 cm in length
Step 5: Attach the Camera Arm to the Glasses
The easiest way to attach the camera arm to the glasses frame is to simply use wire ties to secure the wire along the arm of the glasses. We use around 6-8 small wire-ties.
A more elaborate method involves using aluminum electrical connectors. These can be found at hardware and DIY stores. This method requires a drill, a tap, a tap handle and takes about 20 minutes.
1) drill two appropriately-sized holes for tapping into the aluminum connector. The size of the hole will depend on the size of the screw you intend to use. We used a 4-40 screw. To create a hole ready for a 4-40 tap you would use a #43 drill bit (3/32nd). You can use the tap and die chart linked below as a reference is you intend to use a different-sized screw.
http://www.korit.com/tapndrill.htm
2) use the holes in the aluminum connector as a reference to mark where the holes need to be drilled in your glasses frame and then drill two holes that would accommodate a 4-40 screw (a #38 drill bit).
3) using the tap handle and a 4-40 tap, you should tap the aluminum connector. Remember to move slowly, use machine oil (or olive oil if you dont have professional machine lubricants) and clean the hole before trying to insert the screw
4) you can now assemble the pieces. Use washers, lock washers and lock-tite in order to securely attach the connector to the frame. The connector we used has a flat-head set screw that allows us to screw down the aluminum camera arm and secure it tightly to the glasses.
Step 6: Hack the PS3 Eye
How to take apart the PS3 Eye and remove the IR blocking filter:
How to instal a visible light filter using a floppy disk:
These videos document the process of hacking the PS3 Eye pretty thoroughly. But, In our case we need to use a lens with a shorter focal length than the one provided with the PS3, so some extra hacking is in order. To recap and expand on how to mod the PS3 Eye for use with the EyeWriter software:
1. unscrew the four screws on the back of the PS3 eye
2. Crack open the case using a small flat head screw driver
3. Unscrew the screws that mount the camera circuit board to the plastic housing
4. Unscrew the camera lens mount
5. Either a) throw away the PS3 Eye lens mount and lens and use one of the lens mounts linked to in the parts list (and the 8mm lens from our lens pack) or b) if you want to repurpose the PS3 lens mount you need to dig the IR light filter (as shown in the video).
6. If you want to use the original PS3-native lens mount, you will need to separate the PS3-native lens from the mount, which is attached with some industrial glue. To do this you need to scratch away the glue around the outside lip of the mount. This is hard to do and requires some patience and some luck. You will need to scratch and try to turn the lens to unscrew it. Keep repeating this process until the lens separates and can be unscrewed. If you destroy the lens (which happened to us about half the time) you will be forced to use one of the lens mounts linked to in the parts list.
7. Cut your IR wratten down to fit inside the lens mount
8. If you have successfully separated the PS3 lens from the PS3 lens mount then just screw the PS3 lens mount back onto the camera circuit board. If you have not succeeded in separating the lens from the mount, then screw the new lens mount on the camera circuit board.
9. screw in the 8mm lens into the lens mount
Watch the video and look at the included photos for more tricks and information on how to successfully hack the PS3 Eye.
Step 7: Attach the Camera
To attach your newly modified PS3 Eye camera to the wire armature you will need wire ties and some type of small, insulated substrate that will provide more rigidity to the camera/armature assembly. For our prototype we used a small (3 in x 1 in) piece of hard rubber. You can also use a piece of wood, half of a pop cycle stick, or any other sturdy, insulated material.
1. put the rigid substrate in between the camera and the wire armature. The camera should be pointed toward the eye of the glasses.
2. use 4 wire ties to firmly attach all three pieces together.
You can put a small length of double sided tape between the rigid substrate and the wire armature to make putting the three pieces together and to ensure a more secure assembly. The camera should still be adjustable in terms of pitch and may require occasional adjustments (or even re-assembly) between uses.
Step 8: Lite It Up!
When you illuminate the eye with IR light and observe it through an IR sensitive camera with a visible light filter, the iris of the eye turns completely white and the pupil stands out as a high-contrast black dot. This makes tracking the eye much easier. In order to provide some IR illumination, we have made a quick and dirty IR LED circuit using alligator clips, IR LEDs and a 2x AAA battery holder.
The circuit is a simple 3 volt series circuit with two IR LEDs and a power supply (See the napkin circuit drawing below for more details). Connect an alligator clip, preferably a red one, to the power lead from the battery holder. Connect the other end of the alligator clip to the positive leg of one of the IR LEDs. Connect another alligator clip, preferably white or yellow, to the negative leg of the same IR LED and the positive leg of the second IR LED. Finally, connect an alligator clip, preferably black, to the negative leg of the second IR LED. The other end of the black alligator clip should be connected to the negative lead from the battery holder.
You can test to see if the IR LEDs are working by looking at them using most typical point and shoot cameras. If they are sensitive to IR light, you should see a soft glow coming from both LEDs.
Wrap up the excess cable, wire tie the alligator clips to the arm of the glasses and the camera armature. You can use wire ties to attach the alligator clips to the front of the camera. Bend the IR LEDs so they are pointing in the same direction as the camera, bent in toward the eye. Make sure the LED legs are not touching each other or any part of the camera circuit board. You can use electrical tape to help keep all metal components electrically isolated from one another. You will likely have to adjust the IR LEDs once you are looking at the eye in the EyeWriter software in order to get a strong illumination that removes shadows created by the eyelid, lashes and camera frame.
Step 9: Connect to the EyeWriter Software
The software for both parts has been developed using openframeworks, a cross platform c++ library for creative development. In order to compile and develop the EyeWriter source code, you will need to download openframeworks (pre release v0.06). Documentation, setup guides and more information can be found at http://openframeworks.cc.
For more info on the EyeWriter software click here.
PS3 Eye drivers & QT components
In order to use the PS3 eye you will need to download a driver/component and install it.
For a mac you will need to download the quicktime component here and put it in your-hardrive//Library/QuickTime.
To learn more about using it on a PC click here.
Video Capture
Alternatively, if you plan to use another type of NTSC camera, you will beed a video capture card. We have successfully used the Pinnaccle Dazzle USB DVD recorder. To use this device you will need to install a PC driver or use VideoGlide. This software does require a user-license which costs roughly $25.00 dollars
Eye-Tracking Software
The eye-tracking software detects and tracks the position of a pupil from an incoming camera or video image, and uses a calibration sequence to map the tracked eye/pupil coordinates to positions on a computer screen or projection. Note that we use the GSL (gnu scientific library) for calibration, which is GPL, thus the eye tracking source code is GPL.
The pupil tracking relies upon a clear and dark image of the pupil. The diy glasses we designed use near-infrared leds to illuminate the eye and create a dark pupil effect. This makes the pupil much more distinguishable and, thus, easier to track. The camera setting part of the software is designed so the image can be adjusted with brightness and contrast to get an optimal image of the eye.
The calibration part of the software displays a sequence of points on the screen and records the position of the pupil at each point. It is designed so that a person wearing the glasses should focus on each point as it is displayed. When the sequence is finished, the two sets of data are used to interpolate where subsequent eye positions are located in relation to the screen.
The following software tutorial walk-through will help you to better understand the tracking interface.
eyewriter tracking software walkthrough from thesystemis on Vimeo.
Eye-Drawing Software
The eye-drawing software is designed to work with the EyeWriter tracking software as well as commercial eye-trackers such as the MyTobii. It is currently a separate application from the EyeTracker, but we will also post a combined version that shows the two working together.
The tool allows you to draw, manipulate and style a tag using a time-based interface so that triggering buttons or creating points for drawing is achieved by focusing on the position for a given amount of time. Tags and tag data can also be uploaded via FTP and HTTP Post.
The following software tutorial walk-through will help you to better understand the drawing interface.
EyeWriter - Drawing App walkthrough from Theo Watson on Vimeo.
Step 10: EyeWriter Installation
The EyeWriter interface can be used to create drawings on screen, or using a small projector, you can create drawings on the wall in a hospital room. We have also used the EyeWriter software in conjunction with a special version of the Laser Tag software to project EyeTags at large scale in public space. The following steps and links will help you to create an EyeWriter installation catered to your needs.
On-Screen Drawing
In order to create on-screen drawing you will simply need to follow the steps featured in the previous step. This will work with both the mytobii system software as well as the EyeWriter hardware and software suite.
Co-located and Projected Drawing
In order to create drawing using the EyeWriter hardware/software or the EyeWriter software/Mytobii system, you will need a digital projector and a projection screen or surface. You will need to calibrate the EyeWriter user to the projection surface. We have experimented with using a regular bed sheet as a projection screen successfully.
Remote Drawing and Projection
To do remote projection you will need two computers, both connected to the internet, a mobile broadcast system connected to one computer and the Laser Tag OSC receive software (super-beta). For more information on mobile broadcasting and projection bombing check out GRL's projection bombing tutorial on instructables and the MBU diagram below. We have also used Sprint wireless broadband cards to create wireless remote connection between the two computers.
We are no currently supporting the Laser Tag OSC receive software, but you are welcome to download it and hack around with it. In order to project the GML (graffiti markup language) tags you will need to drag the GML data into /bin/data and rename the file tempt.gml. We will soon release the full EyeWriter send/recieve software with instructions. For more GML tags, visit Evan Roth's unreleased beta site http://000000book.com/ and download the more GML tags here. Stay tuned for more from GRL, Fi5e and the EyeWriter team and if you're interested contact the EyeWriter team and become a developer/collaborator.
