Since Valentine's day is approaching, I thought this would be an appropriate project to write about :-)

The project was intended to 'do something' with the empty windows downstairs in the building were I have my studio. The end result is a projection on little pieces of transparent paper ('paper pixels') that shows a heart shape. The heart shape turns into words when someone is standing in front of the window (detected by using a Kinect). It then shows things that people love: a program scrapes Twitter for messages of the form 'I love ...'. From those messages the terms on the dots (cats, weekends, Bermuda, etc.) are parsed and fit into in a heart shape.

Step 1: The Projection Frame

I wanted the projection screen to be very transparent and light, like it floats in the air. For that I constructed a wooden frame and strung transparent nylon thread between both sides horizontally. Hanging weights at both sides of the thread keeps prevents them from arcing down.

I cut pieces of translucent paper (in Dutch: 'kalkpapier', I could not find the proper translation in English), and folded them around the nylon threads, resulting in a grid of 'pixels' hanging in the air.

Step 2: Setting Up the Projector

I wanted the projection to be as close to the window as possible. Therefore the paper pixels are lit from behind, so no room is required in front of the projection frame. I put a projector on the ground and used a mirror that reflects the projection towards the projection screen.

The projector is a BenQ of about $300. For my installation no high resolution nor high luminosity was needed (unless you want the projection to be visible in bright day light).

As a mirror I used a Plexiglass acrylic mirror of 40cmx40cm. The advantage of this material compared to glass its weight: it is much lighter which comes in handy when the installation needs to be transported somewhere else. In addition, the small construction that keeps the mirror in the correct angle could be kept light and simple.

Step 3: The Projection

Projecting an image via the mirror on the pieces of paper will result in deformation. This is caused by the mirror itself, by the fact that the pieces of paper are not in perfect alignment, and by the fact that some of the pieces of paper curl a bit. Therefore the projected image needs to be deformed in such a way that the resulting projection equals the original image.

I wrote a simple tool that projects rectangles on the paper pixels. The software allows the corners of each rectangle to be aligned with the corners of the pieces of papers (this is done by hand, and is quite tedious: for every pixel each of the four corners needs to be aligned). The video on the introduction page shows this process. Once the deformation for each square is known, the characters can be deformed the same way, so the characters in the projection look like the ones in the original image. The images above show the deformed projected image. When projected, the squares are rectangular again, and the characters look normal.

Step 4: Detecting Motion

To detect that a person is in walking by the projection, I used the Kinect, a motion-sensing device. The Kinect is able to produce a depth-image which comes in handy in this project since I wanted the projection to react on people walking by or standing in front of the window, but not on the cars and bikes passing by on the street, a little further away. With a little bit of code (in Processing) a read the frame coming from the Kinect, and filter out movement in a range from 0.5 - 2 meter distance from the projection.

When something in front of the projection is detected, the projection changes from a heart-shape consisting of filled squares ('pixels'), to a heart shape filled with characters.

Step 5: Find Things That People Love

The words that are used to fill the heart shape are coming from Twitter. Using the Twitter Streaming API and the Java Twitter4J library I wrote a program that reads tweets from the public Twitter timeline containing the words 'I love'. This is done every minute or so. From those tweets the word right after 'I love' is taken, except if it is a 'stop word' (like 'a', 'an', 'the', etc.). To fill the heart shape I need words varying in length from 2 to 8 characters. When new words become available, they are replaced in the heart shape, so the projection constantly changes and rarely shows the same words.

Step 6: The End Result

Our building is right next to a museum, so a lot of people pass it. It was great fun and exciting to see how people reacted on the installation. Next time I need a window without a frame that blocks the projection in the middle :-)