Introduction: Virtual Graffiti

I've seen a few virtual graffiti systems on the web but couldn't find any information published on how to make one (though see final links page). I thought it would be great for my graffiti workshops, so I made one myself and have published everything you need to make your own here!



Features

* all open source and hardware,
* cost < £100 excluding projector and computer,
* detects can's nozzle pressure and distance from the screen,
* models paint dripping if you move too slowly!

Notes

* this instructable is quite high level, but please let me know if I've missed something important,
* the computing setup is for Linux. If you get it working on other systems, please post your instructions!

Skills you'll need

* wood working to make the wooden rear projection screen,
* electronic circuits and programming Atmel AVR micro controllers (or arduino),
* be able to install some libraries on your computer to allow processing to talk to the wiimote.


Step 1: How It Works

* The spray can has an infra red LED that shines through the projector screen and is seen by the wiimote's camera.
* The wiimote sends the X and Y co-ordinates of the can to the computer via a bluetooth radio link.
* The computer is running a simple painting program that uses a projector to "paint" the lines as you draw with the can. It also takes care of mapping the wiimote camera to the screen using a 4 point calibration system.
* The spray can also detect its distance from the screen and the nozzle pressure: the further you are away the bigger the dot drawn, the harder you press the nozzle, the more opaque the paint dot becomes.

Step 2: The Components

Here are all the bits you need to get together:

* computer - needs to be about 1.4Ghz, bluetooth and a usb port,
* processing environment,
* virtualGraffiti software, download from the "computer setup" step,
* nintendo wiimote - buy second hand from ebay,
* projector - it will need to be bright if you're planning to use during the day or inside with lights on,
* rear projection screen - make yourself,
* virtual spray can - make yourself,
* virtual spray can receiver - make yourself.


Costing

* arduino for can receiver (built in usb->serial) £21
* radio rx/tx pair £9
* components for building spray can £18 plus optional enclosure £12
* optional enclosure for receiver £8
* nintendo wiimote - buy second hand from ebay £20

Step 3: Rear Projection Screen

The screen needs to be just the right amount of see through-ness! If it is not transparent enough, the image won't be seen and the infra red LED won't be visible to the wiimote's camera. If it's too transparent then the projector will be blinding and the image washed out. (Though see the last page for ways of mitigating this).

I used lycra, which is stretchy so I can stretch it to make it more transparent. At the moment I hold it on with thumb tacks, but I'm graduating to velcro when I get access to a sewing machine.

I made a wooden frame with the help of a workshop and a carpenter (thanks Lou!) I needed it to collapse so I could transport it on my bike. If you're making one for a fixed venue then it will be easier to make. Just make it at a 4:3 aspect ratio, and stiff enough to stay upright.

I've found people tend to push on the screen material a fair bit so it needs to be a bit rugged.

Step 4: Spray Can

This is the most complicated part of the project and took the longest time to get right. The good news is that you don't need all this stuff to make a fun system work. The simplest thing is just to get a circuit with a switch, and infra red LED and a resistor. When you push the switch the LED lights up and is seen and tracked by the wiimote's camera.

This version is more advanced, because it also measures the distance from the screen and the nozzle pressure. Both these things are important when you're actually spray painting. I wanted to make a training system, so it was important to make the system as "real" as possible (within my cost limits).

The circuit is pretty simple. Take a look at the attached circuit diagram to see for yourself. You need basic soldering skills and to be able to put a circuit onto veroboard. Also, you should feel happy with programming microcontrollers.

Building a circuit from scratch vs using an arduino board

option 1: if you want to use an arduino board in spray can. Use arduino as is and halve the baud rate of the radio tx in the spraycan code.

option 2: you want to save cash but you don't have a fuse programmer. Build the board and use a 16MHz external crystal. Halve the baud rate as in option 1.

option 3: you want to save even more cash and you have a fuse programmer. Build the board, but omit the external crystal. Use the fuse programmer to set the atmel to use its internal clock. I believe this DIY parallel programmer will allow you to program fuses. I use the olimex programmer.

Overview of the circuit

The microcontroller measures the output from the sharp 2d120x distance sensor (great info on this sensor here) and the linear potentiometer. It also measures the output of the LED PWM potentiometer. This is used to adjust the LED's light output. The IR LED I'm using is 100mA and peak wavelength is 950nm (ideal for wiimote).

The microcontroller uses PWM to flash the LED very fast. We use an IRF720 power mosfet so that the micro doesn't burn out its output. Also I wanted to add capacity for a brighter LED in the future.

There is a status LED that flashes everytime a data packet is broadcast on the radio. If everything is working well, this light should be flashing at around 15Hz.

Finally, the radio transmitter module is attached to pin 3 (digital pin 1 for arduino) of the microcontroller so that we can send the information we're measuring to the computer.

You also NEED an aerial attached to the receiver board. I used a 12cm long peice of wire. This is half what is recommended on this excellent info page.

Programming the microcontroller

After you've built the circuit, you'll need to upload the program (attached). I use the arduino programming environment/libaries. You can compile this with the arduino IDE, and then program it however you usually do.

My circuit is made simpler by using the internal 8MHz clock of the micro. If you use this you'll need to set the fuse settings to use the internal 8MHz calibrated RC:  1111 0010 = 0xf2

This means you'll need to have a programmer that can write fuses.

./avrdude -C ./avrdude.conf -V -p ATmega168 -P /dev/ttyACM0 -c stk500v2 -U lfuse:w:0xf2:m

If you don't have this kind of programmer (say you just have the arduino board), just use a 16MHz crystal between pins 9 and 10 and it should all work (untested - you may need a capacitor). You'll also need to modify the program code so that the transmitter baud is halved.

Testing

After you've got it all together and the program loaded, you need to adjust the IR LED brightness. I just wanted to maximise light output without toasting the LED so I blew a few up and ended up with about a 120ma draw average.

If you have a multimeter you can adjust this quite easily, otherwise just adjust the potentiometer to be quite high but not all the way! You can also check the analogue inputs on pins 26, 27 and 28 of the PWM adjust potentiometer, the distance sensor and the nozzle potentiometer.

If you have a scope you can check the pulse train coming out of pin 3 into the radio TX module. Check the pwm output of the LED on pin 11.

You can use a mobile phone camera (or most CCD cameras) to see the IR LED turn on when you press the nozzle button.

Step 5: Spray Can Receiver

If you're going the simple spray can route then you don't need this bit.

Otherwise, I just use an arduino board, with the radio receiver plugged into the pin 2. This makes it easy to get the data into a computer via the USB -> serial chip on the arduino board. If I was going to make a custom circuit I'd probably use an FTDI USB -> serial UART evaluation board.

You also NEED an aerial attached to the receiver board. I used a 12cm long peice of wire. This is half what is recommended on this excellent info page.

Load the graffitiCanReader2.pde sketch into the arduino. With the can powered up, you should see the status LEDs on the can and the receiver board flashing quickly. Everytime the can LED flashes, a data packet is sent. Everytime the receiver board LED flashes, a valid data packet is received. If you're not seeing this then there is something up with the radio link. Something to try is connecting the TX of the can to the RX of the receiver with a piece of wire. If this doesn't work then you probably have a mismatch in the baud rate of virtualwire (see the code).

Assuming you've got lots of flashing going on on the receiver board, you should be able to monitor this on your usb serial port. If you monitor the serial port (usually /dev/ttyUSB0) at 57600 you should see data spewing out like

Got: FF 02
Got: FF 03
.
.
.

The first number is pressure, and the second is distance.

Now you can run processing and use this information to make pretty pictures!

Load the attached processing sketch (canRadioReader.pde) . Start the program and check the output of the program. You should be getting a frequency (that tells you how many updates per second the receiver is getting - you definately want this to be at least 10Hz). Also you'll get a distance and nozzle measurement. Test the can by moving the nozzle potentiometer and by moving a peice of card in front of the distance sensor.

If it's all working, then move on to the next step - getting the computer ready to talk to the wiimote!

Step 6: Computer Setup: Processing and the Wiimote

Our main here is getting processing talking to the wiimote.

These instructions are Linux specific, but it should all work on a mac and windows with some research on how to get the wiimote's data into processing.

After installing processing, I found some instructions on the forum, but I still had some problems. Here is what I had to do:
  1. install processing
  2. install bluez libraries: sudo apt-get install bluez-utils libbluetooth-dev
  3. create ./processing/libraries/Loc and ./processing/libraries/wrj4P5
  4. download bluecove-2.1.0.jar and bluecove-gpl-2.1.0.jar and put into ./processing/libraries/wrj4P5/library/
  5. download wiiremoteJ v1.6, and put the .jar into ./processing/libraries/wrj4P5/library/
  6. download wrj4P5.jar (I used alpha-11) and put into ./processing/libraries/wrj4P5/library/
  7. download wrj4P5.zip and unzip into ./processing/libraries/wrj4P5/lll/
  8. download Loc.jar (I used beta-5) and put into ./processing/libraries/Loc/library/
  9. download Loc.zip and unzip into ./processing/libraries/Loc/lll/

Then I used code inspired from Classiclll to get the buttons and sensor bar working. The attached code/sketch just draws a circle where the 1st infra red source is found by the wiimote.

To check your bluetooth, press buttons one and two on the wiimote, then try

$ hcitool scan

at the terminal. You should see the nintendo wiimote detected. If you don't you'll need to look at your bluetooth setup further.

If it's all good, load the wiimote_sensor.pde (attached) program and start it. In the lower status part of the screen you should see:

BlueCove version 2.1.0 on bluez
trying to find a wii

Press buttons 1 and 2 on the wiimote. After it's detected, wave your infra red source (the spray can) around in front of it. You should see a red circle following your movement!

Make sure this works before moving on. If you can't get it to work, search the processing forum.

Step 7: Setting It All Up

Download the virtualGraffiti software below. Extract it into your sketchbook directory and then follow these steps!

* power up spray can, check status LED light is flashing.
* turn on computer, plug in spray can receiver,
* setup screen and projector,
* check that spray can receiver status LED is flashing,
* start processing and load the virtualGraffiti program,
* check that you're getting both RX and TX serial indicator LEDs are flashing on the arduino board,
* press both buttons on wiimote,
* do 4 point calibration when prompted (put spray can over each target in turn, then press the nozzle till the writing goes red).
* have fun!


Step 8: Resources, Links, Thanks, Ideas

Links

Here are the links that were invaluable in making this project work:

RF info: http://narobo.com/articles/rfmodules.html

Arduino: www.arduino.cc

Processing: www.processing.org

Using wii with processing: http://processing.org/discourse/yabb2/YaBB.pl?num=1186928645/15

Linux: www.ubuntu.org

Wiimote: http://www.wiili.org/index.php/Wiimote, http://wiki.wiimoteproject.com/IR_Sensor#Wavelengths

4 point calibration: http://www.zaunert.de/jochenz/wii/

Thanks!

Without a lot of people publishing their work, this project would have been a lot harder and more expensive.

Massive thanks to all open source crew, people who hacked the wiimote, Classiclll for making wiimote easy to use with processing, Jochen Zaunert for code to do calibration, processing crew, arduino crew, Lou for carpentry help, and all those who explore, make and then publish their findings online!

Other people's systems

* I only just found http://friispray.co.uk/, with open source software and a howto
* this system allows use of stencils: cool!  http://www.wiispray.com/, no code or howto
* yrwall's virtual graffiti system, no code or howto.


Ideas for exploration

* use 2 wiimotes to do 3D volume tracking and do away with distance sensor in the can: http://www.cl.cam.ac.uk/~sjeh3/wii/. This would be good because the distance sensor is currently the weakest part of the system. It would also mean we could use a proper rear projection screen for more vivid images.

* use a wiimote in the can to detect angle of spray can. This would add realism to the spray paint model.


Comments

author
alanblumen made it! (author)2014-09-16

Hello, great tutorial

I'm trying to use processing in w8 to read the wiimote data, so I follow that steps and it's not working, I have the message java.lang.NoClassDefFoundError: wiiremotej/event/WiiDeviceDiscoveryListener

Do you kno how to fix it?

Thanks for your time.

author
Raphango made it! (author)2014-08-23

Great man!! =D

Congratulations!

author
mtwister made it! (author)2012-06-15

Hi Matthew,

Thanks for the detailed instructions. I did everything explained above to connect Processing and the Wiimote, but I get the following error:

The type lll.Loc.Loc cannot be resolved. It is indirectly referenced from required .class files

Do you know how to fix this error?

Thanks a lot for your help.

mtwister

author
SeeDoubleYouNL made it! (author)2012-01-18

Hey Matthew, thanks for you work. I tried your steps but when I run wiimote_sensor.pde processing gives me the following error:

Native Library bluecove not available
!! There may be no USB dongle/device. !!
java.lang.IllegalStateException: Bluetooth failed to initialize. There is probably a problem with your local Bluetooth stack or API.
at wiiremotej.WiiRemoteJ.(WiiRemoteJ.java:74)
at lll.wrj4P5.Wrj4P5.connect(Wrj4P5.java:301)
at lll.wrj4P5.Wrj4P5.connect(Wrj4P5.java:290)
at lll.wrj4P5.Wrj4P5.connect(Wrj4P5.java:284)
at lll.wrj4P5.Wrj4P5.connect(Wrj4P5.java:278)
at wiimotetest.setup(wiimotetest.java:30)
at processing.core.PApplet.handleDraw(Unknown Source)
at processing.core.PApplet.run(Unknown Source)
at java.lang.Thread.run(Thread.java:662)
Caused by: javax.bluetooth.BluetoothStateException: BlueCove library bluecove not available
at com.intel.bluetooth.BlueCoveImpl.loadNativeLibraries(BlueCoveImpl.java:381)
at com.intel.bluetooth.BlueCoveImpl.detectStack(BlueCoveImpl.java:429)
at com.intel.bluetooth.BlueCoveImpl.access$500(BlueCoveImpl.java:65)
at com.intel.bluetooth.BlueCoveImpl$1.run(BlueCoveImpl.java:1020)
at java.security.AccessController.doPrivileged(Native Method)
at com.intel.bluetooth.BlueCoveImpl.detectStackPrivileged(BlueCoveImpl.java:1018)
at com.intel.bluetooth.BlueCoveImpl.getBluetoothStack(BlueCoveImpl.java:1011)
at javax.bluetooth.LocalDevice.getLocalDeviceInstance(LocalDevice.java:75)
at javax.bluetooth.LocalDevice.getLocalDevice(LocalDevice.java:95)
at wiiremotej.WiiRemoteJ.(WiiRemoteJ.java:67)
... 8 more

When I run $ hcitool scan it dows see the wiimote so the dongle is probably not the problem. Any suggestions?

author
matthewvenn made it! (author)matthewvenn2012-01-18

I'm sorry it's been such a long time since I did any work on this project, quite a lot has probably changed.

I'd google the phrase:


Caused by: javax.bluetooth.BluetoothStateException: BlueCove library bluecove not available

And try some of the suggestions.

Sorry I can't help more,
Matt

author
ulfilas made it! (author)2010-04-14

 Having trouble with this :( should this be run in processing? If so - do I need to rename this as another file name? I did try - but I get errors. Was it an earlier version of processing than the one currently available.. do I use the withoiut java one?

author
sanspaper made it! (author)sanspaper2011-02-14

rename it to a .pde file.

author
matthewvenn made it! (author)matthewvenn2010-04-14

Did you manage to get everything running in step 6? If so then this should be straight forward.

I'm using processing 1.0.9.

What errors are you getting?

author
ulfilas made it! (author)ulfilas2010-04-14

 okay so I managed to work out ow to extract the file by changing its name from a tmp file to a tar.gz and extracting it using 7-zip !  I now have the file open in processing - I'm getting a thing saying that "package III does not exist. You might be missing a library"

author
matthewvenn made it! (author)matthewvenn2010-04-15

 good work, sorry I should have provided a .zip.

You need to install all the libraries detailed in the previous step. Have you followed those steps exactly? Can you see the lll directories that need to be created?

Let me know if that helps,

Matthew

author
sc_r made it! (author)2010-01-12

Hey! This is cool, just got all of the bits together but struggling to download the .pde file - it just links to a .tmp file.

What gives? Could you upload it again please? or mail the code over to stuchilds [at] gmail.com

Nice one, good work Matthew :)

author
matthewvenn made it! (author)matthewvenn2010-01-13

download it and rename it to spraycan.pde
I don't know why it changes the name. I tried uploading it again but same result.

author
sc_r made it! (author)sc_r2010-01-13

Thanks for this - I downloaded the .tmp file, renamed to spraycan.pde and looks like arduino code. Will have a play later.

For others: this pinout of the atmega/arduino will help:

http://arduino.cc/en/Hacking/PinMapping168

Next thing I'm struggling with is the external crystal osc:

I don't have the facilities to reset the fuse / timing on the atmega chip so thought would try to use an external 16MHz crystal.
[this one: http://uk.farnell.com/jsp/search/productdetail.jsp?CMP=i-ddd7-00001003&sku=9712526 ]

Now, as I am trying to build this circuit around an actual Arduino board I can't see a way of connecting the external osc. as pins 9/10 of the atmega chip do not have an external connection, they are linked to the onboard crystal.

Will keep updating progress.. Trying to use a pressure sensitive resistor instead of a linear pot. for the cap assembly instead in the hope that it will be a little more durable [plus I have some lying around to play with]

Lovely.
stuart

author
matthewvenn made it! (author)matthewvenn2010-01-13

Stuart, I've updated step 4 with info about crystals etc. See the new section Building a circuit from scratch vs using an arduino board

Basically, you only need to use external crystal if you're making a board from scratch and you don't have a fuse programmer.

author
DIIILIIII made it! (author)2010-01-09

Hey man this is awesome, we made a simpler version in 2008, link here http://friispray.co.uk/infra-red-spray-can/ and we've been looking to implement the arduino and the pressure and distance sensors for the past year, would it be cool to blog about this on our site? we've ordered all the bits and are going to try out a new version based on your findings next week, were massive believers in open source so everything will be credited as you. 

Weve taken the one we made all round the UK and originally made it for back projecting on windows. were also taking it to NY in march time and would be great to see you there.

Super Nice

Dave
JamJarcollective.com

author
matthewvenn made it! (author)matthewvenn2010-01-09

Hey Dave,

I actually found your site after making my own, and don't know why I didn't find it earlier! I found wiispray and some videos on the internet but nothing opensource. I've been meaning to get in contact with you guys anyway!

So yes, please take anything and feel free to ask any questions/post blogs etc.

I'm in London, not NY, where are you based?

About This Instructable

18,745views

43favorites

Bio: I'm an artist/engineer excited about making cool stuff!
More by matthewvenn:Push fit LED moodlampwireless organ pipe doorbellfloppy drawbot
Add instructable to: