I recommend at least skimming over my last two instructables before diving into this one, many of the example patches I use in this instructable are derived from patches built in the last two classes. In this Ible, I'll be going over how to get MaxMSP connected to Reactivision, Kinect, Arduino, MaKey MaKey, and the Novation Launchpad. The last step has some useful links and ideas for more ways to extend Max.
ReacTIVision is a free, open source tool that communicates with MaxMSP and allows you to track objects with a camera by attaching small paper markers to them. The software tracks x and y position and rotation and it's great for incorporating real world objects into your MaxMSP patch or making table top interfaces. It was originally written for the ReacTable, a table top multitouch instrument:
Here are some ReacTIVsion projects:
Here's how to do it: - Download the ReacTIVision vision engine - Download the ReacTIVision TUIO client for MaxMSP - Print out some of the fiducial markers (look in your ReacTIVision vision engine download, under symbols>>default.pdf) - Run the ReacTIVision application and hold up the markers to your webcam - you should see the number of the fiducial track with it as you move it around the screen. You may need to adjust your lighting to help reacTIVision see the markers.
By default ReacTIVision uses background subtraction to help it with tracking, in some cases this can be a hindrance. Use the 'E' key to toggle in and out of background subtraction mode and use the spacebar to recalculate the background until you find a setting that best tracks the fiducials. Pressing the 'S' key will show to the original source image. Pressing 'T' will show the binary tresholded image, and pressing the 'N' key will turn the display off, which reduces the CPU usage. The thresholder gradient gate can be adjusted by hitting the 'G' key, lowering the value can improve the thresholder performance in low light conditions with insufficient finger contrast for example. You can gradually lower the value just before noise appears in the image. The camera options can be adjusted by pressing the 'O' key. You can read more on the reacTIVision website and the README.txt (comes with the ReacTIVision vision engine) about the optimal lighting, camera, and background setup for the software.
- TUIOClient.help/mxe/and mxo are the files that allow you to pull in data from the ReacTIVision vision engine into Max. You must include these files in the same folder as any patch that requires them You can also add them into a directory that Max knows to look for external objects.
First open the file TuioDump.pat in Max (find it in the TUIO_MaxMSP folder), open up the Max window to see the logged output (Command + M / Ctrl + M).
Here is how the TUIOClient sends messages into Max:
When an object comes into view an addObject message is sent: addObject session_id symbol_id session_id is the number of the current session, we don't need to worry about that for most applications. symbol_id is the number of the symbol that came onscreen - these can be found on the defaults.pdf that you printed out.
As the marker is tracked around the screen, an updateObject message is sent for each frame (my computer runs at about a 15fps frame rate, your machine may differ depending on how hard it is working): updateObject session_id symbol_id xpos ypos angle xspeed yspeed rspeed maccel raccel session_id and symbol_id are the same as the addObject message xpos, ypos, and angle are the x position, yposition, and angle of rotation of the marker xspeed, yspeed, and rspeed are the x, y, and rotational velocities of the marker maccel is the acceleration of the marker across the screen raccel is the rotational acceleration of the marker you may find that you need just one or all of these pieces of information, but usually symbol_id, xpos, ypos, and rotation are the most useful
And when the marker is removed a removeObject is sent: removeObject session_id symbol_id session_id and symbol_id are the same as the addObject message
Use the patch below to read in and parse add, update, and remove messages for marker id, xpos, ypos, and rotation. Be sure to save it in the same folder as the TUIO objects you just downloaded.
Try wiring up the data from updateObject to the frequency modulated synth from the last Instructable. See if you can get the rotation of a fiducial to control the rate of frequency modulation. Or use the y position to change the pitch of the main oscillator. Hint: you'll need to use a scale object to scale your range of values to something appropriate for the synth. In case you get stuck: