Multimodal Music Stand

The Multimodal Music Stand (MMMS) is a new way to control audio synthesis and effects while playing a traditional instrument (sax, flute, violin, you name it), and offers quite a few extra possibilities for extended techniques that can be used while playing! There's already a webpage about our group project at UC Santa Barbara in which we developed the MMMS, so instead of repeating that here, I'll post the link - check it out!

Multimodal Music Stand Website

A short clip of the the MMMS in action is here, but there are better videos on the site above...

Multimodal refers to the capability of sensing input in more than one modality (audio input, video input, and sensor-based input). The audio and video input are pretty straightforward, involving a normal microphone and a webcam connected to your computer, but the sensor input is a bit more complex, hence this instructable...

We use the CREATE USB Interface, a simple circuit I developed for a class I teach at UCSB, along with the ThereminVision II kit for E-field (also known as capacitive sensing) as shown here:

CREATE USB Interface

ThereminVisionII kit

If you don't know what a Theremin is, do yourself a favor and search Youtube, you won't be dissapointed! The MMMS is basically equivalent to 2 Theremins, since it has 4 E-field sensing antennas, in addition to the audio/video input of course.

If you're interested in extending your performance techniques with your instrument by interacting with a computer, but don't want to use simple footpedals, build yourself a Multimodal Music Stand and start practicing with it!

Step 1 About the CREATE USB Interface

The CREATE USB Interface (CUI) is a simple PIC-based programmable circuit that can either be built entirely DIY, or you can get a pre-built one from me directly for $50 (+ 5 for shipping), and you won't need to buy a PIC programmer since I put the bootloader on it for you... see the website for details, or just email me to request one:

CREATE USB Interface website

The CUI can function as many different things, and has by now been used in hundreds of interesting projects all over the world... a few examples from students at UC Santa Barbara are in this conference paper (PDF). The CUI can be reprogrammed through the USB cable using the bootloader, which is why it is so easily adaptable - simply change the firmware and it is something else.

For example, it is easy to make the CUI into a wireless sensor interface using one of the Bluetooth modules from spark fun - if anyone is interested in doing this, post a note in the comments requesting another instructable. The CUI has 13 channels of 10-bit analog inputs and 16 general purpose input/output pins. The default firmware shipped with the CUI v1.0 boards sends them all as inputs to the host computer, and works nicely with interactive art-making environments such as Max/MSP/Jitter, Pd/Gem, SuperCollider, Chuck, etc.

I used one of the CUI v1.0 boards to build the Multimodal Music Stand in this instructable. We use Max/MSP/Jitter as an environment, but by all means feel free to adapt this to Pd or your choice of software. I am a big fan of open source (the CUI itself is open source), and it helps greatly if we all stand on each other's shoulders and improve the things we're making!

Comments

[supercatalyst]

This is beautiful...similar to theremin to CV to MIDI, but far more advanced.

[robots199]

Amazing is all that I can say. I am a viola player and love this project.