Touch-Sensitive Musical Painting

49,789

557

55

Introduction: Touch-Sensitive Musical Painting

About: Sophia Brueckner, born in Detroit, MI, is an artist, designer, and engineer. Inseparable from computers since the age of two, she believes she is a cyborg. She recently graduated from the MIT Media Lab and the…
  • This musical painting is a graphical score that you touch to play melodies and harmonies. It incorporates visual composition, musical composition, and gestural interaction to create a new form of visual music where the composition and the instrument are combined together.
  • This Instructable will show you how to make a touch-sensitive musical painting with a Teensy to send MIDI to a Mac Mini that synthesizes sounds with Ableton Live. The painting itself is made with acrylic paint, conductive paint, colored pencil, and nails on a wood panel.
  • To get an overview of how to do capacitive touch sensing in general, check out this Instructable.
  • This technique can be easily applied to other materials and touch sensing applications!
  • This painting was a collaboration between Sophia Brueckner and Eric Rosenbaum.
  • This Instructable is a work in progress. We have been refining our process with each iteration, and feedback is welcome!

Step 1: Materials

  • Wooden art panel
    • These are available at most art stores
    • They can be round or rectangular
    • Choose a thicker one (We used a 1.5" thick panel) to give yourself room for the electronics
  • Gesso primer (available at any art store)
  • Acrylic paints
  • Brushes, stencils, etc.
  • Hammer and nails
    • Use the smallest nails you can as long as they go all the way through the panel and stick out far enough to solder to. Larger nails may split the wood.
  • Wire, solder, copper tape (available on Adafruit or Sparkfun)
  • Teensy board (available on Adafruit or Sparkfun)
  • Computer (We used Ableton Live, but you could use Processing or anything else you like to make sound.)
  • External speakers (optional)

Step 2: Prepare the Surface

We used an unprimed wooden art panel (1.5" thick) because:

  • It's sturdy...people will be touching this, so you want a durable material.
  • It's a great painting surface, and it works well with the conductive paint.
  • There's room in the back to hide the electronics.
  • The rigid surface keeps electrical connections from being jostled too much.

These are available at most art stores. You could also make your own, or experiment with other materials (canvas and conductive thread?).

Prime your surface with Gesso. It's easier to paint on a primed surface, and your colors will look better. I usually do 2-3 coats. You can use a cheap, wide paintbrush to do this. Wait for it to dry after each coat. If your surface is streaky, you can sand it when it's fully dry.

Step 3: Paint the Painting

Next, paint your background and all the parts of the painting that are not going to be touch sensitive. Do this as if you were painting a regular painting. You will paint the conductive shapes on top of this in the next step.

I used Holbein Acryla gouache and Prismacolor colored pencils, but you could use something else. Just remember that people will be touching the painting, so choose something that won't get ruined if it gets wet or if it is rubbed a lot. Acrylic paints are a great choice. If you are not as comfortable with a paintbrush, you could also use stencils and a stencil brush.

Step 4: Paint the Conductive Shapes

After your painting is fully dry, use the conductive paint to paint your touch sensitive shapes on top. Remember these shapes shouldn't touch. Paint somewhat thickly, and avoid watering down the conductive paint. You can mix in a little water with some conductive paint on a palette if you are struggling to paint tiny details or clean edges and need it to flow a bit more. Let the painting dry completely.

Warning: If you attempt to mix colored paint with the conductive paint to make new colors, your shapes may no longer be conductive enough for touch sensing. It's best to keep the paint as pure as possible.

Step 5: Connect Conductive Shapes to Back of Painting

We wanted this to be very sturdy because we put it in a public space and it gets a lot of use, so we hammered regular nails through each conductive shape. You can touch the shape anywhere, not just on the nail head, to produce sound. The nails create a solid connection between the conductive shapes and the electronics on the back of the painting. I painted over some of the nail heads with more conductive paint for visual interest, but this is not needed for functionality.

Tip: You don't necessarily have to have one shape per sound. On the back of the painting you can wire multiple shapes to play the same sound. This simplifies the electronics while allowing you to create a more complex composition.

Step 6: Wire the Conductive Shapes

Connect the nails to the inputs on your board/s. We used a mixture of soldered wires and copper tape. If your painting is complex, it is helpful to label all the nails with a pencil on the back. Keep in mind how many inputs you have available on your board/s. We often connected multiple shapes together to the same input to reduce the number of inputs we needed.

Step 7: Program Your Board

The following code allows you to program your Teensy board to send MIDI to your computer. You can also download the code on GitHub.

int bounceThreshold = 0; // a press must last this many loop cycles (about 20ms each) before triggering.
// higher values make it less sensitive, fewer false triggers, higher latency

#define NUM_INPUTS 44

int channel = 1;

// edit this array to change the MIDI notes sent //int pitches[NUM_INPUTS] = { // 79, 84, 75, 67, 65, 63, // top of board (up, left, down, right, space, click) // 60, 62, 77, 68, 58, 56, // left side of female header, w,a,s,d,f,g // 51, 53, 48, 55, 72, 74 // right side of female header, up,down,left,right,left-click,right-click //};

// teensy makey pin numbers int pinNumbers[NUM_INPUTS] = { 0,1,2,3,4,5,7,8,9,10,11,12,13,14,15,16,17, // left side starting at USB connector, pin D0, skipping D6 26,25,24,23,22,21,20,19,18,38,39,40,41,42,43,44,45, // right side starting at USB connector, pin B6, skipping gnd and aref 36,37,32,33,34,35,28,29,30,31 // these are "interior" through-holes, in the center of the board };

// cap sense thresholds for each pin // this is proportional to the capacitance on the pin that will count as a press // it is units of a very small unit of time, in iterations through an unrolled loop // higher values make it less sensitive (i.e. require larger capacitance) //int capThresholds[NUM_INPUTS] = { // 1, 1, 1, 1, 1, 2, // 2, 2, 1, 1, 1, 1, // 2, 1, 1, 1, 1, 1, //};

int bounceCounter[NUM_INPUTS]; boolean pressed[NUM_INPUTS];

void setup(){ for (int i=0; i

void loop() { for (int i=0; i1){ // if we detect a touch on the pin if (!pressed[i]) { // and if we're not already pressed bounceCounter[i]++; // increment the bounce counter if(bounceCounter[i] > bounceThreshold){ // if we're over the bounce threshold usbMIDI.sendNoteOn(60+i,127,channel); // send a MIDI note on pressed[i] = true; // remember it was pressed bounceCounter[i]=0; // reset the bounce counter } } } else { // if we don't a detect touch on the pin if (pressed[i]) { // if this key was pressed before usbMIDI.sendNoteOff(60+i,127,channel); // send a MIDI note off pressed[i] = false; // remember we are not pressed bounceCounter[i] = 0; // reset the bounce counter } } } //Serial.println(" "); }

// CapacitiveSensor tutorial from http://www.arduino.cc/playground/Code/CapacitiveS... // readCapacitivePin // Input: Arduino pin number // Output: A number, from 0 to 17 expressing // how much capacitance is on the pin // When you touch the pin, or whatever you have // attached to it, the number will get higher

uint8_t readCapacitivePin(int pinToMeasure) { // Variables used to translate from Arduino to AVR pin naming volatile uint8_t* port; volatile uint8_t* ddr; volatile uint8_t* pin; // Here we translate the input pin number from // Arduino pin number to the AVR PORT, PIN, DDR, // and which bit of those registers we care about. byte bitmask; port = portOutputRegister(digitalPinToPort(pinToMeasure)); ddr = portModeRegister(digitalPinToPort(pinToMeasure)); bitmask = digitalPinToBitMask(pinToMeasure); pin = portInputRegister(digitalPinToPort(pinToMeasure)); // Discharge the pin first by setting it low and output *port &= ~(bitmask); *ddr |= bitmask; delay(1); // Make the pin an input with the internal pull-up on *ddr &= ~(bitmask); *port |= bitmask;

// Now see how long the pin to get pulled up. This manual unrolling of the loop // decreases the number of hardware cycles between each read of the pin, // thus increasing sensitivity. uint8_t cycles = 17; if (*pin & bitmask) { cycles = 0; } else if (*pin & bitmask) { cycles = 1; } else if (*pin & bitmask) { cycles = 2; } else if (*pin & bitmask) { cycles = 3; } else if (*pin & bitmask) { cycles = 4; } else if (*pin & bitmask) { cycles = 5; } else if (*pin & bitmask) { cycles = 6; } else if (*pin & bitmask) { cycles = 7; } else if (*pin & bitmask) { cycles = 8; } else if (*pin & bitmask) { cycles = 9; } else if (*pin & bitmask) { cycles = 10; } else if (*pin & bitmask) { cycles = 11; } else if (*pin & bitmask) { cycles = 12; } else if (*pin & bitmask) { cycles = 13; } else if (*pin & bitmask) { cycles = 14; } else if (*pin & bitmask) { cycles = 15; } else if (*pin & bitmask) { cycles = 16; }

// Discharge the pin again by setting it low and output // It's important to leave the pins low if you want to // be able to touch more than 1 sensor at a time - if // the sensor is left pulled high, when you touch // two sensors, your body will transfer the charge between // sensors. *port &= ~(bitmask); *ddr |= bitmask;

return cycles; }

Step 8: Using the MIDI Input

We used Ableton, but there are lots of ways to work with MIDI Input. Here are a few options:

Ableton MIDI Input

MidiBus for Processing

Send and Recieve MIDI Signals with Arduino

Step 9: Put It All Together

With some planning, you could also hide the computer and speakers within the painting itself. We used external speakers and kept the Mac Mini separate.

This Instructable is a work in progress as we build more of these musical paintings. Feedback is welcome!

Be the First to Share

    Recommendations

    • For the Home Contest

      For the Home Contest
    • Big and Small Contest

      Big and Small Contest
    • Game Design: Student Design Challenge

      Game Design: Student Design Challenge

    55 Comments

    0
    laserr
    laserr

    7 years ago

    I love it! I would like to build a small version (12 keys) with the Teensy 2.0. Can you please post the schematic or wiring diagram?

    Thanks!

    0
    vasilem4
    vasilem4

    7 years ago

    Hello...for my science fair i decided to build a makey makey out of Arduino UNO. I am not a pro but i can sort of manage Arduino. I know how to make the circuit (i am going to use a very high resistance : 1M ohms). The problem is that i am not sure on how to program it. Please help me!!!!

    0
    sab-art
    sab-art

    Reply 8 years ago on Introduction

    It was a while ago, but I am pretty sure we used the Teensy++ 2.0 and this: https://www.pjrc.com/teensy/teensyduino.html

    I believe you could use the Teensy 3.1 as well, but the code would be different. I don't know much about the Audio Board. Let me know how that works for you!

    0
    coffeestains
    coffeestains

    Reply 7 years ago on Introduction

    I got the Teensy++ 2.0 ... When I touch a pin, it sends a MIDI Signal to Ableton and I am able trigger the sound. But if behaving funny.

    I put the Teensy on a breadboard and put in a wire to one of the I/O pins. Even though I am holding the shielded portion of the wire (so no touch / capacitive touch) Ableton is getting a MIDI signal.

    Also the program, teensy, Ableton something is crashing. I am not getting consitent signals and after a couple of touches have to reboot ableton, the board. GOD KNOWS WHATS HAPPENING. Totall lost. But am trying to find out. Will share the solution if I find out :)

    0
    jeremy.j.adamowicz
    jeremy.j.adamowicz

    Reply 7 years ago

    How did you get ableton to recognize the teensy? My code seems correct but ableton wont see it. Thanks

    0
    coffeestains
    coffeestains

    Reply 7 years ago on Introduction

    Its like installing a printer driver. If installed properly, it shows up in print command of all applications. So you have to make sure you installed Teensy correctly. It has nothing to do with Ableton. I am working on something now. Gimme a day to go thru my installation procedure, will revert back with more info and hopefully it will help.

    0
    jeremy.j.adamowicz
    jeremy.j.adamowicz

    7 years ago

    So im trying to make this using thr teensy ++2.0 but the code seems a bit funny. Is the void setup really only supposed to be that 1 line that is shown?? Thank you

    0
    cstreit
    cstreit

    7 years ago on Introduction

    I love this idea and decided to see if I could pull it off, but I am not good with coding and just by copying over the code provided, there are a bunch of errors when trying to compile. I am using the teensy ++ 2.0 board so any help would be greatly appreciated.

    0
    sab-art
    sab-art

    Reply 7 years ago on Introduction

    Can you post the error messages? I'll try to help!

    0
    cstreit
    cstreit

    Reply 7 years ago on Introduction

    Arduino: 1.6.3 (Windows 8.1), TD: 1.22, Board: "Arduino Uno"

    P4.ino:2:1: error: expected unqualified-id before numeric constant

    P4.ino:5:1: error: expected unqualified-id before numeric constant

    P4.ino:8:1: error: 'from' does not name a type

    P4.ino:1:1: error: expected unqualified-id before numeric constant

    P4.ino:2:1: error: expected unqualified-id before numeric constant

    P4.ino:9:1: error: expected unqualified-id before numeric constant

    P4.ino:25:1: error: expected unqualified-id before numeric constant

    P4.ino:36:1: error: expected unqualified-id before numeric constant

    P4.ino:39:1: error: expected unqualified-id before numeric constant

    P4.ino:40:1: error: expected unqualified-id before numeric constant

    Error compiling.

    This report would have more information with
    "Show verbose output during compilation"
    enabled in File > Preferences.

    0
    coffeestains
    coffeestains

    Reply 7 years ago on Introduction

    Look at http://www.PJRC.com was installation instructions of the board. In Arduino you have to select Tool>>Board>> Teensy++2.0 Not Arduino Uno.

    Also you have to set Tools>USBTYPE>>MIDI.

    Additionally there is diff in code provided here and the GITHUM link povided here. Take the code from http://www.github.com/ericrosenbaum/makeymakey-midi/blob/master/makey_teensy_midi.ino

    0
    bliedahl
    bliedahl

    7 years ago

    Thanks for the inspiration!! My friend and I are working on our own versions of this project. Plans are for 24" x 24" but I wanted to create a couple of small versions to test before committing to the big one. The photo is of the drawing I did for planning and the video is my first prototype on 9 x 12 canvas board. Here, I have connected it to Makey Makey and my 2nd grade neighbor and fellow crafter is playing it. I have to learn how to reprogram the Makey Makey so I can use all of the pins, but I'm eventually going to use a Teensy board for the big project. I have a lot to learn about Arduino...

    image.jpg
    0
    sab-art
    sab-art

    Reply 7 years ago

    This is awesome!!!!!! I love the center design. Super great work, and I'm glad you were inspired! Thank you so much for sharing!

    0
    sab-art
    sab-art

    Reply 7 years ago on Introduction

    It's been a long time since we made this and it was a bit of a hack, but I'm pretty sure this code is for the Teensy++ 2.0. You could use a MaKey MaKey and do this the normal MaKey MaKey way with having the user touching ground to get this working the fastest. Other boards also can work, but you will have to change the pin mappings. See some of my other comments below, and let me know what you learn and I'll update the Instructable!

    0
    bulta
    bulta

    8 years ago on Introduction

    I have seen incredible things in a nice instructables other functional or curious

    but as this no that creating more beautiful in every way and mind that / minds they were able to imagine prodigious

    congratulations and thanks for sharing art