Touch-Sensitive Musical Painting





Introduction: Touch-Sensitive Musical Painting

  • This musical painting is a graphical score that you touch to play melodies and harmonies. It incorporates visual composition, musical composition, and gestural interaction to create a new form of visual music where the composition and the instrument are combined together.
  • This Instructable will show you how to make a touch-sensitive musical painting with a Teensy to send MIDI to a Mac Mini that synthesizes sounds with Ableton Live. The painting itself is made with acrylic paint, conductive paint, colored pencil, and nails on a wood panel.
  • To get an overview of how to do capacitive touch sensing in general, check out this Instructable.
  • This technique can be easily applied to other materials and touch sensing applications!
  • This painting was a collaboration between Sophia Brueckner and Eric Rosenbaum.
  • This Instructable is a work in progress. We have been refining our process with each iteration, and feedback is welcome!

Step 1: Materials

  • Wooden art panel
    • These are available at most art stores
    • They can be round or rectangular
    • Choose a thicker one (We used a 1.5" thick panel) to give yourself room for the electronics
  • Gesso primer (available at any art store)
  • Acrylic paints
  • Brushes, stencils, etc.
  • Hammer and nails
    • Use the smallest nails you can as long as they go all the way through the panel and stick out far enough to solder to. Larger nails may split the wood.
  • Wire, solder, copper tape (available on Adafruit or Sparkfun)
  • Teensy board (available on Adafruit or Sparkfun)
  • Computer (We used Ableton Live, but you could use Processing or anything else you like to make sound.)
  • External speakers (optional)

Step 2: Prepare the Surface

We used an unprimed wooden art panel (1.5" thick) because:

  • It's sturdy...people will be touching this, so you want a durable material.
  • It's a great painting surface, and it works well with the conductive paint.
  • There's room in the back to hide the electronics.
  • The rigid surface keeps electrical connections from being jostled too much.

These are available at most art stores. You could also make your own, or experiment with other materials (canvas and conductive thread?).

Prime your surface with Gesso. It's easier to paint on a primed surface, and your colors will look better. I usually do 2-3 coats. You can use a cheap, wide paintbrush to do this. Wait for it to dry after each coat. If your surface is streaky, you can sand it when it's fully dry.

Step 3: Paint the Painting

Next, paint your background and all the parts of the painting that are not going to be touch sensitive. Do this as if you were painting a regular painting. You will paint the conductive shapes on top of this in the next step.

I used Holbein Acryla gouache and Prismacolor colored pencils, but you could use something else. Just remember that people will be touching the painting, so choose something that won't get ruined if it gets wet or if it is rubbed a lot. Acrylic paints are a great choice. If you are not as comfortable with a paintbrush, you could also use stencils and a stencil brush.

Step 4: Paint the Conductive Shapes

After your painting is fully dry, use the conductive paint to paint your touch sensitive shapes on top. Remember these shapes shouldn't touch. Paint somewhat thickly, and avoid watering down the conductive paint. You can mix in a little water with some conductive paint on a palette if you are struggling to paint tiny details or clean edges and need it to flow a bit more. Let the painting dry completely.

Warning: If you attempt to mix colored paint with the conductive paint to make new colors, your shapes may no longer be conductive enough for touch sensing. It's best to keep the paint as pure as possible.

Step 5: Connect Conductive Shapes to Back of Painting

We wanted this to be very sturdy because we put it in a public space and it gets a lot of use, so we hammered regular nails through each conductive shape. You can touch the shape anywhere, not just on the nail head, to produce sound. The nails create a solid connection between the conductive shapes and the electronics on the back of the painting. I painted over some of the nail heads with more conductive paint for visual interest, but this is not needed for functionality.

Tip: You don't necessarily have to have one shape per sound. On the back of the painting you can wire multiple shapes to play the same sound. This simplifies the electronics while allowing you to create a more complex composition.

Step 6: Wire the Conductive Shapes

Connect the nails to the inputs on your board/s. We used a mixture of soldered wires and copper tape. If your painting is complex, it is helpful to label all the nails with a pencil on the back. Keep in mind how many inputs you have available on your board/s. We often connected multiple shapes together to the same input to reduce the number of inputs we needed.

Step 7: Program Your Board

The following code allows you to program your Teensy board to send MIDI to your computer. You can also download the code on GitHub.

int bounceThreshold = 0; // a press must last this many loop cycles (about 20ms each) before triggering.
// higher values make it less sensitive, fewer false triggers, higher latency

#define NUM_INPUTS 44

int channel = 1;

// edit this array to change the MIDI notes sent //int pitches[NUM_INPUTS] = { // 79, 84, 75, 67, 65, 63, // top of board (up, left, down, right, space, click) // 60, 62, 77, 68, 58, 56, // left side of female header, w,a,s,d,f,g // 51, 53, 48, 55, 72, 74 // right side of female header, up,down,left,right,left-click,right-click //};

// teensy makey pin numbers int pinNumbers[NUM_INPUTS] = { 0,1,2,3,4,5,7,8,9,10,11,12,13,14,15,16,17, // left side starting at USB connector, pin D0, skipping D6 26,25,24,23,22,21,20,19,18,38,39,40,41,42,43,44,45, // right side starting at USB connector, pin B6, skipping gnd and aref 36,37,32,33,34,35,28,29,30,31 // these are "interior" through-holes, in the center of the board };

// cap sense thresholds for each pin // this is proportional to the capacitance on the pin that will count as a press // it is units of a very small unit of time, in iterations through an unrolled loop // higher values make it less sensitive (i.e. require larger capacitance) //int capThresholds[NUM_INPUTS] = { // 1, 1, 1, 1, 1, 2, // 2, 2, 1, 1, 1, 1, // 2, 1, 1, 1, 1, 1, //};

int bounceCounter[NUM_INPUTS]; boolean pressed[NUM_INPUTS];

void setup(){ for (int i=0; i

void loop() { for (int i=0; i1){ // if we detect a touch on the pin if (!pressed[i]) { // and if we're not already pressed bounceCounter[i]++; // increment the bounce counter if(bounceCounter[i] > bounceThreshold){ // if we're over the bounce threshold usbMIDI.sendNoteOn(60+i,127,channel); // send a MIDI note on pressed[i] = true; // remember it was pressed bounceCounter[i]=0; // reset the bounce counter } } } else { // if we don't a detect touch on the pin if (pressed[i]) { // if this key was pressed before usbMIDI.sendNoteOff(60+i,127,channel); // send a MIDI note off pressed[i] = false; // remember we are not pressed bounceCounter[i] = 0; // reset the bounce counter } } } //Serial.println(" "); }

// CapacitiveSensor tutorial from // readCapacitivePin // Input: Arduino pin number // Output: A number, from 0 to 17 expressing // how much capacitance is on the pin // When you touch the pin, or whatever you have // attached to it, the number will get higher

uint8_t readCapacitivePin(int pinToMeasure) { // Variables used to translate from Arduino to AVR pin naming volatile uint8_t* port; volatile uint8_t* ddr; volatile uint8_t* pin; // Here we translate the input pin number from // Arduino pin number to the AVR PORT, PIN, DDR, // and which bit of those registers we care about. byte bitmask; port = portOutputRegister(digitalPinToPort(pinToMeasure)); ddr = portModeRegister(digitalPinToPort(pinToMeasure)); bitmask = digitalPinToBitMask(pinToMeasure); pin = portInputRegister(digitalPinToPort(pinToMeasure)); // Discharge the pin first by setting it low and output *port &= ~(bitmask); *ddr |= bitmask; delay(1); // Make the pin an input with the internal pull-up on *ddr &= ~(bitmask); *port |= bitmask;

// Now see how long the pin to get pulled up. This manual unrolling of the loop // decreases the number of hardware cycles between each read of the pin, // thus increasing sensitivity. uint8_t cycles = 17; if (*pin & bitmask) { cycles = 0; } else if (*pin & bitmask) { cycles = 1; } else if (*pin & bitmask) { cycles = 2; } else if (*pin & bitmask) { cycles = 3; } else if (*pin & bitmask) { cycles = 4; } else if (*pin & bitmask) { cycles = 5; } else if (*pin & bitmask) { cycles = 6; } else if (*pin & bitmask) { cycles = 7; } else if (*pin & bitmask) { cycles = 8; } else if (*pin & bitmask) { cycles = 9; } else if (*pin & bitmask) { cycles = 10; } else if (*pin & bitmask) { cycles = 11; } else if (*pin & bitmask) { cycles = 12; } else if (*pin & bitmask) { cycles = 13; } else if (*pin & bitmask) { cycles = 14; } else if (*pin & bitmask) { cycles = 15; } else if (*pin & bitmask) { cycles = 16; }

// Discharge the pin again by setting it low and output // It's important to leave the pins low if you want to // be able to touch more than 1 sensor at a time - if // the sensor is left pulled high, when you touch // two sensors, your body will transfer the charge between // sensors. *port &= ~(bitmask); *ddr |= bitmask;

return cycles; }

Step 8: Using the MIDI Input

We used Ableton, but there are lots of ways to work with MIDI Input. Here are a few options:

Ableton MIDI Input

MidiBus for Processing

Send and Recieve MIDI Signals with Arduino

Step 9: Put It All Together

With some planning, you could also hide the computer and speakers within the painting itself. We used external speakers and kept the Mac Mini separate.

This Instructable is a work in progress as we build more of these musical paintings. Feedback is welcome!



  • Epilog Challenge 9

    Epilog Challenge 9
  • Sew Warm Contest 2018

    Sew Warm Contest 2018
  • First Time Author Contest 2018

    First Time Author Contest 2018

We have a be nice policy.
Please be positive and constructive.




I love it! I would like to build a small version (12 keys) with the Teensy 2.0. Can you please post the schematic or wiring diagram?


Hello...for my science fair i decided to build a makey makey out of Arduino UNO. I am not a pro but i can sort of manage Arduino. I know how to make the circuit (i am going to use a very high resistance : 1M ohms). The problem is that i am not sure on how to program it. Please help me!!!!

It was a while ago, but I am pretty sure we used the Teensy++ 2.0 and this:

I believe you could use the Teensy 3.1 as well, but the code would be different. I don't know much about the Audio Board. Let me know how that works for you!

I got the Teensy++ 2.0 ... When I touch a pin, it sends a MIDI Signal to Ableton and I am able trigger the sound. But if behaving funny.

I put the Teensy on a breadboard and put in a wire to one of the I/O pins. Even though I am holding the shielded portion of the wire (so no touch / capacitive touch) Ableton is getting a MIDI signal.

Also the program, teensy, Ableton something is crashing. I am not getting consitent signals and after a couple of touches have to reboot ableton, the board. GOD KNOWS WHATS HAPPENING. Totall lost. But am trying to find out. Will share the solution if I find out :)

How did you get ableton to recognize the teensy? My code seems correct but ableton wont see it. Thanks

Its like installing a printer driver. If installed properly, it shows up in print command of all applications. So you have to make sure you installed Teensy correctly. It has nothing to do with Ableton. I am working on something now. Gimme a day to go thru my installation procedure, will revert back with more info and hopefully it will help.

So im trying to make this using thr teensy ++2.0 but the code seems a bit funny. Is the void setup really only supposed to be that 1 line that is shown?? Thank you

I love this idea and decided to see if I could pull it off, but I am not good with coding and just by copying over the code provided, there are a bunch of errors when trying to compile. I am using the teensy ++ 2.0 board so any help would be greatly appreciated.

Can you post the error messages? I'll try to help!