This musical painting is a graphical score that you touch to play melodies and harmonies. It incorporates visual composition, musical composition, and gestural interaction to create a new form of visual music where the composition and the instrument are combined together.
This Instructable will show you how to make a touch-sensitive musical painting with a Teensy to send MIDI to a Mac Mini that synthesizes sounds with Ableton Live. The painting itself is made with acrylic paint, conductive paint, colored pencil, and nails on a wood panel.
To get an overview of how to do capacitive touch sensing in general, check out this Instructable.
This technique can be easily applied to other materials and touch sensing applications!
This painting was a collaboration between Sophia Brueckner and Eric Rosenbaum.
This Instructable is a work in progress. We have been refining our process with each iteration, and feedback is welcome!

Step 1: Materials

Conductive paint
- Electromagnetic shielding paint works really well, and you can buy it in large quantities (http://www.lessemf.com/paint.html)
- Bare Conductive paints are easily available, but they smear a little bit more (http://www.bareconductive.com/shop/)

Wooden art panel
- These are available at most art stores
- They can be round or rectangular
- Choose a thicker one (We used a 1.5" thick panel) to give yourself room for the electronics
Gesso primer (available at any art store)
Acrylic paints
Brushes, stencils, etc.
Hammer and nails
- Use the smallest nails you can as long as they go all the way through the panel and stick out far enough to solder to. Larger nails may split the wood.
Wire, solder, copper tape (available on Adafruit or Sparkfun)
Teensy board (available on Adafruit or Sparkfun)
Computer (We used Ableton Live, but you could use Processing or anything else you like to make sound.)
External speakers (optional)

Step 2: Prepare the Surface

We used an unprimed wooden art panel (1.5" thick) because:

It's sturdy...people will be touching this, so you want a durable material.
It's a great painting surface, and it works well with the conductive paint.
There's room in the back to hide the electronics.
The rigid surface keeps electrical connections from being jostled too much.

These are available at most art stores. You could also make your own, or experiment with other materials (canvas and conductive thread?).

Prime your surface with Gesso. It's easier to paint on a primed surface, and your colors will look better. I usually do 2-3 coats. You can use a cheap, wide paintbrush to do this. Wait for it to dry after each coat. If your surface is streaky, you can sand it when it's fully dry.

Step 3: Paint the Painting

Next, paint your background and all the parts of the painting that are not going to be touch sensitive. Do this as if you were painting a regular painting. You will paint the conductive shapes on top of this in the next step.

I used Holbein Acryla gouache and Prismacolor colored pencils, but you could use something else. Just remember that people will be touching the painting, so choose something that won't get ruined if it gets wet or if it is rubbed a lot. Acrylic paints are a great choice. If you are not as comfortable with a paintbrush, you could also use stencils and a stencil brush.

Step 4: Paint the Conductive Shapes

After your painting is fully dry, use the conductive paint to paint your touch sensitive shapes on top. Remember these shapes shouldn't touch. Paint somewhat thickly, and avoid watering down the conductive paint. You can mix in a little water with some conductive paint on a palette if you are struggling to paint tiny details or clean edges and need it to flow a bit more. Let the painting dry completely.

Warning: If you attempt to mix colored paint with the conductive paint to make new colors, your shapes may no longer be conductive enough for touch sensing. It's best to keep the paint as pure as possible.

Step 5: Connect Conductive Shapes to Back of Painting

We wanted this to be very sturdy because we put it in a public space and it gets a lot of use, so we hammered regular nails through each conductive shape. You can touch the shape anywhere, not just on the nail head, to produce sound. The nails create a solid connection between the conductive shapes and the electronics on the back of the painting. I painted over some of the nail heads with more conductive paint for visual interest, but this is not needed for functionality.

Tip: You don't necessarily have to have one shape per sound. On the back of the painting you can wire multiple shapes to play the same sound. This simplifies the electronics while allowing you to create a more complex composition.

Step 6: Wire the Conductive Shapes

Connect the nails to the inputs on your board/s. We used a mixture of soldered wires and copper tape. If your painting is complex, it is helpful to label all the nails with a pencil on the back. Keep in mind how many inputs you have available on your board/s. We often connected multiple shapes together to the same input to reduce the number of inputs we needed.

Step 7: Program Your Board

The following code allows you to program your Teensy board to send MIDI to your computer. You can also download the code on GitHub.

int bounceThreshold = 0; // a press must last this many loop cycles (about 20ms each) before triggering. 
// higher values make it less sensitive, fewer false triggers, higher latency
#define NUM_INPUTS 44
int channel = 1;
// edit this array to change the MIDI notes sent 
//int pitches[NUM_INPUTS] = {
//  79, 84, 75, 67, 65, 63,           // top of board (up, left, down, right, space, click)
//  60, 62, 77, 68, 58, 56,           // left side of female header, w,a,s,d,f,g
//  51, 53, 48, 55, 72, 74            // right side of female header, up,down,left,right,left-click,right-click
//};
// teensy makey pin numbers
int pinNumbers[NUM_INPUTS] = {
  0,1,2,3,4,5,7,8,9,10,11,12,13,14,15,16,17,         // left side starting at USB connector, pin D0, skipping D6 
  26,25,24,23,22,21,20,19,18,38,39,40,41,42,43,44,45, // right side starting at USB connector, pin B6, skipping gnd and aref
  36,37,32,33,34,35,28,29,30,31                      // these are "interior" through-holes, in the center of the board
};
// cap sense thresholds for each pin
// this is proportional to the capacitance on the pin that will count as a press
// it is units of a very small unit of time, in iterations through an unrolled loop
// higher values make it less sensitive (i.e. require larger capacitance)
//int capThresholds[NUM_INPUTS] = {
//  1, 1, 1, 1, 1, 2,
//  2, 2, 1, 1, 1, 1,
//  2, 1, 1, 1, 1, 1,
//};
int bounceCounter[NUM_INPUTS];    
boolean pressed[NUM_INPUTS];
void setup(){
  for (int i=0; i
void loop() { 
  for (int i=0; i1){                                                       // if we detect a touch on the pin
      if (!pressed[i]) {                                          // and if we're not already pressed
        bounceCounter[i]++;                                           // increment the bounce counter
        if(bounceCounter[i] > bounceThreshold){                       // if we're over the bounce threshold
          usbMIDI.sendNoteOn(60+i,127,channel);                      // send a MIDI note on
          pressed[i] = true;                                              // remember it was pressed
          bounceCounter[i]=0;                                             // reset the bounce counter
        }
      }
    } 
    else {                                                  // if we don't a detect touch on the pin
      if (pressed[i]) {                                           // if this key was pressed before
        usbMIDI.sendNoteOff(60+i,127,channel);                    // send a MIDI note off
        pressed[i] = false;                                          // remember we are not pressed
        bounceCounter[i] = 0;                                        // reset the bounce counter
      }        
    }
  }
  //Serial.println(" ");
}  
// CapacitiveSensor tutorial from  http://www.arduino.cc/playground/Code/CapacitiveS...
// readCapacitivePin
//  Input: Arduino pin number
//  Output: A number, from 0 to 17 expressing
//  how much capacitance is on the pin
//  When you touch the pin, or whatever you have
//  attached to it, the number will get higher
uint8_t readCapacitivePin(int pinToMeasure) {
  // Variables used to translate from Arduino to AVR pin naming
  volatile uint8_t* port;
  volatile uint8_t* ddr;
  volatile uint8_t* pin;
  // Here we translate the input pin number from
  //  Arduino pin number to the AVR PORT, PIN, DDR,
  //  and which bit of those registers we care about.
  byte bitmask;
  port = portOutputRegister(digitalPinToPort(pinToMeasure));
  ddr = portModeRegister(digitalPinToPort(pinToMeasure));
  bitmask = digitalPinToBitMask(pinToMeasure);
  pin = portInputRegister(digitalPinToPort(pinToMeasure));
  // Discharge the pin first by setting it low and output
  *port &= ~(bitmask);
  *ddr  |= bitmask;
  delay(1);
  // Make the pin an input with the internal pull-up on
  *ddr &= ~(bitmask);
  *port |= bitmask;
  // Now see how long the pin to get pulled up. This manual unrolling of the loop
  // decreases the number of hardware cycles between each read of the pin,
  // thus increasing sensitivity.
  uint8_t cycles = 17;
  if (*pin & bitmask) { 
    cycles =  0;
  }
  else if (*pin & bitmask) { 
    cycles =  1;
  }
  else if (*pin & bitmask) { 
    cycles =  2;
  }
  else if (*pin & bitmask) { 
    cycles =  3;
  }
  else if (*pin & bitmask) { 
    cycles =  4;
  }
  else if (*pin & bitmask) { 
    cycles =  5;
  }
  else if (*pin & bitmask) { 
    cycles =  6;
  }
  else if (*pin & bitmask) { 
    cycles =  7;
  }
  else if (*pin & bitmask) { 
    cycles =  8;
  }
  else if (*pin & bitmask) { 
    cycles =  9;
  }
  else if (*pin & bitmask) { 
    cycles = 10;
  }
  else if (*pin & bitmask) { 
    cycles = 11;
  }
  else if (*pin & bitmask) { 
    cycles = 12;
  }
  else if (*pin & bitmask) { 
    cycles = 13;
  }
  else if (*pin & bitmask) { 
    cycles = 14;
  }
  else if (*pin & bitmask) { 
    cycles = 15;
  }
  else if (*pin & bitmask) { 
    cycles = 16;
  }
  // Discharge the pin again by setting it low and output
  //  It's important to leave the pins low if you want to 
  //  be able to touch more than 1 sensor at a time - if
  //  the sensor is left pulled high, when you touch
  //  two sensors, your body will transfer the charge between
  //  sensors.
  *port &= ~(bitmask);
  *ddr  |= bitmask;
  return cycles;
}

Step 8: Using the MIDI Input

We used Ableton, but there are lots of ways to work with MIDI Input. Here are a few options:

Ableton MIDI Input

MidiBus for Processing

Send and Recieve MIDI Signals with Arduino

Step 9: Put It All Together

With some planning, you could also hide the computer and speakers within the painting itself. We used external speakers and kept the Mac Mini separate.

This Instructable is a work in progress as we build more of these musical paintings. Feedback is welcome!

Be the First to Share

Recommendations

Rubik's Cube Solver by aaedmusa in Gadgets

76 10K

Laser-Cut Infinity Dodecahedron (Fusion 360) by thomasj152 in LEDs

174 15K

Pingo, the Secret Color Clock by Illusionmanager by illusionmanager in Clocks

1 37 3.5K

A Pocket Sundial From a Broken Pocket Watch! by JGJMatt in Metalworking

83 8.1K

For the Home Contest
Big and Small Contest
Game Design: Student Design Challenge

55 Comments

laserr

7 years ago

I love it! I would like to build a small version (12 keys) with the Teensy 2.0. Can you please post the schematic or wiring diagram?

Thanks!

vasilem4

7 years ago

Hello...for my science fair i decided to build a makey makey out of Arduino UNO. I am not a pro but i can sort of manage Arduino. I know how to make the circuit (i am going to use a very high resistance : 1M ohms). The problem is that i am not sure on how to program it. Please help me!!!!

coffeestains

8 years ago on Introduction

There are three kinds of boards, can you please help choose?

Teensy++ 2.0

Teensy Audio Board

Teensy 3.1

jeremy.j.adamowicz

7 years ago

So im trying to make this using thr teensy ++2.0 but the code seems a bit funny. Is the void setup really only supposed to be that 1 line that is shown?? Thank you

cstreit

7 years ago on Introduction

I love this idea and decided to see if I could pull it off, but I am not good with coding and just by copying over the code provided, there are a bunch of errors when trying to compile. I am using the teensy ++ 2.0 board so any help would be greatly appreciated.

bliedahl

7 years ago

Thanks for the inspiration!! My friend and I are working on our own versions of this project. Plans are for 24" x 24" but I wanted to create a couple of small versions to test before committing to the big one. The photo is of the drawing I did for planning and the video is my first prototype on 9 x 12 canvas board. Here, I have connected it to Makey Makey and my 2nd grade neighbor and fellow crafter is playing it. I have to learn how to reprogram the Makey Makey so I can use all of the pins, but I'm eventually going to use a Teensy board for the big project. I have a lot to learn about Arduino...

IMG_6260.MOV

KristinYankey

8 years ago on Introduction

Which Teensy board did you use?

neonix

8 years ago on Introduction

So awesome!

bulta

8 years ago on Introduction

I have seen incredible things in a nice instructables other functional or curious

but as this no that creating more beautiful in every way and mind that / minds they were able to imagine prodigious

congratulations and thanks for sharing art

Introduction: Touch-Sensitive Musical Painting

Step 1: Materials

Step 2: Prepare the Surface

Step 3: Paint the Painting

Step 4: Paint the Conductive Shapes

Step 5: Connect Conductive Shapes to Back of Painting

Step 6: Wire the Conductive Shapes

Step 7: Program Your Board

Step 8: Using the MIDI Input

Step 9: Put It All Together

Be the First to Share

Recommendations

For the Home Contest

Big and Small Contest

Game Design: Student Design Challenge

55 Comments

Teensy++ 2.0

Teensy Audio Board

Teensy 3.1