Introduction: Touch-Sensitive Musical Painting

  • This musical painting is a graphical score that you touch to play melodies and harmonies. It incorporates visual composition, musical composition, and gestural interaction to create a new form of visual music where the composition and the instrument are combined together.
  • This Instructable will show you how to make a touch-sensitive musical painting with a Teensy to send MIDI to a Mac Mini that synthesizes sounds with Ableton Live. The painting itself is made with acrylic paint, conductive paint, colored pencil, and nails on a wood panel.
  • To get an overview of how to do capacitive touch sensing in general, check out this Instructable.
  • This technique can be easily applied to other materials and touch sensing applications!
  • This painting was a collaboration between Sophia Brueckner and Eric Rosenbaum.
  • This Instructable is a work in progress. We have been refining our process with each iteration, and feedback is welcome!

Step 1: Materials

Picture of Materials
  • Wooden art panel
    • These are available at most art stores
    • They can be round or rectangular
    • Choose a thicker one (We used a 1.5" thick panel) to give yourself room for the electronics
  • Gesso primer (available at any art store)
  • Acrylic paints
  • Brushes, stencils, etc.
  • Hammer and nails
    • Use the smallest nails you can as long as they go all the way through the panel and stick out far enough to solder to. Larger nails may split the wood.
  • Wire, solder, copper tape (available on Adafruit or Sparkfun)
  • Teensy board (available on Adafruit or Sparkfun)
  • Computer (We used Ableton Live, but you could use Processing or anything else you like to make sound.)
  • External speakers (optional)

Step 2: Prepare the Surface

Picture of Prepare the Surface

We used an unprimed wooden art panel (1.5" thick) because:

  • It's sturdy...people will be touching this, so you want a durable material.
  • It's a great painting surface, and it works well with the conductive paint.
  • There's room in the back to hide the electronics.
  • The rigid surface keeps electrical connections from being jostled too much.

These are available at most art stores. You could also make your own, or experiment with other materials (canvas and conductive thread?).

Prime your surface with Gesso. It's easier to paint on a primed surface, and your colors will look better. I usually do 2-3 coats. You can use a cheap, wide paintbrush to do this. Wait for it to dry after each coat. If your surface is streaky, you can sand it when it's fully dry.

Step 3: Paint the Painting

Picture of Paint the Painting

Next, paint your background and all the parts of the painting that are not going to be touch sensitive. Do this as if you were painting a regular painting. You will paint the conductive shapes on top of this in the next step.

I used Holbein Acryla gouache and Prismacolor colored pencils, but you could use something else. Just remember that people will be touching the painting, so choose something that won't get ruined if it gets wet or if it is rubbed a lot. Acrylic paints are a great choice. If you are not as comfortable with a paintbrush, you could also use stencils and a stencil brush.

Step 4: Paint the Conductive Shapes

Picture of Paint the Conductive Shapes

After your painting is fully dry, use the conductive paint to paint your touch sensitive shapes on top. Remember these shapes shouldn't touch. Paint somewhat thickly, and avoid watering down the conductive paint. You can mix in a little water with some conductive paint on a palette if you are struggling to paint tiny details or clean edges and need it to flow a bit more. Let the painting dry completely.

Warning: If you attempt to mix colored paint with the conductive paint to make new colors, your shapes may no longer be conductive enough for touch sensing. It's best to keep the paint as pure as possible.

Step 5: Connect Conductive Shapes to Back of Painting

Picture of Connect Conductive Shapes to Back of Painting

We wanted this to be very sturdy because we put it in a public space and it gets a lot of use, so we hammered regular nails through each conductive shape. You can touch the shape anywhere, not just on the nail head, to produce sound. The nails create a solid connection between the conductive shapes and the electronics on the back of the painting. I painted over some of the nail heads with more conductive paint for visual interest, but this is not needed for functionality.

Tip: You don't necessarily have to have one shape per sound. On the back of the painting you can wire multiple shapes to play the same sound. This simplifies the electronics while allowing you to create a more complex composition.

Step 6: Wire the Conductive Shapes

Picture of Wire the Conductive Shapes

Connect the nails to the inputs on your board/s. We used a mixture of soldered wires and copper tape. If your painting is complex, it is helpful to label all the nails with a pencil on the back. Keep in mind how many inputs you have available on your board/s. We often connected multiple shapes together to the same input to reduce the number of inputs we needed.

Step 7: Program Your Board

The following code allows you to program your Teensy board to send MIDI to your computer. You can also download the code on GitHub.

int bounceThreshold = 0; // a press must last this many loop cycles (about 20ms each) before triggering.
// higher values make it less sensitive, fewer false triggers, higher latency

#define NUM_INPUTS 44

int channel = 1;

// edit this array to change the MIDI notes sent //int pitches[NUM_INPUTS] = { // 79, 84, 75, 67, 65, 63, // top of board (up, left, down, right, space, click) // 60, 62, 77, 68, 58, 56, // left side of female header, w,a,s,d,f,g // 51, 53, 48, 55, 72, 74 // right side of female header, up,down,left,right,left-click,right-click //};

// teensy makey pin numbers int pinNumbers[NUM_INPUTS] = { 0,1,2,3,4,5,7,8,9,10,11,12,13,14,15,16,17, // left side starting at USB connector, pin D0, skipping D6 26,25,24,23,22,21,20,19,18,38,39,40,41,42,43,44,45, // right side starting at USB connector, pin B6, skipping gnd and aref 36,37,32,33,34,35,28,29,30,31 // these are "interior" through-holes, in the center of the board };

// cap sense thresholds for each pin // this is proportional to the capacitance on the pin that will count as a press // it is units of a very small unit of time, in iterations through an unrolled loop // higher values make it less sensitive (i.e. require larger capacitance) //int capThresholds[NUM_INPUTS] = { // 1, 1, 1, 1, 1, 2, // 2, 2, 1, 1, 1, 1, // 2, 1, 1, 1, 1, 1, //};

int bounceCounter[NUM_INPUTS]; boolean pressed[NUM_INPUTS];

void setup(){ for (int i=0; i

void loop() { for (int i=0; i1){ // if we detect a touch on the pin if (!pressed[i]) { // and if we're not already pressed bounceCounter[i]++; // increment the bounce counter if(bounceCounter[i] > bounceThreshold){ // if we're over the bounce threshold usbMIDI.sendNoteOn(60+i,127,channel); // send a MIDI note on pressed[i] = true; // remember it was pressed bounceCounter[i]=0; // reset the bounce counter } } } else { // if we don't a detect touch on the pin if (pressed[i]) { // if this key was pressed before usbMIDI.sendNoteOff(60+i,127,channel); // send a MIDI note off pressed[i] = false; // remember we are not pressed bounceCounter[i] = 0; // reset the bounce counter } } } //Serial.println(" "); }

// CapacitiveSensor tutorial from http://www.arduino.cc/playground/Code/CapacitiveS... // readCapacitivePin // Input: Arduino pin number // Output: A number, from 0 to 17 expressing // how much capacitance is on the pin // When you touch the pin, or whatever you have // attached to it, the number will get higher

uint8_t readCapacitivePin(int pinToMeasure) { // Variables used to translate from Arduino to AVR pin naming volatile uint8_t* port; volatile uint8_t* ddr; volatile uint8_t* pin; // Here we translate the input pin number from // Arduino pin number to the AVR PORT, PIN, DDR, // and which bit of those registers we care about. byte bitmask; port = portOutputRegister(digitalPinToPort(pinToMeasure)); ddr = portModeRegister(digitalPinToPort(pinToMeasure)); bitmask = digitalPinToBitMask(pinToMeasure); pin = portInputRegister(digitalPinToPort(pinToMeasure)); // Discharge the pin first by setting it low and output *port &= ~(bitmask); *ddr |= bitmask; delay(1); // Make the pin an input with the internal pull-up on *ddr &= ~(bitmask); *port |= bitmask;

// Now see how long the pin to get pulled up. This manual unrolling of the loop // decreases the number of hardware cycles between each read of the pin, // thus increasing sensitivity. uint8_t cycles = 17; if (*pin & bitmask) { cycles = 0; } else if (*pin & bitmask) { cycles = 1; } else if (*pin & bitmask) { cycles = 2; } else if (*pin & bitmask) { cycles = 3; } else if (*pin & bitmask) { cycles = 4; } else if (*pin & bitmask) { cycles = 5; } else if (*pin & bitmask) { cycles = 6; } else if (*pin & bitmask) { cycles = 7; } else if (*pin & bitmask) { cycles = 8; } else if (*pin & bitmask) { cycles = 9; } else if (*pin & bitmask) { cycles = 10; } else if (*pin & bitmask) { cycles = 11; } else if (*pin & bitmask) { cycles = 12; } else if (*pin & bitmask) { cycles = 13; } else if (*pin & bitmask) { cycles = 14; } else if (*pin & bitmask) { cycles = 15; } else if (*pin & bitmask) { cycles = 16; }

// Discharge the pin again by setting it low and output // It's important to leave the pins low if you want to // be able to touch more than 1 sensor at a time - if // the sensor is left pulled high, when you touch // two sensors, your body will transfer the charge between // sensors. *port &= ~(bitmask); *ddr |= bitmask;

return cycles; }

Step 8: Using the MIDI Input

We used Ableton, but there are lots of ways to work with MIDI Input. Here are a few options:

Ableton MIDI Input

MidiBus for Processing

Send and Recieve MIDI Signals with Arduino

Step 9: Put It All Together

Picture of Put It All Together

With some planning, you could also hide the computer and speakers within the painting itself. We used external speakers and kept the Mac Mini separate.

This Instructable is a work in progress as we build more of these musical paintings. Feedback is welcome!

Comments

laserr (author)2016-03-10

I love it! I would like to build a small version (12 keys) with the Teensy 2.0. Can you please post the schematic or wiring diagram?

Thanks!

vasilem4 (author)2016-01-29

Hello...for my science fair i decided to build a makey makey out of Arduino UNO. I am not a pro but i can sort of manage Arduino. I know how to make the circuit (i am going to use a very high resistance : 1M ohms). The problem is that i am not sure on how to program it. Please help me!!!!

coffeestains (author)2015-03-19

There are three kinds of boards, can you please help choose?

Teensy++ 2.0

Teensy Audio Board

Teensy 3.1

sab-art (author)coffeestains2015-03-22

It was a while ago, but I am pretty sure we used the Teensy++ 2.0 and this: https://www.pjrc.com/teensy/teensyduino.html

I believe you could use the Teensy 3.1 as well, but the code would be different. I don't know much about the Audio Board. Let me know how that works for you!

coffeestains (author)sab-art2015-04-13

I got the Teensy++ 2.0 ... When I touch a pin, it sends a MIDI Signal to Ableton and I am able trigger the sound. But if behaving funny.

I put the Teensy on a breadboard and put in a wire to one of the I/O pins. Even though I am holding the shielded portion of the wire (so no touch / capacitive touch) Ableton is getting a MIDI signal.

Also the program, teensy, Ableton something is crashing. I am not getting consitent signals and after a couple of touches have to reboot ableton, the board. GOD KNOWS WHATS HAPPENING. Totall lost. But am trying to find out. Will share the solution if I find out :)

How did you get ableton to recognize the teensy? My code seems correct but ableton wont see it. Thanks

Its like installing a printer driver. If installed properly, it shows up in print command of all applications. So you have to make sure you installed Teensy correctly. It has nothing to do with Ableton. I am working on something now. Gimme a day to go thru my installation procedure, will revert back with more info and hopefully it will help.

jeremy.j.adamowicz (author)2015-05-05

So im trying to make this using thr teensy ++2.0 but the code seems a bit funny. Is the void setup really only supposed to be that 1 line that is shown?? Thank you

cstreit (author)2015-04-13

I love this idea and decided to see if I could pull it off, but I am not good with coding and just by copying over the code provided, there are a bunch of errors when trying to compile. I am using the teensy ++ 2.0 board so any help would be greatly appreciated.

sab-art (author)cstreit2015-04-13

Can you post the error messages? I'll try to help!

cstreit (author)sab-art2015-04-13

Arduino: 1.6.3 (Windows 8.1), TD: 1.22, Board: "Arduino Uno"

P4.ino:2:1: error: expected unqualified-id before numeric constant

P4.ino:5:1: error: expected unqualified-id before numeric constant

P4.ino:8:1: error: 'from' does not name a type

P4.ino:1:1: error: expected unqualified-id before numeric constant

P4.ino:2:1: error: expected unqualified-id before numeric constant

P4.ino:9:1: error: expected unqualified-id before numeric constant

P4.ino:25:1: error: expected unqualified-id before numeric constant

P4.ino:36:1: error: expected unqualified-id before numeric constant

P4.ino:39:1: error: expected unqualified-id before numeric constant

P4.ino:40:1: error: expected unqualified-id before numeric constant

Error compiling.

This report would have more information with
"Show verbose output during compilation"
enabled in File > Preferences.

coffeestains (author)cstreit2015-04-15

Look at http://www.PJRC.com was installation instructions of the board. In Arduino you have to select Tool>>Board>> Teensy++2.0 Not Arduino Uno.

Also you have to set Tools>USBTYPE>>MIDI.

Additionally there is diff in code provided here and the GITHUM link povided here. Take the code from http://www.github.com/ericrosenbaum/makeymakey-midi/blob/master/makey_teensy_midi.ino

bliedahl (author)2015-04-12

Thanks for the inspiration!! My friend and I are working on our own versions of this project. Plans are for 24" x 24" but I wanted to create a couple of small versions to test before committing to the big one. The photo is of the drawing I did for planning and the video is my first prototype on 9 x 12 canvas board. Here, I have connected it to Makey Makey and my 2nd grade neighbor and fellow crafter is playing it. I have to learn how to reprogram the Makey Makey so I can use all of the pins, but I'm eventually going to use a Teensy board for the big project. I have a lot to learn about Arduino...

sab-art (author)bliedahl2015-04-12

This is awesome!!!!!! I love the center design. Super great work, and I'm glad you were inspired! Thank you so much for sharing!

KristinYankey (author)2015-03-24

Which Teensy board did you use?

sab-art (author)KristinYankey2015-04-12

It's been a long time since we made this and it was a bit of a hack, but I'm pretty sure this code is for the Teensy++ 2.0. You could use a MaKey MaKey and do this the normal MaKey MaKey way with having the user touching ground to get this working the fastest. Other boards also can work, but you will have to change the pin mappings. See some of my other comments below, and let me know what you learn and I'll update the Instructable!

neonix (author)2015-02-25

So awesome!

sab-art (author)neonix2015-03-22

Thank you!

bulta (author)2015-02-25

I have seen incredible things in a nice instructables other functional or curious

but as this no that creating more beautiful in every way and mind that / minds they were able to imagine prodigious

congratulations and thanks for sharing art

sab-art (author)bulta2015-03-22

Thank you!

gabby.reinhart (author)2015-02-27

This would be such a beautiful way to introduce children to creating music both visually and through touch! And even more inspiring if they are part of creation process. It could inspire the love of art & music that could last a lifetime! Great job to the collaborators at sab-art.

sab-art (author)gabby.reinhart2015-03-22

Thank you so much!

bliedahl (author)2015-03-18

Excellent Instructable! We are going to try making one. What version of the Teensy did you use? I just ordered the Teensy ++ 2.0 from SparkFun. Will this work?

sab-art (author)bliedahl2015-03-22

I think this should work fine, and take a look at this: https://www.pjrc.com/teensy/teensyduino.html

It was a while since we made this, so if there are other details missing let me know!

MichiganDave (author)2015-02-22

I am so impressed. I wish I view it in person. So pleased you created this, built it and shared with everyone. A BIG thanks for that!

sab-art (author)MichiganDave2015-02-23

Thank you so much!

bitsy113 (author)2015-02-23

BEAUTIFUL!!!!

milesnorth (author)2015-02-22

Very Cool. Nice and thourough! Thanks.

spylock (author)2015-02-22

Awesome,nuff said.

billie.nenninger (author)2015-02-22

Thanks so much for the idea, I'm not sure my skills are up to this but it looks like fun!

sab-art (author)billie.nenninger2015-02-22

Try a simpler one with a MaKey Makey...instead of doing capacitive sensing, do it the traditional MaKey MaKey way where you have to be connected to ground.

dresch (author)2015-02-22

Say goodbye to doorbells and wreaths that just hang there.

Beautiful... and may I say it: touching.

sab-art (author)dresch2015-02-22

:) :) :)

sally.hastings.71 (author)2015-02-22

I work with elderly frail residents, I can see a market for this wonderful sound painting, I look forward to seeing your work in progress evolve.

that's a really interesting connection and of relevance to my main thread of research...thank you!

joyjoy100 (author)2015-02-22

Absolutely wonderful. Lovely to see talent like this and you are doing something with it. I commend you!!!!! Cheers

sab-art (author)joyjoy1002015-02-22

thank you!

joyjoy100 (author)2015-02-22

Absolutely wonderful. Lovely to see talent like this and you are doing something with it. I commend you!!!!! Cheers

sab-art (author)joyjoy1002015-02-22

thanks so much!

bpahls (author)2015-02-21

do you have a video of it playing music?

sab-art (author)bpahls2015-02-22

the very first image in the instructable is actually a video

GeckoHiker1 (author)2015-02-22

This is a fascinating concept and great instructable. I'm more of a computer geek than an artist, but this is a project I plan to try. I like the 3D appearance of your painting in this instructable, and wondered if you thought of creating something that could sit horizontally, so that the pedestal could hide the electronics and speakers? It could be displayed on a table or as it's own end table. Then you could incorporate some dimensionality into the artwork. And there is no law that says that art must hang on a wall! Great job!

sab-art (author)GeckoHiker12015-02-22

horizontal is a great idea. 3d is an even better idea! why not a sound sculpture? thanks for the kind words!

jtechian (author)2015-02-22

Had an idea you might try. Get some colored foils and cut to shape and apply to the conductive paint that will stick the foil to the art. May add a little more color to the painting rather then just black for the touch parts. With metal foils they will be more conductive etc.

sab-art (author)jtechian2015-02-22

foil is a good idea...I had also considered doing metal inlays but that takes more work!

joyjoy100 (author)2015-02-22

Absolutely wonderful. Lovely to see talent like this and you are doing something with it. I commend you!!!!! Cheers

fenikkusu (author)2015-02-22

My what beautiful ways your minds work!

Stunningly beautiful!

Intellectually playful!

Age defying and transcending!

sab-art (author)fenikkusu2015-02-22

Thank you very much!

The Electrobot (author)2015-02-21

The painting looks great.

Just a thought, how sensitive is your capacitive interface? If it has good sensitivity, you should be able to add a glass/acrylic sheet over the painting. The thicker the sheet the more sensitive your system needs to be....

This way the painting will remain protected and will not be damaged by touching.

sab-art (author)The Electrobot2015-02-21

That's a very good idea! I am not sure ours was sensitive enough to do that, but maybe the next one. Thanks for the compliments!

About This Instructable

41,193views

517favorites

License:

Bio: Sophia Brueckner, born in Detroit, MI, is an artist, designer, and engineer. Inseparable from computers since the age of two, she believes she is a ... More »
More by sab-art:Capacitive Sensor Visualization With ProcessingE-Textile Jumper Wires for SnapsTouch-Sensitive Musical Painting
Add instructable to: