Draw It Yourself - MIDI Controller With Conductive Ink

'Draw It Yourself' is a MIDI controller which uses conductive ink as push-buttons. It is based on Arduino, and uses a capacitive sensor (two, actually) to determine whether the drawn buttons are being touched or not.

http://sanidanz.tumblr.com/drawityourself

The main advantages of this controller are:

-You can create your own templates just drawing the buttons in a paper or any other insulating material, using conductive ink.

-You can have different MIDI controllers, drawing a different one on each paper.

-Incredibly cheap, comparing to the hardware MIDI controllers that can be replaced using Draw It Yourself.

-Easy to carry: you just need the case and all the templates you want in a folder.

-Useful for children: they can draw pictures and make them sound.

However, there are also some drawbacks:

-Buttons are not pressure-sensitive.

-Knobs or sliders cannot be drawn: they should be discrete (up/down or several levels).

-No visual feedback (no LEDs on the paper).

This was my second semester project for the Interactive Music Systems Design Course (CDSIM) that I am doing at the Music Technology Group (MTG) at University Pompeu Fabra of Barcelona. I presented this project at Sonar+D, part of the Sonar festival of Barcelona, held between June 12th and 14th 2014.

On the whole, if you want to be able to build your own MIDI controller templates in a very cheap way, and learn about Arduino, MIDI and capacitive sensing, you should read this project. Feel free to comment with any doubt or suggestion.

Step 1: Materials

MAIN COMPONENTS:

1) Arduino Uno: http://store.arduino.cc/index.php?main_page=produc...

2) MPR121 capacitive sensor (quantity: 2): https://www.sparkfun.com/products/9695

3) Conductive Ink: http://www.bareconductive.com/shop/electric-paint-... or any other.

4) Paper, cardboard, ... anything to draw your controller (must be insulating!!)

OPTIONAL COMPONENTS:

4) LCD screen: https://www.sparkfun.com/products/709

5) Push button (quantity: 2): https://www.sparkfun.com/products/97

6) Rotatory Encoder: https://www.sparkfun.com/products/9117

7) Resistors, wire, protoboard, soldering tools.... everything to build the electronic circuit (see schematic!!)

8) Tin foil (to make the contact between the paper and the circuit)

9) foam and rubber strips (to make pressure and ensure contact)

10) case (handbuilt, 3D-printed....)

Step 2: General Description

In this Instructable I am not going to describe in detail all the steps that I followed to build this controller (if I did, it would be unchallenging and boring for you, wouldn't it?): building it was a long process of mistakes, problems and corrections. Instead, I am going to explain the general guidelines of how you can build a similar controller, or use this knowledge for any other application related with Arduino, Midi or Capacitive Sensing.

So this is, in a nutshell, how Draw It Yourself works: The push-buttons are connected with conductive ink to the case, where the Arduino and the sensors are. The case has 20 small metallic sheets, where each of the buttons are connected. The capacitive sensor is managed by an Arduino, and it charges and discharges every pin continuously to measure its capacity and check if there is a human body touching it. In the box you can see a LCD screen, an encoder and 2 buttons: I use one of the buttons to reset Arduino when I want to change the template. The other button and the encoder are used to select the template that I am going to use. I can select the template I am on at the LCD.

There is one important point: each pin of the different templates needs to be adjusted for touch detection correctly. What I do is save the touch/release threshold values on the Arduino EEPROM memory, which remains the same even when the Arduino is disconnected. If it is the first time I use a template, I do the sensor adjustments with the button and the encoder. Finally, the Arduino sends MIDI messages to the computer when a button is touched/released. This allows me to use the controller on every program that manages MIDI and create my own mappings.

Step 3: The Conductive Ink and the Templates

The ink is probably the worst part of the project: although nowadays there are more and more projects that use conductive ink, it is still hard to find and expensive. As you can see in the 'Materials' Step, I bought it to BareConductive. Their ink works fine, but it is quite sticky and dirty, which makes it difficult to use as a normal pen to draw. However, the electricity conduction is excellent.

My first idea was to be able to design my templates in a computer CAD software and print them using the conductive ink instead of common printer ink, but I found out it was impossible (at least with the BareConductive ink: it is not thin enough). I found a company in Japan that has developed a printer and a conductive ink for it, and they use it to design printed circuits on paper. However their product is very expensive, at least for me.

So I had to look for another solution. What I did is to design templates using SolidWorks 2D, then print them with a laser-cutter. I put the cut paper over another paper and spread the ink over it, so that the buttons and connections will remain on the paper below. Then, I glued another paper over it, which just had the holes of the buttons. This makes it more stylish, and protects the ink-wires. You can see the pictures that are attached to have a better idea the result of what I just explained.

The 4 controller templates that I designed and showed at the Sonar festival were (you can download their CAD designs):

-MPC-Style: similar to a classic MPC sampler or a MASCHINE controller. It allows me to launch different sounds recorded in each pad, and have different presets.

-Keyboard: Just a normal piano keyboard of one octave, which includes an octave selector and a pitch bend. I use it for any software instrument.

-Multi-Track: I used it to mix a Ableton Live project I created with 4 tracks: I can play loops and control FX and volume.

-Step Sequencer: Typical step sequencer layout, to make beats.

Warning: My templates are prepared to be connected to my own case. If you want to design your own case, bear in mind that the paper contact points must match the case's.

The way I designed the contact between the metallic pins inside the case and the paper can be seen on the image. I spent a lot of time improving this, and finally this is how I made the contact: I soldered the small metallic contacts (each one to one sensor electrode) with wire. The contacts were made cutting small pieces of tin foil All the pieces were then glued to foam and put inside the case. The paper (each template) was then introduced sideways to the part of the case where the metallic contacts were. Then I pressed the upper and lower parts of the case with rubber strip, and the foam inside ensured the connections of each metallic sheet to one point in the paper. I don't think this is the best way to make the connections, but I couldn't use a lot of tools to design it better, so this is the best I could do. However, I recommend that you think of something more efficient if you are thinking of building something similar.

Step 4: Arduino

This is a short explanation about the components the Arduino uses and communicates with. You can also find a picture of the schematic of the whole circuit. The schematic file is also attached, it is a Fritzing file. I used the Fritzing software only to draw the schematic, and then I soldered it by hand. You can open it to see the schematic with more detail.

The main controller that I use to manage the sensors and send MIDI to my computer is Arduino. Probably, if you are reading this, you will have some knowledge about it. If not, you should get familiar with it on your own, as some basic knowledge is needed to follow this step. You can learn everything about Arduino on their website: it is full of tutorials and different projects made by users: http://arduino.cc/

The way the Arduino communicates with the MPR121 capacitive sensor is the I2C communication. It is a 2-line serial communication bus: one data line and one clock line. Arduino already has a library to make this kind of bus easy to use. It is called the Wire library. Here you will find all the commands you need to use this kind of communication: http://arduino.cc/en/pmwiki.php?n=Reference/Wire. The way the information is shared between the Arduino and the sensor will be explained in the next step.

I use an LCD, an encoder and buttons to control the Arduino. The buttons and the encoder are easy to use, you should be able to easily find some tutorials about it on the arduino website, if you still do not know how to do it. The way to connect an LCD screen to Arduino, and how to visualise text using the Liquid Crystal library, is very straightforward and is explained here: http://arduino.cc/en/pmwiki.php?n=Tutorial/LiquidCrystal

As I explained previously, I use the arduino EEPROM memory to save the touch/release threshold values of every pin of each different template. And again, Arduino has a specific library for its EEPROM memory, which makes it very easy to use. Just follow the instructions here to learn how to write values on the EEPROM and read them later:

http://arduino.cc/en/pmwiki.php?n=Reference/EEPROM

And finally, the most important step: how to use Arduino as a MIDI controller. Arduino has an specific library for MIDI too: http://playground.arduino.cc/Main/MIDILibrary

You can include MIDI messages in an Arduino sketch and upload it, but the problem is that the programs that use MIDI (such as Ableton Live or any other DAW) will not recognise your Arduino, so you would have to use another program as a bridge between the Arduino and the host program.

There is a solution to this: HIDUINO. Don't ask me about how it works, but , as far as I know, basically a new firmware is loaded into the Atmega 16u2 chip that Arduino uses. This makes that Arduino is not an Arduino anymore: it becomes an USB-MIDI controller, and therefore is recognised as any other MIDI controller by your host software. The disadvantage of this is that you cannot load new sketched while the HIDUINO firmware is loaded. More info here:

https://www.instructables.com/id/Turn-your-Arduino-Uno-into-an-USB-HID-Mididevice/

If you do not have a ISP programmer, you should learn how to turn your Arduino into a MIDI device here:

https://github.com/ddiakopoulos/hiduino/wiki/HIDUINO-via-DFU-Method

Step 5: The MPR121 Capacitive Sensor

https://www.sparkfun.com/products/9695

The Capacitive Sensor I used for this project is the MPR121. The MPR121 is a capacitive touch sensor controller driven by an I2C interface (in this case the master controller is an Arduino). The chip can control up to twelve individual electrodes. As in my project I was using 20 pins (for 20 buttons), I used 2 sensors: I used the 12 electrodes of the 1st and 8 electrodes of the second. There is a very helpful hookup guide on the sparkfun website, which will make it easier to understand how to use this sensor. I recommend to download the code they use and modify it four your needs. If you want to make more advanced stuff than what is explained in this guide, I recommend to take a look at its datasheet to know how to make it communicate with the Arduino via I2C.

Hookup guide:

https://learn.sparkfun.com/tutorials/mpr121-hookup...

Datasheet:

https://www.sparkfun.com/datasheets/Components/MPR...

If you looked at the Fritzing file schematic that I attached, you will find out that lots of the connections of the schematic (pull-up resistors, capacitors....) are already soldered when you buy the MPR121 sensor at Sparkfun.

Next, I will remark some important notes on how to setup the sensor for this project:

1) Vdd:

Be careful!! the MPR121 works with Vdd = 3.3V. You will burn it if you connect it to the 5V output of Arduino.

2) The Setup:

In the hookup guide, you will find this:

void mpr121_setup(void)
{
	set_register(0x5A, ELE_CFG, 0x00); 
	set_register(0x5A,....
	.........

What it does is to set up the MPR121 sensor correctly: select the number of electrodes used, choose the charge/discharge current and time for each sensor, set the touch/treshold release for each electrode... You do not need to know exactly what it is about: you can just copy it and change some of the lines for your project. You just need to call this function on your Arduino Setup, and do it again everytime you want to change the sensor setup (in my case, when I adjust the buttons or change the templates, for example). I will explain some aspects about the sensor setup in the next points.

The 'set_register' function basically makes the I2C communication between the sensor and the Arduino. It is defined like this:

void set_register(int address, unsigned char r, unsigned char v)
{
	Wire.beginTransmission(address); 
	Wire.write(r); 
	Wire.write(v);  
	Wire.endTransmission(); 
}

Two of the most important registers are the AFE Configuration registers 1 and 2 (registers 0x5C and 0x5D). See page 13 of the datasheet for more detail. Changing the values of this registers will define the charge/discharge current and time for the electrodes, the filtering iterations, and so on. I recommend playing a little bit with the CDT and CDC values so that you can find suitable ones for correct touch/release detection in your project (maybe they will not need to be modified, but maybe they will).

3) The address of each sensor:

There is a problem when you want to use two or more MPR121 sensors at a time, because all of them have the same default address for the I2C communication: it is the 0x5A address. If you look at the datasheet, it says that 4 different addresses can be chosen, depending on where you solder the address pin: Vdd, Vss, SDA or SCL. By default, the board comes with the address pin soldered to Vss (ground), so if you want to use more than one sensor, you have to cut this connection: next to the address pin you will find two small metallic squares that are connected with a very thin line: this line is the connection between the address pin and ground. You should cut this small line of one of the sensors with a cutter, and solder the address pin to Vdd, for example, so that the new address of this sensor is not 0x5A anymore: instead, it will be 0x5B.

This is exactly what I did for this project, so I had two sensors: the first one's address was 0x5A and the second one's was 0x5B.

4) Touch/Release Treshold setup:

The touch and release threshold setup is done inside the MPR121 setup. if you follow the hookup guide and look at the mpr121.h library, the default values are 0xA0 for touch_threshold and 0xB0 for release threshold. One possibility is to change the value of the touch and release threshold for each pin, using a byte variable for it. For example, for one sensor:

<p>byte touch_treshold_1st_sensor[12] = { 0xA0, 0X94, 0xB2....  //random values for example<br>byte release_treshold_1st_sensor[12] = { 0xB0,0xC3, 0xA8....
......
.....
void mpr121_1st_sensor_setup(void)
{
	.......
  set_register(0x5A, ELE0_T, touch_treshold_1st_sensor[0]);
  set_register(0x5A, ELE0_R, release_treshold_1st_sensor[0]);
  set_register(0x5A, ELE1_T, touch_treshold_1st_sensor[1]);
  set_register(0x5A, ELE1_R, release_treshold_1st_sensor[1]);
          ........</p>

I find this the simplest way to define different touch/release threshold values for each template: you can save the values on the Arduino EEPROM and just load them into the sensor, depending on the template you are using.

However, I was having problems with this: some of the buttons I drew on some templates were out of range, so I could not make a correct setup. So instead of changing the treshold values, I was changing the Charge/Discharge current for each individual pin of the sensor. You can learn how to do this on page 14 of the datasheet. This worked better for me, becouse the range of touch/release detection for each electrode was greater. This is an example of how I did it:

<p>// 20 byte array: 12 values for 1st sensor and 8 for 2nd sensor. Initial value: 0x28<br>byte CDC_sensor[20] = {0x28, 0x28, 0x28, 0x28, 0x28, 0x28, 0x28, 0x28, 0x28, 0x28, 0x28, 0x28, 0x28, 0x28, 0x28, 0x28, 0x28, 0x28, 0x28, 0x28};</p><p>.......</p><p>//Then I used the encoder to select the CDC_sensor[i] value for each pin</p><p>for (int i=0; i<20; i++)</p><p>{</p><p>	//encoder stuff</p><p>	if... //if encoder position is increased</p><p>	{CDC_sensor[i] ++;}</p><p>	else...//if encoder position is decreased</p><p>	{CDC_sensor[i] --;}</p><p>}</p><p>

//finally, everytime a CDC_sensor value is changed, you need to make the setup again:</p><p>void mpr121_1st_sensor_setup(void)<br>{
	.......
</p><p>      set_register(0x5A, 0x5F, CDC_sensor[0]);   //el 0<br>      set_register(0x5A, 0x60, CDC_sensor[1]);   //el 1
      set_register(0x5A, 0x61, CDC_sensor[2]);   //el 2</p><p>      ........</p>

You should try to use the way that is most suitable for you: changing the electrode threshold values or the individual charge current values.

5) Arduino interrupt:

The MPR121 sensor has an IRQ pin, which has to be set with a pull-up resistor, and goes to LOW everytime the sensor detects a touch or a release in one of its electrodes. So I recommend to solder this pins to the Arduino interrupt pins, which are digital 2 and 3. This way you can use the Arduino Attachinterrupt function:

http://arduino.cc/en/Reference/AttachInterrupt

But be careful!! not many code stuff must be done inside the interrupt: for example, the serial port does not work (therefore, the MIDI messages do not work either, becouse the serial port is used for their transmission). I only used the interrupts to change the value of a boolean variable, which determined whether one pin was being touched or not. Then, I did the corresponding changes in the general loop program. See the code:

//interrupt variables
static boolean touch_1st = false;
static boolean touch_2nd = false;

//in the setup:
//interrupts
attachInterrupt(0, touching_1st_sensor, FALLING);
attachInterrupt(1, touching_2nd_sensor, FALLING);

//outside the loop
void touching_1st_sensor() 
{
  touch_1st = true;
}
void touching_2nd_sensor()
{
  touch_2nd = true;
}

//in the loop
if(touch_1st)
{
  //check for electrodes...
  touch_1st = false;
}
if(touch_2nd)
{
  //check for electrodes...
  touch_2nd = false;
}<br>	

With all of this, I think you have enough information to do some beginner and medium level stuff with the MPR121 sensor.

Step 6: Sending the MIDI Messages (code Example)

Now that you know how to set everything up, I will write a simple code to show what I did to send the MIDI messages to the computer when a button was touched or released. My code is based on the Sparkfun hookup guide code that I recommended in the previous step. It just has some small changes:

-All the setup changes that I explained in the previous step

-I use Arduino iterrupts. Also explained in the previous step

So here goes the code:

<p>if (touch_1st == true) { //chech if interrupt was generated<br>    </p><p>//read data from the sensor
    Wire.requestFrom(0x5A,2); 
    
    byte LSB = Wire.read();
    byte MSB = Wire.read();
    
    uint16_t touched_1st = ((MSB << 8) | LSB);</p><p>// Check which electrodes were touched/released<br>        for (int i=0; i < 12; i++){  
          if (touched_1st & (1<<i)) {</p><p><i>            if(electrode_value[i] == 0){
              //send midi
              MIDI.sendNoteOn(midinote[i],100,1);        //midinote[12] array was previously defined
            }      
           </i>electrode_value[i] = 1;      </p><p><i>          }
          else{
            if(</i>electrode_value[i] == 1){</p><p><i>              //send midi
              MIDI.sendNoteOff(midinote[i],0,1);
           }       
           </i>electrode_value[i] = 0;</p><p><i>          }</i></p>

I hope this example gave you an idea of how to use this sensor to send MIDI messages to a computer or any other device. For any questions or suggestions, please contact me or write a comment.