Music Fruits (capacitive Sensors and Arduino Multidrop Network)

About a year ago I was awarded with a scholarship from EU project "Mazowiecki program stypendialny - najlepsza inwestycja w człowieka". After being awarded I had to make a project. I came up with an idea of building a device that imitates a piano (or a Launchpad), using non-music related objects (such as fruits and vegetables). It is, furthermore, my biggest Arduino project so far.

Step 1: Step 1 - How It Works

It is basically an Arduino net with slaves used as capacitive (touch) inputs and a master as USB MIDI interface. The net is based on UART communication witch is one of most common ways to communicate with and between Arduinos. My network allows to attach 128 slaves to one master, with each slave having 12 capacitive inputs. That’s enough for single octave including semitones. User is able to connect up to 128 independent one octave module which can be used with different tones, volumes and also channels. It works like this: touch is recognised by slave, translated as an MIDI information and sent to master, master sends this information through USB to PC and PC plays the right tone in program like FL Studio (which I used).

Requirements

Arduino pro micro, leonardo, teensy 2.0 or any Arduino compatible board with native USB support
at least one Arduino pro mini, nano, uno, or actually any Arduino compatible board
over 1 MOhm resistors (I used 2.2 MOhm) - 12 for each slave
10 kOhm resistor as pullup
1n4148 diode - one for each slave

Capacitive sensor

This project uses 12 inputs for each slave. Quick math proves that pro micro doesn’t have 24 digital I/O pins so we have to use analog pins as digital I/O (for those who are new to Arduino – yes, it is possible) and use two receive pins for one send pin like on pictures in next step . I set capacitive sensor’s resolution to 1 to make it work as fast as possible and prevent it from reading my hand from distance. Also each inputs out value is an average of 10 previously red values just to make it more stable.

Capacitive sensors – more details

Capacitive sensors are widely used in such devices as laptops touchpads, smartphones touchscreens and most of "touch" inputs. Fortunately, they are pretty easy to use with Arduino. The only required things (except the Arduino itself) are some high resistance resistors and capacitive sensors library which you can find over here

It's very simple to use. Just put resistor between a send pin and a receive pin, then attach anything that is conductive (in this case anything that contains some water) to resistor’s receive pin's side. This method is very stable as long as you use one send pin attached to only one receive pin. If you attach more receive pins to one send pin it will still work, but in some cases it becomes unstable. For instance, if you try to touch two inputs at the same time, it can start acting strange. In most cases output values from libraries function starts jumping between minimum value and maximum value (that’s because I used arithmetic mean of 10 previous value – it helps to stabilise it). The more receive pins attached to one send pin are being touched at the same time, the lower maximum values are, the higher jumping frequency is, and the closer to 0 minimum values are. It happens on each input. This makes recognizing if input was touched very difficult, because threshold can be too high and the jumping values can cross it in a random moment. In this case, when the "musician" is keeping the key pressed, the device randomly plays the sound once again. Graphs helps to see what actually happents there (max value in top right corner is maximum value that can be shown. I used it like a zoom)

There are another important things. High valued resistors allow detecting an organism from distance (yes – it works on cats, dogs, etc., as well as on people). The higher resistor value, the longer the distance is and higher library’s function’s values. A higher sensor resolution causes the same thing, but higher resolution is equal to longer read time for each input. Also there are problems when you are grounded or not. Values are different when you sit on chair, stand, when you wear shoes or not. Mostly when you are grounded values are higher and more stable, but not always.

Why using a very complex network instead of just one board? One board with lots I/O pins like Arduino mega will work too slow and it will be difficult to make it work well. One smaller board like pro mini, nano, or even Uno has to few I/O pins. So what about making a network containing many slaves. It allows it to work fast (each slave is independent so read time doesn’t make large different), stable (only two input pins for one send pin) and has as may inputs as user wishes it to have (1536 possible sounds is enough).

Network

Why UART and not I2C, or SPI? The answer is: wire length and their quantity. I2C is amazing. It uses 2 wires, designed for building fast address based network and has very easy to use Arduino library, but I was confused about what is the maximum length between the devices. I wanted to be able to use longer wires and don’t care if I am doing something wrong. Why not SPI? It uses 3 wires. Nothing more, I just wanted to have 2 or less wires. So I decided to make multidrop UART network . Everything about it is fine. Two wires, by lowering baudrate it can be used with hundreds of meters between two devices (in theory). There is one problem, though. UART is not designed for building network so I had to implement it myself and modify a little the communication between boards by adding diodes for each slave and one pullup resistor on their TX line.

The network – more details

First master sends one bit. 7 bits contains slaves address and 1 last bit is information. Last bit informs slave if it is being asked for MIDI data, or is instructed to do something. In first case it ends with one bit from master and opens transfer with selected slave. If slave has any data in its buffer responds with full MIDI data frame until buffer is empty, then sends no data byte and transfer is being closed then master asks next slave. When buffer is empty and slave is being asked for first time if has data it sends no data byte. When last bit is instruction bit slave waits for next bit containing instruction. Next bit informs how many incoming bits will be there (it’s for future usage) and contain instruction code. For example using this master finds attached slaves at beginning of its work sending question to every possible slave and waiting until it respond. Also slaves works much slower than master because checking each capacitive sensor takes some time and it’s like solar eclipse. Moon and earth pass each other, but only sometimes moon is in right position to be in straight line between earth and sun. There is a similar situation here. It's possible that master would ask slave exactly the moment it can respond. Slave is reading inputs (it takes long period of time) and master is trying to ask it for data. For instance slave is reading inputs at this moment so can’t respond to master. Master will not waste time waiting forever for slave so when time after question is longer than threshold it reaches timeout and asks next slave, but, as I said, master waits some time to make it more possible to “meet” with slave. When slave is being asked and over and over again is not responding and reaching timeouts master treats it as detached and no longer asks for data.

One more thing. This is my own interpretation of multiodrop protocols so it’s not any standard. Probably someone did something similar before. I don’t know.

Step 2: Step 2 - Wire It Up and Test

Connect masters UART 1 (in Leonardo and pro micro UART 0 is over USB) to slaves UART using diodes and pullup resistor. Upload code to each Arduino and check. You can also get most of data as text messages in a serial port. Simply modify slave's code to test it with any USB to TTL converter. Uncomment some lines in Config.h file. Some usable options can be also found in other header files. You can use my processing app (screenschots from it are in previous step) to visualise input data as graphs (I know it is horribly written, but it was my first time I wrote anything in Java also in processing). This app requires connecting slave directly to PC and making some changes in slaves Config.h. Uncomment

#define DEBUG_FORCE_QUESTIONS

#define DEBUG_LOCK_MESSAGES

#define DEBUG_LOCK_NO_DATA_MESSAGE

#define DEBUG_PRINT_CAP_READ_VALUE

#define DEBUG_PRINT_AVG

If you are using pro micro and have no USB to TTL converter like FT232RL based one you can use master's board, Arduino Uno (or any Arduino with USB port) with software serial to receive data from tested Arduino and send it over USB to PC (mega, Leonardo, pro micro and some others don't require software serial, because they have two or more UARTs)

Also during tests you don’t have to use pullup resistor and diodes as long as you are testing only one slave.

And yes. Testing is very important step, because you have to check what threshold value is best for resistors you have chosen. Too low high threshold causes random noise and I wrote about it before, too low causes that untouched input is being red as touched.

Step 3: Step 3 - Make It Look Professional

Professional…
Or should I rather say “look acceptable”… So first, I made a prototype PCB for slave for testing (in most cases that’s enough and is easier to deal with than using breadboard after you have chosen the right resistors), then I made myself PCBs for one slave and one master. After PCB I 3d printed case for master (just for better appearance).

Step 4: Step 4 - Alternatives

Presented method is not the only way to have touch input. There are alternatives such as Arduino ADC Touch and it doesn’t require any additional resistors, moreover Teensy 3.0 and 3.1 have a special function to do it. If you want to use more hardware way, you can use special capacitive sensors (fast, accurate, easy to use, but expensive way), or play around with some resistors and transistors (fast and quite cheap). In the internet you can find (more or less) information for each of these methods.

Alternative for master is to use Arduino uno, or mega (only R3 version with atmega 16u2 not FT232, or CH340 chip as USB converter). You can change software on this tiny atmega 16u2 to HIDUINO firmware, but it also requires some modifications in MidiSending.cpp. Fortunately only a three, maybe four lines. Instructable (not mine, but it is good piece of work) how to change the firmware to HIDUINO can be found here. If you want to use Uno you have to keep in mind that you have to use software serial for multidrop.