First and foremost, I must acknowledge that I am standing on the shoulders of giants, and that every giant is standing on the shoulders of giants (such as all contributors to instructables). If it weren't for the unknowably many people who had the mindfulness to freely share information, this would have been utterly impossible; I reckon the same if even a handful of these people chose otherwise. So, if you like this, don't just thank me, thank a history of humans and the tremendous power of the freedom of information.

Are you bored of sensing things in the same way they've been sensed for the history of humanity? Have the known forms of interacting with computers lost their flair? Is an insufficient auditory neurosystem bogging you down? Then I think I have the project for you. Why not give your ears a rest, and let your fingers do the listening for a change!

Let's prepare ourselves with some background information. Human hearing relies on a bunch of sensors in a structure called the cochlea, which, by way of a very remarkable structure, converts variations of air pressure into neural impulses. In mathematical terms, the cochlea decomposes a waveform into a finite number of dimensions, about 3600 per ear. Have you ever wondered what the sampling frequency of the ear is? It's not a very well-formed question, so to give a not very well-formed answer: from 50 to 250 Hz. Another interesting fact is that the high limit of reasonably good hearing is 20,000 Hz. Together, this means that a sensor at maximum speed is only detecting 1 out of every 80 wavelengths. This is peculiar. It would be like looking at a screen and only being able to see 1 circle every second when 80 circles are shown, and being able to tell anyway that there are actually 80 circles. How can we experience something faster than that which defines our experience? And how can we experience it as continuous when the signal keeps cutting out? With mathemagic. The cochlea's very special coiled construction is a physical, mechanical implementation of a process called the wavelet transformation, which really might as well be magic; conceptually, it's like using part of a torn picture to reconstruct the lost part.

Another key piece of this project is the tactile sensory system. It is an interesting fact that touch sensors also operate in the range of 50-250 Hz. Actually, throughout the body neurons behave in the exact same way. You can move a whole chunk of brain to a completely different spot and have it function as the piece it replaced--scientists have actually done this! As such, it is reasonable to suspect that touch receptors can convey the same information as the cochlea and in the same way, just to different places. Furthermore, some research has indicated that touch does activate some of the auditory processing part of the brain, and one of the best known properties of the brain is that it is extraordinarily adaptable. Thus, we have reason to suspect that you might actually be able to hear through touch! That is, to not just feel the vibration of sounds, like touching a speaker, but to feel sound .

With the idea in place, the rest is simple, right? All we need to do is build a vibrotactile human/computer interface and write some code to send wavelets of sound to the fingertips. The brain will never see it coming! Clearly not, because we're not working with sight (yet); the real question is, will the brain feel it coming, or will the brain hear it coming?

Step 1: Building Tactors

Plenty of research has been done into tactile interfaces. Because of this, there is a fancy scientific name for the critical components here: tactors, which I suspect is a portmanteau of tactile and motor. You could also call them solenoids, linear actuators, moving magnet voice coils, vibrotactile transducers, etc. The specific requirements of their construction requires that they be built by hand, exciting!

You will need:
tiny powerful cylindrical magnets, I used 1/8" diameter with 1/8" height to keep the scale workable, and added a 1/8"x1/16" for more height/mass/magnetic force and to mark the pole
thin walled non-ferrous tubing with an inner diameter that matches the outer diameter of the magnets (ie K&S stock #103, can be found at hardware or hobby/craft stores)
plastic washers that can be press fit onto the tubing
dowel that can be put into the tubing (paper, cut down toothpicks, erasers, etc could also work)
a flexible membrane (latex glove, balloon, etc.)
small rubber bands or heatshrink tubing
electrical tape
some motor winding wire, I used 34 gauge
a lighter and sandpaper
a long bolt with nut to pass through the cut down tubing
a few washers to pass over the tubing
an electric drill with continuously variable speed (most of them)

We are building a bunch of small solenoids that will push the magnets against the skin with just the right amount of force. This will require building a spool, winding that spool, inserting the magnets, and covering the assembly so the magnet can't escape. Note that you can get away with a fair degree of ingenuity here; I made quite a few prototypes leading up to this using things I had laying about.

1. Cut the tubing: Ideally this is done with a tube cutter, because the tube needs to stay round enough that the magnet can move through it. I made the tubes a bit longer than necessary, 3/4", to make them easy to mount later on, just by drilling holes in some wood and pressing in the spare end. Round and ream the tube as needed, a good way to do this is with a drill bit that matches the inner diameter of the tube.

2. Press on the plastic washers: Unless you have an amazing supply at hand, you will probably have to find some with a smaller inner diameter and drill those out. The only way I know how to do this is with some pliers and a hand drill--please, find a better way (and let me know!) or be very careful. We need a strong fit because these will support the coil, and later on, fingers. The outer diameter of the washers and the space between them is going to determine how much wire you can wind and how much power will be needed. You want to keep the power requirements low but still be able to move the magnet well enough; I determined this experimentally, and came up with between 2.5 and 3 spans across a half an inch.

3. Conjure a winding clamp. Use washers to fill the extra tube and to grip the end of the winding wire. Clamp the bobbin with the nut and bolt, there needs to be enough force to stop the tube from rotating while getting a good, tight coil. Be sure to leave enough spare to reach the terminating points later.

4. Wind the winding wire. Clamp the tail or nut of the bolt in the drill and start winding. The wire must be tight and even to achieve the optimum magnetic field. This is probably the most challenging part of the build, so be patient and take your time. Each solenoid should have about the same number of windings in order to make them all produce a similar force; I did this by placing the plastic washers the same distance apart, then spanning that distance the same number of times (2.5 to 3) since that's much easier to keep track of. You can mark the leads and wrap according to the right hand rule, or just change the polarities around later on. This would be a good time to test the continuity and resistance of the solenoid; most of my solenoids came out with a resistance of 2 ohms, which was from around 2.5 spans.

5. When you're done winding, you'll want to secure all that hard work from coming uncoiled. I like to hold it with a little bit of electrical tape first, then put heatshrink over that. Be completely certain that you've removed the insulation from the ends of the wire by burning it off and then taking off what remains with sandpaper.

6. Finishing touches. Put a 1/2" long piece of dowel in the tube so that the magnet rests above the halfway point of the coil and so that it can only go one way, then add the membrane to cover the top so the magnet can only go so far in that direction. Latex gloves will work, but they tend to break down quickly, so a cut up balloon will probably do better. This can be held on with anything non-ferrous , solid aluminum or copper wire works well. Typically the magnet would be secured to the membrane, but there was no obvious way of doing that on this scale. You can test them by rubbing the leads on a battery; these are fundamentally similar to small speakers, so with the right polarity you will hear the scraping of the leads just as you would testing any other speaker.

7. Mount them. I really want this to be attached to a glove or something neat like that, but for a proof of concept a piece of wood will do. I traced my fingers and drilled holes to fit the spare end of the tube where my fingertips would go.

Step 2: Building the Circuit

First, a note of caution: this is pretty much the first circuit I've ever put together myself. Exactly how close to correct it is I cannot say, just that it is close enough! To my knowledge, nothing has yet been damaged--all magic smoke remains tentatively contained. Any comments regarding potential problems, improvements, etc. are absolutely encouraged and appreciated.

With such an exciting idea, it would be silly to let a little unfamiliar territory get in the way, right? That's what I thought too. Actually, with all the amazing work that has been made openly available and easily obtainable, this proved an excellent first circuit.

Here's what you will need:
Arduino (shocking, I know)
power supply (I used a bunch of 9 volt batteries)
breadboard or protoshield (or dead-bug, but is that really an option?)
0.47 ohm 5 watt resistor
wires
toggle switch
LED with appropriate resistor
for each tactor:
2N 2222 A transistor
1N4001 diode
#4x3/8 screws and washers
ring terminals

Here's the big picture: this handful of tactors requires a surprising amount of juice. Using Ohm's law we can figure that at 9 volts and 20 Ohms (all tactors on @ 2 Ohms each), there will be about 500 mA of current, which happens to be about what we need for a distinct vibration that won't cause numbness. These figures are well beyond what the Arduino can handle, so we need an auxiliary power supply. We also need a way for the Arduino to switch the tactors on and off, which is what a transistor does best. Electricity flowing through a coil generates a magnetic field, and a magnetic field passing through a coil generates electricity; these are intensely useful facts, but generally we only want one of these from a magnet/coil assembly, hence we use diodes to protect the electronics from electromagnetism going in the wrong direction. The last piece of the puzzle is the giant resistor, which is used to make the tactors have the same amount of current and thus force whether one is on or all of them are.

If you don't know how to build this circuit, the original size image of the breadboard can be used for reference. Note that because we are using a second power supply, the circuit will need to share a common ground with the Arduino.

Because of the amount of current each tactor needs, it is important to have excellent contact with the winding wire, which is difficult because of its tiny area and fragility. One way to go about this is to compress it between two washers; a washer usually has a rounded side and a flat side with a rough edge. Put the wire between two smooth sides to reduce the chance of damaging it. It is preferrable to use ring terminals for the wires that will join with the circuit, because they will help distribute the pressure evenly. Alternatively, you can try to use 3 washers and just pinch the wire between two that aren't holding the winding wire.

We now have a vibrotactile HCI, ready to vibrostimulate its heart out (which, with 3 9v batteries, doesn't take long). This device could also be used to convey distance and color to the blind, simulate texture, and a whole bunch of other stuff that nobody (myself included) has yet thought of, but without software it will remain a really technical doorstop. Next step!

Step 3: Building the Software

We're on the home stretch. All we need to do is take some sound data, perform a wavelet decomposition, and use the results to tell the Arduino when to flip each switch. This sounds like a monumental task, but once again, high praise to those who have preceded us, and for the miracle of open source and object oriented programming. With a little luck we will be able to bang some bits together to get what we need. Let's start with some of the interesting theoretical stuff.

Everybody knows that a musical note is defined by a specific wavelength, but have you ever wondered why the same note sounds different on different instruments? The reason is that within that overall wave there are a bunch of other tiny waves, too small to overwhelm the big wave but enough to change the specific nature of the wave. Thus, we hear the big wave and we say "middle C," and we hear the smaller waves in that wave and say "piano." The process of sifting out the different waves is an example of a decomposition, and it is what the cochlea does, which means it's what we need to reproduce. As mentioned previously, the cochlea is performing what is called the wavelet transformation, which was actually discovered in trying to figure out what the cochlea was doing; you could call this an example of evolution beating mathematicians to a profoundly useful concept! Now, despite being essentially done with an undergraduate degree in math, I'm barely able to understand the majority of wavelet transformations on my own, so fortunately for this project there's a mysterious black box, the jwave library, which can be abused for our purpose. We will be taking some raw sound data, sending it to jwave, (some magic happens), then receiving some data which looks nothing like sound. We will then turn that data into frequencies that match the operational frequencies of the two kinds of touch receptors (50 to 250 Hz), and finally, turn those frequencies into binary signals to be sent to the Arduino, which simply switches whatever tactor it is told to. Conveniently, the substantial disparity between neural frequency and even a low USB frequency means that we will be able to signal each tactor one bit at a time with plenty of room to spare: for USB at 9600 Mhz, we would be able to signal roughly 38 million tactors.

If you don't feel up to stitching together the code yourself, the source, an executable jar, and the openSCAD code/STL files for the 3D objects can be found over on thingiverse here . If you just want a program ready to run, download the executable, put the sketch on your Arduino, and be sure to read the README. Also, be warned this is far from production quality code.

Step 4: Completion

With all the pieces in place, there's just one last step: load up some music and let her rip!

The first question people ask is probably the one you have in mind right now, "does it work?" There are a few ways to answer this.
Answer 1: Build it and find out!
Answer 2: Yes.
Answer 3: Yes, in a sense. First, remember that each cochlea uses about 3600 neurons to convey sound. Comparing them directly, 8/7200 is about 0.11%, which is a really low rank approximation. I made some attempts to improve on this through my particular use of the wavelet transformation, but without significantly more tactors there's no way this will come close to healthy hearing. However, early in thinking about this project I decided that a good criteria for success would be the ability to distinguish music from speech with this device; so far, all but one person has been able to do that. Another person, a musician, was able to pick out the higher structures of music. Actually listening to music in addition to feeling it is a very interesting experience, like a tactile version of a visualizer, and any doubts disappear; it rapidly becomes clear that there is a direct correspondence between the sound and what is happening at your fingertips.

Suppose you had no hearing, would .11% more ability to hear be useful? I'm going to say yes, and probably more than just .11% more useful. One of my greatest motivations for this project was the possibility of enabling somebody without hearing to experience music. To be certain, there's a lot of technology out there that holds a lot more promise to restore hearing in a conventional sense. I think this is fantastic news, and takes nothing away from me or this device.

Another significant motivator was the chance to do something that nobody has ever done before, which is to say something extraordinarily creative, something to challenge and extend my skillset. I'm still not sure if anybody else has done this, but I can say with absolute certainty that this pushed and tested my abilities; I deeply hope that this is only the beginning of the innovations that come from my life, but I must admit I'm so impressed by it that I have a hard time seeing how I could top this. Naturally, it is difficult to say what I would do with a laser cutter (besides engrave everything in sight). What I can say, is that like all tools, it would extend my ability to transform imagination into reality, even extend my ability to imagine, and I'm interested to see what could come of that... aren't you? Actually, the same is true of all the other prizes too, if to lesser degrees.