Instructables

Earfingers: hear with your hands

Featured
First and foremost, I must acknowledge that I am standing on the shoulders of giants, and that every giant is standing on the shoulders of giants (such as all contributors to instructables). If it weren't for the unknowably many people who had the mindfulness to freely share information, this would have been utterly impossible; I reckon the same if even a handful of these people chose otherwise. So, if you like this, don't just thank me, thank a history of humans and the tremendous power of the freedom of information.

Are you bored of sensing things in the same way they've been sensed for the history of humanity? Have the known forms of interacting with computers lost their flair? Is an insufficient auditory neurosystem bogging you down? Then I think I have the project for you. Why not give your ears a rest, and let your fingers do the listening for a change!

Let's prepare ourselves with some background information. Human hearing relies on a bunch of sensors in a structure called the cochlea, which, by way of a very remarkable structure, converts variations of air pressure into neural impulses. In mathematical terms, the cochlea decomposes a waveform into a finite number of dimensions, about 3600 per ear. Have you ever wondered what the sampling frequency of the ear is? It's not a very well-formed question, so to give a not very well-formed answer: from 50 to 250 Hz. Another interesting fact is that the high limit of reasonably good hearing is 20,000 Hz. Together, this means that a sensor at maximum speed is only detecting 1 out of every 80 wavelengths. This is peculiar. It would be like looking at a screen and only being able to see 1 circle every second when 80 circles are shown, and being able to tell anyway that there are actually 80 circles. How can we experience something faster than that which defines our experience? And how can we experience it as continuous when the signal keeps cutting out? With mathemagic. The cochlea's very special coiled construction is a physical, mechanical implementation of a process called the wavelet transformation, which really might as well be magic; conceptually, it's like using part of a torn picture to reconstruct the lost part.

Another key piece of this project is the tactile sensory system. It is an interesting fact that touch sensors also operate in the range of 50-250 Hz. Actually, throughout the body neurons behave in the exact same way. You can move a whole chunk of brain to a completely different spot and have it function as the piece it replaced--scientists have actually done this! As such, it is reasonable to suspect that touch receptors can convey the same information as the cochlea and in the same way, just to different places. Furthermore, some research has indicated that touch does activate some of the auditory processing part of the brain, and one of the best known properties of the brain is that it is extraordinarily adaptable. Thus, we have reason to suspect that you might actually be able to hear through touch! That is, to not just feel the vibration of sounds, like touching a speaker, but to feel sound .

With the idea in place, the rest is simple, right? All we need to do is build a vibrotactile human/computer interface and write some code to send wavelets of sound to the fingertips. The brain will never see it coming! Clearly not, because we're not working with sight (yet); the real question is, will the brain feel it coming, or will the brain hear it coming?
 
Remove these adsRemove these ads by Signing Up

This looks awesome. I think you mention at the beginning that you got ideas from other instructables (or at least you've looked at others) - could you provide some links? Maybe something simpler that an amateur could start with before working up to your instructable? Thanks!

dreamer.redeemer (author)  melvinsghost29 days ago

Thanks! I was speaking in general, there actually wasn't anything specific I was building off of. To start somewhere simpler, I suggest looking into winding your own electromagnet and any basic Arduino projects. Since the code and circuit is provided, that pretty much covers it; despite how complicated it seems it's fairly simple to build.

qualia2 years ago
even if its not original, its an impressive re-realisation of a concept, you should give yourself more credit (without going O.T.T of course, your humble personality is pleasant, thanks to the appeal to the general arrogance of myself and the majority of the human race) about the washers, a better way is to start with a piece of flat material the thickness you want, and a hole saw with the inner diameter of the washer size drill or flycutter with blades spaced for the inner and outer diameter. for a cheap flycutter you can use a bit of steel or aluminium silver-soldered or welded or turned as a single piece to a shaft that can fit in the chuck of your drill(-press preferably) and sharpen the shanks of dead drill bits into blades and bolt another piece of thinner steel with a sliver of wood in between to the first bit so that you can adjust their position and then lock them into place against the wood (or lead or copper or other semi-hard material)
chansen92 years ago
Hi :) interested in exploring this myself but probably because im a total noob at Java I don't get how the code in the files was used by java to create the wavelet decomposition. Can you possibly put more detail in that section? Totally think the idea is amazing :) great work
dreamer.redeemer (author)  chansen92 years ago
Thanks for the compliments. I can go into more detail, but specifically what detail depends on what you know/how much work you want to do. If you've never done OOP, or if JDK, JRE, IDE, and SDK are mostly meaningless to you, then it would probably be best for me to upload a version of the program that you can just double click to run (if I can get such a thing to work). Otherwise, acknowledging that my code is a mess, the decomposition/transformation is performed by passing chunks of the appropriately structured data through iterative method calls to a Transform object from the jwave library, in essence:
Transform t = new Transform(new AncientEgyptianDecomposition(new FastWaveletTransform(new Daub02( ) ) ) );
decompositions = new double[nSamples][nDivisions];
for(int i=0; i<nSamples-1; i++){
   for(int j=0; j<nDivisions; j++){ samples[j]=scaledAudio[marker++]; }
   decompositions[i]= t.forward( samples ); //the transformations happen here
}
OK that does make more sense now. I should get my friend who programs Java to help :P playing around with the idea of increasing the number of actuators. Which would mean modifying the code. Way out of my league, shouldn't be a problem for my friend though. Have you thought about having more than 10 actuators?
Thanks for your help :)
dreamer.redeemer (author)  chansen92 years ago
I've definitely thought about having more tactors, the biggest hangups for me were multiplexing the signals and the increased power requirements. If I were to do more, I would definitely try to make tactors with smaller diameter magnets and seek to reduce the noise they generate, normalize magnet travel (as in being attached to the membrane), etc. I would also revisit the mounting, because having 8 mounted to a board is awkward enough; actually one of the better ideas I had in this regard was building them into a ball. Unfortunately, the code would need to be modified a little more extensively than changing the right numbers here and there, but hopefully your friend can figure it out based on some of the comments I left in the code. One trick I did in hopes of getting better results was asking for a higher dimensionality and only using parts of it--otherwise, most of the signal would be sent to one tactor, which would always be active, and all the others would just have the occasional blip. Let me know how it goes!
One idea I had for increasing the number of tactors is to use a matrix system for turning them on and off. the idea is that the board will only turn one tactor on or off at once but it cycles though them all so quickly that you cant tell the difference. Same as a computer screen or tv. Im guessing that that would take a Real lot of modification though :P
Jon29802 years ago
Cool project. Reminds me of some work done at MIT back in the 90's...
(see http://www.rle.mit.edu/media/pr142/23_Reed.pdf )
dreamer.redeemer (author)  Jon29802 years ago
Oh, the irony. I've found a reference for another paper by Jacob Kirman dating back to 1973 with the same general concept and the same title as the MIT paper. Apparently I was insufficient in searching for references, and it would seem even this mistake in this specific context has a precedent. An interesting coincidence, that one.
dreamer.redeemer (author)  Jon29802 years ago
Awesome, I can't believe that I didn't find this!! You really would think that searching for the word "vibrotactile" would significantly narrow the results to some very specifically relevant information. Thanks for sharing this, I'm far happier to have the question resolved than I am let down by not being the first. On the other hand, jeez, are there any original ideas left!? Well, at least attempting to distinguish what I think I know from what I really don't know pays off again.
dreamer.redeemer (author) 2 years ago
I'm honored to be on the front page, it satisfies me to know that I am capable of producing something agreeable. Thank you.

If this system can be improved so the definition between sunds is better think of how it could help the deaf!
Yup, with a little bit more than common materials cobbled together by a hand tool wielding mathematician this could easily scale, and hopefully the effect would scale as well. Scaling would also enable the simulation of virtual textures, so, for instance, you could actually feel a surface in a video game. Like I said in the last part, there are more promising, conventional avenues for overcoming deafness (ie cochlear implants, which are similar in principle), but this does have the advantages of low cost and no surgery. There would be many challenges though; a major one would be moving it off the hands for maximum usability, but the distribution of touch sensing neurons is much lower around most of the rest of the body, so the resolution would be limited. Regardless, I think that this device as-is may well provide sufficient stimuli to give a deaf person a substantial portion of the experience of music. I wish I knew a deaf person so I could find out!
Pro

Get More Out of Instructables

Already have an Account?

close

PDF Downloads
As a Pro member, you will gain access to download any Instructable in the PDF format. You also have the ability to customize your PDF download.

Upgrade to Pro today!