Haptic Feedback device for the Visually Impaired [Project HALO]

I recently watched an episode of Stan Lee’s Superhumans which featured a blind
man who used a series of clicks, like a bat, to echo locate his surroundings. I
got to thinking about other blind people and their ability to navigate freely –
without the use of a guide dog or cane. I came up with the idea to use a series
of rangefinders that would take input from sensors and output feedback to pulse
vibration motors placed on a person’s head. As a person gets closer to an object
the intensity and frequency of the vibration would increase – it’s directly
proportional to the distance of an object. If a region was lacking feedback,
then it would be safe to proceed in that direction.

I call my submission the H.A.L.O. - the Haptic Assisted Locating of Obstacles. I
believe this can serve very useful for the visually impaired to have the freedom
to possibily move about hands-free without the assistance of a cane or seeing
eye dog. Technology has undoubtedly made our daily lives better. By using a few
inexpensive components and sensors, I’ve made a device that will allow the blind
to navigate their surroundings and avoid collisions.

Step 1: Overview and Parts List

Major Build Portions of the Project:
- Building the Halo
- Building the Motor Modules
- Building the Haptic Headband
- Wiring the Controller
- Creating the Software

The following is the parts list that will be relevant in the subsequent steps of this Instructable.  

Halo:
 - Rigid frame (I used a round embroidery frame)
 - Female headers (for the sensors)
 - Ultrasonic Rangefinders (Parallax Ping Rangefinders)
 - Wire (Wires with male and female leads are convenient)
 - Glue
 - Twist ties to tidy up wiring
 - Soldering station
 - Male headers (for creating a bridge to feed 5v and ground
 - RJ-45-Term Screw Terminal (2)
 - RJ-45 Cable
- Marker

Motor Modules:
 - Vibration Motors (5) - Motot, VIB,3V/60mA, 7500RPM
 - Grid-Style PC Board
 - Male header pins
 - Motor "shroud" (to prevent things getting sucked into the motor)

Haptic Headband:
 - Headband
 - Sewing Kit
 - 5 Motor Modules
 - Wire (Wires with male and female leads are convenient)
 - Safety Pins
- Female headers
- Soldering station
- RJ-45-Term Screw Terminal (2)
- RJ-45 Cable
- Marker

Wiring the Microcontroller:
 - Arduino Mega 2560
 - Wire (Wires with male and female leads are convenient)
 - 5 LEDs
 - Darlington IC - ULN 2803A
 - 2 port screw terminal
 - 9v battery
 - 5v regulator

Building the Software:
- USB cable
- PC (for editing code and downloading to Arduino)
- Arduino
- Arduino development environment (www.arduino.cc)
- Source Code, modified Ping.h library 

Comments

[comander01]

Easily one of the most useful ideas on this site. I could definitely see this changing hundreds of lives. Possibly extend the range, and use a variable-vibration motor(if it exists), so that you can 'see' things across a room, and the vibration gets consistently stronger as the distance narrows. Also, perhaps if you used electric pulses instead of vibrations, the hardware could be slimmed down considerably. This is definitely worth patenting. I can also think of several military applications for a cheap, easy to use, set of night vision goggles that offer 360° vision.

[polymythic]

Thanks for the comments, comander01. One of my design objectives was to keep the project at a resonable cost, but far more could be done with different sensors (Kinect, anyone?) The form factor can be considerably improved, I agree. I could get this into a normal hat. I do in fact use variable vibration motor. I use the PWM output for the intensity of the motor, and frequency between pulses as well.

[SaberToothTyler]

I seriously made an account just to tell you how wonderful this is... i honestly could see this being used by visually impaired people around the world. The idea is astounding. maybe a suggestion would be to put motors elsewhere as well as the head, it seems like it could get a little crowded up there with all the buzzing and im sure you have considered this, but still its such an astounding idea.

[polymythic]

SaberToothTyler,
Thanks so much for your nice words. You are right in that if there is a lot of detection in an area, multiple adjacent sensors are firing. Teasing out the bit of quiet in the "hallway" situation (left and right firing, forward is no vibration) requires some focus and training. However, it is amazing that as a sighted person, when blindfolded, you quickly focus your brain computing on those stimulus. I would like to see how I do after being blindfolded with the rig for an hour or so.

[bishopdante]

I think this is the coolest thing I've ever seen on Instructables.

I think it'd be great to do a glove which you can feel distance on, too!

Also, would be great for able-bodied people, eyes in the back of the head...

[polymythic]

Wow, bishopdante. That is high praise considering some of the AMAZING things that come up on this site. There are many directions to take this project for sure. Its a different experience having vision and using the device vs. being blindfolded. Your brain really does reach out for some data when you are blindfolded.

[lovemachinesix]

Great Job poly! Breaking new ground with something we've had access to for a long time. Such a great idea. I think using this with an assist from a can for the low stuff, curbs and the like, the blind could really speed up and move more freely in unfamiliar places. Really good idea. Good luck with this. Benmansfield's advice to hit up a sight center sounds right on and could really help refine the design and bring up some 'real world' issues seeing impaired have that sighted people aren't even aware of. Again... good luck man!

[polymythic]

Thanks for the feedback. Agreed that the curb and low angle stuff is another part of the solution that should be integrated into the haptic picture. This project and people's commentary has fired up the mental gears thinking about assistive tech ideas.

[polymythic]

Yes, I did back-and-forth several times with a straight analog scaling. Its still in some of the #defines in the code as well. I decided on "threat levels" type of behavior rather than a full analog sliding scale. Its a quick replace in the code (or just uncomment it and put the computation back in.) There are several variables to tweak in the code as well, and the sensors will go out to 72 inches.

[emattrose]

Hi polymythic, I'm so happy to read this instructable.
I worked on a very similar project last year, for the Intel STS competition. It also uses 5 sensors and vibrating motors to convey distance information to the user. I like your idea of placing the motors around the head - I put them on a belt, against the lower back. I suppose that makes sense because whereas you have the sensors spread around 180 degrees, mine all face forward at slightly different angles, so the feedback doesn't really need to wrap around the body. The sensor placement is an interesting design choice - I wonder whether it's more useful to have good resolution in front of your face or be able to sense things in all directions around you. I guess if you added enough sensors, you could have both! Too bad vibrating motors are so imprecise...
I haven't had chance to check out your code yet, I'm looking forward to it.
Best of luck, I'll be voting for you.

[polymythic]

Great to hear from you emattrose. Sounds like you've been down this road! The sensor layout I selected because I wanted to be like "natural" vision with some peripheral. Someone else had commented on making the front stronger and peripheral weaker, which is interesting. The vibration sensors are a bit imprecise, but with the right material to minimize mechanical transferrance, you should be able to get some good resolution on the haptic. Hmm.. Perhaps one of those beds that you jump on and the next person doesnt feel it. The code still has some artifacts of my free play, so feel free to do whatever. Also I have heard of a threading library for Arduino which could have made my life easier than this crazy leapfrogging state machine with timeouts and scheduling.

[nolte919]

DX sells an ultrasonic range finder for $3.20 (USD) shipped, $2.60 if you buy 3 or more. It would cut down on the cost of making this considerably.
http://dx.com/p/hc-sr04-ultrasonic-sensor-distance-measuring-module-133696

[tolstoyan]

when we do what you told we will not encounter another problem like bchua3 had? or we will also have this problem " the motor kept on spinning and it wont detect the proximity of the obstacle. " ?
also i really found it hard to use the figure 1 because in the figure the motor has only 2 pins while in the your instruction it has 3 pins hehehe im confused, i know i might be a burden but thank you for being understanding.

[polymythic]

What is the debug output telling you. Look for //DEBUG. Also, this function fireAndPrintSensor should be telling you all kinds of stuff. What is the serial output telling you? It should be telling you the range and intensities. Do you have some debug output you could send?

// Display the range information
Serial.print("************ SENSOR ");
Serial.println(sensorNumber, DEC);
Serial.print(" Inches:");
Serial.println(inputSensorArray[sensorNumber].inches(), DEC);
Serial.print(" Feet:");
Serial.println(calculateRangeInFeet(inputSensorArray[sensorNumber].inches()),DEC);
Serial.print(" Gap in Milliseconds:");
Serial.println(motorStopTimeGap[sensorNumber]);
Serial.print(" Intensity:");
Serial.println(motorIntensity[sensorNumber]);
Serial.println();

[tolstoyan]

i will try to do everything you sent but i don't know how to debug can you instruct me how?

[tolstoyan]

i hope you do not loose your patience with us, sorry for trying to do a hard project for a beginner like us. your project is perfect for what we want to build. thank you for your full support and i hope you will keep on entertaining our questions, this is my email "[e-mail]" i hope you can send us your old schematic diagram. also when i delete the RESCHEDULE_THRESHOLD_CONSTANT there is no error regarding about that but still the error C:\Users\CAIMOL\AppData\Local\Temp\Rar$EXa0.454\arduino-1.0.2\libraries\Ping2/Ping2.h:22:22: error: WProgram.h: No such file or directory pop up but i already copy the ping2 library in the my arduino/libraries/ping2. Again thank you very much and i hope yo will respond ^^,

[tolstoyan]

HI, we are already starting building your project but
we are having trouble with the codes. The following errors is below:

'RESCHEDULE_THRESHOLD_CONSTANT' was not declared in this scope.

In file included from HALO_Sketch_12_10_v8_2010.pde:15:
C:\Users\CAIMOL\AppData\Local\Temp\Rar$EXa0.454\arduino-1.0.2\libraries\Ping2/Ping2.h:22:22: error: WProgram.h: No such file or directory
HALO_Sketch_12_10_v8_2010.pde: In function 'int pingAndAdjustSchedule(int)':
HALO_Sketch_12_10_v8_2010:184: error: 'RESCHEDULE_THRESHOLD_CONSTANT' was not declared in this scope

[polymythic]

Here is a similiar comment from another developer:

From: bchua3
Date: Dec 13, 2012. 8:01 PM
Subject: Project HALO

Hi Sir,
Good day to you.
I am actually trying out the project HALO that you submitted.
I uploaded the code to our Arduino Mega 2560 directly.
There were a few errors, the RESCHEDULE_THRESHOLD_CONSTANT was not found so I had to delete the // before it. Wprogram.h was not found as well so I had to replace it with Arduino.h as we are using the latest Arduino IDE Build.
I compiled the program and it was a go, however, the motor kept on spinning and it wont detect the proximity of the obstacle.
Any advice Sir?
Thanks

[polymythic]

It was commented out in the code for some reason. Just delete the // preceding the RESCHEDULE_THRESHOLD_CONSTANT up top.

Also, you will need to copy the ping2 library into /your_arduino_directory/library/Ping2

[tolstoyan]

do i have to delete all these lines:

// RESCHEDULE_THRESHOLD_CONSTANT - Note: This may be replaced by "debouncing?" the motor
// This is the threshold over which the distance is large enough to trigger a
// reschedule event. If too small, there may be a lot of rescheduling, and if
// it is too large, then there may not be enough granularity.
//#define RESCHEDULE_THRESHOLD_CONSTANT 20.83

or all these lines:

// CALIBRATION VARIABLES
// INTERPULSE_LANTENCY (milliseconds) - Defines the period of time between when one ping
// fires and the next fires. This should allow for the sound to travel to
// its furthest detectable echo distance so that the next sensors RX will not
// pick up splash from the previous ping's TX. This may/will result in
// a lower than real distance result from the next sensor.
#define INTERPULSE_LATENCY_DURATION 25

// MOTOR_PULSE_DURATION (milliseconds) - Defines the duration that the motor will pulse
// This is a constant at the moment, and the MOTOR_VIBRATION_FACTOR will
// shorte between the pulses of the motor
#define MOTOR_PULSE_DURATION 250

// RESCHEDULE_THRESHOLD_CONSTANT - Note: This may be replaced by "debouncing?" the motor
// This is the threshold over which the distance is large enough to trigger a
// reschedule event. If too small, there may be a lot of rescheduling, and if
// it is too large, then there may not be enough granularity.
//#define RESCHEDULE_THRESHOLD_CONSTANT 20.83

[tolstoyan]

hi, we are having trouble with the codes. The following errors is below:

'RESCHEDULE_THRESHOLD_CONSTANT' was not declared in this scope.

In file included from HALO_Sketch_12_10_v8_2010.pde:15:
C:\Users\CAIMOL\AppData\Local\Temp\Rar$EXa0.454\arduino-1.0.2\libraries\Ping2/Ping2.h:22:22: error: WProgram.h: No such file or directory
HALO_Sketch_12_10_v8_2010.pde: In function 'int pingAndAdjustSchedule(int)':
HALO_Sketch_12_10_v8_2010:184: error: 'RESCHEDULE_THRESHOLD_CONSTANT' was not declared in this scope

[tolstoyan]

please we really need the solution ASAP

[tolstoyan]

hi polymythic remember me? hehehe we are currently building your prototype we have all the components ready and we plan to start working on it on 29th of december, just curious though, what is the LED for? only for testing? and if we are going to create the project itself, does that mean the diagram in part 1 will changed? i need clarity please reply ASAP tnx a lot :)

[polymythic]

No problem. I am here to help. The LEDs are for debug/testing really. The motors do not come through well in videos, so the LEDS show how the motors are pulsed. Also, they do make nice debug indicators.

[tolstoyan]

so it means that we can removed the LED's when doing it on actual? by simply eliminating the connection of LED's in the diagram, it means that the motor will be directly connected in the IC?

[tolstoyan]

hello i hope you will respond, i seriously want to built this project, but i find it hard to software, which exact code do i have to use because i am very poor at programming. can you tell me which one of the files i downloaded from this page is the code, if it is not there can you please send me or email it to me please please i really want to know the codes thank you and GOD bless

[polymythic]

Certainly! Here you are! Good luck my friend.

http://polymythic.com/HALO_SourceCode_v8.zip

[tolstoyan]

what is the folder ping2 for? can i install the program HALO_Sketch_12_10_v8_2010.pde directly to the arduino without doing any changes? i am confused, what is the purpose of the ping2 folder? please respond i have all the materials bought. i will post some pictures and videos after i finish your project tnx a lot :)

[polymythic]

The ping library calls to the IO that is connected to the digital input (the motion sensor). If nothing is available immediately, it will wait. This is called a "blocking read". That will screw lots of timing up if it cannot read from the motion sensor. Ping2 just gets rid of that blocking read so the system can continue on its merry way.

Copy ping2 library into arduino/libraries/Ping2 and it should be good to go. You'll see the "include ping2.h" in the sketch.

[tolstoyan]

thank you very much! your are so helpful i am building exactly what is in your instruction and if i hopefully understand it fully i might be able to make some modification hehehe but since im a beginner, i will make exactly this.
by the way im louie from the Philippines and a computer engineering student.here thanks again and GOD Bless...

[polymythic]

The diagram is in the instructable. I did not produce a schematic as such. You should be able to understand how to hook everything up by looking at the pin assignment in the code and making sense of the image in step 1:

http://www.instructables.com/id/Haptic-Feedback-device-for-the-Visually-Impaired/

Zoom in and the pins should be right.

[tolstoyan]

im sorry if i am quite demanding hehe i dont want to put my components to waste if i made some mistake, coz items here in the philippines is quite expensive. anyway you really are helpful to your audience. wish you the best :)

[polymythic]

No problem! The Maker Movement is about helping eachother out! Good luck from a friend in the United States!

[tolstoyan]

last request can i have a copy of your schematic diagram :) tnx

[axeman911]

I was thinking of doing this for a science fair, but much differently. Ur idea is way too bulky and only takes account for the upper-body not the lower.

[polymythic]

Go forth and improve, Axeman! That is the beauty of open projects. Let me know how it goes and I hope you win!

[ishamsul]

Hi! Great project!!

I am a Honours Industrial Design student and I've decided to replicate this project as part of my Major.

I've gone for the budget HC-SR04 ultrasonic sensor, and now trying to modify your library to suit these ping sensors - i'm a novice with coding so i will see how I go. If you happen to know of any existing mods to your code could you please let me know?

The project is based around Low Vision and Location Sensing.


Best,
imran

www.imranshamsul.com

[polymythic]

Excellent! Let me know how your work goes. I have not modified any of the code, so it is as it was in the project I posted on instructables.

If you have any questions, I will try to answer them. Email me at [e-mail].

Good luck!

[ishamsul]

It now has its own instructables!

http://www.instructables.com/id/Haptic-Proximity-Module-HPM-for-Low-Vision-users/

Thank you again for the inspiration!!

[HardwareDeveloper]

Hi, Steve. Haptic Feedback device for the Visually Impaired [Project HALO] can be commercialized. HALO - project for blind too. If you make such a device on an industrial scale - you can make many people happy.

[polymythic]

Thanks! I appreciate the encouragement. Once several of my other projects clear of out the queue, I may pick it back up. I hope others will see how easy it is to make and take it to the next level! It could be you, my friend...

[ishamsul]

hi steve, must give you a big thanks for inspiring me to develop this idea into another level! I have developed a haptic proximity module that builds on all this kind of work with range finding and haptic output. best, imran

[polymythic]

Woohoo! Great stuff! Keep me updated! I think a tandem solution would be really good. Once my present craziness dies down a bit, perhaps we'll all have to compare notes and talk next steps!

[ishamsul]

thank you! i've just sent you an email. looking forward to future work/solutions! I really want to take this to the next level and make it accessible to the 200+ million low vision users out there.

[HardwareDeveloper]

Hi.I think we should make a separate circuit board to work with our device. Special electronic board for this application. Existing electronic circuit seems to me too much.
I can try to make a model of such a board, but I want to know what devices are existing circuit board used in your project.

[zrb01]

about how much does it cost to make it anyway?

[Revanchist]

well... just think of how much the Ping)) sensors are... they're around 100 bucks for 4 of them you can find them in parallax.com

or you can also use the el-cheap-o ultrasonic range finders from China but the adv of the ones on Parallax, is that you just need one pin for data transmission send and receive... vs. the other ones you need more code to control.. that's pretty much it...

I invite you to read the following book: make: arduino bots and gadgets... they show you how to build lots of really cool projects.

good luck

[polymythic]

I mention its <250 bucks. About 65 for Mega, and 130 for sensors. Then some for wiring, motors, IC, etc. You could take get by with another arduino, and cheaper motors. May be able to pull it off for 170ish.

[bond31]

if there are any pits in front of the person then how?

[rsokolov]

Can you explain me how it is possible to supply 3V motors with 5V from regulator?
Does the Darlington IC convert 5V to 3V?

I have 1.3V vibro motor but I don't know how to drive it.

[polymythic]

Nope, I was overdriving them with the 5v. For you motors you could either get a 1.3v regulator, or just use a voltage divider to get it down to what you want. If you are not familiar, simply put, its using a pair of different value resistors and the voltage across one will be what you want based on the resistor values. Just google "voltage divider".

[vhyde]

Just for kicks, i am going to suggest that at the very least you do a poor man's copyright of the material you have worked on so far. it is an important thing you are doing and while you are willing to allow others to bring it forward with changes...do not lose the right to have some control over your idea....again, i am very excited about what you have accomplished....as for the issues about how much of a "field of vision issue, a person who would use this would need as much input as possible...try getting with a cane user and determining how much field seems important to them with the cane and then go from there is my opinion...probably about a three to four foot distance across the front usually.

[tinker234]

nice tech this is really helpfull i know a poor liltle boy this could help him

[polymythic]

I hope someone can fine tune the design and make it simple and cheap. The packaging of the hardware needs to be improved for wearability and durability, as I've mentioned elsewhere I just wanted to see/show what was possible with cheap-ish components.

[tinker234]

ok wow

[vhyde]

as a teacher for the visually impaired and a person with visual impairment, i am fascinated by this...i am curious...one of the things to consider as you progress, is the ease of use aspect...i have had many AT devices that while the concept is wonderful, they were so difficult to use that it made it more work than it was worth...my newest personal and portable CCTV operates with three buttons...there is no learning curve at all really...even my youngest students will be able to learn to use it...

[polymythic]

Thanks for your comments and for taking a look at this, vhyde. I completely agree that ease-of-use should be at the top of the list for virtually any project. Especially for something that is trying to analog a natural sense. It needs to be a close to seamless as possible. As I've mentioned elsewhere, I think the real upgrades that I hope someone does is to vastly improve the "packaging" of the sensors (little plastic enclosures, electric thread, better material) and gets input from visually impaired users as to what are the real desired ranges for 1) initial detection, 2) ramp up of intensity, 3) "danger range". Best of luck and let me know how it goes!

[cptchaos]

Good work, how does it work in comparison with the "Haptic Radar", developed in 2008 at the University of Tokyo?
Link:
http://www.k2.t.u-tokyo.ac.jp/perception/HapticRadar/

[polymythic]

I had not seen that video prior to my attempt. They have an excellent debugging rig that puts my little LEDs to shame. In terms of effectiveness, I don't know. The range on the ultrasonic rangefinders is pretty good. People were avoiding boards and walls a good ways out with my system. It appeared that the objects were close for the IR, but I don't know the effective range. I really like their resolution (8 sensors vs. 5) and self-contained modules with sensor/feedback combined. I would guess ultrasonic would give them a better range.

[graffix]

This is a great idea and product you have.I'm gonna build this.Probably use a parallax controller and these vibration motors they are flat and shrouded http://www.pololu.com/catalog/product/1638 should work great
FYI My GrandMother(84) is gonna stay with me now she recently lost all her vision and is quite hard of hearing so the vibration should be good.
If this helps a little its worth it.the design is pefect for my application. I'm interested to see how shes feels about the buzzing on her head though.LOL THANKS.

[polymythic]

Graffix, thank you so much for your comment. I have been really excited for a case study to see the project applied more. Please don't hesitate to contact me if you need help, or at least let me know how your project is progressing. I would love to blog about your progress. [e-mail]

[jankins]

People have developed ideas like this before; the ones I've seen used sound instead of vibrations to indicate how far away objects were. I don't believe that a device like this will ever replace a dog or cane. It just isn't able to warn somebody about the hole in the sidewalk, or the drop-off in front of them, and to do so fast enough that they can respond to it while walking at a normal pace. A cane provides much more immediate and much more precise response. And what if your batteries run out? With some major technological leaps, a device like this may someday be useful as a supplement, but I don't think it will never replace the need for a blind individual to have skill in getting around with a cane.

Have any real blind or visually impaired individuals been involved with this project?

[polymythic]

Jankins, your concerns are similiar to other replies I have seen as well. This is not meant to be a replacement, it is meant to be a supplemental solution. For example, in an unfamiliar indoor setting the risk of sidewalks and drop offs is reduced, so with the right form factor this would be good additional data points upon which to base your movement decision. Also, the haptic feedback system could be integrated with other data sources (better rangefinders, warning transmitters on the walls if a very blind-friendly setting) and permit the wearer a better experience rather than a distracting series of beeping in their ears. As shown in the video, the wearers were having a conversation and their hearing was unaffected while receiving feedback.

[digitalia]

If you're interested in learning more about how various sensory systems work beyond our conscious awareness and how neuroplasticity and synesthesia allow the transfer of stimuli from one sensory system to another, read the book "See What I'm Saying" by Lawrence Rosenblum. Some of the examples used in comments here can be found in that text, along with countless others. I've pinged the author to let him know about your remarkable invention.

[Dilong_paradoxus]

This would be awesome for night vision, especially if you could find some way to get higher resolution...

[Helder4u]

I had a similar idea some years ago.
I believe it would work better with an analogue feedback instead of using 4 steps.
And also would be quite simple to program too.. :)

[will772]

Would it be possible for someone without a disability to use this? Im sure you could train to use this. My reason being that i could move around at night and not need to see anything (not that i could anways :D)

[polymythic]

Well, all the people in the videos were sighted. Take off the bindfold, turn off the lights and its the same experience!

[Sccrfreek92]

This is absolutely wonderful! I am wondering if you could take this idea to an efficiency company and have it looked at, then after it's fine tuned you could get it into a mass production facility and have all the technology built into one compact unit, just like kinect you could enter the world of gaming and biomedical engineering! Once again it's absolutely wonderful! I'm also wondering if it is possible to use the same basic idea but for deaf. For example if someone talks, the device vibrates on the skin... Well that idea needs more care. But once again, absolutely amazing device! Can I buy one? Haha

[polymythic]

Better than buy ibe Sccrfreek, you can BUILD one. Yeah, if I go again on this project I have a laundry list of improvements in form factor. The HALO will be integrated into a fedora, and everything will be through conductive thread. Good extension for the deaf too. As in life, loud noises just get out attention so our eyes can do the work, so could haptic feedback for the deaf.

[WingDings]

This is very nice. It reminds me of the Haptic Torch project that was developed at the University of Reading (in the UK).

That device was nice in that the user could wave it around to be alerted to things that were near them, in any direction it was pointed, though your project has the advantage of having several sensors working simultaneously at multiple angles.

Maybe a combination of these could provide a good solution, with multiple simultaneous sensors on the head, and maybe a couple of sensors in/on the hands that could be pointed to give feedback about a particular location? I'm envisaging something that looked like the things Iron Man had in the palms of his hands (of course). :)

[polymythic]

Yes, somewhat like llanyort's suggestion. Having a powerful device in your hand for long range work could give you some pre-sighting for as your get closer and Halo takes over. Or this torch could do the low angle work.

[deusXmchna]

Wow. This is great, and great "instructable"-ness(?) as well. Just wanted to say "Wow". I've got a relative that works with tech for the disabled, will fwd this link.

(FWIW- I first saw a story on a blind guy using verbal "sonar" almost 10 years ago. I then blindfolded myself for 2 days, and tried it out. I was surprised how quickly I adapted; walls and large objects were easy.. but then again my brain & ears were 10 years younger :) )

Thank you for the best thing I've read in a bit!

[polymythic]

By all means! It would be great for you to bring some of the areas of interest/research into the discussion from your relative. There are wicked smart people on the board. Perhaps people could start their Humana projects a year in advance. I'll be looking for teammates :-)

[llanyort]

I would say that the most economical and simplistic way to build this device would be in a single handed batons or gloves with audio feedback. one hand could be looking at a "distance" and then the other hand looking at up close. using different tones representing the variable distance to object.

maybe use vibraration for the "up close" hand and "sounds" for the distance hand.

just contributing to the sandbox.

[polymythic]

Having vision on the hands is a good idea. Certainly easier to "look around" with, and you have the advantage of the ultimate collision preventers, your hands, in front of you. I just had a mental image of a visually impaired person with a bundle of Christmas gifts in their hands, and how hard it would be to pull that off with a cane or dog. Hence my desire for a hands-free solution...this time.

[lockpick]

This is great!
I think one that could also see down low would be great. The whole project is great but imagine if you built it so that it showed objects on a screen. Of course that wouldn't be helpful for blind but imagine the applications for military use. And medical use. Before performing surgery put inside the wound and detect what needs to come out first. You should definitely get a patent for this because I'm sure someone would try to make money off of this before you do (if you plan on maing money?)? Anyways great instructable.

[polymythic]

Thanks lockpick. Perhaps biomedical companies have far better precision sensors than this. I would suspect that NVG would be preferable, but perhaps this could support that, or work in fog/smoke conditions as our firefighter friend mentions.

[Culturespy]

It's published now. The author has a year from publication date to apply. Their work is protected for now.

[Pader]

For Comander01

Whereas I agree with your comments on this 'ible', unfortunately, the author can no longer patent his idea as he has published it on this page and it is now in the public domain. If anyone is intending to patent a device, he/she must do so before the idea is published.

[ewilhelm]

In the US, one has a year from the first public disclosure to patent something.

[boygenuis32]

This is brilliant. I might integrate this into my pseudo-combat suit as a back-watcher, so I can feel the people behind me.
Only problem I see is shin biters. Got to watch low level obstacles. Great job!

[rwest202]

I have greatly enjoyed your Instructable. A couple weeks ago I was present in a car accident(nothing serious). the car was very icy and My friend, who was driving, could not see a car that was closing in on us. The car bumped us and we slid a bit.

If this technology could be present in cars and other transportation, then i think that the rate of car accidents would decrease significantly. I know that we have this in some newer vehicles. However, a device that fits around the head and emits a low tone when a car comes close would be very helpful. The range would not have to be very far, and it would be inexpensive.

Again I Appreciate your Instructable and hope that research in this area thrives to great extents. Increasing the safety of people all ages and benefiting the disabled din all ways possible.

(end of rant)

[buteman]

Would it help if the vibrations from the sensor directly in front had a higher output than the others? I am thinking that, just as with sight our attention is directed at where we are looking and usually that is straight ahead. Possibly then, if something was moving towards you, but not directly ahead, the output from there could be made to increase so it would stand out.

[Biggsy]

You know what, well done, that's a fantastic concept! Great I'ble :)

[polymythic]

Thanks Biggsy. See you posted a treb 'ible too. Check mine out over at (http://www.polymythic.com/tag/trebuchet/) if you ever get bored.

[Biggsy]

Ooh, I will do, my scouts love mini pioneering ... and large scale too if i give them a helping hand :D

[fcohen]

This would a great game device. Imagine the wearer is a player in a game. Their goal is to reach a location that has an award - for example a candy bar. The players opponents stand with boards in the way of the path to the candy.

What fun!

-Frank

[polymythic]

Very clever fcohen! I am sure there would be lots of gaming angles if you thought about it. Post 'em up!

[manumanu764]

If you want to make the "calibration"(learning) professionally, you should make it something like the training chamber in Half Life 1, what I mean is going from a clear space for about 1 hour (no obstacles, no walls just to dont soffocate without vision), then to a clear room, then something more complex like a room with some furniture(not so much), a full room, and after all, moving obstacles, so you can get a full training without get confused.
Cheers!

[polymythic]

Agreed. We all needed some training, but it was also good so show new users getting accustomed to it.

[fredfx]

This is Fantastic. Genius actually. And will do so much good once developed. Congratulations. And big deposit in the Karma bank account for you.

[polymythic]

Thanks, Fred. I hope the email I send to the Columbia Lighthouse for the Blind gets answered. Better fabrication, funding, and smarter people than I will take this way further.

[the_keiser]

Nice design. I did something similar for my kinetic arts class a few months ago. I hid the sensor inside a set of sunglasses and rigged up a pair of headphones to produce a tone in relation to distance.

[geonav1]

Really great project Polymythic. I'm going to share this with my group. A sensor that could range downwards to detect either stairs or a curb ( up OR down) would be good- maybe an audio tone whose volume indicates the height or perceived depth of the curb, stair, hole or low wall- louder means higher/deeper.

I look forward to seeing more progress with this wonderful idea that could help so many people!

[polymythic]

Geonav1. Great to hear you liked it. Feel free to email me at [e-mail] if you pick up the ball and run with it. More sensors would offer more of the "Y" axis with some offset from the HALO ring. With training, a combination haptic and auditory "image" could give perhaps a fairly complete picture for an impaired user. I tried contacting a member from an organization for the blind. Hope to hear back!

[green_gooseman]

this is cool,man :D well done!

i was working on a sensor for blind people too(sometime back) but that was with a tiny torch sized gadget giving out beeps at different BPM according to distance.
there is one point to consider though,have u tested it in a crowded environment for long time to see if it causes any headache coz of different vibrators being off/on most of the time.probably u can overcome that by having a voice guide the person(like:"Obstacle at 6o'clock".lol..but that'll need coding again.darn!)...have been designing a something similar to yours but for people who have sight,needs heavy coding,will start working on it soon.

i have a suggestion(stupid one nonetheless), to use my torch sized sensor thingy and connect the output of that one and the the outputs of your HALO to an AND gate so that u can confirm whats nearby(coz these sensors have certain drawbacks when the signal projected falls on funnily curved surfaces),know the approx distance and possibly have a switch to choose weather u want any 1 or both to be working.

Cheers! :D

[polymythic]

Green, thanks for your comment. From the commentary I have been getting from other projects, it sounds like each sensor has its own nuances and strengths. This is part of A solution, not necessarily the total solution. There could be some cleverness applied to gather more of a complete picture of the surrounding, and sum them together as you proposed. I would apply the "AND"-ing in SW, because the PING))) gives a range and more of a filtering algorithm could be worked out. Anything is possible!!

[brawns214]

Your idea here resembles similar areas of research already in progress. In fact there is significant progress utilizing different sensory systems in place. The coolest part of this technology is that our brain isn't as hardwired as people previously believed. It actually learns to adapt to these new systems like yours. So while a system might seem clumsy at first, people develop crazy abilities.

Here is an example using a stimulating pad placed on the tongue.
http://io9.com/5338054/brainport-device-lets-the-blind-use-their-taste-buds-to-see

This one actually uses a video camera and encodes the information audio thats called soundscapes. The encoding isn't something that is very intuitive, but its trainable. So people learn to see with their ears. The craziest part is that people who can see normally process audio information in the audio cortex of the brain and visual information in our visual cortex. For blind who use this sort of system, the audio signals that correspond to visual information eventually get decoded in the visual cortex! This means that their most important sense (hearing) isn't disrupted. In fact I heard that people with this system can even watch TV decoding visual and audio independently.
http://www.israel21c.org/technology/teaching-the-blind-to-see-through-sound

This is such a fascinating topic to me and I encourage you to study and keep developing your systems. You're awesome!

[polymythic]

Thanks! I have heard crazy stories about neuroplasticity on TV. Even in short visual deprivation my brain was crying out for data to process and latched quickly on the haptic. With high-res sensors like the Kinect, I think once you can map that 3d space into a haptic or audio signal, it may yield incredible results with training (HINT HINT, someone brilliant out there with some time). So much cool stuff.

[polymythic]

The range is 4 feet that I capped it at. I should have had everyone tap their hand as they picked up anything with direction so you could get a sense of when they were detecting anything. That seems to convey the haptic feedback in the video well.

[tlrice]

I am greatly impressed at the accuacy and ease of which this devise works. while this will not replace sight, it would be a wonderful aid for the impaired, probably best used within a confined area, especially where the individual is not used to the surroundings. With additional sensors this application could receive information, much like the smart cars driving via magnetic impulse, but used to locate specific areas within a building. Great job, throughly impressed.

[polymythic]

Thanks for your comments, tlrice. Yes, there are a lot of other avenues to explore from here. Perhaps just avoiding things as you mentioned in unfamiliar settings. I do have a GPS module as well. Haptic GPS + obstacle avoidance could make for good navigation from A to B.

[benmansfield]

hey buddy... it's been too long since we've been able to catch up, and even though we're separated by a distance you and your ideas never cease to amaze me. This is a good one. Specific to the ultrasonics, any idea how they'd work on something of lower density, like a screen door? I'm sure there are quite sophisticated (expensive) sensors available that would allow you to discern the difference. Of course, as the sensing capabilities got more sophisticated then of course the feedback system could get more sophisticated as well - folks above have already made some good suggestions. You should contact a local sight center to see if they have a doctor or volunteer who would be willing to test it... It would be interesting to see how quickly someone with an actual vision deficiency would adapt to the stimuli. anyway, take care man!

[polymythic]

Thanks for the reply! I don't have specific data on how a ultrasonics would respond to a screen, but I would suspect the response would be weak enough to not even trigger. You can put screen over a speaker and not diminish the sound, so I would think the same would apply for ultrasonics, hence little return. Its great that so many creative users are united in the bond of trying to help people with suggestions for improvements to these projects.

[AlternateLives]

Your really on to something here...

Now if it is possible to do the same thing with the limbs as well, such as a vibrating motor on the hands to detect an object when the subject reaches for something.

The possibilities are endless...

[thecapacity]

I could see that idea being useful, i.e. if your hands had LIDAR or some other more sensitive sensor / feedback system. Then you could 'touch' at a distance!

[polymythic]

AlternateLives, it very well could be. More sensor arrays on the limbs is a good extension of this concept.

[AlternateLives]

That's the Idea!

[omnibot]

What a wonderfully goofy prototype, complete with crazy headband.

[polymythic]

Every project needs a crazy headband, Omnibot :-)

[canida]

Whoa, neat!

[polymythic]

Thanks a bunch, Canida!

[thecapacity]

I've seen some vibration motors that are enclosed, but nice use of construx to shroud the motor!

There's no particular reason why the motors have to be on your head with the sensors, it would be interesting if other parts of your body have more granular sensitivity to the buzzing (i.e. a belt) or a necklace vs. a headband.

[polymythic]

Yeah, it almost would not feel like a Polymythic project without Construx. Actually, I never intended them. I went to various places, and even got a Slurpee for the first time in years for the largest diameter straw I could find. The motor would not fit, but snapped RIGHT into the Construx. Sorry, I can't get a way from them :-)

[kryptofacist]

This is a great idea. I have often thought of sonar devices for the blind, but don't really have the knowledge to create such a device. My main suggestion would be fitting the sensor array to a belt in order to pick up objects from a lower point of view (for furniture). Concerns include outdoor settings, such as crossing the street or navigating in a crowd. As it is, a seeing-eye dog still wins out for versatility. However, this is a nice first step.
I hope I didn't sound mean, because I am really impressed.

[polymythic]

Thanks for your comments! I completely agree. As is the case with prototypes, there is much more to do. Also I wish we could have spent some time training with the devices to become adept at moving through the room. It has a 4 foot detection radius. The good thing is that it this project has me thinking of other ways to solve parts of the visually impaired existing solutions. Technology does not have to be the complete solution, but can be a part of it, or connect other pieces.