Haptic Feedback device for the Visually Impaired [Project HALO]

 by polymythic
Contest WinnerFeatured
I recently watched an episode of Stan Lee’s Superhumans which featured a blind
man who used a series of clicks, like a bat, to echo locate his surroundings. I
got to thinking about other blind people and their ability to navigate freely –
without the use of a guide dog or cane. I came up with the idea to use a series
of rangefinders that would take input from sensors and output feedback to pulse
vibration motors placed on a person’s head. As a person gets closer to an object
the intensity and frequency of the vibration would increase – it’s directly
proportional to the distance of an object. If a region was lacking feedback,
then it would be safe to proceed in that direction.

I call my submission the H.A.L.O. - the Haptic Assisted Locating of Obstacles. I
believe this can serve very useful for the visually impaired to have the freedom
to possibily move about hands-free without the assistance of a cane or seeing
eye dog. Technology has undoubtedly made our daily lives better. By using a few
inexpensive components and sensors, I’ve made a device that will allow the blind
to navigate their surroundings and avoid collisions.


 
Remove these adsRemove these ads by Signing Up

Step 1: Overview and Parts List

HALO_Project_www.polymythic.com_bb.jpg
Major Build Portions of the Project:
- Building the Halo
- Building the Motor Modules
- Building the Haptic Headband
- Wiring the Controller
- Creating the Software

The following is the parts list that will be relevant in the subsequent steps of this Instructable.  

Halo:
 - Rigid frame (I used a round embroidery frame)
 - Female headers (for the sensors)
 - Ultrasonic Rangefinders (Parallax Ping Rangefinders)
 - Wire (Wires with male and female leads are convenient)
 - Glue
 - Twist ties to tidy up wiring
 - Soldering station
 - Male headers (for creating a bridge to feed 5v and ground
 - RJ-45-Term Screw Terminal (2)
 - RJ-45 Cable
- Marker

Motor Modules:
 - Vibration Motors (5) - Motot, VIB,3V/60mA, 7500RPM
 - Grid-Style PC Board
 - Male header pins
 - Motor "shroud" (to prevent things getting sucked into the motor)

Haptic Headband:
 - Headband
 - Sewing Kit
 - 5 Motor Modules
 - Wire (Wires with male and female leads are convenient)
 - Safety Pins
- Female headers
- Soldering station
- RJ-45-Term Screw Terminal (2)
- RJ-45 Cable
- Marker

Wiring the Microcontroller:
 - Arduino Mega 2560
 - Wire (Wires with male and female leads are convenient)
 - 5 LEDs
 - Darlington IC - ULN 2803A
 - 2 port screw terminal
 - 9v battery
 - 5v regulator

Building the Software:
- USB cable
- PC (for editing code and downloading to Arduino)
- Arduino
- Arduino development environment (www.arduino.cc)
- Source Code, modified Ping.h library 
1-40 of 117Next »
comander01 says: Dec 13, 2010. 12:32 PM
Easily one of the most useful ideas on this site. I could definitely see this changing hundreds of lives. Possibly extend the range, and use a variable-vibration motor(if it exists), so that you can 'see' things across a room, and the vibration gets consistently stronger as the distance narrows. Also, perhaps if you used electric pulses instead of vibrations, the hardware could be slimmed down considerably. This is definitely worth patenting. I can also think of several military applications for a cheap, easy to use, set of night vision goggles that offer 360° vision.
polymythic (author) in reply to comander01Dec 13, 2010. 1:23 PM
Thanks for the comments, comander01. One of my design objectives was to keep the project at a resonable cost, but far more could be done with different sensors (Kinect, anyone?) The form factor can be considerably improved, I agree. I could get this into a normal hat. I do in fact use variable vibration motor. I use the PWM output for the intensity of the motor, and frequency between pulses as well.
SaberToothTyler says: Dec 13, 2010. 3:00 PM
I seriously made an account just to tell you how wonderful this is... i honestly could see this being used by visually impaired people around the world. The idea is astounding. maybe a suggestion would be to put motors elsewhere as well as the head, it seems like it could get a little crowded up there with all the buzzing and im sure you have considered this, but still its such an astounding idea.
polymythic (author) in reply to SaberToothTylerDec 13, 2010. 4:44 PM
SaberToothTyler,
Thanks so much for your nice words. You are right in that if there is a lot of detection in an area, multiple adjacent sensors are firing. Teasing out the bit of quiet in the "hallway" situation (left and right firing, forward is no vibration) requires some focus and training. However, it is amazing that as a sighted person, when blindfolded, you quickly focus your brain computing on those stimulus. I would like to see how I do after being blindfolded with the rig for an hour or so.
bishopdante says: Dec 19, 2010. 6:46 AM
I think this is the coolest thing I've ever seen on Instructables.

I think it'd be great to do a glove which you can feel distance on, too!

Also, would be great for able-bodied people, eyes in the back of the head...
polymythic (author) in reply to bishopdanteDec 19, 2010. 9:57 AM
Wow, bishopdante. That is high praise considering some of the AMAZING things that come up on this site. There are many directions to take this project for sure. Its a different experience having vision and using the device vs. being blindfolded. Your brain really does reach out for some data when you are blindfolded.
lovemachinesix says: Dec 20, 2010. 11:52 AM
Great Job poly! Breaking new ground with something we've had access to for a long time. Such a great idea. I think using this with an assist from a can for the low stuff, curbs and the like, the blind could really speed up and move more freely in unfamiliar places. Really good idea. Good luck with this. Benmansfield's advice to hit up a sight center sounds right on and could really help refine the design and bring up some 'real world' issues seeing impaired have that sighted people aren't even aware of. Again... good luck man!
polymythic (author) in reply to lovemachinesixDec 20, 2010. 6:57 PM
Thanks for the feedback. Agreed that the curb and low angle stuff is another part of the solution that should be integrated into the haptic picture. This project and people's commentary has fired up the mental gears thinking about assistive tech ideas.
polymythic (author) in reply to Helder4uDec 20, 2010. 7:09 PM
Yes, I did back-and-forth several times with a straight analog scaling. Its still in some of the #defines in the code as well. I decided on "threat levels" type of behavior rather than a full analog sliding scale. Its a quick replace in the code (or just uncomment it and put the computation back in.) There are several variables to tweak in the code as well, and the sensors will go out to 72 inches.
emattrose says: Dec 23, 2010. 8:54 PM
Hi polymythic, I'm so happy to read this instructable.
I worked on a very similar project last year, for the Intel STS competition. It also uses 5 sensors and vibrating motors to convey distance information to the user. I like your idea of placing the motors around the head - I put them on a belt, against the lower back. I suppose that makes sense because whereas you have the sensors spread around 180 degrees, mine all face forward at slightly different angles, so the feedback doesn't really need to wrap around the body. The sensor placement is an interesting design choice - I wonder whether it's more useful to have good resolution in front of your face or be able to sense things in all directions around you. I guess if you added enough sensors, you could have both! Too bad vibrating motors are so imprecise...
I haven't had chance to check out your code yet, I'm looking forward to it.
Best of luck, I'll be voting for you.
polymythic (author) in reply to emattroseDec 24, 2010. 6:29 PM
Great to hear from you emattrose. Sounds like you've been down this road! The sensor layout I selected because I wanted to be like "natural" vision with some peripheral. Someone else had commented on making the front stronger and peripheral weaker, which is interesting. The vibration sensors are a bit imprecise, but with the right material to minimize mechanical transferrance, you should be able to get some good resolution on the haptic. Hmm.. Perhaps one of those beds that you jump on and the next person doesnt feel it. The code still has some artifacts of my free play, so feel free to do whatever. Also I have heard of a threading library for Arduino which could have made my life easier than this crazy leapfrogging state machine with timeouts and scheduling.
nolte919 says: May 5, 2013. 2:09 PM
DX sells an ultrasonic range finder for $3.20 (USD) shipped, $2.60 if you buy 3 or more. It would cut down on the cost of making this considerably.
http://dx.com/p/hc-sr04-ultrasonic-sensor-distance-measuring-module-133696
tolstoyan says: Dec 31, 2012. 2:19 AM
when we do what you told we will not encounter another problem like bchua3 had? or we will also have this problem " the motor kept on spinning and it wont detect the proximity of the obstacle. " ?
also i really found it hard to use the figure 1 because in the figure the motor has only 2 pins while in the your instruction it has 3 pins hehehe im confused, i know i might be a burden but thank you for being understanding.
polymythic (author) in reply to tolstoyanDec 31, 2012. 8:03 AM
What is the debug output telling you. Look for //DEBUG. Also, this function fireAndPrintSensor should be telling you all kinds of stuff. What is the serial output telling you? It should be telling you the range and intensities. Do you have some debug output you could send?

// Display the range information
Serial.print("************ SENSOR ");
Serial.println(sensorNumber, DEC);
Serial.print(" Inches:");
Serial.println(inputSensorArray[sensorNumber].inches(), DEC);
Serial.print(" Feet:");
Serial.println(calculateRangeInFeet(inputSensorArray[sensorNumber].inches()),DEC);
Serial.print(" Gap in Milliseconds:");
Serial.println(motorStopTimeGap[sensorNumber]);
Serial.print(" Intensity:");
Serial.println(motorIntensity[sensorNumber]);
Serial.println();
tolstoyan in reply to polymythicJan 3, 2013. 4:00 PM
i will try to do everything you sent but i don't know how to debug can you instruct me how?
tolstoyan in reply to tolstoyanJan 7, 2013. 4:53 PM
i hope you do not loose your patience with us, sorry for trying to do a hard project for a beginner like us. your project is perfect for what we want to build. thank you for your full support and i hope you will keep on entertaining our questions, this is my email "ralphlouietionghoy@ymail.com" i hope you can send us your old schematic diagram. also when i delete the RESCHEDULE_THRESHOLD_CONSTANT there is no error regarding about that but still the error C:\Users\CAIMOL\AppData\Local\Temp\Rar$EXa0.454\arduino-1.0.2\libraries\Ping2/Ping2.h:22:22: error: WProgram.h: No such file or directory pop up but i already copy the ping2 library in the my arduino/libraries/ping2. Again thank you very much and i hope yo will respond ^^,
tolstoyan says: Dec 29, 2012. 5:52 PM
HI, we are already starting building your project but
we are having trouble with the codes. The following errors is below:

'RESCHEDULE_THRESHOLD_CONSTANT' was not declared in this scope.

In file included from HALO_Sketch_12_10_v8_2010.pde:15:
C:\Users\CAIMOL\AppData\Local\Temp\Rar$EXa0.454\arduino-1.0.2\libraries\Ping2/Ping2.h:22:22: error: WProgram.h: No such file or directory
HALO_Sketch_12_10_v8_2010.pde: In function 'int pingAndAdjustSchedule(int)':
HALO_Sketch_12_10_v8_2010:184: error: 'RESCHEDULE_THRESHOLD_CONSTANT' was not declared in this scope
polymythic (author) in reply to tolstoyanDec 29, 2012. 8:18 PM
Here is a similiar comment from another developer:

From: bchua3
Date: Dec 13, 2012. 8:01 PM
Subject: Project HALO

Hi Sir,
Good day to you.
I am actually trying out the project HALO that you submitted.
I uploaded the code to our Arduino Mega 2560 directly.
There were a few errors, the RESCHEDULE_THRESHOLD_CONSTANT was not found so I had to delete the // before it. Wprogram.h was not found as well so I had to replace it with Arduino.h as we are using the latest Arduino IDE Build.
I compiled the program and it was a go, however, the motor kept on spinning and it wont detect the proximity of the obstacle.
Any advice Sir?
Thanks
polymythic (author) in reply to tolstoyanDec 29, 2012. 7:33 PM
It was commented out in the code for some reason. Just delete the // preceding the RESCHEDULE_THRESHOLD_CONSTANT up top.

Also, you will need to copy the ping2 library into /your_arduino_directory/library/Ping2

tolstoyan in reply to polymythicJan 3, 2013. 4:10 PM
do i have to delete all these lines:

// RESCHEDULE_THRESHOLD_CONSTANT - Note: This may be replaced by "debouncing?" the motor
// This is the threshold over which the distance is large enough to trigger a
// reschedule event. If too small, there may be a lot of rescheduling, and if
// it is too large, then there may not be enough granularity.
//#define RESCHEDULE_THRESHOLD_CONSTANT 20.83

or all these lines:

// CALIBRATION VARIABLES
// INTERPULSE_LANTENCY (milliseconds) - Defines the period of time between when one ping
// fires and the next fires. This should allow for the sound to travel to
// its furthest detectable echo distance so that the next sensors RX will not
// pick up splash from the previous ping's TX. This may/will result in
// a lower than real distance result from the next sensor.
#define INTERPULSE_LATENCY_DURATION 25

// MOTOR_PULSE_DURATION (milliseconds) - Defines the duration that the motor will pulse
// This is a constant at the moment, and the MOTOR_VIBRATION_FACTOR will
// shorte between the pulses of the motor
#define MOTOR_PULSE_DURATION 250

// RESCHEDULE_THRESHOLD_CONSTANT - Note: This may be replaced by "debouncing?" the motor
// This is the threshold over which the distance is large enough to trigger a
// reschedule event. If too small, there may be a lot of rescheduling, and if
// it is too large, then there may not be enough granularity.
//#define RESCHEDULE_THRESHOLD_CONSTANT 20.83
tolstoyan says: Dec 29, 2012. 2:22 AM
hi, we are having trouble with the codes. The following errors is below:

'RESCHEDULE_THRESHOLD_CONSTANT' was not declared in this scope.

In file included from HALO_Sketch_12_10_v8_2010.pde:15:
C:\Users\CAIMOL\AppData\Local\Temp\Rar$EXa0.454\arduino-1.0.2\libraries\Ping2/Ping2.h:22:22: error: WProgram.h: No such file or directory
HALO_Sketch_12_10_v8_2010.pde: In function 'int pingAndAdjustSchedule(int)':
HALO_Sketch_12_10_v8_2010:184: error: 'RESCHEDULE_THRESHOLD_CONSTANT' was not declared in this scope
tolstoyan in reply to tolstoyanDec 29, 2012. 2:36 AM
please we really need the solution ASAP
tolstoyan says: Dec 21, 2012. 6:42 PM
hi polymythic remember me? hehehe we are currently building your prototype we have all the components ready and we plan to start working on it on 29th of december, just curious though, what is the LED for? only for testing? and if we are going to create the project itself, does that mean the diagram in part 1 will changed? i need clarity please reply ASAP tnx a lot :)
polymythic (author) in reply to tolstoyanDec 22, 2012. 11:34 AM
No problem. I am here to help. The LEDs are for debug/testing really. The motors do not come through well in videos, so the LEDS show how the motors are pulsed. Also, they do make nice debug indicators.
tolstoyan in reply to polymythicDec 26, 2012. 4:12 PM
so it means that we can removed the LED's when doing it on actual? by simply eliminating the connection of LED's in the diagram, it means that the motor will be directly connected in the IC?
tolstoyan says: Dec 7, 2012. 6:10 PM
hello i hope you will respond, i seriously want to built this project, but i find it hard to software, which exact code do i have to use because i am very poor at programming. can you tell me which one of the files i downloaded from this page is the code, if it is not there can you please send me or email it to me please please i really want to know the codes thank you and GOD bless
polymythic (author) in reply to tolstoyanDec 9, 2012. 10:18 AM
Certainly! Here you are! Good luck my friend.

http://polymythic.com/HALO_SourceCode_v8.zip
tolstoyan in reply to polymythicDec 9, 2012. 7:00 PM
what is the folder ping2 for? can i install the program HALO_Sketch_12_10_v8_2010.pde directly to the arduino without doing any changes? i am confused, what is the purpose of the ping2 folder? please respond i have all the materials bought. i will post some pictures and videos after i finish your project tnx a lot :)
polymythic (author) in reply to tolstoyanDec 9, 2012. 7:32 PM
The ping library calls to the IO that is connected to the digital input (the motion sensor). If nothing is available immediately, it will wait. This is called a "blocking read". That will screw lots of timing up if it cannot read from the motion sensor. Ping2 just gets rid of that blocking read so the system can continue on its merry way.

Copy ping2 library into arduino/libraries/Ping2 and it should be good to go. You'll see the "include ping2.h" in the sketch.
tolstoyan in reply to polymythicDec 9, 2012. 8:40 PM
thank you very much! your are so helpful i am building exactly what is in your instruction and if i hopefully understand it fully i might be able to make some modification hehehe but since im a beginner, i will make exactly this.
by the way im louie from the Philippines and a computer engineering student.here thanks again and GOD Bless...
polymythic (author) in reply to tolstoyanDec 10, 2012. 5:47 AM
The diagram is in the instructable. I did not produce a schematic as such. You should be able to understand how to hook everything up by looking at the pin assignment in the code and making sense of the image in step 1:

http://www.instructables.com/id/Haptic-Feedback-device-for-the-Visually-Impaired/

Zoom in and the pins should be right.
tolstoyan in reply to polymythicDec 10, 2012. 4:07 PM
im sorry if i am quite demanding hehe i dont want to put my components to waste if i made some mistake, coz items here in the philippines is quite expensive. anyway you really are helpful to your audience. wish you the best :)
polymythic (author) in reply to tolstoyanDec 10, 2012. 5:33 PM
No problem! The Maker Movement is about helping eachother out! Good luck from a friend in the United States!
tolstoyan in reply to tolstoyanDec 9, 2012. 8:52 PM
last request can i have a copy of your schematic diagram :) tnx
axeman911 says: Nov 7, 2012. 8:54 PM
I was thinking of doing this for a science fair, but much differently. Ur idea is way too bulky and only takes account for the upper-body not the lower.
polymythic (author) in reply to axeman911Dec 9, 2012. 10:18 AM
Go forth and improve, Axeman! That is the beauty of open projects. Let me know how it goes and I hope you win!
ishamsul says: Jul 22, 2012. 7:37 PM
Hi! Great project!!

I am a Honours Industrial Design student and I've decided to replicate this project as part of my Major.

I've gone for the budget HC-SR04 ultrasonic sensor, and now trying to modify your library to suit these ping sensors - i'm a novice with coding so i will see how I go. If you happen to know of any existing mods to your code could you please let me know?

The project is based around Low Vision and Location Sensing.


Best,
imran

www.imranshamsul.com
polymythic (author) in reply to ishamsulJul 23, 2012. 12:09 PM
Excellent! Let me know how your work goes. I have not modified any of the code, so it is as it was in the project I posted on instructables.

If you have any questions, I will try to answer them. Email me at steve@polymythic.com.

Good luck!
ishamsul in reply to polymythicOct 25, 2012. 11:33 AM
It now has its own instructables!

http://www.instructables.com/id/Haptic-Proximity-Module-HPM-for-Low-Vision-users/

Thank you again for the inspiration!!
HardwareDeveloper in reply to polymythicOct 14, 2012. 11:02 PM
Hi, Steve. Haptic Feedback device for the Visually Impaired [Project HALO] can be commercialized. HALO - project for blind too. If you make such a device on an industrial scale - you can make many people happy.
1-40 of 117Next »
Pro

Get More Out of Instructables

Already have an Account?

close

PDF Downloads
As a Pro member, you will gain access to download any Instructable in the PDF format. You also have the ability to customize your PDF download.

Upgrade to Pro today!