Instructables

Haptic Feedback device for the Visually Impaired [Project HALO]

FeaturedContest Winner
I recently watched an episode of Stan Lee’s Superhumans which featured a blind
man who used a series of clicks, like a bat, to echo locate his surroundings. I
got to thinking about other blind people and their ability to navigate freely –
without the use of a guide dog or cane. I came up with the idea to use a series
of rangefinders that would take input from sensors and output feedback to pulse
vibration motors placed on a person’s head. As a person gets closer to an object
the intensity and frequency of the vibration would increase – it’s directly
proportional to the distance of an object. If a region was lacking feedback,
then it would be safe to proceed in that direction.

I call my submission the H.A.L.O. - the Haptic Assisted Locating of Obstacles. I
believe this can serve very useful for the visually impaired to have the freedom
to possibily move about hands-free without the assistance of a cane or seeing
eye dog. Technology has undoubtedly made our daily lives better. By using a few
inexpensive components and sensors, I’ve made a device that will allow the blind
to navigate their surroundings and avoid collisions.


 
Remove these adsRemove these ads by Signing Up

Step 1: Overview and Parts List

Picture of Overview and Parts List
Major Build Portions of the Project:
- Building the Halo
- Building the Motor Modules
- Building the Haptic Headband
- Wiring the Controller
- Creating the Software

The following is the parts list that will be relevant in the subsequent steps of this Instructable.  

Halo:
 - Rigid frame (I used a round embroidery frame)
 - Female headers (for the sensors)
 - Ultrasonic Rangefinders (Parallax Ping Rangefinders)
 - Wire (Wires with male and female leads are convenient)
 - Glue
 - Twist ties to tidy up wiring
 - Soldering station
 - Male headers (for creating a bridge to feed 5v and ground
 - RJ-45-Term Screw Terminal (2)
 - RJ-45 Cable
- Marker

Motor Modules:
 - Vibration Motors (5) - Motot, VIB,3V/60mA, 7500RPM
 - Grid-Style PC Board
 - Male header pins
 - Motor "shroud" (to prevent things getting sucked into the motor)

Haptic Headband:
 - Headband
 - Sewing Kit
 - 5 Motor Modules
 - Wire (Wires with male and female leads are convenient)
 - Safety Pins
- Female headers
- Soldering station
- RJ-45-Term Screw Terminal (2)
- RJ-45 Cable
- Marker

Wiring the Microcontroller:
 - Arduino Mega 2560
 - Wire (Wires with male and female leads are convenient)
 - 5 LEDs
 - Darlington IC - ULN 2803A
 - 2 port screw terminal
 - 9v battery
 - 5v regulator

Building the Software:
- USB cable
- PC (for editing code and downloading to Arduino)
- Arduino
- Arduino development environment (www.arduino.cc)
- Source Code, modified Ping.h library 
1-40 of 123Next »
emattrose3 years ago
Hi polymythic, I'm so happy to read this instructable.
I worked on a very similar project last year, for the Intel STS competition. It also uses 5 sensors and vibrating motors to convey distance information to the user. I like your idea of placing the motors around the head - I put them on a belt, against the lower back. I suppose that makes sense because whereas you have the sensors spread around 180 degrees, mine all face forward at slightly different angles, so the feedback doesn't really need to wrap around the body. The sensor placement is an interesting design choice - I wonder whether it's more useful to have good resolution in front of your face or be able to sense things in all directions around you. I guess if you added enough sensors, you could have both! Too bad vibrating motors are so imprecise...
I haven't had chance to check out your code yet, I'm looking forward to it.
Best of luck, I'll be voting for you.
polymythic (author)  emattrose3 years ago
Great to hear from you emattrose. Sounds like you've been down this road! The sensor layout I selected because I wanted to be like "natural" vision with some peripheral. Someone else had commented on making the front stronger and peripheral weaker, which is interesting. The vibration sensors are a bit imprecise, but with the right material to minimize mechanical transferrance, you should be able to get some good resolution on the haptic. Hmm.. Perhaps one of those beds that you jump on and the next person doesnt feel it. The code still has some artifacts of my free play, so feel free to do whatever. Also I have heard of a threading library for Arduino which could have made my life easier than this crazy leapfrogging state machine with timeouts and scheduling.
Great Job poly! Breaking new ground with something we've had access to for a long time. Such a great idea. I think using this with an assist from a can for the low stuff, curbs and the like, the blind could really speed up and move more freely in unfamiliar places. Really good idea. Good luck with this. Benmansfield's advice to hit up a sight center sounds right on and could really help refine the design and bring up some 'real world' issues seeing impaired have that sighted people aren't even aware of. Again... good luck man!
polymythic (author)  lovemachinesix3 years ago
Thanks for the feedback. Agreed that the curb and low angle stuff is another part of the solution that should be integrated into the haptic picture. This project and people's commentary has fired up the mental gears thinking about assistive tech ideas.
bishopdante3 years ago
I think this is the coolest thing I've ever seen on Instructables.

I think it'd be great to do a glove which you can feel distance on, too!

Also, would be great for able-bodied people, eyes in the back of the head...
polymythic (author)  bishopdante3 years ago
Wow, bishopdante. That is high praise considering some of the AMAZING things that come up on this site. There are many directions to take this project for sure. Its a different experience having vision and using the device vs. being blindfolded. Your brain really does reach out for some data when you are blindfolded.
I seriously made an account just to tell you how wonderful this is... i honestly could see this being used by visually impaired people around the world. The idea is astounding. maybe a suggestion would be to put motors elsewhere as well as the head, it seems like it could get a little crowded up there with all the buzzing and im sure you have considered this, but still its such an astounding idea.
polymythic (author)  SaberToothTyler3 years ago
SaberToothTyler,
Thanks so much for your nice words. You are right in that if there is a lot of detection in an area, multiple adjacent sensors are firing. Teasing out the bit of quiet in the "hallway" situation (left and right firing, forward is no vibration) requires some focus and training. However, it is amazing that as a sighted person, when blindfolded, you quickly focus your brain computing on those stimulus. I would like to see how I do after being blindfolded with the rig for an hour or so.
comander013 years ago
Easily one of the most useful ideas on this site. I could definitely see this changing hundreds of lives. Possibly extend the range, and use a variable-vibration motor(if it exists), so that you can 'see' things across a room, and the vibration gets consistently stronger as the distance narrows. Also, perhaps if you used electric pulses instead of vibrations, the hardware could be slimmed down considerably. This is definitely worth patenting. I can also think of several military applications for a cheap, easy to use, set of night vision goggles that offer 360° vision.
polymythic (author)  comander013 years ago
Thanks for the comments, comander01. One of my design objectives was to keep the project at a resonable cost, but far more could be done with different sensors (Kinect, anyone?) The form factor can be considerably improved, I agree. I could get this into a normal hat. I do in fact use variable vibration motor. I use the PWM output for the intensity of the motor, and frequency between pulses as well.
amaproject2 months ago

It's a great project you've done it, so could you please provide me how

Darlington IC - ULN 2803A pins are connected in the circuit?

My email ahmed.ebrahem@gmail.com

Patuko3 months ago

Oh man you guys rock so hard :D

I was wondering to build a project similar like this and i'm seeing really hard work here about it! :D Really nice.

Did you guys have iterate on it? Seen here also another hactip project with a glove that also measures analog ir on fingers to vibe, really nice thoughts.

Will really want to know if you have been testing it on over this years and if you can "sense" enviroment better.

Thank you!

janandh10 months ago
sir will you please send me the source coding..my mail id is r.jamesanandh@gmail.com
polymythic (author)  janandh10 months ago
Hello! Source code is on the site in the instructable. It's fallen into disrepair because the Arduino environment has updated a couple times. I've made some modifications in the last year because others have contacted me. Way behind on updating. Will get to it when I've got time.
janandh10 months ago
sir will you please send me the source coding..my mail id is r.jamesanandh@gmail.com
janandh10 months ago
sir will you please send me the source coding..my mail id is r.jamesanandh@gmail.com
nolte9191 year ago
DX sells an ultrasonic range finder for $3.20 (USD) shipped, $2.60 if you buy 3 or more. It would cut down on the cost of making this considerably.
http://dx.com/p/hc-sr04-ultrasonic-sensor-distance-measuring-module-133696
tolstoyan1 year ago
when we do what you told we will not encounter another problem like bchua3 had? or we will also have this problem " the motor kept on spinning and it wont detect the proximity of the obstacle. " ?
also i really found it hard to use the figure 1 because in the figure the motor has only 2 pins while in the your instruction it has 3 pins hehehe im confused, i know i might be a burden but thank you for being understanding.
polymythic (author)  tolstoyan1 year ago
What is the debug output telling you. Look for //DEBUG. Also, this function fireAndPrintSensor should be telling you all kinds of stuff. What is the serial output telling you? It should be telling you the range and intensities. Do you have some debug output you could send?

// Display the range information
Serial.print("************ SENSOR ");
Serial.println(sensorNumber, DEC);
Serial.print(" Inches:");
Serial.println(inputSensorArray[sensorNumber].inches(), DEC);
Serial.print(" Feet:");
Serial.println(calculateRangeInFeet(inputSensorArray[sensorNumber].inches()),DEC);
Serial.print(" Gap in Milliseconds:");
Serial.println(motorStopTimeGap[sensorNumber]);
Serial.print(" Intensity:");
Serial.println(motorIntensity[sensorNumber]);
Serial.println();
i will try to do everything you sent but i don't know how to debug can you instruct me how?
i hope you do not loose your patience with us, sorry for trying to do a hard project for a beginner like us. your project is perfect for what we want to build. thank you for your full support and i hope you will keep on entertaining our questions, this is my email "ralphlouietionghoy@ymail.com" i hope you can send us your old schematic diagram. also when i delete the RESCHEDULE_THRESHOLD_CONSTANT there is no error regarding about that but still the error C:\Users\CAIMOL\AppData\Local\Temp\Rar$EXa0.454\arduino-1.0.2\libraries\Ping2/Ping2.h:22:22: error: WProgram.h: No such file or directory pop up but i already copy the ping2 library in the my arduino/libraries/ping2. Again thank you very much and i hope yo will respond ^^,
tolstoyan1 year ago
HI, we are already starting building your project but
we are having trouble with the codes. The following errors is below:

'RESCHEDULE_THRESHOLD_CONSTANT' was not declared in this scope.

In file included from HALO_Sketch_12_10_v8_2010.pde:15:
C:\Users\CAIMOL\AppData\Local\Temp\Rar$EXa0.454\arduino-1.0.2\libraries\Ping2/Ping2.h:22:22: error: WProgram.h: No such file or directory
HALO_Sketch_12_10_v8_2010.pde: In function 'int pingAndAdjustSchedule(int)':
HALO_Sketch_12_10_v8_2010:184: error: 'RESCHEDULE_THRESHOLD_CONSTANT' was not declared in this scope
polymythic (author)  tolstoyan1 year ago
It was commented out in the code for some reason. Just delete the // preceding the RESCHEDULE_THRESHOLD_CONSTANT up top.

Also, you will need to copy the ping2 library into /your_arduino_directory/library/Ping2

do i have to delete all these lines:

// RESCHEDULE_THRESHOLD_CONSTANT - Note: This may be replaced by "debouncing?" the motor
// This is the threshold over which the distance is large enough to trigger a
// reschedule event. If too small, there may be a lot of rescheduling, and if
// it is too large, then there may not be enough granularity.
//#define RESCHEDULE_THRESHOLD_CONSTANT 20.83

or all these lines:

// CALIBRATION VARIABLES
// INTERPULSE_LANTENCY (milliseconds) - Defines the period of time between when one ping
// fires and the next fires. This should allow for the sound to travel to
// its furthest detectable echo distance so that the next sensors RX will not
// pick up splash from the previous ping's TX. This may/will result in
// a lower than real distance result from the next sensor.
#define INTERPULSE_LATENCY_DURATION 25

// MOTOR_PULSE_DURATION (milliseconds) - Defines the duration that the motor will pulse
// This is a constant at the moment, and the MOTOR_VIBRATION_FACTOR will
// shorte between the pulses of the motor
#define MOTOR_PULSE_DURATION 250

// RESCHEDULE_THRESHOLD_CONSTANT - Note: This may be replaced by "debouncing?" the motor
// This is the threshold over which the distance is large enough to trigger a
// reschedule event. If too small, there may be a lot of rescheduling, and if
// it is too large, then there may not be enough granularity.
//#define RESCHEDULE_THRESHOLD_CONSTANT 20.83
polymythic (author)  tolstoyan1 year ago
Here is a similiar comment from another developer:

From: bchua3
Date: Dec 13, 2012. 8:01 PM
Subject: Project HALO

Hi Sir,
Good day to you.
I am actually trying out the project HALO that you submitted.
I uploaded the code to our Arduino Mega 2560 directly.
There were a few errors, the RESCHEDULE_THRESHOLD_CONSTANT was not found so I had to delete the // before it. Wprogram.h was not found as well so I had to replace it with Arduino.h as we are using the latest Arduino IDE Build.
I compiled the program and it was a go, however, the motor kept on spinning and it wont detect the proximity of the obstacle.
Any advice Sir?
Thanks
tolstoyan1 year ago
hi, we are having trouble with the codes. The following errors is below:

'RESCHEDULE_THRESHOLD_CONSTANT' was not declared in this scope.

In file included from HALO_Sketch_12_10_v8_2010.pde:15:
C:\Users\CAIMOL\AppData\Local\Temp\Rar$EXa0.454\arduino-1.0.2\libraries\Ping2/Ping2.h:22:22: error: WProgram.h: No such file or directory
HALO_Sketch_12_10_v8_2010.pde: In function 'int pingAndAdjustSchedule(int)':
HALO_Sketch_12_10_v8_2010:184: error: 'RESCHEDULE_THRESHOLD_CONSTANT' was not declared in this scope
please we really need the solution ASAP
tolstoyan1 year ago
hi polymythic remember me? hehehe we are currently building your prototype we have all the components ready and we plan to start working on it on 29th of december, just curious though, what is the LED for? only for testing? and if we are going to create the project itself, does that mean the diagram in part 1 will changed? i need clarity please reply ASAP tnx a lot :)
polymythic (author)  tolstoyan1 year ago
No problem. I am here to help. The LEDs are for debug/testing really. The motors do not come through well in videos, so the LEDS show how the motors are pulsed. Also, they do make nice debug indicators.
so it means that we can removed the LED's when doing it on actual? by simply eliminating the connection of LED's in the diagram, it means that the motor will be directly connected in the IC?
tolstoyan1 year ago
hello i hope you will respond, i seriously want to built this project, but i find it hard to software, which exact code do i have to use because i am very poor at programming. can you tell me which one of the files i downloaded from this page is the code, if it is not there can you please send me or email it to me please please i really want to know the codes thank you and GOD bless
polymythic (author)  tolstoyan1 year ago
Certainly! Here you are! Good luck my friend.

http://polymythic.com/HALO_SourceCode_v8.zip
what is the folder ping2 for? can i install the program HALO_Sketch_12_10_v8_2010.pde directly to the arduino without doing any changes? i am confused, what is the purpose of the ping2 folder? please respond i have all the materials bought. i will post some pictures and videos after i finish your project tnx a lot :)
polymythic (author)  tolstoyan1 year ago
The ping library calls to the IO that is connected to the digital input (the motion sensor). If nothing is available immediately, it will wait. This is called a "blocking read". That will screw lots of timing up if it cannot read from the motion sensor. Ping2 just gets rid of that blocking read so the system can continue on its merry way.

Copy ping2 library into arduino/libraries/Ping2 and it should be good to go. You'll see the "include ping2.h" in the sketch.
thank you very much! your are so helpful i am building exactly what is in your instruction and if i hopefully understand it fully i might be able to make some modification hehehe but since im a beginner, i will make exactly this.
by the way im louie from the Philippines and a computer engineering student.here thanks again and GOD Bless...
polymythic (author)  tolstoyan1 year ago
The diagram is in the instructable. I did not produce a schematic as such. You should be able to understand how to hook everything up by looking at the pin assignment in the code and making sense of the image in step 1:

http://www.instructables.com/id/Haptic-Feedback-device-for-the-Visually-Impaired/

Zoom in and the pins should be right.
im sorry if i am quite demanding hehe i dont want to put my components to waste if i made some mistake, coz items here in the philippines is quite expensive. anyway you really are helpful to your audience. wish you the best :)
polymythic (author)  tolstoyan1 year ago
No problem! The Maker Movement is about helping eachother out! Good luck from a friend in the United States!
last request can i have a copy of your schematic diagram :) tnx
axeman9111 year ago
I was thinking of doing this for a science fair, but much differently. Ur idea is way too bulky and only takes account for the upper-body not the lower.
1-40 of 123Next »