Introduction: Smart Glasses to Help the Blind, With Pivothead LiveModPro and Intel Edison
You want to do cool computer vision tricks with Intel Edison ? Yes, but why not work on a project to help the blind and put your coding skills to good use !
My project was to perform basic but robust computer vision tricks to help a blind person. Tricks such as :
- detect barcodes with zbar, get the description from a local source or from internet databases, read the text with epseak
- detect colors at the center of the frame, find the html and pantone names, read the name
- find colors that would look good with this color, to select matching clothes, and read them
- find faces in front of you, say where they are compared to you and how far. read the text in stereo to give spatial information on where the face is located
Step 1: Select the Glasses
First step is to pick a camera. Pivothead SMART glasses were an obvious choice. Image quality is awesome, hardware codec too, there's cheap and super light. Turns out they have an Intel Edison extension card called "LiveModPro" to do computer vision in the glasses, on battery. Perfect for my project.
Step 2: Install Edison
- flash edison with build v3
- setup the board with internet access
- install repositories from repo.opkg.netupgrade node, but not all the packages (kernel would not work well if you do)
- install packages : opkg install fswebcam nano espeak ffmpeg-x264-presets gps-utils htop git lighttpd ofono opencv opencv-dev opencv-staticdev opencv-apps opencv-dev opencv-samples opencv-samples-dev python-opencv python-pip python-numpy zbar mjpg-streamer gstreamer1.0-plugins-good-interleave gstreamer1.0-plugins-good-audiofx
- install node packages : npm install -g fs sleep tinycolor2 array-unique striptags color-namer color-scheme onecolor util request shelljs-nodecli linux-input-device canvas okrabyte ocra.js
- unpack the source from http://dl.free.fr/l31NwuXWp
- setup webserver on port 81, and root folder as /home/root/www/ by editing the file /etc/lighttpd.conf
- test the webcam :fswebcam -d /dev/video0 /home/root/www/shot.png and use your browser to see the file http://EDISON_IP:81/shot.png
- optional : setup a bluetooth headset or use headphones connected to the jack of the glasses
Step 3: Demo and Code
go to pivothead-intel and launch one of the demos, with a command like : node demo_barcode.js
you can hear the messages, or go in your web browser to see the debug interface (frame grab, faces detected, audio messages, ...)
Enjoy the video demo video and code comments
6 years ago
It's a great topic !
Would you mind sharing the resource about the Intel Edison extension card
7 years ago
You are great taking time out of your life to make this for those with disabilities May god bless you
7 years ago
Great! Keep with the good work. Congratulations!
7 years ago
That is a great tool. That could even be useful for people who are color blind too.