Introduction: Prosopagnosia
Prosopagnosia is the problem of not recognizing faces. A good way to illustrate that is to call it "Face blindness". Our goal in this project is to build a device that recognizes faces to help people that suffer from this cognitive disorder.
Step 1: Material
So first, what do we need ?
- A Raspberry Pi 3 and a way to control it (either via SSH or a keyboard + screen, there are plenty of tutorials out there that treat this subject)
- A camera. The project has been tested with an official Raspberry Pi camera
- Three buttons
- At least one resistor (380 ohms)
- Some wires and a breadboard
- A Headphone set and a microphone and an USB-audio adapter
Step 2: Setting Up Your Raspberry Pi
Go to https://www.raspberrypi.org/downloads/raspbian/ and download the latest Raspberry Pi image.
Copy the image to an SD card. You can use https://www.raspberrypi.org/documentation/install... to help yourself with that step.
To install OpenCV it is best to have internet on the Raspberry Pi.
Use those ressources to do that :
https://cdn-learn.adafruit.com/downloads/pdf/adafr...
https://www.raspberrypi.org/documentation/configur... (for wifi)
You would also need to install the wiringPi library.It provides an easy-to-use interface to acces the GPIOs of the RaspberryPi from a C++ program.
Download the latest version of wiringPi from the following link: https://git.drogon.net/?p=wiringPi;a=summary
You will then need to go the directory where you downloaded wiringPi, extract the archive and build it.
$ cd
$ tar xfz wiringPi-98bcb20.tar.gz
$ cd wiringPi-98bcb20
$ ./build
To see if the installation is successful run the following commands:
$ gpio -v
$ gpio readall
Step 3: Install OpenCV
Now we assume that the Raspberry Pi has some kind of distribution installed into it.
For this part, you're going to need an internet connection for the Raspberry Pi. There is a bunch of tools to install to install OpenCV :
sudo apt-get install build-essential
sudo apt-get install cmake
sudo apt-get install pkg-config
sudo apt-get install libpng12-0 libpng12-dev libpng++-dev libpng3
sudo apt-get install libpnglite-dev libpngwriter0-dev libpngwriter0c2
sudo apt-get install zlib1g-dbg zlib1g zlib1g-dev
sudo apt-get install pngtools libtiff4-dev libtiff4 libtiffxx0c2 libtiff-tools
sudo apt-get install libjpeg8 libjpeg8-dev libjpeg8-dbg libjpeg-progs
sudo apt-get install ffmpeg libavcodec-dev libavcodec52 libavformat52 libavformat-dev
sudo apt-get install libgstreamer0.10-0-dbg libgstreamer0.10-0 libgstreamer0.10-dev
sudo apt-get install libxine1-ffmpeg libxine-dev libxine1-bin
sudo apt-get install libunicap2 libunicap2-dev
sudo apt-get install libdc1394-22-dev libdc1394-22 libdc1394-utils
sudo apt-get install swig sudo apt-get install libv4l-0 libv4l-dev
sudo apt-get install python-numpy
sudo apt-get install libpython2.6 python-dev python2.6-dev
sudo apt-get install libgtk2.0-dev pkg-config
Now we download OpenCV :
wget https://github.com/Itseez/opencv/archive/2.4.11.z...
Once it has been downloaded, extract the content :
unzip 2.4.11.zip
Run those commands to choose the installation options :
cd opencv-2.4.11
mkdir release
cd release
cmake ../
Type c then g to generate and exit the configuration files.
Use make to start compilation.
Once it's done, run the installation process :
sudo make install
Step 4: Camera Installation
Plug the Camera.
Enable the Camera :
We need to enable the camera . For this, type in a terminal :
sudo raspi-config Go to menu 6 and pick Enable Camera from Overclock option.
To test the camera you can use the raspistill command :
raspistill -o photo.jpg
The red LED on the Pi camera should turn on and a photo file should appear.
Step 5: Set Up the Sound Interface
If you want a full audio interface (so that the project is suitable for blind people), you're going to need an USB-audio adapter. You will then be able to plug a microphone and a headset into your Raspberry Pi.
The microphone will let the user name the person and the sound will let the user know the results of the recognition.
Step 6: Using Text-to-speech
As we want an audio interface, it will be necessary to generate audio files to inform the user that the training has been done and the different instructions on how to use the device.
You can use espeak to generate the audio files.
To install it :
sudo apt-get install espeak
To generate an audio file with it, prepare your text in a text file (for example text.txt) and then use this command :
espeak -f text.txt -w text.wav
Use aplay (aplay text.wav) to listen to it.
Step 7: Preparing the Breadboard
Just reproduce the circuit that is shown in the schematics.
If you're short on resistors, you can use just one if you plug it via the 3.3V.
Step 8: Compile & Run the Code
Download the package containing the code :
Extract the content and go into the Prosopagnosie directory.
Use the make command to compile
Execute ./picam to run the program.
You can now have fun with the device ! It will recognize faces and tell their names while the blue button is pushed.
You can register new faces by pushing the green button at start (push it to record the name with the mike, then push the red button for the photoshoot).
Photos are stored in ressources/photos. Sounds are stored in ressources/sons.
Details on the algorithm :
The code is based on a Finite State Machine (FSM). The machine have different states wich result in different outputs. Each of these states can be seen as a different step of the algorithm and result in the different behaviors.
Attachments
Step 9: Extra Step
You can modify the etc/rc.local file to have your program run whenever the raspberry pi is on.
To do so :
sudo nano etc/rc.local
Add the command to launch the program before the exit command :
/path/to/Prosopagnosie/picam
And that's it !
Step 10: How to Use the Device ?
