Introduction: Who Is at the Coffee Machine? Facial Recognition Using Raspberry Pi, OpenCV and Sigfox

By

Introduction

Have you ever wonder how facial recognition works? Have you heard of Sigfox? Do you like Raspberries?

In this tutorial, we will see how to develop a prototype using a Raspberry Pi to recognise faces with OpenCV and send the Id of the recognised people through Sigfox Network.

Who am I

My name is Louis Moreau, I'm an intern at Sigfox. My mission is to create prototypes and PoC using Sigfox network. I want these projects to be fun and instructive. I want to show you how easy is it to develop the first steps of your ideas. I will try to make this tutorial as complete as I can. Do not hesitate to ask me for more details if needed.

What is Sigfox

Sigfox is a connectivity solution dedicated to the Internet of Things. The operated network is currently operating in +15 countries, on every continent. Focused on tiny messages (up to 12 bytes) & low energy consumption, it currently powers 7 million devices. Various Sigfox-compatible technical solutions are available, from different silicon vendors. This project uses a shield SNOC (Raspberry Pi compatible).

In this tutorial, we will see that even 12 bytes can but more than enough to deal with video processing.

Contrib

To realise this tutorial, I have been using three main projects:

Step 1: Hardware Requirements

In this tutorial, we will be using:

  • A Raspberry Pi 2 or 3
    I built this project using the Raspberry Pi 2. However, in order to write the tutorial, take screenshots, or just try, I've been using the model 3. It should work on both raspberries then.
  • An SD card
  • Shield SNOC with an antenna: http://yadom.fr/carte-rpisigfox.html
  • Camera Raspberry
  • An LCD display:
    I used the Grove-LCD RGB backlight: https://www.seeedstudio.com/item_detail.html?p_id=...
  • Few wires
  • Some carton to make the box
  • To prepare the Raspberry Pi, you also need a screen with HDMI port, a mouse and a keyboard.

Step 2: Prepare Your Raspberry Pi

Install a fresh Raspbian image on your Raspberry Pi

Download the latest image: https://www.raspberrypi.org/downloads/raspbian/

Install the image on your SD card.

The installation guide on the official Raspberry Pi website is well explained. You just have to follow the procedure whether you are on:

Configure your Raspberry Pi

Once you have prepared your SD card with a fresh raspbian, just insert it in the Raspberry Pi and plug it. You might need to plug the Raspberry Pi into a screen and add a mouse and a keyboard to configure it. If you have a Raspberry Pi 2, you also need an ethernet cable or a wifi USB adapter.

Now, the first thing you should do is connect your raspberry Pi to the internet and run in the terminal:

$ sudo apt-get update

$ sudo apt-get upgrade -y

$ sudo reboot

Note that the default password is "raspberry"

Now go the "Menu" -> "Preferences" -> "Raspberry Pi Configuration" and go the "Interfaces" tab

We need to enable:

- Camera
- SSH
- I2C
- SPI

Get the source files

Clone the project's repository:

https://github.com/luisomoreau/sigfox_face_recogni...

$ sudo git clone https://github.com/luisomoreau/sigfox_face_recogni...

Step 3: Plug the LCD Display

Installation

I cloned the https://github.com/youngage/grove-lcd python libraries under the lcd/ folder in the project's files

Run in the terminal:

$ sudo apt-get install python-smbus

$ sudo apt-get install i2c-tools

Test

To test if the LCD display works fine, go to the project's folder and run the lcd/backlight.py script:

$ sudo python lcd/backlight.py

The result should be like the following video:

Step 4: Recognise Faces With OpenCV

What is openCV?

OpenCV means Open Source Computer Vision. It is this library that will allow use to detect faces and then recognise the person.

In this step, we will need to run three scripts:

  • The first script will allow to capture pictures using the Raspberry Pi camera and save it with the right colours and dimensions
  • The second script will train the OpenCV algorithm using the EigenFaces method
  • The third script will allow us to test the results and eventually adapt different parameters

I adapted under the opencv/ folder some parts of Tony DiCola's code from his Treasure Box tutorial: https://learn.adafruit.com/raspberry-pi-face-recog...

Install OpenCV

We are going to use the OpenCV python libraries. At first, I tried to compile it myself but I couldn't really make it. I spent hours trying different tutorials but with no success.

Finally, I found that OpenCV could be installed using apt-get. I read that few features are missing but at least it is enough for what we want to do!

(https://www.raspberrypi.org/forums/viewtopic.php?f...)

Open your terminal again and run:

$ sudo apt-get install python-dev

$ sudo apt-get install python-opencv

$ sudo apt-get install libopencv-dev

$ sudo pip install imutils

$ sudo pip install numpy

However, if you wish to compile it yourself, feel free to have a look at pyimage website: http://www.pyimagesearch.com/2015/02/23/install-op...

It can be a bit long, though.

Test the installation

In your terminal run:

pi@raspberrypi:~ $ python
Python 2.7.9 (default, Mar 8 2015, 00:52:26) [GCC 4.9.2] on linux2 Type "help", "copyright", "credits" or "license" for more information.

>>> import cv2

>>>

If you don't see any errors, OpenCV has been installed correctly.

Now press ctrl + d to exit

Capture pictures to be recognise

First, plug the Raspberry Pi camera in the RPI.

Go to opencv/ directory.

Then, open the capture_positive.py file and replace the POSITIVE_FILE_PREFIX = 'username_' by your name.

Save the file.

Run the script: $ python capture_postive.py

Fix the camera and press "c" and "Enter" to capture an image. If your face is detected and is the only one detected in the picture, the script will convert it with the right format (cropped + in grey scale + good dimension) in the training/positive folder.

Try to take around 10 pictures for each person. You may want to take it at different moments of the day to take into account the brightness evolution and to take different angles of your face.

Press "ctrl + c" to exit the script if you want to change the person name.

Note that if the script has trouble to detect your face you can change the value of the HAAR_SCALE_FACTOR in config.py

Prepare the training

Now open the config.py file and replace the provided USERNAME_LABEL by the names you used to take pictures.

Run the training script

Open the train.py script in the main function and replace the if(path='username'): by your name as you wrote it in the capture_positive.py file and the config.USERNAME_LABEL by the one you wrote in the config.py file.

Do it for each people, save the file and then run it:

$ python train.py

Depending on how many people you want to train, the script can take some time to run.

Once it is finished, you should see the training.xml file, the positive eigenface image, the negative eigenface image and the mean one.

Test the facial recognition

Go back to the project's root folder and run the test.py

The LCD display should write "Hello [your-name]". Depending on the pictures you took, errors might occur. You can put a lower confidence to have better results.

Step 5: Extend With Sigfox Connectivity

Configuration

I followed the following tutorial https://github.com/SNOC/rpisigfox

You can find this files under the sigfox/ folder in the project's files

Below is a copy/past of the requirements:

Disable Raspberry Pi terminal on serial port with raspi-config utility:

$ sudo raspi-config

9 Advanced Options >> A8 Serial >> NO

Install pyserial

$ sudo apt-get install python-serial

Pi3 requirements

In '/boot/config.txt' disable if present 'dtoverlay=pi3-miniuart-bt' by adding a '#' character at line beginning

Add if necessary :

dtoverlay=pi3-disable-bt

enable_uart=1

Then reboot :

$ sudo reboot

Send your first message

Now it's time to send your first Sigfox message:

go to the project's folder and run the script:

$ python sigfox/sendsigfox.py [message] [path to the serial port]

You can pass two arguments:

- Your message: In the screenshot, you can see I tried to send CAFE, 1234CAFE.
It's the message in hexadecimal up to 12 Bytes

- The path to the serial port: The default port is /dev/ttyAMA0

Activate your RPISIGFOX shield

To activate your dev kit, go to https://backend.sigfox.com/activate and choose your provider.

Depending on your location, pick your country and then fill the device's ID, the PAC number and your details. Both ID and PAC number are written on your dev kit, your box or has been given by your provider.

Check your emails and log in. Your device will appear in your account.

See Sigfox messages

Click on your device's ID and go to messages to check if you received anything.

Edit: To facilitate sending messages, I updated the sendsigfox.py from the official rpisigfox repository to create an object reusable in the project

Step 6: All Together

Now every that every part of this tutorial is available, you just need to gather everything to make it work!

We create a new python script, I called it run.py where I copied the test.py code.

Disable the video stream

In our case, we won't need the video so we will comment the lines where the video is concerned.

Send the IDs with Sigfox

Then, we want to count the number of people recognised by the device, add their ID to a list and then send this list using Sigfox.

For each face recognised, add them to an empty list if it is not present already.

Sigfox can send 12 bytes per message. You can store up to 256 people in your database (I would not recommend it, we're dealing with a Raspberry Pi) and send up to 12 different people each time.

Sigfox allows you to send one message every 10 minutes. So we configure a timer to send this list every 10 minutes and then reset the list.

Your script as a service

You can also run your script as a service to run it using ssh and then kill your terminal or autorun the script on reboot:

Add the following lines:

cd /home/pi/Documents/tuto/sigfox_face_recognition
sudo python run.py
cd /

Then, make this script executable:

$ chmod 755 launcher.sh

You can test the script to check everything went well by hitting:

$ sh launcher.sh

Save the logs in a file:

$ cd ~
$ mkdir logs

Now we will create a Cron task:

$ sudo crontab -e

Add the following line:

@reboot sh /home/pi/Documents/tuto/sigfox_face_recognition/launcher.sh >/home/pi/logs/cronlog 2>&1

This last line will launch the script on reboot and save the logs under the logs directory in the cronlog folder.

Now you can reboot and check if everything works fine by checking the logs.

Step 7: Going Further - Slack Integration

And now what?

From now on, the possibilities and endless.

We haven't discussed about setting a backend service. You can have a look on my other tutorials to know how to send a twit when you receive a Sigfox message or to send a message on a Slack channel.

Slack integration

In my case, I think I will create a channel on Slack called #coffeeMachine and send who is at the coffee machine on this channel to share a coffee for those who are interested. Let your imagination guide you ;)

Go to https://your-team.slack.com/apps/manage/custom-integrations

Click on "Incoming WebHooks"

Click on "New Integration"

Click on "Create a channel"

Choose your channel and click on "Add Incoming WebHook"

Create a custom callback in Sigfox backend to post a text message on slack given the url provided

Back in Slack, fill the form and you are ready to go :)

Firebase Integration

Another possibility is to store the received data in Firebase and to create a complete application around the project. This surely goes beyond the scope of this tutorial.

Feel free to modify, adapt, share and let me know what you did with the project.

Thank you for your attention :) I will be glad to answer your questions.

Automation Contest 2016

Second Prize in the
Automation Contest 2016