ESP32 CAM Face Recognition With MQTT Support | AI-Thinker

Introduction: ESP32 CAM Face Recognition With MQTT Support | AI-Thinker

Hello!

I wanted to share my code for a project were I needed to have an ESP CAM with Face recognition, that could send out Data to MQTT. So well.. after maybe 7 Hours of looking trough multiple code examples and looking for what is what, I have finished the MQTT integration!

Supplies

What you need:

Step 1: Setting Up the TTL Programmer (OPTIONAL)

If you are using a Computer, that has a newer version than Windows 7, you most likely need the supported drivers for the programmer.

If the device is not able to start, then I recommend you this easy tutorial to install the drivers manually

Step 2: Setting Up the Connection and Configuring in the IDE

You need to wire it like in the picture (It is better to use 5v instead of 3v!)

The grey cable is only needed to be connected if you want to program it!

I am assuming you already have the ESP32 package installed, if so you need to roll the version back, from whatever you are using currently to version 1.01 this step is necessary for the Face recognition, otherwise it will not work!

The next step is to go into TOOLS in your IDE and set Partition Scheme to Huge APP, and select ESP32 Wrover Module as the Board!

Step 3: Code and Libraries

You will only need the PubSubClient Library, all other libraries are automatically installed.

(Don´t forget to change your credentials before uploading)

Download the attachment and click upload don´t forget the grey Cable mentioned above!

After Upload open the Serial Monitor and select 115200 as baud rate.

You should see a IP address to connect to. NOTE: The face recognition is already enabled at start, so it will scan your face! EDIT: It will NOT automatically do that anymore!

On the MQTT part you should see an info tab and one face tab. EDIT: And id tab

Step 4: That´s It | EDIT

This is it for now, I think I will be updating it some time if I see bugs or any other thing.

I hope you enjoyed!

EDIT:

I have made some changes to the code!

It now saves your Face in internal memory, so when you start it up, it will load all previously saved faces from Flash!

It automatically saves enrolled faces to Flash.

I also removed the auto Enroll face on startup.

I have made a new MQTT Topic called "id", this is where it shows the last ID recognized!

Be the First to Share

    Recommendations

    • Explore Science Challenge

      Explore Science Challenge
    • Colors of the Rainbow Contest

      Colors of the Rainbow Contest
    • Arduino Contest

      Arduino Contest

    27 Comments

    0
    ionlef
    ionlef

    Question 2 months ago on Step 3

    Great project, how can I install dl_lib.h ?

    0
    psbest
    psbest

    5 months ago

    Great project, learning a lot. The instructions are clear: >>I am assuming you already have the ESP32 package installed, if so you need to roll the version back, from whatever you are using currently to version 1.01 this step is necessary for the Face recognition, otherwise it will not work!<<
    What is needed to make it work in 1.0.4, I tried to roll back but no luck, no errors and uploads fine, but the code will not run after boot. I also tried to convert the new camera example that works with 1.0.4 with this project, but failed, too complex for me.

    0
    ftpgonzo
    ftpgonzo

    Question 1 year ago on Introduction

    The Code is Damage says Arduino :(

    0
    psbest
    psbest

    Answer 5 months ago

    Had the same issue, when i removed the + in the filename and dir it worked!

    0
    dobbler
    dobbler

    11 months ago

    Hi, I'm loving this project and have it working successfully with my home automation. Am I right in thinking that the face recognition doesn't work unless the Stream is started on the web interface? Is there a way for this to happen automatically/on boot?

    Thanks again!

    0
    danielterletzkiy
    danielterletzkiy

    Reply 11 months ago

    Yes you are right, I had solved the problem with installing a mjpg Library for Node Red, where it would just open the stream, then I dumped the Idea of having Face Processing on the ESP32 Cam and installed "FaceBox" on Docker it worked beatifully and the ESP32 didn´t have to stutter anyymore (Face Recognition on ESP32 turned off).
    It worked well with Home Assistant, but I don´t know what Home Automation or HArdware you have.. :/

    0
    hddev
    hddev

    Reply 5 months ago

    Hello, thanks a lot for the post here.
    Does it mean, that the webserver does not run anymore on the esp32 cam and instead it runs on node red using the mjpeg?

    How do you feed the "stream" data to mjpeg reside within the node red environment? Are these data raw data? Are they sent via MQTT to node red?

    It will be interesting, to decouple the functions, e.g. 1. esp32 cam capturing, 2
    Captured data from cam are processed in node red, 3. Face recognition etc. is then performed in docker etc

    Thanks a lot!

    0
    dobbler
    dobbler

    Reply 11 months ago

    Nice.. Facebox looks very interesting, must check it out. I'm running Openhab for my home automation on a RPi3+.. Works well, save for a fair bit of maintenance.. but that's what we like to do! Thanks for the reply.

    0
    tcsmaxx
    tcsmaxx

    1 year ago

    Hi,
    can I do the face enrole via the web interface?
    Nothing happens to me when I press 'Enroll Face'.

    ---------
    Connecting to intern
    [D][WiFiGeneric.cpp:336] _eventCallback(): Event: 0 - WIFI_READY
    [D][WiFiGeneric.cpp:336] _eventCallback(): Event: 2 - STA_START
    .....[D][WiFiGeneric.cpp:336] _eventCallback(): Event: 4 - STA_CONNECTED
    [D][WiFiGeneric.cpp:336] _eventCallback(): Event: 7 - STA_GOT_IP
    [D][WiFiGeneric.cpp:379] _eventCallback(): STA IP: 192.168.43.232, MASK: 255.255.255.0, GW: 192.168.43.1
    .
    WiFi connected
    MQTT connected!
    Friday, June 12 2020 16:44:17
    SD_MMC Card Type: SDHC
    SD_MMC Card Size: 30191MB
    Total space: 30173MB
    Used space: 0MB
    <DBG>failed to open file for reading</DBG>
    <DBG>Load face id list fail</DBG>
    Starting web server on port: '80'
    Starting stream server on port: '9601'
    Camera Ready - Use 'http://192.168.43.232' to connect, the live stream is on a different port channel 9601
    Stream Ready - Use 'http://192.168.43.232:9601/stream'
    Image Ready - Use 'http://192.168.43.232/capture'

    jpg-encode-stream !index (with FACE recognition pass here)
    FACE: 11680B 425ms 0
    jpg-encode-stream !index (with FACE recognition pass here)
    FACE: 11948B 424ms 0


    0
    danielterletzkiy
    danielterletzkiy

    Reply 1 year ago

    Yes.. I don´t know any other way other than webinterface or some more coding

    0
    csgt01
    csgt01

    Reply 6 months ago

    Did you sort it out? I have the same problem. I know with some debugging that FACE_ID_PATH "/faceid.dat" is a directory. Why ever.

    0
    candy0543
    candy0543

    Question 11 months ago

    Hi, I wanna do same project but i can't open code file which name of ESP32+CAM+MQTT. Could u plz send me an email or if you have any github leave comment below plz

    0
    Elzershark
    Elzershark

    Question 11 months ago

    how can i delete the face?

    0
    zbcamera1
    zbcamera1

    1 year ago

    cif camera doesn't work. Can't get face detection or face recognation on or off.
    When i put the esp32 cam on. i see it in the mqtt list. So thats working.
    Can you give some help?

    0
    danielterletzkiy
    danielterletzkiy

    Reply 1 year ago

    I cant really look into it right now but I think it doesnt support Face recognition at higher resolutions, I think that is why I set it to QVGA in the first place.
    I hope this helps!

    0
    zbcamera1
    zbcamera1

    Reply 1 year ago

    Gonna try it again... Thanks.

    Edit.
    I've try it, but in QVGA or others, there's no face recognition.
    Hoop you find the time to fix it...
    Thanks.

    Edit 2, Found the problem, my mistake, face recognation works.
    Thanks.

    1
    multimediaex
    multimediaex

    Reply 1 year ago

    Can you tell how you fixed the problem? Same problem here

    0
    dobbler
    dobbler

    Reply 12 months ago

    Yeah, me too!

    0
    dobbler
    dobbler

    Reply 12 months ago

    I managed to get it going by (reading the instructions again!) downgrading the ESP32 package in the Arduino IDE to 1.01, that seems to have done the trick for me..

    1
    eBär007
    eBär007

    1 year ago

    Can not complie the source, Lib 1.04 nor 1.01 both with errors.