ESP32 CAM Face Recognition With MQTT Support | AI-Thinker

Hello!

I wanted to share my code for a project were I needed to have an ESP CAM with Face recognition, that could send out Data to MQTT. So well.. after maybe 7 Hours of looking trough multiple code examples and looking for what is what, I have finished the MQTT integration!

Supplies

What you need:

• ESP32 Camera (~5$)
• TTL programmer (~2$)
• 5 Jumper Cables

Step 1: Setting Up the TTL Programmer (OPTIONAL)

If you are using a Computer, that has a newer version than Windows 7, you most likely need the supported drivers for the programmer.

If the device is not able to start, then I recommend you this easy tutorial to install the drivers manually

Step 2: Setting Up the Connection and Configuring in the IDE

You need to wire it like in the picture (It is better to use 5v instead of 3v!)

The grey cable is only needed to be connected if you want to program it!

I am assuming you already have the ESP32 package installed, if so you need to roll the version back, from whatever you are using currently to version 1.01 this step is necessary for the Face recognition, otherwise it will not work!

The next step is to go into TOOLS in your IDE and set Partition Scheme to Huge APP, and select ESP32 Wrover Module as the Board!

Step 3: Code and Libraries

You will only need the PubSubClient Library, all other libraries are automatically installed.

(Don´t forget to change your credentials before uploading)

Download the attachment and click upload don´t forget the grey Cable mentioned above!

After Upload open the Serial Monitor and select 115200 as baud rate.

You should see a IP address to connect to. NOTE: The face recognition is already enabled at start, so it will scan your face! EDIT: It will NOT automatically do that anymore!

On the MQTT part you should see an info tab and one face tab. EDIT: And id tab

Step 4: That´s It | EDIT

This is it for now, I think I will be updating it some time if I see bugs or any other thing.

I hope you enjoyed!

EDIT:

I have made some changes to the code!

It now saves your Face in internal memory, so when you start it up, it will load all previously saved faces from Flash!

It automatically saves enrolled faces to Flash.

I also removed the auto Enroll face on startup.

I have made a new MQTT Topic called "id", this is where it shows the last ID recognized!

Comments

[snehasishtestengineer]

i was looking for the similar project but i want the face id to be sent to the MQTT local broker(running MQTT Broker/client in RaspberryPi).Please help

[ionlef]

Great project, how can I install dl_lib.h ?

[psbest]

Great project, learning a lot. The instructions are clear: >>I am assuming you already have the ESP32 package installed, if so you need to roll the version back, from whatever you are using currently to version 1.01 this step is necessary for the Face recognition, otherwise it will not work!<<
What is needed to make it work in 1.0.4, I tried to roll back but no luck, no errors and uploads fine, but the code will not run after boot. I also tried to convert the new camera example that works with 1.0.4 with this project, but failed, too complex for me.

[ftpgonzo]

The Code is Damage says Arduino :(

[psbest]

Had the same issue, when i removed the + in the filename and dir it worked!

[dobbler]

Hi, I'm loving this project and have it working successfully with my home automation. Am I right in thinking that the face recognition doesn't work unless the Stream is started on the web interface? Is there a way for this to happen automatically/on boot?

Thanks again!

[danielterletzkiy]

Yes you are right, I had solved the problem with installing a mjpg Library for Node Red, where it would just open the stream, then I dumped the Idea of having Face Processing on the ESP32 Cam and installed "FaceBox" on Docker it worked beatifully and the ESP32 didn´t have to stutter anyymore (Face Recognition on ESP32 turned off).
It worked well with Home Assistant, but I don´t know what Home Automation or HArdware you have.. :/

[hddev]

Hello, thanks a lot for the post here.
Does it mean, that the webserver does not run anymore on the esp32 cam and instead it runs on node red using the mjpeg?

How do you feed the "stream" data to mjpeg reside within the node red environment? Are these data raw data? Are they sent via MQTT to node red?

It will be interesting, to decouple the functions, e.g. 1. esp32 cam capturing, 2
Captured data from cam are processed in node red, 3. Face recognition etc. is then performed in docker etc

Thanks a lot!

[dobbler]

Nice.. Facebox looks very interesting, must check it out. I'm running Openhab for my home automation on a RPi3+.. Works well, save for a fair bit of maintenance.. but that's what we like to do! Thanks for the reply.

[tcsmaxx]

Hi,
can I do the face enrole via the web interface?
Nothing happens to me when I press 'Enroll Face'.

---------
Connecting to intern
[D][WiFiGeneric.cpp:336] _eventCallback(): Event: 0 - WIFI_READY
[D][WiFiGeneric.cpp:336] _eventCallback(): Event: 2 - STA_START
.....[D][WiFiGeneric.cpp:336] _eventCallback(): Event: 4 - STA_CONNECTED
[D][WiFiGeneric.cpp:336] _eventCallback(): Event: 7 - STA_GOT_IP
[D][WiFiGeneric.cpp:379] _eventCallback(): STA IP: 192.168.43.232, MASK: 255.255.255.0, GW: 192.168.43.1
.
WiFi connected
MQTT connected!
Friday, June 12 2020 16:44:17
SD_MMC Card Type: SDHC
SD_MMC Card Size: 30191MB
Total space: 30173MB
Used space: 0MB
<DBG>failed to open file for reading</DBG>
<DBG>Load face id list fail</DBG>
Starting web server on port: '80'
Starting stream server on port: '9601'
Camera Ready - Use 'http://192.168.43.232' to connect, the live stream is on a different port channel 9601
Stream Ready - Use 'http://192.168.43.232:9601/stream'
Image Ready - Use 'http://192.168.43.232/capture'

jpg-encode-stream !index (with FACE recognition pass here)
FACE: 11680B 425ms 0
jpg-encode-stream !index (with FACE recognition pass here)
FACE: 11948B 424ms 0

[danielterletzkiy]

Yes.. I don´t know any other way other than webinterface or some more coding

[csgt01]

Did you sort it out? I have the same problem. I know with some debugging that FACE_ID_PATH "/faceid.dat" is a directory. Why ever.

[candy0543]

Hi, I wanna do same project but i can't open code file which name of ESP32+CAM+MQTT. Could u plz send me an email or if you have any github leave comment below plz

[Elzershark]

how can i delete the face?

[zbcamera1]

cif camera doesn't work. Can't get face detection or face recognation on or off.
When i put the esp32 cam on. i see it in the mqtt list. So thats working.
Can you give some help?

[danielterletzkiy]

I cant really look into it right now but I think it doesnt support Face recognition at higher resolutions, I think that is why I set it to QVGA in the first place.
I hope this helps!

[zbcamera1]

Gonna try it again... Thanks.

Edit.
I've try it, but in QVGA or others, there's no face recognition.
Hoop you find the time to fix it...
Thanks.

Edit 2, Found the problem, my mistake, face recognation works.
Thanks.

[multimediaex]

Can you tell how you fixed the problem? Same problem here

[dobbler]

Yeah, me too!

[dobbler]

I managed to get it going by (reading the instructions again!) downgrading the ESP32 package in the Arduino IDE to 1.01, that seems to have done the trick for me..