WatchEye - One Eyed Pet With Face Tracking

Intro: WatchEye - One Eyed Pet With Face Tracking

WatchEye - One-eyed Pet With Face Tracking

I enjoyed the Instructable "Twitch" very much and it was very inspiring: an electronic pet that reacts to touch and blink with its eye.

But I wanted a pet that could really look at me.

My project looks like "Twitch - Your Robotic Pet" and I copied the blinking eye, but the inner life and function are quite different. Here is WatchEye

STEP 1: The Electronics: ESP32-cam

The ESP32-cam does not have a USB launch. For the upload, I recommend an adapter, but you can also do it yourself with an FTDI adapter and a breadboard.

A face recognition program is integrated in the ES32-sketch. Among everything the camera sees, the shape "face" is recognised and imaginarily surrounded with a frame. It does not recognise your face, but a face

If this frame is in the middle of the picture and WatchEye is looking straight ahead, everything is fine.

If the frame is e.g. on the right side WatchEye is moving to the right until the frame is in the center again.

For the image size I use VGA in the sketch. You can also try SVGA, but the image processing will then be quite slow.

config.frame_size = FRAMESIZE_VGA;

For setting up and programming I recommend this instructable:

https://www.instructables.com/Getting-Started-With-ESP32-CAM-Streaming-Video-Usi/

STEP 2: Electronics: LED Matrix

An eye is depicted on the matrix, which moves and blinks at random. This makes the whole thing come alive.

Yes, I copied that from Twitch and changed it a little. Thank you, drum303.

STEP 3: Pan & Tilt & Servos

I built a pan&tilt device myself in 2020 from plastic sheets. But a pan&tilt kit is easier to handle and not expensive. It can hold the weight of the PCB, ESP32 and LED matrix.

STEP 4: Electronics: PIR Sensor

Our pet should sleep and save power when we are not there. As soon as someone enters the room, WatchEye wakes up from a deep sleep and turns his eye to the visitor. The eye follows our movements.

STEP 5: Software

Some parts of the programme I found at https://www.instructables.com/Twitch-Your-Robotic-Pet/ and

https://techtutorialsx.com/2020/06/13/esp32-camera-face-detection/

For setting up and programming I recommend this instructable:

https://www.instructables.com/Getting-Started-With-ESP32-CAM-Streaming-Video-Usi/

Download the three files and put them all to one folder. Then open WatchEye.ino file with Arduino IDE. Include some Libraries:

Sketch --> Include Libraries --> Manage Libaries

Enter "Adafruit LED" and install "Adafruit LED Backpack Library" and depending libraries.

Send the sketch via USB and a COM port to the ESP32-cam.

STEP 6: Putting All Together

Now it is up to you to put everything together. I soldered a board with ESP32-cam, LED matrix and connections for the servos. The board is attached to the pan&tilt mechanism.

STEP 7: Housing

I built a plywood box for the housing. Cardboard, plastic or an (insulated) metal box would also work. Don't forget the switch and enough space for the battery. The PIR sensor is placed at the front to detect visitors and wake up WatchEye .

For a better look I made a cover out of black cardboard for the eye.

STEP 8: Next Step: Animatronic Eye Mechanism?

... with cameras in the eyeballs.

Comments

[pingsinoca]

Hello author, I like this project. After compiling the esp32 cam and uploading it successfully, I connected it correctly according to your wiring diagram and I am close to the PIR. The two Servos have no action, and the LED (i2c) eyeball lights remain the same and have no action. I have changed to an esp32 cam development board, but they are still the same and have no action. I changed the detection pixel (VGA) in your code. ~QVGA) I moved closer to the cam, but there was still no movement. Please, please help, thank you

[Markus Opitz]

Hello pingsinoca,
so nice that you want to rebuild the project. Please excuse this questions, but did you check all contacts, are pin numbers in program and pins used identical? Are the pins on the ESP32 cam well soldered? I am only asking about the mistakes I made myself.
Does the power supply provide enough amps for the servos?
Are there any error messages? What do the Serial.print messages in the Serial Monitor say? I could not find any error in the program.
Feel free to contact me at [e-mail]
Kind regards

[jojo0910]

Hi, I wanted to build this project myself as well but I am unsure about the power supply. What kind of battery would you recommend in terms of voltage, current and amp/hrs? Thanks

[Markus Opitz]

Hello, nice that you want to do the project!
The power supply is quite simple. I use a 2000 mAh 5V powerbank myself. The servos need 5V and also the ESP32-cam to run properly. The power consumption is not very high, although the device also has an automatic switch-off function and can go into deep sleep.
If you want to make the project smaller, I would recommend a 1000 mAh lipo battery with a 5V step-up board. Another suggestion would be to use the new ESP32S3 from XIAO instead of an ESP32-Cam as described in
https://www.instructables.com/Camera-NanoTank/ . I just got one today. It gets a bit hot, but it's a great device.
Good luck with the rebuild!
Markus

[jojo0910]

Hello and thank you for the quick response. I'm considering the bigger powerbank, but it makes a somewhat silly little robot a bit more expensive as well. Anyways I will experiment with a few solutions now. Thanks for the help