WatchEye - One-eyed Pet With Face Tracking

I enjoyed the Instructable "Twitch" very much and it was very inspiring: an electronic pet that reacts to touch and blink with its eye.

But I wanted a pet that could really look at me.

My project looks like "Twitch - Your Robotic Pet" and I copied the blinking eye, but the inner life and function are quite different. Here is WatchEye

Supplies

ESP32-cam (with FTDI)

Pan & Tilt kit

2x SG90 servo motors

PIR sensor

LED matrix 8x8 (green)

switch

box

So that you are not confused:

Some of the pictures look slightly different from the description. I made this project in 2020, before I became a member of Instructables. I built my own pan and tilt mechanism back then.

Step 1: The Electronics: ESP32-cam

The ESP32-cam does not have a USB launch. For the upload, I recommend an adapter, but you can also do it yourself with an FTDI adapter and a breadboard.

A face recognition program is integrated in the ES32-sketch. Among everything the camera sees, the shape "face" is recognised and imaginarily surrounded with a frame. It does not recognise your face, but a face

If this frame is in the middle of the picture and WatchEye is looking straight ahead, everything is fine.

If the frame is e.g. on the right side WatchEye is moving to the right until the frame is in the center again.

For the image size I use VGA in the sketch. You can also try SVGA, but the image processing will then be quite slow.

config.frame_size = FRAMESIZE_VGA;

For setting up and programming I recommend this instructable:

https://www.instructables.com/Getting-Started-With-ESP32-CAM-Streaming-Video-Usi/

Step 2: Electronics: LED Matrix

An eye is depicted on the matrix, which moves and blinks at random. This makes the whole thing come alive.

Yes, I copied that from Twitch and changed it a little. Thank you, drum303.

Step 3: Pan & Tilt & Servos

I built a pan&tilt device myself in 2020 from plastic sheets. But a pan&tilt kit is easier to handle and not expensive. It can hold the weight of the PCB, ESP32 and LED matrix.

Step 4: Electronics: PIR Sensor

Our pet should sleep and save power when we are not there. As soon as someone enters the room, WatchEye wakes up from a deep sleep and turns his eye to the visitor. The eye follows our movements.

Step 5: Software

Some parts of the programme I found at https://www.instructables.com/Twitch-Your-Robotic-Pet/ and

https://techtutorialsx.com/2020/06/13/esp32-camera-face-detection/

For setting up and programming I recommend this instructable:

https://www.instructables.com/Getting-Started-With-ESP32-CAM-Streaming-Video-Usi/

Download the three files and put them all to one folder. Then open WatchEye.ino file with Arduino IDE. Include some Libraries:

Sketch --> Include Libraries --> Manage Libaries

Enter "Adafruit LED" and install "Adafruit LED Backpack Library" and depending libraries.

Send the sketch via USB and a COM port to the ESP32-cam.

Step 6: Putting All Together

Now it is up to you to put everything together. I soldered a board with ESP32-cam, LED matrix and connections for the servos. The board is attached to the pan&tilt mechanism.

Step 7: Housing

I built a plywood box for the housing. Cardboard, plastic or an (insulated) metal box would also work. Don't forget the switch and enough space for the battery. The PIR sensor is placed at the front to detect visitors and wake up WatchEye .

For a better look I made a cover out of black cardboard for the eye.