Face Aware OSD Photo Frame

1,916

15

6

Introduction: Face Aware OSD Photo Frame

About: Make it yourself if you cannot buy one!

This Instructables shows how to make a photo frame with face aware On Screen Display (OSD).

The OSD can show time, weather or other internet information you want.

Step 1: Why OSD Photo Frame?

I have 2 photo clock projects at Instructables before:

https://www.instructables.com/id/ESP32-Photo-Clock...

https://www.instructables.com/id/Arduino-BiJin-ToK...

Both retrieve a photo with beauties and a time board from the Internet every minutes and display on the LCD.

Displaying beauties is good, but they all are strangers to me. How about using personal favorite photos and add current time and further instant information on it?

This project is exploring how to make it.

Step 2: Why Face Aware?

Let's check how to add instant information OSD to a photo first:

  1. Random select a photo from a specific folder
  2. Retrieve time
  3. Retrieve instant information from Internet
  4. draw time and instant information on the photo

Step 1-3 is straight forward; Step 4 also look simple, but determine where to draw the text is not so easy.

If the text size too small, it is hard to read in a reasonable distance; If the text size too big, it most likely cover the photo objects. Especially if it is portrait photo, text covered the faces is not preferred.

Since the position of faces for each photo not the same, to avoid OSD covered faces we need a face detect process first. Then we can find a no-face-area to draw the text.

Step 3: 2-tiers Design

The face detect process require some processing power, in contrast, photo frame can be very light weight. So I split it into 2-tiers:

Server

The face aware photo engine is a Node.js app server. For each HTTP request, it will:

  1. Select a photo from the photo folder randomly
  2. Face detection
  3. determine no face or least faces area
  4. At the mean time, retrieve weather or other useful instant information from the Internet every certain period
  5. Draw time and instant information on the photo
  6. Return the photo with OSD in JPEG format as HTTP response

Client

The client can be a web browser, an applet or an IoT device.

E.g. an ESP32 dev boasrd with a 2-4 inches LCD is very fit for placing on the desktop as a tiny photo frame.

Step 4: Setup Photo Server Option 1: Docker Image

For convenience, I have pre-builded a Docker image for the face aware photo OSD Node.js app server.

In case you are not yet setup Docker, please follow the Docker Get Started guide:

https://www.docker.com/get-started

Then run the following command: (replace /path/to/photo to your own photo path)

docker run -p 8080:8080 -v /path/to/photo:/app/photo moononournation/face-aware-photo-osd:1.0.1

Test it by browse to http://localhost:8080/

You may find the displaying time is not in your time zone:

docker run -p 8080:8080 -e TZ=Asia/Hong_Kong -v /path/to/photo:/app/photo moononournation/face-aware-photo-osd:1.0.1

If you live in Hong Kong like me, you can add Hong Kong weather information:

docker run -p 8080:8080 -e TZ=Asia/Hong_Kong -e OSD=HK_Weather -v /path/to/photo:/app/photo moononournation/face-aware-photo-osd:1.0.1

If you would like to develop your own OSD information:

mkdir -p ~/git
cd ~/git
git clone https://github.com/moononournation/face-aware-photo-osd.git
docker run -it -p 8080:8080 -e TZ=Asia/Hong_Kong -e OSD=HK_Weather -e DEBUG=Y -v /path/to/photo:/app/photo -v ~/git/face-aware-photo-osd/app.js:/app/app.js moononournation/face-aware-photo-osd:1.0.1

Modify update_osd() function in app.js to tailor-made your own OSD information. After development, simply remove DEBUG=Y environment from the docker command.

Step 5: Setup Photo Server Option 2: Build From Source

If you are familiar with Node.js, you can build the app server from source.

Get the source:

git clone https://github.com/moononournation/face-aware-photo-osd.git

Install packages:

cd face-aware-photo-osd

npm install

Create photo folder and copy your own photos to the folder.

Run app server:

node app.js

Step 6: Client Option 1: Web Browser

Simply browser to http://localhost:8080/

The page is scripted auto load a fit page size image every minute.

P.S. If you browse from another machine that are not running the app server, remember change localhost to the app server hostname or IP address.

Step 7: Client Option 2: ESP32 + LCD

A photo frame client can be as simple as a ESP32 dev board and a LCD.

Here are the hardware required:

ESP32 Dev Board

Any ESP32 dev board should be ok, this time I am using a board called MH-ET LIVE.

LCD Display

Any Arduino_GFX supported LCD, you may find currently supported display at GitHub readme:

https://github.com/moononournation/Arduino_GFX

Jumper Wire

Some Jumper Wires, depends on the dev board and LCD pins layout. In most case 6-9 female to female jumper wires are enough.

LCD Stand

Some support help the LCD standing straight, this time I am using a card holder stand.

Step 8: ESP32 + LCD Assembly

The ESP32 with pin header on the upper side is preferred. If the pin header at the lower side, simply put the board upside down ;>

Connect ESP32 and LCD with jumper wires, then fit it to the stand.

Here are the sample connection summary:

ESP32   -> LCD
Vcc     -> Vcc
GND     -> GND
GPIO 5  -> CS
GPIO 27 -> DC (if available)
GPIO 33 -> RST
GPIO 18 -> SCK
GPIO 19 -> MISO (optional)
GPIO 22 -> LED (if available)
GPIO 23 -> MOSI / SDA

Step 9: ESP32 + LCD Software

Arduino IDE

Download and install Arduino IDE if you are not yet do it:

https://www.arduino.cc/en/main/software

ESP32 Support

Follow the Installation Instructions to add ESP32 support if you re not yet do it:

https://github.com/espressif/arduino-esp32

Arduino_GFX Library

Download latest Arduino_GFX libraries: (press "Clone or Download" -> "Download ZIP")

https://github.com/moononournation/Arduino_GFX

Import libraries in Arduino IDE. (Arduino IDE "Sketch" Menu -> "Include Library" -> "Add .ZIP Library" -> select downloaded ZIP file)

Compile & Upload

  1. Open Arduino IDE
  2. Open ESP32PhotoFrame sample code ("File" -> "Example" -> "GFX Library for Arduino" -> "WiFiPhotoFrame")
  3. Fill your WiFi AP settings into SSID_NAME and SSID_PASSWORD
  4. Replace your server hostname or IP and port in HTTP_HOST and HTTP_PORT
  5. Press Arduino IDE "Upload" button
  6. If you find the orientation not correct, change the "rotation" value (0-3) in new class code

Step 10: Enjoy Photo!

It's time to put the IoT photo frame on your desktop and enjoy!

Step 11: What's Next?

  • Add your own instant information
  • Fine tune the source photo size for better face detection accuracy
  • Automatic task to put latest photos to the server photo folder
  • Take more photos ;>

Be the First to Share

    Recommendations

    • Metal Contest

      Metal Contest
    • First Time Author Contest

      First Time Author Contest
    • DIY Summer Camp Contest

      DIY Summer Camp Contest

    6 Comments

    0
    nobitasayhi
    nobitasayhi

    1 year ago

    Hi 陳亮,
    pls. help me. I use your code "ESP32PhotoFrame" in esp8266 and monitor serial port, the connection it does not work because of return code -1.
    server ok and open in chrome it's work.
    many thanks
    note

    error.png
    0
    nobitasayhi
    nobitasayhi

    Reply 1 year ago

    The picture not show display on LCD => ILI9488 ???

    0
    nobitasayhi
    nobitasayhi

    Reply 1 year ago

    OK, now I can connect. It's can not use name server, use IP Address is success.

    0
    hichm
    hichm

    1 year ago

    OK after your replay 陳亮, i post it here:
    i have LCD 2.1 inch (6cm)
    W = 36.72 mm, H = 48.96 mm, 36 pins :
    FPC-X024N12-36A
    2016/10/20
    YH

    https://i.pinimg.com/564x/c9/69/2f/c9692f2fa2605d0d2285fd02da79e1ed.jpg

    i find a TFT LCD Screen, with 36pins same size but other mark, 240 x 320 (RGB)
    PIN No | Name
    01 | TE
    02 | NC
    03 | NC
    04 | NC
    05 | NC
    06 | RESET
    07 | VSYNC NC
    08 | HSYNC NC
    09 | DOTCLK NC
    10 | DEN NC
    11 | DB17
    12 | DB16
    13 | DB15
    14 | DB14
    15 | DB13
    16 | DB12
    17 | DB11
    18 | DB10
    19 | DB09
    20 | DB08
    21 | DB07
    22 | DB06
    23 | DB05
    24 | DB04
    25 | DB03
    26 | DB02
    27 | DB01
    28 | DB00
    29 | RD
    30 | WR
    31 | RS
    32 | CS
    33 | GND
    34 | VCC
    35 | LED-K
    36 | LED-A

    i try with multi-meter to check if the ground pin is the same as the diagram, i found it same, is that enough to conclusion that this pins are for my LCD?

    0
    陳亮
    陳亮

    Reply 1 year ago

    If the pins layout is correct, it is not a SPI display and not supported by Arduino_GFX yet.

    0
    jessyratfink
    jessyratfink

    1 year ago

    That's so nice! I love the simplicity of the frame.