TTGO T-Watch Mnist Dataset Experiment With or Without TTGO T-Watch APIs

As a continuation of my previous TTGO T-Watch experiment as described in my post Starting an ESP32 Experimentation Journey With TTGO T-Watch, this post will demonstrate a way to use the TFT touchscreen of the TTGO T-Watch to implement a very simple UI for the Mnist Dataset, as in Mnist Dataset -- From Training to Running With ESP32 / ESP32S3.

It is often simpler to use the APIs provided by TTGO T-Watch. Nevertheless, I hope to demonstrate that this Mnist Dataset experiment can also be implemented without using TTGO T-Watch APIs.

Step 1: Preparation With VSCode/PlatformIO

Like last time, this time I will also be using VSCode/PlatformIO for the development.

The project folder is something like TWatchExperiments
In the project folder, there is the sub-folder src, which is the root for all source files.
In the sub-folder src, the file src/main.cpp is the entry point to the sketch/program to build.
In the project folder, there is the file platformio.ini for PlatformIO configuration

Since the same sketch/program will be built for "with APIs" and "without APIs", two PlatformIO environments will be stated in platformio.ini:

[platformio]
default_envs =tw3

[env]
monitor_speed = 115200

[env:tw3]
platform = espressif32
board = ttgo-t-watch
framework = arduino
lib_deps =
    https://github.com/Xinyuan-LilyGO/TTGO_TWatch_Library
    tanakamasayuki/TensorFlowLite_ESP32@^1.0.0
    https://github.com/aselectroworks/Arduino-FT6336U
build_flags = 
    -D LILYGO_WATCH_2020_V3

[env:tw3-bare]
platform = espressif32
board = ttgo-t-watch
framework = arduino
lib_deps =
    tanakamasayuki/TensorFlowLite_ESP32@^1.0.0
    https://github.com/lewisxhe/AXP202X_Library
    bodmer/TFT_eSPI
    https://github.com/lewisxhe/FocalTech_Library
build_flags = 
    -D TWATCH_BARE

Two environments are defined -- tw3 and tw3-bare
The environment tw3 is for using TTGO T-Watch APIs. Notice the dependency https://github.com/Xinyuan-LilyGO/TTGO_TWatch_Library
The environment tw3-bare is for not using TTGO T-Watch APIs; in such a case the macro TWATCH-BARE is defined

There is one more extra step for setting up for not using TTGO T-Watch APIs case -- modify TFT_eSPI's User_Setup_Select.h to select Setup45_TTGO_T_Watch.h, like .pio\libdeps\tw3-bare\TFT_eSPI\User_Setup_Select.h

...
//#include <User_Setup.h>           // Default setup is root library folder
...
#include <User_Setups/Setup45_TTGO_T_Watch.h>      // Setup file for ESP32 and TTGO T-Watch ST7789 SPI bus TFT  240x240
...

comment out #include <User_Setup.h>
uncomment #include <User_Setups/Setup45_TTGO_T_Watch.h>

Step 2: Start With Blink Test

Before implementing for Mnist Dataset, I will start with a simple blink test sketch -- src/INO/tw_blink.ino -- that you can download here.

Here are some important differences between using APIs and not using APIs.

If TTGO T-Watch APIs are to be used, all you need to include is

  #include <LilyGoWatch.h>

However, if APIs are not to be used, "include section" will be like

  #include "axp20x.h"
  #include <TFT_eSPI.h>

  #define I2C_BUS_SDA   21
  #define I2C_BUS_SCL   22

  #define TWATCH_TFT_BL (GPIO_NUM_15)
  #define BL_CHANNEL    0

  AXP20X_Class*   power = new AXP20X_Class();
  TFT_eSPI        tft   = TFT_eSPI();

  #include "focaltech.h"

  #define TOUCH_SDA     (GPIO_NUM_23)
  #define TOUCH_SCL     (GPIO_NUM_32)

  FocalTech_Class* touch = new FocalTech_Class();

The header file apx20x.h is for AXP20X_Class. You will need it to power the TTGO T-Watch up [after reboot].
The header file TFT_eSPI.h is for TFT_eSPI. You will need it to use the TFT screen.
The header file focaltech.h is for FocalTech_Class. You will need it to use the touch layer on the TFT screen.

If TTGO T-Watch APIs are to be used, initialization is basically done with the single API call -- TTGOClass::begin()

  // create the T-Watch API object 
  TTGOClass *ttgo = TTGOClass::getWatch();

  // init the T-Watch API object and turn on backlight
  ttgo->begin();
  ttgo->openBL();

  // from the T-Watch API object, get the TFT_eSPI object
  TFT_eSPI& tft = *ttgo->tft;

Notice that you get the TFT_eSPI object view the TTGOClass object ttgo.

However, if APIs are not to be used, you will need to initialize the different components of the T-Watch individually

  // some power management initialization
  Wire.begin(I2C_BUS_SDA, I2C_BUS_SCL);
  power->begin();
  power->setPowerOutPut(AXP202_LDO2, AXP202_ON);

  // init TFT
  tft.init();

  // turn on backlight
  ledcSetup(BL_CHANNEL, 1000, 8);
  ledcAttachPin(TWATCH_TFT_BL, BL_CHANNEL);
  ledcWrite(0, 255);

  // init touch
  Wire1.begin(TOUCH_SDA, TOUCH_SCL);
  touch->begin(Wire1);

Notice that the tft object is declared previously in the "include section" like

  TFT_eSPI    tft  = TFT_eSPI()

The code to check and get where the TFT screen is touched is similar for both using APIs and not using APIs cases

#if !defined(TWATCH_BARE)

  int16_t x, y;
  if (ttgo->getTouch(x, y)) {
    Serial.println(String("- TOUCH: ") + x + " " + y);
  }

#else

  uint16_t x, y;
  if (touch->getPoint(x, y)) {
    Serial.println(String("- TOUCH: ") + x + " " + y);
  }

#endif

The TTGO T-Watch is packed with a list of features that I didn't use in this simple blink test sketch, including

vibrate
dim screen
sleep

Nevertheless, the above-mentioned features will be used in the Mnist Dataset UI sketch next.

Assuming you have copied the sketch to your project src/INO/tw_blink.ino, to build it, src/main.cpp can just be the following line

#include "INO/tw_blink.ino"

You should be able to build the sketch with either one of the two environments -- tw3 or tw3-bare

Step 3: Mnist Dataset UI Sketch

The Mnist Dataset DL implementation using ESP32 with TensorFlow Lite is actually the same as described in Mnist Dataset -- From Training to Running With ESP32 / ESP32S3. Hence, if you want more details on the DL model, please refer to that post.

You can download the sketch -- src/INO/MNIST/tw_mnist.ino -- here.

And you can download the model -- src/INO/MNIST/mnist_model.h -- here.

You are to write a digit (0 to 9) on the TFT screen. Shortly after you write a digit (i.e. you stop touching the screen for a second), your T-Watch starts to predict what digit you have written and then shows the prediction result on the screen. Notice that after prediction, your T-Watch will also vibrate a bit.

After the prediction result is shown, you can start to write another digit for prediction.

If you do not write on the screen for more than 30 seconds, the screen will turn dim. After 30 seconds more of inactivity, your T-Watch will put itself in sleep mode. You bring your T-Watch out of sleep mode by touching the TFT screen once.

Assuming you have copied the sketch to your project src/INO/MNIST/tw_mnist.ino, and you have also copy the model src/INO/MNIST/mnist_model.h, to build it, src/main.cpp can just be the following line