A Bluetooth ESP32 TFT + Touch Macro Keypad)

29,848

452

106

Introduction: A Bluetooth ESP32 TFT + Touch Macro Keypad)

About: Hi! I'm Dustin. I'm a electronics enthusiast/hobbyist who likes to build projects around microcontrollers and sometimes all discrete components. Most of my instructables come with a YouTube video.

Controlling your computer by using hotkeys and macros is a great way of speeding up your workflow. Unfortunately a device dedicated to doing that (for example Elgato's Stream Deck) will cost you about $120.

That is why I designed FreeTouchDeck. FreeTouchDeck will cost you about $20 (including shipping!) to make. It uses an ESP32 and a 3.5" touch screen. Keystrokes and macros are send via Bluetooth to your Windows, MacOS, or Linux computer.

You can customize your FreeTouchDeck by using the configurator. The configurator is a webpage that is hosted on the ESP32 itself. You can customize menus, buttons, logos and colours. You can also create your own button logos and upload them.

This is a really simple build and will take up less then an hour of your time + a few cups of coffee. I designed it so it can be build using parts that can be easily found on AliExpress and Banggood but can also be found locally if you do not want to wait a while before the postman comes.

Supplies

  • A 38-pin ESP32-WROOM-32 Development Board AliExpress

  • An ILI9488 TFT screen with XPT2046 touch controller AliExpress

  • An optional ESP32 + TFT Combiner PCB

Tools:

  • A soldering iron and solder.
  • Wire strippers (if you are not using the PCB).
  • Flush cutter.

Step 1: Learn More About FreeTouchDeck

Before you start this project it might be a good idea to watch the video I made. The video in itself isn't a how-to, but more of an introduction to what FreeTouchDeck is and what it can do for you!

Watched the video? Let's get the parts you'll need!

Step 2: Get the Parts You'll Need

If you do not already have an ESP32 and an ILI9488 TFT + Touchscreen lying around. You can find them pretty cheap on AliExpress. I used these parts:

- A 38-pin ESP32-WROOM-32D Development Board from AliExpress*

- An ILI9488 TFT screen with XPT2046 touch controller from AliExpress*

- An optional ESP32 + TFT Combiner PCB (Gerber file)

Not all ILI9488 TFT screens are created equally. It is important to make sure you have selected a screen with touch!

*Full disclosure: these are affiliate links

Step 3: Hardware: Connect the TFT Screen to the ESP32

The wiring may seem a bit daunting at first. But don't let all the wires scare you. It is pretty straight forward. The images above will help you when you wire your TFT + Touchscreen to your ESP32. This is also decision making time. There are few options when it comes to connecting the two together. You can use a breadboard, you can use prototyping board or you can order a PCB specifically to connect the ILI9488 + touch to the 38-pin ESP32 DevKitC. I'd like to point out that the breadboard option is only an option for testing your connections and screen. It is not very practical to have on your desk and loose connections can cause problems.

It is important to know that these screens run at 3.3V. Connecting them to 5V can cause damage!

These are the connections you need to make from the ESP32 -> ILI9488:

3.3V -> VCC

GND -> GND

GPIO15 -> CS

GPIO4 -> RESET

GPIO2 - > DC/RS

GPIO23 -> SDI(MOSI) and T_DIN

GPIO18 -> SCK and T_CLK

GPIO32 -> LED

GPIO21 -> T_CS

GPIO19 -> T_DO

GPIO27 -> T_IRQ

SDO(MISO) is not used for the TFT screen, so you are left with one unconnected pin on the TFT module. That's ok!

If you like to order a PCB designed to make it easy to connect the ESP32 to the TFT screen, you can download the gerber files here: https://github.com/DustinWatts/ESP32_TFT_Combiner

You can also order them directly from PCBWay where I had mine made:

https://www.pcbway.com/project/shareproject/ESP32_TFT_Combiner_V1.html

Step 4: Software: Installing Arduino IDE Libraries

Installing the Arduino IDE ESP32 core.

Go to Arduino -> Preferences and click on the icon behind the input field for Additional Board Managers URLs.

Next, copy and paste the following link (without quotes) in the popup box:

"https://raw.githubusercontent.com/espressif/arduino-esp32/gh-pages/package_esp32_index.json"

Click OK and OK.

Next, go to Tools -> Board: -> Board Manager and search for "ESP32". Install the latest version and click "Close".

Installing the necessary libraries.

FreeTouchDeckuses a few (some might call 6 more than a few) libraries. Three of these libraries can be installed though the the Library Manager. Go to Sketch -> Include Library -> Manage Libraries and search for, and install the following libraries:

  • Adafruit-GFX-Library

  • TFT_eSPI

  • ArduinoJson

The other three libraries need to be downloaded manually. For each of them the process is the same so I'll describe just one of them.

Go to "https://github.com/DustinWatts/ESP32-BLE-Keyboard", click on "Code" and "Download .ZIP". In the Arduino IDE, go to Sketch -> Include Library -> Add .ZIP library

Repeat for the following libraries:

  • "https://github.com/me-no-dev/ESPAsyncWebServer"
  • "https://github.com/me-no-dev/AsyncTCP"

Step 5: TFT Screen Configuration

Before compiling and uploading the FreeTouchDeck.ino sketch, you will have to edit the user_setup.h file included with the TFT_eSPI library. This can be found in your Arduino sketchbook folder under "libraries". If you have not renamed the TFT_eSPI library folder, the file user_setup.h can be found in TFT_eSPI-master. Here you will have to uncomment the lines that apply to your hardware configuration.

To make things easier, you can find the user_setup.h file you will need for this project in the downloaded repository in the "user_setup.h Examples" folder. Copy the file "ESP32_Dev_Kit_V1_ILI9488_Resistive.h" to "/TFT_eSPI-master/". Rename the "User_Setup.h" that is already there to "User_Setup.old" (this way you keep the original in case something goes wrong). Now rename "ESP32_Dev_Kit_V1_ILI9488_Resistive.h" to "User_Setup.h". (<- CASE sensitive!) Save and close the file.

Step 6: Download FreeTouchDeck.ino and Upload to the ESP32

ESP Sketch Data Upload Tool

FreeTouchDeck uses the SPIFFS (ESP32 flash memory) to store configuration and images that are used. You'll need to upload these to the ESP32 before you upload the sketch to the ESP32. For this you'll need the ESP32 Sketch Data Upload tool. You can download this from Github: "https://github.com/me-no-dev/arduino-esp32fs-plugin". Follow the instructions on the Github to install the tool:

  • Download the tool archive from releases page.
  • In your Arduino sketchbook directory, create tools directory if it doesn't exist yet.
  • Unpack the tool into tools directory (the path will look like /Arduino/tools/ESP32FS/tool/esp32fs.jar).
  • Restart Arduino IDE.

(On MacOS create the tools directory in ~/Documents/Arduino/ and unpack the files there).

FreeTouchDeck.ino

On Github you can find the full source code for FreeTouchDeck. Go to the FreeTouchDeck Github repository and click "Code" and "Download .ZIP": https://github.com/DustinWatts/FreeTouchDeck

Extract and rename the extracted folder to "FreeTouchDeck". Open the FreeTouchDeck.ino sketch in the Arduino IDE. This will also open a few other header (.h) files. You do not need to touch (pun intended) these.

**Since version 0.9.11 the next step is no longer necessary! You can still edit this if you want but if you do not edit the config before uploading, if you start the configurator and it can't connect to an Access Point it will start one. You can then enter the configurator and edit you wifi settings in there.**

Set your WiFi Settings

Once you have the FreeTouchDeck.ino sketch open go to the folder you have the FreeTouchDeck.ino sketch in, open "/data/config/wificonfig.json". This is where you can enter you're WiFi SSID and Password so you can connect to your FreeTouchDeck to configure it

  • change "YOUR_WIFI_SSID" to the WiFi SSID you want FreeTouchDeck to connect to.
  • change "YOUR_WIFI_PASSWORD" to the Password that WiFi network uses.

Upload the data folder

Before you upload the data folder to the ESP32, you'll first have to select the right partitioning scheme.

  • Go to Tools -> Board and select ESP32 Dev Module.
  • Still under Tools, select Partition Scheme. -> "No OTA (2MB APP/ 2MB SPIFFS)".
  • Plug in the ESP32.
  • Click on Tools and select "ESP Sketch Data Upload".

Tip!: If the data upload fails, chances are you have the serial monitor open. If this happens, close the serial monitor and try again.

Upload the sketch to the ESP32

After the data folder is successfully uploaded, you can go ahead and upload the FreeTouchDeck.ino sketch to the ESP32. The settings under tools besides the Partition Scheme can be left to the default.

Go to "Sketch" and select "Upload". This may take a while because it is a large sketch.

Step 7: Printing a Case (optional)

Off course not necessary, but when you have built your FreeTouchDeck you can print a case for your FreeTouchDeck. This design will fit the screen and ESP32 with or without the combiner PCB.

The case is designed to be a press-fit. You could however add a touch of CA glue (crazy glue) to keep the top firmly attached to the bottom.

You can find the case on Thingiverse: https://www.thingiverse.com/thing:4661069

I printed it in PLA with a 0.2 mm layer height. The top doesn't need support if you print it face down. The bottom needs some minimal support if you print it with the opening down.

If you like to modify the case to your liking, you can find the Fusion 360 files on Github: https://github.com/DustinWatts/FreeTouchDeck/tree/master/case/ESP32_TFT_Combiner_Case

I've included some images of my slicer.

Step 8: First Boot

After uploading the sketch and all the files to the ESP, at first boot a touch calibration screen is displayed. This is so that FreeTouchDeck knows where all the corners are and can determine if a touch falls within the boundaries of a button.

A calibration file is saved on the ESP filesystem. So, if you re-upload files to the ESP using "ESP32 Sketch Data Upload", the calibration file is deleted and you will be presented with the calibration screen again.

The home screen has 6 buttons. These are all "menu buttons" in that they take you to an underlying menu. These buttons have a different colour as the function buttons in the menu's. On the home screen you will also find a "settings" menu. This will take you to a menu were you can enter the configurator.

Once the home screen is loaded, BLE is also started. You can now connect with your computer to the FreeTouchDeck. You might have to refresh your Bluetooth device list on your computer. After you see "FreeTouchDeck", you can connect to it. To test if Bluetooth works properly you can go in to the "Music" menu. And press "Mute" to see if a BLE connection is established.

Step 9: Configure Your FreeTouchDeck

You can now customize your FreeTouchDeck. FreeTouchDeck hosts a website on the ESP32 itself where you can configure the menu's and the button's each menu has. You can also customize the colors and upload your own logo's.

To start the configurator. Touch the bottom right button so go to the "Settings" page and then touch the "WiFi" button on the top left. Once the configurator has started you can open a web browser and go to freetouchdeck.local.

To learn more about how to use the configurator check out the Wiki on Github.

15 People Made This Project!

Recommendations

  • Laser Challenge

    Laser Challenge
  • Made with Math Contest

    Made with Math Contest
  • Hour of Code Speed Challenge

    Hour of Code Speed Challenge

106 Comments

0
ronthema
ronthema

5 days ago

LOVE this project! Got it all up and running with a few bumps here and there, but was a great learning experience! Is there a way to remove the white border from the buttons?

0
yacsf
yacsf

5 weeks ago

Hi Dustin.When upload. There is error message 'NUM_0' was not declared in this scope. How to add in?

0
007mng.
007mng.

Question 2 months ago

Hi Dustin, thanks for sharing this amazing project, I'm on the same point than "Rapidbit", I've followed all the steps and the screen remains blank (white).

I've this:


settings. I don't know what could be wrong. I were searching in the Serial Monitor, and thats the result:


I've setted the wifi settings and seems to be all ok, what can I do? Thanks for all you have done yet, impressive work.

(Sorry if my english is not good enough, I'm from Spain, regards!)

0
DustinWatts
DustinWatts

Answer 2 months ago

Hi! I think the uploading of the images didn't go right. There is nothing for me to see :) Do you have Discord? That will make debugging a lot easier for us!

0
007mng.
007mng.

Reply 2 months ago

I've reuploaded the images and still white, here's my discord "007mng#8367", it would be awesome if you help me to the debug the problem.

0
Rapidbit
Rapidbit

2 months ago

Hi Dustin, thanks for sharing this great project.
I have followed all the indications, reviewed connections, verified file path as well as the User_Setup.h modification.
The loading of the sketch apparently correct, it does not throw any error, the Arduino IDE, once the loading is finished, I reboot ESP32 and the TFT screen remains blank (white).
When I open the serial port and do a reset it indicates error in the SPIFFS according to attached image.
Can you help me please?
Thank you.
Best regards.
Luis


2021-09-03_11h33_37.png
0
DustinWatts
DustinWatts

Reply 2 months ago

Hi Rapidbit, Did you do the Sketch Data Upload step? Selecting 2MB APP/2MB SPIFFS is also important when uploading the sketch....

0
Rapidbit
Rapidbit

Reply 2 months ago

Hi Dustin, thanks for your quick response.
Now I have the same problem as "erwinsanders ".
I have managed to upload the sketch and the "data" folder without errors.
From the mobile I see by BLE the device and I can connect, but I see nothing on the screen, I have checked several times the wiring and the path to the User_Setup.h file, I use the one from the repository.
Thanks

0
ranseyer
ranseyer

10 months ago

Solved:

Hi, one more question about touch... I have ordered exact the linked display from aliexpress. I can clearly see the ~4 wires in the flat-cable from the touchscreen layer and the XPT2046 on backside of the pcb. The display is mounted on your PCB and i used as an template :ESP32_Dev_Kit_V1_ILI9488_Resistive.h
(WIFI is not configured and connected)

Is there an video with the touch calibration ? (i see two times a symbol. I think it wants to tell me: "touch me in in the middle of the circle to start calibration) But i dosnt matter if i touch the display or not, it will go on after ~2 seconds and i see the 6 symbols...)

PS: I have two of this ESP32 Modules and two displays with the same behavior... So i should be an configuration problem on my side (?)

Solved:

Hi, the problem is lost while im searching it...
A) i had 2,5MOhm / 6MOhm between the Pin numbered with 21 at the ESP32 Module and between GND and 3V3, That looks good for me...
B) i resoldered all Pins with Flux
C) i used another Power Supply
=> I saw no change
D) auf setting #define REPEAT_CAL to true the calibration can be don and the device works...
I finally can't reproduce the reason. For example i tested calibration with the protection film on the display that i had at the beginng an it works also...
Thank you very much for your support and the great project !

0
JordiKoki
JordiKoki

Reply 3 months ago

I have the same problem but I cannot solve it by performing the procedures you mention.

EDIT: My solution comment this line in the sketch: //#define USECAPTOUCH

0
S-v-e-n
S-v-e-n

Question 5 months ago

Hello Dustin,
I would like to implement the project wired and on a Raspberry Pi Pico. Can you tell me if I can run the source code on the Raspi?
Thanks & greetings, Sven

0
glenn.edvardsen
glenn.edvardsen

7 months ago

Hi!
love the project but i got a issue when im compiling the sketch i get an error saying
could any one point me to what is wrong?

Arduino:1.8.13 (Windows Store 1.8.42.0) (Windows 10),
In file included from C:\Users\glenn\Documents\FreeTouchDeck\FreeTouchDeck.ino:301:0:
sketch\Touch.h: In function 'void touch_calibrate()':
Touch.h:39:9: error: 'class TFT_eSPI' has no member named 'setTouch'
tft.setTouch(calData);
^
Touch.h:61:9: error: 'class TFT_eSPI' has no member named 'calibrateTouch'
tft.calibrateTouch(calData, TFT_MAGENTA, TFT_BLACK, 15);
^
C:\Users\gleedv846\Documents\FreeTouchDeck\FreeTouchDeck.ino: In function 'void loop()':
FreeTouchDeck:676:19: error: 'class TFT_eSPI' has no member named 'getTouch'
pressed = tft.getTouch(&t_x, &t_y);
^
FreeTouchDeck:714:19: error: 'class TFT_eSPI' has no member named 'getTouch'
pressed = tft.getTouch(&t_x, &t_y);
^
FreeTouchDeck:753:19: error: 'class TFT_eSPI' has no member named 'getTouch'
pressed = tft.getTouch(&t_x, &t_y);
^
FreeTouchDeck:831:19: error: 'class TFT_eSPI' has no member named 'getTouch'
pressed = tft.getTouch(&t_x, &t_y);
^
Multiple libraries were found for "WiFi.h"
Använd: C:\Users\gleedv846\Documents\ArduinoData\packages\esp32\hardware\esp32\1.0.6\libraries\WiFi
Oanvänd: C:\Program Files\WindowsApps\ArduinoLLC.ArduinoIDE_1.8.42.0_x86__mdqgnx93n4wtt\libraries\WiFi
exit status 1
'class TFT_eSPI' has no member named 'setTouch'
This report would have more information with
"Show verbose output during compilation"
option enabled in File -> Preferences.

0
DustinWatts
DustinWatts

Reply 7 months ago

Hi Glenn! That compile error shows up if in User_Setup.h, TOUCH_CS is not defined (probably still commented out). Open the file:

/Arduino/libraries/TFT_eSPI/User_Setup.h

and make sure that it is the same as in the example (in the FreeTouchDeck repository) file.

Have fun!

0
glenn.edvardsen
glenn.edvardsen

Reply 7 months ago

:) some times you need to read the instructions more the one time!
Thanks for your reply now it compiled just fine thanks for the help and what a great project!

0
RudSpring
RudSpring

7 months ago

Very nice project indeed! Especially with the WiFi setup page that allow easy configuration. I got everything working, almost. Except that my Linux Mint computer does not recognize the Bluetooth connection (my android phone does however). Which brought me to the following thought: As I have to power the Esp32 with a USB adapter anyway, would it somehow be possible to convert this from a 'Bluetooth' keyboard device into a native 'USB' keyboard device?

0
DustinWatts
DustinWatts

Reply 7 months ago

Hi Rudolf! The ESP32 BLE can have some trouble with certain Bluetooth connections. There are however some good USB BT adapters that help with connection issues. Although I do not know about Linux Mint support....

The ESP32 doesn't have native USB support unfortunately. There is a possibility to use and external IC that has USB HID support which then communicates over serial to the ESP32. I'm looking in to that atm. So it is possible, but I do not have a clear answer right away....

0
jamok
jamok

Question 8 months ago

Love the work you put into making this and letting us get the pcb's. I am at step 5 and cannot figure out where TFT_eSPI-master folder is or should be.

0
DustinWatts
DustinWatts

Answer 8 months ago

Hi Jamok, if you have installed the library using the library manager it is in the sketchbook/libraries folder. By default this is in /Documents/Arduino/libraries.

0
jamok
jamok

Reply 8 months ago

I have a TFT_eSPI folder. Should I rename this to TFT_eSPI-master?