ESP32 Bluetooth BLE Remote Control

Introduction: ESP32 Bluetooth BLE Remote Control

About: I am a retired Embedded Systems Engineer. I enjoy playing around with microcontrollers, SBCs, 3D printing and woodworking.

This project is an example of how to connect an inexpensive Bluetooth BLE joystick to an ESP32. The code was written using the Arduino IDE Ver 1.8.5 with the ESP32 addon. The BLE joystick used is a commercial device that you can purchase on the Internet for less than $20.00 or from your local fiveBELoW store for $5.00.

The joystick that I used for this project is a Spektrum VR Control Bluetooth Remote Controller. It is being marketed as working with Android and IOS tablets as a joystick/mouse for use with VR headsets.

The VR Control Bluetooth Remote Controller has a single joystick with X and Y axis, two trigger buttons in front and six buttons on the handle. Two of the buttons are for power on/off and mode control. The other four buttons can be used for whatever you want. This project is a skeleton or framework that handles all of the Bluetooth interfacing and decoding of the buttons & joystick. All you have to do is add function calls to handle what you want the buttons and joystick to do. You do not need to know anything about Bluetooth to use this framework.

There are numerous web sites with detailed instructions for installing the Arduino IDE and the ESP32 addon. I am not going to attempt to repeat that information here. Google it and follow the directions.

The framework is an adaptation of a BLE client example published by IoT Sharing. You can find it here. You can study this code to get an idea of how BLE works. Expressif has a complete example for a GATT BLE client and explains the operation in detail (not written for Arduino IDE). You can get it here.

This is not a BLE tutorial. I will not be explaining how the code works in detail. I will use BLE terminology to describe some of the features of the joystick. The part of the code that you need to modify for your project will be explained in detail to help you to modify it. The scope is limited to keep this Instructable short and focused on using the joystick.

Step 1: A Little About Bluetooth Low Energy (BLE)

This is not intended to be a tutorial on BLE. When I started this project, I did not know the difference between BLE and Classic Bluetooth. I just wanted to see if I could get the joystick I bought to work with the ESP32. In the following text I am using BLE terminology to give a simplified overview of how BLE works.

BLE uses a client/server architecture. One device is a server that provide services. The other device is a client that consumes services. In order to keep the power requirements down, BLE only transmits small packets of information when a change occurs. In the case of the joystick, the joystick device is a server. As a server, it advertises itself and will transmit a list of the services it provides when asked. The joystick device advertises five services. The only service that we are interested in is the BLE HID (Human Interface Device) service. A BLE service has what are known as Characteristics associated with it. A Characteristic is typically a source of data. The HID service of the joystick has ten Characteristics. Some of the Characteristics are duplicates and are ignored. We are only interested in the BLE Report Characteristics that have Read and Notify capabilities. Three of the characteristics meet these requirements and provide data about the position of the joystick and the state of the buttons. When notification is enabled, the server will send data packets when a change is detected on the associated characteristic.

The framework verifies that the server it finds has the BLE HID service and will then enable Notifications on the three Report Characteristics that provide joystick and button state information. Then, when a button is pressed or released or the joystick is moved, the ESP32 receives a packet of data telling it what the new joystick position is and/or the state of some buttons.

Step 2: Scanning and Connection Indicators

The framework defines two LEDS, GREENLED and BLUELED and assigns them to two of the ESP32's GPIO pins. The GREENLED is lit up when the ESP32 is searching for the BLE joystick. When the joystick is found the GREENLED is turned off and the BLUELED is lit up to indicate that the connection has been established and you are ready to go. If the connection is lost, the BLUELED is turned off, the ESP32 is reset, the GREENLED is lit and scanning starts again. If the joystick is not found within thirty seconds then scanning stops and the GREENLED turns off. After five seconds, scanning starts again and the GREENLED is turned on.

The end result is that the ESP32 will continue to scan for the joystick until it finds it. Once the connection is made, if it is then lost, the ESP32 will reset itself and start scanning over again. The ESP32 is reset because there are no ESP32 SDK functions to reset the Bluetooth stack to restart scanning.

Step 3: Deciphering Joystick and Button Events.

One callback event on the ESP32 receives three different data packets from the server for the three Characteristics that were setup to provide Notifications. One packet is four bytes long. Three of the bytes contain the X axis position, Y axis position and the trigger buttons, which are bit mapped in the byte. The other two packets are two bytes each and have a single byte that has the bit mapped button state. The received packets are decoded and copied into a byte array in memory. The joystick axis data goes into the X and Y data bytes and each of the three bit mapped button bytes is directed into the appropriate byte for those buttons.

A FreeRTOS task is created to handle the data received by the notifications. One task for the joystick and trigger buttons, one task for the A & B buttons and one task for the C & D buttons. Each of these tasks has clearly marked areas where you should your add code to do what you want with the event. Look for the "//===== add your code here =====" comment in the body of the task and add your code after it. Each task has a comment indicating what it is used for and uses a Serial.println() to print a message about the event that occurred.

Here is an example from the A/B button task;

void taskButtonAB(void *parameter)
{ uint8_t buttons;

//===== if the task requires any one time initialization, put it here ===== while(true) { // give up the CPU, wait for new data vTaskSuspend(NULL); // we just woke up, new data is available buttons = VrBoxData[VB_BTNAB]; Serial.printf("A/B Buttons: %02X\n", buttons); if (buttons & VB_BUTTON_A) { // button A pressed or is being held down Serial.println("Button A"); //===== add your code here ===== }

if (buttons & VB_BUTTON_B) { // button B pressed or is being held down Serial.println("Button B");

//===== add your code here ===== } } // for } // taskButtonAB

Step 4: VR Box Operation: the Joystick

If the joystick is left in the center position, no joystick notifications are sent. Once the joystick is moved off center, a notification message with joystick data and trigger button data is sent about every 15mS. When the joystick is moved back to center, a notification that it has moved to center is not sent. In other words, it tells you the joystick has moved off center, but not that it has moved to center. The end result is that you receive messages indicating the joystick is moving toward center, but not that it has reached center. Very annoying. The two trigger buttons are included with the joystick data. Pressing one of the trigger buttons after returning the joystick to center will update the joystick position to zero. The Framework has a timeout timer built-in to it that automatically simulates a joystick notification message a short time after all joystick/trigger button notification messages stop arriving. The timer sets the joystick to zero. The joystick has a range of about +/- 25 on each axis.

Step 5: VR Box Operation: Trigger Buttons

The trigger buttons will send a notification message once when pressed and again when released. The pressed notification message will indicate the button that was pressed. The release notification message indicates that both buttons are released.

Holding the lower trigger button will prevent the server from detecting that the upper trigger button has been pressed. Holding the upper trigger button and pressing the lower trigger button results in the server sending a notification message that the lower trigger button is pressed (the upper trigger button will be zero!). Releasing the lower trigger button will cause the server to send a notification that the upper trigger button is pressed and the lower trigger is released.

In other words, the lower trigger button is dominant over the upper trigger button and will override it when both are pressed. You have to determine how to handle the case of both buttons being pressed.

Step 6: VR Box Operation: A/B Buttons

The A and B buttons act like the joystick and continuously send notification messages when pressed and held down. The messages stop when the button is released. The A and B buttons work similar to the Trigger buttons in that the A button dominates the B button just like the lower trigger button dominates the upper trigger button.

Step 7: VR Box Operation: C/D Buttons

The C and D buttons send a notification message once when pressed and again when released. If held down, no further messages are sent until they are released. Holding down either one of the C or D buttons will prevent the server from detecting activity on the other button.

Step 8: Conclusion

The operation of the buttons is in my opinion a little wonky. The Framework provides for places to put your code to act on when a button is pressed. If you also need to detect button releases, that is left for you to figure out how to do.

It is entirely up to you to determine what you want each button to do and what moving the joystick should do. How you handle the differences in the trigger, A & B and the C & D buttons is up to you.


Look in the code for the; taskJoyStick(), taskButtonAB(), taskButtonCD() functions and add your code after the "//===== add your code here =====" comment.

You will need up to four functions to handle the joystick (forward, backward, right & left) and up to six functions to handle the various buttons. Implement them all or just what you need. The choice is yours.

If you use this framework. I would love a shout out about what you used it for and if you found it easy to use.

If you have questions about how it works or need help getting it to work, contact me.

The code is available on GitHub here.

Enjoy.

Microcontroller Contest

Participated in the
Microcontroller Contest

Be the First to Share

    Recommendations

    • Pocket-Sized Speed Challenge

      Pocket-Sized Speed Challenge
    • Super-Size Speed Challenge

      Super-Size Speed Challenge
    • Audio Challenge 2020

      Audio Challenge 2020

    25 Discussions

    0
    adolph52vc
    adolph52vc

    5 weeks ago

    Hi BigjBehr. Thank you for this tutorial, I still confuse about BLE, trying to reproeduce your proyect some times it goes all the way trough but must of the time after initialization it stops at - Connected to server and stays there, after several reboots I can get go trough. What could be probably the reason? Thank you so much in advance. Regar

    0
    BigjBehr
    BigjBehr

    Reply 4 weeks ago

    Did you change the "VR BOX" in the line; #define ServerName "VR BOX" to match the advertised name of your BLE joystick device ? Must be an exact match. Do you have a good battery in you joystick device ? Are you in range (<30ft) ?

    0
    tpower4
    tpower4

    Question 6 weeks ago

    hey behr, got it to compile, weird but removing all references to librairies seemed to work. .. it finds the joystick but only after resetting the board. then I do the @ D on the joystick for 5 secs. then try to get some notifications about buttons and joystick pressed, nothing on the serial monitor. output attached

    utopia.png
    0
    BigjBehr
    BigjBehr

    Reply 6 weeks ago

    After you do the @+D the device should be in mouse mode. You only need to do this once. The ESP32 looks for the joystick on startup. If it does not find it then the scanning stops after about 30 seconds. You should always turn the joystick on first, then power up the ESP32. The line, "pBLEScan->start(30);" in startup() sets the timeout to 30 seconds. Change the "30" to increase or decrease the time. Once a connection is established the ESP32 is able to re-connect if the connection is lost.

    You can fix this problem by modifying the code to do an ESP32 soft-reset if a no connection timeout occurs before a connection is established. This is the only way that I know of to reset the BLE stack so that it will try to connect again.

    If you made the connection before being in the proper mode then the ESP32 will have established what services and characteristics were available at the time the connection is made. Changing the joystick mode after connection will do nothing. You will need to break the connection and then re-establish it. Also, if your joystick is not exactly the same as mine then you may need to find which mode provides the features that you want to use.The easy way is to select a mode, establish a connection and press buttons & move the joystick and see what you get. You might have to re-write the notifyCallback() function to parse the data packets that you receive.

    Your device is different. Mine indicates its name as "VRBOX", yours says "Utopia 360 Remote". The printout was cutoff so that I could not see the UUID of the service. You can modify the code to print the UUIDs of services found and the UUIDs of the characteristics for each service. The code looks for a remote server with the name "VRBOX" and then if it has a service with the HID UUID. and then for characteristics that use the REPORT UUID. Due to the name of your device being different you are not actually connecting to the joystick. Line 76 (or there abouts) defines ServerName as "VRBOX". You need to change this to "Utopia 360 Remote" and recompile. You can change the code to connect to a remote server that has a matching MAC address or name, I chose to use the name. If I was going to use multiple remotes with multiple ESP32s in the same vicinity then I would use the MAC address instead because it has to be unique, whereas the name does not.. Because your joystick is different you are going to have to do some experimenting to find out which mode has the data you want and how to parse it. Your output does not indicate that a HID service was found, so no connection is made. When I started this project, I used an app on an Android device to display all the services and characteristics that the joystick was advertising. If you cannot figure out how to have the ESP32 display the advertised services then try to find an Android app to do it for you, then use your phone or tablet. I do not use Apple products, so I do not know if there is an IOS app for BLE snooping.

    I hope this helps, Good luck,

    0
    tpower4
    tpower4

    Reply 6 weeks ago

    BIG SHOUT OUT TO YOU BEHR!!! That was it. I'm a little sloppy with the programming but should have seen that one, as plain as day. I used the strstr as opposed to the strcmp in the callback. Its now pumping all kinds of data. First of all, you say in your article you are not a programmer.I profoundly disagree, YOU ARE!! Now you wonder what this will be used for. Have a mildly disabled roommate with a manual electric bed. We are automating it for him and this little joystick will give him a lot more functionality. Thanks so much.

    0
    BigjBehr
    BigjBehr

    Reply 6 weeks ago

    I never said I was not a programmer. I said that I am a retired Embedded Systems Engineer. I developed hardware and software. So, yes, a programmer. I probably said that I am not a BLE expert. I just learned enough about how BLE works to be able to program the ESP32. Glad to hear that it works for you.

    BTW, I got my VRBOX controller from my local 5Below store for $5.00. Where did you get your Utopia 360 controller & what did you pay for it ?

    0
    tpower4
    tpower4

    Reply 6 weeks ago

    I paid $10 for the remote on Kijiji (online garage sale). now thinking how handy these little things are with the advent of your well written program. And for those folks looking over our shoulders they are well worth the money, https://www.bestbuy.ca/en-ca/product/retrak-retrak...

    0
    ильяса1
    ильяса1

    11 months ago

    Hi, thanks for this nice tutorial. I'm trying to do the same with game pad "Gamesir-G3s". I scanned it with nRF Connect APP. The scan result is on the picture. Among services I can't see a "HID". Instead, information about buttons and joystick is stored in the "Unknown service". This device works with any smartphone. So how should I use it without the "HID" service? Thanks in advance.

    WeChat Image_20190717110621.pngWeChat Image_20190717110630.png
    0
    Tomsmsgt
    Tomsmsgt

    1 year ago

    hi BigJBehr,

    I'm getting the following error when I try to compile,what am I missing?
    Thanks for your help,
    Tom


    Arduino: 1.8.5 (Windows 10), Board: "ESP32 Dev Module, Disabled, Default 4MB with spiffs (1.2MB APP/1.5MB SPIFFS), 240MHz (WiFi/BT), QIO, 80MHz, 4MB (32Mb), 921600, None"

    C:\Users\Tom\Desktop\bluJoy\ESP32-Bluetooth-BLE-Remote-Control-master\Gatt-VRBOX-Tasks\Gatt-VRBOX-Tasks.ino:42:24: fatal error: controller.h: No such file or directory

    compilation terminated.

    exit status 1
    Error compiling for board ESP32 Dev Module.

    This report would have more information with
    "Show verbose output during compilation"
    option enabled in File -> Preferences.


    1
    BigjBehr
    BigjBehr

    Reply 1 year ago

    You can comment out the line: #include controller.h.
    This project will no longer compile. The latest release of the Expressif SDK for the Arduino IDE no longer has the Bluetooth support files for this program.

    There is a new BLE_client example that uses Neil Kolban's class based Bluetooth library (which is built-in to the SDK). You will have to modifiy it to change the UUIDs of the service to connect to and the characteristic to use. This can be made to work. However, there is a bug in the Expressif Bluetooth stack that will crash the stack if you try to use more than one characteristic with a service. I have code somewhere that sort of works, only the joystick and trigger buttons work. Let me know if you would like this and I will look for it.

    0
    smallwire
    smallwire

    Question 1 year ago

    Hi BigjBehr!

    Thank you very much for this nice instructable!

    I need some help please. Scanning for devices works, as I can find things like my Fitbit or the BB-8, but I cannot find the remote.

    It looks almost like yours, but it's a different brand.

    Do you know what may be causing this?


    Best regards!

    IMG_20190406_171705.jpg
    0
    BigjBehr
    BigjBehr

    Answer 1 year ago

    Your device looks a lot like mine. Are you sure that it is a Bluetooth BLE device ?
    Did you turn it on before trying yo scan for it ?
    Are you using a fresh battery ?
    On my device you press the power button and the blue LED starts blinking. Does yours do the same ?

    Some issues with my code have come up in the past few months. It seems that somewhere between when I wrote the code and posted it on Instructables, a new Arduino SDK for the ESP32 came out. The new SDK broke a few things and has a bug in it that prevents you from using more than one characteristic with a service. I have posted updated code that fixes some of the broken parts, but I cannot fix the issue with one characteristic per service. None of this matters if you cannot find your device and none of the issues had to do with finding a device.

    Make sure your device is on when you start the ESP32 code running. There are come Android apps that can be run on an Android smartphone or tablet that provide information about Bluetooth devices. I suggest that you download and install one of these apps (look in the Google Play store) and see if they can find your device.

    Can you capture the debug messages that are output on the serial port when you are trying to find your device and post it here.

    Can you tell me the exact manufacturer's name and model number and where you bought it.

    I have talked with a fellow in Germany who bought a German version of the device. It was exactly the same, different color and branded different, but worked identically. His came in a different mode than mine. He could not get it to work until he tried a different mode. Modes are changed by holding down one button and pressing another. Check with you instruction booklet and try all the different modes.

    Hope this helps

    0
    JohnJ248
    JohnJ248

    1 year ago

    Hi, I'm interested in the red board you have mounted the ESP to. did you make it yourself or is it purchased,if so from where. Thanks John in the UK.

    0
    BigjBehr
    BigjBehr

    Reply 1 year ago

    Hello John in the UK,
    Nice to hear from someone across the pond. The red board that you are asking about is one that I made myself. The board was designed in either Eagle or Kicad and manufactured for me in China. The bare bards are hand soldered by myself. If you are fairly good with a soldering iron then you should be able to solder the board yourself.
    I have designed several breakout boards for the ESP32 and ESP8266. The one you are asking about will accept ESP32 boards that have two rows of fifeteen pins that are 1.0 and 1.1 inch apart. The board has all the GPIO pins broken out to headers, a pushbutton switch, tri-color LED and a place for an IR sensor.
    I have other ESP32 breakout boards. All have headers for all of the GPIO pins and a tri-color LED. Some have places to connect a 5V HC-SR04 Sonar sensor with a voltage divider on the Echo signal to make it compatible with the ESP32. One has a place on board for a TB6612FNG Dual H bridge daughter board. Another is more of a universal breakout board that can accept ESP32, ESP8266, Arduino and STM32F103xx, LPC1768 and similar boards.
    If you tell me what features you are interested in, I will create a github entry for that board and post the link here. Then you can down load the files needed to have boards made. Once you have the files you can have the boards made in China by Seed Studio, AllPCB or any of the others manufacturers.

    0
    JohnJ248
    JohnJ248

    Reply 1 year ago

    Hi, Thank you for replying. I'm ok at soldering and have just built my first robot with my daughter.
    In the UK the most popular boards are ESP8266 - due to the better library support and these are the ones I have started to buy.
    https://www.ebay.co.uk/itm/NodeMcu-Development-Boa...

    Your universal board for the ESP32 and ESP8266 look amazing with the all the GPIO pins broken out to headers. The board with all the GPIO pins broken out to headers, a pushbutton switch, tri-color LED and a place for an IR sensor - would be great for my daughter as she is only 3 years old but starting to take some interest in how it all works!

    Thank you for your time.

    John

    IMG_20181216_233222597_BURST000_COVER_TOP.jpg
    0
    BigjBehr
    BigjBehr

    Reply 1 year ago

    Cute robot. The controller in the picture, is it Bluetooth ?
    The board that you want is part of a three board set. The other two boards are an IR blaster and an HC-11 Bluetooth module breakout board. All three are together as one layout. When you get the boards you will have to cut them apart. I use a bandsaw with a metal cutting blade to separate the boards. A hacksaw will also work, but takes longer. There are no cut lines on the board, but it is fairly obvious where to cut them apart.
    Order the board as one layout. This will minimize the price.
    I will post all the files in a day or two. If you are adept at using Eagle, you can modify the schematic to remove the other two boards and then change the board outline. You would then have to make all new files. I could do this for you, but will not be able to do it quickly due to the Holidays coming up.
    I usually order the headers on ebay. I use so many that I keep them on hand. I buy the forty pin snap apart headers and then break them down to the size I need. Same with the female socket strips. However, you cannot cut the female socket strips between two pins. You have to burn a pin to cut them down. So a forty pin strip will yield a 20 pin strip and a 19 pin strip, not two 20 pin strips.
    Let me know if you want me to separate the boards for you.

    0
    JohnJ248
    JohnJ248

    Reply 1 year ago

    Hi Ben,
    It's late here in the UK (1AM!), the controller is a chinese (eBay) 2.4Ghz version of a PS2 controller.

    http://www.lynxmotion.com/p-1096-ps2-robot-control...

    Look forward to your layout file, as the process doesn't sound impossible and I've got friends with the tools :) don't worry about seperating them.

    Happy Christmas

    John

    0
    BigjBehr
    BigjBehr

    Reply 1 year ago

    John,
    I recently posted a new project on Instructables that you may be interested in. It is for an ESP32 breakout board with a built-in dual H bridge motor controller

    0
    Martin Prochazka
    Martin Prochazka

    2 years ago

    Hello BigjBehr, I am trying upload code to Heltec ESP32 and have some problem with compiling error:

    C:\Users\Peter Brown\Documents\Gatt-VRBOX-Tasks\Gatt-VRBOX-Tasks.ino: In function 'void esp_gap_cb(esp_gap_ble_cb_event_t, esp_ble_gap_cb_param_t*)':

    Gatt-VRBOX-Tasks:222: error: cannot convert 'bool' to 'esp_ble_addr_type_t' for argument '3' to 'esp_err_t esp_ble_gattc_open(esp_gatt_if_t, uint8_t*, esp_ble_addr_type_t, bool)'

    esp_ble_gattc_open(test_profile.gattc_if, scan_result->scan_rst.bda, true);

    ^

    exit status 1

    cannot convert 'bool' to 'esp_ble_addr_type_t' for argument '3' to 'esp_err_t esp_ble_gattc_open(esp_gatt_if_t, uint8_t*, esp_ble_addr_type_t, bool)'

    ----------------------------

    Ble_client from sketches compile well.

    What version of BLE library did you use ? I already tryied those from expressiff and BLE from 0.4.0 to 0.4.7 Thank you for the tip.