Gesture-controlled Mouse Using Micro:bit

INTRODUCTION

The project is a gesture-controlled mouse, in the form of a glove with a micro:bit embedded in it. It is a project inspired by an early idea I had as an embedded engineer. I always wish to have access to a wireless mouse and keyboard set, for interfacing with any microprocessor I may wish to use. A mouse is essential in case I am running an OS on a microprocessor. Having a wireless mouse and keyboard has proved to be a savior quite a number of times, and in this regard, I was very interested in developing a wearable piece of technology which can be used as a mouse, and conceivably, a keyboard.

In this instructable, I am going to share the steps and procedure of how I used a micro:bit with some Python code to make a glove-mounted gesture-controlled mouse.

Supplies

1 micro:bit with battery

1 glove for attaching micro:bit to

1 bluetooth-enabled computer Linux (Raspberry Pi used here)

PC for programming micro:bit and opening VNC session to Raspberry Pi

Step 1: How It Works

Step 2: Interfacing Micro:bit to Raspberry Pi Via Bluetooth

1. On start, I show a sequence of LEDs in the shape of "b*", and initialize the bluetooth accelerometer, button, LED and temperature services.

2. In event of bluetooth connection, the LED screen shows a "C", indicating the micro:bit has established bluetooth connection.

3. In event of bluetooth disconnection, the LED screen shows a "D", indicating the micro:bit has lost bluetooth connection, and will need to re-connect.

4. In Project Settings, select the "No Pairing" mode. The micro:bit website recommends "Just Works" pairing, but pairing always fails with the latter. For some inexplicable reason, it works with the former.

Step 3: Setting Up Bluetooth on Raspberry Pi

1. Install the bluezero package using "sudo pip3 install bluezero"

2. Open a bluetooth command prompt by typing "bluetoothctl". The new prompt should look like:

$ [bluetooth] #

3. Put micro:bit into scanning mode by holding buttons A and B, and pressing the reset button. Wait until a bluetooth symbol appears on the LED screen, then release all buttons. On the raspberry pi bluetooth terminal, type "scan on". Once the micro:bit is visible in the list of new devices, type "scan off", and note the device address of the micro:bit as "yy:yy:yy:yy:yy:yy".

4. Pair and connect with the micro:bit by typing "pair yy:yy:yy:yy:yy:yy".

5. To verify that pairing worked, type "devices" and check if micro:bit appears in the list of devices. If connection is successful, the LED screen on the micro:bit should change to a "C". If not, reset the micro:bit and simply try to pair again on the bluetoothctl terminal on the Raspberry pi.

Step 4: Run Code on the Raspberry Pi

1. Run the code on the Raspberry Pi by typing "python3 mouse_control.py", which is attached at the end of this step:

2. A prompt should appear on the terminal saying "Connected". Press both buttons A and B for a second to 'switch on' the mouse. A prompt saying "Move cursor around" should appear now.

3. To 'switch off' the mouse, again press both buttons together. A prompt of "Continue or exit?" appears. Hit c to continue from step 2, or just enter to exit the program.

4. If the mouse is held in an inverted position (LED screen facing downwards), the programs throws an error asking the user to hold the mouse correctly, before proceeding to step 2.

5. For a single click, hold the mouse parallel to the ground (to make cursor stop moving) and press button A only. For a double click, press button B only.

6. For code walkthrough, refer the attached file, which is sufficiently well-commented.

7. The bluetooth connection is quite sporadic and may break up quite frequently, especially if any other bluetooth-connected devices are in use nearby. If this happens, running the code again should work. If that too fails, open the bluetoothctl command window and pair with the micro:bit again.

Step 5: Future Scope

1. The mouse controls can be fine-tuned to a great degree. If an exponential acceleration instead a linear acceleration algorithm is used, it might provide even greater control.

2. Since the bluetooth connection is quite sporadic, an event for detecting connection failures can be added, which would have the response of re-establishing a connection on every failure.

3. Some keyboard gestures can be added, like a sudden jerk could cause a shutdown, or a pre-defined gesture, when detected, could trigger some other frequently used command/keyboard stroke.