Intro: Arm and Hand Gesture Control in Python
For those that haven't heard of the Myo before - it's an armband that reads the electrical impulses from your muscles, letting you control devices by simply waving or gesturing with your hand.
If you have seen it and want to develop on it, you'll soon come to realize that the library is C++ only. This is great for code bases that are already in C++ (*cough* video games *cough*), but not if you just want to make a quick script that does something with the input data. So, why not use Python?
We created a small library for the supported systems (currently OSX, Win32 and Win64) that bundles all of the required C++ code into a separate process and exposes it via a simple python class. In later steps we'll also go over the protocol, so you can port it to a language of your choice!
Let's get started.
Like this Instructable? Don't forget to follow us, favorite it, and check us out on Facebook!
Note 11/2014: If you get repeated "unable to connect" messages and the vanilla hello-myo.exe program from Thalmic works, let us know and we'll fix it - new releases of Myo Connect can break old versions of the executable.
Time Required: 15 minutes (once Myo is set up)
- Myo Armband (we had the Alpha version) plus included BLE dongle
- Computer running OSX or Windows 32/x64
- Some basic knowledge of Python (how to run a script)
Step 1: Setting Up
To get a Myo talking to Python, you'll need a running installation of Myo Connect, a Python installation, and our custom library.
This comes with the developer SDK, in the myo-connect folder for Windows or the root folder for OSX. Run it and go through the setup process to register your Myo with the computer.
We're using version 2.7.8 of python - which can be found here. Installation should be as simple as downloading and running the file. If you're using OSX, python should come preinstalled.
Myo for Python:
Our library is available on GitHub as always - click the Download Zip button on the right side for an archive of all files.
Step 2: Go PyMyo!
Double-click the MyoTest.py script - you should immediately see a printout of roll, pitch, yaw, hand state, and arm side. This should be pretty familiar for those of you that previously tried the hello-myo binary, as it's exactly the same.
That's literally all there is to it - you now have the Myo armband talking to a Python script!
Read on to learn more about the test code, class, and executable.
Step 3: MyoTest.py
MyoTest is a python implementation of the hello-myo code that comes with the Myo sdk. It connects to the myo and prints all the same stuff to the console.
How it Works:
The main difference between MyoTest and hello-myo.exe is that the hello-myo code stops and polls for event data, while MyoTest instead registers a function (during the Myo() class instantiation) that's called when the myo sends data to our wrapped C code. For our test program, this callback function is printData, which gets/creates string representations of the pose, orientation, and arm state of the Myo.
Step 4: Myo.py
The actual Myo class (inside myo.py) handles the output data from the wrapped executable and makes it easy to use in a python script. It has methods to get any data that the myo currently allows the user to access, plus a convenient conversion from quaternions to roll/pitch/yaw rotation.
How it Works:
Calling .start() on the class object starts it as a new thread. The object executes our C code as a separate process, keeping a handle to the process I/O. It then blocks and waits for the executable to send us a data packet of the Myo's state before unpacking and saving the packet. Methods like getAcceleration() operate on this saved packet data.
Whenever a call to vibrate() occurs, our Myo object sends a (1) to STDIN of the process, which recognizes it as a request to vibrate and acts accordingly.
Step 5: PyMyo (.exe)
This executable is where the actual communication to the Myo occurs. It's a modification of hello-myo that prints out a compact version of the state of the Myo armband - only 9 bytes - every 50ms.
How it Works:
This works similarly to hello-myo (see here for extensive documentation); the majority of our changes were to catch accelerometer data, print out the data compactly, and handle console input to vibrate the Myo.
For a closer look at these changes, check out our inputThread() function (which runs the input as a separate thread) and DataCollector::print(), which prints out the Myo state for Myo.py to receive.
The executable sends a packed string of bytes at 20Hz - the format is:
Acceleration X, Y, and Z (3 bytes) form the acceleration vector.
Rotation W, X, Y, and Z (4 bytes) form a quaternion representing the current rotation.
Hand Pose (1 byte) indicates the state of the hand (Fist, WaveIn, etc.). It uses the same enum values as the myo library.
Arm # (1 byte) is either 0 or 1 for Right/Left respectively, or ASCII 'A' (0x41) if the arm is not present.
Step 6: What's Next?
We built a Myo wrapper for Python because it's easier to get things done quickly. Python is "batteries included," meaning it has a truly incredible number of modules.
Here are a few ideas for projects that are quite Python-friendly that could benefit from a Myo:
- Control a cheap pan-tilt camera remotely with a wave of your arm
- Control your Windows computer and automate your tasks with PyAHK
- Build a cool Myo-enabled video game!
- Control an arduino robotic arm with your fleshy non-robotic arm and pySerial
We've got plenty more exciting new content on the way this summer, so if you enjoyed this Instructable, don't forget to favorite it and follow us on Facebook!