Introduction: Moving LED Eyes Using a Range of Inputs Including Gesture Control
During the COVID-19 pandemic Melbourne has been under some of the strictest lockdowns in the world, which has essentially meant that I've had lots of spare time to learn something new. So I've decided to turn my hand to some hobby electronics to see what sorts of things I can create.
I had spent some time looking at different sensors from the electronics shop I had ordered some things from and came across a gesture sensor! I thought it would be pretty cool to see what I could use it for and so I ordered one.
I'm also learning lots about LEDs and the LED Matrix, so I thought it would be good to combine the two and create a set of moving eyes that could be used on a prop, as a puppet, or just something to make on a rainy afternoon. Once you understand the commands and how it all ties in, the options really are endless and only limited by your imagination.
I wanted a few other controls too, so I decided to throw a potentiometer and a push button in the mix too to see what I could create and I used some LEDs to shine when the gesture sensor sensed a gesture.
If you like my work, please:
OK, let's jump in!
Supplies
For this project, you'll need:
- Arduino Uno (or other microcontroller like a Nano)
- 2 x 8x8 LED Dot Matrix Modules (you can use more or less, depending on how many eyes you want - bearing in mind they will all display exactly the same)
- Adafruit APDS9960 Gesture Sensor
- Potentiometer
- Push Button
- LEDs (I chose one each in red, green, yellow and blue) (Optional)
- 4.7k Resistor
- M-M jumper cables
- M-F jumper cables
- Solderless Breadboard
Step 1: What Will It Do?
I wanted to try out a range of different inputs and use each of them to animate the eyes in some way. We will have three different inputs, a push button, a potentiometer and a gesture sensor.
The push button will make the eyes blink once each time it's pressed.
The potentiometer will be used to turn the eyes from left to right, animating over 7 'frames' being far left, left, slight left, neutral, slight right, right, far right. If no other inputs are detected, the eyes will remain in the position based on the potentiometer.
Lastly, the gesture sensor will sense 4 gestures - swiping left, right, up and down. This will make the eyes go left, right, blink or close respectively.
There will also be 4 coloured LEDs that will light up when their respective gesture is sensed.
Each of these inputs can be used at any time, so you can choose which one you want to control the eyes with or alternate between them.
Check out the video here:
Step 2: Hooking It All Up
This should be fairly straightforward, all of the connections are shown in the diagram attached and you can see how it physically looks in the photos.
The gesture sensor uses a range of sensors and its own processor to determine different hand gestures. The connections to the Uno are as follows:
APDS9960 -> Arduino Uno
VIN -> 5V
GND -> GND
SCL -> SCL
SDA -> SDA
The LED Matrices can be setup in a daisy chain arrangement. They have pins on both ends and you just match them up (output of one, connects to the same pin for the input on the next matrix). Connecting these to the Uno, you would connect the pins as follows:
LED Matrix -> Arduino Uno
CLK -> Pin 13
CS -> Pin 10
DIN -> Pin 11
GND -> GND
VCC -> 5V
When I was first putting this together, I used LEDs to indicate which gesture the gesture sensor was detecting. You may choose to leave these out, but I thought it added another level of interactivity with the gestures. I arranged them so they showed the direction of the gesture being activated (up, down, left, right). You might choose to use resistors with these LEDs as well, but as they are only on for a second (when activated) I plugged the Cathode (-) directly into Ground on the breadboard. The anodes should be arranged as follows:
LED -> Arduino Uno
Red (Right) -> Pin 2
Yellow (Up) -> Pin 3
Blue (Down) -> Pin 4
Green (Left) -> Pin 5
The button is pretty easy to hook up. 5V to one side, the other side to Pin 6, with a 4.7k resistor running to Ground.
The potentiometer works by sending a voltage reading to a pin on the Arduino board, depending on the position of the 'dial'. We can then use these readings to tell the board where the eyes should be looking. You'll notice the potentiometer has 3 pins. The 2 on one side should be connected to 5V and GND. The Output pin should be hooked up to Pin A0 on the Arduino - this pin reads analog signals and is appropriate for the potentiometer readings.
Double check your wiring matches the diagram and you should be in business.
If you are confident in using Arduino IDE and it's libraries, then you might like to use the (examples) in the LEDControl (LCDemoMatrix) and Adafruit APDS9960 Library (gesture_sensor) to test that the different functions are working.
Step 3: Programming - Drawing the Pixel Art
The LED Matrices are drawn in frames, similar to how cartoons are animated. There's a number of frames we need for the eyes. So make a list of all the different positions of the eyes you'd like to be able to display, the ones we'll use for this project are:
- Open eyes,
- Partially open,
- Partially closed,
- Closed eyes,
- Looking really left,
- Looking left,
- Looking slightly left,
- Looking slightly right,
- Looking right,
- Looking really right
These frames will help with the animation, to give it a more lifelike look. It's up to you how many frames you want to make, if you want to start off simple, you might just choose a neutral eye, and a left and right looking eye and start from there.
This is also a good time to think about how you want your eye to look. You might just choose a round eye like I have. Or you might want your eye to look like a cat, or square eyes, oval shaped eyes, it's really up to you. It's best to start your designing with the open eye, as this is the frame that you will see the most. You can then design the rest of your eyes, based off this design.
The best way to design your eye is using a tool like this (you can also search LED Matrix Editor). We will be working with byte arrays, so make sure you tick that box. You can click on each LED to light it up and see how it looks, once you're happy with it, click the 'Update' button and it will update the details of the byte array. I've done this for my Open Eye in the screenshot. The result I get is:
B00111100,
B01000010,
B10000001,
B10011001,
B10011001,
B10000001,
B01000010,
B00111100
If you look carefully, the 1's in the array are the LED's that are 'on' and the 0's are 'off'. So you can kind of see the pattern you are creating in the byte array itself. Continue creating the rest of your frames and saving the byte arrays.
Step 4: Programming - the Code
To get started with the code, you'll need Arduino IDE. This is available here and you can download for Windows, Mac OS X and Linux. Once you've followed the instructions on how to download, you'll need to add a couple of libraries.
The libraries required are:
- LEDControl
- Adafruit APDS9960 Library
If you're not sure how to install a library - it's very simple. Open Arduino IDE, click Tools, Manage Libraries... Then in the search bar in the window that opens, type the name of the library, scroll to find the right one if required and then hover over it and an 'Install' button will appear. Click install and the library is installed. Repeat the process for the other library.
The Declarations
The first part of the program includes the libraries (LEDControl, Adafruit APDS9960 and binary). We also setup the pins that each of the LEDs and the button is connected to, as well as variables where we can store values to check if the blink button has been pressed or the potentiometer has been turned.
We also tell the program how many LED matrices we have and how they are connected to the Arduino. If you need to, you can edit any of these as you see fit.
The Setup
The next part is the Setup - this part runs once when you turn on the Uno or reset it. We give a small delay, start the serial and then tell the program which pins will be input or output pins (sending or receiving things to/from the user). We also initialise the gesture sensor and the LED matrices. Finally we set the eyes to 'open'.
The Bitmaps
Next we include all the bitmaps - these are the frames that display each part of the eye movement animation. I've called each byte a meaningful name like 'eyeopen' or 'eyeclosed' so I can tell which one refers to each frame. It's then followed by the 1's and 0's that we got from the website earlier.
When originally tested my bitmaps on my screen, some of them were rotated the wrong way. If this happens, you can use the website to rotate the bitmap (for me, it just needed to be rotated 90 degrees - as the blink was happening vertically rather than horizontally). Please note: You want to test these out on an eye that's partially closed, as the open one (in the example I've given) will look the same regardless of rotation, as it's just a circle with a smaller circle in the middle.
The Loop
Then you have the loop. This is the part that will just run over and over again - this occurs many many times per second. First we'll ask it to check if the button is being pressed - if it is, then we say, make the eyes blink. This is done by using functions closeeyes and openeyes with a small delay in between to give the appearance of a blink. These functions will be discussed in a little more detail below, but we use them to prevent having to type lots of code over and over.
The next part is checking for gestures. This is done by using 4 if statements linked to each of the gestures. Up, down, left and right. For each if statement we check if there's a gesture, if there is, we print the gesture captured to the serial along with the action the eyes will take, we light up the relevant coloured LED, we run a function related to that animation (closeeyes, openeyes, lookleft, lookright) and then have a 1 second delay before allowing the program to move on.
Once the if statements have run, the program resets all the coloured LED's to off, and the LED matrices continue to show the last frame drawn.
Lastly, we call the drawLooking function which checks the potentiometer for any changes and draws the eyes to match where the dial has been turned.
The Functions
At the end we define the functions and what will happen when each function is called. As mentioned, it means that the code within the Setup and Loop functions is more succinct and easy to read. It also means that if we want the eyes to open or close at different times, we only have to define the function once and can then call it in many different sections.
We first choose a meaningful name for the function - usually what it will do. Then we set out what will happen. For most of these functions is quite simple, we just tell the program how to display the bitmaps we've made. This is done by setting the rows of LEDs in each matrix to on or off depending on the binary array that we defined earlier on in the 'bitmaps' section. If you do this with small delays in between drawing each frame, then it will appear more life like when you actually view the changes occurring on the matrix.
We do the same for the drawLooking function. However we also check the value of the potentiometer and map it, so the system calculates the number and scales it to a number between 0 and 7, as this is the number of frames we have for the eyes looking from left to right.
Download the code and check it out for yourself. I'm by no means excellent, but it's a great way to get to know how the LED matrix works and what sorts of images you can draw on them. The options really are endless!
Attachments
Step 5: Try It Out!
OK so you've come this far. Now it's time to upload your code to the Uno board. Plug it in via USB. Go to Tools in Arduino IDE and make sure you select the right board under 'Board'. You're looking for 'Arduino Uno'. Then check that you've selected the right 'Port'. For this, if it's plugged in, you should be able to scroll down to 'Port' and select the right COM port from the list. Then click the arrow that's pointing right (Upload) in the bar just under the toolbar.
Once it's uploaded, the eyes should appear on the LED matrices and you should be able to start testing it all out. You may have to move your hand close to the gesture sensor and then away before it starts sensing gestures. Here's what mine looks like:
Have some fun with it, let's see what sorts of eyes you can make in the comments below. Maybe you even want to include eyebrows or eyelashes? Maybe some evil eyes. Or scared eyes. Include another LED matrix and create byte arrays for mouths. Your options really are unlimited! Good luck.
