How to Create a Gesture-controlled Universal Remote Control

This is my first Instructable...my goal was to build a Gesture-controlled Universal Remote control for my entertainment center.

Parts for remote: Arduino Nano, APDS 9960 Gesture Sensor, IR Send module.

Other: Simerec PCS-1 (see further information later in Instructable)

I've tried to think of everything, but if I'm missing something, or something's not clear, please let me know....I am more than willing to share what little I know!!

Step 1: Gesture Controlled Universal Remote Control

I wanted to create an all-in-one remote for my entertainment system, which consists of a TV, DVD/Surround Sound Amp, and a computer. But, at the same time, I had begun tinkering with an APDS 9960 Gesture Sensor, and thought, "Hey, why not combine them to have a Gesture-controlled universal remote?". And that's what I did!

(The video is a little rough, as I found it difficult to video both my hand performing the gestures, and the system responding at the same time...I hope you see enough to understand what I have created!)

Step 2: IR Receiver for Computer Power

The first thing I needed to do was to somehow wire the computer to turn on/off via IR remote. I found this product, which was exactly what I was looking for. (More on the tremendous customer support later!!)

http://simerec.com/PCS-1a.html

This product came with everything needed to install the IR unit, with absolutely fantastic step-by-step instructions!

Step 3: Obtaining the IR Codes From the Various Remotes

The next step was to obtain IR remote codes from the various remote controls. From the TV, I obtained the power code (it was the same to turn on/off). From the amp, I obtained the power code (again, the same for on/off), the volume up code, the volume down code, and the function code. And after following the instructions provided from Simerec to program the computer to use an IR signal from an extra button or other remote for power, I obtained the IR code for it. To obtain the codes, I used this IR receiver (http://www.ebay.com/itm/Brand-New-Digital-38KHz-In... I purchased mine from KingElectronics. I'm sure any IR receiver would work just as well. It was a simple hook up to my Arduino Uno, VCC to 5v, gnd to gnd, and Dat pin to pin 2 on Uno.

I used the following code to receive the raw codes. I used RAW codes, as that's what seemed to work best from everything I read online...

IR receive code

#include

int RECV_PIN=2;

IRrecv irrecv(RECV_PIN);

decode_results results;

void setup()

{

Serial.begin(9600);

irrecv.enableIRIn();

}

int c=1;

void dump(decode_results *results){

int count=results->rawlen;

Serial.println(c);

c++;

Serial.println("For IR Scope:");

for (int i=1;i

if((i%2)==1){

Serial.print("+");

Serial.print(results->rawbuf[i]*USECPERTICK,DEC);

}

else{

Serial.print(-(int)results->rawbuf[i]*USECPERTICK,DEC);

}

Serial.print(" ");

}

Serial.println("");

Serial.println("For Arduino sketch: ");

Serial.print("unsigned int raw[");

Serial.print(count,DEC);

Serial.print("]={");

for(int i=1;i

if((i%2)==1){

Serial.print(results->rawbuf[i]*USECPERTICK,DEC);

}

else{

Serial.print((int)results->rawbuf[i]* USECPERTICK, DEC);

}

Serial.print(",");

}

Serial.print("};");

Serial.println("");

Serial.print("irsend.sendRaw(raw,");

Serial.print(count, DEC);

Serial.print(",38);");

Serial.println("");

Serial.println("");

}

void loop(){

if(irrecv.decode(&results)){

dump(&results);

irrecv.resume();

}

}

You can see the codes I used in the Arduino code sketch.

Step 4: Writing the Arduino Code

The next step was to use the codes I obtained to write the Arduino sketch. I used an Arduino Nano, as it had enough pins to do what I needed, and was small enough to make a nice sized enclosure.

My goal was to have a "left gesture" control power to all three (TV, Amp, Computer), "right gesture" to control the Amp function (it simply toggled through TV, Audio, FM Radio, AM Radio, DVD). The "up" gesture sent a volume up signal, a "down" gesture sent a volume down signal. I didn't use the "near" or "far" gestures (at least not yet...I may want one of them to just turn the Amp on without the other two components turning on, in the case that I am playing a vinyl record and only want the amp on...perhaps in the future!).

Here's the Arduino sketch:

/*

All functions on one NANO board, using APDS 9960 gesture sensor as trigger for IR Send.

This code has the tv on/off (raw1), Stereo on/off(raw2), Stereo function(raw3), stereo vol up (raw4), stereo vol down(raw5), and computer on/off(raw6).

IRremote: IRsendDemo - demonstrates sending IR codes with IRsend

An IR LED must be connected to Arduino PWM pin 3.

Version 0.1 July, 2009

Copyright 2009 Ken Shirriff

http://arcfn.com

*/

#include //library for IR

#include //library for gesture

#include //library for gesture

#define APDS9960_INT 1 // interrupt pin for gesture

SparkFun_APDS9960 apds = SparkFun_APDS9960(); //gesture

int isr_flag = 0; //gesture

IRsend irsend;

unsigned int raw1[68] = {4450, 4500, 500, 1700, 500, 1750, 500, 1700, 500, 650, 500, 600, 450, 650, 500, 600, 500, 600, 500, 1750, 500, 1700, 500, 1750, 500, 600, 450, 650, 500, 600, 500, 650, 450, 650, 450, 650, 500, 1750, 450, 650, 500, 600, 450, 650, 500, 600, 500, 650, 450, 650, 450, 1750, 500, 600, 500, 1750, 450, 1750, 500, 1750, 450, 1750, 500, 1750, 450, 1750, 500,};

unsigned int raw2[32] = {2300, 650, 1100, 700, 500, 650, 1100, 700, 500, 650, 1100, 650, 550, 650, 550, 600, 550, 700, 500, 650, 500, 650, 550, 650, 1100, 650, 550, 650, 1150, 600, 550,};

unsigned int raw3[32] = {2300, 650, 1150, 600, 550, 650, 550, 650, 1100, 650, 550, 600, 1150, 650, 1150, 650, 500, 650, 550, 650, 550, 600, 550, 650, 1150, 600, 550, 650, 1150, 650, 1100,};

unsigned int raw4[32] = {2250, 700, 500, 700, 1050, 700, 500, 700, 500, 650, 1100, 650, 550, 650, 500, 650, 550, 650, 550, 650, 500, 700, 500, 700, 1050, 700, 500, 650, 1100, 700, 500,};

unsigned int raw5[32] = {2300, 700, 1100, 650, 1100, 650, 550, 650, 550, 650, 1100, 650, 550, 600, 600, 600, 550, 650, 550, 600, 550, 650, 550, 650, 1100, 650, 550, 650, 1150, 600, 550,};

unsigned int raw6[68] = {9000, 4500, 550, 550, 550, 600, 550, 550, 550, 550, 600, 500, 600, 550, 550, 600, 550, 550, 550, 1700, 550, 1700, 550, 1700, 550, 1650, 600, 500, 600, 1700, 550, 1700, 500, 1700, 550, 1700, 550, 1700, 550, 550, 550, 600, 550, 600, 500, 600, 550, 600, 500, 600, 500, 600, 550, 600, 550, 1650, 550, 1750, 500, 1700, 550, 1700, 550, 1700, 500, 1700, 550,};

void setup()

{

// Set interrupt pin as input

pinMode(APDS9960_INT, INPUT);

// Initialize interrupt service routine

attachInterrupt(0, interruptRoutine, FALLING);

// Initialize APDS-9960 (configure I2C and initial values)

if ( apds.init() ) {

//Serial.println(F("APDS-9960 initialization complete"));

} else {

// Serial.println(F("Something went wrong during APDS-9960 init!"));

}

// Start running the APDS-9960 gesture sensor engine

if ( apds.enableGestureSensor(true) ) {

// Serial.println(F("Gesture sensor is now running"));

} else {

//Serial.println(F("Something went wrong during gesture sensor init!"));

}

}

void loop() {

if ( isr_flag == 1 ) {

detachInterrupt(0);

handleGesture();

isr_flag = 0;

attachInterrupt(0, interruptRoutine, FALLING);

}

}

void interruptRoutine() {

isr_flag = 1;

}

void handleGesture() {

if ( apds.isGestureAvailable() ) {

switch ( apds.readGesture() ) {

case DIR_UP:

//Serial.print("up");

for (int i = 0; i < 3; i++) {

irsend.sendRaw(raw4, 32, 38); //amp volume up. Must be sent 3 times as per Sony

delay(40);

}

break;

case DIR_DOWN:

//Serial.print("down");

for (int i = 0; i < 3; i++) {

irsend.sendRaw(raw5, 32, 38); //amp volume down. Must be sent 3 times as per Sony

delay(40);

}

break;

case DIR_LEFT:

//Serial.print("left");

apds.disablePower();

delay(220);

irsend.sendRaw(raw6, 68, 38); //computer power. The APDS 9960 Gesture Sensor must be disabled so the Simerec module doesn't read it's IR

delay(220);

apds.enablePower();

irsend.sendRaw(raw1, 68, 38); //TV power

//Serial.println("tv");

for (int i = 0; i < 3; i++) {

irsend.sendRaw(raw2, 32, 38); //amp power. This must be sent 3 times as per Sony

delay(40);

}

break;

case DIR_RIGHT:

for (int i = 0; i < 3; i++) {

irsend.sendRaw(raw3, 32, 38); //amp function. Must be sent 3 times as per Sony

delay(40);

}

break;

case DIR_NEAR:

break;

case DIR_FAR:

break;

default:

Serial.println("NONE");

}

}

}

Step 5: Two Problems I Encountered

I ran into two problems. The first was that my Stereo Amp is a Sony. When Sony sends IR signals, it sends them 3 times quickly. In the Arduino code, if you look at the Cases, each time a Sony signal was sent, it was sent three times. I discovered this fix online. Everything worked perfectly. Except the computer. The TV would turn on, the Amp would turn on, but the computer wouldn't turn on with the Gesture sensor. It worked perfectly with the remote. It also worked sending the code with a separate Arduino, wired with a button. So the code, and the Simerec product worked perfectly as it should, but it wouldn't work with the Gesture Sensor.

Step 6: Unbelievable Customer Support

I grew frustrated, and was at the end of my limited knowledge. So on a whim I sent an email to Steven at Simerec customer support. I fully expected them to tell me that as far as they were concerned, their product was functioning properly, and that I was on my own. Wasn't I in for a wonderful surprise! After going back and forth with information, Steven re-created my project after going out and getting the same modules. He worked at it and came up with a solution. He felt that the TV and Amp were already programmed to receive the IR Signals, but the Simerec IR Receiver was more of a universal receiver, and that the IR used by the APDS 9960 Gesture Sensor was confusing the Simerec Receiver. He wrote an adapted section of code, which temporarily disables the Gesture sensor, sends the IR signal to the computer, then re-enables the Gesture Sensor. You can see that in the "Left gesture" Case. It worked perfectly!! Without Steven's diligence, this project would not have been completed, as I was out of my league!

Step 7: From Breadboard to Perfboard

I then took my project from the breadboard to a perfboard. I'm quite new to all this, so everything might look a little rough...but it works, and I'm getting better with each project! ONE NOTE OF CAUTION...the APDS 9960 operates on 3.3V....the IR Send module on 5V. Don't try to feed the APDS 9960 5v or you'll fry it....I forgot with my first module!

I used standoffs and screws from a kit I purchased on Ebay...(260 Pcs M3 Nylon Bolt Hex Spacers Screw Nut Stand-off Assortment Kit Set Box

( 262516253269 ))

`Not sure why my wiring diagram is upside down...but here is the connections:

From Nano D2 to 9960 INT

From Nano D3 to IR Send S (signal pin)

From Nano Ground to 9960 GND and IR Send -

From Nano 5v to IR Send +

From Nano A5 to 9960 SCL

From Nano A4 to 9960 SDA

From Nano 3.3v to 9960 VCC

That's it!! Now onto building a project enclosure!!

Step 8: Project Enclosure

I 3D printed my project enclosure....I designed it on Tinkercad, then used a Ultimaker 3 Extended to do the printing. I just designed a box big enough to hold everything. I put the usb connection hole into the design, as I know where it would be, but I left the IR LED hole and the 4 holes to screw the lid to the project for after, so I could make sure everything would line up. I'm not sure how to upload my 3D printer design, but it is essentially a box with a square hole in it for the USB connection to the Nano. My material was .04 PLA, extra fine profile, 50% infill. I then covered the box with a carbon-fibre design vinyl wrap, just to finish it off. I had to continue to play with the size of the hole on the top for the APDS 9960 sensor's IR signal. At first I had a hole that was quite small...too small as it turns out. I kept enlarging it until I had consistent gesture reads from the sensor.

I plug it in directly to a USB outlet in the wall, and it sits beside my chair. Although it may be seen as unnecessary, it was more the process of figuring it out that I enjoy, although it is a great conversation starter!!

Share

    Recommendations

    • 1 Hour Challenge

      1 Hour Challenge
    • Beauty Tips Contest

      Beauty Tips Contest
    • Growing Beyond Earth Maker Contest

      Growing Beyond Earth Maker Contest

    Discussions

    Hello there!

    We're glad you want to share something with the Instructables community!

    And we’re here to help you out.

    In order to be published live on the site, Instructables must consist of the following things:

    - Multiple steps showing how you made your project

    - Written instructions in each step

    - Your own original images

    Beyond making your Instructable simply publishable, this guide (https://www.instructables.com/id/How-to-Create-a-Feature-Worthy-Instructable/) explains what is required to have your Instructables featured by our site editors. It’s very helpful, and definitely worth checking out.

    We would love to review your project again after you have made the necessary edits, and we will publish your project if it is eligible.

    If you have any questions, please feel free to ask right here or send us an email at service@instructables.com.

    Best,

    Instructables Community Manager