Gesture Controlled Gaming

Intro: Gesture Controlled Gaming

Hi friends, In this instructable I will show you how to make a gesture controlled gaming with a Arduino UNO without getting into lot of stuff(like getting Arduino into DFU programmer mode or custom drivers or things like that).

I tried some games to check the compatibility of the system I find it is not compatible with NFS and GTA and I find my program stops responding when the game is loaded.

Here I used the serial input and then used a c# program to convert the serial inputs to keystrokes. Even though some Arduino models like Arduino Micro, Leonardo, etc comes with inbuilt capabilities for emulating as a keyboard or a mouse but unfortunately our Arduino UNO does not come with that feature.

I have googled "How to emulate a keyboard using Arduino" but the results are too annoying and it says to get into DFU programmer mode and install a driver and then you can emulate UNO as a Keyboard.

Here I can give you a pretty simple solution for this. In this I used the Serial input from the Arduino to control the game. I just converted the inputs to Keypress and played the game with it.

STEP 1: Setup Your Arduino

Here for simplicity sake I used a ultrasound rangefinder module(HC-SR04) to get the inputs. You may use sensors like Accelerometer(like ADXL330) or any other module to get better gaming experience.

The circuit diagram is shown above for our HC-SR04 module. I also used some leds to know the status and toggle switch to control the input.

The major drawback with this system is your mouse pointer become unresponsive if you try to input at faster rates.So I used the toggle switch and some LED indicators to know the status of the system.

A simple diagram showing how to connect the ultrasound module is also provided. To get better control over your mouse pointer I recommend you to use a switch to stop and start the entire process in the Arduino.

I had calibrated the arduino to work within 60 cms

The arduino sketches for this project is also right here..

#define echoPin 7
#define trigPin 8
int ld = 0;
void setup() {
  Serial.begin(9600);
  pinMode(trigPin, OUTPUT);
  pinMode(echoPin, INPUT);
  pinMode(3, OUTPUT);
  pinMode(4, OUTPUT);
}
void loop() {
  digitalWrite(trigPin, LOW);
  delayMicroseconds(2);
  digitalWrite(trigPin, HIGH);
  delayMicroseconds(10);
  digitalWrite(trigPin, LOW);
  int distance = pulseIn(echoPin, HIGH) / 58.2;
  int del = 0;
  if (distance > 60) {
    Serial.println("p|39");//p|39 = p for press and r for release (39 - keycode for right arrow)
    del = distance - ld;
    delay(200);
    Serial.println("r|39");
    return;
  } else if (distance < 15) {
    Serial.println("p|37");
    delay(200);
    Serial.println("r|37");
    return;
  }
  if ( ld - 1 > distance) {
    Serial.println("p|37");
    del = ld - distance;
    delay(del * 25);
    Serial.println("r|37");
  } else  if (ld + 1 < distance) {
    Serial.println("p|39");
    del = distance - ld;
    delay(del * 25);
    Serial.println("r|39");
  }
  ld = distance;
  delay(10);
}

Actually the sketches seems simpler the time delays and some other statements are provided there for better calibration for the game.

STEP 2: Set Up Your PC

Next is to program your PC to listen to the COM port which the Arduino is connected. In windows systems you can find it in your device manager. You can download the C# program which I used to convert the serial inputs to keystrokes form my Github here.

Then open up the program and specify your COM port and the baud rate (here it is 9600). Then load your game and start to play then press the start button. Now you can play your games from Arduino UNO.

Comments

[tashawn]

which game did you use?

[krishanindia4]

Generic 9axis 9DOF IMU Sensor Breakout Board Module Gyroscope Compass for Arduino. availabla on amazon: https://www.amazon.in/Sensor-Breakout-Gyroscope-Compass-Arduino/dp/B01KT7XCKS?tag=price-in-india-21

[aish ece]

When I tried uploading the code, I am getting an error 'keyboard' not found.Does your sketch include the line '#include<keyboard.h>'?.After downloading and running arduino uno serial to key.exe and connecting the board I clicked on start ,it started listening.Am I following your steps correctly?.

[Nekhil_r]

can i control the mouse using hand gestures

[Jobin Johnson]

Of course. But you need to update the software or you need an Arduino variant which supports emulating mouse.

[saneesh123]

Which game you played

[ChetanP10]

Awesome project dude. Although its cool, can you plz clear some doubts

1. Is is necessary to use those resistors and LEDs, or is the arduino and the ultrasonic sensor combo fine?

2. What is the resistance of the resistors used?

3. Can you please provide a more clearer picture so that i can know where the resistors are to be connected?

4. Where is the toggle switch to be attached?

Plz answer it ASAP.

Thanks and good work

[Jobin Johnson]

No the ultrasonic sensor is enough. The led and the resistors are used only as indicators.

As the toggle switch is used to stop the serial communication from the arduino you can attach to the power supply so that you can cut the power supply to arduino. Or you can attach to a pin of arduino and edit this code a little bit or it can be attached to the wire which power the ultrasound sensor.

[Prans]

Can you upload a schematic for the LEDs and Resistors part.

[pathakmayank]

i am not able to understand the last step . would you please explain it to me

[Seeed Studio]

Cool project！I made a gestured-controlled Pac-man game two weeks ago,which is similar to your project, haha! I used Grove-Gesture module to implement gestured-controlled, and I think it's simpler.Anyway, l like your project ; )

[omer ib]

its so amazing ^^ ...

please can you give use more picture for schematic :D ...

[jjohns5]

keep up the good work :)

[Jobin Johnson]

thanks bro

[Octus]

pls can you give a list of all the components you used??

[Jobin Johnson]

Arduino UNO, ultrasound rangefinder module(HC-SR04),LEDs, Ressistors, and a toggle switch and the software