Introduction: How to Control Almost Anything With a 3D Camera (including Your Arduino)
If you control your application by keyboard, mouse or joystick then it’s easy to control it with hand gestures using a 3D camera. This tutorial will show you how it can be done using Device Emulator, a neat open source piece of software.
Bonus: DeviceEmulator can also control your Arduino!
Step 1: Select a Camera and Install the Drivers and Software
For far interaction aka full-body tracking (1m to 4m) use the SoftKinetic DS311, Mesa Swissranger 4000, Panasonic D-Imager or Asus Xtion.
In this tutorial we use the Creative Senz3D.
3. Download and install SoftKinetic’s iisu middleware
Go to www.softkinetic.com, click on the ‘Download iisu’ button on the homepage, download iisu Free (you will need to register first) and install it.
Launch and test the application.
Now that everything is installed you can launch DeviceEmulator.exe and start emulating
Step 2: Choose What You Want to Control and How You Want to Control It
1. Load one of the scripts by clicking on the "..." icon.
- If you have a close range camera use DeviceEmulator\CloseRange\CI_Example_Script_With_Gestures.iid or any file from the CloseRange folder
- • If you have a long range camera, use DeviceEmulator\FullBody\GestureLibrary_Hands_Manipulation.iid... or any file from the FullBody folder
3. Depending on the device, you can specify what you want to emulate.
- For a keyboard, choose which key will be mapped to your value. Keyboard emulation only works with Digital Values (on/off).
- For a joystick or a mouse:
- With Digital Values (On/Off), choose which button you want to map
- With an Analog Value (continuous value), choose which axis you want to emulate
- For Arduino, you have to choose the Pin and Pin Type. More details on how to use the Device Emulator with an Arduino on the next step.
6. Press "play" to start your emulation
Step 3: Control an Arduino Board
For this we use the Firmata protocole. Its fairly simple to setup.
2. In the DeviceEmulator folder, edit the Settings.ini file with a text editor to set the Com Port on which the Arduino is plugged
3. Choose your values mapped to the DeviceEmulator by following the instructions from the previous step..
You are now ready to make things move with hand gestures :o))