Head Tracking With a Wii Remote Camera (War Thunder)

Hello everyone ! I want to share with you my first real finished Arduino project. I tried to make a kind of homemade augmented reality. Let me explain it to you :

It is basically a system that uses a camera to track your head moves to adapt it as an X and Y joystick axis. Until there, the project can be adapted on any game you want.

Then, we will link those axes to the head movement in the War Thunder settings, to give a bit more sensations in the cockpit view.

To make this project, you will need :

• An Arduino Uno (the joystick library only works with those but you can make it work with many other ways),
• A Wii remote camera (i really recommend to take an old remote, don't buy a new one, because it's still expensive),
• A 100 kΩ resistor,
• A 100 nF capacitor,
• A 25MHz oscillator (4 pins),
• An IR LED,
• A 3V button cell (to power the LED),
• A little breadboard (17 holes long to hold the components and the camera, 2x more to hold the Nano (if you used this one) besides),
• Some wires and soldering stuff.

I want to thanks Eric Jacob with his Instructable Wii Remote IR Camera Hack With Arduino Interface which gave me this idea.

Step 1: Extract the Camera From the Wii Remote

I took an old broken Wii remote, and i suggest you to do the same. You can still buy one, but it will cost you abot 40$ (expensive !) i think. The remote is sealed with four 3 pointed crosses. Find the right tool or crush the frame. You see the square camera ? Unsold it ! (I tried to but i didn't want to burn the camera. I soldered wires directly on the remote PCB)

Step 2: Add the Components

Now, solder wires to the camera's pins so we can plug it on the breadboard.

The camera works in I²C, but can't work when plugged directly on the Arduino. We will add some components to simulate the DFRobots IR camera. Follow the electric diagram to complete the camera.

Try to make the camera looking at you, at your head's height, typically like a webcam onto your screen.

Step 3: The LED Tracker

Simply tape the LED to the button cell and attach it to your headphone's top. Orientate the LED toward the camera for more efficiency.

Obviously, don't forget to remove the LED to avoid power loss in your battery.

Step 4: The Code

I used the UnoJoy and DFRobotIRPosition library to emulate the joysticks. the principle is very simple :

• We declare the camera as a serial communication and the Arduino as a game controller.
• We get the X and Y value of the recorded IR LED.
• We map it to fit the 10 bits (0-1023) returned value into an 8 bits (0-255) value for the joystick.
• The Arduino sends via the serial com port the joystick values.

Step 5: Configure the "controller"

Once the code is flashed, we need to turn the Arduino into a joystick controller.

• Connect the two pins that are the closest from the USB port to get into DFU mode,
• you can now execute the "TurnIntoAJoystick.bat" file in the library file,
• now, you can unplug/replug your Arduino, and it will be detected as a controller.

Now, your Arduino will appear as a controller for Windows. Go to Config panel > Hardware > Peripherals > Right click on your new controller > Settings > Properties. You can see here a panel which gives you a preview of the joystick axes and button evolution. Note that if the led is not detected, the returned value from the camera will be 255,255. The default position of the stick will be in the down right corner.

Important note : If the arduino doesn't returns anyvalue, even if the code and the board are correct, try to unplug/replug your wires and reset it until the cross shows the down right corner (which means the camera communicates now).

To recover the "Arduino mode", follow the same pattern :

• Jumper the two pins near the USB port,
• execute the "TurnIntoanArduino.bat",
• then unplug/replug your Uno.

Step 6: Link the Joystick in the Game Settings

The controls we want to set are in the "Camera control" settings. Link the X axis to the left-right axis and the Y axis to the upward-downward axis. Test in-game the moves it makes, you can rescale to make the moves as realistic as it can with the "scale" settings.

And it's finished ! well done ! you can now enjoy a more realistic gameplay for your aircrafts game. Note that the depth axis is not present. I will probably make a 2.0 version with 2 LEDs to have a depth notion, but I don't think it will be enough accurate.