Introduction: The Oculus Cardboard Project: DIY Virtual Reality Gun With Tracking Using Arduino and the ESP8266

By Check me out on YouTube!
About: My name is Matthew and I attend the University of Pittsburgh. Currently I am a senior, going for a bachelors in Information Science with a minor in CS. Current interests include augmented reality, virtual real…

I always wanted to shoot things in virtual reality but I'm broke so I did what I could. This is my attempt at an Oculus Rift style experience with Google Cardboard. This is actually a really fun project and its extremely easy to replicate. The parts cost around $15 total and will take about 45 minutes if you are new to Arduino. It utilizes the smartphone that you already have, the only caveat being it must be compatible with google cardboard. The game was made in Unity 3D with the Vuforia augmented reality plugin. You do need a virtual reality headset which can be had for around $15 as well. This is the one I use: http://amzn.to/296KwbJ

The Android App and iPhone xCode project can be downloaded here as well as the Arduino code (there is also a list of parts with links to exactly everything I used): http://www.wirebeings.com/virtual-reality-gun.htm...

Stay tuned for some updates by subscribing to my youtube channel https://www.youtube.com/channel/UClm2DY6pj3ygKoKhE...

I will be posting the .stl files for 3D printing a more user friendly version of the gun soon, as well as making the tracker smaller and making a better game.

Consumer version coming soon!
Want to buy one? Sign up for the preorder list here: http://www.wirebeings.com/virtual-reality-gun.htm...

Step 1: Gather the Materials:

You will need:

-A Google Cardboard style virtual reality headset.

-2 pieces of cardboard. (8.5in by 11in and 6in by 12.25in)

-ESP8266 ESP-12E NodeMCU

-2 small buttons

-2 male to female jumper wires

-4 female to female jumper wires

-Tape

-A printer

-Printer paper

-A 3.7v rechargable lithium ion battery

-Scissors

Links to all the parts are here: http://www.wirebeings.com/virtual-reality-gun.html

Step 2: Print the Image Target.

Print the image above and tape it to the large piece of cardboard. This is the image target that will be tracked by the front facing camera on your smart phone.

Only put tape around the edges covering the white border. If tape covers the image target it can create reflections that hinder the tracking process.

This game should also be played in well lit areas for a better, more reliable experience. It can still work in low light but you would need to illuminate the flashlight on your phone (reducing battery life).

Step 3: Mount the ESP-8266

Place the ESP-8266 according to the first picture above on your small piece of cardboard.

Push the board down with your thumb so that the pins make indentations in the cardboard.

Take one of your jumper wires and poke those indentations all the way through.

Now you should be able to push the ESP-8266 all the way through the cardboard.

Step 4: Mount the Buttons

Roll up the cardboard according to the first picture and find a grip that feels comfortable.

Using a pen, mark the spot where your thumb and pointer finger hit the cardboard.

Attach 2 female wires to each button as shown in the second picture.

They can either be soldered or you can use needle nose pliers to (gently) secure the wires to the leads on the button. Make sure to connect the wires to the two closest leads on each button, doesn't matter which ones you choose as long as they are on the same side.

Use a pen or something comparable to make two holes in the cardboard where you previously marked according to the third picture.

Pull the wires from the button through those holes and use a jumper wire to make two more smaller holes in the cardboard for the unused button leads.

Push the buttons down into the cardboard as shown in the last picture.

Step 5: Make the Connections and Bring It All Together

Connect one wire from the trigger button to d1 on the ESP-8266, connect the other to a ground pin (labeled GND)

Connect one wire from the walk button to d2 on the ESP-8266, connect the other to a ground pin

Using your male to female wires, connect one to a ground pin (GND) and the other to the VIN pin. Let both of these hang out of the bottom for now as shown in the first picture.

Again these wires can be soldered, or just gently squeeze them on with needle nose pliers if they won't stay in place.

Now, roll the cardboard together as shown in the second picture and you will see there is a problem with the trigger button wires being in the way.

Mark the spot on the cardboard where they are going to hit and undo the connection. Cut a hole there in the cardboard, pull the wires back through the hole you just made, and everything should now sit flush.

It is now safe to tape everything in place like the third picture.

Grab the battery and tape it somewhere along the side of the cardboard like the fourth picture.

Finally, cut two small slits in the top and slide in the image target according to the last picture.

Step 6: Prepare the ESP8266 for Programing

Download the Arduino IDE if you don't already have it: https://www.arduino.cc/en/Main/Software

Now you need to add the ESP8266 Node MCU 12E as a selectable board inside the Arduino IDE.

Follow the instructions detailed here: http://www.whatimade.today/esp8266-easiest-way-to-...

Now you should have all the ESP options as selectable boards when you click Tools -> Boards in the IDE

Copy and paste this code into the Arduino IDE: 

#include <ESP8266WiFi.h>
 #include <WiFiUdp.h>

int buttonPin = 5; //d1 on esp8266 nodemcu
int buttonPin2 = 4; //d2 

bool walking = false;
bool sentWalking = false;
bool sentNotWalking = false;

bool firing = false;
bool sentFiring = false;
bool sentNotFiring = false;

const char* ssid     = "***********"; // wifi network name
const char* password = "*******"; // wifi network password

//IPAddress ipBroadCast(255, 255, 255, 255); //ip of computer network
IPAddress ipBroadCast(255, 255, 255, 255); //ip of mobile network

unsigned int udpRemotePort=2000;
const int UDP_PACKET_SIZE = 28;
char udpBuffer[ UDP_PACKET_SIZE];
WiFiUDP udp;

void setup() {
  Serial.begin(115200);
  delay(10);
  // We start by connecting to a WiFi network
  Serial.print("Connecting to ");
  Serial.println(ssid);
 
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
      delay(500);
      Serial.print(".");
  }
  Serial.println("WiFi connected"); 
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
  Serial.println("Starting UDP");
  
  pinMode(buttonPin, INPUT_PULLUP);
  pinMode(buttonPin2, INPUT_PULLUP);
  
  //send connected message
  strcpy(udpBuffer, "Connected");
  Serial.println("Connected"); 
  udp.beginPacket(ipBroadCast, udpRemotePort);
  udp.write(udpBuffer, sizeof(udpBuffer));
  udp.endPacket();
}

void sendFIRE(){
  
  strcpy(udpBuffer, "FIRE"); 
  udp.beginPacket(ipBroadCast, udpRemotePort);
  udp.write(udpBuffer, sizeof(udpBuffer));
  udp.endPacket();
  }

 void sendWALK(){

  strcpy(udpBuffer, "WALK"); 
  udp.beginPacket(ipBroadCast, udpRemotePort);
  udp.write(udpBuffer, sizeof(udpBuffer));
  udp.endPacket(); 
  }

 void sendSTOP(){

  strcpy(udpBuffer, "STOP"); 
  udp.beginPacket(ipBroadCast, udpRemotePort);
  udp.write(udpBuffer, sizeof(udpBuffer));
  udp.endPacket(); 
  }

   void sendDONT(){

  strcpy(udpBuffer, "DONT"); 
  udp.beginPacket(ipBroadCast, udpRemotePort);
  udp.write(udpBuffer, sizeof(udpBuffer));
  udp.endPacket(); 
  }

void loop() {
  int buttonValue = digitalRead(buttonPin);
  int buttonValue2 = digitalRead(buttonPin2);

 if (buttonValue == LOW){
      firing = true;
   } else {
      sentFiring = false;
   }
   
   if (buttonValue == HIGH){
      firing = false;
   } else {
    sentNotFiring = false;
   }

   if (firing == true && sentFiring == false){
      Serial.println("FIRE");
      sendFIRE();
      delay(200);
      sentFiring = true;
   }

   if (firing == false && sentNotFiring == false){
      Serial.println("DONT");
      sendDONT();
      delay(200);
      sentNotFiring = true;
   }
   
   //end firing
   //start walking
   
   if (buttonValue2 == LOW){
      walking = true;
   } else {
      sentWalking = false;
   }
   
   if (buttonValue2 == HIGH){
      walking = false;
   } else {
    sentNotWalking = false;
   }

   if (walking == true && sentWalking == false){
      Serial.println("WALK");
      sendWALK();
      delay(200);
      sentWalking = true;
   }

   if (walking == false && sentNotWalking == false){
      Serial.println("STOP");
      sendSTOP();
      delay(200);
      sentNotWalking = true;
   }
   
}

Step 7: Upload the Code and Download the App!

Plug the module into your computer with the USB cable.

Now you need to go to your wifi settings on your phone and click the network that you are connected to. It should display an IP address underneath.

Find these lines in the code: 

const char* ssid = "***********"; // wifi network name
const char* password = "*******"; // wifi network password

//IPAddress ipBroadCast(255, 255, 255, 255); //ip of computer network 

IPAddress ipBroadCast(255, 255, 255, 255); //ip of mobile network

On the line where it says "ip of mobile network" replace the "255"s in the code with the IP address you just found (keeping each set of digits separated with a comma as shown).

Replace the stars with your wifi network name, and password, respectively (inside the quotes).

Now, click Tools at the top of the IDE and make sure you have selected "NODEMCU 1.0 (ESP 12-E Module)", also make sure you are on the right COM port and click upload.

Finally, download the app here: http://www.wirebeings.com/virtual-reality-gun.html

If you are using Android you can simply go to that address on your mobile device and directly download the app.

If you need the app for iPhone you need to download the project from the link above and open it in Xcode in order to get the app onto your phone (because of signing permissions). If you don't already have a developer account you will be prompted to sign up for one in order to build to your iPhone.

To power on the board plug in the loose GND wire to the black terminal on the battery and plug in the loose wire connected to the VIN pin into the red battery terminal (unplug these when it is not in use...). The battery should last a decent amount of time, but to charge it you can buy a small USB charger online.

Once the app is open you may need to press the small reset button on the ESP board if it does not connect immediately. The trigger button fires and if you hold the other button down you will walk forward in whatever direction your phone is facing. The game can either be played standing up in one place or in a swivel chair.

Any questions, leave a comment, and I will be glad to help!

DIY Summer Camp Challenge

Participated in the
DIY Summer Camp Challenge

Beyond the Comfort Zone Contest

Participated in the
Beyond the Comfort Zone Contest

Trash to Treasure Challenge

Participated in the
Trash to Treasure Challenge