HackerBoxes 0009: Virtual Worlds


Introduction: HackerBoxes 0009: Virtual Worlds

Virtual Worlds: This month, HackerBox Hackers are exploring Virtual Reality technology. This Instructable contains information for working with HackerBoxes #0009. If you would like to receive a box like this right to your mailbox each month, just subscribe at HackerBoxes.com and join the revolution!

Topics and Learning Objectives for this HackerBox:

  • Understand and Define Virtual Reality (VR)
  • Work with VR Headsets
  • Experience VR Software
  • Explore Bluetooth Communications
  • Interface and Program ATmega32U4 Microcontrollers
  • Interface an Inertial Measurement Unit (IMU)
  • Implement an Inertial Mouse Control Device using an IMU
  • Configure the Inertial Mouse into a VR Glove
  • Work with the HC-05 Bluetooth Module

HackerBoxes is the monthly subscription box service for DIY electronics and computer technology. We are hobbyists, makers, and experimenters.

Hack the Planet!

Step 1: HackerBoxes #0009: Box Contents

  • HackerBox #0009 Collectible Reference Card
  • VR Smartphone Headset
  • Bluetooth Gamepad Controller
  • USB Bluetooth Adapter CSR 4.0
  • Arduino Pro Micro with ATmega32U4
  • HC-05 Bluetooth Module
  • MPU-92/65 Inertial Motion Sensor Module
  • Green Prototype PCB (4x6 cm)
  • Pair of Waterproof Gloves
  • Two Micro Buttons
  • Two Pairs of Velcro Tabs (16x45 mm)
  • DuPont Jumpers 20cm F-F
  • HackerBoxes Decal
  • Exclusive DARKNET Decal

Some other things that will be helpful:

  • Soldering Tools
  • Smartphone
  • Computer with Arduino IDE
  • Glue gun or epoxy

Most importantly, you will need a sense of adventure, DIY spirit, and hacker curiosity. Some of these VR technologies are still fairly cutting edge and will work differently depending upon what type of computer or smartphone you are using. This type of hobby electronics isn't always easy, but when you persist and enjoy the adventure, a great deal of satisfaction may be derived from overcoming the frustration and making things work!

Step 2: Welcome to Virtual Reality

Virtual reality (VR), also known as immersive multimedia or computer-simulated reality, is a computer technology that replicates an environment, real or imagined, and simulates a user's physical presence and environment to allow for user interaction.

Virtual realities may be displayed on either computer monitor or through a virtual reality headset (also called head-mounted display). Some simulations include additional sensory information and focus on real sound through speakers or headphones targeted towards the user. Advanced haptic systems may include tactile information, generally known as force feedback.

VR may include remote communication environments which provide virtual presence of users with the concepts of telepresence and telexistence either through the use of standard input devices such as a keyboard and mouse, or through multimodal devices such as a wired glove or omnidirectional treadmills.

An immersive virtual environment can be similar to the real world in order to create a lifelike experience (for example, in simulations for pilot or combat training), or it may differ significantly from reality, such as in VR games. (from Virtual Reality)

Here is a nice overview entitled, "Explained: How does VR actually work?"
VR was the all the rage at CES 2016. For example, the author of this TIME article reports, "I Finally Tried Virtual Reality and It Brought Me to Tears."

For further (much further) details, check out this MOOC and accompanying free text book from Computer Science Professor Steven M. LaValle at UIUC.

Step 3: VR Headgear for Smartphones

Google Cardboard introduced the world to easily-accessible, low-cost virtual reality by leveraging smartphone technology for processing and display. A simple mobile application is launched on the smartphone and then the smartphone is simply placed into the viewer (also known as the headset).

The smartphone display is split into left and right stereoscopic components. When viewed through the lenses of the headset, the two images are fused into one image with depth for three-dimensional effect.

While Google Cardboard was named for the original fold-out cardboard viewers supplied by Google and others, applicable headsets are no longer limited to those made of cardboard. The nicer plastic variety, such as those seen here, work the same way, but are much more durable and comfortable to use.

Install the Google Cardboard mobile application on your smartphone and give it a try. If your smartphone is running Android, some applications may be controlled by the Bluetooth Gamepad Controller.

The original cardboard versions of the smartphone headsets had magnetic switches. Sadly, the nicer ones do not. In some instances, the Bluetooth Gamepad can be used for triggering inputs. There are some interesting hacks to be found for adding the button functionality such as this one using a simple magnet.

Google's VR offerings will not be ending with Cardboard. Google Daydream was announced at I/O 2016. Here is the Keynote. There was a very nice presentation at Google I/O 2016 on Designing with Daydream if you are interested in some of the nuanced considerations for designing VR apps and media.

Step 4: Virtual Reality Software

Once you have played with the Google Cardboard App a bit, try some others...

YouTube (Android and iOS)

Check out "The best 360 degree and VR videos on YouTube"

FULLDIVE (Android)

Within VR (formerly Vrse) (Android and iOS)

Roller-Coaster (Android and iOS)

Mindroid (Android)

Of course you can find many, many more by searching for "VR" under Apps in the Apple iOS App Store of the Google Play Store for Android.

This project is quite creative and shows some interesting examples of head tracking (using Free Track), display mirroring (using Splashtop), and game controller support (using Keysticks).

Step 5: Bluetooth Wireless Technology

Bluetooth is a wireless technology standard for exchanging data over short distances. It uses short-wavelength UHF radio waves in the ISM band around 2.4 GHz. This presentation on Bluetooth Basics has a lot of history and details on Bluetooth technology.

The Miniature Bluetooth Gamepad can be used as an input device on computers and certain tablets and smartphones. (Newer non-rooted iOS devices do not support this type of interface).

The HC-05 Bluetooth Module can be easily interfaced to a microcontroller platform (such as an Arduino board) as presented later in this Instrucable.

The USB Bluetooth Adapter is based on a CSR 4.0 USB interface that may be used on your PC if it did not come with Bluetooth. Note that the CSR chipset is not supported by certain OSX versions, but Mac laptops generally provide builtin Bluetooth support.

Step 6: Arduino Pro Micro 5V/16MHz

The Arduino Pro Micro is based on the ATmega32U4 microcontroller which has a builtin USB interface. This means that there is no FTDI, PL2303, CH340, or any other chip acting as an intermediary between your computer and the Arduino microcontroller.

We suggest first working with the Pro Micro without soldering the pins into place. You can perform the basic configuration and testing without using the header pins. Also, delaying soldering on the module gives one less variable to debug should you run into any complications.

If you do not have the Arduino IDE installed on your computer, start by getting it form arduino.cc. WARNING: Be sure to select the 5V version under tools > processor prior to programming the Pro Micro. Having this set for 3.3V will work once and then the device will appear to not ever connect to your PC until you follow the "Reset to Bootloader" instructions in the guide discussed below, which can be a little tricky.

Sparkfun has a great Pro Micro Hookup Guide. The Hookup Guide has a detailed overview of the Pro Micro board and then a section for "Installing: Windows" and a section for "Installing: Mac & Linux." Follow the directions in the appropriate version of those installation instructions in order to get your Arduino IDE configured to support the Pro Micro.

We usually start working with an Arduino board by loading and/or modifying the standard Blink sketch. However, the Pro Micro does not include the usual LED on pin 13. Luckily, we can control the RX/TX LEDs and Sparkfun has provided a neat little sketch to demonstrate how. This is in the section of the Hookup Guide entitled, "Example 1: Blinkies!" Verify that you can compile and download this Blinkies! example before moving on.

Step 7: Inertial Measurement Unit (IMU) Module

The Inertial Measurement Unit (IMU) is based on a MPU-9250. The MPU-9250 is a second generation 9-axis MotionTracking device comprising a 3-Axis Gyroscope, 3-Axis Accelerometer, and a 3-Axis Magnetometer.

The schematic shown here demonstrates how to wire up the Arduino Pro Micro and the MPU-9250 to create an Inertial Mouse. One mouse button is also added.

This project (with video) includes an Arduino library for the MPU-9250 and also has some example code to start with for our inertial mouse:

  1. copy the code as shown
  2. remove (comment out) all reference to the right button
  3. change the left button pin to pin 4 (or whichever pin you used)
  4. change pinMode for leftbutton from OUTPUT to INPUT_PULLUP
  5. reverse the active level for leftstate

As wired here (as opposed to in the example code), the button signal is active low since it is pulled up when released and grounded when the button is pressed.

if (leftstate == LOW) { //grounded when pressed
Serial.print(" Left Click! ");
if (leftstate == HIGH) { //pulled up when not pressed

Step 8: Virtual Reality Glove

Once you test out the Inertial Mouse, you can mount it to the Prototyping PCB (optional) or to a piece of cardboard (optional) and then use the provided velcro tabs to adhere it to the back of a glove. The "mouse button" can be affixed to a fingertip using hot glue (or epoxy) so that it can be pressed using the thumb while operating the Inertial Mouse.

Step 9: HC-05 Bluetooth Module

The Bluetooth Module (HC-05) can easily be used in place of the serial monitor interface. For example, get any sketch running on the Arduino that uses serial communications. Do not wire up the HC-05 Bluetooth module yet. From the start, set the data rate to 9600bps. That is the default for the HC-05 so this just makes it easier to transition later.

Get the program running on the Arduino and working with the serial monitor on your computer (make sure to set the computer serial rate to 9600 as well or it will not work). Now you can disconnect the computer (or just turn the IDE and serial monitor off if you still need it for power) and wire up the HC-05. Now Bluetooth is taking the place of the serial monitor. Sync your mobile device's Bluetooth radio to the HC-05 and run an app like BlueTerm on the mobile device. This will let you type to the serial port from the mobile device just as you did from the Arduino serial monitor, but this time, you are wireless.

As an advanced option, if you want to be able to leave pins 0 and 1 connected to the PC (via the USB interface), you can modify whatever sketch you are working with to use the SoftwareSerial and wire the HC-05 rx and tx lines onto two other pins.

Check out this Virtual Reality Skateboard example using a Bluetooth interface.

Step 10: Hack the Planet

We hope you are enjoying your time working with Virtual Worlds. If you enjoyed this Instrucable and would like to have a box like this delivered right to your mailbox each month, please join us by SUBSCRIBING HERE.

Please share your success in the comments below and/or on the HackerBoxes Facebook page. Certainly let us know if you have any questions or need some help with anything. Thank you for being part of the HackerBoxes adventure. Please keep your suggestions and feedback coming. HackerBoxes are YOUR boxes. Let's make something great!

3 People Made This Project!


  • Oil Contest

    Oil Contest
  • Clocks Contest

    Clocks Contest
  • Casting Contest

    Casting Contest

We have a be nice policy.
Please be positive and constructive.




My quick attempt with the Google Cardboard app and my Nexus 5 leads me to believe that the Bluetooth joy pad doesn't work to "select" in the app. Though I have seen a few websites that says similar devices can be made to work. I did grab a magnet and waved it around my phone to manage selection. For those that don't know the actual Google Cardboard devices had a "switch" that was made with a magnetic trapped in the layers of cardboard. The phones magnetometer will pick up the movement.

If anyone has a better way to use the Google Cardboard app and these goggles, please let me know. In addition we should list apps that can successfully be used. I did fire up Shadowrun VR using the goggles and joy pad.

2 replies

Did anyone get the bluetooth gamepad to work with an android phone and the cardboard app?

There are a few different "hacks" for the trigger. This one is fairly elegant if it works for you: https://youtu.be/rQuuE31g4Hs

is this something that would only work with a smartphone? I'd be more interested in a standalone vr kit


This is my first box, and I am struggling with the Bluetooth section, I have used two different Windows computers and can't communicate over serial with the device. I can pair with the HC-05, but the serial port associated with it is always "busy" it also looks like the device disconnects instantaneously. What information do you need to point me in theright direction?

4 replies

If you're trying to use the arduino serial monitor that may not work for the bluetooth connection. Try downloading tera term or putty to interact with the bluetooth serial port. Also make sure your arduino code is using Serial1 and not just Serial for all the Serial1.begin, Serial1.read, and Serial1.write commands.

Serial through the built in usb connection, Serial1 for an addon module using the Tx/Rx pins.

Thank You, I have tried Putty to the same result. I even tried controlling just the bluetooth adapter using the Blinkies sketch from http://sfe.io/t121 and saw the same kind of port busy message. I have wired up the adapter both with and without a voltage divider on the bluetooth RX pin. I have even tried powering the bluetooth adapter off of a separate 5 V supply, I always seem to get the same response. (I even purchased a new adapter from amazon thinking I had fried the RX pin with 5V.) It's like windows doesn't know what to do with the device and stops communicating. I wondered if it was windows 10, but my windows 7 machine is the same, except I don't get ports that are clearly attached to the bluetooth adapter. Again, Thanks for the quick reply.

When I was struggling with basic bluetooth communication I went with a very simple echo program so I could see the back and forth data flow. I'd recommend loading up the below code to your arduino, then connect via putty to the outgoing com port showing up in the bluetooth port details:

void setup()
Serial1.println("Type Something and press enter");

void loop()

Serial1.println("You wrote:");

Ok sorry for your difficulties , this is probably a silly question but have ensured that your arduino IDE serial monitor is not running ? I think you are going to have have to chase your problem down through the device manager . The fact that it pairs is a good thing and means the hardware is talking just fine, so none of the voltages or hardware crap is interfering. That leaves things in the protocol realm or device enumerator / blue tooth enumerator and type , I am using the bluetooth term from the windows app store ( freebie ) , putty is good but this app specifically looks for the bluetooth device ports. Good luck post your progress

Here's the completion of my inertial mouse from this month's box. I did get bluetooth communication working between the arduino and mu computer but I abandoned getting the glove to work via bluetooth and I explain why in the video. A couple things I may adjust in the future.

- Change the delay so the mouse is a little more responsive.
- Add an additional button to set where level is, so I can have my hand resting more naturally without the mouse moving.
Overall I had a lot of fun and frustration but I learned a lot during this project. Can't wait for the next box.
3 replies

Take a look at the BLE libraries , you will find some wonderful things like BLEmouse .move commands and so forth.

Did you get it to work with those libraries?
I took a cursory look, I don't think this gets around the fundamental problem....that windows doesn't see this device as a human interface device(HID). The BLE seems to just change how it communicates, it doesn't maintain an open communication channel just sends data when there's data to send.
If you've had success with this I'd be very interested, but I don't think I'll pursue that with the info I've read about it so far.

If you load the BLE library using the Arduino IDE , then go select Examples from the file menu , near the bottom of the list will be BLEPeripheral , when you mouse over it a drop down list will show with HID mouse over it and you will find an example called HID_joystick_mouse This example code will run if you change the board specific references ...... If you are not comfortable writing the code flag me and I'll code up an example for you when I get a little time Cheers !

There is another way to get it to work remotely without having to buy a $25+ HID Bluetooth module and without a listener program on the computer.

This solution would require 2 arduino devices and 2 Bluetooth modules. The idea would have one arduino, the motion tracker, and a Bluetooth module in slave mode on the glove powered by a battery. The arduino would feed the motion tracker and button info to the bluetooth and that's it. Then connect the other Bluetooth module in master mode to another arduino plugged into the computer. We know that the Pro Micro is recognized as a human interface device by windows, so this should work fine. This module will read the input from the Bluetooth, parse the data and send mouse commands to the computer. This would turn the arduino into a Bluetooth receiver for your machine emulating a HID so you wouldn't need the little Bluetooth receiver for you computer.

Extra components beyond what was in the hackerbox:
- Bluetooth module - either another HC05(master or slave) or an HC06(slave only)
- Arduino, probably something like the pro mini or micro
- batteries.

For the extra arduino you could use the pro-mini that came in box 0008 for the glove since it only needs to communicate via Bluetooth. If I were to buy another arduino specifically for this I would purchase the 3.3V arduino pro mini since that would require less weight for the batteries to supply that voltage. That voltage works fine for all the components on the glove. Note the Box 0008 came with the 5V version of that board, so you'll want a 6V or 9V battery for that one.

The zipper came off my glove the first time I zipped. Love the monthly boxes.

I connected the HC-05 to the Arduino. But what code do I need to make it actually recognize it? Also, is anyone else having the issue that it disconnects the Bluetooth after a few seconds?

Hey all you PC challenged (i.e. Mac OS X) people out there. Has anyone had any luck getting the HC-05 to communicate with their Mac Computer? I'm using El Capitan. So far I can connect via Bluetooth and the HC-05 tty port shows up in the terminal, but I've had no luck with moving data back and forth. I also suspect that the iPhone is a lost cause. Any detailed advice would be appreciated.

2 replies

The data piece may be as mentioned below replace all serial in the sketch with serial1 then recompile you should start seeing data

Indeed. Replacing Serial1 for Serial doest the trick! SoftwareSerial does not seem to work with the Arduino Pro Micro.

Okay here's an update the sketch won't start sending data over bluetooth till you do a find and replace Serial with Serial1 . I set up a terminal to watch the bluetooth data flow and had nothing till I recompiled with the Serial1, Still if you power this circuit separately from the computer the mouse will not work but you will still see the data flowing across the serial link. I downloaded a free bluetooth terminal programmer from the app store ( WIN 10 ) to troubleshoot with. I used the default microsoft bluetooth mgr. to Pair the HC05 and then the app found and connected fine. I though I had this beat when I saw the position data flowing and the mouse cursor moving as I moved the board ..... but then I noticed if I unplugged and ran off a straight power only USB connector the mouse would no longer work , the data for the serial did still flow. This must have something to do with the way the Mouse.h lib defines the port .... Someone didn't do a very good job of really checking this code out. Oh well let you know when I figure it out.