Virtual Worlds: This month, HackerBox Hackers are exploring Virtual Reality technology. This Instructable contains information for working with HackerBoxes #0009. If you would like to receive a box like this right to your mailbox each month, just subscribe at HackerBoxes.com and join the revolution!

Topics and Learning Objectives for this HackerBox:

Understand and Define Virtual Reality (VR)
Work with VR Headsets
Experience VR Software
Explore Bluetooth Communications
Interface and Program ATmega32U4 Microcontrollers
Interface an Inertial Measurement Unit (IMU)
Implement an Inertial Mouse Control Device using an IMU
Configure the Inertial Mouse into a VR Glove
Work with the HC-05 Bluetooth Module

HackerBoxes is the monthly subscription box service for DIY electronics and computer technology. We are hobbyists, makers, and experimenters.

Hack the Planet!

Step 1: HackerBoxes #0009: Box Contents

HackerBox #0009 Collectible Reference Card
VR Smartphone Headset
Bluetooth Gamepad Controller
USB Bluetooth Adapter CSR 4.0
Arduino Pro Micro with ATmega32U4
HC-05 Bluetooth Module
MPU-92/65 Inertial Motion Sensor Module
Green Prototype PCB (4x6 cm)
Pair of Waterproof Gloves
Two Micro Buttons
Two Pairs of Velcro Tabs (16x45 mm)
DuPont Jumpers 20cm F-F
HackerBoxes Decal
Exclusive DARKNET Decal

Some other things that will be helpful:

Soldering Tools
Smartphone
Computer with Arduino IDE
Glue gun or epoxy

Most importantly, you will need a sense of adventure, DIY spirit, and hacker curiosity. Some of these VR technologies are still fairly cutting edge and will work differently depending upon what type of computer or smartphone you are using. This type of hobby electronics isn't always easy, but when you persist and enjoy the adventure, a great deal of satisfaction may be derived from overcoming the frustration and making things work!

Step 2: Welcome to Virtual Reality

Virtual reality (VR), also known as immersive multimedia or computer-simulated reality, is a computer technology that replicates an environment, real or imagined, and simulates a user's physical presence and environment to allow for user interaction.

Virtual realities may be displayed on either computer monitor or through a virtual reality headset (also called head-mounted display). Some simulations include additional sensory information and focus on real sound through speakers or headphones targeted towards the user. Advanced haptic systems may include tactile information, generally known as force feedback.

VR may include remote communication environments which provide virtual presence of users with the concepts of telepresence and telexistence either through the use of standard input devices such as a keyboard and mouse, or through multimodal devices such as a wired glove or omnidirectional treadmills.

An immersive virtual environment can be similar to the real world in order to create a lifelike experience (for example, in simulations for pilot or combat training), or it may differ significantly from reality, such as in VR games. (from Virtual Reality)

Here is a nice overview entitled, "Explained: How does VR actually work?"
VR was the all the rage at CES 2016. For example, the author of this TIME article reports, "I Finally Tried Virtual Reality and It Brought Me to Tears."

For further (much further) details, check out this MOOC and accompanying free text book from Computer Science Professor Steven M. LaValle at UIUC.

Step 3: VR Headgear for Smartphones

Google Cardboard introduced the world to easily-accessible, low-cost virtual reality by leveraging smartphone technology for processing and display. A simple mobile application is launched on the smartphone and then the smartphone is simply placed into the viewer (also known as the headset).

The smartphone display is split into left and right stereoscopic components. When viewed through the lenses of the headset, the two images are fused into one image with depth for three-dimensional effect.

While Google Cardboard was named for the original fold-out cardboard viewers supplied by Google and others, applicable headsets are no longer limited to those made of cardboard. The nicer plastic variety, such as those seen here, work the same way, but are much more durable and comfortable to use.

Install the Google Cardboard mobile application on your smartphone and give it a try. If your smartphone is running Android, some applications may be controlled by the Bluetooth Gamepad Controller.

The original cardboard versions of the smartphone headsets had magnetic switches. Sadly, the nicer ones do not. In some instances, the Bluetooth Gamepad can be used for triggering inputs. There are some interesting hacks to be found for adding the button functionality such as this one using a simple magnet.

Google's VR offerings will not be ending with Cardboard. Google Daydream was announced at I/O 2016. Here is the Keynote. There was a very nice presentation at Google I/O 2016 on Designing with Daydream if you are interested in some of the nuanced considerations for designing VR apps and media.

Step 4: Virtual Reality Software

Once you have played with the Google Cardboard App a bit, try some others...

YouTube (Android and iOS)

Check out "The best 360 degree and VR videos on YouTube"

FULLDIVE (Android)

Within VR (formerly Vrse) (Android and iOS)

Roller-Coaster (Android and iOS)

Mindroid (Android)

Of course you can find many, many more by searching for "VR" under Apps in the Apple iOS App Store of the Google Play Store for Android.

This project is quite creative and shows some interesting examples of head tracking (using Free Track), display mirroring (using Splashtop), and game controller support (using Keysticks).

Step 5: Bluetooth Wireless Technology

Bluetooth is a wireless technology standard for exchanging data over short distances. It uses short-wavelength UHF radio waves in the ISM band around 2.4 GHz. This presentation on Bluetooth Basics has a lot of history and details on Bluetooth technology.

The Miniature Bluetooth Gamepad can be used as an input device on computers and certain tablets and smartphones. (Newer non-rooted iOS devices do not support this type of interface).

The HC-05 Bluetooth Module can be easily interfaced to a microcontroller platform (such as an Arduino board) as presented later in this Instrucable.

The USB Bluetooth Adapter is based on a CSR 4.0 USB interface that may be used on your PC if it did not come with Bluetooth. Note that the CSR chipset is not supported by certain OSX versions, but Mac laptops generally provide builtin Bluetooth support.

Step 6: Arduino Pro Micro 5V/16MHz

The Arduino Pro Micro is based on the ATmega32U4 microcontroller which has a builtin USB interface. This means that there is no FTDI, PL2303, CH340, or any other chip acting as an intermediary between your computer and the Arduino microcontroller.

We suggest first working with the Pro Micro without soldering the pins into place. You can perform the basic configuration and testing without using the header pins. Also, delaying soldering on the module gives one less variable to debug should you run into any complications.

If you do not have the Arduino IDE installed on your computer, start by getting it form arduino.cc. WARNING: Be sure to select the 5V version under tools > processor prior to programming the Pro Micro. Having this set for 3.3V will work once and then the device will appear to not ever connect to your PC until you follow the "Reset to Bootloader" instructions in the guide discussed below, which can be a little tricky.

Sparkfun has a great Pro Micro Hookup Guide. The Hookup Guide has a detailed overview of the Pro Micro board and then a section for "Installing: Windows" and a section for "Installing: Mac & Linux." Follow the directions in the appropriate version of those installation instructions in order to get your Arduino IDE configured to support the Pro Micro.

We usually start working with an Arduino board by loading and/or modifying the standard Blink sketch. However, the Pro Micro does not include the usual LED on pin 13. Luckily, we can control the RX/TX LEDs and Sparkfun has provided a neat little sketch to demonstrate how. This is in the section of the Hookup Guide entitled, "Example 1: Blinkies!" Verify that you can compile and download this Blinkies! example before moving on.

Step 7: Inertial Measurement Unit (IMU) Module

The Inertial Measurement Unit (IMU) is based on a MPU-9250. The MPU-9250 is a second generation 9-axis MotionTracking device comprising a 3-Axis Gyroscope, 3-Axis Accelerometer, and a 3-Axis Magnetometer.

The schematic shown here demonstrates how to wire up the Arduino Pro Micro and the MPU-9250 to create an Inertial Mouse. One mouse button is also added.

This project (with video) includes an Arduino library for the MPU-9250 and also has some example code to start with for our inertial mouse:

copy the code as shown
remove (comment out) all reference to the right button
change the left button pin to pin 4 (or whichever pin you used)
change pinMode for leftbutton from OUTPUT to INPUT_PULLUP
reverse the active level for leftstate

As wired here (as opposed to in the example code), the button signal is active low since it is pulled up when released and grounded when the button is pressed.

if (leftstate == LOW) { //grounded when pressed
Serial.print(" Left Click! ");
Mouse.press(MOUSE_LEFT);
}
if (leftstate == HIGH) { //pulled up when not pressed
Mouse.release(MOUSE_LEFT);
}

Step 8: Virtual Reality Glove

Once you test out the Inertial Mouse, you can mount it to the Prototyping PCB (optional) or to a piece of cardboard (optional) and then use the provided velcro tabs to adhere it to the back of a glove. The "mouse button" can be affixed to a fingertip using hot glue (or epoxy) so that it can be pressed using the thumb while operating the Inertial Mouse.

Step 9: HC-05 Bluetooth Module

The Bluetooth Module (HC-05) can easily be used in place of the serial monitor interface. For example, get any sketch running on the Arduino that uses serial communications. Do not wire up the HC-05 Bluetooth module yet. From the start, set the data rate to 9600bps. That is the default for the HC-05 so this just makes it easier to transition later.

Get the program running on the Arduino and working with the serial monitor on your computer (make sure to set the computer serial rate to 9600 as well or it will not work). Now you can disconnect the computer (or just turn the IDE and serial monitor off if you still need it for power) and wire up the HC-05. Now Bluetooth is taking the place of the serial monitor. Sync your mobile device's Bluetooth radio to the HC-05 and run an app like BlueTerm on the mobile device. This will let you type to the serial port from the mobile device just as you did from the Arduino serial monitor, but this time, you are wireless.

As an advanced option, if you want to be able to leave pins 0 and 1 connected to the PC (via the USB interface), you can modify whatever sketch you are working with to use the SoftwareSerial and wire the HC-05 rx and tx lines onto two other pins.

Check out this Virtual Reality Skateboard example using a Bluetooth interface.

Step 10: Hack the Planet

We hope you are enjoying your time working with Virtual Worlds. If you enjoyed this Instrucable and would like to have a box like this delivered right to your mailbox each month, please join us by SUBSCRIBING HERE.

Please share your success in the comments below and/or on the HackerBoxes Facebook page. Certainly let us know if you have any questions or need some help with anything. Thank you for being part of the HackerBoxes adventure. Please keep your suggestions and feedback coming. HackerBoxes are YOUR boxes. Let's make something great!