Introduction: Use Your HTC Vive With Processing!

This guide will show you how to add a library Processing that will enable you to use your HTC Vive with Processing so you can view and interact with your sketches in Virtual Reality! It will also cover how to implement the library in a basic sketch.

There are many great avenues for creating immersive experiences with the Vive and other VR headsets (Unity, Unreal, InstaVR), and through creating the ViveP5 library I hope to extend the potential to create VR experiences to Processing so that creators are able to work across an even wider range of software to create weird and wonderful experiences. If you're just starting out, or even if you're a Processing-wizard and are having issues with a particular problem, Processing has an extremely helpful community across its Forum and Stack Overflow that might be able to help you, or have already solved your problem.

In order to undertake this instructable you will need:

- Processing 3 installed (if you don't have it installed you can download it here)

- HTC Vive Headset (controllers are supported, but optional for this tutorial)

- SteamVR installed (this is included with Steam, but if you dont have Steam you can download it here)

- About 20-30 minutes of your time

Step 1: Installing the ViveP5 Library

To support the Vive headset in Processing we first need to add a library that will do the heavy work behind the scenes and help us to interface the headset and controllers with Processing in a simplified way.

  • To start, click this link to download the download the .zip of the most recent build - "ViveP5.zip"

Now that we have the zip we need to extract and move it to the Sketchbook location that Processing references to find your downloaded libraries.

  • To be certain of the referenced library location let's quickly open Processing and go to File > Preferences. The top of this window shows the location of our "Sketchbook".
  • Copy ViveP5.zip. Navigate to your sketchbook folder > libraries, paste the .zip and extract the contents of ViveP5.zip here. Once the ViveP5 folder is extracted feel free to delete the .zip file.

Our library is now in the right location! Now lets check that Processing finds it so we can use it in our sketches.

  • Restart processing and go to Sketch > Import Library and if ViveP5 is visible you're good to go!

______

If you had any issues please refer to these instructions to make sure your installation is correct and that you're running Processing 3.3

If it still doesn't show please feel free to leave a comment / message me or raise an issue on the ViveP5 Bitbucket

Step 2: Understanding the Basics of ViveP5

The following shows how a normal sketch runs in processing. The comments on the right explain each part of the typical sketch (don't copy the following code into processing - it doesn't work yet).

void setup(){        	    		// We setup our sketch window and create
  size(1028,1028, P3D);     		// any objects
}         
void draw(){
  // Process stuff          	// The draw loop is used to run processes
  calculation1();            	// on our objects and make calculations
		    	    	//   eg. (move a box or add two numbers)
                            	// The draw loop is used to then draw all
  // Draw stuff            	// of our objects
  drawStuff();			//   eg. (draw the moved box)
}

When we are using ViveP5 we need to apply a slightly different logic when we're using Processing because we need to draw everything twice (once for each eye as they see from two different perspectives). So the way we have to use the draw() function is different. We want to use draw() function to still do all of our calculations for our objects, but we want to draw everything in a separate function - VRdraw(). Our code now needs to look like this:

void setup(){			// Still we setup our sketch normally
  size(1028,1028, P3D);
}

void draw(){
  // Process stuff		// We still run our processes and calculations
  calculation1();		// in the draw loop
}				// We do not draw in the draw loop anymore

void VRdraw(){			//We do all of our drawing in the VRdraw now!
  // Draw stuff
  drawstuff();
}

Now that we've covered this we can move on and begin to create a working sketch. With a few lines of code we can easily setup a Vive in our scenes.

Step 3: Creating a Vive in Your Sketch

Lets look at a basic sketch that puts a grid of pink spinning boxes in our scene.

void setup() {           <br>  size(1028, 1028, P3D);
}                        
void draw() {
  background(200,100,150);		     // Note: Draw loop is filled with
  camera(1500+(millis()/50), 1500, 	     // all drawing related functions
      1500, 0, 0, 0, 0, 1, 0);	
  perspective(PI/2, PI/3.0, 1, 100000);
  translate(500, 500, 0);
  drawBoxes();
}

void drawBoxes() {			     // function that draws 8x8x8 grid
  for (int i = -1000; i < 1000; i+=200) {    // of rotating boxes
    for (int j = -1000; j < 1000; j += 200) {
      for (int k = 0; k < 1000; k += 200) {
        pushMatrix();
        translate(i, j, k);
        fill(255, 120, 140);
        rotateX(millis() / 1000.0f);
        rotateY(millis() / 900.0f);
        box(50);
        popMatrix();
      }
    }
  }
}

Now lets make some changes to our sketch to add a vive into the scene through implementing ViveP5.

To first import ViveP5 go to Sketch > Import Library > ViveP5 which will add a line to the top of your sketch

import core.viveP5.*;

We can now create an instance of our Vive class and initialize it in the setup loop.

Vive vive;<br>void setup() {           
  size(1028, 1028, P3D);
  vive = new Vive(this);
}

If we look at the previous draw loop - all of the functions are either related to setting up the camera, or drawing the boxes.

The only tricky thing we need to do is remove the background command (this ruins the Vive drawing for some strange reason) and replace it with vive.setBackground(200,100,150).

We no longer need to setup the camera now that we're using the vive so we can remove all of the camera related stuff.

Anything that is drawn needs to be moved to VRdraw() so the draw loop is essentially empty, which in this case means drawBoxes needs to be moved to VRdraw()

We finally need to use the draw loop to tell the vive to run its draw function every frame - vive.draw();

void draw() {<br>  vive.draw();
  vive.SetBackground(200,100,150);
}
void VRdraw(){
  drawBoxes();
}
void drawBoxes() {
  for (int i = -1000; i < 1000; i+=200) {
    for (int j = -1000; j < 1000; j += 200) {
      for (int k = 0; k < 1000; k += 200) {
        pushMatrix();
        translate(i, j, k);
        fill(255, 120, 140);
        rotateX(millis() / 1000.0f);
        rotateY(millis() / 900.0f);
        box(50);
        popMatrix();
      }
    }
  }
}

The .pde of each of these scripts is attached below.

Attachments

Step 4: Get Creative!

Now that you have the basis for a Processing VR sketch and hopefully everything worked - get creative and create some immersive experiences!

If you want to know more about the code and its capabilities refer to the source code in the ViveP5 folder in your library or the Javadoc which is

Step 5: Troubleshooting Issues

"What other functions are included in this package?"

Refer to the Javadoc which is inside your sketchbook folder > libraries > ViveP5 > reference > index.html

"My sketch crashes the first time I run it"

This is normal because usually processing is waiting for SteamVR to launch and if SteamVR takes longer than 5 seconds to launch the sketch will automatically timeout.

"I can think of a faster way to implement some of these functions"

Please contribute to the BitBucket and make your own fork of the repository if you have some crazy ideas you want to test.

"I followed everything and nothing works"

Please leave a comment, message me or raise an issue on BitBucket

"The Vive has eyes the wrong way around"
Write this line into your setup loop after you have initialized the vive: 

vive.swapEyes();

"One eye draws, but the other doesnt / is black"

Look at the Javadoc (inside your sketchbook folder > libraries > ViveP5 > reference > index.html) for the command "setEyes" (try different combinations of numbers between 1 and 5 for each eye int). Write this line into your setup loop after you have initialized the vive with different values for int left and int right: 

vive.setEyes(int left, int right);