Markerless Augmented Reality App for BEGINNERS!

This tutorial goes through how to do markerless augmented reality for beginners using the Unity 3D game engine. The app we will be making will work for Android or IOS mobile devices. The app is a space shooter of sorts where you can shoot down spaceships that are flying around your living room. Most of my videos have been using Vuforia so I wanted to do something a little different. It seems the standard in Markerless AR is the Kudan plugin but I decided against that because it costs money to use. Instead we are going to do a augmented reality app from scratch (for free) and augment everything with respect to the phones orientation using it's gyro.

All you need to follow along is an Android or IOS mobile phone to run the app.

All the code and assets used in the game are here for reference:

http://wirebeings.com/markerless-augmented-reality...

Lets get started!

Step 1: Download Unity and Create a New Project

Download Unity if you don't already have it from here: https://unity3d.com/unity

Unity is a free video game engine and that is what we are going to use to make this app.

Make sure to install the packages for IOS or Android depending on what mobile device you are making the app for.

Open Unity and create a new project.

Right click in the hierarchy and create a new 3D object -> plane.

Rotate the plane off to the right by an x value of -90.

Drag your plane onto the main camera making it a child.

Click on the camera and you will notice that some white lines appear coming out of it. That is the camera's field of view.

Position the plane approximately 7 or 8 scene units away from the camera (using its z position) and make sure it is still within the camera's field of view.

Step 2: Add a Webcam Texture to the Plane.

Click on the camera and then off to the right click to add a component.

Add a new script and call it "webCamScript.cs".

Double click that new script and it will open in MonoDevelop.

Add this line of code above the start function to create a public GameObject:

public GameObject webCameraPlane;

Now go back to the scene and you will see the script showing under the camera in the inspector (off to the right) when you click on the camera again.

There is an empty space for the "webCameraPlane" so we need to drag in a reference to it.

To do this drag in the plane we created into this slot.

Now go back to the script in MonoDevelop and add these couple lines:

WebCamTexture webCameraTexture = new WebCamTexture();
webCameraPlane.GetComponent().material.mainTexture = webCameraTexture;

webCameraTexture.Play();

Step 3: Size Your Webcam Texture.

Now if you click play in the editor you will see yourself.

Your webcam feed is being sent to the texture on the plane we created.

Its going to be upside down but that is OK, I will explain why shortly.

For now lets get everything resized and scaled correctly.

While still in the game view click on the plane and change it's x and z scale until the image fills up the entire view.

Try not to change the z position but you may also need to play with the x and y position of the plane to get it centered properly.

Once everything looks satisfactory click the gear in the top right of the rect transform there and copy component values.

Click play again to exit play mode and click the same gear to paste the component values.

We must do this because changes made in play mode to the scene do not get saved.

Step 4: Prepare Texture for IOS or Android.

Now go back to the webcam script and add these couple lines to the start function:

if (Application.isMobilePlatform) {
GameObject cameraParent = new GameObject ("camParent");

cameraParent.transform.position = this.transform.position; this.transform.parent = cameraParent.transform; cameraParent.transform.Rotate (Vector3.right, 90);

}

We need to add this because on mobile platforms the plane and camera setup gets moved 90 degrees. So that code will create a parent object of the camera and rotate the parent object leaving the cameras transforms intact.

Now beyond that there are some differences between building for Android and IOS too.

IOS will invert the image and rotate 180 degrees. So thats why are webcam feed is upside when we click play in the editor. If you are building for IOS leave this as it is. The only other thing you are going to need to change is the x scale so click on the plane and make its x scale value negative, this will invert the image.

If you are building to Android the image should display normally, but this could vary from device to device. So, that being said if you are building or Android click on the plane and rotate it until the plane is right side up and the webcam feed is displaying normally in the Unity Editor.

Now all of the camera functionality should be straightened out and we are ready to add some 3D objects to the scene.

Step 5: Lets Add Some Objects to the Scene.

Go back to the scene and click on the plane. You will see it has a z value of 7 or 8. That is going to be its distance in scene units away from the camera (which should be at 0,0,0).

Right click in the hierarchy and create a cube. Change its scale to 14,10,14.

Move the cube such that the camera is in it's center.

Uncheck the cube's mesh collider and mesh renderer. You should be left with a green outline of the cube.

This is going to be the bounds in which 3D objects can exist. If they go beyond the bounds of this box (through the webcam texture), when you are viewing this on your phone, the objects will disappear from view.

Go to this website and follow the link to download the orange spaceship that can be seen in the demo video: http://wirebeings.com/markerless-augmented-realit...
Unzip the file and drag the ship's .obj file into your assets folder in Unity. Do the same thing with the folder called "Maps."

Step 6: Lets Make the Ship Move!

Drag the ship's .obj file from your assets folder into the hierarchy, placing it in the scene.

Change its scale to (.005,.005,.005).

Click on it's name in the hierarchy to expand it and go down to "mesh01" and click to expand that.

Click on the first child of that called "mesh01_MeshPart0" to highlight it.

Now go to the "maps" folder in your asset folder and drag one of the orange images onto the spaceship. That should give it some color.

Add a component -> new script to the ship and call it "enemyScript.cs".

We are going to add a script to this ship that will cause the ship to move constantly forward and turn 180 degrees every 3.5 seconds.

So, paste in this code:

using UnityEngine;

using System.Collections;

public class enemyScript : MonoBehaviour {

// Use this for initialization

void Start () {

StartCoroutine ("Move");

}

// Update is called once per frame void

Update () {

transform.Translate(Vector3.forward * 3f * Time.deltaTime);

}

IEnumerator Move() {

while (true) {

yield return new WaitForSeconds (3.5f);

transform.eulerAngles += new Vector3 (0, 180f, 0);

}

}

}

Step 7: Almost Done!

Press play in the editor and click back to the scene view.

You will see the ship moving back and forth. Click on the cube and make sure the ship is staying within the bounds of its green bounding box. If not move the ship forward or backward until it does. Don't forget the changes will not be saved in play mode!

Finally, we need to add functionality such that the camera moves according to the position of the phones gyro.

Go back to the webCamScript.cs and make sure it looks like this:

using UnityEngine;

using System.Collections;

using UnityEngine.UI;

public class webCamScript : MonoBehaviour {

public GameObject webCameraPlane;

// Use this for initialization

void Start () {

if (Application.isMobilePlatform) {

GameObject cameraParent = new GameObject ("camParent");

cameraParent.transform.position = this.transform.position;

this.transform.parent = cameraParent.transform;

cameraParent.transform.Rotate (Vector3.right, 90);

}

Input.gyro.enabled = true;

WebCamTexture webCameraTexture = new WebCamTexture();

webCameraPlane.GetComponent().material.mainTexture = webCameraTexture;

webCameraTexture.Play();

}

// Update is called once per frame

void Update () {

Quaternion cameraRotation = new Quaternion (Input.gyro.attitude.x, Input.gyro.attitude.y, -Input.gyro.attitude.z, -Input.gyro.attitude.w);

this.transform.localRotation = cameraRotation;

}

}

Step 8: Build Out to IOS or Android

We are done!

Go to file -> build settings and change your build platform to IOS or Android.

If you are building to Android you should just be able to click build and run, then plug your phone in to your computer via usb.

If you are building to IOS there are a few extra steps.

First plug your phone in and make sure you have xCode downloaded.

Click player settings and change your bundle identifier to something like com."YourName"."YourAppName"

Go to the bottom where it says "camera usage description" and put anything in there.

Now you should be able to click build and run!

When the app opens you will be able to see the space ship flying around you!

If you want to take this a little further watch this video for more:

https://www.youtube.com/watch?v=T6bd_MQ2ass