How to Markerless Augmented Reality Tutorial: SLAM

This Instructable will show you how to make a markerless augmented reality app for your mobile device. This will work with Android or IOS. We will use the Unity 3D video game engine with the Wikitude SDK and their instant tracking, or SLAM technology. SLAM stands for simultaneous localization and mapping. Wikitude’s 3D computer vision engine tracks the users surroundings in order to localize the device. This will allow us to augment 3D objects without the use of a fiducial marker, or image target.

I have done a few other Instructables on markerless AR but we have only been able to float objects in the air. Wikitude’s instant tracking will allow us to actually detect surfaces so we can place objects on the ground. Today we are going to go through setting up Wikitude’s sample scene for instant tracking. We will also go through modifying this sample scene in order to augment whatever 3D model you want.

So, to get started the free trail Wikitude SDK can be downloaded here:

https://www.wikitude.com/download/

If you don't already have the Unity 3D video game engine, it can be downloaded here:

https://store.unity.com/download

Step 1: Setting Up the Sample Scene.

Once you download the Wikitude SDK you will be redirected to a registration page, after that is completed you will receive a trail license key sent to your email.

Start a new unity project and call it whatever you want. Unzip the Wikitude folder and find the unity project file. Drag this into your assets folder.

Go to file, build settings and switch your platform to Android or IOS.

Go to Wikitude, samples, scenes, and select the instant tracking scene. Go to edit, duplicate, and rename the scene. Drag it into your assets folder so its easy to find. Double click the scene to open it.

The instant tracking algorithm works in two distinct states, there is an initialization state that defines the origin of the tracking, which the user has to actively confirm. Next is the tracking state where augmentations can actually be placed.

To get the most accurate results the user must also enter the height of their device in order to accurately adjust the scale of augmentations within the scene.

Lucky for us this is already done in the sample scene here. If we were to build this out to Android or IOS we would have a working app. The touch inputs are already in place for positioning and scaling the objects as well.

Step 2: Lets Customize It.

Now, say for example we want to add our dinosaur 3D model to this scene. Go to the asset store and download and import this free Allosaurus.

Before we do anything with it lets try to minify the scene a little bit without screwing anything up.

First remove the DockUI component from the dock gameobject.

Delete the model buttons background.

Delete the 4 buttons below the clock button, and rename the first one DinoButton.

Enable the dock game object so we can see the button, then change its sprite to something else.

Change its scale to something a little more manageable, and lets make it black.

Just delete the child text for now.

Lets also get rid of the image and title from the example header.

With the Controller game object selected, expand buttons and models.

Change the size to 1 on each.

Step 3: Create Your Dinosaur Prefab.

Now lets create a dinosaur prefab so it can be added to our scene.

Go to the allosaurus folder, sources, fix and drag in the allosaurus idle into the scene. Change its position to 0,0,0.

Double click the animation in the inspector to reveal it in the asset folder. With that highlighted click edit and change the wrap mode to loop and hit apply.

Create an empty gameObject called dino and make the allosaurus idle its child.

Change the scale of the child component to .5,.5,.5. Rotate the child about its y axis 180 degrees.

Make sure the dino’s parent position is 0,0,0.

With the Dino selected, add a box collider component. This is the area in which the touch events are detected so scale it appropriately.

Now drag this whole game object into your assets folder, this will create a prefab. Remove the current one from the scene.

Click on the controller and drag in the Dino prefab to element 0 of the models.

Step 4: Customizing the Functionality.

In my demo video I had the mesh grid and the button disappear when the model was active in the scene, so lets make that happen now.

We will access everything from Wikitude's scripts from our own script on our dinosaur as to attempt to stick to proper object oriented programming. What I mean by that is we will have our object perform the desired actions without directly changing the scripts already in the scene.

So add a script to the dinosaur prefab and call it DinoController.

Open this in monoDevelop. Also, open up the instantTrackerController script on the Controller game object. Change the modifier from private to public on the grid renderer so we can access it from our other script. Go to the SetSceneActive() function and make sure the grid renderer gets enabled before we loop through the models and set them active or inactive.

Step 5: Turning Off the Grid Renderer.

Switch over to our DinoController Script and now we have to get the button and grid renderer to do what we want.

Inside DinoController we need a first create an InstantTrackerController, just call it trackerScript.

Create a private GameObject and call it ButtonsParent.

Inside the start function lets create the references for our tracker script and ButtonsParent.

Make sure the names of the objects are the exact same as in the scene because they are case sensitive.

Lets make sure that the grid renderer and buttons are disabled when the dinosaur gets instantiated by setting them inactive in the start functions.

Now use Unity’s OnEnable function to set them inactive in the case where tracking is lost and started again.

Use unity’s OnDisable function to set the ButtonsParent active again when tracking stops. We do not have to do this with the grid renderer because the SetSceneActive function of instant tracker controller takes care of this already.

Step 6: Scripts for Reference:

DinoController.cs

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class DinoController : MonoBehaviour {

	private InstantTrackerController trackerScript;
	private GameObject ButtonsParent;

	// Use this for initialization
	void Start () {

		trackerScript = GameObject.Find ("Controller").gameObject.GetComponent<InstantTrackerController> ();
		ButtonsParent = GameObject.Find ("Buttons Parent");

		trackerScript._gridRenderer.enabled = false;
		ButtonsParent.SetActive (false);
	}

	void OnEnable(){

		trackerScript._gridRenderer.enabled = false;
		ButtonsParent.SetActive (false);
	}

	void OnDisable(){

		ButtonsParent.SetActive (true);
	}

}

InstantTrackerController.cs

using UnityEngine;
using UnityEngine.UI;
using System.Collections.Generic;
using Wikitude;

public class InstantTrackerController : SampleController 
{
	public GameObject ButtonDock;
	public GameObject InitializationControls;
	public Text HeightLabel;
	public Text ScaleLabel;

	public InstantTracker Tracker;
	
	public List<Button> Buttons;
	public List<GameObject> Models;

	public Image ActivityIndicator;

	public Color EnabledColor = new Color(0.2f, 0.75f, 0.2f, 0.8f);
	public Color DisabledColor = new Color(1.0f, 0.2f, 0.2f, 0.8f);

	private float _currentDeviceHeightAboveGround = 1.0f;

	private MoveController _moveController;
	public GridRenderer _gridRenderer;

	private HashSet<GameObject> _activeModels = new HashSet<GameObject>();
	private InstantTrackingState _currentState = InstantTrackingState.Initializing;
	private bool _isTracking = false;

	public HashSet<GameObject> ActiveModels {
		get { 
			return _activeModels;
		}
	}

	private void Awake() {
		Application.targetFrameRate = 60;

		_moveController = GetComponent<MoveController>();
		_gridRenderer = GetComponent<GridRenderer>();
	}

	#region UI Events
	public void OnInitializeButtonClicked() {
		Tracker.SetState(InstantTrackingState.Tracking);
	}

	public void OnHeightValueChanged(float newHeightValue) {
		_currentDeviceHeightAboveGround = newHeightValue;
		HeightLabel.text = string.Format("{0:0.##} m", _currentDeviceHeightAboveGround);
		Tracker.DeviceHeightAboveGround = _currentDeviceHeightAboveGround;
	}

	public void OnBeginDrag (int modelIndex) {
		if (_isTracking) {
			// Create object
			GameObject modelPrefab = Models[modelIndex];
			Transform model = Instantiate(modelPrefab).transform;
			_activeModels.Add(model.gameObject);
			// Set model position at touch position
			var cameraRay = Camera.main.ScreenPointToRay(Input.mousePosition);
			Plane p = new Plane(Vector3.up, Vector3.zero);
			float enter;
			if (p.Raycast(cameraRay, out enter)) {
				model.position = cameraRay.GetPoint(enter);
			}

			// Set model orientation to face toward the camera
			Quaternion modelRotation = Quaternion.LookRotation(Vector3.ProjectOnPlane(-Camera.main.transform.forward, Vector3.up), Vector3.up);
			model.rotation = modelRotation;

			_moveController.SetMoveObject(model);
		}
	}

	public override void OnBackButtonClicked() {
		if (_currentState == InstantTrackingState.Initializing) {
			base.OnBackButtonClicked();
		} else {
			Tracker.SetState(InstantTrackingState.Initializing);
		}
	}
	#endregion

	#region Tracker Events
	public void OnEnterFieldOfVision(string target) {
		SetSceneActive(true);
	}

	public void OnExitFieldOfVision(string target) {
		SetSceneActive(false);
	}

	private void SetSceneActive(bool active) {

		_gridRenderer.enabled = active;

		foreach (var button in Buttons) {
			button.interactable = active;
		}

		foreach (var model in _activeModels) {
			model.SetActive(active);
		}

		ActivityIndicator.color = active ? EnabledColor : DisabledColor;
		

		_isTracking = active;
	}

	public void OnStateChanged(InstantTrackingState newState) {
		Tracker.DeviceHeightAboveGround = _currentDeviceHeightAboveGround;
		_currentState = newState;
		if (newState == InstantTrackingState.Tracking) {
			InitializationControls.SetActive(false);
			ButtonDock.SetActive(true);
		} else {
			foreach (var model in _activeModels) {
				Destroy(model);
			}
			_activeModels.Clear();

			InitializationControls.SetActive(true);
			ButtonDock.SetActive(false);
		}
		_gridRenderer.enabled = true;
	}
	#endregion
}

Step 7: IOS or Android?

Before we build out to a device, let's make sure everything will look correct.

Click on the Canvas game object, and go to the inspector off to the right.

To make this look a little better in landscape mode change the reference resolution on the canvas scaler to 960 by 640.

Lastly, click on WikitudeCamera and paste in the license key from your email.

Now we should be ready to build to our device. I am going to go through getting this built out for IOS in the next step because it is a little more involved.

If you are building out to Android, everything will work as expected. Just plug in you phone and go to file, build and run.

If you are going through Android Studio, follow these instructions here:

http://www.wikitude.com/external/doc/documentation...