Introduction: Apple ArKit Augmented Reality App

Picture of Apple ArKit Augmented Reality App

This augmented reality tutorial is a first look at the Apple ARkit. In this Instructable we will use the Unity 3D video game engine and the Apple ARkit to create an augmented reality zombie app for your iPhone or iPad. This app will allow us to place a virtual zombie in the real world, move it around, and make it bigger or smaller. These instructions are designed for beginners so even if you don't have any previous knowledge of programming or Unity 3D you should still be able to follow along!

The ARkit from Apple uses SLAM tracking and sensor fusion in order to place 3D objects on the ground and other surfaces. To follow along you will need to download some software. You will need Xcode 9 beta, IOS 11 beta, the Arkit Unity package and the newest patch version of Unity 5.6.1p1.

IOS 11: http://www.redmondpie.com/ios-11-beta-download-fe...

Xcode 9: https://developer.apple.com/xcode/

Unity (newest patch version): http://beta.unity3d.com/download/74c1f4917542/Uni...

Unity ArKit Plugin: https://oc.unity3d.com/index.php/s/3hfM9T05P9vOpC...

You will also need an apple device with an A9 or A10 chip. These include:

-iPhone SE

-iPhone 6s

-iPhone 6s Plus

-iPhone 7

-iPhone 7 Plus

-iPad Pro

- 9.7-inch iPad 2017

Step 1: Open the Apple ARkit Sample Scene.

Picture of Open the Apple ARkit Sample Scene.

Make sure to download and install all the software from the intro. When you are instaling Unity, make sure to check the box in order to install the packages for IOS support.

Open up the Unity app and click the button to start a new project. Call it whatever you want.

Find the Apple ARkit Unity package and drag it into the assets folder of your project. Click import.

Go to file, build settings, and switch your platform to IOS.

In the hierarchy off to the left uncheck the "PointCloudExample," "ARKitControl," and "GeneratePlanes" game objects. Right click the "RandomCube" game object and delete it.

Now lets download our zombie model. Click on the asset store tab and search "Free Zombie Walk."

Find the second choice and download and import that model into your project. You will be prompted to create a free Unity account if you don't already have one.

Step 2: Lets Add Our Zombie.

Picture of Lets Add Our Zombie.

Click the "HitCubeParent" game object to expand it and select it's child. Right click it's child to delete it.

Expand the zombie folder and find "z@walk." This is our zombie model that includes a walking animation. Drag that game object on top of the "HitCubeParent" to make it a child.

Select the "HitCubeParent" object again and make its z position 0.

Select zombie object and set it's x, y, and z scale to 1.

Click the cog wheel at the top right of it's Animator component and remove it.

Click the add component button and add an Animation component. Uncheck "Play Automatically."

Find the zombie object back down in the assets folder and select it. Look up to the inspector and select the "rig" tab. Change the animation type to legacy and hit apply.

Expand that same zombie object in your assets folder and go back up to the zombie object in the hierarchy and select it. Drag the walk animation from the zombie in the assets folder to the slot in the animation component of the zombie in the hierarchy.

Step 3: Add the Zombie Script.

Picture of Add the Zombie Script.

Click the add component button again with the zombie game object selected.

Type in "UnityArHitTestExample" and add that script.

There is an empty slot for a transform, so drag the "HitCubeParent" game object into that slot to create the reference.

Add another component to the zombie and type in "ZombieControl" and click to add that as a new C# script.

Double click that new script to open it in MonoDevelop. Before we go any further, minimize MonoDevelop and go back to Unity. Click on the "UnityARHitTestExample" script to open that as well, because we will need to make some changes to it shortly.

Now we need to add some functionality for our zombie. We are going to create some functions so that our zombie can move and so that we can scale it's size up and down. We will have a bigger and smaller function that will take care of changing the scale of the zombie, we will have a look at function so the zombie will turn and look at the camera, and we will also have a walk button that will move our zombie forward and play its walking animation.

Step 4: ZombieControl.cs

Picture of ZombieControl.cs

Copy and paste this code into the Zombie.cs file.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class ZombieControl : MonoBehaviour {

	private Animation animation;
	private bool shouldMove = false;

	// Use this for initialization
	void Start () {

		animation = GetComponent<Animation> ();
	}
	
	// Update is called once per frame
	void Update () {

		if (shouldMove) {

			transform.Translate (Vector3.forward * Time.deltaTime * (transform.localScale.x * .05f));
		}
	}

	public void Walk(){

		if (!animation.isPlaying) {

			animation.Play ();
			shouldMove = true;
		} else {
			animation.Stop ();
			shouldMove = false;
		}
	}

	public void LookAt(){

		transform.LookAt (Camera.main.transform.position);
		transform.eulerAngles = new Vector3 (0, transform.eulerAngles.y, 0);
	}

	public void Bigger(){

		transform.localScale += new Vector3 (1, 1, 1);
	}

	public void Smaller(){

		if (transform.localScale.x > 1) {
			transform.localScale -= new Vector3 (1, 1, 1);
		}
	}
}
?>

Step 5: Modify the Touch Input.

Picture of Modify the Touch Input.

We actually only need to modify a small portion of this script. We need to reset the local position of the zombie when we touch on the screen since we are allowing it to move or walk from its original position. We also need to prevent touch events on our UI buttons from registering as touch events in this script. To make it easier just copy and paste this code into the UnityARHitTestExample.cs script, replacing everything that is already there.

using System;
using System.Collections.Generic;
using UnityEngine.EventSystems;

namespace UnityEngine.XR.iOS
{
	public class UnityARHitTestExample : MonoBehaviour
	{
		public Transform m_HitTransform;

        bool HitTestWithResultType (ARPoint point, ARHitTestResultType resultTypes)
        {
            List<ARHitTestResult> hitResults = UnityARSessionNativeInterface.GetARSessionNativeInterface ().HitTest (point, resultTypes);
            if (hitResults.Count > 0) {
                foreach (var hitResult in hitResults) {
                    Debug.Log ("Got hit!");
                    m_HitTransform.position = UnityARMatrixOps.GetPosition (hitResult.worldTransform);
                    m_HitTransform.rotation = UnityARMatrixOps.GetRotation (hitResult.worldTransform);
                    Debug.Log (string.Format ("x:{0:0.######} y:{1:0.######} z:{2:0.######}", m_HitTransform.position.x, m_HitTransform.position.y, m_HitTransform.position.z));
                    return true;
                }
            }
            return false;
        }
		
		// Update is called once per frame
		void Update () {
			if (Input.touchCount > 0 && m_HitTransform != null)
			{
				var touch = Input.GetTouch(0);
				if (touch.phase == TouchPhase.Began && !EventSystem.current.IsPointerOverGameObject(0))
				{
					transform.localPosition = Vector3.zero;

					var screenPosition = Camera.main.ScreenToViewportPoint(touch.position);
					ARPoint point = new ARPoint {
						x = screenPosition.x,
						y = screenPosition.y
					};

                    // prioritize reults types
                    ARHitTestResultType[] resultTypes = {
                        ARHitTestResultType.ARHitTestResultTypeExistingPlaneUsingExtent, 
                        // if you want to use infinite planes use this:
                        //ARHitTestResultType.ARHitTestResultTypeExistingPlane,
                        ARHitTestResultType.ARHitTestResultTypeHorizontalPlane, 
                        ARHitTestResultType.ARHitTestResultTypeFeaturePoint
                    }; 
					
                    foreach (ARHitTestResultType resultType in resultTypes)
                    {
                        if (HitTestWithResultType (point, resultType))
                        {
                            return;
                        }
                    }
				}
			}
		}
	}
}

Step 6: Add Some Buttons.

Picture of Add Some Buttons.

Save both of the scripts in MonoDevelop and close them. Return to Unity.

Now we are going to create all of our buttons that will call the functions we created in the ZombieControl.cs file.

Right click anywhere in the hierarchy and create a UI Button.

This will automatically create an even system and a canvas.

Click on the canvas game object and change the UI scale mode from constant pixel size to scale with screen size.

Click the button game object and change the alignment to center on both groups of buttons.

Change both the horizontal and vertical overflow from wrap to overflow.

Click on the button's child text and make it say "Look At" or something similar.

Scale the button however you would like and up the font size to fill the button.

Change the buttons anchor point to bottom left.

Step 7: Finish the User Interface.

Picture of Finish the User Interface.

Duplicate the button we just made 3 times.

Drag each button to the appropriate place according to the image above.

Change each buttons child text to read like the image above.

Change the anchor point to center for the middle two buttons, and bottom right for the right most button.

The last thing we need to do is map each button to its respective function in the zombie control script.

So click on each button and find the button (script) component. There is a section that says OnClick() with a plus symbol at the bottom right corner. Click that plus button and drag the zombie object into the game object slot. Under the function section find the zombie control script and find the appropriate function for each button.

Do this same process for each button and we are done!

Step 8: Get the AR App on Your IPhone or IPad!

Picture of Get the AR App on Your IPhone or IPad!

Now to get this AR app on your phone go to file, build settings, and click add open scenes.

Click player settings and put in something for the bundle identifier.

It must be in this format: com.YourName.YourAppName

Make sure to add something for the camera usage description. I typically type Augmented Reality.

Click build and save the file somewhere that you will remember!

If you don't already have one, create a free apple developer account from here: http://www.idownloadblog.com/2015/12/24/how-to-cr...

Open up Xcode 9 beta and click to open an existing project from the bottom right. Choose the project that we just built out.

Choose your personal development team and plug your phone in via USB. Click play to load the app onto your device!

Let me know if you have any questions in the comments! Thanks for looking!

Comments

Swift45 (author)2017-09-21

Hi

I made the entire project and at the end, I encountered a problem. It is like when we are adding the functions to the UI Buttons, then after selecting the z@walk object, there is no way I can assign each button their respective functions.

Please help me out. Thanks in Advance!

Check the screenshots provided.(Mainly the 3rd one)

MaxK80 (author)Swift452017-10-18

Hey this was like a month ago, but it's because there are a few errors on the script ( ZombieController), first Animation animation... replace all the animation ( lower case ) with something else like zombieanimation as animation is an existing method and causes problem and then the the whole line inside the walk method, comment it out or create a temp variable for it.

Then when you save it with no errors it will let you select them on the IDE, all of this because you are using a newer version of Unity

UsmanK39 (author)2017-10-05

followed all the steps and the zombie appeared walking on my computer.
However, when I install the app on my iphone, all I see is a blue screen
with 4 buttons (Look At, -, +, Walk). Why is that so?

kimele0n (author)2017-09-11

Hello! breezed through the unity tutorial, thanks! had some difficulties with the Xcode section but finally got it to build. However I now have the same question as last post. No camera feed. Blue screen with 4 buttons. Did anyone manage to solve this? Thanks for the tutorial! I've also just done another tutorial using the Xcode AR project and the camera works fine, so not sure where it's going wrong, it must be a setting in Unity? Where else do you reference "Make sure to add something for the camera usage description. I typically type Augmented Reality." besides in the Player settings?

chenyd00 (author)2017-07-31

need your help! same question as last post. blue screen with 4 buttons. camera die? Check attach picture.

CrazyLeopard128 (author)chenyd002017-08-19

You probably need to allow camera access?

CrazyLeopard128 (author)2017-08-18

Any chance that you could provide a download link for an IPA file, please?

yinjiab (author)2017-08-11

I follow your step and can't find function with 4 button and it said some mistake .I don't no how to do now

buikhanhtoan1998 (author)2017-07-26

I followed all the steps and the zombie appeared walking on my computer. However, when I install the app on my iphone, all I see is a blue screen with 4 buttons (Look At, -, +, Walk). Why is that so?

talthomp made it! (author)2017-07-16

I'm having trouble seeing my buttons in the viewer once I add them to the project.

wuchao00427 (author)2017-07-10

Hello, it is really cool. I follow your steps , finished it on the xcode, and launched it on the ipad mini 2. however,it can not show the zombie and the reality. just a blue screen, seems that the camera died. should i need to set something on my ipad?

michaelgcypher (author)2017-07-08

It is still passing through the touch... even though that script is in there to prevent it

BTP2 (author)2017-07-02

Hi, I have an error in Xcode. It's about "Thread 38". I don't know what to do...

I hope for an answer, thanks

CrazyLeopard128 (author)2017-06-30

Hi, is there any way that i can finish this tutorial and get it onto my device without xcode? I don't own a mac, only windows.

yeah you should be able to get xcode for windows now, not sure about the new beta version though unfortunately.

Nope - Clicked on download for Xcode 8 and it just took me to the web version of the Mac App Store without offering me a download for windows. Thanks for trying to help though! :)

piginhat (author)2017-06-29

Awesome tutorial thanks. How would I create the dotted square that is in the Apple AR example, it then becomes solid when a plane is detected ready to add the 3D object?

SriyaaS (author)2017-06-20

Hi ! this looks great, thanks for sharing it! I was wondering once i have created this app on an iOS device, can it be viewed by someone who on android/windows.

enzym (author)2017-06-19

Do we need to do everything in Mac? Or I could use Unity in Windows, then switch to XCode in Mac? Currently my MacBook cannot run Unity (too old), but can still do XCode 9.

TobiasR27 (author)2017-06-13

Awesome Tutorial, but I think there is a little error?!

Shouldn't it be "animation = GetComponent ()<Animation>;" ?

in your code above?

matthewh8 (author)TobiasR272017-06-13

My bad, the code editor here sees it as a tag. Thanks for pointing that out!

About This Instructable

27,954views

22favorites

License:

Bio: My name is Matthew and I attend the University of Pittsburgh. Currently I am a senior, going for a bachelors in Information Science with a ... More »
More by matthewh8:Apple ArKit Augmented Reality AppAugmented Reality Fidget SpinnerFacial Recognition With Tracking.js
Add instructable to: