Introduction: How to Augmented Reality Tutorial: Virtual Buttons

Picture of How to Augmented Reality Tutorial: Virtual Buttons

This tutorial is geared towards beginners so you won't need any prior experience with AR, Unity, or Vuforia. The video above goes through everything step by step. Overall, the video shows you how to implement virtual buttons with the augmented reality Vuforia SDK in Unity. This tutorial shows you how to create a zombie scene on top of a fiducial marker and then play a walking animation when your hand goes over the virtual button. Upon trying to figure out how to use these virtual buttons I quickly found that there is not much information online. I hope this video helps some people!

All you need to follow along is a computer with a webcam.

As far as software you will need Unity 3D: https://store.unity.com/?_ga=1.97655965.303547796....

and the Vuforia SDK: https://developer.vuforia.com/downloads/sdk

Thanks for watching!

Step 1: Start a New Unity Project.

Picture of Start a New Unity Project.

Start a new Unity project and delete the main camera on the left (in the hierarchy).

Drag in the Vuforia SDK to your assets folder.

Navigate to the Vuforia folder, prefabs, and drag in the ARCamera prefab into the scene.

Create an account on Vuforia.com if you don't already have one.

Find an image you want to use as a target, could be anything as long as it is rich in detail.

Print out the image and then upload a copy to your Vuforia developer portal where it says target manager.

Download the image database from Vuforia for Unity.

Drag that image database into your asset folder in Unity.

Go back to Vuforia one last time and copy your license key.

Click on the ARCamera in your scene (in Unity) and off to the right you will see a place to paste your license key.

Step 2: Set Up Your Unity Scene.

Picture of Set Up Your Unity Scene.

Once your license key is in, go down to database load behavior script and check the box to load image targets and activate.

Go to the Vuforia folder again and drag in the ImageTarget prefab on top of the ARCamera so it becomes a child.

Now, click on the ImageTarget prefab off to the left and to the right you will see Image Target Behavior Script, choose ImageTarget under database and choose your image from the ImageTarget dropdown.

Right click the ImageTarget off to the left and create a 3d object, plane.

Resize the plane with the controls on the right so that the plane just covers your image.

Now, grab the grass texture from here and drag it into your assets folder. Now drag that same image onto your plane in the scene.

When you click play in the editor now, you should be able to see the grass over your image when you hold it in front of your camera.

Step 3: Create Your Virtual Button.

Picture of Create Your Virtual Button.

Find the VirtualButton prefab in the Vuforia prefabs folder and drag that on top of your image target in the scene to make it a child.

Rename your virtual button "actionButton" in off to the left.

Click on the virtual button and change its name (off to the right) "actionButton" as well under where it says Virtual Button Behavior.

In the scene view drag your button over a feature rich portion of your image. Try to make it approximately 10% of your total image size, by resizing it with the controls on the top left of the screen.

Now, click on the Image Target off to the left, and go to the inspector off to the right. Scroll to the bottom and click add component, new c sharp script.

This will open up Mono Develop.

Step 4: Add This Script!

Call this new script: virtualButtonScript.cs

Delete everything currently in the script and replace it all with this:


using UnityEngine;
using Vuforia;

public class virtualButtonScript : MonoBehaviour, IVirtualButtonEventHandler
{

	GameObject zombie;
	/// <summary>
	/// Called when the scene is loaded
	/// </summary>
	void Start() { 
	
		zombie = GameObject.Find ("zombie");
			
			GameObject virtualButtonObject = GameObject.Find ("actionButton");
			virtualButtonObject.GetComponent<VirtualButtonBehaviour> ().RegisterEventHandler (this);
			

	}

	/// <summary>
	/// Called when the virtual button has just been pressed:
	/// </summary>
	public void OnButtonPressed(VirtualButtonAbstractBehaviour vb) { 
	
		Debug.Log("button Pressed");
		zombie.GetComponent<Animation> ().Play ();
	
	}

	/// <summary>
	/// Called when the virtual button has just been released:
	/// </summary>
	public void OnButtonReleased(VirtualButtonAbstractBehaviour vb) { 
	
		zombie.GetComponent<Animation> ().Stop ();


	}
}

Hit control "s" to save.

Step 5: Almost There!

Picture of Almost There!

Now, all the button functionality should be there so we just need to add a physical button so you can see it when the reality is getting augmented (lol idk).

Right click the virtual button (actionButton) and create a new plane. Resize the plane so it just covers the transparent blue button.

If you want to change the color, right click in the assets folder and create a new material, change the color of that material and then drag it onto your new button.

If you want to add text to the button like I have done, right click the plane and add 3D text to it. You will have to resize the text and rotate it until it looks correct.

Step 6: Add Your Zombie.

Picture of Add Your Zombie.

Go to the asset store tab and search for "free zombie with animation."

Find the one pictured above and hit download and then import it into your scene.

In your new zombie folder go to model and click "z@walk."

From the inspector change animation type from generic to legacy.

Drag your zombie in the scene and resize it how you want it.

Change its name in the hierarchy on the left to "zombie."

Under its animation component off to the right uncheck "play automatically."

Step 7: Click Play and Prepare to Be Amazed.

Now simply click play and you should be able to put your hand over the portion of your physical image that you assigned a button to, and the zombie will play its walking animation.

This works much better on a smart phone so you can go to file, build settings, and switch them to either Android or IOS. This will take a minute or two. Now, go to File build settings again and click player settings. Change the bundle identifier to something like "com.yourName.YourAppName."

Plug in your phone and click build and run.

For android just follow the prompts it will have you download a few things if you don't already have them, if you have IOS you will need to download xCode and create a free developer account.

Any Questions don't hesitate to ask.

Check the video for more step by step instructions.

Thanks for looking!!!

Comments

ЕвгенийК51 (author)2017-06-08

Hi!

I look the video https://www.youtube.com/watch?v=Fgd21lbhikU and try to do the same, but... if I lay paper in front of me webcam I see zombie and green grass but if I make "Build&Run" I see on my mobile only zombie without green grass. Where the grass was gone? What is wrong?

Thanks.

kdewitt (author)2017-01-12

Mathew I would like to make this, however I'd like to swap out the Zombie figure for a dinosaur. How are these figures created? What tools are needed to create a 3d animated figure? Also if I want to create various augmented reality scenes how would I share them for others to use and experience? Aurasma has its own community and app a viewer must install. I could export this to my phone but how would I share with others? Thanks so much!

matthewh8 (author)kdewitt2017-01-12

So I don't know much about 3d modeling and character creation but you can just google free 3d model and import any of those into unity by dragging them in, or just search for free characters on the asset store in Unity. To share with others you need to actually put the app on the google play store or apple app store.

kdewitt (author)matthewh82017-01-24

Mathew, I posted a new comment this morning. I've started making this and got to the animation step and realized that my Unity environment is different from yours for some reason. I don't have the Animation tab next to the console or projects tabs. I'm stuck at this step. Any ideas?
Thanks
Also note at the bottom of the image there are some error and warning messages. These appeared when I first imported the vuforia packages. Not sure if they will mess things up.

kdewitt (author)matthewh82017-01-12

Mathew would I be able to put the app on my website as another option?
Thanks,
Keri

nagarjunabujji (author)2016-11-22

i followed the tut wat u uploaded by i'm nt able to control the buttons, and i'm writing script in visual studio an updated of an unity, mono developer. plz help me to solve it.

Thanks.

OferH made it! (author)2016-09-25

I had no Unity or Vuforia skills, i'm totally beginner at AR. I followed the instruction and easily succeed having this Zombie walking on my mouse pad.

Matthew, it is not simple to write so clear and simple instruction that actually works well - thanks! also thanks to you I would try making more stuff with Vuforia and Unity.

and of course thanks to Vuforia for making this amazing plugin and let us (hobbyist) use it for free.

and uni

matthewh8 (author)OferH2016-09-26

Thats awesome, looks good!

GGriffinGG (author)matthewh82016-11-13

Hey Matt, I've followed up to the point where I'm supposed to test the image with my camera. When I hit play, nothing happens. No video. Why isn't my webcam hooked up for this?

cuyler1 (author)2016-09-22

I got my 22 year old nephew to view this and it blew him away! I think you need an internet patent lawyer,BIG TIME! Getting ahead of the others attempting to do what you/them have accomplished will be a heady race. I'll never attempt this but think it is wonderful and I'd like to have some creature dancing on my motorcycle helmet!

matthewh8 (author)cuyler12016-09-23

lol yeah but honestly I cannot take the credit on this one, the real credit goes to the people at Vuforia that created this amazing plugin.

Renauld (author)2016-09-22

First, I am amazed what can do now a days. I am in computer since 50
years (yes started wtih 2nd generation, the one with transistors, octal
and 4 Kbytes of memory). And I am proud of myself that I could follow
and understand your explanation. The implication is that say on Skype
you can pretend that somebody is with you as long as you have a movie
from long ago. I can see a Krimi built on that.

matthewh8 (author)Renauld2016-09-23

Yeah that's exactly right, when projection based AR comes out that will be a game changer too.

snoop911 (author)2016-09-17

Do you have any thoughts on AruCo, the AR library for OpenCV? I'm interested in detecting markers and overlaying information, something like the following but not sure what it uses:

The only Unity / AruCo tutorial I could find is not in English :(

http://tips.hecomi.com/entry/2015/09/16/014119

sgelbart (author)snoop9112016-09-23

super cool!

matthewh8 (author)sgelbart2016-09-23

Thanks so much!

matthewh8 (author)snoop9112016-09-18

Actually After watching that video I think all of that stuff could be done in Unity pretty easily using the Vuforia plugin.

matthewh8 (author)snoop9112016-09-18

So I have not even heard of that yet, but that's something I would like to start playing with soon. I have had a few people ask me to do custom AR stuff that I don't think is possible with Kudan or Vuforia so I was probably going to start looking into learning some stuff with OpenCV soon.

About This Instructable

20,588views

249favorites

License:

Bio: My name is Matthew and I attend the University of Pittsburgh. Currently I am a senior, going for a bachelors in Information Science with a ... More »
More by matthewh8:Apple ArKit Augmented Reality AppAugmented Reality Fidget SpinnerFacial Recognition With Tracking.js
Add instructable to: