Augmented Reality Tutorial- PizzaSalad! (Android)

Hey all!

This tutorial is about making your own augmented reality app on android. I only came across this just the other week, and have been making them non-stop; it might be fairly straight forward, but awesomely rewarding when completed. Firstly, I would like to give credit to this website, www.dannygoodayle.com/2013/03/01/making-your-firs... , for a good portion of the learning process.

I am also entering this in the Graphic Design contest. Although this isn't purely graphic design, it does incorporate it, and technically wouldn't work without it.

To help make this instructable as clear as possible, I added LOTS of pictures. This whole instructable can be done at a whopping amount of 0 dollars! Free is always good... Well it is technically free assuming you have access to a computer, android phone, and printer. This instructable can also be done on Mac or pc, the process is the same with the only thing to take note of is to make sure you download the correct versions in regards to your machine. Altogether the downloads take up quite a hefty amount of space, so make sure you aren't down to your last few precious gigabytes. In terms of required skills, just basic computer know-how, such as navigating folders and doubling-clicking :). No coding is needed either. Having skills in any CAD program will allow you to put your own models into the app, but I'll provide the ones I used. (And if you are interested in learning 3d modeling, I would suggest Blender, it is free, quite easy to pick up after watching a few YouTube videos, and has lots of forums and online support for any troubles)...

For my example in this instructable I will be using a pizza salad! What do you get when you add a salad and pizza?? A PIZZASALAD!! Awesome. (And obviously the pizza replaces the lettuce because it tastes soooo much better). Without further ado, let's get started.

Step 1: Creating the Graphic Design

The objective of this step is to create your target image, which is the image you point your camera at to make a 3d model appear over it on the phone

The first thing you want to do create your design. The design can be literally be anything you want. Design to your heart's content! Just two requirements 1) when done it has to be saved as either a .png or .jpeg, and 2) the design has to be asymmetrical, meaning it can't be a circle in the middle of the page, it needs an undisputed top, bottom, left, and right so the camera can orient itself properly. Some cool ideas might be to make it something you put on a birthday card to give it some pizazz, or make it a picture of a stop sign so when you are walking down the street you can impress your family/friends.

There is no required size, it can be scaled down to a business card or up to a giant wall mural, just be aware that if it is too small then the camera might have a hard time registering it.

The image in this step is what I created in Microsoft paint. My graphic design skills don't go beyond that, but programs like InkScape or Adobe Illustrator work too, for more professional looking images. As this instructable is being entered in the Graphic Design contest sponsored by Creative Market you can hop on over there for inspiration, and even download free designs weekly.

Once satisfied with your design save it as a .jpeg or .png, don't print it just yet, however. You might need to edit it, as you will see later on.

F.Y.I I have provided my image as a .png for those who don't plan on making their own 3d model. You could make your own, but it wouldn't make much sense in regards to the whole theme

Step 2: Getting All the Downloads in Order

There are only a few downloads needed, the process is just like any other download; click and let the computer do its thing. You don't need to open them just yet, so once they finish just remember where they are. This is also where your are going to need all that harddrive space. When all said and done it should be roughly 10 gigabytes (I think mine was that). This step can be quite lengthy depending on your internet connection, so make sure it is strong and be prepared to make some coffee.

Download 1: Unity- https://unity3d.com/get-unity/download?ref=persona...

This is where all the magic happens and everything comes together. The link takes you to the page for the free version, which is sufficient for the instructable but there is an option for a more professional version which costs $75 a month. Make sure you download the proper version regarding your machine and software version.

Download 2: Vuforia- https://developer.vuforia.com/downloads/sdk

From here you want to download the "Download Unity Extension." (As seen in 2nd picture). This is the extension that allows the augmented reality to work as it does.

Download 3: Android SDK- http://developer.android.com/sdk/index.html

This is the download that allows us to make the app for android. Once this downloads, open it up and a little install window should pop up. The instructions are like any other install, but what we are looking for here is the sdk. Once it is done and click finish you can exit out of it. The most important part here is to take note of where the "android" folder was created because you will need to reference it later. From the same Vuforia page in download 2 there is a download for andoird sdk, I haven't used that one, and I'm not sure if it is the same, but if anyone does know from experience drop a comment down below!

Step 3: Setting Up Unity

Once all the downloads are completed it is time to start making the magic! I added pictures for reference and clarity for what needs to be clicked/dragged, they are also annotated in case the are out of order/describe what they are for

Open up the first download, which is the Unity program and click new project. It will ask you to create a name for the project, this part is up to you. There will also be some options below, just ignore those leaving them to the default.

Once the new project is created, go to your downloads and double-click on the Vuforia Unity extension (download 2). Once clicked it should bring you back to Unity with a new window in the top left (picture 2). Leave all the checks as they are and click import. Once done, you should see a bunch of new folders inn the bottom window under the projects tab.

Within the "Assets" folder click on the folder named "Qualcomm Augmented Reality." This is what was just imported. Now Withing the Qualcomm folder click choose "Prefabs."

Now there should be blue boxes one one folder (picture 5). Click and drag the box "AR Camera" and drop it in the center window (the one with the grids, that is the viewing window where you see everything come together). It doesn't matter where you drop, as long as it is in the window it's good. The second box we need it the "ImageTarget," and just like the first one, drag and drop anywhere.

In the left window is the hierarchy tree. In there right click on the "main camera" and delete it. It isn't needed because a new "AR Camera" was added. Once done with this step, your screen should look like the last picture: a white grided field and a hierarchy tree with three objects in it.

Step 4: Adding the Image Target

This step is where we add the image from earlier. However, we cannot just throw it in.

Head on over to https://developer.vuforia.com/target-manager, same website as download 2 but a different tab. If you didn't have to before, you are going to need to create an account and log in. Once logged in, within the develop page you should see a "License Manager" and "Target Manager" tabs. Go to Target Manager and click "Add Databse." In the window enter a name, (enter something similar to what you named your project simply to help avoid confusion), and select the "Unity Editor" bubble (picture 3). Note: the first time around it should say "create" instead of "download," download will be in just a bit.

After clicking "create" the window should disappear, and you should see the same "Add Database" button, but with the new database below it, as in the 2nd picture. Next click on the name, which should be blue, then click a button similar to the "add database" that reads "Add Target." That should bring you to a pop-up window that looks like the last picture. Browse for your glorious work of art from the first step, and enter a name in the bottom field. The middle field "width" just sets the width of the image and proportions the rest of the file in respect to that number, I just put 100, but the it doesn't matter because we can always scale the image in Unity.

Go ahead an click "add." The name and image you just added should now be under the "Target Name" list as in the 6th picture. However, once you click add, the image will have to be processed. This means it will be given a 1 through 5 star rating on how well the camera will be able to perceive the image. That is why it is important to make the graphic design with an obvious right-side-up. Sometimes the processing will only take a moment, but it is possible to take longer. After a minute or so try refreshing the page if hasn't given you a star rating yet. I've found that useful.

Now you have to download the image. Click the check box to the left of the images name, then on the right side press the "Download Dataset" button. The download should only take a moment, and once done click on the downloaded file, which should bring you to Unity, just like the Vuforia Unity extension did. This will show you a similar "Import" window, and almost naturally click the "Import" button.

After all that we now have to tell Unity where the Image is. In the hierarchy tree select "ImageTarget" and the window on the right should change to something like picture 5. This is the inspector window which allows you to change settings and a particular object. From the image target to x, y, z coordinates. While "ImageTarget" is still selected open the "Image Target Behaviour" section, find the "Data Set" drop box and the name you gave your Image Target on the Vuforia website should be there. Once selected the white grid system should change to your image (you may need to zoom out to get a better view).

Quite a lot of wording here... sorry :P In short this step was basically taking our image and allowing Unity to accept as a Image Target.

Step 5: AR Camera Settings

One last step before we insert our 3d model. Woohoo!

Back on the Vuforia website where the Image Target was upload, to the left of the "Target Manager" tab there is a "License Manager" tab. Within that tab click the "Add License Key" button and enter an application name in the appropriate field. The name can be whatever you want it to be, but I chose the same as the Image Target to help avoid confusion. The other settings you want are "mobile" because this is going to be phone, and "starter" because it is free. After clicking "Next," you will be asked to confirm the License Key, and just like we all do when accepting terms and services click confirm! You will now be presented with a list of your license keys, if this is your first one then your list might be a little small.

Click on your newly made Key and you should see a gray box with a whole bunch of letters and such (picture 5). Simply highlight and copy it, and head back on over to Unity.

Now this time select "AR Camera" from the list and in the same "Inspector" window, all new setting should appear. First in the "QCAR Behaviour" section, paste the License Key you just copied in the "App License Key" box (picture 4). Then scroll down a bit to the "Data Set Load Behaviour" and check "Load Data Set..." where ... should be the name you created, and once you check that box, a second box named "Activate" will appear just below it and check that one too (picture 1).

Step 6: Importing and Inserting the 3d Model

If you made it this far with no problems, then give yourself a pat on the back! Almost done!

For the people who have no experience in CAD and just want to download what I made I included a .fbx file. Along with some images of the build process.

For people who want to edit what I made go right ahead and share some of your edits in the comments! For those purposes I included a .blend and .stl files.

Whatever route you decide take the important thing is to make sure you export/save it as an fbx file. For other projects I used Solidworks but there is not .fbx option, so I would save it as a .stl then open it in blender to make it a .fbx. The last picture shows where to export it as a .fbx in blender. I would also recommend adding materials while in blender. I have been using the diffuse just to keep it simple. If you want to give your objects texture you are going to need to UV unwrap it and export the texture independently. Also when exporting form blender, while choosing the location and before actually exporting, on the bottom left you have the option to "export selected models only" which you can use if you want. just make sure to select the models before actually exporting.

So once you have decided on a model and exported as a .fbx go into Unity and click the "Assests" tab at the top (I believe it is just to the right of edit). Then move down about half way and click "Import New Assest," navigate to your .fbx file and click import.

Once completed, check in the "Qualcomm Augmented Reality" folder and your file should be there, e.g. picture one. And just like before drag and drop it into the middle window. It doesn't have to be right on top of your Image Target because you can always change its coordinates in the "Inspector" window.

In the hierarchy tree your newly added model should be in there, click and drag it on top of "Image Target." This will make the Image Target a parent to the model (picture 2). If the object is too small in proportion the the image target you can click on it and in the "Inspector" window change its scale, location, and rotation. Keep in mind that if you have print your graphic design big then the 3d model will be big, if you print it smaller then the model will be smaller; it will all appear in scale to one another. A helpful tip, sometimes the Image Target might not be a suitable location either, so you can also click on it from the viewer window and set its scale and location there.

Step 7: Adding Color to Your Model

If you want your 3d model to be gray then you can skip this step, in addition I know this process works when importing Blender files, I have not tried it with other software.

So once you are happy with scaling and everything, click on the "Materials" folder within the "Qualcomm Augmented Reality" folder. You should see a bunch of spheres with different names. Since in Blender I created four different materials, there are four spheres labeled "Material.0..." I could have changed the names while working in Blender, but didn't. Click on one of the (in this case) four spheres and change its color in the "Inspector" window to something like it was in Blender (just below and to the right of "Main Maps" there is a gray rectangle with a dropper icon next to it). For those who downloaded the file I made you'll need to guess and check which material belongs to which portions. I like to keep my materials simple while working in Blender because it makes it simple when adding the colors in Unity. I haven't tried anything complex yet, neither the material options Unity gives you.

Step 8: Preventing a White Screen

This step is a solution I have been using to prevent a white screen appearing when the camera sees the Target Image. I don't know if it will happen every time but on my first two projects this solved the problem, so I do it every time now.

Withing the same "Materials" folder there are two spheres names "EmulatorVideo" and "VideoMaterial." One at a time, click on the material then in the "Inspector," at the top, in the "Shader" drop down menu, select "Unlit" then "Texture." Do this for both "EmulatorVideo" and "VideoMaterial."

Step 9: Getting Ready for Android

Get ready for the home stretch! Just a few last settings tweaks...

Firstly we are going to need to locate the android sdk from earlier on. If you are on a Mac go to "Unity" then "preferences" and for pc go to "edit" then "preferences" all which is in the top navigation area. The preferences window should open up, as in image two, and select the "External Tools" section (picture 2). Then in the "Android SDK Location" box, navigate to where it was installed and saved earlier and select the folder. In case you are having trouble finding it, the install creates its own folder, most likely called ".android" (well that is what it was called on my machine). Once the location is set you can exits out of preferences.

Next go to "File" > "Build Settings." There you should see a list of all sorts of platforms, select "Android" and them "Switch Platfrom" at the bottom (picture 3). Then click "Player Settings," this will open up settings in the "Inspector" window. Under "Other Settings" scroll down to "Bundle Identifier" withing the "Identification" header. There change "Company" to anything you want, it doesn't have to be a real company. Just make sure it says "com.blahblahblah" (picture 1).

As an option you can have the app lock its orientation to either landscape or portrait. Within the same "Player Settings" "Inspector" window go to "Resolution and Presentation" and select which ever check boxes please in the "Allowed Orientation for Auto Rotation" section. Leaving them all checked means it can flip any directions, which I find useful sometimes depending on how wide or tall the 3d model is.

Step 10: Creating an App Icon

This step isn't 100% necessary for making the app work, however it does add finish it off like a ribbon on a present.

Again withing the "Player Settings" navigate to the "Icon" section and there should be a whole bunch of gray boxes. Clicking select will allow you to choose an image of your liking, I just chose the Target Image. Depending on your phone model, you can select the correct pixel dimensions of the icon. I'm using an old Samsung Note II, and not really sure what the proper dimensions are, so I just did the first to and they worked.

Step 11: Finishing Up

FINALLY!! I know you're all thinking that was long and tiring, but hey once you get the hang of it, it should be fairly quick!

At this point plug in your android phone to your computer, and make sure its charged and on. And at this time you should also print out your Target Image (it can be either black and white or in color, makes no difference, but you may opt for color simply for the looks)

From the "Build Settings" window there are two "Build" buttons one build and one build and run. First try the "Build and Run" button, this should launch the app on your phone. However, that didn't work for me, and in that case you should click on just "Build" which will then prompt you to save the file as an .apk. So give it any name you would like, then go to your documents, copy the file you just made, and paste it the phone. (pictures 1 and 2).

If you go about the second method, you will need to download "ASTRO File Manager" from the store, don't worry it's free! (picture 4). From the "Astro" appclick the arrow in the top left then open up "Locations" and select "sdcard" (last picture). Now scroll the bottom of the files and there should be an icon with the name you gave when saving as a .apk. If you changed the Icon earlier on then it will be that, otherwise it will be a black and white box as in picture 5. Once you click on it, it will ask you if you want to install it, of course you do :D, so click "Install," (picture 6), then when it is done click "Open."

Tips: when clicking "Build" a loading box should appear, however if it flashes for only a second, or closes before it reaches the end, click build again because we want the app to load 100%.

If you go back and make a change then hit "Build" again, what I have been doing is clicking replace (when saving and copying to the phone) because there I no need for two apps, unless you want to of the same at which point it might make sense to give them different names. After pasting, head back to "Astro" and click the arrow again then sdcard, as to refresh it, then click on the icon again and this time it will ask you if you want to update it.

Step 12: AND WE HAVE LIFT OFF!!!!

Launch up the app on your phone (at this point you can disconnect it from the computer). Point the camera at your printed out Target Image... and watch magic happen. It is a really awesome feeling you get the first time you get it working! You can move the camera around the paper and in and out to get different views. If the Target Image gets out of sight then the 3d model will be "stuck" in its last position until you point the camera at the paper again.

Step 13: Final Rundown

This instructable was a combination of graphic design and 3d modeling. This was fairly basic, just some pizza in a bowl, but it can become a whole lot more complex, for example making the Target Image some blueprints, and have a 3d model of a building pop-up.

This technology, augmented reality, is also very similar to how Google Glass and Microsoft Hololens work, they use cameras to identify certain objects and place graphics, either 2d or 3d, on top of them.

Virtual reality is when your entire field of view if looking at a screen, displaying real world or computer generated ones. (Oculus Rift for example).

In the Future: I would definitely recommend looking into making more interactive and dynamic augmented realities. A quick search on Google and you'll be able to find all sorts of ideas, Unity a very awesome program. What really caught my eye was making virtual buttons, so when you cover up a specific portion of the target Image, the 3d model would change, either in color or into something completely new!

P.s. This may be a PizzaSalad and taste delicious, but don't forget to eat your veggies too! :P