AR Plane Detection & Placing an Object

In the previous blog, we have seen 2D Image Tracking, let's see how Unity makes easier to detect the plane in the real world and use that plane to place or spawn any object.

Use case

The most famous living scenario is IKEA. You might know how IKEA places a digital furniture on our living rooms through mobile app.

Source

We'll try to build something similar.

Let's Begin

For this tutorial our goal is to spawn a 3D car on the detected plane.

Settings for Unity

In Unity, create a project and choose an empty 3D template.

First, we need to configure Player Build Settings, Switch the Build platform to your favourite device, then after add the package AR Foundation from Package Manager and a device specific plugin package like ARCore or ARKit, in this case I've used iOS device so downloading ARKit XR Plugin. You can check the configuration steps from here.

Next, delete the MainCamera object added by the 3D template and add the mandatory objects for AR experience to your Scene, AR Session and AR Session Origin.

And inside AR Session Origin, there is a AR Camera, tag it to MainCamera.

Till now, we have completed setting up Unity for AR development.

Next we have to setup few things for Plane Detection and Placing object over detected plane. So let's first do very basic setup for Plane Detection then we can improvise later while doing things for placing object on the plane.

Plane Detection

For this, we need to add AR Plane Manager, a predefined script as a component into the AR Session Origin.

It has 3 fields, Script which contains it's own script, Plane Prefab, this is where we need to add a Game Object of Plane characteristics, and then whenever AR camera detects surfaces or planes on real world, that Prefab will be drawn on those surfaces. Detection Mode, default is Everything means manager will detect every type of plane whether it's Vertical or Horizontal, It can be set as per our requirement. Like only Vertical or only Horizontal.

After adding AR Plane Manager we need to assign some Game Object to Plane Prefab field.

We have 3 ways to do this, we can use various 3D Plane Meshes available in Unity Asset Store or we can use default AR Plane provided by AR Foundation or we can build our custom plane and add components of plane like Mesh Filter, Mesh Renderer, Line Renderer etc.

For this, I'm choosing default AR Plane, this can be found in XR submenu.

And that's it we are done, now drag that AR Plane from Hierarchy to the folder structure of Assets and now it become a Prefab, and remove it from the Hierarchy.

Lastly, drag that prefab to Plane Prefab field of AR Plane Manager in AR Session Origin.

Here, we are done with very basic Plane Detection setup. Build and run it on the device to see how it works.

If everything works fine, then let's move forward with object placement.

Placing an Object

This process requires some understanding about Raycasting.

Raycasting - projecting a ray to help estimate where the AR object should be placed in order to appear in the real-world surface in a believable way; used during hit testing.

Then what is hit testing, Hit-testing lets you establish a pose for virtual objects and is the next step in the AR user process after feature-tracking and plane-finding.

In our case, we found the plane, now we have to perform raycasting to place objects.

To achieve this, we need to write some code here. You can download it from Github

On Assets folder > Right click > Create C# script > Name eg, PlaceObjectScript

double-click and Open in an editor

Import XR libraries

using UnityEngine.XR.ARSubsystems;
using UnityEngine.XR.ARFoundation;

The most important component that we need is ARRaycastManager, so we will let the class know that we need this guy, it's mandatory.

We know declare some variables that will be needed in the script further.

 public GameObject gameObjectToInstantiate;

 private GameObject spawnedGameObject;

 private ARRaycastManager _aRRaycastManager;

 static List<ARRaycastHit> hits = new List<ARRaycastHit>();

gameObjectToInstantiate, in our case we'll assign our 3D car prefab.

spawnedGameObject, when the user tap on screen the gameObject will be assigned here.

_aRRaycastManager, through this variable we can perform raycasting and then place the object on the position where the user tapped.

hits, to get the list of RayCast hits.

right after this, we would need to initialise _aRRaycastManager in the Awake() method.

Remove the Start() method, we're not going to use it for our app.

Next, we need the positions where the user tapped or touched on the screen, for this create a new method as below,

 bool TryGetTouchPosition(out Vector2 touchPosition)
 {        
       if(Input.touchCount > 0)
       {            
          touchPosition = Input.GetTouch(0).position;
          return true;
       }

       touchPosition = default;
       return false;
 }

Now we need to instantiate our gameObject to spawnedGameObject when we get positions of tapped area on screen and true value from above method.

Inside the Update(), write below lines.

We're done with the scripting. Now back to Unity.

Simply drag the created script to the AR Session Origin Inspector section, it must be added as a component.

AR Raycast Manager is automatically added into the components, it's because we told our PlaceObjectScript at the top as RequireComponent(). Just leave that as it is.

Lastly, drag your prefab to the Game Object To Instantiate field.

We can also change the Detection Mode to Horizontal in AR Plane Manager, to avoid detecting Vertical planes.

Voilà, we made it.

Thanks for reading this blog. You can find the source code and demo in Github.

Leave a comment if you found any obstacle while implementing this.

Until Then.

Keep Building Yourself

Photo by Michael Dziedzic on Unsplash