Scanning the rooms of a single structure

Create an AR experience that enables people to scan a building that contains multiple rooms.

Overview

RoomPlan can combine the data from multiple scan sessions to create a single captured result that includes multiple rooms. Having a single structure object (CapturedStructure) lets you create a USDZ file and inspect for particular shapes, dimensions, or items that RoomPlan observes in the structure. Scanned structures support rooms with varying floor heights and rooms that reside on different floors in the building.

[Image]

To scan a structure, your app guides a person to scan each room in the building, one by one. To start scanning a room, your app calls run(configuration:) on a room-capture session instance (RoomCaptureSession). If your app uses the framework-provided view (RoomCaptureView) to facilitate scanning, access the view’s captureSession property. If your app renders its own graphical interface, create and manage the room-capture session directly.

Your app guides a person through the scanning process by presenting informational text on how to move the device through the room. To indicate a room scan is complete, a person interacts with an interface that your app provides, such as tapping the Finish Room button in the toolbar. Your app stops the room-capture session by calling stop(pauseARSession:), passing an argument of false to keep the session open, as required to scan another room.

Each time your app calls stop(pauseARSession:), the system invokes your app’s captureSession(_:didEndWith:error:) method, in which you process the raw scan results and create a finished CapturedRoom. Store each CapturedRoom in an array until a person finishes scanning the whole structure.

A person indicates they’re done scanning all the rooms by interacting with an interface your app provides, such as by tapping the Finish Structure button in the toolbar. With the finalized room array, use the framework to merge the rooms into a single structure. Instantiate a (StructureBuilder) instance and pass the array to the capturedStructure(from:) method, which outputs a CapturedStructure object.

Create a room-capture session and begin a scan

To capture a structure that contains multiple rooms, scan each room one by one. Start by creating a single RoomCaptureSession instance for the first room:

var myCaptureSession = RoomCaptureSession()

If your app runs its own AR experience, you can pass your existing ARSession to RoomPlan using the init(arSession:) method. Apps that utilize the framework-provided UI (RoomCaptureView) for scanning can hand off the AR session similarly:

var roomCaptureView = RoomCaptureView(frame:myFrame, 
                                  arSession:existingARSession)

Begin the room-capture session by running it with a configuration:

myCaptureSession.run(configuration: captureSessionConfig)

Guide a person by displaying instructional text

As a person holds the device, the framework models the room by analyzing data from the device’s camera and LiDAR Scanner. To enable the device to acquire sufficient data to model the whole room, your app guides a person on how to move and where to point the device.

The framework-provided view automatically presents text instructions that guide a person through the process. If your app renders its own graphical interface, display your own text according to the instructions that RoomCaptureSessionDelegate provides in its captureSession(_:didProvide:) method. The RoomCaptureSession.Instruction object that the system passes in assists you with crafting the right information to display to someone.

Finalize a room scan and keep the AR session running

After a person completes scanning the room, stop the room-capture session. To keep subsequent rooms compatible for merging, make the AR session continue running by calling stop(pauseARSession:) with an argument of false.

myCaptureSession.stop(pauseARSession:false)

If your app uses framework-provided UI, stop the scan by referencing the view’s room-capture session:

roomCaptureView.captureSession.stop(pauseARSession:false)

When the room-capture session ends, the system calls your delegate’s captureSession(_:didEndWith:error:) method, which provides the scan results. Create a CapturedRoom object from the scan results and store them in an array for later:

func captureSession(_ session: RoomCaptureSession, 
              didEndWith data: CapturedRoomData,
                        error: Error?) {    
    let roomBuilder = RoomBuilder(options: [.beautifyObjects]
    /// ... 
    let capturedRoom = try? await roomBuilder.capturedRoom(from: data) else { return }
    if let room = capturedRoom {  
        self.roomsArray.append(capturedRoom)

Repeat this process until the person captures all the rooms. To start the next scan, call run(configuration:) again on the same room-capture session object. For the final room, you can call the regular stop() function at the end. However, apps that want to keep the AR experience going after scanning is done can call stop(pauseARSession:) with an argument of false to keep the AR session open.

For an example app that implements room scanning with the framework-provided UI, see Create a 3D model of an interior room by guiding the user through an AR experience.

During the structure scanning process, your app manages two kinds of sessions: RoomCaptureSession, and ARSession. To successfully merge rooms, each RoomCaptureSession needs to share the same common coordinate space as the ARSession. The coordinate space is common when:

The rooms are close to each other (for example, in the same building).
The captured session utilizes a continuous AR session or a relocalized AR session.

An ARSession is continuous when a person completes all scans without interruption, or the RoomCaptureSession contains the same AR session object that your app maintains after each room scan by calling stop(pauseARSession:) with an argument of false.

If the person sends the app to the background or the AR session experiences tracking problems while a person moves room to room, your app needs to relocalize the AR session before continuing to scan. Relocalizing instructs ARKit to restore a common coordinate space by inspecting a world-map (ARWorldMap) object that you provide.

Relocalize an AR session after an interruption

When a person sends the app to the background or restarts the app before finishing the structure, or if ARKit encounters a tracking error, your app has to restart the AR session. By default, new or restarted AR sessions define a coordinate space that’s incompatible with prior runs because the world origin and orientation are different. To scan more rooms that are compatible with prior scans, you need to restore a common coordinate space by using relocalization.

To enable relocalization, the following code implements an ARSessionObserver and responds to its sessionShouldAttemptRelocalization(_:) callback:

func sessionShouldAttemptRelocalization(_ session: ARSession) -> Bool { return true }

When you enable relocalization, ARKit automatically informs your app when it’s time to restore the prior coordinate space by setting the camera tracking state to ARCamera.TrackingState.limited(_:) with reason ARCamera.TrackingState.Reason.relocalizing. The following code responds to this tracking-status change:

func session(_ session: ARSession, cameraDidChangeTrackingState camera: ARCamera) {
    respondToStatusChange(for: session.currentFrame!, trackingState: camera.trackingState)
}

When the tracking-state reason is .relocalizing, ask the person scanning to return to their prior location in the physical environment where the interruption occurred. The following code conveys this request by displaying a label that tells people what to do:

private func respondToStatusChange(for frame: ARFrame, trackingState: ARCamera.TrackingState) {
// ...
case (.limited(.relocalizing), _) where isRelocalizingMap:
    message = "Move your device to the most recently scanned room."
    snapshotThumbnail.isHidden = false

As ARKit processes the camera feed, it identifies similarities in an ARWorldMap object, which acts as the session’s memory of where the scan left off. When ARKit recognizes the environment, it reorients itself, which completes the relocalization process. ARKit sets the camera tracking status to normal, and your app resumes scanning.

Your app manages the ARWorldMap object that acts as the session’s memory. For a complete example app that demonstrates relocalization using an ARWorldMap, see Saving and loading world data.

With the relocalized ARSession object, you can resume scanning the rest of the structure by creating a new room-capture session with the init(arSession:) intializer:

var roomCaptureSession = RoomCaptureSession(arSession:relocalizedARSession)

Merge the rooms to create the structure

After you collect all of the captured rooms in an array, hand it off to the capturedStructure(from:) method of StructureBuilder to create the CapturedStructure instance, as the following code demonstrates:

// Access the newly captured structure.
let capturedStructure = structureBuilder.capturedStructure(from: capturedRoomArray)

For a complete example app that merges a collection of prescanned rooms, see Merging multiple scans into a single structure.