LiveAssistance

Declaration

enum LiveAssistance

Overview

To provide interpreter, captioning, or other services that work with FaceTime calls, create a container app that uses the LiveAssistance APIs, as well as an app extension that conforms to the LiveAssistanceExtension protocol.

When a participant in a FaceTime calls wants to use live assistance services, they primarily interact with the app extension. If the extension needs further information to set up the service, it launches the app by returning LiveAssistanceRequest.Response.requiresUserInput(reason:) to the request from the framework. This response causes the system to launch the container app, which uses the APIs in this type.

Handling app launch

To prepare your container app, start by adding com.apple.conversation-accessibility.container-app-launch to the NSUserActivityTypes in your app’s information property list. This allows the framework to launch your app when the extension needs the person using the app to sign in or provide other information.

The framework launches your app with onContinueUserActivity(_:perform:) (SwiftUI) or application(_:continue:restorationHandler:) (UIKit). Inspect the NSUserActivity to verify that the activity type is containerAppLaunchActivityType. If so, retrieve the launch reason and the request identifier from the userInfo. The launch reason indicates whether the extension needs the app to authenticate the person requesting the service, or if they need to perform some other configuration or approval. Once your app resolves that, you use the request identifier and call returnToCall(requestID:) to resume or relaunch the extension and continue setting up the live assistance service.

The following example shows how a SwiftUI container app would handle being launched in response to a request from its app extension. It retrieves the request identifier and launch reason from the NSUserActivity, and then calls an asynchronous myHandleAccessibilityLaunch(requestID:reason:) method to handle any needed login or configuration and then resume the request.

var body: some Scene {
    WindowGroup {
        ContentView()
            .onContinueUserActivity(LiveAssistance.containerAppLaunchActivityType) { activity in
                guard
                    let requestID = activity.userInfo?[LiveAssistance.containerAppLaunchRequestIDKey] as? /// UUID,
                    let rawReason = activity.userInfo?[LiveAssistance.containerAppLaunchReasonKey] as? /// String,
                    let reason = LiveAssistanceLaunchReason(rawValue: rawReason)
                else { return }
                Task { await myHandleAccessibilityLaunch(requestID: requestID, reason: reason) }
            }
    }
}

Topics

Identifying the app extension

• extensionPointName

Responding to a container app launch

• containerAppLaunchActivityType
• containerAppLaunchRequestIDKey
• containerAppLaunchReasonKey
• LiveAssistanceLaunchReason

Returning to a call

• returnToCall(requestID:)

See Also

Conversation assistance

• LiveAssistanceExtension