Contents

Integrating your photo app with Apple Intelligence

Adopt photo schemas so people can edit and manage photos with Siri.

Overview

This sample app uses the App Intents framework to make its photo functionality discoverable by Apple Intelligence and Siri. After choosing photos from their camera roll, people use Siri to perform actions like rotating a photo, adding it to their Favorites, and more.

Make content available in Spotlight

To make its content discoverable by Apple Intelligence and let people find photos using semantic search in Spotlight, the sample app describes its assets as app entities that conform to the IndexedEntity protocol:

@AppEntity(schema: .photos.asset)
struct AssetEntity: IndexedEntity {

    // MARK: Static

    static let defaultQuery = AssetQuery()

    // MARK: Properties

    let id: String
    let asset: Asset

    @Property(title: "Title")
    var title: String?

    var creationDate: Date?
    var location: PlaceDescriptor?
    var assetType: AssetType?
    var isFavorite: Bool
    var isHidden: Bool
    var hasSuggestedEdits: Bool
    var aperture: Double?
    var exposure: Double?
    var saturation: Double?
    var warmth: Double?
    var filter: AssetFilter?
    var isPortraitModeEnabled: Bool?

    var displayRepresentation: DisplayRepresentation {
        DisplayRepresentation(
            title: title.map { "\($0)" } ?? "Unknown",
            subtitle: assetType?.localizedStringResource ?? "Photo"
        )
    }
}

By conforming app entities to IndexedEntity and donating them to a Spotlight index, the sample app enables people to search for its content in Spotlight and enable semantic search results.

For more information about making content available to Spotlight, see Making app entities available in Spotlight.

Make app entities shareable

Describing app data as app entities and indexing them in Spotlight is the first step toward making your app discoverable by Apple Intelligence and bringing your app to Siri. To make sure the system can pass the sample app’s content to other apps, the app adopts the Transferable protocol for its app entities:

extension AssetEntity: Transferable {
    // ...

    static var transferRepresentation: some TransferRepresentation {
        DataRepresentation(exportedContentType: .png) { entity in
            try await entity.asset.pngData()
        }
    }
}

Provide onscreen context

When a person makes a request that references onscreen content, for example, “Rotate this photo”, Apple Intelligence needs to know which photo is visible onscreen. When you provide this information as context, Siri can respond to a request and perform the action. In addition to conforming to the Transferable protocol, the AssetEntity also conforms to the asset schema. Schema conformance is a prerequisite for providing cues to the system about what’s visible onscreen. To bridge visible content to an app entity that conforms to a schema, you annotate a view or user activity object with an app entity. The MediaView shows one photo at a time, so the app annotates a NSUserActivity with the AssetEntity that represents the currently visible photo:

var body: some View {
    // ...
    MediaView(
        image: image,
        duration: asset.duration,
        isFavorite: asset.isFavorite,
        proxy: proxy
    )
    .userActivity(
        "com.example.apple-samplecode.PhotosDomainExample.ViewingPhoto",
        element: asset.entity
    ) { asset, activity in
        activity.title = "Viewing a photo"
        activity.appEntityIdentifier = EntityIdentifier(for: asset)
    }
    // ...

For more information about making onscreen content discoverable by Apple Intelligence, see Providing contextual cues to Apple Intelligence and Siri.

Create app intents for photo actions

Transferable app entities, Spotlight donations, and user-activity annotations give the system context about your app’s content. To let people perform actions on the content with Siri, create app intents that conform to intent schemas in the .photos domain that match photo actions. For example, the sample app implements an intent that conforms to the createAlbum schema to let people create a photo album in the app with Siri:

@AppIntent(schema: .photos.createAlbum)
struct CreateAlbumIntent: AppIntent {
    var name: String

    @Dependency
    var library: MediaLibrary

    @MainActor
    func perform() async throws -> some ReturnsValue<AlbumEntity> {
        let album = try await library.createAlbum(with: name)
        return .result(value: album.entity)
    }
}

For more information about making photo actions discoverable by Apple Intelligence and Siri, see Making actions and content discoverable by Apple Intelligence and Photos.

See Also

Sample code