perso-ai/perso-interactive-ondevice-sdk-swift

Key Features

• Realistic AI Human Avatars: Natural expressions with AI Lip Sync
• Conversational AI Pipeline: Speech-to-Text, Large Language Model, and Text-to-Speech
• Customization: Integrate your own STT, LLM, and TTS models into the Conversational AI Pipeline
• On-Device Processing: Private, low-latency ML inference powered by Apple Neural Engine

<br/>

Prerequisites

• iOS 18.0+ / macOS 15.0+ / visionOS 2.0+
• Swift 6.0+
• Xcode 16.0+
• A14 Bionic or later (iPhone / iPad) · M1 or later (Mac)

<br/>

Installation

Swift Package Manager

1. Open your Xcode project

1. Navigate to File > Add Package Dependencies...

1. Enter the following URL of this repository :

`` https://github.com/perso-ai/perso-interactive-ondevice-sdk-swift ``

1. Choose the version range or specific version

1. Click Add Package to add PersoInteractiveOnDeviceSDK to your project

<br/>

Once this setup is complete, you can import PersoInteractiveOnDeviceSDK and start using the SDK in your Swift code.

<br/>

Quick Look

import PersoInteractiveOnDeviceSDK

// 1. Initialize SDK
PersoInteractive.apiKey = "YOUR_API_KEY"
PersoInteractive.computeUnits = .ane

// 2. Fetch and prepare model-style
let modelStyles = try await PersoInteractive.fetchAvailableModelStyles()
guard var modelStyle = modelStyles.first else {
    return
}

// Download model-style resources if needed
if case .unavailable = modelStyle.availability {
    let stream = PersoInteractive.loadModelStyle(with: modelStyle)

    for try await progress in stream {
        switch progress {
        case .progressing(let value):
            print("Downloading: \(Int(value.fractionCompleted * 100))%")
        case .finished(let updated):
            modelStyle = updated
        }
    }
}

// 3. Load on-device models
try await PersoInteractive.load()
try await PersoInteractive.warmup()

// 4. Configure audio session (iOS/visionOS only)
#if os(iOS) || os(visionOS)
try PersoInteractive.setAudioSession(
    category: .playAndRecord,
    options: [.defaultToSpeaker, .allowBluetooth]
)
#endif

// 5. Fetch features
let sttModels = try await PersoInteractive.fetchAvailableSTTModels()
let llmModels = try await PersoInteractive.fetchAvailableLLMModels()
let ttsModels = try await PersoInteractive.fetchAvailableTTSModels()
let prompts = try await PersoInteractive.fetchAvailablePrompts()

// 6. Create a session
let session = try await PersoInteractive.createSession(
    for: [
        .speechToText(type: sttModels.first!),
        .largeLanguageModel(llmType: llmModels.first!, promptID: prompts.first!.id),
        .textToSpeech(type: ttsModels.first!)
    ],
    modelStyle: modelStyle,
    statusHandler: { status in
        print("Session status: \(status)")
    }
)

// 7. Display AI Human
let videoView = PersoInteractiveVideoView(session: session)
videoView.videoContentMode = .aspectFit
try videoView.start()

// 8. Start conversation
let userMessage = UserMessage(content: "Hello!")
let stream = session.completeChat(message: userMessage)

for try await message in stream {
    if case .assistant(let assistantMessage, _) = message,
       let chunk = assistantMessage.chunks.last {
        try? videoView.push(text: chunk)
    }
}

<br/>

Documentation

Start with the Getting Started guide, and explore other articles in the documentation as needed. Check out the Sample Project to see the Perso Interactive SDK in action.

<br/>

License

Perso Interactive SDK for Swift is commercial software. Contact our sales team.

Package Metadata

Repository: perso-ai/perso-interactive-ondevice-sdk-swift

Default branch: main

README: README.md