Contents

LanguageModelSession

An object that represents a session that interacts with a language model.

Declaration

final class LanguageModelSession

Mentioned in

Overview

A session is a single context that you use to generate content with, and maintains state between requests. You can reuse the existing instance or create a new one each time you call the model. When creating a session, provide instructions that tells the model what its role is and provide guidance on how to respond.

let instructions = """
    You are a motivational workout coach that provides quotes to inspire \
    and motivate athletes.
    """
let session = LanguageModelSession(instructions: instructions)
let prompt = "Generate a motivational quote for my next workout."
let response = try await session.respond(to: prompt)

The framework records each call to the model in a Transcript that includes all prompts and responses. If your session exceeds the available context size, it throws LanguageModelSession.GenerationError.exceededContextWindowSize(_:).

When you perform a task that needs a larger context size, split the task into smaller steps and run each of them in a new LanguageModelSession. For example, to generate a summary for a long article on device:

  1. Separate the article into smaller sections.

  2. Summarize each section with a new session instance.

  3. Combine the sections.

  4. Repeat the steps until you get a summary with the size you want, and consider adding the summary to the prompt so it conveys the contextual information.

Use Instruments to analyze token consumption while your app is running and to look for opportunities to improve performance, like with prewarm(promptPrefix:). Because some generation tasks can be resource intensive, consider profiling your app with other instruments — like CPU Profiler and Power Profiler — to identify where your app might be using more system resources than expected. For more information on Instruments, see Analyzing the runtime performance of your Foundation Models app.

For more information on managing the context window size, see TN3193: Managing the on-device foundation model’s context window.

Topics

Creating a session

Creating a session from a transcript

Preloading the model

Inspecting session properties

Generating a request

Streaming a response

Generating feedback

Getting the error types

See Also

Prompting