Foundation Models
Perform tasks with models that specialize in language understanding, structured output, and tool calling.
Mentioned in
Overview
The Foundation Models framework provides access to any large language model, like the on-device and Private Cloud Compute models designed for Apple Intelligence. These models help you perform intelligent tasks specific to your use case.
[Image]
On-device models excel at a diverse range of text generation tasks, like summarization, entity extraction, text and image understanding, refinement, dialog for games, generating creative content, and more. When you need more reasoning capabilities and context size, use Private Cloud Compute or any server model provider.
The dynamic profile API provides the flexibility to select the best model configuration for your task, and lets you build many useful abstractions, such as agents or skills.
Generate entire Swift data structures with guided generation. With the @Generable macro, you can define custom data structures and the framework provides strong guarantees that the model generates instances of your type.
Use Tool to create custom tools that the model can call to assist with handling your request. For example, the model can call a tool that searches a local or online database for information, or calls a service in your app.
To use Apple Foundation Models, people need to turn on Apple Intelligence on their device. For a list of supported devices, see Apple Intelligence.
What’s new
- Adding server-side intelligence with Private Cloud Compute
- Composing dynamic sessions with instructions and profiles
- Analyzing images with multimodal prompting
- Build agentic app experiences with the Foundation Models framework
- Bring an LLM provider to the Foundation Models framework
- Debug and profile agentic app experiences with Instruments
Topics
Essentials
Foundation Models updatesGenerating content and performing tasks with Foundation ModelsAdding intelligent app features with generative models
Sessions and prompts
Prompting an on-device foundation modelManaging the context windowUpdating prompts for new model versionsLanguageModelSessionInstructionsPromptTranscriptTranscriptErrorHandlingPolicyGenerationOptionsContextOptions
Prompt attachments
Analyzing images with multimodal promptingAttachmentAttachmentContentImageAttachmentContentImageReference
Dynamic profiles
Composing dynamic sessions with instructions and profilesOrigami: Crafting a dynamic tutorial for Apple IntelligenceDynamicInstructionsLanguageModelSession.DynamicProfileLanguageModelSession.DynamicProfileModifierLanguageModelSession.Profile
Structured output
Generating Swift data structures with guided generationGenerableGenerationSchemaDynamicGenerationSchemaGeneratedContentConvertibleToGeneratedContentConvertibleFromGeneratedContent
Tools
Expanding generation with tool callingGenerate dynamic game content with guided generation and toolsTool
System language model
Supporting languages and locales with Foundation ModelsCategorizing and organizing data with content tagsSystemLanguageModelLanguageModelError
Private Cloud Compute
Adding server-side intelligence with Private Cloud Computecom.apple.developer.private-cloud-computePrivateCloudComputeLanguageModel
Custom language model provider
Optimizing key-value caching in language model sessionsLanguageModelLanguageModelCapabilitiesLanguageModelExecutorLanguageModelExecutorGenerationChannelLanguageModelExecutorGenerationRequest