Foveated Streaming
Foveated Streaming enables visionOS apps to display high-resolution, low-latency immersive content from streaming endpoints.
Overview
The Foveated Streaming framework provides a session-based API for establishing connections from Apple Vision Pro to local and cloud streaming endpoints. The endpoint host streams high quality content only where necessary based on information about the approximate region where the person is looking, ensuring performance. Applications and games can integrate with the NVIDIA CloudXR™ SDK to become streaming endpoints.
On Apple Vision Pro, Foveated Streaming allows you to display visionOS spatial content alongside streaming content. For example, a flight simulator app can render a cockpit using RealityKit, and stream a processor-intensive landscape from a remote computer to the device.
The core workflow involves creating a FoveatedStreamingSession, establishing a connection to the external streaming endpoint, and presenting the streamed content inside the app’s ImmersiveSpace. All of the features of ImmersiveSpace are available, including progressive and mixed immersion styles. A bidirectional message channel system allows visionOS apps to exchange custom data with streaming endpoints, allowing apps to configure the streamed content with a native SwiftUI interface.
Topics
Essentials
Streaming a CloudXR application to Apple Vision Pro with foveationEstablishing foveated streaming sessions with Apple Vision ProCreating a foveated streaming client on visionOSAnalyzing the performance of a foveated streaming sessionFoveatedStreamingSessionFoveatedStreamingSpaceContent