Privacy and Security Group Lab

Transcript

Engineers from the privacy and security teams introduce themselves and set up a session covering agentic AI security risks, Private Cloud Compute, App Privacy Nutrition Labels, data protection and CloudKit encryption, passkeys, and secure-by-design development.

Agentic technologies introduce a new category of risk, notably indirect prompt injection — where malicious instructions are hidden in content the model processes rather than typed by the user. Apple designs these features secure-by-design, constraining what the model is allowed to do and treating untrusted content as data, not instructions, to limit what an injected prompt can achieve.

Private Cloud Compute's guarantees — no retention, no access, verifiable — apply only to Apple's PCC path. When you route data to a third-party provider through the protocol, those guarantees don't carry over; that provider's own data handling governs it. Disclose third-party processing to users and reflect it in your privacy practices, since you can't promise PCC-level protections for data you send elsewhere.

This maps onto App Privacy Nutrition Labels, which let users compare apps' data practices at a glance. Data sent to a third party purely for processing on your behalf — not retained or used for their own purposes — is treated differently from data you collect. Understand each service's role and declare your practices accordingly so the nutrition label accurately reflects what happens to user data.

Like Siri and Safari's AI features, Xcode's agentic features are built secure-by-design with the prompt-injection risks in mind. Xcode adds mitigations such as allow-listing the common tools the agent may run and permission prompting, so the agent can't take sensitive actions freely. Review what tools you grant, keep the agent scoped to what it needs, and don't blindly trust generated actions.

Read the in-depth PCC Security Guide on apple.com, which documents the architecture and guarantees at several levels of detail. PCC is designed so that user data is used only to fulfill the request and is not accessible to Apple or anyone else, with the guarantees made verifiable — the security guide is the authoritative place to examine the design and check the claims.

SwiftData with CloudKit is a strong foundation because data syncs through the user's own iCloud with encryption. A newly practical audit tool is AI itself: point a model at your code and ask it to enumerate your privacy and security guarantees and where they might break. Combine that with reviewing your data protection classes and encryption settings to confirm the guarantees hold.

For privacy, lean on CloudKit for syncing data within the user's own account, Keychain for secrets, and design so users own their data. For security, use CryptoKit for cryptography and encourage passkeys over passwords. These frameworks are fundamental building blocks that give you strong protections without rolling your own.

Use data protection classes: when data is local (or in Keychain), create a key to encrypt it and set an appropriate data protection class so it's inaccessible when the device is locked. For sync, CloudKit offers encryption so sensitive fields stay protected in transit and at rest, while you keep the searchable structure you need on device.

The existing code-completion features are entirely on device and work offline, so they're fine air-gapped. Features that integrate external agents like Claude Code or OpenAI's Codex inevitably need internet access, so those won't work offline. It varies by feature — on-device completion works walled off, cloud-backed agents do not.

Siri does as much as possible on device, and when it needs larger models it calls Private Cloud Compute, which extends the on-device privacy guarantees to the server. That combination — on-device processing plus PCC for heavier work — is what lets Siri use your personal context powerfully while keeping that information private, alongside the platform's sandboxing and permission systems.

Apple Intelligence and Siri features are manageable through MDM, so organizations can configure and restrict these capabilities via their device-management policies as the features roll out. Check the MDM configuration options for the specific controls available to your deployment.

ITP is a long-evolving technology Apple has invested in for years and continues to advance. There were no specific new ITP features to highlight this year, but it remains a priority area of ongoing investment; the WebKit resources are the place to learn more about how it protects users from cross-site tracking.

Make trust verifiable rather than asserted. App Privacy Nutrition Labels let users see, before downloading, exactly what data an app collects and whether it's linked to them — an at-a-glance, standardized signal. Back claims with observable facts (what you collect, how it's used, what stays on device) instead of marketing language.

Start with the Apple Platform Security Guide, which goes deeper than standard documentation on how the OSes are secured. Build from there into the security frameworks (CryptoKit, Keychain, data protection). Apple also hires for security and privacy roles, so this foundation is directly relevant to the field.

There's a new Signal API through which a relying party's app or website can inform the system that credentials have changed and need updating — addressing stale, revoked, or invalid passkeys. This is part of the broader passkey standard Apple participates in, so the lifecycle cleanup is being handled in a standardized way.

Start with Hardened Runtime on macOS — strongly recommended for any privacy- or security-sensitive app. It prevents other processes from reading your app's memory and blocks the common attacks (spinning up a thread to read process memory, or attaching a debugger). Before reaching for encryption, ask whether destruction is the better answer: if a secret like a token only needs to live briefly, use it and then destroy it rather than encrypting it in RAM — long-lived in-memory encryption is really DRM territory, not general security. For keys and other sensitive values, use CryptoKit, which zeroizes the memory backing those keys (rolling your own can't guarantee the Swift runtime will). And you can bind a key to the device's Secure Enclave so that even if your app, process, or the device is compromised, the key can't be exported off the device — with an attestation interface to prove it's Enclave-held.

The guiding principle is great features and great privacy — both, not a trade-off of one for the other. Achieving that often takes extra engineering: collect the minimum you need, prefer privacy-preserving techniques (aggregation, on-device processing, de-identification), and design telemetry so it informs development without exposing individual users.