Stateless Computation

Overview

[Image]

Serving a request requires unencrypted access to the user’s personal data. To protect that data, Private Cloud Compute must use it only to fulfill the user’s request, and must not retain access to it once it completes the request.

This requirement constrains what PCC does with user data throughout the life cycle of a request. Each stage of PCC’s request-handling and inference pipeline bounds how long user data persists, leaving no accessible state behind in storage or in memory once processing completes.

Stateless inference

PCC bounds the lifetime of user data across every stage of processing and at three layers: the inference engine, the address spaces that process a request, and the storage beneath them. No single failure leaves user data accessible after a request completes.

We designed PCC’s inference engines to process each request independently. Conventional inference servers often pool computed state across requests, for example by building a prefix cache from the traffic they observe. PCC doesn’t derive shared state from traffic this way, and the requirement runs in both directions: no request may leave state that another request can read or that changes how another request is processed. Pooled state breaches both, letting one user’s data outlive the request that produced it and letting one user’s traffic shape another user’s results. Instead, PCC relies on a combination of a pre-provisioned prefix cache, whose contents come only from measured material we supply rather than from traffic, and per-user encrypted caches reused across turns.

This stateless design is accompanied by bounds on the lifetime of the address spaces that process user data. The request-handling infrastructure and inference engine apply one of two techniques wherever user data is present:

Per-request address spaces

An address space scoped to a single request is Per-request isolation.

Periodic recycling

A long-lived address space is periodically recycled, bounding how long user data can persist in freed or leaked memory.

The inference engines on The Inference Engine and Google Cloud Stateless Inference each apply both techniques.

Beneath the applications, neither platform retains user data that survives a reboot:

Apple silicon nodes

Ephemeral Data Mode is a backstop against long-term persistence. The Secure Enclave randomizes the data volume’s encryption keys on every boot and retains no copy of them, so rebooting the node’s Secure Enclave Processor cryptographically erases the data volume. Any data that PCC application software might inadvertently write to the volume is unrecoverable after a reboot.

Google Cloud nodes

A node retains no writes across a reboot. The one Host and guest OS hardening is encrypted under a key the node generates at boot and holds only in memory, and everything else writable is either memory-backed or bind-mounted onto that file system, so a restart erases it.

A few large assets on an Apple silicon node are Persistent asset caches from that erasure so a node need not refetch them on every boot, such as model assets and the filters the Private Filter Service uses. Two rules keep the exemption from becoming a way to retain user data: only components that never handle user data may write to that storage, and its contents are revalidated against expected measurements on each boot.

These mechanisms work alongside PCC’s other restrictions on observability and debugging, which help ensure that no diagnostic path inadvertently persists state. For example, a node keeps no log that survives a reboot, and Operating the service without privileged access bounds the log lines PCC’s software can emit.

Tool calls

Some requests need information or services a PCC node doesn’t hold, such as the world knowledge a tool call retrieves during an agentic loop. When a model requests a tool PCC doesn’t implement, PCC returns that request to the user’s device, which runs the tool locally and supplies the result on a later turn. Where PCC implements a tool itself, it holds that tool to the same standard of privacy the user gets on device, using the most privacy-preserving mechanism available:

Within the trust boundary

Many tools run entirely within the trust boundary, exposing nothing beyond what the request already does; the workflow runs such tools directly or issues Delegated tool calls to other PCC nodes. Query-derived data stays inside the trust boundary, the strongest privacy posture available for a tool.

Private information retrieval

PCC issues some queries against an external database using Private Information Retrieval, which uses homomorphic encryption to return the requested record without the server learning which one the query selected, so the lookup reveals nothing about the content of the user’s request.

External tool call

World knowledge and similar capabilities require information or services that can’t reside inside the trust boundary. PCC reaches these through an External tool calls to a dedicated tool gateway. Such calls necessarily send query-derived data outside the trust boundary, so we designed them to offer the same privacy as on-device tool access. The Callout transparency requirement covers how such a call is kept non-identifiable, recorded, and made inspectable.

Multi-turn interactions

A multi-turn interaction can reuse computed state between turns rather than recomputing it from scratch each time. Reusing this state must preserve PCC’s statelessness.

PCC preserves this invariant in three ways:

Per-request and per-turn keys

The keys that protect cached state come from key material that the user’s device supplies with each request.

Statelessness across turns

A node encrypts cached state that it keeps for a later turn before storing it, and PCC retains no cache-decryption keys between turns. Stored state is not decryptable by any process on the node until the device resupplies the key.

Key custody returns to the device

Continuing an interaction requires the device to resupply the keys before the node can reuse any cached state. When the interaction ends, only the user’s device holds the keys to anything the node kept.

These three properties help ensure that reusing cached state across turns preserves the stateless-computation guarantee: the ability to decrypt a user’s data returns to the user’s device between turns. For the mechanism that enforces this, see KV cache reuse on an Apple silicon node and Cache privacy on a Google Cloud node.