Google Cloud Node Attestation

Overview

A user’s device — or a proxy node acting on its behalf — sends a request to a PCC node only after Node validation: cryptographic proof of the node’s identity, bound to the boot chain’s measurements, checked against the append-only transparency log. No single Multiple hardware roots of trust measures the whole system, so the node assembles its proof from three roots. The Titan Trusted Platform Module (TPM), the Intel TDX CPU, and the NVIDIA GPUs each attest to the layer they measure, and the node binds those separate attestations together into one attestation bundle.

Producing attestation evidence

Three sources emit attestation evidence, each rooted in a different hardware root of trust that measures a distinct layer of the node:

CPU evidence

An Intel TDX quote over the confidential VM. The quote reflects the VM’s initial memory in its MRTD field and carries the Runtime Measurement Registers (RTMRs) that record firmware, boot loader, and container-launch measurements, analogous to TPM Platform Configuration Registers (PCRs); see Appendix: Google Cloud Measurement Registers for the register-level mapping of both the host PCRs and the guest RTMRs.

Host evidence

A Titan TPM quote over the host boot chain — baseboard management controller (BMC) firmware, BIOS, and the host kernel — extended into TPM PCRs, together with an endorsement that includes Titan’s own firmware measurement. A Host Services Daemon that runs on the host, isolated from the workload, produces this quote on request.

GPU evidence

NVIDIA attestation reports for the node’s GPUs, captured by the drivers inside the confidential VM.

The workload obtains Titan TPM and TDX evidence only through the Workload Services Daemon inside the confidential VM, which services requests from the workload container over Unix domain sockets and orchestrates the other daemons. It exposes two logically separate interfaces, an Attestation Service and a key-management interface. PCC exposes each socket only to the containers that require it; the per-request cloud app that parses untrusted request contents does not receive them. The Attestation Service gathers the CPU, host, and GPU evidence and, on request, returns an endorsement that binds a specific key to that evidence. For more on the full REST surface for these services, see Appendix: Google Cloud Workload Services Daemon.

[Image]

Titan TPM evidence

Google’s Titan hardware security chip is attached to the BMC; this chip exposes a standard TPM interface. Attestation evidence from the Titan TPM comes in the form of a standard TPM2 Quote, specifically a TPMS_ATTEST structure. These quote objects are signed by an Endorsement Key (EK).

Titan’s EK is a restricted ECDSA P-256 signing key in the firmware endorsement hierarchy, TPM_RH_FW_ENDORSEMENT, and the chip quotes under that key directly, so the endorsed key and the signing key are one. That differs from the conventional TPM arrangement, where the endorsement key is a restricted decryption key that activates a credential for a separate attestation key, and the attestation key signs the quote. Titan’s EK is also firmware-limited, a property TCG TPM 2.0 revision 1.83 introduces: the chip derives it under a firmware-versioned seed, and its certificate records that seed’s firmware major version. The firmware-versioned Device Identifier Composition Engine (DICE) alias key can therefore certify it. A verifier recomputes the TPM qualified name of the EK public key under that hierarchy and Google’s firmware-limited template, requires the EK certificate to carry the same value, and requires that certificate’s firmware major version to equal the alias key’s, so an endorsement minted under any other template or firmware version fails.

The EK’s certificate is signed by the chip’s DICE alias key, that key’s certificate by the chip’s DeviceID key, and the DeviceID certificate by one of the Titan scribe roots that Google publishes in its platform-attestation repository. A verifier pins those roots rather than validating them: CloudAttestation embeds the published set and checks each endorsement against two values, the root the DeviceID certificate names in its header and the scribe key carried on the accompanying DeviceID-scribe certificate. Verifying that a TPM quote was correctly signed by an endorsed EK yields a trusted set of Platform Configuration Register (PCR) values and the attached extraData field.

A verifier uses the PCR values to establish trust in an event log, where each individual event in the log can be “extended” to compute an expected PCR value, which in turn can be compared to those obtained from the quote. This verification confirms that the TPM device extended the same sequence of events into its PCRs that was presented to the attestation verifier. Measurements over the host platform firmware and operating system are stored in this event log. The Titan TPM keeps the node’s warm-reset counters in a separate NV index and signs that index independently of the quote.

Intel TDX evidence

Intel TDX provides attestation evidence as a TDX quote. PCC currently uses the Version 4 Quote Format. A TDX quote contains measurements and security version numbers describing the firmware and microcode state of the CPU. These quote objects also contain ratchet-capable RTMRs that function nearly identically in form to TPM PCRs. These quoted registers are also used to verify an event log containing measurements of the virtual machine, as well as container launch events and measurements recorded from userspace.

The CPU’s quoting enclave signs each Intel TDX quote with an attestation key, and a Provisioning Certification Key (PCK) signs the enclave report that commits to that key. An X.509 certificate certifies the PCK, and that certificate validates to an Intel certificate authority root. The PCK certificate contains a stable device-unique identifier, the Intel TDX PIID.

NVIDIA GPU evidence

The NVIDIA driver running inside the confidential VM captures GPU measurements at launch. For each GPU, it issues a Security Protocol and Data Model (SPDM) GET_MEASUREMENTS request that carries a fresh anti-replay value and records the signed MEASUREMENTS response. The GPU also returns its certificate chain, and the driver’s SPDM requester verifies the response’s signature with the public key in that chain. The VM then reflects the full transcript and the certificate chain into its own attestation, as part of the RTMR event log. A verifier anchors that chain to NVIDIA’s root and compares the measurements against NVIDIA’s reference values. Because the evidence is carried inside the VM’s own quote, GPU trust is bound to the same CPU attestation as the rest of the workload.

Blackwell GPUs implement device attestation over the DMTF SPDM 1.1 protocol, and a GPU’s evidence is the transcript of the SPDM GET_MEASUREMENTS request and MEASUREMENTS response. The response includes DMTF-format measurement blocks covering both the GPU’s VBIOS and the NVIDIA driver, distinguished by fixed block indices, alongside tagged type-length-value opaque-data fields that report firmware claims such as the driver version, VBIOS version, board identifier, and architecture. The transcript ends with a fixed-length signature — 96 bytes for the known architectures — made by the GPU’s attestation identity key, which a verifier checks against the public key in the accompanying certificate chain, and that chain validates up to NVIDIA’s certificate authority. The leaf certificate includes a stable identity as a SAN otherName of the form NVIDIA:<architecture>:<serial> under OID 1.3.6.1.4.1.412.274.1.

A verifier that doesn’t use NVIDIA’s attestation SDK parses these fields itself and checks each measurement against a trusted reference set obtained from NVIDIA.

Binding the evidence together

The CPU, host, and GPU evidence comes from separate roots of trust, so a client cannot trust any piece in isolation: the client must confirm that all three describe the same machine at the same time. Without that binding, an attacker could assemble a valid-looking bundle from evidence emitted by different machines, or replay stale evidence. The node’s evidence is therefore bound on two axes:

Spatial binding

Confirms the roots of trust reside on the same physical machine. The CPU’s TDX evidence is bound to the platform through the Intel TDX PIID carried in the TDX quote’s certification chain; because the Key Protection Service runs in a separate confidential VM on the same machine, its quote carries the same PIID. The host extends the same PIID into a TPM PCR early in boot, so the TPM quote carries it as well. A verifier extracts the PIID from each attestation and confirms all values match; a mismatch means the attestations were not emitted from roots of trust on one machine, and binding fails.

Temporal binding

Confirms the evidence is fresh and not replayed. The workload generates a challenge over the metadata it attaches to the bundle, and each of the three quotes incorporates that challenge: both TDX quotes fold it into their report data, and the Titan quote includes it verbatim in its qualifying data. The challenge covers the certificate-transparency log head the node witnessed at attestation time, so a verifier can bound the bundle’s age against a timestamp the log assigned rather than one the node chose. Validation accepts a quote only when its report data reproduces the challenge over the bundle’s own metadata, so an attacker cannot substitute or strip that metadata.

Google Cloud Roots of Trust describes the relationship between the PIID and the machine’s ledgered identity, and Appendix: Google Cloud Attestation Bundle Contents gives the wire format of the bound attestation bundle.

Establishing hardware roots

Three independent signature chains must be validated. Until they are, every field behind them is attacker-controlled.

For each of the two Intel TDX-enabled VMs, a verifier validates the PCK certificate chain against Intel’s SGX root and verifies the quoting enclave’s report under the PCK public key. That report’s data commits to the attestation key, which the verifier then uses to check the quote’s header and body.

For the host, the verifier validates the Titan EK certificate against its DICE chain and a pinned Google root, then checks the TPM quote’s signature under the endorsed EK public key.

For each GPU, the verifier validates the attestation certificate chain against an NVIDIA device-identity root, then checks the EC-P384 signature over the SPDM measurement response under the leaf certificate’s public key.

A failure at any of these steps leaves the evidence behind it unverified, and policy rejects the bundle.

Binding each raw field to its quote

A verifier must validate the label, challenge, and extra_data fields, which ship out of band in protobuf, against the TDX quote. The bundle carries a The Google Cloud variant in its attested_metadata field, and the node sets challenge to SHA256(attested_metadata). Recomputing SHA512(label ‖ SHA512(challenge ‖ SHA512(extra_data))) from the raw values and requiring it to equal the quote body’s REPORTDATA binds those values to the signed quote. The verifier applies this check to both the workload and key-protection VMs.

The host TPM attestation contains raw bytes of challenge inside its TPMS_ATTEST.extraData field.

Binding event logs to quoted registers

Event logs are unsigned. Each one becomes trustworthy only by replaying it and reproducing a register value that can be compared to an expected value from a signed attestation quote.

Verifying the TDX quote of each VM yields a trusted event log for each VM, and validating the TPM quote from the Titan chip yields a trusted event log for the host platform itself.

[Image]

Validating platform configuration from event logs

With a set of trusted event logs anchored, the verifier can check each component’s security state.

Each VM’s TD configuration must have: debug off, migratable off, key-locker off, perf-mon off, PKS off, SEPT #VE disabled, XFAM PT and LBR off, zero-vector SEAM attributes, and a quoting enclave that is initialized, 64-bit, and not in debug mode.

The verifier also checks each VM’s boot chain: UEFI debug mode must be off, DMA protection must not be disabled, and the boot chain must match a layout the verifier recognizes. A node’s guests load a single measured Unified Kernel Image (UKI) as their sole EFI application.

The container launch log must also log the container image digest and, on a production node, must not contain a populated runtime-configuration override. The container launch log also contains the list of GPU attestation reports; the verifier confirms that the count matches the expected value, that each attestation validates successfully, and that each GPU shares the same set of measurements.

The host event log also describes the host platform’s boot chain, which must also record only a single UKI as the sole EFI application booted.

Anchoring hardware identifiers

The evidence must describe a single machine. The Intel TDX PIID recorded in the host launch event log must equal the PIID carried in the PCK certificate of each VM’s quote, so the Titan-attested host and the two TDX-attested VMs are provably the same physical machine.

That machine must also be one Apple expects. After verifying all attestation evidence and event logs, the verifier extracts the stable platform identifiers for the Titan chip, NVIDIA GPUs, and Intel CPU and compares them against those in the Apple-issued platform certificate. This helps ensure that the pairing of hardware in the Google-operated data center is consistent with Apple’s expectations for a particular physical platform.

Detecting a warm-reset attack

The platform is a shared-root-of-trust design: the Titan TPM sits behind the BMC, which proxies commands to it, manages the BIOS flash, and holds reset control over the node. The BMC and BIOS deposit CPU measurements into TPM dynamic-root-of-trust PCRs, which are cleared on a CPU warm reset. Without a further check, a BMC compromised at runtime could exploit that combination: it could stage arbitrary BIOS code in flash, warm-reset the CPU into it, and then, because the reset clears the PCRs, extend legitimate-looking measurements that mask the code that ran. The measured boot chain cannot distinguish that from a clean boot, because the PCRs replay identically either way.

Titan therefore counts warm resets itself. It autonomously writes to a non-volatile (NV) TPM index whenever it senses a CPU warm reset, and only a full power cycle clears that index, so the BMC cannot reset the counter along with the PCRs. Alongside the TPM quote, which covers PCRs, the node produces an auxiliary attestation over that NV data, signed with the same EK. The verifier expects the count to be exactly 2, because the boot flow performs exactly two warm resets before the workload launches; any other value is out of spec. Titan’s cold-boot measurements of the BMC firmware are likewise never cleared from the TPM.

The auxiliary attestation also carries the challenge in its own TPMS_ATTEST.extraData, which a verifier can compare against the challenge in the quote to bind the two to the same request. If a node fails attestation because of a warm reset, the operator relinquishes the instance and requests a new one; a power cycle precedes instance scheduling, so the replacement passes the check.

Verifying the evidence

The client — or the proxy node acting on its behalf — verifies a node’s evidence before routing a request to it. Verification chains each attestation to a trusted hardware root, replays the measurement logs, and checks that the measurements match releases recorded in the transparency log. The steps are:

1. Verify the signatures on the TDX and TPM quotes and chain them to their trusted roots.

2. Replay the firmware boot event log and the launcher event log, extracting the measurements they record.

3. Project those measurements into a software release and a firmware release, and verify an inclusion proof in the transparency log for each. Apple builds each firmware release from the vendors’ reference data: Google’s software bill of materials (SBOM) for the host’s Titan, BMC, and BIOS firmware, and NVIDIA’s reference integrity manifests (RIMs) for the GPU. A device therefore accepts a firmware state only once Apple has recorded it in the log. Google Cloud Software Transparency covers what is published and how a researcher reproduces these measurements.

4. Verify the NVIDIA GPU evidence: check each GPU’s signed report, and confirm that every GPU reports the same driver and VBIOS measurements.

5. Confirm temporal binding by recomputing the challenge from the bundle’s metadata and requiring each quote to carry it, and spatial binding by confirming the PIID matches across the CPU, host, and key-protection attestations.

6. Confirm the warm-reset count matches the expected value.

Because firmware updates can change a node’s measurements, the software-bill-of-materials interface can report the firmware versions expected after a pending maintenance update as well as the current ones; a node still running older firmware appears as a measurement mismatch until it updates. Appendix: Google Cloud Workload Services Daemon documents the endpoint reference.

Protecting request keys

Attestation gates the keys that decrypt a request: a client sends user data to a node only after verifying its attestation. Each request is encrypted to an attested key-encapsulation mechanism (KEM) key pair that the node holds. PCC protects these keys with an attested, isolated key-management service. The design goal is that even an attacker who compromises the Workload VM’s kernel cannot exfiltrate the private key to which a request is encrypted.

To withstand a compromise of the Workload VM’s kernel, the KEM private key lives in a dedicated confidential VM, the Key Protection Service VM. That VM is isolated from the Workload VM, and it communicates with the Workload Services Daemon over a gRPC bridge. Inside it, a Rust-based module owns the private key material and backs it with Linux’s memfd_secret, which allocates memory the kernel removes from its direct map, so the resident copy of a long-term key stays out of the VM’s general memory and out of reach of its guest kernel. The module copies a key into ordinary memory only for the duration of one decapsulation, and zeroes that copy afterward.

The component exposed to a request’s untrusted input holds that request’s shared secret but not the long-lived KEM private key.

Attested key release

The Key Protection Service is designed to release a decapsulated secret only to a Workload VM that booted to measurements a verifier recognizes, and to do so without exposing the unencrypted KEM private key or the decapsulated secret to any other VM on the host. To make this possible, the Workload Services Daemon generates its own binding Hybrid Public Key Encryption (HPKE) key pair, and the Key Protection Service encrypts decapsulated secrets to that binding public key. Key generation and endorsement proceed as a periodic attestation exchange:

1. The Workload Services Daemon generates a fresh random binding HPKE key pair for each KEM key it requests and passes the binding public key to the Key Protection Service with the request.

2. The Key Protection Service generates a random KEM key pair, associates it with the given binding public key, and sets a lifespan after which it autonomously deletes the KEM key from memory. The key pair supports a single operation: decapsulate a ciphertext and encrypt the resulting secret to the binding public key supplied when the key pair was generated. There is no operation that returns a decapsulated secret in the clear.

3. The Key Protection Service emits an attestation over its measurements, its KEM public key, and the associated binding public key.

4. The Workload Services Daemon emits an attestation over its measurements and the binding public key.

5. The verifier checks both attestations and confirms the binding public keys in each are equal.

The pair of attestations is what makes the KEM public key safe to encrypt a request to. The Key Protection Service’s attestation shows which binding key its KEM key releases secrets to; the Workload Services Daemon’s attestation shows that the holder of that binding key is a Workload VM whose measurements the verifier recognizes. Neither attestation on its own establishes that the KEM public key is safe to encrypt a request to. Having checked both, the verifier caches the KEM public key and rotates it on a fixed schedule with a fresh attestation.

[Image]

When a request arrives, the proxy node encapsulates against the cached KEM public key and encrypts the request to the resulting shared secret. Releasing that secret to the workload proceeds through the key-management interface of the Workload Services Daemon:

1. The proxy node presents the KEM ciphertext with the encrypted request.

2. The workload container passes the ciphertext to the Key Protection Service through the Workload Services Daemon.

3. The Key Protection Service decapsulates the ciphertext inside its boundary and returns the shared secret encrypted to the associated binding public key, so the unencrypted secret does not cross the bridge and the long-lived KEM private key does not leave the Key Protection Service.

4. The Workload Services Daemon, which holds the binding HPKE private key, decrypts the shared secret and returns it to the workload container, which uses it to decrypt the request. The response is encrypted with a key derived from the same shared secret.

The workload container uses these keys to On-Node Request Termination on the node. For the API surface for key generation, endorsement, and decapsulation, see Appendix: Google Cloud Workload Services Daemon.