TEE Attestation & Fleet Admission

Why TEE fleet nodes exist

Every other way a node joins a scope starts with a human: an admin issues an invitation, and the joiner redeems it (see Governance). A TEE fleet node is the exception. It is a hardware-attested read replica that admits itself by proving — cryptographically, against the hardware — that it is running an approved measurement inside a Trusted Execution Environment. No operator clicks “invite”.

The node joins in one of two TEE roles, chosen per namespace by the admission policy’s mode: as a ReadOnlyTee TEE replica, or as a RelayTee TEE relay. Either can decrypt a namespace’s data and answer sync requests, and neither ever writes in its own name; a relay may additionally carry members’ writes under their signed warrants. The fleet exists so a namespace’s data stays available and queryable even when the human-operated owner node is offline, NAT’d, or asleep.

The trust substitution is the whole point: where an invitation says “an admin vouches for this identity”, an attestation quote says “Intel’s DCAP chain vouches that this identity runs measurement X inside genuine TDX hardware”. The namespace’s admins decide, ahead of time, which measurements they trust — an admission policy — and from then on any node matching it can join unattended.

Note

This chapter is about the per-group encryption keys a replica receives so it can read a namespace, and the governance ops that admit and evict it. It is distinct from the node’s own storage/disk key that merod fetches from a KMS at startup. Those are unrelated mechanisms — do not conflate them.

The attestation quote

A TDX quote is a signed measurement of a confidential VM. Calimero treats it as an opaque blob plus a verified verdict; the crate that produces and checks it is crates/tee-attestation.

What a verified quote proves:

Field	What it is	Role in admission
mrtd	Measurement of the TD firmware the platform loads, before anything from the image runs	Matched against the policy’s allowed_mrtd
rtmr0	The firmware’s configuration: the virtual hardware and ACPI tables	Matched against allowed_rtmr0 when it is non-empty (optional)
rtmr1–rtmr3	The image: the boot chain and kernel (RTMR1); the kernel command line, which carries the root-filesystem hash, and the initrd (RTMR2); the role, profile and root hash the image’s init extends at boot (RTMR3)	Matched against allowed_rtmr1..3, each required
tcb_status	Platform TCB level reported by DCAP verification	Matched against allowed_tcb_statuses
report_data	64 caller-supplied bytes baked into the quote	Binds the quote to a fresh nonce and a public key

The MRTD does not name the image

On a cloud platform the MRTD measures the provider’s TD firmware, so it is the same for every TD there, whatever it runs. On GCP every mero-tee image, every profile (debug, debug-read-only, locked-read-only), every release and the KMS images share one MRTD, and one RTMR0. What tells them apart is RTMR1–3. Anything that decides whether to trust a TD must pin RTMR1–3 too: admission does, and so do the client verifiers (VerifierPolicy, mero-js createQuoteVerifier), which refuse a policy that leaves any of them empty.

The report_data binding is what stops a captured quote from being replayed for a different identity. It is built as:

report_data[0..32]  = nonce          (32 bytes, random per announce)
report_data[32..64] = SHA-256(pubkey) or app-hash (32 bytes)

For fleet admission the second half is SHA-256 of the joining node’s namespace-identity public key (fleet_join.rs). For the standalone /tee/attest endpoint it is the optional application bytecode hash, unless the request sets bindNodeKey.

With bindNodeKey: true, the second half is attest_key_binding(app_hash, node_key): SHA-256("calimero.tee-attest.key-binding.v1" || app_hash || node_key), with 32 zero bytes when no application was named. The response returns the key as boundPublicKey. Without the binding, a quote proves only that some genuine TEE ran this image, and a relay could forward the attest call to that TEE while answering everything else itself. A client that recomputes the binding from boundPublicKey knows that anything signed by that key comes from the attested machine, including the deltas and delegated-execution results it signs. The node signs with one identity for every namespace, so that is the key bound.

With bindTransportKey: true, the second half wraps whatever would have been there in a commitment to the node’s X25519 transport key: attest_transport_binding(inner, transport_key) = SHA-256("calimero.tee-attest.transport-key.v1" || inner || transport_key), where inner is the key binding, else the app hash, else 32 zero bytes. The response returns the key as transportPublicKey. It is the static key a client opens a sealed session to: it authenticates the node in the handshake, while the session keys come from ephemeral keys, for forward secrecy. The server generates it at startup and holds it only in memory, so it never leaves the TD and a restart replaces it.

The protected POST /admin-api/tee/registration-attest takes only a nonce and fills the second half with attest_registration_binding() = SHA-256("calimero.tee-attest.registration.v1"). /attest never produces that value (it produces zeros, an app hash, or one of the two bindings above), so a verifier that requires it knows the quote was asked for through the node’s protected router, not through the public attest endpoint. A fleet node uses it to register with its manager, which quotes over a nonce that commits to what it registers.

With includeCollateral: true, the response also carries collateral: the TCB info, QE identity, CRLs and certificate chains the quote is verified against, in dcap-qvl’s QuoteCollateralV3 form. A client that has nothing but the node, such as a browser or a mobile app, can then verify the quote offline, as mero-js’s createQuoteVerifier does. Taking the collateral from the node being verified is safe because Intel signs it: the node can choose only which valid collateral to serve, and the verifier still checks it against Intel’s root and against the time now. The node fetches it at most once an hour and shares it, so an unauthenticated flood of attestations does not become a flood of requests to the collateral source. A mock quote carries none.

Verification (verify_attestation) does four things:

1. Parses the TDX quote.

2. Fetches collateral from Intel PCS and verifies the quote’s cryptographic signature and certificate chain, extracting tcb_status and any advisory IDs.

3. Checks the nonce — report_data[0..32] must equal the expected nonce.

4. Checks the bound hash — report_data[32..64] must equal the expected public-key hash (or app hash), when one is supplied.

A result is is_valid() only when the signature verified and the nonce matched and the bound-hash check passed (or was not requested).

Caution

Mock quotes. On non-Linux/non-TDX hosts, and under merod run --mock-tee, the crate emits a structurally valid but cryptographically invalid quote carrying a MOCK_TDX_QUOTE_V1 header. verify_mock_attestation skips the signature check (it reports quote_verified: true unconditionally) and only re-checks the nonce and bound hash. A mock quote is accepted only by a policy that explicitly sets accept_mock = true. It exists for laptop/CI testing of the full join path and must never be trusted in production.

The admission handshake

Admission is a gossip exchange on the namespace governance topic ns/<hex(namespace_id)>. A namespace is its own root group, so the namespace id is the admission group id. The joining replica announces; every member receives the announce, but only an admin of the namespace or an already-admitted TEE node acts on it as a verifier. Other members stand down.

▶ Replay

Announce (the replica)

POST /admin-api/tee/fleet-join (fleet_join.rs) drives the replica side. It resolves the node’s namespace identity (the per-root-group keypair it joins under, not a throwaway key), builds report_data = nonce || SHA-256(pubkey), generates the quote, and broadcasts:

BroadcastMessage::TeeAttestationAnnounce {
    quote_bytes,
    public_key,        // the namespace-identity pubkey the quote is bound to
    nonce,             // the random nonce embedded in report_data
    node_type: SpecializedNodeType::ReadOnly,
}

Note

The nonce field’s doc-comment in snapshot.rs calls it a “Group DAG head hash for freshness binding”, but the code path sets it to rand::random() and verification matches it against report_data[0..32]. Trust the code: it is the per-announce random nonce. The node_type is carried but the admission handler ignores it.

A single gossip publish into an empty mesh is lost forever — gossipsub has no replay, and a NAT’d owner’s mesh forms only intermittently. So fleet-join re-announces: it republishes roughly every 2 s for up to ~30 s per call, runs a namespace bootstrap pull each cycle to seed the mesh, and polls for its own admission between announces. If the budget elapses without admission, the call returns { admitted: false, status: "announced" } and the fleet sidecar retries.

Asking an admitter directly (the replica)

The broadcast only works if a peer that may vouch is in the gossip mesh to hear it. When the only such peer is an owner’s laptop behind a relay, that mesh may form late or never, and the miss is silent. So fleet-join also takes admitter_addrs — libp2p multiaddrs ending in /p2p/<peer id>, the shape an invitation’s admitter_addrs has — and, when given any, asks those peers directly right after its first announce.

The request (InitPayload::TeeAdmissionRequest) carries exactly what the announce carries, over the sync stream, with a proof of possession binding the requester’s namespace key to its transport. The responder runs the same verify_and_admit the broadcast receiver runs, so the quote, the credential, the namespace policy and the vouching rule are decided exactly as below — and it answers (TeeAdmissionResponse { admitted, reason }). “Already a member” reads as admitted; a peer that may not vouch says so, and the replica asks the next address. The addresses therefore decide who is asked, never who may admit: a wrong or hostile one costs a dial and a refusal.

On a direct admission the replica pulls namespace governance immediately, then continues the loop below. The broadcast keeps running either way, for peers that predate the request and for a node given no addresses. The fleet manager supplies the addresses: the namespace’s admitted TEE nodes, then the owner’s node.

Verify and admit (the verifier)

The inbound handler is handle_tee_attestation_announce (crates/node/src/handlers/tee_attestation_admission.rs). It rejects an announce that did not arrive on a well-formed ns/<hex> topic, verifies the quote (mock or real) against SHA-256(public_key), drops it silently if invalid, then computes quote_hash = SHA-256(quote_bytes) and delegates to admit_tee_node with the extracted measurements.

admit_tee_node (crates/context/src/handlers/admit_tee_node.rs) is where policy is enforced. It first asks whether this node may vouch at all — only an admin of the group or a TEE member (ReadOnlyTee or RelayTee) of it may — and a node that may not returns Ok(()) without publishing, since peers would refuse its op. A node that may vouch then:

1. Read the policy for the namespace root (read_tee_admission_policy). No policy set → reject.

2. Mock gate. is_mock && !policy.accept_mock → reject.

3. Signed release (signed-release policies only; steps 3–4 are skipped). The TEE must name the mero-tee node release it runs, no older than the policy’s min_release_version. The admitter fetches that release’s published-mrtds.json with its detached signature and Sigstore bundle, refuses it unless the Release mero-tee workflow of calimero-network/mero-tee signed it, and requires the quote’s MRTD and RTMR1–3 (RTMR0 when the profile pins it) to match one of the policy’s allowed_profiles. See Signed-release policies.

4. Fail-closed allowlist. allowed_mrtd must be non-empty; an empty MRTD allowlist is rejected outright. The quote’s mrtd must be in it.

5. Runtime registers. allowed_rtmr1, allowed_rtmr2 and allowed_rtmr3 must each be non-empty, and the quote’s value must be in each. RTMR3 is the only register that names the image (profile and release). But calimero-init extends it from public inputs, so it proves something only when the kernel (RTMR1) and kernel command line + initrd (RTMR2) that ran before it are pinned as well. Without them, a custom kernel or initrd can reproduce a locked profile’s RTMR3. allowed_rtmr0 (the VM’s hardware configuration, which varies with machine shape) is checked only when non-empty. tcb_status fails closed: an empty allowlist enforces UpToDate.

6. Idempotency. If the member already holds a direct row, admission is a no-op Ok(()) — safe under the re-announce loop. The exception is a TEE row in the other TEE role than the policy’s mode: that node is re-admitted, and the fresh admission converts its row (see converting admitted TEEs).

7. Replay guard. is_quote_hash_used scans the group op log; a quote_hash already consumed in this scope is rejected, blocking replay of a captured announce.

8. Publish. Sign and publish GroupOp::MemberJoinedViaTeeAttestation { member, quote_hash, mrtd, rtmr0..3, tcb_status, role }, where role is the policy mode’s: ReadOnlyTee for replica, RelayTee for relay.

9. Deliver the key (next section).

The op’s apply handler (member_joined_via_tee_attestation.rs) re-checks the same invariants on every receiving peer, so a forged op cannot slip through replication: the role must be the one the namespace’s policy mode names, the signer must be an admin of the group or a TEE node already admitted to it (require_tee_attestation_verifier), a policy must exist, and the recorded measurements must still satisfy the allowlists — under a signed-release policy only tcb_status is re-checked, and the measurements are the voucher’s word, since a replaying peer holds neither the quote nor the release file. On success it admits the member and clears any prior deny-list entry.

Caution

Normative. A verifier MUST be an admin of the group whose attestation it admits (directly, as its genesis admin, or inherited from an ancestor) or a TEE member (ReadOnlyTee or RelayTee) of that same group. Plain membership is NOT sufficient: peers never see the quote — the op carries only the measurements the verifier claims — so whoever may sign the op decides who receives the group key. The admission op MUST record the role the namespace’s policy mode names — ReadOnlyTee in replica mode, RelayTee in relay mode; any other role is rejected at apply, so no admitter can mint a relay under a replica policy or the reverse. The TEE admission policy MUST live on the namespace root — TeeAdmissionPolicySet on a subgroup is rejected, and policy reads resolve to the root before reading. allowed_mrtd, allowed_rtmr1, allowed_rtmr2 and allowed_rtmr3 MUST each be non-empty, so a misconfigured policy fails closed rather than admitting everything. These are refused when the policy is set, when a node admits, and on every peer that applies an admission.

Key delivery (the verifier, again)

A membership row is not enough — the replica needs the namespace’s group encryption key to decrypt anything.

The admission itself is cleartext: RootOp::MemberJoinedViaTeeAttestation, readable by every peer. It has to be, because it carries the replica’s account credential and the peers who must verify that credential are precisely the ones who do not yet share a key with the joiner. Publishing it in the clear costs nothing — the measurements are not secret, an admission policy is a public allow-list of them, and the quote is verified before the op is signed.

The earlier encrypted form (GroupOp::MemberJoinedViaTeeAttestation) is retained for the subgroup fan-in, where the node is already a namespace member: bindings are namespace-keyed, so admitting it inward binds nothing new and the encrypted op stays correct there.

The verifier (a key-holder) therefore calls deliver_group_key_to_member, which publishes NamespaceOp::Root(RootOp::KeyDelivery { group_id, envelope }) on the namespace DAG. The envelope is ECDH-wrapped for the recipient’s public key, so only that node can unwrap it. Delivery is best-effort and one-shot; the durable fallback is the joiner-side recovery pull (recover_missing_group_keys), which sends a GroupKeyRequest that any key-holding member answers regardless of role. Key acquisition is thus a self-healing loop rather than a fragile single broadcast.

Self-confirm and auto-follow (the replica)

Back in fleet-join, the poll loop watches list_group_contexts. Once the membership op and KeyDelivery have propagated and been applied, the replica sees itself in the group, joins every context in the namespace, and publishes its own MemberSetAutoFollow { auto_follow_contexts: true, auto_follow_subgroups: true } — signed with its own namespace identity, which satisfies that op’s admin-or-self authorization rule. The verifier does not do this on the replica’s behalf: it does not hold the replica’s signing key, and when the verifier is another TEE it has no admin authority either. From then on, new contexts and subgroups auto-join without further sidecar polling (see Governance for the auto-follow machinery).

Worked example: a replica joins a fleet

Concretely, here is how a brand-new TEE replica brings itself online as a read-only follower of an existing namespace — no operator invite at any point.

1. An admin sets the policy, once. Ahead of time, a namespace-root admin publishes the measurements the fleet trusts. This is the only human step, and it happens long before any replica exists:

   # the namespace is its own root group, so the group id is the namespace id
   curl -X PUT \
     "$NODE/admin-api/groups/$NS/settings/tee-admission-policy" \
     -d '{ "allowed_mrtd": ["<approved-td-measurement>"],
           "allowed_rtmr1": ["<approved-kernel-measurement>"],
           "allowed_rtmr2": ["<approved-initrd-measurement>"],
           "allowed_rtmr3": ["<approved-image-measurement>"],
           "allowed_tcb_statuses": ["UpToDate"],
           "accept_mock": false }'

   This publishes TeeAdmissionPolicySet on the namespace root. The values come from the release’s published-mrtds.json. allowed_mrtd and allowed_rtmr1..3 must each be non-empty; an empty one is refused rather than admitting every quote.

2. The replica boots and announces. The fleet sidecar calls the protected route on the new node. The node resolves its namespace identity, builds report_data = nonce || SHA-256(pubkey), generates a TDX quote over it, and broadcasts TeeAttestationAnnounce on ns/<hex>:

   curl -X POST "$NODE/admin-api/tee/fleet-join" -d '{ "group_id": "'"$NS"'" }'

   The call re-announces every ~2 s for up to ~30 s, seeding the gossip mesh each cycle, because a single publish into a not-yet-formed mesh would be lost.

3. An admin or an admitted TEE verifies and admits. Such a node receives the announce, runs verify_attestation (DCAP signature + nonce + SHA-256(pubkey) bind), checks the quote’s mrtd/tcb_status against the policy, guards against a replayed quote_hash, and publishes RootOp::MemberJoinedViaTeeAttestation { role, quote_hash, mrtd, …, account }, role being the policy mode’s (ReadOnlyTee or RelayTee). A plain member does not: peers refuse an admission it signs, because they trust the verifier’s claimed measurements rather than re-checking the quote.

   The account is the replica’s own credential, which travelled on the announcement. The verifier checks it against the key the quote binds to before admitting, so a credential lifted from another replica’s announcement is refused — the membership still stands, the device simply does not bind.

4. The verifier delivers the key. Membership does not confer readability, so the verifier also publishes RootOp::KeyDelivery carrying the namespace group key ECDH-wrapped to the replica’s public key. Only the replica can unwrap it; if this best-effort delivery is missed, the replica’s own GroupKeyRequest recovery pull fills the gap.

5. The replica self-confirms and follows. Once the membership op and key have propagated, the replica’s poll loop sees itself in the group, joins every context in the namespace, and signs its own MemberSetAutoFollow { contexts, subgroups } — authorized by the admin-or-self rule because it signs with its own namespace identity. From here new contexts and subgroups auto-join with no further sidecar polling. fleet-join returns { admitted: true }.

The node is now a TEE member — a ReadOnlyTee replica or a RelayTee relay, as the policy’s mode says: it decrypts and serves the namespace’s data and answers sync requests, but never authors a write in its own name — keeping the namespace available even while the human-operated owner node is offline. A relay additionally carries members’ writes under their warrants.

The TEE roles: ReadOnlyTee and RelayTee

ReadOnlyTee (the TEE replica) and RelayTee (the TEE relay) are the two group member roles only attestation mints (identities covers the keypair model). GroupMemberRole::is_tee() is true for both, and everywhere the question is “is this an attested TEE node” — sync and availability anchoring, trusted-anchor and KeyDelivery signing, vouching for another TEE’s admission, TEE authorship, auto-follow, eviction and self-purge — the two are treated alike. They differ in one thing only: a relay may author members’ writes under their signed warrants, and a replica may not (delegated authorship). Their constraints:

• Directly-rowed, never inherited. A ReadOnlyTee membership is always a stored (member, group) row written by admission; it is never conferred by the inherited-membership parent-walk. A replica is a member only of the scopes it was explicitly admitted to.
• Read-only for its own writes, unless the namespace grants authorship. Neither role can author governance ops or state deltas in its own name; a JSON-RPC write on a TEE node is discarded and refused (ReadOnlyWriteRefused), as a ReadOnly member’s is. The state-delta receive path rejects any delta whose author’s effective role in the context’s group is ReadOnly, ReadOnlyTee or RelayTee, in every apply path (gossip, parent-fetch, buffered). Effective means held there or inherited from an ancestor through an Open subgroup, so a TEE admitted at the namespace root is read-only in every subgroup context it reaches the same way (the role of an inherited member). It exists to decrypt and serve, not to mutate. The one exception is a TEE whose verified attestation evidence reports an MRTD the namespace’s TEE authoring policy names: it may author as the TEE authority, and only from an #[app::tee] method its own scheduler fired. See TEE authorship.
• Relaying is the relay’s alone. Which of the two a TEE is, for relaying, is read from its namespace root row wherever it acts (see converting admitted TEEs). A delegated write executed by a ReadOnlyTee is refused — at POST .../intents with a 403, and by every peer at the cut — even when the replica holds CAN_AUTHOR_ON_BEHALF from a default mask or an explicit grant. A RelayTee relays by its role, with no capability bit. Its relayed writes are the author’s writes, so they are not discarded as the relay’s own would be.
• Attestation-only assignment. MemberJoinedViaTeeAttestation is the only op that mints either role, and MemberAdded refuses both. Manually setting a member to a TEE role via update_member_role is rejected. MemberRoleSet may only move an already-attested TEE row between the two TEE roles, and only to the one the policy mode names — the conversion an admin’s mode switch publishes.
• Locked to the TEE roles. The reverse is refused too: an attested TEE row is never moved to Member, ReadOnly or Admin. MemberRoleSet on a TEE row may target only the other TEE role (the conversion above); MemberAdded over a TEE row and AdminChanged naming a TEE are refused the same way, with TeeMemberRoleLocked (HTTP 403 from update_member_role and add_group_members, which refuse before signing). An enclave’s key given ordinary authorship in its own name would mix the two trust roles, and every check keyed on is_tee() would silently stop covering it. A TEE is still removed like any member (MemberRemoved, or its own leave); once removed it has no TEE row, so it can be re-added as anything.
• Auto-follow on. Admission plus the replica’s self-signed op leave both auto-follow flags set, so the replica tracks new contexts and subgroups in the scopes it holds.

Replica or relay: the admission mode

A namespace chooses one mode for every TEE it admits, carried inside the admin-signed admission policy: replica (the default) or relay.

Mode	Role	Replicates, anchors sync and availability, TEE authorship	Relays members’ writes
replica	ReadOnlyTee	yes	no — refused with a 403, whatever capability it holds
relay	RelayTee	yes	yes, under each member’s signed warrant, without CAN_AUTHOR_ON_BEHALF

Choose relay for a namespace whose clients have no node of their own and reach it through a hosted TEE relay — a browser tab or an agent holding one signing key. Choose replica (or leave it) for a namespace whose TEEs exist to keep data available and serve reads, where a node that could write on members’ behalf is authority you do not want to hand out.

meroctl --node node1 group settings set-tee-admission-policy <NAMESPACE_ID> \
  --profile locked-read-only --tcb-status UpToDate --mode relay
meroctl --node node1 group settings get-tee-admission-policy <NAMESPACE_ID>

Over the admin API it is the optional mode field of PUT /admin-api/groups/:group_id/settings/tee-admission-policy, "replica" or "relay"; absent means replica, and the GET always reports it.

The tee-admission-mode-relay.yml merobox scenario in apps/scaffolding-e2e covers both modes end to end. Two mock TEEs fleet-join two namespaces that differ only in mode. The one admitted under relay is a RelayTee, and it runs a member’s POST .../intents (200). The one admitted under the default is a ReadOnlyTee, and it refuses the same request by role (403).

The mode is part of the policy op, so every peer derives the role from the same governance state; an admitting node has no local say. A policy set before the mode existed (TeeAdmissionPolicySet, TeeReleaseAdmissionPolicySet) still decodes and reads as replica. New policies are published as TeeAdmissionPolicySetV2 / TeeReleaseAdmissionPolicySetV2, appended to GroupOp so no stored discriminant moves.

Upgrade a namespace’s peers together. RelayTee was appended to GroupMemberRole and the two policy variants to GroupOp, and SIGNED_NAMESPACE_OP_SCHEMA_VERSION went from 11 to 12 with them. Stored ops still decode, so nothing is re-bootstrapped; the bump is for the other direction. A v11 node could decode neither a RelayTee join nor a v2 policy, and would still let a ReadOnlyTee relay members’ writes that a v12 node refuses at the cut, so the version gate keeps the two out of one namespace rather than letting them diverge partway through its DAG.

Converting TEEs already admitted

Setting the policy with a different mode converts the TEEs the namespace already holds: after publishing the policy, the admin’s node publishes a MemberRoleSet for every direct TEE row in the namespace whose role differs, and every peer checks each one against the policy it has just applied. A TEE inheriting into an Open subgroup carries its root row’s role, so converting the root converts it there too. A row in a Restricted subgroup the admin does not administer — the copy tee_subgroup_admit carried there — cannot be converted by the namespace admin, because admin authority does not cross a Restricted boundary; it is logged and left for that subgroup’s admin, and the TEE’s next attestation there converts it as well, since an admission for a TEE row in the other role converts rather than no-ops. Neither may ever happen (a TEE’s fleet-join does not re-run for a namespace it already joined), so the stored copy is not what decides relaying: a TEE’s replica/relay role is read from its namespace root row wherever it acts. When the root row holds a TEE role it replaces the TEE role on any subgroup row, so converting the root is enough — a TEE switched back to replica stops relaying in every subgroup at once, and one switched to relay relays in them. A TEE with no TEE row at the root (admitted into a subgroup alone) keeps its subgroup row’s role. This is warrant_gate::executor_standing, the one function POST .../intents, the relay descriptor and every peer at the cut share; because peers must agree on it, it arrived with SIGNED_NAMESPACE_OP_SCHEMA_VERSION 13, a coordinated upgrade.

The read is the policy at apply time, the same state the admission’s allowlists are checked against. A mode switch that races an admission can therefore be seen in different orders by different peers — the same window the allowlists already have; the admin’s conversion ops, which are causally after the policy, close it for every TEE they cover.

The admission policy

The policy is itself a governance op, GroupOp::TeeAdmissionPolicySet, settable only by a namespace-root admin:

TeeAdmissionPolicySet {
    allowed_mrtd: Vec<String>,
    allowed_rtmr0: Vec<String>,
    allowed_rtmr1: Vec<String>,
    allowed_rtmr2: Vec<String>,
    allowed_rtmr3: Vec<String>,
    allowed_tcb_statuses: Vec<String>,
    accept_mock: bool,
}

There is no materialized policy row — the op log is the storage. read_tee_admission_policy scans the root’s op log and takes the last policy op of any form — TeeAdmissionPolicySet, TeeReleaseAdmissionPolicySet, or their mode-carrying …V2 successors — so a later op simply supersedes an earlier one of either form (last-writer-wins, like the rest of the projection). The apply handler validates only that the signer is an admin and that the target is a namespace root, not a subgroup.

Signed-release policies

A measurement list has to gain RTMR3 for every release: a fleet node on a new image is refused until an admin copies the new value in. The second form trusts the release’s signature instead:

TeeReleaseAdmissionPolicySet {
    allowed_profiles: Vec<String>,        // e.g. ["locked-read-only"]
    min_release_version: Option<String>,  // oldest release admitted
    allowed_tcb_statuses: Vec<String>,
    accept_mock: bool,
}

mero-tee’s release workflow keyless-signs each node release’s published-mrtds.json with cosign, publishing a detached signature and a Sigstore bundle next to it. The admitter fetches all three for the release the TEE names (calimero-tee-release, cached per release), verifies the Fulcio certificate names the Release mero-tee workflow on refs/heads/master of calimero-network/mero-tee and is issued to its workflow file (.github/workflows/release-node-image-gcp.yaml), checks the file’s tag is that release, and matches the quote against the allowed profiles. A TEE that names a release it does not run, or one never published, matches nothing.

The release travels as a claim: BroadcastMessage::TeeReleaseAttestationAnnounce and InitPayload::TeeReleaseAdmissionRequest are the old forms plus release_version, appended at the tail of their enums. fleet-join sends the named form when merod knows its release (MERO_TEE_VERSION, written by calimero-init from the instance’s tee-release-version) and always sends the old announce too, so admitters that predate the new form still hear the node. An admitter under a signed-release policy refuses a TEE that names no release.

Two properties follow from the choice to trust the voucher on replay:

• A replaying peer checks the voucher’s role and tcb_status, as it does for every admission, and records the measurements without comparing them. What it relies on is that the voucher is an admin or an admitted TEE, which the list form already relies on for the quote itself.
• A subgroup admission of an already-admitted TEE (tee_subgroup_admit) does not re-fetch the release: its record carries measurements, not the version, and the root admission checked it.
• TeeAuthorityEvidence is held to the same rule. Its quote still verifies offline against the collateral it carries, and its TCB status against the policy, but its measurements are not compared with a list the policy does not have. Which admitted TEEs may author as the TEE authority stays decided by TeeAuthoringPolicySet’s MRTD list.
• The release is checked only for a new admission. An already-admitted TEE re-announcing so its evidence gets published (tee::evidence_retry) sends the plain announce, and the admitter’s already-member branch runs before any release is required.

Only node releases published with a .bundle.json can be verified; releases before that are refused under this form.

Eviction and self-purge

Decommissioning is a coordinated, cross-repo flow. The control plane never reaches into the node; it uses soft disable — it drops the namespace from a fleet node’s assignments, and the fleet sidecar (in the mero-tee repo) notices and runs a namespace leave. That publishes a removal op (MemberLeft, or an admin’s MemberRemoved). Core owns the rest.

When a removal’s captured role is a TEE role (ReadOnlyTee or RelayTee), the apply path emits a role-scoped TeeMemberRemoved event in addition to the generic MemberRemoved. A removal at the namespace root cascades: because a replica’s presence in any subgroup came from namespace-level attestation (not a subgroup admin’s choice), root authority extends to it, so the apply walks the whole subtree and emits TeeMemberRemoved per descendant subgroup the node held a direct row in.

control plane drops assignment
  → sidecar runs `namespace leave`
    → MemberLeft / MemberRemoved (role = ReadOnlyTee or RelayTee)
      → TeeMemberRemoved emitted (root + each subgroup row, cascaded)
        → self_purge on the evicted node

self_purge (crates/context/src/self_purge.rs) is a node-local listener that reacts only to TeeMemberRemoved for this node’s identity — deliberately not the generic MemberRemoved. The split matters:

• Non-TEE removals stay soft-leave. Local rows remain so kick-and-readd, rejoin-via-keyshare, and inheritance-rejoin flows can reuse them.
• TEE removals hard-purge. A TEE node has no rejoin pathway — the only admission op for either TEE role re-derives identity from a fresh attestation — so leaving key material on disk buys nothing and hurts forward-secrecy hygiene.

For a TEE eviction the handler deletes the group’s local replicated data, signing keys, and AES group encryption keys, and (for a namespace-root removal) cascade-purges the subtree and unsubscribes from the namespace gossip topic; a subgroup-only removal purges just that group’s rows and keeps the topic subscription for the node’s other memberships.

Note

Self-purge does not rotate any key. Forward secrecy on the namespace’s new writes is already provided by the standard MemberRemoved rotation pipeline, which mints a fresh group key wrapped for everyone except the removed member. Self-purge only deletes what the evicted node held locally — including the now-orphaned old key material. A purge interrupted by a crash is finished by a durable pending-self-purge marker plus a startup reconcile sweep.

Admin API surface

The /admin-api/tee routes back this flow. fleet-join and registration-attest are on the protected (authenticated) router; the others are unprotected.

Method · Path	Body	What it does
GET /tee/info	—	Returns { cloud_provider, os_image, mrtd } for the host TEE.
POST /tee/attest	{ nonce, application_id?, bindNodeKey?, bindTransportKey?, includeCollateral? }	Generates a TDX quote over nonce || app-hash (or the key / transport bindings above); rejects a mock result unless --mock-tee. With includeCollateral, also returns the Intel-signed collateral the quote verifies against (see below).
POST /tee/registration-attest (protected)	{ nonce }	Generates a TDX quote over nonce || attest_registration_binding(), for a fleet node registering with its manager.
POST /tee/fleet-join (protected)	{ group_id, admitter_addrs? }	Runs the full announce → admission → join loop above, asking admitter_addrs directly first when given.

The admission policy is set separately via PUT /admin-api/.../groups/:group_id/settings/tee-admission-policy, which publishes TeeAdmissionPolicySet — or, when the body carries signedRelease: { allowedProfiles, minReleaseVersion? } and no measurement lists, TeeReleaseAdmissionPolicySet. GET on the same path reports signedRelease for the second form.

Verifying from outside Rust

Everything above happens between nodes, where the verifier is merod and can call verify_attestation directly. A service written in something else cannot: it is a library function with no binary in front of it. calimero-tee-verify exists for that caller — the fleet manager verifying a quote that a node submits when it registers is the motivating case, and it is written in Python.

Build it from the calimero-tee-attestation crate under the cli feature; a default build produces no binary and pulls no async runtime.

cargo build -p calimero-tee-attestation --features cli

It reads one JSON object on stdin and writes one on stdout, so there is no argument parsing to get wrong:

{"quote_b64": "...", "nonce_hex": "<64 hex>", "app_hash_hex": "<64 hex>"}
  -> {"valid": true, "quote_verified": true, "nonce_verified": true,
      "application_hash_verified": true, "tcb_status": "UpToDate",
      "advisory_ids": [], "tcb_evaluation_data_number": 17,
      "measurements": {"mrtd": "...", "rtmr0": "...", "rtmr1": "...",
                       "rtmr2": "...", "rtmr3": "..."}}

quote_hex may be given in place of quote_b64. The collateral endpoint is whatever CALIMERO_TEE_COLLATERAL_URL names, as for any other caller of this crate — see config.

The exit code separates a verdict from the absence of one, which matters more here than it looks. Collateral is fetched from Intel PCS over the network, so if an outage and a rejection were reported the same way, an outage would become an open door.

Exit	Meaning
0	A verdict was reached. Read valid — it may be false.
1	The input was unusable: bad JSON, bad hex, wrong lengths.
2	No verdict. The quote could not be parsed, or collateral could not be fetched. This is not a rejection.

Caution

valid mirrors VerificationResult::is_valid(), so it is crypto and structural only: signature and certificate chain, the nonce, and the app-hash binding. It deliberately says nothing about whether the platform’s TCB is acceptable or whether the measurements are ones you allow. Both are reported — tcb_status and measurements — and both are the caller’s to enforce. A Revoked platform still yields valid: true, so admitting on valid alone rebuilds exactly the gap the admission policy exists to close.

The binary has no mock path under any feature combination. A mock quote is valid by construction, so accepting one would let an untrusted caller present something that verifies with no hardware behind it; here it simply fails the real parse and exits 2.