Home/TEE Mode

TEE Mode (storage key / KMS)

Trusted Execution Environment operation — the node's storage/disk encryption key

Which key this page is about

This page covers the node's storage / disk encryption key obtained from the KMS — a per-node, MRTD-gated key fetched at startup and used to encrypt the local datastore and blobstore. It is distinct from the per-group GroupKeyring encryption keys delivered via KeyDelivery, which protect fleet-HA application data shared across a group. The two are unrelated mechanisms — do not conflate them. For the fleet / per-group key story, see TEE Fleet HA.

Overview

merod in TEE mode runs inside a confidential VM and obtains its storage encryption key from a KMS at startup. Only nodes with a [tee] section perform the KMS key-fetch and attestation flow; other nodes use libp2p as usual without KMS.

Deployment on GCP, Phala, and related images is documented in mero-tee.

KMS provider

Supported path today: Phala Cloud — mero-kms-phala from the mero-tee repository (merod in a Phala CVM with dstack).

Configuration

Run merod … init first, then add KMS settings (CLI config or edit config.toml):

[tee]
[tee.kms.phala]
url = "https://<kms-host>:8443/"

[tee.kms.phala.attestation]
enabled = true
accept_mock = false
allowed_tcb_statuses = ["UpToDate"]
allowed_mrtd = ["<trusted_kms_mrtd_hex>"]
allowed_rtmr0 = ["…"] … allowed_rtmr1..3 similarly in production

Optional TLS/mTLS paths under [tee.kms.phala.tls]; external policy via policy_json_path / binding_b64.

Release policy environment variables

When set, merod fetches attestation policy from the official release and verifies the KMS via POST /attest before key requests. Precedence:

MERO_KMS_RELEASE_TAG > MERO_KMS_VERSION > MERO_TEE_VERSION

(MERO_KMS_RELEASE_TAG accepts mero-kms-vX.Y.Z or X.Y.Z.) If a version is set and policy fetch fails, startup aborts (fail closed). Air-gapped: USE_ENV_POLICY=true with a verified local policy script.

Startup flow

  1. If attestation is enabled: KMS self-attestation (POST /attest) — quote validity, nonce/binding in reportData, policy for TCB, MRTD, RTMR0–3.
  2. Request a challenge from the KMS.
  3. Build a TDX quote with the challenge nonce and peer ID hash.
  4. Sign with the node identity key from config.toml.
  5. Submit to the KMS and receive the storage key for datastore/blobstore.

GCP operators: MRTD verification

Compare deployed nodes to published measurements using published-mrtds.json from mero-tee releases, e.g.:

https://github.com/calimero-network/mero-tee/releases/download/<X.Y.Z>/published-mrtds.json

Step-by-step: mero-tee verify-mrtd.

Deployment model

  • Isolate each merod release line with its own KMS; pin attestation inputs to a specific signed mero-tee release tag (do not auto-follow latest).
  • Avoid mixing old/new TEE cohorts against one KMS unless deliberately governed.
Troubleshooting
  • Key fetch fails — Confirm tee.kms.phala.url is reachable; KMS /health returns 200; KMS can access the TEE attestation socket (e.g. dstack on Phala).
  • KMS rejects attestation — Dev: mock attestation per mero-tee docs. Prod: align measurement policy with mero-kms-phala.
  • Mock vs production — accept_mock and MEROD_ALLOW_MOCK_KMS_ATTESTATION are for development only. The generate_mock_attestation(report_data) function in calimero-tee-attestation is a public, OS-independent API that produces a structurally valid but cryptographically invalid quote; it is accepted only by KMS policies with accept_mock = true and must never be used in production.
  • Local fleet testing without TDX hardware — Pass merod run --mock-tee to enable the dev/test-only mock-TEE harness. The node produces and accepts a synthetic attestation quote (via generate_mock_attestation) through the real fleet-join / announce / admit path, allowing full TEE/HA fleet lifecycle testing on a laptop. This mode is disabled by default and will refuse to start if a real KMS attestation endpoint is configured — it must never be used in production.

Known fixes (PR #2855)

  • Buffered ContextRegistered re-driven after GroupCreated — A buffered encrypted ContextRegistered event that arrived before its subgroup's GroupCreated had applied was previously stranded. It is now re-driven once GroupCreated applies, ensuring the context is correctly registered (fixes #2848).
  • Fleet-join first announce no longer 500s on empty mesh — fleet-join's initial announce into an empty gossipsub mesh previously returned HTTP 500. This is now handled gracefully (fixes #2491).
  • KeyDelivery-bootstrapped replica seeds CAN_JOIN_OPEN_SUBGROUPS — A TEE replica bootstrapped via a KeyDelivery seed now correctly seeds the CAN_JOIN_OPEN_SUBGROUPS capability so it can replicate Open subgroups via inheritance, matching the behaviour of nodes that joined through the normal fleet path.

Attestation flow

  1. merod starts and detects a [tee] configuration block.
  2. Requests KMS self-attestation (POST /attest) — verifies the KMS quote, nonce, binding, and MRTD/RTMR policy.
  3. Requests a challenge from the KMS.
  4. Generates a TDX attestation quote embedding the challenge nonce and peer ID hash.
  5. Signs the payload with the node identity key from config.toml.
  6. Submits the signed request to the KMS.
  7. Receives the storage encryption key (AES-256).
  8. Uses that key for datastore and blobstore encryption.