Encryption

There are two encryption boundaries, and between them nothing is in the clear: in transit, every request and response travels in a sealed session bound to an attested key; at rest, every value a node writes is sealed under the cluster’s storage key. In between, data exists in plaintext only inside the TD’s memory.

In transit

Hop	Protection
Client → node	Sealed session (Noise NK to the attested transport key, AES-256-GCM per request and per response frame)
Node → node	The same sealed session, to the peer’s attested transport key, plus the cluster secret on every internal route
Anything unsealed	Only /attestation and /health

The session keys come from ephemeral keys on both sides, so a session that has ended (after an hour at most) cannot be opened later, not even with the node’s transport key. See Networking & sealed channels.

At rest: the storage key K_r and the cluster key K_c

▶ Replay

Attestation comes before any key: a node gets nothing from mero-kms until its quote matches a release the KMS’s policy accepts, and everything it writes is sealed under keys derived from what it got. Every node of a cluster derives the same K_r, so a follower opens a leader’s snapshot as it is; a different cluster name gives a different K_r from the same KMS key. The keys that identify a node to clients — its transport and receipt keys — are attested on their own, never derived from K_r. The transport key is made in memory at start; the receipt key, which also signs sessions, is made on the node’s first start and kept in its data directory wrapped under K_r, so it survives a restart and only the release that holds that K_r opens it.

Each node’s data — every Raft entry, vote, state row, snapshot frame and blob — is sealed under one 32-byte cluster key, K_c, which the node keeps on disk wrapped under its storage key, K_r, from mero-kms.

K_r = HKDF-SHA256(ikm = KMS key, salt = "calimero.cloud.storage.v1", info = cluster name)

• From mero-kms, after attestation. A TDX node fetches the KMS key itself (--kms-url with --kms-identity) through calimero-kms-client. The client trusts the KMS only if the KMS’s own quote matches the signed mero-tee release the environment names (MERO_TEE_VERSION, MERO_KMS_VERSION, MERO_KMS_RELEASE_TAG); its request carries this TD’s quote over the KMS’s challenge, and the key comes back sealed to that request.
• Never from the host. Without --dev, --storage-key and --storage-key-file are refused (kms::key_source): a key the host could hand the node would let whoever runs the host read the disks. merod kms disk-key fetches the same key for a boot script and writes it only to tmpfs or ramfs; merod run accepts such a file only with --dev today. A --dev node without a key derives a development key from the cluster name.
• Shared by the cluster’s nodes of one release. mero-kms derives a key per identity, and the nodes of one cluster share one disk identity, so they derive the same K_r.
• Bound to the cluster. The cluster name is the HKDF info, so the same KMS key gives another cluster a different K_r.
• Bound to the release. Each mero-tee release has a mero-kms cluster of its own whose random root lives only in its replicas’ memory (mero-tee docs/design/gcp-tdx-kms.md, “Lifecycle”), so a node of the next release gets another K_r, and once a release’s KMS is retired its K_r is gone for good.

The cluster key K_c

K_r changes with every release, but a cluster’s data outlives releases: a release upgrade starts nodes on the new image, moves every app onto them and retires the old ones (Upgrade the cluster’s release). So nothing the nodes share is keyed by K_r itself (calimero_node::cluster_key):

• Minted once. The founding node (--bootstrap) takes its own K_r as K_c. A node whose store was written before K_c existed does the same: its data is sealed under K_r, which was then the cluster’s only key, so nothing on disk changes.

• Wrapped per release. Every node keeps K_c in <data-dir>/cluster-key, AES-256-GCM under HMAC-SHA256(K_r, "calimero.cluster-key.wrap.v1"), and unwraps it at each start. A K_r that does not open it (the same disk booted on another release’s KMS) stops the node: a node does not move to a new release in place. The node’s signing key (receipts and sessions) is kept beside it the same way, in <data-dir>/signing-key under HMAC-SHA256(K_r, "calimero.signing-key.wrap.v1"), made on the first start.

• Delivered to new nodes by attestation. A node starting on an empty data directory without --bootstrap serves only /health (with a waiting field), /attestation and a delivery route, and asks its --peers for K_c (POST /internal/cluster-key { cluster, node, token }, inside the sealed channel to a peer whose quote it verified), every second for up to 10 minutes (cluster_key::WAIT). The peer never answers with the key. It delivers it to the address the cluster has for that node id (registered with /ops/nodes, or in the peer’s own --peers; not a removed node), inside a sealed channel to the key the TD at that address attests, judged like any peer (--peer-profiles, --peer-min-release) (POST /internal/cluster-key/deliver { token, key }). The waiting node takes it only with its own random token, which travelled only to an attested peer. So K_c reaches only a genuine TD at an address the operator chose, the same bar the shared disk identity sets for K_r (mdma stamps it only after attesting the instance). At most MAX_CONCURRENT_KEY_RELEASES (4) requests are handled at once.

• Kept with backups, per release. A node with --backup-dir writes K_c there wrapped under its own K_r (cluster-key/{release}, the same wrapping as <data-dir>/cluster-key), once per release. A cluster founded again from its backups (--restore-from) takes K_c from the copy its founder’s K_r opens, so it is the same cluster; no other key opens any copy, and a copy dies with its release’s mero-kms (Found a cluster again).

The route takes no cluster secret, since the new node has none yet. Release approvals still decide which apps’ data reaches the new node (Node::may_send): K_c opens what a node holds, not what it may receive.

How a value is sealed

calimero-log’s store seals every value with AES-256-GCM under K_c, with a fresh random nonce, and binds it (as associated data) to its column, its group and its key. A value moved to another row, group or column fails to open. A state machine can also hide a key: the row is then keyed by an HMAC of the key and the key itself travels inside the sealed value, so the disk does not even show which keys an app uses. App state rows and idempotency records are hidden this way. See Storage schema.

Snapshots are files of sealed frames, each frame’s associated data naming its group, snapshot, position and whether it is the last, so frames cannot be reordered or truncated unnoticed. Backups are copies of those files, each with a manifest sealed under HMAC-SHA256(K_c, "calimero.backup.manifest.v1") and bound to its app and file name, so a backup opens and restores only where K_c is: on the cluster that wrote it, whatever release its nodes run.

Keys derived from K_c

Key	Derivation	Used for
Cluster secret	HMAC-SHA256(K_c, "calimero.cluster.v1|" ‖ cluster)	x-cluster-auth on internal routes
App TEE key	HMAC-SHA256(K_c, "calimero.tee.v1|" ‖ app)	Values sealed to the app’s TEE (TeeSecret, a timer’s own state)
Member key	HMAC-SHA256(K_c, "calimero.member.v1|" ‖ app ‖ "|" ‖ account)	A member’s sealed share, opened only in that member’s own runs
Hidden-key HMAC key	HMAC-SHA256(K_c, "calimero.log.hidden-keys.v1")	Hiding row keys in the store
Snapshot key	HMAC-SHA256(K_c, "calimero.log.snapshot.v1")	Sealing snapshot frames (streamed to followers as they are)
Backup manifest key	HMAC-SHA256(K_c, "calimero.backup.manifest.v1")	Sealing a backup’s manifest, bound to its app and file name
Component cache key	from K_c (component_cache_key)	The MAC on compiled components in <data-dir>/components

Every node, of every release, derives the same keys, and nothing outside the cluster can. These derivations are the ones K_r fed before K_c existed, so a cluster founded earlier keeps every value, and a next-release node opens them (regression: nodes_on_the_next_release_keep_the_data_of_the_ones_before). Keys per app from mero-kms, which would let an app’s data be opened only where that app is placed, are not done yet: today one K_c covers every app on the cluster.

What an observer can still see

Position	Sees	Does not see
The network or a TLS-terminating proxy	That a client talks to a node; request sizes and timing; unsealed attestation documents	Paths, tokens, arguments, results, events
The host, reading the disks	Sizes of sealed values, row counts per column; which groups exist	Keys of hidden rows, values, entries, snapshots, blobs
The operator, through the Ops API	Counts, Raft status, node list, governance, quotas, Prometheus metrics	App names, state, events, accounts’ details, credentials

The host also controls availability: it can stop, freeze or restart a node, or withhold the network. Raft keeps an app available while a majority of its replicas run. A restored older copy of a node’s disk is not detected yet: the proposed rollback counter (a mero-kms epoch, with restored nodes rejoining as learners) is not done.