Deploy the node image

How to build a GCP TDX merod node image with this repository and boot it as an Intel TDX Confidential VM. This is the layer underneath the runbooks: the runbooks point an already-running merod at the KMS; this page gets a node image built, launched, and healthy in the first place. Everything below is grounded in the mkosi build and Ansible roles under mero-tee/ and the GCP workflows — confirm exact flags against --help and your checked-out version before running against production.

Caution

Mero TEE is pre-1.0. Image names, machine types, pinned versions, and defaults can change between releases. Boot a debug or debug-read-only image first; never make a fresh locked-read-only build your only path to a fleet.

What gets baked into an image

A node image is an Ubuntu 26.04 LTS (Resolute Raccoon) image built by mkosi (mero-tee/mkosi-image) from mero-tee/playbook.yml’s roles. Its packages come from a pinned snapshot of the Ubuntu archive (Snapshot= in mkosi.conf), so two builds of one commit give the same image. RTMR3 sysfs support needs kernel 6.17+. The image boots as a UKI with an EROFS root behind dm-verity. Component versions are pinned in mero-tee/versions.json:

Component	Version key	Value at time of writing
Node image	imageVersion	2.3.123
merod / meroctl / mero-auth (core tag)	merodVersion	0.11.0-rc.90
Traefik	traefikVersion	3.5.0
node-exporter	nodeExporterVersion	1.9.1
vmagent	vmagentVersion	1.132.0
Vector	vectorVersion	0.50.0

The playbook.yml applies these roles in order: mero-traefik, calimero-core (bakes merod/meroctl/mero-auth at the core tag so calimero-init skips the runtime download), merotee (the init script, systemd units, and — for read-only profiles — the fleet sidecar), node-exporter, vmagent, vector, cleanup, and — only for locked-read-only — merod-lockdown followed by merotee-conformance.

Profiles

The PROFILE the build is given selects one of three profiles (mero-tee/mkosi-image/build.sh accepts exactly this set):

Profile	Intended use	merod mode	Lockdown role
debug	Local / dev	full	not applied
debug-read-only	Integration / pre-production	read-only	not applied
locked-read-only	Production	read-only	applied + conformance

merod_mode is derived in ansible/roles/merotee/defaults/main.yml: it is read-only for debug-read-only and locked-read-only, and full for debug. In read-only mode the boot script initializes merod with --mode read-only and disables the client-facing servers (see the boot sequence below).

The merod-lockdown role (ansible/roles/merod-lockdown/tasks/main.yml) hardens the production image so its measurements differ from the debug profiles. It:

• R1 removes openssh-server/openssh-client and deletes /etc/ssh;
• R2 masks all getty@/serial-getty@ttyS0/console-getty/getty.target units; the image’s kernel cmdline carries no console= (the build refuses one);
• R3 masks emergency/rescue/debug-shell (and overrides emergency/rescue to reboot -f);
• R4 locks the root account and empties /etc/securetty;
• R5 masks google-startup-scripts and the cloud-init* units so no GCP metadata script is ever executed — metadata is only read as data by the init script’s whitelist.

Note

Each profile deliberately produces distinct on-disk content (/etc/calimero/image-profile, calimero.role/calimero.profile in the kernel cmdline) so its MRTD / RTMR measurements differ. That is what lets the KMS release keys only to a matching profile — see Verification.

Prerequisites

• mkosi (v27, as CI pins it) and Python with Jinja2 and PyYAML. CI installs the rest with scripts/release/mkosi/install-tools.sh.
• Root: build as root, as CI does; unprivileged, mkosi’s user namespace gives every file root as its owner.
• GCP credentials only to upload and boot the image.

Build an image

1. Build one role and profile, with every version read from versions.json:

   sudo ROLE=node PROFILE=locked-read-only mero-tee/mkosi-image/build.sh OUT

   It writes OUT/merotee-node.raw (the disk), merotee-node.efi (the UKI) and mkosi’s package manifest. ROLE is node, kms or agent.

2. Compute the measurements and upload, as the releases do: scripts/release/mkosi/build-image.sh builds, checks the command line, computes RTMR1–3 and packages the disk, and scripts/release/mkosi/create-gce-image.sh creates the GCE image.

To build, boot and probe an image on GCP TDX from a branch, run node-image-mkosi.yaml (see Build the images with mkosi).

Where images are published

Each release image is published into the GCP project with a deterministic name and family (scripts/release/mkosi/release-image-name.sh):

• Image name: merotee-ubuntu-questing-25-10-<profile>-<version-with-dashes> (e.g. merotee-ubuntu-questing-25-10-locked-read-only-2-3-51).
• Image family: merotee-ubuntu-questing-<profile>.
• Disk: the disk mkosi writes, about 2 GB.

The questing-25-10 in those names is a frozen identifier, not the base release — the base is 26.04 LTS. mdma’s dispatcher matches on these prefixes, so they do not move when the base does.

The release-node-image-gcp.yaml workflow builds the images in CI and records the published project / family in its release provenance. Its VM-facing defaults (used for the post-build attestation probe) match Calimero Cloud MDMA: project cloud-486420, zone europe-west4-a, machine type c3-standard-4.

Launch a TDX Confidential VM

Boot the image as an Intel TDX Confidential VM. The flags below are exactly those the release-node-image-gcp.yaml and node-image-gcp-staging-probe.yaml workflows use to create their ephemeral VMs:

gcloud compute instances create <instance-name> \
  --project <vm-project> \
  --zone europe-west4-a \
  --machine-type c3-standard-4 \
  --confidential-compute-type TDX \
  --maintenance-policy TERMINATE \
  --min-cpu-platform "Intel Sapphire Rapids" \
  --no-service-account \
  --no-scopes \
  --image <image-name> \
  --image-project <image-project> \
  --subnet <subnetwork> \
  --metadata merod-version=<merod-version> \
  --tags merotee

Note

TDX requires --confidential-compute-type TDX with --maintenance-policy TERMINATE and --min-cpu-platform "Intel Sapphire Rapids" on a c3 machine type (c3-standard-4 is the tested size). The CI probes run with --no-service-account --no-scopes — the node reads metadata as data but never needs GCP API credentials.

GCP metadata the boot script reads

calimero-init reads a fixed whitelist of instance metadata attributes (ansible/roles/merotee/templates/calimero-init.sh.j2) — never arbitrary scripts. All are optional except where noted:

Metadata key	Effect	Default
server-port	merod admin/server port (1–65535; anything else falls back to the default)	2428
swarm-port	merod libp2p swarm port (1–65535; anything else falls back to the default)	2528
tee-release-version	written as MERO_TEE_VERSION into /etc/calimero/merod.env (sanitized/escaped), with MERO_TEE_MIN_VERSION (the image’s own version) and MERO_TEE_PROFILE (the image’s profile), so merod verifies the KMS against that profile’s signed policy. Refused if older than the image (downgrade protection)	unset → env file removed
kms-url	the VPC-internal URL of the release’s KMS cluster, which the data-disk key and the store key come from; verified against tee-release-version’s signed policy	unset → locked-read-only refuses to create a node; debug profiles create an unencrypted one
metrics-endpoint	if set, configures and starts vmagent and node_exporter. Must be https:// except on the debug and debug-read-only profiles	unset → vmagent stays off
logs-endpoint	if set, configures and starts vector. Must be https:// except on the debug and debug-read-only profiles	unset → vector stays off
observability-token	the bearer token both shippers present; written to /etc/vector/provided_token and used via the provided secret provider	unset → the node falls back to the credential MDMA delivers on a should-join poll
merod-version	only used to download binaries at boot if they were not baked into the image — debug profiles only; locked-read-only refuses to boot rather than run an unmeasured binary	required only when unbaked

Observability is off by default (R7): vmagent and vector are stopped and disabled at boot and started only when the matching *-endpoint metadata is present. node_exporter is started alongside vmagent, because it is the only thing the default scrape config targets — its unit file is installed by the image but is not enabled, so without that it would be a dead scrape target.

Shipped logs are readable by whoever sets logs-endpoint

Instance metadata picks the log sink, so treat every shipped line as readable by the VM’s operator. Node, KMS and agent images send their journal through the guard_egress transform before the sink. It runs mero-tee/ansible/roles/vector/files/egress-guard.vrl, which is baked into the image and measured.

The guard does three things:

• Allowlists fields. Only timestamp, message, unit, hostname, instance_name, instance_type, instance_profile and priority leave. journald’s _CMDLINE, _PID, _BOOT_ID, _MACHINE_ID and the rest are dropped.
• Masks credentials in the message: PEM private keys, JWTs, Bearer and Basic credentials, and the value of any secret-named key (password, token, secret, api_key, private_key, seed, mnemonic, cookie, authorization and similar).
• Masks application data merod logs at its default level. That is a guest’s execution log lines, a method’s error payload, guest panic text and the panic hook. It also masks admin login names, client values that serde errors echo, a refresh token’s jti, the JSON-RPC client id, KMS error bodies, URL userinfo, token query parameters and 24-word recovery phrases. Public keys, context IDs and peer IDs are kept, because diagnosis needs them.
• Caps each message at 8 KiB.

A line the program fails on is dropped, never shipped unguarded. configure_vector.sh refuses a config without the guard, and the locked-read-only conformance role refuses to build one. scripts/ci/tests/log-egress-guard-test.sh runs the guard with the pinned Vector release.

This is a backstop. Services still must not log secrets.

The observability credential comes only from MDMA

observability-token is the only way a node gets the token its shippers present, and the provided provider simply reads it from disk. There is no Secret Manager path: MDMA creates node instances with no serviceAccounts field at all, so a node has no cloud identity to read a secret with. The logs-secret-name and metrics-secret-name keys that used to select that path are no longer read.

Instance metadata is set at creation only

It is never backfilled. A node created before any of these keys existed will never pick them up and must be recreated. The one exception is the observability credential, which MDMA also delivers on the should-join poll — that is how rotation reaches a node already running.

The calimero-init boot sequence

calimero-init.service is a oneshot unit that runs /usr/local/lib/calimero/init.sh after network-online.target; merod, mero-auth, traefik, and (in read-only profiles) fleet-sidecar all order themselves After=calimero-init.service, and merod, mero-auth and fleet-sidecar also Require it, so none of them writes to /mnt/data if the data disk could not be unlocked. Its log is /run/calimero/calimero-init.log (RAM) and the journal; the journal is volatile, so nothing it logs persists on the boot disk. The script (ansible/roles/merotee/templates/calimero-init.sh.j2) runs, in order:

1. Extend RTMR3 for attestation. It reads the dm-verity root hash (roothash=) from /proc/cmdline, computes an SHA-384 digest of calimero-rtmr3-v3:<role>:<profile>:<roothash>, and writes the 48-byte digest into the TDX guest sysfs (/sys/class/misc/tdx_guest/measurements/rtmr3:sha384, falling back to .../mr/rtmr3:sha384). Every block of the root filesystem is checked against that hash, so RTMR3 names the image’s whole root, its role and its profile. On locked-read-only a missing roothash= or RTMR3 interface stops the boot; the debug profiles log a warning and go on. (Releases up to 2.3.118 extended calimero-rtmr3-v2:... with calimero.root_hash, a build-time hash of a few directories.)

2. Disable observability by default (stops/disables vmagent and vector), then reads the metadata whitelist above.

3. Resolve binaries. If merod/meroctl/mero-auth are baked in /usr/local/bin (they are, for images built here), it uses them; otherwise, on the debug profiles only, it downloads them from the core release matching merod-version metadata. locked-read-only stops instead: a downloaded binary is in no measurement.

4. Apply tee-release-version. If set, it refuses a release older than the image’s own version (/etc/calimero/min-tee-release-version, baked at build), then writes MERO_TEE_VERSION, MERO_TEE_MIN_VERSION and MERO_TEE_PROFILE (the image’s profile, which selects the release’s per-profile KMS policy) to /etc/calimero/merod.env, which the merod unit loads via EnvironmentFile, and exports them for the steps below.

5. Unlock the data disk. Swap must be off (it is refused on locked-read-only). Then, for /dev/disk/by-id/google-data, the link GCE’s disk-naming udev rule makes for the disk attached as data (the image ships the rule and its google_nvme_id helper, vendored from GoogleCloudPlatform/guest-configs; if the link is still missing after 10 s, the one whole disk with nothing on it is used, and the journal says so):

   • blank (blkid -p finds no signature): fetch a key with merod kms disk-key --create-identity into a dedicated tmpfs, cryptsetup luksFormat LUKS2 (aes-xts-plain64, --integrity hmac-sha256), store the disk-unlock identity in a calimero-kms-identity LUKS2 token, open it as /dev/mapper/calimero-data, create ext4 on the mapper and mount it at /mnt/data. The integrity wipe covers the whole disk, so the first boot is slow (about 17 minutes for 200 GB pd-balanced); the journal logs Integrity wipe: N% (… of … GB, …m…s left) every 30 s while it runs, and mdma’s MDMA_TEE_POST_BOOT_VERIFY_TIMEOUT_SECONDS must outlast it. No fstab entry is written.
   • LUKS: read the identity from the token, fetch its key, open, mount.
   • plain ext4 (a node from before disk encryption, or a debug node with no KMS): mounted as before, with a loud warning. Never reformatted.
   • anything else, a failed probe, a LUKS header with no token, or an opened volume without ext4: the boot stops. Nothing is ever formatted.

   The key is shredded as soon as the disk is open. locked-read-only refuses a blank disk without kms-url + tee-release-version, or on a merod without kms disk-key. With ephemeral-store=true, a tmpfs is mounted over /mnt/data instead and the disk is untouched.

6. Initialize the node at /mnt/data/calimero (node default) if not already initialized, with --advertise-address and the resolved ports. In read-only mode it passes --mode read-only.

7. Apply read-only config. In read-only mode it runs merod config to set server.jsonrpc.enabled=false, server.sse.enabled=false, and server.websocket.enabled=false — the client-facing write/subscribe surfaces are turned off, leaving only the read/admin path. A relay (a read-only profile built with fleet_delegated_access, the default for both read-only profiles) keeps server.sse.enabled=true: delegated clients subscribe there, behind auth-node, scoped to their own contexts. In full (debug) mode these remain enabled.

8. Start services. It daemon-reloads, enables and non-blocking-starts merod, mero-auth, and traefik; in read-only mode it also starts fleet-sidecar when that unit exists.

Caution

The three servers disabled in read-only mode (jsonrpc, sse, websocket) are exactly the client mutation/subscription interfaces. Do not expect to drive a locked-read-only or debug-read-only node through those APIs — it is a replication/read node. The read-only fleet node instead converges via the fleet HA sidecar; see the fleet sidecar.

Verify the booted node

1. Confirm it reached RUNNING and answers. After the VM is RUNNING, the admin API is reachable on its external IP at the admin_api_port (80 in the CI probes). The node-image-gcp-staging-probe.yaml workflow automates exactly this: it boots a TDX VM from a named image, waits for readiness, collects a TDX quote, and verifies it against Intel Trust Authority. Run it via Actions → Node image GCP staging probe with profile, image_name, and image_project (see the probe entry in the runbooks).

2. Attest the measurements. Confirm the node’s MRTD and RTMR values (the RTMR3 extend from step 1 of the boot sequence, plus calimero.role=node / calimero.profile measured into RTMR[2]) match the published, signed set. See Verification for the operator verify procedure and how to verify a release for the release-asset side.

Next steps

• Runbooks — point the booted node at the KMS and roll versions.
• Configuration reference — every runtime knob.
• Verification — MRTD/RTMR, root hash, operator verify.
• Verify a release — trust the release assets before you deploy them.