Skip to content

Deploy the node image

How to build a GCP TDX merod node image with this repository and boot it as an Intel TDX Confidential VM. This is the layer underneath the runbooks: the runbooks point an already-running merod at the KMS; this page gets a node image built, launched, and healthy in the first place. Everything below is grounded in the Packer/Ansible sources under mero-tee/ and the two GCP workflows — confirm exact flags against --help and your checked-out version before running against production.

A node image is an Ubuntu 26.04 LTS (Resolute Raccoon) GCP image produced by mero-tee/ubuntu.pkr.hcl + mero-tee/playbook.yml. The base is pinned at ubuntu-2604-lts-amd64 (from ubuntu-os-cloud): RTMR3 sysfs support needs kernel 6.17+, and an LTS is used because Canonical delists an EOL release from that project, which fails the build — the previous pin on the interim 25.10 did exactly that when it reached EOL. Component versions are pinned in mero-tee/versions.json:

Component Version key Value at time of writing
Node image imageVersion 2.3.102
merod / meroctl / mero-auth (core tag) merodVersion 0.11.0-rc.67
Traefik traefikVersion 3.5.0
node-exporter nodeExporterVersion 1.9.1
vmagent vmagentVersion 1.132.0
Vector vectorVersion 0.50.0

The playbook.yml applies these roles in order: mero-traefik, calimero-core (bakes merod/meroctl/mero-auth at the core tag so calimero-init skips the runtime download), merotee (the init script, systemd units, and — for read-only profiles — the fleet sidecar), node-exporter, vmagent, vector, cleanup, and — only for locked-read-only — merod-lockdown followed by merotee-conformance.

The lockdown_profile Packer variable selects one of three profiles (mero-tee/ubuntu.pkr.hcl validates against exactly this set):

Profile Intended use merod mode Lockdown role
debug Local / dev full not applied
debug-read-only Integration / pre-production read-only not applied
locked-read-only Production read-only applied + conformance

merod_mode is derived in ansible/roles/merotee/defaults/main.yml: it is read-only for debug-read-only and locked-read-only, and full for debug. In read-only mode the boot script initializes merod with --mode read-only and disables the client-facing servers (see the boot sequence below).

The merod-lockdown role (ansible/roles/merod-lockdown/tasks/main.yml) hardens the production image so its measurements differ from the debug profiles. It:

  • R1 removes openssh-server/openssh-client and deletes /etc/ssh;
  • R2 masks all getty@/serial-getty@ttyS0/console-getty/getty.target units and strips console= from the kernel cmdline;
  • R3 masks emergency/rescue/debug-shell (and overrides emergency/rescue to reboot -f);
  • R4 locks the root account, empties /etc/securetty, and removes the ubuntu user (done last, since Packer SSHes in as ubuntu);
  • R5 masks google-startup-scripts and the cloud-init* units so no GCP metadata script is ever executed — metadata is only read as data by the init script’s whitelist.
  • Packer and Ansible — the build driver and provisioner.
  • GCP credentials with rights to create an image in the target project.
  • The Packer plugins pinned in ubuntu.pkr.hcl: googlecompute 1.2.1 and ansible 1.1.3 (installed by packer init).
  1. Install the pinned Packer plugins:

    Terminal window
    cd mero-tee
    ./init.sh # runs: packer init ubuntu.pkr.hcl
  2. Build. For a single profile with all versions read from versions.json, use the wrapper — it resolves every pinned version and passes the Packer vars for you:

    Terminal window
    ./build-and-release.sh locked-read-only

    With no profile argument it builds all three profiles (locked-read-only, debug-read-only, debug) in turn. To drive Packer directly instead:

    Terminal window
    packer build -var-file=ubuntu-x86.pkrvars.hcl ubuntu.pkr.hcl

The build runs on a widely-available n2-standard-2 host (instance_type in ubuntu-x86.pkrvars.hcl) — no TDX is needed to build the image, only to run it. The build host defaults are project_id calimero-p2p-development, region europe-west4, zone europe-west4-a, overridable via PACKER_GCP_PROJECT_ID / PACKER_GCP_REGION / PACKER_GCP_ZONE / PACKER_GCP_SUBNETWORK. The base source image is hardcoded for release reproducibility and cannot be overridden.

Packer publishes each image into the GCP project with a deterministic name and family (from ubuntu.pkr.hcl):

  • Image name: merotee-ubuntu-questing-25-10-<profile>-<version-with-dashes> (e.g. merotee-ubuntu-questing-25-10-locked-read-only-2-3-51).
  • Image family: merotee-ubuntu-questing-<profile>.
  • Disk: 20 GB pd-ssd.

The questing-25-10 in those names is a frozen identifier, not the base release — the base is 26.04 LTS. mdma’s dispatcher matches on these prefixes, so they do not move when the base does.

The release-node-image-gcp.yaml workflow builds the images in CI and records the published project / family in its release provenance. Its VM-facing defaults (used for the post-build attestation probe) match Calimero Cloud MDMA: project cloud-486420, zone europe-west4-a, machine type c3-standard-4.

Boot the image as an Intel TDX Confidential VM. The flags below are exactly those the release-node-image-gcp.yaml and node-image-gcp-staging-probe.yaml workflows use to create their ephemeral VMs:

Terminal window
gcloud compute instances create <instance-name> \
--project <vm-project> \
--zone europe-west4-a \
--machine-type c3-standard-4 \
--confidential-compute-type TDX \
--maintenance-policy TERMINATE \
--min-cpu-platform "Intel Sapphire Rapids" \
--no-service-account \
--no-scopes \
--image <image-name> \
--image-project <image-project> \
--subnet <subnetwork> \
--metadata merod-version=<merod-version> \
--tags merotee

calimero-init reads a fixed whitelist of instance metadata attributes (ansible/roles/merotee/templates/calimero-init.sh.j2) — never arbitrary scripts. All are optional except where noted:

Metadata key Effect Default
server-port merod admin/server port (1–65535; anything else falls back to the default) 2428
swarm-port merod libp2p swarm port (1–65535; anything else falls back to the default) 2528
tee-release-version written as MERO_TEE_VERSION into /etc/calimero/merod.env (sanitized/escaped), with MERO_TEE_MIN_VERSION (the image’s own version) and MERO_TEE_PROFILE (the image’s profile), so merod verifies the KMS against that profile’s signed policy. Refused if older than the image (downgrade protection) unset → env file removed
kms-url the VPC-internal URL of the release’s KMS cluster, which the data-disk key and the store key come from; verified against tee-release-version’s signed policy unset → locked-read-only refuses to create a node; debug profiles create an unencrypted one
metrics-endpoint if set, configures and starts vmagent and node_exporter unset → vmagent stays off
logs-endpoint if set, configures and starts vector unset → vector stays off
observability-token the bearer token both shippers present; written to /etc/vector/provided_token and used via the provided secret provider unset → the node falls back to the credential MDMA delivers on a should-join poll
merod-version only used to download binaries at boot if they were not baked into the image — debug profiles only; locked-read-only refuses to boot rather than run an unmeasured binary required only when unbaked

Observability is off by default (R7): vmagent and vector are stopped and disabled at boot and started only when the matching *-endpoint metadata is present. node_exporter is started alongside vmagent, because it is the only thing the default scrape config targets — its unit file is installed by the image but is not enabled, so without that it would be a dead scrape target.

calimero-init.service is a oneshot unit that runs /usr/local/lib/calimero/init.sh after network-online.target; merod, mero-auth, traefik, and (in read-only profiles) fleet-sidecar all order themselves After=calimero-init.service, and merod, mero-auth and fleet-sidecar also Require it, so none of them writes to /mnt/data if the data disk could not be unlocked. Its log is /run/calimero/calimero-init.log (RAM) and the journal; the journal is volatile, so nothing it logs persists on the boot disk. The script (ansible/roles/merotee/templates/calimero-init.sh.j2) runs, in order:

  1. Extend RTMR3 for attestation. It reads calimero.root_hash from /proc/cmdline, computes an SHA-384 digest of calimero-rtmr3-v2:<role>:<profile>:<root_hash>, and writes the 48-byte digest into the TDX guest sysfs (/sys/class/misc/tdx_guest/measurements/rtmr3:sha384, falling back to .../mr/rtmr3:sha384). This makes each image’s runtime measurement unique and unforgeable. It is best-effort: if the sysfs node is unavailable the node still boots, logging a warning.

  2. Disable observability by default (stops/disables vmagent and vector), then reads the metadata whitelist above.

  3. Resolve binaries. If merod/meroctl/mero-auth are baked in /usr/local/bin (they are, for images built here), it uses them; otherwise, on the debug profiles only, it downloads them from the core release matching merod-version metadata. locked-read-only stops instead: a downloaded binary is in no measurement.

  4. Apply tee-release-version. If set, it refuses a release older than the image’s own version (/etc/calimero/min-tee-release-version, baked at build), then writes MERO_TEE_VERSION, MERO_TEE_MIN_VERSION and MERO_TEE_PROFILE (the image’s profile, which selects the release’s per-profile KMS policy) to /etc/calimero/merod.env, which the merod unit loads via EnvironmentFile, and exports them for the steps below.

  5. Unlock the data disk. Swap must be off (it is refused on locked-read-only). Then, for /dev/disk/by-id/google-data, the link GCE’s disk-naming udev rule makes for the disk attached as data (the image ships the rule and its google_nvme_id helper, vendored from GoogleCloudPlatform/guest-configs; if the link is still missing after 10 s, the one whole disk with nothing on it is used, and the journal says so):

    • blank (blkid -p finds no signature): fetch a key with merod kms disk-key --create-identity into a dedicated tmpfs, cryptsetup luksFormat LUKS2 (aes-xts-plain64, --integrity hmac-sha256), store the disk-unlock identity in a calimero-kms-identity LUKS2 token, open it as /dev/mapper/calimero-data, create ext4 on the mapper and mount it at /mnt/data. The integrity wipe covers the whole disk, so the first boot is slow (about 17 minutes for 200 GB pd-balanced); the journal logs Integrity wipe: N% (… of … GB, …m…s left) every 30 s while it runs, and mdma’s MDMA_TEE_POST_BOOT_VERIFY_TIMEOUT_SECONDS must outlast it. No fstab entry is written.
    • LUKS: read the identity from the token, fetch its key, open, mount.
    • plain ext4 (a node from before disk encryption, or a debug node with no KMS): mounted as before, with a loud warning. Never reformatted.
    • anything else, a failed probe, a LUKS header with no token, or an opened volume without ext4: the boot stops. Nothing is ever formatted.

    The key is shredded as soon as the disk is open. locked-read-only refuses a blank disk without kms-url + tee-release-version, or on a merod without kms disk-key. With ephemeral-store=true, a tmpfs is mounted over /mnt/data instead and the disk is untouched.

  6. Initialize the node at /mnt/data/calimero (node default) if not already initialized, with --advertise-address and the resolved ports. In read-only mode it passes --mode read-only.

  7. Apply read-only config. In read-only mode it runs merod config to set server.jsonrpc.enabled=false, server.sse.enabled=false, and server.websocket.enabled=false — the client-facing write/subscribe surfaces are turned off, leaving only the read/admin path. A relay (a read-only profile built with fleet_delegated_access, the default for both read-only profiles) keeps server.sse.enabled=true: delegated clients subscribe there, behind auth-node, scoped to their own contexts. In full (debug) mode these remain enabled.

  8. Start services. It daemon-reloads, enables and non-blocking-starts merod, mero-auth, and traefik; in read-only mode it also starts fleet-sidecar when that unit exists.

  1. Confirm it reached RUNNING and answers. After the VM is RUNNING, the admin API is reachable on its external IP at the admin_api_port (80 in the CI probes). The node-image-gcp-staging-probe.yaml workflow automates exactly this: it boots a TDX VM from a named image, waits for readiness, collects a TDX quote, and verifies it against Intel Trust Authority. Run it via Actions → Node image GCP staging probe with profile, image_name, and image_project (see the probe entry in the runbooks).

  2. Attest the measurements. Confirm the node’s MRTD and RTMR values (the RTMR3 extend from step 1 of the boot sequence, plus calimero.role=node / calimero.profile measured into RTMR[2]) match the published, signed set. See Verification for the operator verify procedure and how to verify a release for the release-asset side.