State, Projection & the Root Hash

What projection is

You never query the DAG directly. You query state — the current value of every entity, who may write it, who belongs to the scope. Projection is the act of replaying the operation log to compute that state, plus a single hash that fingerprints it.

A bank balance is the projection of its transaction history. The balance isn’t a fact you trust on its own — it’s recomputed from the list of deposits and withdrawals, and anyone holding the same transactions arrives at the same balance. A Calimero scope works the same way: the DAG of operations is the ledger, the state is what you get by folding that ledger down, and the scope root is a 32-byte fingerprint of the result.

▶ Replay

Everything below unpacks three claims:

• the fold is order-independent — replay the same operations in any order and you land on the same state and the same root;
• the root is a total fingerprint — it covers data and authorization, so no change can move state without moving the root;
• comparing two roots is therefore enough to tell whether two nodes agree.

A worked example: three ops to one root

Start concrete. Take three data-plane operations in one scope. Each entity id is derived from its key, so it is identical on every node (hashes are abbreviated and illustrative):

op1:  Put    entity=0x9a…m1  value="hi"   hlc=1750.000100:#0
op2:  Put    entity=0xb3…m2  value="yo"   hlc=1750.000120:#0
op3:  Delete entity=0x9a…m1               hlc=1750.000150:#0

Step 1 — fold the ops into slots. The projection treats each entity as a slot and keeps, per slot, the stamp of the last operation to write it. op1 and op3 both touch slot 0x9a…m1; op3’s stamp (…150) outranks op1’s (…100), so the delete wins. A late-arriving op1 can’t resurrect the entity — its lower stamp loses.

Slot	Resolved value	Why
0x9a…m1	absent	op3 Delete stamp …150 > op1 Put stamp …100
0xb3…m2	"yo"	only one writer

Step 2 — hash the resolved state. Only m2 survives, so the data hash is taken over that one live entity, in ascending-id order. The two authorization planes are empty here, so each hashes to the empty-input digest:

entities_hash = SHA-256( 0xb3…m2 ‖ u64_le(2) ‖ "yo" )       = 0x1f44…ab
acl_hash      = SHA-256( <no objects> )                      = 0xe3b0…55   (empty plane)
groups_root   = SHA-256( <no governance> )                   = 0xe3b0…55   (empty plane)

scope_root = SHA-256( entities_hash ‖ acl_hash ‖ groups_root ) = 0x8d27…f0

Why two nodes get the same hash. A node that received op3, op1, op2 and a node that received op1, op2, op3 walk different paths, yet they resolve the same slots to the same bytes — because each slot keeps only its highest-stamped write, and a stamp doesn’t depend on arrival order. Same set of ops ⇒ same slots ⇒ same scope_root = 0x8d27…f0. The two nodes are converged, and neither ever had to agree on an order.

The fold, formally

The example’s “highest stamp wins” rule is the whole engine. Formally, the projection is last-writer-wins per slot: every addressable piece of state — an entity, an object’s writer set, a member’s role — is a slot, and applying an operation overwrites a slot only if it carries a higher stamp.

// applying one operation — pure, deterministic, order-independent
fn apply(op):
  for (slot, value) in op.writes():
    stamp = (op.hlc, generation(op), op.id)
    if stamp > clock[slot]:
      state[slot] = value
      clock[slot] = stamp

The stamp is a triple, compared left to right:

• hlc — the hybrid logical clock. It tracks wall-clock time but is nudged forward whenever a higher clock is observed, so it stays causally monotonic: an operation that causally follows another always carries a higher (or equal) hlc.
• generation — the operation’s causal depth in the DAG (1 + max(parent generation), 0 at a root). This breaks ties where hlc alone can’t. Governance operations are authored with hlc = 0, so generation is what orders them — a later structural change always sits deeper in the DAG than the state it supersedes.
• id — the content hash, a final deterministic tiebreaker so two genuinely concurrent writes still resolve the same way on every node.

A Delete records a stamp exactly like a write — which is why the stale re-add in the example above could never win.

This is also where order-independence comes from: because a slot only ever remembers the single highest stamp it has seen, the order operations arrive in can’t change the outcome. Replay the same set in any order — authored, gossiped, or backfilled — and every slot lands on the same value.

What the projection holds

The fold maintains one set of slots per plane. Together they are the complete materialized state of a scope:

Plane	Slots	Resolved value
Data	entities	the latest value written to each entity (or absent, if deleted)
Access control	acl	the writer set and capability mask for each object
Membership	groups	each group’s members and their roles
Admin	root_admin, policy	the scope’s admin identity and policy bytes
Structure	subgroups	the subgroup tree: parent links and visibility
Capability	default_caps, member_caps	default and per-member capability grants

The data plane is the application’s state; the other planes are the governance planes. They all fold the same way — that is the unification.

The scope root

The whole projection collapses to one 32-byte hash: the concatenation of three plane hashes, each taken over its slots in sorted, deterministic order.

scope_root = SHA-256( entities_root ‖ acl_hash ‖ groups_root )

// each component is a hash over its plane's slots in sorted key order
entities_root = the data-plane root (on the wire, the storage Merkle root — see below)
acl_hash      = H( for each object:  id ‖ writer ‖ capability_mask )
groups_root   = H( memberships ‖ admin ‖ policy ‖ subgroups ‖ caps )

Because every node sorts the same way and hashes the same bytes, equal operation sets give equal roots. The exact byte layout — and the subtlety behind entities_root — is in the specification below.

Why it matters: detecting divergence

Order-independence plus a total fingerprint is what turns “are these two nodes in sync?” into a single comparison.

The root covers authorization, not just data

Equal sets give equal roots — and unequal state gives unequal roots, including state with no data effect at all.

Key property

A change with no data effect still moves the scope root.

Rotate an object’s writers, or remove a member, and the data entities may be byte-for-byte identical — yet because acl_hash and groups_root are folded into the root, the scope root changes. Returning to the worked example: add a fourth op that only rotates a writer set, and the data slots stay identical while acl_hash (and so scope_root) moves. This closes the gap that would otherwise let a silent access-control or membership split-brain hide behind matching application data. Convergence is checked on the whole scope, not just its data.

Convergence is a hash comparison

With an order-independent fold and a single total root, checking whether two nodes agree is trivial: compare scope roots. No agreement on ordering, no quorum, no coordination — just two hashes.

▶ Replay

Equal roots mean the nodes hold the same state and are done. Unequal roots mean one is missing operations the other has — which is exactly the signal the sync engine acts on.

Reading the past: authorization at a cut

The fold above produces the current state. But authorization often needs the state as it was at a particular point in history — specifically, the membership and access-control state an operation’s author had observed when they wrote it.

The projection answers this by folding only the causal cut: the operation’s parents and everything those parents transitively descend from, and nothing later. This yields an access-control view “as of” that point.

Why it matters

Forward-only authorization.

A write authored before a member was revoked is judged against the access-control state that existed before the revocation — so it stays valid regardless of the order in which a peer happens to receive the write and the revocation. Authority is evaluated at the author’s cut, never at the receiver’s present. Governance builds the full membership and capability rules on top of this primitive.

Specification

The exact composition of the scope root. Every node must hash identical bytes in identical order, so the rules below are normative for interop. All integers are little-endian; every map is iterated in ascending key order (the determinism source).

scope_root = SHA-256( entities_root ‖ acl_hash ‖ groups_root )  // three 32-byte inputs

Important

Two entity roots — pick the right one for the wire.

The entities_root input has two sources. The projection can hash its own entity map (a self-contained root, used in tests and local checks). But the convergence signal exchanged on the wire folds the storage layer’s Merkle root as entities_root, with this projection’s acl_hash and governance_hash on top. The storage Merkle root is authoritative for the data plane; the projection never re-hashes entity state for the wire signal. A second implementation must fold its storage root here, not a re-hash of the entity map.

entities_hash (projection-local data plane)

// for each (id, value) in entities, ascending by id
h.update( id )       // 32 bytes
h.update( u64_le(len(value)) )
h.update( value )

acl_hash (access-control plane)

// for each (object_id, writers) ascending; writers ascending by key
h.update( object_id )  // 32 bytes
for (writer, mask) in writers:
  h.update( writer )  // 32 bytes
  h.update( [mask.bits()] )  // 1 byte bitflag

groups_root (membership + admin + policy + structure + caps)

The third component. The implementation names the function that computes it governance_hash, because it spans the scope’s admin, policy, and subgroups in addition to group memberships and capabilities — all of it folded in below.

// 1) groups — ascending; empty groups are skipped entirely
for (group, members) in groups, if members not empty:
  h.update( group )
  for (member, role) in members: h.update( member ‖ [role_byte] )
// 2) root admin
h.update( admin ? [1] ‖ admin : [0] )
// 3) policy
h.update( u64_le(len(policy)) ‖ policy )
// 4) live subgroups — ascending; only those whose latest exists=true
for (child, slot) in subgroups, if slot.live:
  h.update( child ‖ [parent if set] ‖ [restricted_byte] )
// 5) capability plane — each map ascending
for (group, caps) in default_caps: h.update( group ‖ u32_le(caps) )
for ((group,member), caps) in member_caps: h.update( group ‖ member ‖ u32_le(caps) )
for (group, admin) in group_admin: h.update( group ‖ admin )

Stable role encoding

Roles hash to an explicit byte — fixed independently of any in-memory enum order, so the root survives refactors:

Role	role_byte
Admin	0
Member	1
ReadOnly	2
ReadOnlyTee	3

Note the empty-group skip and the live-subgroup filter: materialized state must hash identically no matter how it was reached, so a phantom empty group or a deleted subgroup must never perturb the root.

Where this leads

A signed operation, a causally-ordered DAG, an order-independent fold to state, and a single scope root that makes convergence a hash comparison — that is the core of the protocol. Everything else applies this machinery to a purpose.

Next is Anatomy of State — a visual tour of how the DAG, the Merkle tree, and the on-disk column families actually lay this state out. After that, Execution shows where data operations come from (a running WASM application), and Governance shows how the authorization planes encode membership and authority.