Modeling your state
Your #[app::state] struct is the whole data model of your
app. Unlike a traditional database schema, every field is a CRDT: concurrent
writes on different nodes merge deterministically with no coordination, and all
replicas converge. This page is about the design shift that follows from that —
how to pick collection types for the behavior you want, and the patterns that keep
a convergent app correct as it grows.
The mental shift
Section titled “The mental shift”A traditional DB gives you a single authoritative copy, transactions, and read-your-write consistency. Calimero gives you none of those by default. Design around four facts:
- There is no global lock. Two members can write the same logical thing at the same time on different nodes. The collection’s merge rule — not your code — decides the outcome. So choosing the collection is choosing the conflict policy.
- Writes are eventually consistent. A write on node A is not visible on node B until they sync. Do not design flows that depend on reading back what another node just wrote.
- Anything any member can write, every member can write. Plain state is group-public. If a field must be owner-only or per-user, you model that with an access-controlled wrapper — see Access-controlled storage.
- Merge replays your writes deterministically. The same logical operation must produce the same result on every node. That rules out clocks and randomness inside state mutations (more below).
Choose the CRDT type for your intent
Section titled “Choose the CRDT type for your intent”The Collections page is the full reference. The design question is what should concurrent writes do:
| Intent | Reach for | Concurrent-write behavior |
|---|---|---|
| Latest write should win | LwwRegister<T> | Newest HLC timestamp wins (ties broken by node id) |
| Accumulate a count | Counter / GCounter / PNCounter | Per-executor increments are summed — none are lost |
| Membership where adds must survive | UnorderedSet<V> / SortedSet<V> | Add-wins union: every concurrent add survives |
| Key→value store | UnorderedMap<K, V> / SortedMap<K, V> | Add-wins keys; shared keys merge their values recursively |
| Ordered, append + index | Vector<V> | Append-only; each push mints a stable element id |
| Collaborative text | ReplicatedGrowableArray | Per-character RGA ordering; concurrent edits interleave |
The recursive-merge property is the lever you compose with: a
UnorderedMap<String, LwwRegister<String>> is “a map where each key is its own
last-writer-wins cell” — the pattern apps/kv-store uses. A
UnorderedMap<String, Counter> is “a count per key.” Build the behavior you want by
nesting.
Synced vs node-local state
Section titled “Synced vs node-local state”Everything above is synced state: it lives in your #[app::state] struct, is
replicated to every member, counts toward the Merkle root, and is multi-writer — so
it must be a CRDT. That is the default, and almost all of your data belongs here.
Calimero also has a second, deliberately separate plane: node-local private state,
declared with #[app::private]. It lives in its own storage column (PrivateState),
is never synchronized, sits outside the Merkle root, and has exactly one
writer — this node. Because nothing about it crosses the wire, two nodes running the
same app will hold different private state, and that per-node divergence is the whole
point, not a bug.
Synced state (#[app::state]) | Node-local state (#[app::private]) | |
|---|---|---|
| Replicated to other nodes | Yes — in the Merkle root | No — outside it, never on the wire |
| Writers | Every member (multi-writer) | This node only (single-writer) |
| Field types | CRDTs (merge rule required) | Plain values: u64, String, Vec, BTreeMap, and the structural UnorderedMap / UnorderedSet / Vector (auto-pointed at private storage) |
| Convergence | All replicas converge | Intentionally per-node — no convergence |
Because a private namespace has a single writer, CRDT and access-control machinery is
meaningless there — so #[app::private] rejects those types at compile time rather
than silently mis-storing them. A LwwRegister, Counter, SharedStorage,
UserStorage, FrozenStorage, AuthoredMap, and friends inside a private struct each
produce a targeted compile error pointing at the plain type to use instead (a
LwwRegister<u64> becomes a bare u64; a Counter becomes a plain integer).
Reach for #[app::private] when data is meant to stay on one node:
- Secrets — a plaintext value where only its hash should ever sync. The
apps/private_datagame keeps each guess’s secret in private state and publishes onlysha256(secret)to the shared map. - A per-node cache — a derived lookup you can rebuild locally and don’t want to pay to replicate or merge.
- A derived index or local bookkeeping — per-node counters, “have I attempted this?” sets, an append-only local log — state that describes this node’s view, not the group’s.
The how — the typed private_load / private_load_or_default accessors and the
save-on-drop write pattern — lives on
Access-controlled storage,
next to the other write-boundary tools.
A worked domain model: a marketplace
Section titled “A worked domain model: a marketplace”The patterns below are easier to see in one cohesive schema. Here is a small marketplace, and the modeling decision behind each field:
#[app::state]#[derive(Default, BorshSerialize, BorshDeserialize)]struct Marketplace { // Primary store: listing id -> listing. Every access is a point lookup by // id and there is no global ordering need, so UnorderedMap (not SortedMap). listings: UnorderedMap<String, Listing>,
// Secondary index: (seller id ++ big-endian seq) -> listing id. There are no // DB indexes here, so "this seller's listings, newest first" is a SortedMap // you maintain yourself. One index for all sellers; `prefix(seller)` slices // one seller's listings in order without scanning the whole marketplace. by_seller: SortedMap<Vec<u8>, LwwRegister<String>>,
// Per-buyer slot: each buyer writes only their own shipping profile, and // that owner-only rule is enforced at merge, not just locally. buyers: UserStorage<LwwRegister<String>>,}
#[derive(Default, app::Mergeable, BorshSerialize, BorshDeserialize)]struct Listing { title: LwwRegister<String>, price: LwwRegister<u64>, sold: LwwRegister<bool>, // a flag, never a `remove` — see below}Publishing a listing writes the primary entry and its index entry together; both maps are convergent, so the index converges too:
listings.insert(id.clone(), listing)?;let mut key = seller_id.as_bytes().to_vec();key.extend_from_slice(&seq.to_be_bytes()); // big-endian: keep the index time-orderedby_seller.insert(key, id.into())?;
// One seller's listings, newest-first, without touching anyone else's:let theirs = by_seller.prefix(seller_id.as_bytes())?;
// A buyer sets their own shipping address; no other member can overwrite it:buyers.insert(address.into())?;The decisions worth calling out:
soldis a flag, not aremove.listingsis add-wins, so a hardremoveloses to a concurrent edit and the listing resurrects. Modeling “sold” as anLwwRegister<bool>resolves edit-vs-sold by HLC instead — the tombstone pattern below.- The index is a hint. After concurrent edits, an index entry can outlive its
target. Treat a
by_sellerhit as a pointer and confirm againstlistingson read — the secondary-index caveat. - Composite key over nesting. A single
SortedMapwithseller ++ seqkeys, sliced byprefix, is simpler than aUnorderedMap<seller, SortedMap<...>>and needs only one index collection to keep in sync.
The rest of this section unpacks each of these patterns in isolation.
Patterns & recipes
Section titled “Patterns & recipes”Ordered / paginated data with SortedMap
Section titled “Ordered / paginated data with SortedMap”SortedMap / SortedSet are the BTreeMap/BTreeSet to the unordered types’
HashMap/HashSet: same add-wins CRDT, plus a node-local ordered index that unlocks
range(a..b), prefix(p), page(offset, limit), first(), and last().
The catch: ordering is byte order over the key. The index seeks keys by their
raw bytes, so your Ord must match as_ref() byte order. String / &str /
Vec<u8> sort lexicographically and just work. Multi-byte integers must be stored
big-endian, or 255 sorts before 256:
// Time-ordered log: big-endian key so range/pagination scan in real order.let mut log: SortedMap<[u8; 8], LwwRegister<String>> = SortedMap::new();log.insert(seq.to_be_bytes(), entry.into())?; // big-endian!let first_page = log.page(0, 20)?; // ascending by keylet window = log.range(start.to_be_bytes()..end.to_be_bytes())?;The index is not synchronized — each node rebuilds its own after sync — so you
pay a little write overhead but ordering is always correct post-merge. See
apps/sorted-kv-store.
Per-user data with UserStorage
Section titled “Per-user data with UserStorage”When each member owns exactly one slot — a profile, a preference, a per-user
cursor — UserStorage<T> keys by identity automatically. Each executor writes only
its own slot; reads are open:
let mut profiles: UserStorage<LwwRegister<String>> = UserStorage::new();profiles.insert("Alice".into())?; // writes env::executor_id()'s slotlet mine = profiles.get()?; // my slotlet theirs = profiles.get_for_user(&bob_key)?; // anyone's slot, read-onlyA member cannot overwrite another member’s slot — that rule is enforced at merge, not
just locally. See apps/kv-store-with-user-and-frozen-storage.
An open keyspace with per-entry owners via AuthoredMap
Section titled “An open keyspace with per-entry owners via AuthoredMap”When anyone may add entries but only the author may edit theirs — a shared
catalog, a comment thread, a classifieds board — use AuthoredMap<K, V>. Insert is
open and stamps the inserter as owner; update / remove are owner-only:
let mut listings: AuthoredMap<String, LwwRegister<String>> = AuthoredMap::new();listings.insert("listing-42".into(), body.into())?; // open; caller becomes ownerlistings.update(&"listing-42".into(), edit.into())?; // only the owner succeedslet owner = listings.owner_of(&"listing-42".into())?;AuthoredVector<V> is the same idea for an append-only log: any member pushes, only
the author updates or tombstones their slot.
Soft-delete via tombstones
Section titled “Soft-delete via tombstones”Add-wins collections make “remove” subtle: if node A removes a key while node B concurrently re-adds (or edits) it, add wins and the entry comes back. For data that must stay deletable, model deletion as state rather than absence:
struct Note { body: LwwRegister<String>, deleted: LwwRegister<bool>, // tombstone flag, last-writer-wins}A concurrent edit-vs-delete now resolves by HLC instead of silently resurrecting.
AuthoredVector::tombstone follows this model (it replaces the slot with
V::default() rather than physically removing it, so merges stay sound). Reserve
hard remove for cases where a concurrent re-add genuinely is fine.
Per-user counters
Section titled “Per-user counters”Compose a map of counters when you need “a count per member” that still merges correctly under concurrent increments:
let mut points: UnorderedMap<String, Counter> = UnorderedMap::new();points.entry("Alice".into())?.or_default().increment()?;Each Counter already sums per-executor increments, so two nodes bumping the same
user’s count both land.
A secondary index you maintain in a second map
Section titled “A secondary index you maintain in a second map”There are no DB indexes here — if you need lookups by a non-primary attribute, keep a second map and write both. The shapes are convergent, so the index converges too; just update both on every write:
// Primary: id -> user. Secondary: email -> id.users.insert(id.clone(), user)?;by_email.insert(email, id.into())?;Because both maps are add-wins, a stale index entry can outlive its target after concurrent edits — treat index hits as hints and confirm against the primary map on read.
Anti-patterns
Section titled “Anti-patterns”Huge collections under one parent
Section titled “Huge collections under one parent”A single collection with very many direct children gets expensive: operations and
merges touch the parent’s child set, and that cost grows with the number of children.
Prefer sharding (e.g. a map keyed by a prefix bucket, or a SortedMap you page
through) over one flat collection of hundreds of thousands of entries. See
Storage performance & Big-O for the exact costs and
how each collection scales.
time_now / randomness in state mutations
Section titled “time_now / randomness in state mutations”Migrations and merges replay your writes on every node. Anything that reads a
clock, a random source, or any non-deterministic input will diverge between
replicas. Keep state mutations a pure function of their inputs. If you need a
timestamp, take it from the deterministic execution context and store it as data —
never branch state on now() inside a merge-replayed path.
Expecting cross-node read-after-write
Section titled “Expecting cross-node read-after-write”There is no read-your-write across nodes. After a member writes on node A, a read on node B returns the old value until they sync. Do not gate one node’s logic on another node’s just-written state, and do not treat “I don’t see it yet” as “it didn’t happen.” Design for convergence, not immediacy — see CRDT internals for the merge model underneath.