Choosing a collection, and who may write it
Every field in #[app::state] answers two questions, and they are separate:
- How is it read? By key, in key order, or filtered and sorted by what the
values contain. That picks the collection:
UnorderedMap,SortedMaporIndexedMap. - Who may change it? Anyone in the context, only whoever wrote an entry,
a set of writers, or nobody once it is written. That picks the wrapper:
plain,
Authored,WriteOnce,Moderated,SharedStorage,Frozen, and so on.
The wrappers are type parameters over the collections, so the two choices
combine freely: Moderated<IndexedMap<String, Post>> is posts their authors own,
moderators can remove, and feeds can filter by board.
This page covers both choices, then what of each syncs, then how permissions reach structs and nested collections. For the full API of each type see Collections; for costs, Storage performance.
Reading: UnorderedMap, SortedMap or IndexedMap
Section titled “Reading: UnorderedMap, SortedMap or IndexedMap”All three store their entries identically, byte for byte, and report the same kind of entry to sync. What differs is a node-local index each node builds for itself from the entries, which never crosses the wire. Because the stored bytes are the same, a field can switch between the three with no migration.
UnorderedMap<K, V> |
SortedMap<K, V> |
IndexedMap<K, V> |
|
|---|---|---|---|
| Fast reads | get(key), contains |
also range, prefix, page, first, last, all in key order |
also query(index).eq(..).range(..).desc().limit(..) and count() on fields of the value |
| Local index | none | one: keys in order | one per index the value type declares with #[derive(app::Indexed)] |
| Extra cost per write | none | one index row | about one read and three writes per declared index |
| First read after a peer’s change | normal | rebuilds the key order, O(n) |
rebuilds the indexes, O(n) |
| Reach for it when | you look entries up by key | your keys are hierarchical (<post>/<time>/<id>) and you read slices |
you filter, sort or count by what is in the values: status, board, owner, tags |
A rule of thumb that holds up:
- Start with
UnorderedMap. Profiles by account, settings by name, anything you only everget. - Use
SortedMapwhen the key already encodes the order you read in. A thread of comments keyed<post>/<created_at>/<id>is oneprefixcall. - Use
IndexedMapwhen a list view filters by a field. “The newest 20 open issues”, “posts indevtaggedrust”, “how many are pinned” are all seeks, without reading the other entries:
#[derive(BorshSerialize, BorshDeserialize, AbiType, app::Mergeable, app::Indexed)]#[index(board_feed(board, created_at))]pub struct Post { #[index] pub board: LwwRegister<String>, #[index] pub tags: LwwRegister<Vec<String>>, // one row per tag pub created_at: LwwRegister<u64>,}
posts.query("board_feed").eq("dev").desc().limit(20).entries()?;posts.query("tags").eq("rust").count()?;A score or vote count cannot be an index key, because an index key has to come
from the value and a converging count lives in its own collection. See
apps/indexed-forum, which keeps each vote as its voter’s own entry at the
post’s id in an AuthoredMap, and reads the tally as the number of entries
there (entries_at).
Writing: who may change it
Section titled “Writing: who may change it”Every stored entity carries a stamp that says who may write it, and every
node checks that stamp when it applies a peer’s write. A modified node can
skip your method’s checks, but it cannot get a write it was not allowed to
make accepted anywhere else. The checks inside your methods (only_owner(),
is_moderator(), …) only make a refused write fail early; see
the two planes.
| You want | Use | Enforced by every node |
|---|---|---|
| Anyone in the context may write | a plain field: UnorderedMap, SortedMap, IndexedMap, LwwRegister, Counter, … |
nothing to enforce |
| Anyone adds entries; only an entry’s author edits or removes it | Authored<C> (AuthoredMap, AuthoredSortedMap are aliases) |
the author’s signature on every update and delete |
| Anyone adds entries; nobody, the author included, ever changes or removes one | WriteOnce<C> |
the entry’s bytes are fixed at creation; deletes refused |
| Authors own and edit their entries; moderators remove any | Moderated<C> |
owner signature for edits; owner or moderator for deletes |
| Nobody edits an entry; moderators remove spam | ModeratedOnce<C> |
bytes fixed; only moderators delete |
| One value, set once when the context is created, then fixed | Frozen<T> |
the value’s only writer may create it and never change it |
| An immutable, de-duplicated set keyed by content hash | ContentAddressed<C>, or FrozenStorage<T> |
the key must be the hash of the value; no updates or deletes |
| One slot per account | UserStorage<T> |
only the slot’s account writes it |
| A group of writers you can change over time | SharedStorage<T> / Ownable<T> / PermissionedStorage<T, A> |
the writer set as of each write, with per-writer capabilities |
Named roles ("editor", …) |
AccessControl |
the admin writer set |
| A name, seat or slug that at most one account may own | Registry<K, V, A> |
claims are their claimants’ own; only the authority A writes verdicts |
| Data that never leaves this node | #[app::private] |
nothing: it never syncs |
C is any of the three maps. Authored<IndexedMap<String, Post>> stores
exactly an AuthoredMap’s bytes, so an authored map can gain indexes without a
migration.
Written once, and moderated
Section titled “Written once, and moderated”WriteOnce, Moderated and ModeratedOnce put rules on each entry:
pub struct EntryRules { pub immutable: bool, // no update with different bytes, no delete pub moderators: Option<Id>, // the writer set that may delete it}The rules ride in the entry’s stamp, are covered by its signature, and are fixed when the entry is created. Every node refuses an update or delete that names different rules, so an author cannot relax them later, and rules altered in transit fail the signature. A collection reads only entries carrying exactly its rules: an entry a patched author wrote with no moderators, to escape moderation, is stored by every node and returned by none.
A moderated collection’s moderators are a writer set, verified and rotated
exactly as a SharedStorage’s. The founder is the first moderator:
pub struct Forum { posts: Moderated<IndexedMap<String, Post>>,}
fn init() -> Self { Self { posts: Moderated::new() } }
self.posts.remove(&id)?; // the author, or any moderatorself.posts.set_moderators(accounts)?; // a moderator onlyself.posts.is_moderator(&account);Every node checks a moderator’s delete against the moderators as of that delete, so revoking someone stops their later removals and leaves their earlier ones standing.
Frozen<T>
Section titled “Frozen<T>”charter: Frozen<String>,
fn init() -> Self { Self { charter: Frozen::new("be kind".to_owned()) } }self.charter.get()?; // there is no setThe value’s only writer is whoever created it, holding a single capability:
write once. On every node, a later write with different bytes is refused even
if that writer signs it, removing it is refused, and nobody can be given the
right to change it. A byte-identical redelivery is accepted, which is what sync
sends. Like SharedStorage’s writer set, the writer comes from genesis, the
state the creator writes, so a new node trusts the peer it first takes genesis
from.
Use it for plain data fixed at creation. For many immutable entries use
WriteOnce (keyed by the app) or ContentAddressed (keyed by content hash).
Unique names: Registry
Section titled “Unique names: Registry”No write rule gives a name one owner. Two members on either side of a
partition who both claim alice each see a state in which only they asked, so
anything that lets one node call the name theirs lets the other do the same.
“Earliest claim wins” is worse: a node checks that a write’s timestamp is not
in the future, never that it is not in the past, so a patched node claims with
a timestamp of 1 and takes any name, including one held for years.
A Registry separates the claims members make from the verdicts an
authority writes, and calls nothing owned until there is a verdict:
names: Registry<String, Profile>, // the TEE decides (the default)
self.names.claim(name.clone(), profile)?; // any member; writes only their own claimapp::emit!((Event::Claimed { name: &name }, "tee:resolve_name"));
#[app::tee]pub fn resolve_name(&mut self, name: String) -> app::Result<()> { self.names.resolve(&name)?; // idempotent: firing is at least once Ok(())}
self.names.status(&name)?; // Free / Pending / Owned { owner, stable } / Lostself.names.owner_of(&name)?; // Some only once a verdict names an ownerGate every use of a name on owner_of, never on holding a claim. The first
claim to reach the authority wins; claims it sees together are split by a hash
of the name, the epoch and the claimant, which no clock and no claim content
can move. An owner releases a name, the authority answers with a vacancy at
the next epoch, and everyone claims afresh.
A |
Who decides | Pick it when | Offline authority |
|---|---|---|---|
Tee (default) |
an attested TEE, from #[app::tee] methods (a trigger per claim, plus an every = .. sweep of resolve_all_pending) |
nobody, admins included, may pick owners | claims stay pending |
Admin |
the admins, a writer set an admin rotates; an admin calls resolve |
the context has trusted operators and no TEE | claims stay pending |
NoAuthority |
nobody: status reports Contested, owner_of is always None |
you only need to show who wants what | nothing to wait for |
With one TEE, or one admin device, a verdict is final. With several deciding at
once (TEE failover fires at least once), two can grant different claimants the
same epoch: every node converges on the same one, and until they meet each side
believes its own. Owned { stable: false } says a node has seen such a rival.
apps/name-registry is the Tee example; apps/name-registry-admin is the
Admin one. With Admin, the account that ran init is the only admin until
an admin calls set_admins, and every node checks a verdict against the admins
as of that verdict, so an admin who handed the role on can no longer resolve.
What syncs, and what stays on the node
Section titled “What syncs, and what stays on the node”| Piece | Syncs? | How |
|---|---|---|
The root state blob (plain fields, LwwRegisters) |
yes | one entity, merged field by field by the code #[app::state] generates |
| Each collection’s container | yes | its own entity, holding the collection’s id |
| Each entry of a map, set or vector | yes | its own entity, merged per entry |
| A collection nested in an entry’s value | yes | its own entity, at an id derived from the entry’s, so every node computes the same one |
| A stamp: owner, writer set, anchor, rules | yes | carried and signed with each write, checked on apply |
SortedMap / SortedSet key order |
no | node-local index, rebuilt after a sync |
IndexedMap indexes, and their validity markers |
no | node-local index, rebuilt after a sync |
#[app::private] state |
no | a separate column, outside the Merkle root, one writer: this node |
An entry’s schema_version |
yes | in its metadata, but outside the Merkle hash, so a migration does not look like divergence |
Nothing node-local reaches the Merkle root, so nodes that index differently, or not at all, still agree on every hash.
Merging: the entry is the unit
Section titled “Merging: the entry is the unit”Knowing what an entity is tells you what survives two nodes writing at once:
- Root state merges field by field. Two nodes setting different
LwwRegisterfields of the root both keep their write. - A map entry holding a struct is one entity. By default, concurrent writes
to it resolve last-write-wins on the whole entry, so two nodes editing
different fields of the same entry keep one of the two edits. Declare the
type
#[app::mergeable], with aMergeableimpl, to have the storage layer call your merge instead; seeapps/team-metrics-custom. That holds in a signed entry too (anAuthoredmap, aSharedStorageorTeeOnlycell): a write that arrives after a newer one still reaches your merge, so the result does not depend on delivery order. - A collection field inside that struct is its own entity and merges on
its own: a
Counterfield sums every node’s increments, however the rest of the entry resolved.
So keep a value that several people edit at once either split into entries,
or in a collection field, or behind #[app::mergeable].
Permissions on plain fields, structs and nested collections
Section titled “Permissions on plain fields, structs and nested collections”The rule is short: an entry’s stamp covers the entry’s bytes, and every collection inside it, at any depth. Everything else follows.
Plain fields and the root
Section titled “Plain fields and the root”The root state is Public: any context member may write any plain field, and
any unguarded collection. Guarding one field (say, checking only_owner() on
an Ownable) does nothing for the field next to it:
pub fn set_secret(&mut self, value: String) -> app::Result<()> { self.owner_cell.only_owner()?; // checks one entity... self.secret.set(value); // ...and writes a different, public one Ok(())}Put what needs guarding inside the wrapper that guards it.
Struct fields
Section titled “Struct fields”There are no per-field permissions. An entry’s owner may rewrite any field of the struct it holds, and nobody else may rewrite any. If different people must own different parts, make them different entries, or different collections.
Collections nested in a guarded entry
Section titled “Collections nested in a guarded entry”A collection inside an entry’s value takes the entry’s rule, at every depth:
| The entry is | Collections inside it are |
|---|---|
owned by an author (Authored, Moderated, UserStorage, AuthoredVector) |
owned by that same author. Only they can add, change or remove entries in them. |
written once (WriteOnce, ModeratedOnce) |
sealed: they cannot hold entries. A value with filled nested collections is refused when inserted. |
a member of a writer-set cell (SharedStorage) |
members of the same cell: only its writers, with the right capability. |
frozen (ContentAddressed, FrozenStorage) |
sealed. |
| public | public. |
posts: Authored<UnorderedMap<String, Post>>,pub struct Post { title: LwwRegister<String>, tags: UnorderedMap<String, LwwRegister<u64>> }
// Alice's post: its `tags` map is Alice's too.let mut tags = posts.get(&alices_post)?.unwrap().tags;tags.insert("spam".into(), 1.into())?; // as Bob: refusedA wrapper nested inside keeps its own policy. An Authored map of comments
stored in Alice’s post lets anyone comment, and each comment is owned by whoever
wrote it; Alice cannot rewrite Bob’s comment in her own post.
How this holds against a patched peer: a receiving node cannot always tell
which entry a new write belongs to, because a write names its parent itself.
So a peer can make every node store a Public entry that claims to sit in
Alice’s nested map. It cannot make any node read it. A collection in a
guarded domain returns only entries stamped for that domain: owned by Alice,
or a member of that cell. Those stamps are signature-checked on apply, so they
cannot be forged, and every node applies the same filter, so they all agree.
Known limits
Section titled “Known limits”- Keys are first come, first served. In an authored collection, whoever first writes a key owns it. If squatting a predictable key matters, include something the squatter cannot choose: the author’s account, or a random id.
- Genesis is trusted. Writer sets (
SharedStorage, aFrozenvalue’s writer, a moderated collection’s moderators) come from the state the creator writes. A node that took genesis from a malicious peer holds that peer’s version, as it would for any writer set. - A writer can race themself. A
Frozenvalue’s writer, or a write-once entry’s author, running patched code on two devices can create two versions concurrently. Only they can, and it is their own value. - Reads are never restricted. Every member can read everything that syncs.
For confidentiality, seal the value (
TeeSecret,Sealed) or keep it in#[app::private].
A worked example
Section titled “A worked example”apps/indexed-forum uses most of this in one app, with a two-node scenario that
exercises it on real nodes:
| Field | Type | Why |
|---|---|---|
charter |
Frozen<String> |
set once in init, changeable by nobody |
posts |
Moderated<IndexedMap<String, Post>> |
authors edit their posts, moderators remove spam, feeds filter by board, tag, pin and author |
comments |
AuthoredSortedMap<String, …> |
a thread is one key prefix, and only a comment’s author edits it |
votes |
AuthoredMap<String, LwwRegister<bool>> |
one entry per voter per post, the voter’s own: none is lost, and nobody votes in someone else’s name |
apps/permissions-showcase covers the rest, one field per policy, with a
two-node scenario that checks each rule on the node that did not write:
| Field | Type | Why |
|---|---|---|
founding, quorum |
Frozen<Founding>, Frozen<u64> |
fixed at init |
messages |
WriteOnce<SortedMap<String, Message>> |
signed messages nobody edits, a channel is one prefix slice |
announcements |
ModeratedOnce<UnorderedMap<String, Message>> |
never edited, removed only by a moderator |
pages |
Authored<UnorderedMap<String, Page>> |
the author owns the page and the revisions nested in it |
evidence |
ContentAddressed<IndexedMap<[u8; 32], Evidence>> |
de-duplicated by content, found by kind |