Blobs

A blob is immutable, content-addressed bytes kept outside every log: an attachment, an image, a model. Log entries are capped at 1 MiB and should stay small, so anything large goes to the blob store and the log carries only its id. The same store holds bundles and uploaded .mpk releases.

The store

Each node keeps blobs in its own RocksDB blobs column, sealed under the cluster’s storage key like everything else (Storage schema). A blob’s id is hex(SHA-256(bytes)); the API also accepts the base58 form apps print (BlobId).

• Writes reach a majority. Storing a blob writes it locally and pushes it to every other node (internal/blob); the write succeeds once a majority of the cluster’s nodes hold it.
• A node that missed one fetches it. A node asked for a blob it does not hold — it was down when the blob was stored — fetches it from a peer (internal/blob/get), checks the hash, and keeps it.

Every node holds every blob it was reachable for, so there is nothing to announce or discover.

Which blobs are an app’s

Bytes in the store belong to no one. What makes a blob an app’s is a mark in the app’s state, under the host’s key space (\0calimero.blobs\0 ‖ id → size). Marks are written only by the node, as part of an entry, so they replicate with the app and are ordered with its calls.

A run opens only blobs its app’s state marks (services::AppBlobs). A run on a node that misses the bytes fetches them from a peer, waiting at most 5 seconds.

Getting a blob in

• Upload. POST /apps/{app}/blobs (raw bytes) by a signed-in caller who reaches the app and may write it — not a read-only API key and not a read-only team member. The node stores the bytes on a majority, then commits AppCmd::Blob { id, size } to the app’s log. The answer names the blob and the entry’s index.
• From a run. env::blob_create / blob_write / blob_close in a call produce blobs on the run’s outcome, never through the store directly. The leader stores them locally before proposing the entry, the entry’s diff marks them as the app’s, and after the entry commits the leader spreads them to the other nodes. Bytes are never part of the log entry.

Reading and listing

• GET /apps/{app}/blobs/{id} downloads a blob the app marks, by anyone who reaches the app, except a published compute output (a grant is needed). A blob the app does not mark is 404, whether or not the bytes exist.
• GET /apps/{app}/blobs and /blobs/{id}/info list the app’s blobs and their sizes, from a scan of a replica’s state.
• Inside the sealed channel, a download is sealed frame by frame like any response.

Deleting

DELETE /apps/{app}/blobs/{id}, by the app’s owner or an admin of its organization, commits AppCmd::BlobForget: the app no longer has the blob, its runs cannot open it, and the API answers 404. Then the node collects the bytes (Node::collect_blob): it deletes them on every node unless

• another app still marks the same content — asked of each app’s leader (blob_held_strong), because a lagging replica could miss a mark committed a moment ago, and an app whose leader does not answer counts as holding it; or
• the content is a bundle or a release .mpk.

Known window: if the same content is uploaded to another app between that check and the deletion, the other app’s mark survives but its bytes are gone.

Component apps (calimero:app)

A component app (the v2 SDK, calimero-sdk) keeps its blobs in the same store, with the same upload, download, list and delete routes. What differs is which blobs it holds and for how long (spec app-runtime-v0.14 R8):

• Host records, not marks. The node records each of the app’s blobs (size, the uploader’s Content-Type, a reference count) as host-owned rows of the app’s group (rows::CBLOB, D39), outside the keyspace the app writes. An upload commits componentBlob { id, size, mime, tsMs }; the owner’s (or an org admin’s) delete commits componentBlobForget.
• Typed references. State refers to a blob through a BlobRef, which both SDKs store as CBOR tag 51,739 (0xCA1B) around the blob’s 32-byte SHA-256 and send in JSON as the 64-hex-digit id. A run that stores a reference to a blob the app does not hold traps HostDenied.
• Host-tracked retention. Every replica counts references from the diffs it applies and the values they replace. A blob nothing refers to is held for one hour after its upload or after its last reference went (BLOB_IDLE_MS), so a client uploads first and attaches it with a later call. After that the app forgets it, judged against entry timestamps so every replica forgets it in the same entry; the leader then collects the bytes as above and logs componentBlobsCollected.
• No bytes in runs. The WIT blobs interface has info (size, SHA-256, content type; none for a blob the app does not hold) and grant(id, ttl). Calls, views, timers, init and xcall deliveries may use them; actions and migrations may not.
• Grants. grant returns <expiresMs>.<mac>, an HMAC under a key derived from the cluster key over the app, the blob and the expiry (the run’s clock plus 1 s to 24 h). GET /apps/{app}/blobs/{id}?grant=<grant> downloads with no session on any node, until it expires; a bad or expired grant answers 404. Downloads carry the recorded content type.

Files of component apps

A component app’s runs can also make sealed files: a file is written in a call, tick or init, and its contents are read only inside a compute run. A file is a sealed blob. It shares the blob id namespace and the BlobRef type, and blobs.info answers for it with an empty content type. It cannot be downloaded or granted, and no blob upload can claim its id. A call, view, timer or init may create a file and ask for its size and hash, and files.open is refused there. So a sealed file’s contents can be read only inside a compute run of the app release the organization approved, and leave only as a compute output published through a logged call. The node seals a file in the TD as 64 KiB AES-256-GCM chunks under a random data key, and wraps that key under a key derived from the cluster key for this app. The blob id is the SHA-256 of the ciphertext, so a replica can check what a peer sends. The app’s blob record for the file holds the plaintext size and SHA-256 and the wrapped key, in the hidden rows the guest cannot reach. The log entry carries these marks and never the ciphertext.

A client uploads a file with POST /apps/{app}/files (the raw bytes, up to one part of 16 MiB) or, for a larger file, through an upload session. The answer is { id, blobId, size, sha256, index } for the plaintext. A node that does not lead the app passes the request to the leader. The leader seals the file as a run’s files are sealed, stores it on a majority of the app’s voters, and then commits the blob record as a command of the node’s own, which no /intents call can name. Only the app’s owner and the members (not guests) of its organization may upload and list files, even where the app is public.

An upload session lives in the leader’s memory, belongs to the account that began it, and takes the file one part per segment. POST /apps/{app}/files/uploads { size } answers { uploadId, partBytes, parts, expiresAt } (expiresAt in Unix seconds). PUT /apps/{app}/files/uploads/{uploadId}/parts/{k} takes part k as raw bytes, exactly partBytes but for the last, and answers { next } once the part is sealed and stored; its segment is pushed to a majority of the voters in the background, at most four segments ahead (past that the answer waits for a slot, up to 20 seconds, then is 503 FILES_UNAVAILABLE and the resent part replays its answer). Parts go in order: a part ahead of next is 409 PART_OUT_OF_ORDER naming next, a resent part answers its first answer, and a resent part with other bytes is 409 PART_CONFLICT. GET .../uploads/{uploadId} answers { uploadId, size, partBytes, parts, next, expiresAt }, for a client that resumes. POST .../uploads/{uploadId}/complete waits up to 20 seconds for every segment to reach a majority (else 503 FILES_UNAVAILABLE, and a retry resumes), charges the quota, commits the mark and answers like the single request; before every part is in it is 409 UPLOAD_INCOMPLETE naming next. It commits under the key node:upload-{uploadId}, so a retried complete answers the same file, even after a leader change. DELETE .../uploads/{uploadId} aborts the session and deletes what it stored. A session that is unknown, belongs to another account, has run out (no part for 15 minutes, or six hours old) or whose leader changed answers 404 UPLOAD_UNKNOWN, and the client begins again. An app holds at most 4 sessions, a node 32, and a leader’s open sessions declare at most 16 GiB together (429 TOO_MANY_UPLOADS). Nothing is charged before complete; a complete past the quota is 429 QUOTA_EXCEEDED and deletes the bytes. GET /apps/{app}/files lists the files ({ files: [{ id, blobId, size, sha256 }] }) and GET /apps/{app}/files/{id} answers one, to the app’s owner and its organization’s members; /apps/{app}/blobs lists plain blobs only and answers 409 NOT_SUPPORTED for a file id. The reason is that a file’s SHA-256 is the plaintext’s, and would let anyone confirm a guessed document. GET /apps/{app}/blobs lists the files after the blobs, and GET /apps/{app}/blobs/{id}/info answers a file’s size and SHA-256, to those people only.

The committed entry also records the account that uploaded the file, taken from the authenticated session or API key of the upload, never from a field of the request; a node that forwards the upload to the leader names it, and the leader refuses a name that is no account id. The app reads it with runs.uploader (files::uploader in the Rust SDK), which answers none for a file a call or a run made and for an unknown id. Every upload is sealed under a key of its own, so the same bytes uploaded twice are two files with two ids and two uploaders; the entry of an id the app already holds is refused, so a later upload never replaces the first uploader. A compute run does not see it. Everyone else gets the list without files, and 404 for a file’s info, as for an id the app does not hold. Neither answer ever carries the key or any content. An uploaded file is held like any file: for an hour unless the app stores a BlobRef to it.

There is no delete. files.finish returns the blob id, and the app keeps the file by storing it as a BlobRef, exactly as for an uploaded blob. The entry that makes the file also applies the state write that stores the reference, so the file is never collectable in between. A file nothing refers to is held for an hour from its entry, and one whose last reference went is forgotten the same way a blob is. Every replica forgets the record in the same entry, and the bytes are then collected from the nodes that hold them.

The ciphertext lives on the app’s replicas, not on every node. Before the leader proposes the entry, the new files are stored on it and pushed to the app’s other replicas. The call proceeds only once a majority of the app’s voters hold every new file; otherwise it fails with a retryable 503, nothing is logged, and its idempotency key is not spent. A node holds at most 256 MiB of new files’ ciphertext across all runs until their entries are proposed; a call that cannot get its share fails with a retryable 503, is not logged, and does not spend its idempotency key. A file is stored as segments of 256 sealed chunks (16 MiB of plaintext), each under its own hash, and the file’s id is the hash of that list; a replica that misses a segment fetches it from the other replicas when a run opens the file. If no replica can serve it, a call, tick or init is aborted with a retryable 503 and not logged, while a view is told the bytes are not available. A run reads a file by decrypting it as it goes, and the end of the stream checks the plaintext size and SHA-256 against the mark. A file that was altered or cut short fails the run with a trap, so no app can ignore it. Each file has its own random key, so no other app can mark the same ciphertext.

The plaintext bytes of an organization’s files (a personal account’s, for an app outside one) count against maxFileBytesPerOrg (default 50 GB). A file counts from the moment it is charged until retention collects it. The charge is taken before any bytes move, so an upload or a call that would pass the quota fails with 429 QUOTA_EXCEEDED, nothing is logged and the stored bytes are removed. When the leader collects the bytes of files the app forgot, the same entry that logs the collection tells the node how many plaintext bytes it released. Deleting an app or an organization frees its files in the same system command. A node that stops hosting a deleted app drops the bytes of its files. The count and the app’s log are not one atomic step. When a record’s commit fails, the leader checks whether it was logged and settles the count either way. A crash between the two can still leave an over-count. Transferring a personal app is refused if its files would pass the new owner’s quota. GET /auth/usage shows an account’s file bytes per payer.

Publishing a compute output

A sealed file never leaves the cluster, with one exception: an app’s logged write can publish the output of one of its finished compute runs with runs.publish (compute::publish in the Rust SDK). The node answers a plain blob of the app, named by the SHA-256 of the plaintext. From then on blobs.info answers it, and blobs.grant gives a time-limited link, for example to an outside auditor. Until an app publishes, the same download answers 409.

A published blob downloads only through a grant, for every caller including the owner: the plain GET /apps/{app}/blobs/{id} answers 404. blobs.grant of it works only in logged writes (calls, timers, init, xcall deliveries), never in a view, so the log records who shared it. The owner or an org admin can revoke every grant made so far with POST /apps/{app}/blobs/{id}/revoke, after which a new grant holds. The blob record counts the revokes and a grant names the count it was made under, so every replica refuses a revoked grant, and a new leader does too. The app also keeps a floor above the count of every published blob it forgot, and a blob made again starts there, so a revoked grant stays refused when the same output is published again. DELETE /apps/{app}/blobs/{id} takes a published blob down: the app forgets it, its bytes are collected and its charge is released.

The platform enforces one rule: the id must be an output of a done run of this app. An uploaded file, a file a call made, another app’s output, a failed or cancelled run’s output and an unknown id answer the soft error NOT_A_RUN_OUTPUT. Each app decides who may publish, in the method that calls it. The rule is in the log, not in the leader. The leader decrypts the output, stores the plaintext on a majority of the app’s voters as it does a new file, and only then proposes the entry. The entry carries the source, the plain blob’s id and its size, never the bytes. Every replica applies the entry only if the source is a file the app’s records mark as a done run’s output, and the id and size are that file’s plaintext hash and size. Otherwise it answers NOT_A_RUN_OUTPUT and changes nothing, so a leader cannot publish anything else.

The plaintext counts against the organization’s file quota from the entry until retention collects it. Retention is R8: the plain blob is held while a BlobRef in state names it, and for an hour after. Publishing an output the app already holds as a plain blob stores and charges nothing again. A run publishes at most 16 outputs, each at most 47 MiB (the plain blob it becomes; a larger output answers file too large) and 64 MiB together. Past that publish answers the soft error run published too much and records nothing.

Limits

Limit	Value
A sealed request (and so an upload)	64 MiB
Open blob handles per run	100
One read or write chunk	10 MiB
All blobs one run writes	512 MiB
A run waiting for a missing blob	5 s
Blobs one component app holds	100,000 (429 QUOTA_EXCEEDED past it)
A component blob’s content type	127 bytes, printable ASCII
A component blob held with no reference	1 hour
A sealed file held with no reference	1 hour
Plaintext bytes of an organization’s files	maxFileBytesPerOrg, 50 GB (429 QUOTA_EXCEEDED past it)
New-file ciphertext all runs of a node hold before their entries	256 MiB
A grant	1 s to 24 h