Call an external service
Task: call a service outside the cluster (a captcha check, a payment API, a model, any HTTPS API) and write what it answers into state.
A call may ask its node for one HTTPS request. The node makes it only when every gate below holds and answers with its reason otherwise. Outbound requests are off by default: an app that needs them runs on a cluster whose operator turned them on, for its hosts or for any host an app declares.
The gates
Section titled “The gates”| Gate | Who sets it |
|---|---|
[outbound] enabled = true, and the request’s host is in hosts (or hosts holds "*": any host an installed app declares) |
The cluster’s operator |
The request’s URL is under a prefix in the app’s signed release manifest (outbound): the same host, and the prefix’s path or one below it at a /, with any query string |
The app’s publisher |
| A prefix covering the URL is approved for the app’s publisher | Whoever holds MANAGE_APPLICATION org-wide in the app’s organization (its owner and admins; a member granted it), its personal organization included; the operator where the cluster was deployed with --outbound-approvals operator |
The run is a call that writes: not a view, an init, a migration, or a call by a read-only member |
The node |
One request per run; GET, POST, PUT, PATCH or DELETE; https on port 443 to a DNS name that resolves only to public addresses; no fragment; no redirect followed; no proxy |
The node |
| Request body ≤ 64 KiB, URL ≤ 2 KiB, ≤ 8 headers of ≤ 1 KiB, response body ≤ 1 MiB (a longer one is refused, not cut), 10 s for the whole request | The node (calimero_primitives::outbound) |
Host, Cookie, Connection, Content-Length, Transfer-Encoding, Proxy-* and X-Forwarded-* headers are refused |
The node |
A refusal is an AppError with code Fetch and a message field. A refusal
for a missing approval starts with a stable word an app can match on:
NotApproved:nobody approved the prefix the URL is under. Nothing was ever approved for this publisher, or the owner saw the prefix and left it out or revoked it. Tell a member to ask the owner.AwaitingApproval:a later release of the app declares the prefix and the owner has not been asked about it yet. The prefixes approved before keep working. Show the owner what to approve.
A run with no outbound client at all (outbound is off, the app is not a signed release declaring a prefix, or the run is not a writing call) and a second request in one run trap the run.
Secrets
Section titled “Secrets”A release declares the secrets its requests name in its signed manifest, with the hosts each may go to, so whoever deploys it sees what to supply:
[package.metadata.calimero.secrets.NEAR_AI_API_KEY]hosts = ["cloud-api.near.ai"] # each the host of one of `outbound`'s prefixesdescription = "NEAR AI Cloud API key"required = truemeroctl app pack writes it as secrets and refuses a host no outbound prefix
covers, as a node does at install (400 BAD_SECRETS). A node then fills only a
declared name, and only into a request to one of its declared hosts (and one the
operator bound it to). A release that declares none is filled from the operator’s
bindings alone, as before. The release’s capability card
(GET /packages/{package}/{version}/capabilities) lists them, and an upgrade’s
consent page names a secret it adds or sends elsewhere. A release declaring
secrets installs only once every node runs 1.0.0-rc.31 (declaredSecrets).
Whoever deploys an instance may supply a declared secret’s value for that instance alone, instead of the operator:
PUT /apps/{app}/secrets/{name} { value }(the owner, or whoever manages the organization’s apps, with a passkey step-up over the name and the value’s fingerprint, never the value): kept in the app’s log on its replicas, sealed, out of the app’s own reach (no host function reads it), and filled only into a request to that secret’s declared hosts. It wins over the operator’s value for that name.GET /apps/{app}/secretslists each declared secret withsource: app(its fingerprint, who set it and when) orcluster, never a value.DELETE /apps/{app}/secrets/{name}removes it; the deployment’s or the operator’s applies again.
A deployment’s managers may supply one value for every instance created from
it (PUT /deployments/{id}/secrets/{name} { value }, a passkey step-up over
deployment:{id} / __set_secret / { name, fingerprint }; GET answers
fingerprints, DELETE clears it; meroctl deployment secret set|list|clear), for a name a release the deployment offers
declares, or a release one of those references in its uses declares (for the
apps an instance creates with it). It is kept in the system group’s state and read by the outbound gate
on the node that runs the request. A request is filled with the instance’s own
value, else its deployment’s, else the operator’s, each only for a name the
release declares and only to its declared hosts, and the deployment’s only for
a release of a package the deployment offers; an instance’s
GET /apps/{app}/secrets then says source: deployment, nothing more of it.
See Deployments.
Read it left to right: the first value found wins, each source only for a name your release declares, and whichever wins goes out only to that secret’s declared hosts and never comes back in a response.
The app never holds a secret. A request names one as {{secret:NAME}} in a
header value or the body; the node fills it in from its operator’s
secrets, for this app only (the package and signer of the installed release,
never anything the app says), and only into a request to a host the operator
bound that secret to. Any copy of it in the response is replaced with
[redacted] before the app sees the response. So a method cannot write the
secret into state, where it would replicate to every member.
-
Declare the URL prefixes in the release’s
manifest.json, underoutbound(calimero_bundle::BundleManifest::outbound). They are signed with the rest of the manifest and shown to whoever approves them, so declare the narrowest that works. Each ishttps://with a host and no credentials, query or fragment; the node refuses a bundle with any other:"outbound": ["https://challenges.cloudflare.com/turnstile/v0/siteverify","https://api.stripe.com/v1/"]https://api.stripe.com/v1/covers every path below/v1/.meroctl app packwrites it from--outbound, once per prefix, and refuses a prefix the node would (nothttps://, credentials, a query or a fragment):Terminal window meroctl app pack --wasm app.wasm --package com.example.shop \--app-version 1.2.0 --key my-signing-key.json \--outbound https://challenges.cloudflare.com/turnstile/v0/siteverify \--outbound https://api.stripe.com/v1/ --out dist/com.example.shop-1.2.0.mpk -
Make the request from a method that writes, and store the decision:
use calimero_sdk::action::Request;/// Whether Cloudflare Turnstile accepts `token` (`apps/fixture-captcha`).pub fn verify_turnstile(&mut self, token: String) -> app::Result<bool> {let body = format!("secret={{{{secret:TURNSTILE_SECRET}}}}&response={}",form_escape(&token));let request = Request::post(TURNSTILE_VERIFY_URL, body).header("Content-Type", "application/x-www-form-urlencoded");let Ok(response) = env::fetch(&request) else {return Ok(false); // an unreachable verifier is never a pass};Ok(response.status == 200&& response.json::<Siteverify>().is_ok_and(|answer| answer.success))} -
Have the owner approve the prefixes (see approving), and the operator allow the host and supply the secret (see the operator’s side).
Approving the prefixes
Section titled “Approving the prefixes”The organization’s data is what a call can send, so the organization agrees to
the hosts too: the operator allowing a host and the publisher declaring it are
not enough. Someone holding MANAGE_APPLICATION org-wide approves with a
passkey step-up over the exact change:
POST /orgs/{org}/outbound{ "package": "com.acme.forms", "prefixes": "manifest" }"manifest" (the default) approves every prefix the package’s latest release
declares; a list approves those. An app that declares no outbound needs and
asks for nothing.
- Per publisher. The approval is keyed by the package’s pinned signer, so a release signed by someone else starts with nothing approved.
- Per organization. It holds for every app of the package in the
organization. An app created without naming an organization is in its
owner’s only organization, or one made with it, approved the same way on
/orgs/{org}/outbound(there are no per-account approvals since 1.0.0-rc.20). - Releases only add. A prefix a later release declares that the owner was
never shown answers
AwaitingApprovaluntil approved; the rest keep working. - Revoke by setting the list.
"replace": truesets the publisher’s list exactly; an empty list with it revokes everything. It holds from the next call. - Operator-governed clusters. A cluster deployed with
--outbound-approvals operatortakes approvals from its operator instead, for every app:POST /ops/outboundwith the same body, over the Ops API. The owner’s route then answers that the operator holds them.
With meroctl: org outbound <org> --package <pkg> [--prefix <p>]… [--replace],
and account outbound --package <pkg> …, which does the same on the account’s
only organization. On a development cluster add --dev-passkey <file> to
approve the step-up.
An app’s frontend can also ask for the approval on the node’s consent page
(POST /consent/requests with kind: outbound), where the person approves
with their passkey.
The operator’s side
Section titled “The operator’s side”The [outbound] table of merod’s configuration, or of the file
--settings names (which replaces it). Give every node of a cluster the same
table: the call runs on the app’s leader, whichever node that is.
[outbound]enabled = truehosts = ["challenges.cloudflare.com", "api.stripe.com"] # or ["*"]
# Secrets from environment variables, read once at start: a missing one# fails the start.[outbound.env_secrets.TURNSTILE_SECRET]env = "MERO_TURNSTILE_SECRET"packages = ["com.calimero.mero-forms"]signers = ["did:key:z6Mk…"] # the package's signerhosts = ["challenges.cloudflare.com"]hosts = ["*"] lets every installed app reach the hosts its own signed
manifest declares and its owner approved, so a new app needs no config
change. Every other gate still holds, and a secret still goes only to the hosts
bound to it.
Instead of env_secrets, a node can ask a secrets service beside it
(exclusive with env_secrets):
[outbound.sidecar]url = "http://127.0.0.1:7070/v1/secret"token_env = "MERO_SECRETS_TOKEN"The node POSTs {"package", "signer", "name"} with the token as a bearer and
expects 200 {"value", "hosts"} or 404. The URL must be http on a loopback
address. A managed deployment answers there with the secrets a deployer set
for their app, so the secret lives on the nodes that serve the app and nowhere
else.
The node logs every request it makes or refuses, with the app and the URL,
never the secret, and counts the requests it sends
(calimero_app_outbound_requests_total).
A secret is matched on both the package and the release’s signer: anyone can name a package, only its publisher can sign it.
Test it
Section titled “Test it”env::fetch needs a component: on the host target it answers an error. In
TestApp, mock_http(..) answers it in every run, so a test sets the response
it wants and checks what the app decided (Testing).
End to end, use a node built with --features calimero-node/outbound-testing:
the scenario runner’s in-process nodes are, and so is
cargo build -p merod --features calimero-node/outbound-testing. Its
[outbound.testing] table, which a release build refuses, answers .test
hosts in-process and takes literal secrets. A scenario gives its nodes that
TOML as cluster.settings:
[outbound]enabled = truehosts = ["echo.outbound.test"]
[outbound.testing]echo = true # .test hosts are answered in-process
[outbound.testing.secrets.ECHO_KEY]value = "s3cr3t-e2e"packages = ["com.example.my-app"]signers = ["did:key:z6Mk…"]hosts = ["echo.outbound.test"]The echo answers 200 with the request the node sent, as JSON: method, URL,
path, query, headers, body and the number of secrets filled in. It answers only
after every gate has passed, and its answer is redacted like any other, so a
scenario sees exactly what left the node. The node’s cluster tests use it
this way: actions_run_on_the_leader_and_reach_only_declared_approved_hosts
(declared and approved prefixes, secrets, redaction) and
a_call_checks_a_turnstile_token_through_a_captcha_service (a call’s
env::fetch through apps/fixture-captcha), both in
crates/node/tests/component_cluster.rs:
cargo test -p calimero-node --test component_cluster -- reach_only_declared_approved_hostsRetries and actions
Section titled “Retries and actions”env::fetch works from a call, a timer or a delivered xcall. The request goes
out once, from the leader, and the call’s log entry records it with the
response, so a retry with the same idempotency key replays the answer instead
of fetching again (what a single-use token like Turnstile’s needs). A timeout
or a refusal comes back as an error the call handles. Example:
apps/fixture-captcha (verifyTurnstile), a captcha service other apps
call synchronously (Synchronous xcalls).
Protocol detail:
Outbound HTTP from component calls.
For work that needs several requests, or a request that should not hold up the
call, an #[app::action] runs outside the log on the leader: it fetches with
ctx.fetch, reads the app with ctx.view and writes through ctx.call to an
#[app::allow(action)] method. A call or timer starts one with
env::start_action(method, &args)
(Timers and actions).
Where it stops
Section titled “Where it stops”- A request never runs in a view. Read the answer back from state.
- Redaction removes the secret as sent. A service that answers with it encoded (base64, URL-escaped) defeats it, so bind a secret only to hosts that do not echo it.
- The response is not authenticated beyond TLS: a service you call can answer anything, so call only services you trust for the decision you store.