Skip to content

Call an external service

Task: call a service outside the cluster (a captcha check, a payment API, a model, any HTTPS API) and write what it answers into state.

A call may ask its node for one HTTPS request. The node makes it only when every gate below holds and answers with its reason otherwise. Outbound requests are off by default: an app that needs them runs on a cluster whose operator turned them on, for its hosts or for any host an app declares.

Gate Who sets it
[outbound] enabled = true, and the request’s host is in hosts (or hosts holds "*": any host an installed app declares) The cluster’s operator
The request’s URL is under a prefix in the app’s signed release manifest (outbound): the same host, and the prefix’s path or one below it at a /, with any query string The app’s publisher
A prefix covering the URL is approved for the app’s publisher The organization’s owner (or the account, for an app outside an organization); the operator where the cluster was deployed with --outbound-approvals operator
The run is a call that writes: not a view, an init, a migration, or a call by a read-only member The node
One request per run; GET, POST, PUT, PATCH or DELETE; https on port 443 to a DNS name that resolves only to public addresses; no fragment; no redirect followed; no proxy The node
Request body ≤ 64 KiB, URL ≤ 2 KiB, ≤ 8 headers of ≤ 1 KiB, response body ≤ 1 MiB (a longer one is refused, not cut), 10 s for the whole request The node (calimero_primitives::outbound)
Host, Cookie, Connection, Content-Length, Transfer-Encoding, Proxy-* and X-Forwarded-* headers are refused The node

A refusal is an AppError with code Fetch and a message field. A refusal for a missing approval starts with a stable word an app can match on:

  • NotApproved: nobody approved the prefix the URL is under. Nothing was ever approved for this publisher, or the owner saw the prefix and left it out or revoked it. Tell a member to ask the owner.
  • AwaitingApproval: a later release of the app declares the prefix and the owner has not been asked about it yet. The prefixes approved before keep working. Show the owner what to approve.

A run with no outbound client at all (outbound is off, the app is not a signed release declaring a prefix, or the run is not a writing call) and a second request in one run trap the run.

The app never holds a secret. A request names one as {{secret:NAME}} in a header value or the body; the node fills it in from its operator’s secrets, for this app only (the package and signer of the installed release, never anything the app says), and only into a request to a host the operator bound that secret to. Any copy of it in the response is replaced with [redacted] before the app sees the response. So a method cannot write the secret into state, where it would replicate to every member.

  1. Declare the URL prefixes in Cargo.toml. They are signed into the release’s manifest and shown to whoever approves them, so declare the narrowest that works:

    [package.metadata.calimero]
    outbound = [
    "https://challenges.cloudflare.com/turnstile/v0/siteverify",
    "https://api.stripe.com/v1/", # every path below /v1/
    ]
  2. Make the request from a method that writes, and store the decision:

    use calimero_sdk::action::Request;
    /// Whether Cloudflare Turnstile accepts `token` (`apps/v2-fixtures/captcha`).
    pub fn verify_turnstile(&mut self, token: String) -> app::Result<bool> {
    let body = format!(
    "secret={{{{secret:TURNSTILE_SECRET}}}}&response={}",
    form_escape(&token)
    );
    let request = Request::post(TURNSTILE_VERIFY_URL, body)
    .header("Content-Type", "application/x-www-form-urlencoded");
    let Ok(response) = env::fetch(&request) else {
    return Ok(false); // an unreachable verifier is never a pass
    };
    Ok(response.status == 200
    && response.json::<Siteverify>().is_ok_and(|answer| answer.success))
    }
  3. Have the owner approve the prefixes (see approving), and the operator allow the host and supply the secret (see the operator’s side).

The organization’s data is what a call can send, so its owner agrees to the hosts too: the operator allowing a host and the publisher declaring it are not enough. The owner approves with a passkey step-up over the exact change:

POST /orgs/{org}/outbound
{ "package": "com.acme.forms", "prefixes": "manifest" }

"manifest" (the default) approves every prefix the package’s latest release declares; a list approves those. An app that declares no outbound needs and asks for nothing.

  • Per publisher. The approval is keyed by the package’s pinned signer, so a release signed by someone else starts with nothing approved.
  • Per organization. It holds for every app of the package in the organization. An app outside any organization uses its owner’s own approval (POST /auth/outbound, same body).
  • Releases only add. A prefix a later release declares that the owner was never shown answers AwaitingApproval until approved; the rest keep working.
  • Revoke by setting the list. "replace": true sets the publisher’s list exactly; an empty list with it revokes everything. It holds from the next call.
  • Operator-governed clusters. A cluster deployed with --outbound-approvals operator takes approvals from its operator instead, for every app: POST /ops/outbound with the same body, over the Ops API. The owner’s route then answers that the operator holds them.

With meroctl: namespace outbound <org> --package <pkg> [--prefix <p>]… [--replace], and account outbound --package <pkg> … for apps outside an organization.

The [outbound] table of merod’s configuration, or of the file --settings names (which replaces it). Give every node of a cluster the same table: the call runs on the app’s leader, whichever node that is.

[outbound]
enabled = true
hosts = ["challenges.cloudflare.com", "api.stripe.com"] # or ["*"]
# Secrets from environment variables, read once at start: a missing one
# fails the start.
[outbound.env_secrets.TURNSTILE_SECRET]
env = "MERO_TURNSTILE_SECRET"
packages = ["com.calimero.mero-forms"]
signers = ["did:key:z6Mk…"] # the package's signer
hosts = ["challenges.cloudflare.com"]

hosts = ["*"] lets every installed app reach the hosts its own signed manifest declares and its owner approved, so a new app needs no config change. Every other gate still holds, and a secret still goes only to the hosts bound to it.

Instead of env_secrets, a node can ask a secrets service beside it (exclusive with env_secrets):

[outbound.sidecar]
url = "http://127.0.0.1:7070/v1/secret"
token_env = "MERO_SECRETS_TOKEN"

The node POSTs {"package", "signer", "name"} with the token as a bearer and expects 200 {"value", "hosts"} or 404. The URL must be http on a loopback address. A managed deployment answers there with the secrets a deployer set for their app, so the secret lives on the nodes that serve the app and nowhere else.

The node logs every request it makes or refuses, with the app and the URL, never the secret, and counts the requests it sends (calimero_app_outbound_requests_total).

A secret is matched on both the package and the release’s signer: anyone can name a package, only its publisher can sign it.

env::fetch needs a component: on the host target it answers an error. In TestApp, mock_http(..) answers it in every run, so a test sets the response it wants and checks what the app decided (Testing).

End to end, use a node built with --features calimero-node/outbound-testing: the scenario runner’s in-process nodes are, and so is cargo build -p merod --features calimero-node/outbound-testing. Its [outbound.testing] table, which a release build refuses, answers .test hosts in-process and takes literal secrets. A scenario gives its nodes that TOML as cluster.settings:

[outbound]
enabled = true
hosts = ["echo.outbound.test"]
[outbound.testing]
echo = true # .test hosts are answered in-process
[outbound.testing.secrets.ECHO_KEY]
value = "s3cr3t-e2e"
packages = ["com.calimero.outbound-probe"]
signers = ["did:key:z6Mk…"]
hosts = ["echo.outbound.test"]

The echo answers 200 with the request the node sent, as JSON: method, URL, path, query, headers, body and the number of secrets filled in. It answers only after every gate has passed, and its answer is redacted like any other, so a scenario sees exactly what left the node. apps/outbound-probe and its scenarios (scenarios/cluster/outbound.json, and outbound-off.json for a cluster with outbound off) cover every gate this way:

Terminal window
cargo test -p calimero-node --test scenarios -- outbound-probe

A component app (calimero:app, the v2 SDK) sends from a call, a timer or a delivered xcall with env::fetch, through the same gates. The request goes out once, from the leader, and the call’s log entry records it and the response, so a retry with the same idempotency key replays the answer instead of fetching again (what a single-use token like Turnstile’s needs):

let request = Request::post(
"https://challenges.cloudflare.com/turnstile/v0/siteverify",
format!("secret={{{{secret:TURNSTILE_SECRET}}}}&response={token}"),
)
.header("Content-Type", "application/x-www-form-urlencoded");
let passed = env::fetch(&request).is_ok_and(|r| r.status == 200 /* and success */);

The node waits at most 1.5 s for the answer; a timeout or a refusal comes back as an error the call handles. Example: apps/v2-fixtures/captcha (verifyTurnstile), a captcha service other apps call synchronously (Synchronous xcalls). Protocol detail: Outbound HTTP from component calls.

  • A request never runs in a view. Read the answer back from state.
  • Redaction removes the secret as sent. A service that answers with it encoded (base64, URL-escaped) defeats it, so bind a secret only to hosts that do not echo it.
  • The response is not authenticated beyond TLS: a service you call can answer anything, so call only services you trust for the decision you store.