Home/System Overview

System Overview

High-level architecture, data flow, and component relationships

Click
CLI framework
Docker
primary backend
YAML
workflow format
asyncio
runtime

Architecture Diagram

The complete data flow from CLI invocation through workflow execution to node management and external integrations.

User / CI Click CLI merobox run workflow.yaml · merobox stop · merobox health Config Parser YAML load · ${ENV_VAR} expansion · validation WorkflowExecutor nuke → image_pull → sandbox_start → contract_deploy → node_start → remote_register → steps → cleanup Manages workflow_results dict, placeholder resolution, step factory NodeResolver remote → URL → Docker → binary StepFactory BaseStep subclass instantiation AuthManager JWT tokens · /auth/token · disk cache RemoteNodeManager Register nodes to portal DockerManager Container lifecycle Traefik · auth stack · ports BinaryManager Native subprocess PID files · signal cleanup SandboxManager near-sandbox binary RPC · NearDevnetClient calimero-client-py JSON-RPC client Async HTTP · auth integration Docker Daemon merod Binaries NEAR Sandbox RPC merod JSON-RPC
CLI / Config
Workflow / Client
Resolution
Docker
Binary / Auth
NEAR

Execution Flow

When you run merobox run workflow.yaml, the following sequence occurs:

1

Parse & Validate

YAML file is loaded, environment variables (${ENV_VAR}) are expanded, and the workflow schema is validated. Node definitions and step sequences are extracted.

2

Nuke (optional)

If the workflow or CLI flags request it, all existing Docker containers and volumes matching the merobox prefix are removed. Binary processes are killed via PID files.

3

Image Pull

Docker images for merod are pulled (or verified to exist locally). The image tag comes from the workflow YAML or defaults.

4

NEAR Sandbox Start

If the workflow uses NEAR integration, SandboxManager downloads (if needed) and starts a local near-sandbox process. Waits for RPC readiness.

5

Contract Deploy

Smart contracts referenced by the workflow are resolved (path or URL), compiled if needed, and deployed to the NEAR Sandbox via NearDevnetClient.

6

Node Startup

NodeResolver determines the backend for each node. DockerManager creates containers (with Traefik routing) or BinaryManager spawns native merod processes. Config TOML is generated per node.

7

Remote Registration

If remote nodes are declared, RemoteNodeManager registers them via the portal API. AuthManager handles JWT authentication for each node.

8

Step Execution

Steps run sequentially (or in parallel for parallel steps). Each step resolves placeholders, executes via calimero-client-py JSON-RPC, and stores results in workflow_results.

9

Cleanup

On success or failure, cleanup runs: containers are stopped/removed, binary processes are killed, sandbox is shut down, and temporary files are cleaned.

Component Responsibilities

WorkflowExecutor

  • Orchestrates the full lifecycle from YAML to completion
  • Manages workflow_results dictionary
  • Resolves placeholders and dynamic values between steps
  • Handles step factory for creating step instances
  • Coordinates cleanup on both success and failure paths

NodeResolver

  • Determines backend for each declared node
  • Resolution chain: registered remote → URL → Docker → binary
  • Integrates AuthManager for authenticated endpoints
  • Integrates RemoteNodeManager for portal registration
  • Returns resolved node endpoints for step execution

AuthManager

  • Manages JWT authentication tokens per node
  • Calls /auth/token and /auth/refresh endpoints
  • Implements disk-based token cache for persistence
  • Handles automatic token refresh on expiry

Config System

  • YAML workflow files with ${ENV_VAR} expansion
  • TOML config for merod node configuration
  • Bootstrap node injection (apply_bootstrap_nodes)
  • E2E defaults (apply_e2e_defaults)
  • NEAR devnet config (apply_near_devnet_config_to_file)

External Dependencies

calimero-client-py

JSON-RPC client library for communicating with merod nodes. Used by steps to execute methods, query state, and manage contexts.

library

Docker SDK (docker-py)

Python Docker API for container lifecycle management. Used by DockerManager for creating, starting, stopping, and removing containers.

library

aiohttp

Async HTTP client/server framework. Used for health checks, sandbox RPC communication, and async workflow execution.

library

py-near

Python NEAR SDK for account management, key operations, and contract interactions with the NEAR sandbox.

library

Click

CLI framework providing command groups, argument parsing, options, and help generation for all merobox commands.

cli

PyYAML

YAML parsing library for reading workflow definition files and configuration.

library