# Calimero Network > Calimero Network is a framework which enables building fully decentralized applications, ensuring everyone's data privacy. This file contains all documentation content in a single document following the llmstxt.org standard. ## What is Calimero ## What is Calimero Network Calimero is a development framework that enables building and running peer-to-peer Self-Sovereign applications focusing on data ownership and harnessing the power of verified off-chain computing. ### Framework Architecture The Calimero framework is built around several key components that work together to create a seamless development experience: **Client Nodes**: These are the core participants in the network - any device or computer that runs Calimero applications. Each node acts as a gateway that runs applications and connects with other peers, encapsulating the complexities of decentralized networking. **WebAssembly (WASM) Applications**: Applications in Calimero are compiled to WebAssembly format, ensuring secure, isolated execution. Each application runs in its own sandboxed environment within the WASM virtual machine, preventing interference between different applications while maintaining high performance. **Contexts**: The heart of the Calimero ecosystem, contexts are application-specific networks that enable direct communication between users. Each context consolidates all necessary components into a secure, isolated environment where participants only synchronize with each other, not the entire network. **Runtime Environment**: The framework provides a robust runtime that manages multiple applications concurrently, handles state synchronization, ensures atomic transactions, and provides scoped storage for each context. This runtime abstracts away the complexity of distributed systems from developers. **Development SDKs**: Calimero provides comprehensive development tools: - **Protocol SDK**: For building the core application logic in Rust, defining how applications behave and communicate - **Client SDK**: For creating user interfaces in TypeScript, handling authentication and data interactions **Specialized Nodes**: For applications requiring additional capabilities, specialized nodes provide services like encrypted storage, delegated execution, and administrative functions while maintaining the decentralized nature of the network. ## Core Principles Calimero is built on the principle of **local-first execution with eventual consistency**. This means that each actor can independently drive their own local state, and the network will eventually reconcile to the same state. This approach pegs scalability to the number of actors in the network, making executions practically instantaneous. ### Context-Aware State Reconciliation All activity in Calimero revolves around an instance of an app, called a context. Participants of a context only synchronize with each other, not the entire network. This eliminates the need for global consensus and gas fees, with execution costs borne directly by the actor. ### Inherent Privacy Calimero leverages direct-actor relationships to encrypt all network traffic between participants, making the network inherently private. Unlike blockchains that achieve integrity through transparency, Calimero enables private environments while maintaining data integrity. ### Embeddable Architecture Calimero is designed to be embeddable, meaning that client apps themselves become the network actors, and the network is the app itself. This creates a seamless integration where the framework becomes part of your application rather than a separate entity. ## Open Web tech stack Calimero is designed to complement and enhance existing technologies rather than replace them: ### Blockchain Integration While Calimero offers an alternative to traditional blockchain approaches, it can also integrate with blockchains for specific use cases. The framework can leverage blockchain for settlement, identity verification, or as a source of truth when needed, while handling the majority of application logic locally. ### Storage Solutions Calimero includes built-in encrypted data storage capabilities, but can also integrate with existing storage solutions like IPFS when additional storage requirements arise. The framework focuses on compute while providing flexible storage options. ### Zero-Knowledge (ZK) Enhancements ZK solutions can complement Calimero by providing enhanced privacy features within its framework. This combination offers the best of both worlds - Calimero's local-first execution with ZK's privacy guarantees - without inheriting the synchronization complexities of traditional ZK implementations. The rest of this documentation explores how to use Calimero with these and other technologies to build powerful self-sovereign applications. --- ## Use Cases There are a lot of use cases where Calimero can be used and here are some of them: ## Private DAOs Management Migrate sensitive on-chain and off-chain data into a DAO-owned private Calimero application, enabling you to interact with the base chain of the DAO. DAO members keep the core contract logic on-chain, while shielding all the sensitive data. ## Decentralized Voting Systems Enhance trust in voting by creating a secure and tamper-proof voting process application, ensuring each vote is counted accurately and cannot be altered. This technology enhances trust in voting by providing verifiable and immutable records while protecting voter privacy. It empowers voters, reduces the risk of fraud, and promotes democratic participation through a transparent and secure voting platform. ## Decentralized Social Platforms Build social experiences which need scalable but private data like social groups, direct messages and others, while eliminating high transaction costs (gas fees associated with blockchains). This can include exclusive groups and gated communities based on ownership of NFTs or token payments which would allow participation and decryption of the community data content. ## Decentralized Productivity (DeProd) SaaS Create a decentralized productivity tools suite by self hosting your data on Calimero’s private applications and move away from the control of centralized entities. The new platform would mitigate the burden of lifetime deals and hard migration processes for the current users of Productivity SaaS solutions. ## Decentralized Gig Economy Disrupt the current gig economy players and compose a fully decentralized ride sharing, freelancing, temporary work, mechanical turk platform, or any other shared economy model where each network participant gets their fair share of the profits. ## Trustless Gaming Resolve the transparency and verification problems in games with turn mechanics (i.e. Battleship), and in those where private data holds significant importance (i.e. poker). ## Decentralized Edge Compute Most of the general compute today happens on cloud service because that is where most of the data lives. With Calimero, all the applications data is stored locally, whereas Calimero enables edge compute on that data. Combined with the use of decentralized AI and Edge Inference LLMs, there are numerous use cases how Calimero can enhance user experience: Edge AI on users data for social spam filtering, e-commerce shopping recommendations across channels, to name a few. This would allow users to finally maximize the endless possibilities from their data, on their own rules. For example, when you used to purchase from a certain merchant, the data was fully controlled by that merchant or the platform where the data was hosted, which would yield to your recommendations being exclusively tied to that merchant. But if the data would have lived on your device, your data recommendations would have been across various market channels, and not exclusively tied to that specific merchant. ## Decentralized Identity Verification Create a private Calimero application network which provides transparent and trusted identity verification for service providers. With Calimero, sensitive procedures such as opening a bank account or applying for Visas would not require service providers to host the data of their users at centralized entities any more. ## Decentralized Intellectual Property Management The traditional platforms for managing and enforcing intellectual property rights are not fully transparent, resulting in creators not being able to maximize the utility of their work. Developing a decentralized IP management platform would result in a fairer compensation system for all creators. --- ## Key Features ### No Central Authority The networks deployed on Calimero are fully decentralized and managed by the network participants. You own your data! ### Client Side Verified All actions inside the Self-Sovereign Apps are verified using identity off-chain signatures on the client device to ensure validity. ### Strong Encryption The peer interactions inside the Self-Sovereign Apps are encrypted and protected from any unauthorized third parties. ### Local Data Vault The data is stored on the user's device and all state transitions are applied locally ensuring data ownership. ### Developer Friendly Build privacy preserving applications in any language without the knowledge of cryptography and networking. ### Cross Platform Support Calimero is built using a WASM runtime allowing it to be run on different platforms including servers, desktops, browsers and mobile. ### Sovereign Digital Identity Unique digital identity for every application, guarded by your wallet of choice or a on-device private key. --- ## Setup import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; # Setup This guide will help you set up Calimero in your device. 1. If you want to build on Calimero, or just want to use Calimero apps, [**choose one of these recommended options**](./packaged). 2. If you want to contribute to Calimero framework, or do some of our paid [bounties](https://calimero.network/bounties) - [**build it from Source**](./build-from-source). ### Supported Platforms Calimero currently supports the following operating systems and architectures: - **Operating Systems**: macOS, Linux - **Architectures**: x86_64, arm64 (Apple Silicon) :::note If your platform is not supported, please [create an issue on GitHub](https://github.com/calimero-network/core/issues). ::: --- ## Packaged installation import MerodInstallation from '../shared/install-merod.mdx'; ### Installation {#installation} ### Next Steps Initialize and run node If this is your first time initializing the node, the process may take a little bit more time until all dependencies are downloaded. It will take around 7 minutes but can vary depending on your internet connection. #### 1. Initialize node ```bash title="Terminal" merod --node-name node1 init --server-port 2428 --swarm-port 2528 ``` Node configuration file contains protocol defined metada and is located at `~/.calimero/your_node_name/config.toml`. #### 2. Run node ```bash title="Terminal" merod --node-name node1 run ``` Wait for a few moments and node logs should appear. --- ## Build From Source import InitNode from '../shared/node-init-source.mdx'; import RunNode from '../shared/node-run-source.mdx'; ## Setup For Contributors If you want to contribute to the Calimero framework or customize its functionality, you can build it from source. ### Prerequisites - Ensure Rust is installed. If not, follow the instructions on the [Rust website](https://www.rust-lang.org/tools/install). ### Troubleshooting - If Rust dependencies fail, ensure your Rust installation is up-to-date: ```bash rustup update ``` - Verify that you have all required permissions to run the node. For further support, open an issue on [GitHub](https://github.com/calimero-network/core/issues). ### Setup steps Setup consists of few steps and is around 9 minutes long. (mostly waiting for project dependencies to download) 1. Clone repository from GitHub (1min) 2. Position in the root of the project ( ```bash title="Clone using HTTPS" git clone https://github.com/calimero-network/core.git ``` ```bash title="Clone using SSH" git clone git@github.com:calimero-network/core.git ``` ### 2. Position in the root of the project ```bash title="Terminal" cd core ``` ### 3. Initialize and run node You can do it in two ways. - Using CLI - Using Docker compose If this is your first time initializing the node, the process may take a little bit more time until all dependencies are downloaded. It will take around 7 minutes but can vary depending on your internet connection. #### 1. Initialize node #### 2. Run node #### 1. First build the image ```bash title="Terminal" docker buildx build -t 'merod' . ``` #### 2. Then setup and run the nodes defined in `docker-compose.yml` file ```bash title="Terminal" docker compose up ``` #### 3. Connect to the available node by listing running containers ```bash title="Terminal" docker ps ``` and then attach to the selected container ```bash title="Node Terminal" docker attach core-app_node_run-1 ``` Wait for a few moments and node logs should appear. --- ## Run as a systemd service # Run as a systemd service When you want your Calimero node to run continuously on a server or system, you can set it up as a systemd service. This ensures automatic startup, restart on failure, and proper logging. :::note This guide assumes you're using the packaged Calimero installation. For systemd services, packaged binaries are recommended over building from source. ::: ## Create systemd service file Create a new systemd service file: ```bash title="Terminal" sudo nano /etc/systemd/system/calimero-node.service ``` Add the following content (adjust paths and node name as needed): ```ini title="/etc/systemd/system/calimero-node.service" [Unit] Description=Calimero Node Service After=network.target Wants=network.target [Service] Type=simple User=calimero Group=calimero WorkingDirectory=/home/calimero ExecStart=/usr/local/bin/merod --node-name node1 run Restart=always RestartSec=10 StandardOutput=journal StandardError=journal SyslogIdentifier=calimero-node # Security settings NoNewPrivileges=true PrivateTmp=true ProtectSystem=strict ProtectHome=true ReadWritePaths=/home/calimero/.calimero # Resource limits LimitNOFILE=65536 LimitNPROC=4096 [Install] WantedBy=multi-user.target ``` :::tip The default path `/usr/local/bin/merod` assumes a standard packaged installation. If you installed Calimero to a different location, adjust the `ExecStart` path accordingly. ::: ## Create dedicated user (recommended) For security, create a dedicated user to run the Calimero service: ```bash title="Terminal" sudo useradd -r -s /bin/false -d /home/calimero calimero sudo mkdir -p /home/calimero/.calimero sudo chown -R calimero:calimero /home/calimero/.calimero ``` ## Set up the service 1. **Reload systemd configuration:** ```bash title="Terminal" sudo systemctl daemon-reload ``` 2. **Enable the service to start on boot:** ```bash title="Terminal" sudo systemctl enable calimero-node.service ``` 3. **Start the service:** ```bash title="Terminal" sudo systemctl start calimero-node.service ``` 4. **Check service status:** ```bash title="Terminal" sudo systemctl status calimero-node.service ``` ## Service management commands ```bash title="Terminal" # Stop the service sudo systemctl stop calimero-node.service # Restart the service sudo systemctl restart calimero-node.service # View logs sudo journalctl -u calimero-node.service -f # View recent logs sudo journalctl -u calimero-node.service --since "1 hour ago" ``` ## Configuration considerations - **Ports**: Ensure the ports specified in your node configuration (default: 2428 for server, 2528 for swarm) are open in your firewall - **Storage**: The service will store data in `/home/calimero/.calimero/` by default - **Logs**: All logs are captured by systemd journald and can be viewed with `journalctl` - **Updates**: When updating Calimero, restart the service with `sudo systemctl restart calimero-node.service` ## Troubleshooting If the service fails to start, check: 1. **Service status:** ```bash title="Terminal" sudo systemctl status calimero-node.service ``` 2. **Detailed logs:** ```bash title="Terminal" sudo journalctl -u calimero-node.service -n 50 ``` 3. **Common issues:** - Permission errors: Ensure the `calimero` user owns the `.calimero` directory - Port conflicts: Check if ports are already in use with `netstat -tulpn | grep :2428` - Configuration errors: Verify your `config.toml` file is valid - Binary path: Ensure the path in `ExecStart` points to your `merod` binary (usually `/usr/local/bin/merod` for packaged installations) :::tip For server environments, consider using a reverse proxy (like nginx) in front of your Calimero node and implementing proper SSL/TLS termination. ::: :::note The systemd service configuration above is optimized for security and reliability. Adjust the paths, user, and resource limits according to your specific environment and requirements. ::: --- ## Terminology As projects grow, it's important to have a shared vocabulary to help communicate effectively. This page provides a list of terms used in the documentation and throughout the codebase. - **Node** is any individual device or computer that participates in the network. To avoid confusion with network layer used in the protocol, instead of network we are using term **Context**. - **Peer** is a specific instance of a node within a P2P network that interacts with other peers. Peer represents user. - **Peer Id** is a unique identifier assigned to each peer in the network. It is used to distinguish between different peers and ensure that messages are delivered to the correct recipient. - **Context** is the core of the Calimero ecosystem. It is an application specific network designed to enable direct communication between users, eliminating the need for intermediaries - **Application** is a software program designed to perform specific tasks or solve particular problems. To ensure compatibility and functionality, it should be developed according to the protocol SDK instructions provided. Once developed, the application should be published in a format that others can use during runtime, specifically in WebAssembly (WASM) format. Developer can also build frontend for an application, deployed separately, allowing users to interact with an app directly. This user interface facilitates interaction with the underlying software, making the application accessible and user-friendly. - **Root key** is the public part of a wallet cryptographic key pair used to verify the signature of sensitive actions. This public key is used to ensure that any data or actions signed with the corresponding private key can be trusted. Essentially, the root key serves as a trust anchor, enabling users to validate the authenticity and integrity of operations or communications associated with the node. It does not grant direct control over the node but ensures that actions authenticated with the private part of the root key are legitimate. - **Client key** is a cryptographic key tailored for each user session, acting as a session key or token. Each client key must be signed with the root key to be valid which is done automatically during login. This ensures that only sessions authenticated by the trusted root key can interact with the node. - **Specialized node** is third-party node that augment a context's capacity and reliability. It participates in a context but have additional capabilities, providing various services while maintaining the decentralized nature of the network. --- ## Identity ## Calimero Decentralized Identity Calimero’s decentralized identity framework is designed to provide secure, anonymous, and decentralized control across the network. It leverages multiple types of cryptographic keys to manage identities and ensure the integrity of operations within the network. ### Network identity Network identity is defined within the configuration file on each node. This identity is derived from [network keys](#network-keys), which generate a unique `Peer ID` for each node, allowing peers to identify themselves to other peers in the network. #### Network Keys Network keys are used to authenticate nodes within the network, with each node having a unique private-public key pair that it uses to verify its identity to other nodes. During connection setup, nodes exchange public keys to ensure they are communicating with the intended parties. More in [Network](../architecture/node/network#identify) ### Context identity Context identity allows users to identify themselves within a specific context. Users can have arbitrary number of identities within a specific context which remains encapsulated within the context. These identities are linked to [runtime keys](#runtime-keys), with each context identity connected to one runtime key. ### Decentralized Identity (DID) document All context-specific identities and node keys are managed through a Decentralized Identity (DID) document stored on the node. This document includes: - [Root Keys](#root-keys) - [Client Keys](#client-keys) - [Runtime Keys](#runtime-keys) - [Encryption Keys](#encryption-keys) Calimero's key management utilizes multiple keys for various purposes, ensuring secure, anonymous, and decentralized control across the network. This structure underpins secure and efficient operations within the Calimero Network, facilitating both node and application functionalities. #### Root keys Root keys provide full access for node operations, including managing root keys, listing identifiers, and generating new client keys. Web3 wallets can be used as fundamental for root keys, simplifying the setup process. #### Client keys Client keys are generated on login and used by the frontend. Stored in browser local storage, these keys initiate applications and serves as permissioned access to the node. **Client Key Usage**: 1. **Key Creation**: Users generate a new keypair in their browser. 2. **Verification**: - A Verifiable Presentation Request is sent to the node, which responds with a challenge. - The challenge and public key are signed using the root key. - Upon node verification of the request and signature, the new key is cleared for JSONRPC API communication from the browser to the node. Calimero's TypeScript SDK supports developers in building browser and CLI applications by simplifying interaction with the network. #### Runtime keys Runtime keys are used for signing transactions within a context, with the appropriate key selected based on the context identity. #### Encryption keys Keys used for encrypting and decrypting messages between peers in the context. Each context has its own encryption key. More in [Encryption](./encryption) --- ## Contexts Contexts are the core of the Calimero ecosystem. These are application specific networks designed to enable direct communication between users, eliminating the need for intermediaries. Here's a closer look at how they operate: #### **How Contexts Work:** 1. **Initialization**: A user selects a WebAssembly (WASM) module from a repository, which contains the logic and rules for the application. With this, they initiate a new Application Network, creating a unique identity for this specific network and setting the initial parameters and update strategies. 2. **Joining the Network**: - **Inviting Peers**: The initial user can invite others directly or set up Access Control Lists (ACLs) to govern how new members can join the network. - **New Member Process**: When a new user joins, they generate a unique identity for the network, download the relevant WASM module, and synchronize with the existing data on the network. This process ensures they're up to speed and ready to engage fully with the network's activities. 3. **Data Interaction**: Within the network, users can query and modify data according to the application's rules. Each Application Network autonomously manages data interactions, ensuring consistency and integrity. 4. **Security and Privacy**: All communications within an Application Network are secured with end-to-end encryption, utilizing the Double Ratchet Algorithm. This ensures that data exchanged between peers remains private and secure. 5. **Offline Capability and Consistency**: Calimero's design is 'offline-first,' accommodating the reality that peers may not always be online. When a peer goes offline and later returns, they synchronize with the network to update and reconcile any changes, maintaining eventual consistency across the network's state. 6. **Governance and Updates**: Application Networks can be governed by the users themselves, with decisions made through a consensus mechanism. Updates to the network, including changes to the WASM module or network parameters, can be proposed and voted on by the network's members. 7. **Extending network capabilities**: In addition to the core components of Application Networks, Calimero introduces Specialized Nodes to further enhance network capabilities. These nodes are designed to perform specific functions that go beyond the standard operations of client nodes, such as heavy data processing, advanced encryption tasks, or providing additional storage solutions. They play a crucial role in scaling the network's functionality and performance, ensuring that even as demand grows, the network remains efficient and responsive. Specialized Nodes can be deployed by any participant in the network, including Calimero, third-party developers, or users themselves, offering a flexible and decentralized approach to augmenting the network's capabilities. By leveraging these nodes, Application Networks can meet the diverse needs of different applications, ensuring that each network can be customized and optimized for its unique requirements, all while maintaining the overarching principles of privacy, security, and decentralization inherent to Calimero. --- ## Applications Applications in Calimero define the core logic governing how peers interact within a decentralized network. Developers can programmatically augment the protocol to create customized functionalities, supporting a wide range of applications tailored to various use cases, from direct messaging and communication channels to interactive games and collaborative editing. ### Examples of Applications 1. **Communication Platforms** - **Direct Messages and Channels**: Support for private, encrypted messaging between users and group communications in shared, secure spaces. This setup can scale from one-on-one conversations to large group discussions, similar to Slack channels or Discord communities. - **Community Platforms**: Decentralized alternatives to platforms like Reddit or Hacker News, enabling independent contexts for different purposes, fostering discussions, and sharing content securely. 2. **Interactive Games** - **Privacy-Focused Games**: These games require the privacy of each player's moves until both have played, ensuring fair play and maintaining the confidentiality of strategies. Examples include: - **Rock-Paper-Scissors**: Players' choices are revealed simultaneously after both have made their moves. - **Battleship**: The positions of ships are kept secret until revealed through gameplay. - **Non-Privacy Games**: These games do not require concealment of moves and allow all actions to be visible to both players. Examples include: - **Chess**: A strategy game where all moves are visible to both players, with game logic running locally. - **Checkers**: Another strategy game where all moves are open and visible to both players. 3. **Collaborative Work** - **Document Collaboration**: Real-time collaboration on documents, akin to Google Docs, but decentralized and secure. - **Creative Projects**: Shared canvases or environments for drawing, designing, or working on various creative projects, where all contributions are securely encrypted. ### Benefits of Decentralized Applications - **Resilience**: Distributed application logic ensures the system functions smoothly even if some devices go offline. - **Autonomy**: Users control their data and activities without relying on a central authority. - **Scalability**: The system scales effectively as more users join due to optimistic execution, with state being conflict-free and eventually reconciled. - **Privacy**: Keeping data local and encrypted prevents exposure to third parties, crucial for sensitive information. ### Security and Data Management All non-state-transitional data, such as attached files in DMs, collaborative document assets, and game resources, inherit the same level of security as state-transitional transactions. This ensures that all forms of data within the Calimero network are encrypted and secure. Calimero also functions as a decentralized filesystem for these non-state-transitional, encrypted blobs of data. Similar to BitTorrent or IPFS, nodes can lazily share the data without needing any centralized storage options. This decentralized approach allows for efficient and secure data distribution across the network. ### Building Applications Developers can leverage Calimero's framework to programmatically build and augment a wide range of applications using provided tools and documentation. This includes setting up the development environment and deploying applications within the network. 1. **Development Tools** Calimero offers the Rust SDK and intuitive APIs, enabling developers to create applications that integrate seamlessly with the network. 2. **Documentation and Support** Comprehensive documentation and community support assist developers in every step of the application development process, ensuring that they can build and deploy high-quality, secure applications efficiently. --- ## Blockchains Integration import InitNode from '../shared/node-init-binary.mdx'; # Blockchain Integrations Calimero offers integration with multiple blockchain protocols, allowing users to interact with the blockchain and perform read and write operations. ## Current Blockchain Integrations Calimero currently supports the following blockchain protocols: - NEAR - Starknet - Internet Computer (ICP) - Stellar - Ethereum ```mermaid graph TD A[Calimero] --> B[Integrations] B --> C[NEAR] B --> D[Starknet] B --> E[ICP] B --> F[Stellar] B --> G[Ethereum] ``` ## Operations - Read - Read the data from the Calimero contracts. - Write - Write data to any contract on the same blockchain protocol through cross-contract calls in the Proxy Contract. ```mermaid graph TD A[Calimero] B[Context Contract] C[Proxy Contract] D[Transfers] E[Cross-Contract Calls] A B A C C --> D C --> E ``` ## Functionalities - Cross-contract calls - Transfers - Storage of arbitrary data - Configuration of the internal configuration variables - Proposal creation and execution ```mermaid graph TD A[Proxy Contract] -->|Cross-contract Calls| B[Other Contracts] A -->|Transfers| C[Token Transfers] A -->|Data Storage| D[Key-Value Storage] A -->|Configuration| E[Internal Variables] E -->|Change Votes Required| F[Vote Configuration] E -->|Change Proposal Limits| G[Proposal Limits] ``` ## Functionality Details - **Cross-contract calls**: Proxy contract allows making cross-contract calls to other contracts on the same blockchain protocol. Users need to specify the contract address, the method to call, and the arguments for that method. - **Transfers**: Proxy contract allows transferring tokens to the specified address. Users need to specify the recipient address and the amount of tokens to transfer. - **Storage of arbitrary data**: Proxy contract allows storing arbitrary data in the context storage. Users need to specify the key and the value for the data to be stored. - **Configuration of the internal configuration variables**: Configure the internal configuration variables of the Calimero contracts: - **Vote configuration**: Number of votes required for a proposals to be approved (u32) - **Proposal limits**: Maximum number of proposals that one user can have at the same time(u32) - **Proposal creation and execution**: Proposals are created by users and contain one or multiple actions that are voted on by the users of the context. If a proposal is approved, it is automatically executed by the Calimero Proxy Contract. ## Calimero Contracts Calimero integrates two types of contracts for each blockchain protocol: - Context Contract - Proxy Contract ```mermaid graph TD A[User] -->|Creates Context| B[Context Contract] B -->|Creates| C[Context] B -->|Adds Admin| D[User as Admin] B -->|Deploys| E[Proxy Contract] E -->|Handles| F[Proposals, Votes, Execution] F -->|Executes Actions| G[Blockchain Operations] ``` ### Context Contract The Context Contract is responsible for creating contexts, adding and removing members, editing their privileges, and associating specific applications with a context. ### Proxy Contract Each context has its own Proxy Contract instance which is deployed during the context creation. ```mermaid graph TD A[User] -->|Creates Proposal| B[Proxy Contract] B -->|Stores Proposal| C[Proposal Storage] C -->|Voting by Members| D{Required Votes Met?} D -->|Yes| E[Execute Proposal] D -->|No| F[Store Vote Count] ``` Tutorial on how to use proxy contract functionalities is available in [tutorials section](../../tutorials/proxy-contract-interaction) ### Deployment of Contracts #### Context Contract Context Contract is predeployed on each blockchain by the Calimero team. #### Proxy Contract Proxy Contract is deployed during the context creation. ```mermaid graph TD A[Calimero Team] -->|Pre-deploys| B[Context Contract] B -->|During Context Creation| C[Proxy Contract] C -->|Unique Instance| D[Context-specific Proxy] ``` :::info Proxy Contract is deployed on the same blockchain as the Context Contract. ::: :::warning Users are responsible for funding the Proxy Contract. ::: ### Example Application We have an example application that connects to Calimero Proxy Contract and allows creating proposals, voting on them, and executing the approved proposals. ```mermaid graph TD A[Application] -->|Connects to| B[Calimero Proxy Contract] B -->|Handles| C[Proposals] C -->|Create| D[New Proposals] C -->|Approve| E[Proposal Voting] C -->|Delete| J[Delete Proposal] E -->|Check Votes| F{Required Votes Met?} F -->|Yes| G[Execute Proposal] G -->|Perform Actions| H[Blockchain Operations] F -->|No| I[Store Vote Count] ``` Application showcases all the actions that can be done on the Proxy Contract. To create context on specific blockchain user check out our [Create Context](../06-tutorials/02-create-context.mdx) guide. Full source code is available in our GitHub [repository](https://github.com/calimero-network/demo-blockchain-integrations). --- ## Encryption Encryption in Calimero ensures data security in transit over the network, maintaining confidentiality and integrity. ### Key Principles 1. **Forward Secrecy**: Ensuring past messages remain secure even if a key is compromised in the future. 2. **Post-Compromise Security**: Ensuring future messages remain secure even after any previous message has been compromised. 3. **Zero Trust in Third Parties**: No reliance on intermediaries for security. 4. **Verifiable End-to-End Encryption**: Confirming that only the intended recipients can read the messages. 5. **Asynchronous Communication**: Ability to start communications without recipients being online. 6. **Multi-Device Support**: Ensuring seamless use across multiple devices. 7. **Deniability**: Providing plausible deniability for message authorship to non-context members. 8. **Non-Interactive Group Management**: Adding and removing context members without requiring interaction. ### Double Ratchet Algorithm Each network message uses a distinct encryption key derived from the ratchet state, providing forward secrecy by ensuring that the compromise of one key does not affect the security of previous messages. Each context can configure Diffie-Hellman reset parameters. For one-on-one peer interactions, resets can occur instantaneously, while for larger groups, resets can happen at non-deterministic intervals to balance security and performance. ### Tree-Based Diffie-Hellman Key Exchange All contexts use a tree-based Diffie-Hellman key exchange. This method efficiently manages shared secrets among multiple members, ensuring that keys are updated and propagated correctly. The reset of keys occurs at the leaf nodes of the tree, guaranteeing post-compromise security. Adding a new member involves existing members using their prekeys to complete an X3DH (Triple Diffie-Hellman) exchange, securely adding the new member without requiring direct interaction. Removing a member involves invalidating their keys and updating the shared secrets among remaining members, ensuring efficient and secure updates. By leveraging advanced encryption techniques such as the Double Ratchet Algorithm and tree-based Diffie-Hellman key exchange, Calimero ensures that all data in transit is protected, maintaining the confidentiality and integrity of network messages. --- ## Specialized Nodes Specialized nodes in the Calimero Network are third-party nodes that augment a context's capacity and reliability. They participate in a context but have additional capabilities, providing various services while maintaining the decentralized nature of the network. ### Key Concepts - **Incentivization**: These nodes can be incentivized through contracts on blockchains that pay them for their services, ensuring they remain motivated to perform their roles effectively. - **Permissions and Roles**: Specialized nodes can have different levels of permissions, ranging from being subscribed to encrypted network events to fully participating as part of the context. ### Types of Specialized Nodes 1. **Storage Nodes**: - **Encrypted Transaction Storage**: These nodes store encrypted transactions without the ability to read them. They guarantee 100% uptime, ensuring that all transactions are available even when peers are offline. When peers come back online, the node provides missed transactions and new transactions for replication. - **Blob Storage**: These nodes also store encrypted blobs of non-state-transitional data, ensuring that all necessary data is always available without holding decryption keys. 2. **Delegated Execution Nodes**: - **Context Maintenance**: These nodes fully join a context and maintain the context state on their end. They can handle resource-intensive executions, delegating specific calls to optimize performance and resource utilization. 3. **Administrative Nodes**: - **Event Observation and Action**: These nodes observe real-world events and act on them within the context. For example, in a billionaire's club context, an administrative node could monitor members' accounts on a blockchain and evict them if their balance falls below a certain threshold. ### Reliability Through Decentralization Specialized nodes ensure that the context remains operational and consistent, even when primary peers are offline. They provide the necessary data and transactions to keep the context up-to-date. The decentralized nature ensures that the state is eventually consistent. Fragmentation does not cause issues as the system reconciles itself when peers come back online, highlighting the network's reliability without reliance on any single specialized node. --- ## Node --- ## State Reconciliation --- ## System Overview Calimero Network offers a robust framework for developing and running peer-to-peer (P2P) applications. Our framework allows users to participate in the network or build applications for others to use. ## Participate Users participate in the network with a client node. By encapsulating the complexities of operating a client node, we aim to make it easy and intuitive for everyone to engage in the decentralized world. - Client node acts as a gateway that runs applications and connects with other peers - Each application is loaded and isolated from other applications into a separate context. This ensures that each application runs independently while still allowing interactions through shared states or messages. - Context consolidates all necessary components into a secure, isolated environment. Some networks may require specialized functionalities, which are provided by a dedicated compute market. Users can integrate these special functionalities from a pool of available specialized nodes. ![Calimero Architecture](/learn/architecture.png) ## Build To develop applications on the Calimero Network, we provide comprehensive SDKs: - Protocol SDK to define how the application should behave and communicate with the node. - Client SDK to connect to node and use data in user interface and authenticate using wallets UI. Developer applications are shared through application registry where developers upload their applications and share it with other users Explore other sections to learn more about each component, and how they contribute to a seamless decentralized experience. --- ## Client Node ### Runtime Overview: The runtime environment of a client node in the Calimero Network is crucial for the execution of decentralized applications (DApps), particularly those compiled to WebAssembly (WASM). Functionality: - State Synchronization: Each node can download and synchronize the state with existing applications, ensuring that all nodes participating in a particular application network are consistent and up-to-date. - Application Settings: Nodes can be configured with specific settings for each application, including which WASM modules to run, source URLs for fetching these modules, encryption protocols to be used, and more. - Network Topology & Update Rules: Defines the structure of the network and how nodes communicate and update each other. Proper update rules are crucial for application security and integrity, particularly in a decentralized setting where trust is distributed. #### Recommendations for Developers: Thorough testing of applications in a controlled environment is advised before deploying them in production to ensure stability and security. Additionally, developers are encouraged to implement locked update rules to prevent unauthorized modifications to the application's behavior. ### Storage Overview: Storage on client nodes involves maintaining the state and data required for the decentralized applications they support. Functionality: - Local Storage: Each node stores application data locally, contributing to the overall decentralized storage model of the network. This ensures that data is distributed across the network, enhancing privacy and resilience against central points of failure. ### Encryption: Data stored on client nodes can be encrypted, providing an additional layer of security and privacy for user data. ### Identity Management Overview: Managing identities on the Calimero Network is fundamental for ensuring secure and private interactions between nodes and applications. Functionality: ### Authentication Nodes implement mechanisms for authenticating users and applications, ensuring that interactions are secure and that entities are who they claim to be. ### Key Management The management of cryptographic keys is an integral part of identity management, enabling secure communication and data encryption across the network. ### Application Marketplace Current State: The marketplace for decentralized applications within the Calimero Network is facilitated by a smart contract on the NEAR blockchain, with application data and metadata hosted on IPFS. This setup serves as a temporary solution while further community engagement and discussions are underway to refine the marketplace's infrastructure and governance. The Calimero Network's approach to client nodes emphasizes security, decentralization, and privacy, with a strong recommendation for users to engage with applications that have securely locked update mechanisms. These applications are more reliable for critical use cases and are the only ones featured in the official marketplace, ensuring a curated and trustworthy selection of DApps for users. This framework demonstrates Calimero Network's commitment to building a secure and user-centric decentralized ecosystem. --- ## Runtime The runtime environment in the Calimero Network is essential for executing decentralized applications (DApps). It acts as a bridge between the application logic, the network, and storage layers, ensuring seamless operation and integration. The runtime ensures secure, isolated, and efficient execution of applications by managing resources effectively, supporting real-time event handling, enabling scalability, and providing robust storage and transaction management. ### Core Capabilities - **Security and Isolation**: The runtime provides a secure execution environment for Calimero applications using WebAssembly (WASM). Each application is sandboxed in the WASM VM, ensuring proper isolation and preventing interference between applications. This setup also ensures that applications cannot access unauthorized resources, maintaining a secure environment. - **Multi-Application Support**: The runtime allows multiple applications to run concurrently on the same node and supports multiple instances (contexts) of the same application, each with its own state. This capability enhances the flexibility and scalability of the network. - **Scoped Storage**: The runtime manages storage by partitioning it and governing where each context stores its state. These implementation details are abstracted from the app developer, ensuring that storage management is handled seamlessly and securely. - **Atomic Transactions**: The runtime guarantees atomic transactions, ensuring that if a transaction fails, it is completely rolled back with no state updates or side effects detected. This guarantees consistency and reliability in the application's state and any connected clients. - **Log Collection and Relaying Events**: The runtime facilitates log collection and relays events emitted by the applications to connected clients, enabling real-time monitoring and interaction. - **Resource Management**: The runtime defines resource limits for applications to ensure fair usage and prevent malicious behavior. This includes limiting CPU, memory, and network usage to prevent any single application from monopolizing system resources or compromising the host system. - **Task Management and Performance**: The runtime keeps track of WASM instances up to a defined threshold, effortlessly queueing transactions to reuse live instances and shutting down stale ones to reclaim system resources. These optimizations ensure efficient resource utilization and improved performance. --- ## Server The server is a core component for interacting with a Calimero node. Calimero can be embedded with your client to make it a self-contained node, or it can run as a remote node that multiple clients can connect to, allowing centralized state management. ### Core Capabilities - **JSON-RPC API**: Provides a standardized way for clients to query or mutate the state of their counterpart applications on the node, ensuring seamless integration and communication. - **WebSocket Interface**: Allows clients to subscribe to events emitted from applications, enabling real-time reactions to activity triggered by other peers in the network. - **Admin API**: Manages various aspects of the node, including: - **Context Administration**: Create, delete, invite others to contexts, and accept invitations. - **Storage Management**: Track usage, view raw state storage for each context, and view encrypted blobs. - **State Management**: Manually garbage collect state-transitional transactions. - **Network Management**: Manually connect to peers and manage blocklists. - **Application Management**: Manage installed applications, create contexts from applications, delete applications if no contexts are associated, and manually sideload applications. - **Peer Identity Management**: Rotate peer identities without affecting context identities. - **Node Metrics**: Track network bandwidth usage, both total and by context, to manage resource usage effectively. For comprehensive documentation of all available admin API endpoints, see the [Admin API Reference](/developer-tools/admin-api). --- ## Storage The storage component in the Calimero Network is essential for managing and maintaining the data generated and utilized by decentralized applications (DApps). It ensures data integrity, security, and efficient access, enabling seamless operation of applications within the network. ### Core Capabilities - **Generic Storage Interface**: Calimero provides a flexible storage interface that allows app developers to choose their preferred database. By default, Calimero uses RocksDB, but it can also support LevelDB, Sled, TigerBeetle, SQLite, or even cloud storage solutions like S3. - **Context State Storage**: The context state is backed by a Patricia-Trie structure flattened into the key-value map of the datastore. This structure ensures efficient state management and retrieval. - **Data Blobs**: The storage system handles non-state-transitional, encrypted blobs of data, similar to BitTorrent or IPFS. Nodes can lazily share these data blobs without needing centralized storage, ensuring efficient and secure data distribution across the network. By default, the blobstore is the local filesystem, but it can be configured to use any cloud storage option or content-addressed storage like IPFS. - **Data Encryption**: All data stored within the network is encrypted at rest, ensuring that sensitive information remains protected. This includes both state-transitional data and non-state-transitional data like attached files in DMs or collaborative document assets. - **Efficient Data Operations**: The storage component is optimized for quick data operations, ensuring that applications can access, retrieve, and update the data they need promptly. Caching mechanisms are employed to further improve data access speeds. - **Garbage Collection**: The system includes mechanisms for garbage collection using reference counting for trie data, allowing for the cleanup of obsolete or redundant data. This helps in maintaining optimal storage performance and resource utilization. - **Metrics and Monitoring**: The storage component provides detailed metrics on storage usage, including total usage and breakdowns by context. This allows for effective monitoring and management of storage resources. --- ## Network ## Overview This document provides an overview of the networking component of Calimero Network, which is implemented using the `libp2p` library. The network consists of two types of peers: client nodes and boot nodes, each serving distinct roles and utilizing specific protocols to facilitate peer-to-peer communication. Client node is the component which hosts and runs client applications, communicates and shares data between other client nodes. Boot node is the component used for the initial discovery of the peers in the network. ## Node Types ### Client Node - **Deployment:** Can run on any machine - **Protocols Utilized:** - [dcutr](#dcutr-direct-connection-upgrade-through-relay) - [gossipsub](#gossipsub) - [identify](#identify) - [kad](#kademlia-kad) - [mdns](#mdns-multicast-dns) - [ping](#ping) - [rendezvous](#rendezvous) - [relay](#relay) - **Behavior:** - **Configuration:** A client node can be configured to use zero boot nodes. - **External Address:** - **Direct Public External Address:** Nodes with a direct public external address do not require reservation at the relay server. These nodes publish their public external address to the Kademlia DHT, making them directly accessible to other peers. - **Relayed External Address:** Nodes that do not have a direct public external address, typically those behind a NAT or firewall, can obtain a relayed external address by requesting a reservation at a relay server. Once the reservation is accepted, the node publishes its new external address to the rendezvous server. This allows other nodes to discover relayed addresses of a peers in a certain rendezvous namespace. The relay server can be used for the coordination of the hole punching between two nodes. If the hole punching attempt fails, the relay server will bridge the traffic. - **Discovery Protocols:** `mDNS`, `rendezvous` and `Kademlia` - **Connection Management:** A peer, identified via PeerId, can be discovered either via mDNS, rendezvous or Kademlia. mDNS discovery provides local network addresses, rendezvous discovery provides relayed addresses, and Kademlia discovery provides direct public external addresses. The node maintains information about its connections to peers, including the discovery source. For a discovered external address, either relayed or direct public, the node will only attempt to dial the peer if the same peer is not already connected via a discovered local address. This ensures that local connections have higher priority and that there are no unnecessary hole punching attempts. - **Message Relaying:** The node participates in the `gossipsub` protocol, relaying messages to all connected peers that support it. This enables efficient and scalable message dissemination across the network. ### Boot Node - **Deployment:** Must run on a publicly available machine with a static IP address. - **Protocols Utilized:** - [identify](#identify) - [kad](#kademlia-kad) - [ping](#ping) - [rendezvous](#rendezvous) - [relay](#relay) - **Behavior:** - **Characteristics:** Boot nodes are publicly available, long-running nodes that provide stable entry points to the network. - **Functions:** - **Bootstrap Node:** Acts as a well-known peer for the Kademlia protocol, facilitating peer discovery and network join operations. - **Circuit Relay Server:** Serves as a generic relay that provides the medium that facilitates the hole punching, enabling peers to establish direct connections even when they are behind NAT or firewalls. The relay server is used for the coordination of the hole punching between two nodes, and briding traffic if the hole punching attempt fails. - **Rendezvous Server:** Facilitates peer discovery by allowing nodes to register their presence and query for other peers within a shared rendezvous namespace. This enables dynamic and efficient peer-to-peer connections without relying on a static list of peers. ## P2P protocols and techniques ### Protocol Descriptions #### DCUtR (Direct Connection Upgrade through Relay) - DCUtR is used to upgrade connections through relay nodes, allowing peers to establish direct connections even if they are behind NATs or firewalls. Peers initially connect via a relay node, then use the DCUtR protocol to attempt a direct connection, which reduces latency and bandwidth usage. - **Reference:** [libp2p DCUtR Documentation](https://github.com/libp2p/specs/blob/master/relay/DCUtR.md) #### Gossipsub - Gossipsub is a scalable and efficient pub-sub protocol for message dissemination. It combines the best aspects of gossip protocols and topic-based pub-sub systems. It minimizes bandwidth usage by only gossiping metadata and ensuring that messages are only sent once per peer. - **Reference:** [libp2p Gossipsub Documentation](https://github.com/libp2p/specs/tree/master/pubsub/gossipsub) #### Identify - The Identify protocol allows peers to identify themselves and share their capabilities with other peers. Peers exchange identification information such as supported protocols, listen addresses, and public keys. This helps peers make informed decisions about connecting and interacting. - **Reference:** [libp2p Identify Documentation](https://github.com/libp2p/specs/blob/master/identify/README.md) #### Kademlia (Kad) - Kademlia is a distributed hash table (DHT) protocol used for peer discovery and data routing. It uses an XOR metric to ensure efficient and scalable peer lookup. Each node maintains a routing table with information about other nodes, facilitating quick lookups and robust network operation. - **Reference:** [libp2p Kademlia DHT Documentation](https://github.com/libp2p/specs/tree/master/kad-dht) #### mDNS (Multicast DNS) - mDNS enables local network peer discovery without the need for a central server. It uses multicast DNS to allow peers to find each other on the same local network by broadcasting their presence and listening for broadcasts from other peers. - **Reference:** [libp2p mDNS Documentation](https://github.com/libp2p/specs/tree/master/discovery/mdns) #### Ping - The Ping protocol measures the round-trip time (latency) between peers. It regularly pings connected peers and measures the time it takes for a response. This helps in maintaining healthy connections and understanding network latency. - **Reference:** [libp2p Ping Documentation](https://github.com/libp2p/go-libp2p-ping) #### Relay - The Relay protocol supports relay-based communication, allowing peers to communicate through intermediary nodes when direct connections are not possible. Nodes can use relay nodes to forward their traffic, which is especially useful for nodes behind NATs or firewalls. The protocol includes mechanisms for reserving relay slots and managing relay connections. - **Reference:** [libp2p Relay Documentation](https://github.com/libp2p/specs/tree/master/relay) #### Rendezvous - The Rendezvous protocol enables peers to discover each other by registering at and querying a shared rendezvous point. This is useful for dynamically finding peers without needing a central directory or pre-established list of peers. Peers register their presence at a rendezvous server and can also query the server to find other peers. - **Reference:** [libp2p Rendezvous Documentation](https://github.com/libp2p/specs/tree/master/rendezvous) ### NAT Traversal Techniques One of the common techniques used for NAT traversal in P2P networks is **Hole Punching**. This technique allows two peers, each behind a NAT, to establish a direct connection with each other. Here's a brief explanation: - **Hole Punching:** This technique involves three steps: - **Step 1 - Connection to Public Server:** Both peers initially connect to a public server (in this case, the relay server). This creates a NAT mapping (a "hole") for outgoing packets to the server. - **Step 2 - Exchange of Address Information:** The server shares the public address information of each peer with the other. This information includes the IP address and port number that the NAT has assigned for the connection to the server. - **Step 3 - Direct Connection:** Each peer sends a packet to the other peer's public address. Since a mapping for this address already exists in the NAT (from the connection to the server), the NAT forwards the packet to the appropriate internal address, and a direct connection is established. This technique is particularly useful in P2P networks, as it allows peers to communicate directly, reducing the load on relay servers and improving network efficiency. However, it's worth noting that hole punching may not work with all types of NATs, and success can depend on the specific NAT implementation and configuration. - **Reference:** [Hole punching in libp2p](https://blog.ipfs.tech/2022-01-20-libp2p-hole-punching/) - **Reference:** [How NAT traversal works](https://tailscale.com/blog/how-nat-traversal-works) --- ## Developer Tools We provide different tools for building decentralized applications. ### CLI Tools for interacting with the node and its functionalities You can use CLI to interact with the node using the terminal: - **[Merod](./CLI/merod)**: CLI tool for managing nodes. - **[Meroctl](./CLI/meroctl)**: Tool for performing various operations with the node such as context creation or app installation. - **[Merobox](./merobox)**: Advanced CLI tool for managing Calimero nodes in Docker, automating workflows with YAML configuration, and integrating with testing frameworks. ### SDKs for building the application Multiple SDKs are available for different development needs: [Protocol SDK](./SDK/protocol-sdk/protocol-sdk): Used for building WebAssembly (WASM) application logic in Rust. Provide macros for identifying different application components such as state, logic and events. [Client SDKs](/developer-tools/SDK/client-sdk): Enable frontend and backend development: - **[TypeScript Client](/developer-tools/SDK/client-sdk/client-ts-sdk)**: Enables frontend development in TypeScript, supporting wallet authentication and context data handling via JSON-RPC and WebSockets. - **[Python Client](/developer-tools/SDK/client-sdk/calimero-client-py)**: High-performance Python client library built with Rust and PyO3, perfect for backend services, data analysis, and automation workflows. ### Apps for node administration throught the UI - [Admin dashboard](./apps/admin-dashboard) is web interface for node admin api. - [Desktop application](./apps/desktop-app) is a desktop interface for the node where node is available as a service on the local machine. ### Examples We have few repositories that demonstrate how SDK can be used. Check our [GitHub](https://github.com/calimero-network) --- ## Merod import MerodInstallation from '../../shared/install-merod.mdx'; Merod is a command-line tool that allows you to initialize, configure, and run Calimero nodes. Installation ## Usage ```bash title="Terminal" merod [OPTIONS] --node-name ``` ## Examples with Real Values ```bash title="Terminal" # Basic usage with a node named "node1" merod --node-name node1 init # Using a different node name merod --node-name my-production-node run # With custom home directory merod --home ~/.calimero-custom --node-name node1 init # With additional options merod --node-name node1 init --server-port 3000 --swarm-port 4000 ``` ## Parameter Reference | **Parameter** | **Description** | **Real Examples** | | ------------- | --------------------------- | ------------------------------------------------ | | `` | Name of the node to manage | `node1`, `prod-node`, `test-node` | | `` | Subcommand to execute | `init`, `config`, `run`, `relay` | | `` | File system path | `/path/to/node`, `./my-node`, `~/.calimero` | | `` | Network host address | `127.0.0.1`, `0.0.0.0`, `143.34.182.202` | | `` | Network port number | `2428`, `2528`, `3000`, `63529` | | `` | Remote service URL | `http://relayer.com`, `https://api.example.com` | | `` | Network identifier | `calimero-dev`, `ipfs` | | `` | Blockchain protocol | `near`, `starknet`, `icp`, `stellar`, `ethereum` | | `` | Network address | `127.0.0.1:2428`, `192.168.1.100:2528` | | `` | Uniform Resource Identifier | `0.0.0.0:63529`, `127.0.0.1:63529` | ## Commands ### Initialize Node Configuration **Syntax:** ```bash title="Terminal" merod --node-name init [OPTIONS] ``` **Commands:** - `init` - Initialize node configuration with default or custom settings **Options:** - `--boot-nodes ` - List of bootstrap nodes - `--boot-network ` - Use nodes from a known network [default: calimero-dev] [possible values: calimero-dev, ipfs] - `--swarm-host ` - Host to listen on [default: 0.0.0.0,::] - `--swarm-port ` - Port to listen on [default: 2428] - `--server-host ` - Host to listen on for RPC [default: 127.0.0.1,::1] - `--server-port ` - Port to listen on for RPC [default: 2528] - `--relayer-url ` - URL of the relayer for submitting NEAR transactions - `--protocol ` - Name of protocol [default: near] [possible values: near, starknet, icp, stellar, ethereum] - `--mdns` - Enable mDNS discovery - `--advertise-address` - Advertise observed address - `--rendezvous-registrations-limit ` - Maximum number of rendezvous registrations allowed [default: 3] - `--relay-registrations-limit ` - Maximum number of relay registrations allowed [default: 3] - `--autonat-confidence-threshold ` - Minimum number of successful autonat probes required to be confident about NAT status [default: 2] - `--force` - Force initialization even if the directory already exists **Real Examples:** ```bash # Initialize a node with default settings merod --node-name node1 init # Initialize with custom ports merod --node-name node1 init --server-port 3000 --swarm-port 4000 # Initialize with custom home directory mkdir data merod --home data/ --node-name node1 init # Initialize with specific protocol and network merod --node-name node1 init --protocol ethereum --boot-network ipfs # Initialize with custom swarm host merod --node-name node1 init --swarm-host 192.168.1.100 --swarm-port 2428 # Force initialization merod --node-name node1 init --force ``` ### Configure Node **Syntax:** ```bash title="Terminal" merod --node-name config [ARGS]... ``` **Commands:** - `config` - Configure an existing node with key-value pairs **Arguments:** - `[ARGS]...` - Key-value pairs to be added or updated in the TOML file **Real Examples:** ```bash # Configure server host and port merod --node-name node1 config --server-host 143.34.182.202 --server-port 3000 # Configure swarm settings merod --node-name node1 config --swarm-host 0.0.0.0 --swarm-port 2428 # Configure protocol and relayer merod --node-name node1 config --protocol near --relayer-url https://relayer.near.org # Configure multiple settings at once merod --node-name node1 config --server-host 192.168.1.100 --server-port 8080 --swarm-port 9090 ``` ### Run Node **Syntax:** ```bash title="Terminal" merod --node-name run ``` **Commands:** - `run` - Start and run the configured node **Real Examples:** ```bash # Run a node with default configuration merod --node-name node1 run # Run a node with custom home directory merod --home ~/.calimero-custom --node-name node1 run # Run a production node merod --node-name prod-node run ``` ### Relay Mode **Syntax:** ```bash title="Terminal" merod --node-name relay [OPTIONS] ``` **Commands:** - `relay` - Spin up a relay for external client interactions **Options:** - `-l, --listen ` - Sets the address to listen on [default: 0.0.0.0:63529] [env: PORT] **Real Examples:** ```bash # Start relay with default port merod --node-name node1 relay # Start relay on specific address and port merod --node-name node1 relay --listen 127.0.0.1:8080 # Start relay on custom port merod --node-name node1 relay --listen 0.0.0.0:9000 # Start relay using environment variable PORT=7000 merod --node-name node1 relay ``` ## Global Options **Syntax:** ```bash title="Terminal" merod [OPTIONS] --node-name ``` **Options:** - `--home ` - Directory for config and data [env: CALIMERO_HOME] [default: /Users/anton/.calimero] - `-n, --node-name ` - Name of node - `-h, --help` - Print help - `-V, --version` - Print version **Environment Variables:** - `CALIMERO_HOME` - Directory for config and data - `NEAR_API_KEY` - NEAR API key for blockchain operations **Real Examples:** ```bash # Use custom home directory merod --home ~/.calimero-custom --node-name node1 init # Set home via environment variable export CALIMERO_HOME=~/.calimero-custom merod --node-name node1 init # Check version merod --version # Get help for specific command merod --node-name node1 init --help ``` ## Complete Workflow Examples ### Basic Node Setup ```bash # 1. Initialize a new node merod --node-name node1 init # 2. Configure the node (optional) merod --node-name node1 config --server-port 3000 --swarm-port 4000 # 3. Run the node merod --node-name node1 run ``` ### Production Node Setup ```bash # 1. Create custom directory mkdir -p /opt/.calimero/prod-node # 2. Initialize with production settings merod --home /opt/.calimero/prod-node --node-name prod-node init \ --server-host 0.0.0.0 \ --server-port 8080 \ --swarm-host 0.0.0.0 \ --swarm-port 9090 \ --protocol near \ --relayer-url https://relayer.near.org # 3. Run the production node merod --home /opt/.calimero/prod-node --node-name prod-node run ``` ### Multi-Protocol Node ```bash # 1. Initialize node supporting multiple protocols merod --node-name multi-node init \ --protocol near \ --boot-network calimero-dev \ --mdns \ --advertise-address # 2. Configure additional protocols merod --node-name multi-node config \ --protocol ethereum \ --relayer-url https://ethereum-relayer.com # 3. Run the multi-protocol node merod --node-name multi-node run ``` :::tip Run `merod --help` or `merod --node-name --help` to get detailed information about available options and commands. ::: :::info The default location of the config and data directory is `~/.calimero`. You can change this location by using the `--home` option or setting the `CALIMERO_HOME` environment variable. ::: ``` ``` --- ## Meroctl import MeroctlInstallation from '../../shared/install-meroctl.mdx'; Meroctl is a command-line tool that enables you to interact with your Calimero node directly from the shell. Installation ## Usage ```bash title="Terminal" meroctl [OPTIONS] --node ``` ### Examples with Real Values ```bash title="Terminal" # Basic usage with a node named "node1" meroctl --node node1 app list # Using a different node name meroctl --node my-production-node context list # With additional options meroctl --home ~/.calimero --node node1 app install my-app ``` ### Parameter Reference | **Parameter** | **Description** | **Real Examples** | | -------------- | --------------------------------- | ---------------------------------------------- | | `` | Name of the node to connect to | `node1`, `prod-node`, `test-node` | | `` | Subcommand to execute | `list`, `install`, `get`, `create` | | `` | Identity/alias executing the call | `3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf` | | `` | Unique context identifier | `3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf` | | `` | Method name to call | `get_value`, `set_value`, `create_user` | | `` | Application identifier | `3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf` | | `` | Blob storage identifier | `3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf` | | `` | Context identifier | `3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf` | | `` | User-friendly alias name | `alice`, `bob`, `admin`, `user123` | | `` | File system path | `/path/to/node`, `./my-node`, `~/.calimero` | | `` | Remote node URL | `http://node.com`, `https://api.example.com` | ### Commands: - `app` Command for managing applications - `blob` Command for managing blobs - `context` Command for managing contexts - `call` Call a method on a context - `peers` Return the number of connected peers - `node` Command for managing nodes - `help` Print this message or the help of the given subcommand(s) :::tip Run help to get more information about the available options and commands. ```bash title="Terminal" meroctl --help ``` ::: ### Options: - `--home ` Directory for config and data - `-n`, `--node-name ` Name of node - `-h`, `--help` Print help - `-output-format ` [default: plain-text] [possible values: json, plain-text] - `-V`, `--version` Print version :::tip Default location of the config and data directory is `~/.calimero`. You can change the location by using the `--home` option. ::: :::note All of the communication with your node through the meroctl CLI is protected by your nodes private key. ::: ### Examples: | **Syntax** | **Description** | | --------------------------------------------------------------------------------- | ------------------------------------ | | `meroctl --node app ` | Command for managing applications | | `meroctl --node blob ` | Command for managing blobs | | `meroctl --node context ` | Command for managing contexts | | `meroctl --node call [OPTIONS] --as ` | Call a method on a context | | `meroctl --node peers` | Return the number of connected peers | | `meroctl --node node ` | Command for managing nodes | ### Practical Examples with Real Values | **Command** | **Description** | | ------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------- | | `meroctl --node node1 app list` | List all applications on node1 | | `meroctl --node node1 blob list` | List all blobs on node1 | | `meroctl --node node1 context list` | List all contexts on node1 | | `meroctl --node node1 call --as 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf get_value` | Call get_value method as executor | | `meroctl --node node1 peers` | Show connected peers on node1 | | `meroctl --node node1 node list` | List configured nodes | ### Manage Applications **Syntax:** ```bash title="Terminal" meroctl --node app ``` **Commands:** - `get ` - Fetch application details - `install ` - Install an application - `list` - List installed applications - `uninstall ` - Uninstall an application **Real Examples:** ```bash # List all applications meroctl --node node1 app list # Install an application meroctl --node node1 app install 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf # Get details of a specific application meroctl --node node1 app get 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf # Uninstall an application meroctl --node node1 app uninstall 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf ``` Commands: - `get` Fetch application details - `install` Install an application - `list` List installed applications - `uninstall` Uninstall an application ### Manage Blobs **Syntax:** ```bash title="Terminal" meroctl --node blob ``` **Commands:** - `list` - List all blobs - `info ` - Get information about a specific blob - `delete ` - Delete a specific blob **Real Examples:** ```bash # List all blobs meroctl --node node1 blob list # Get information about a specific blob meroctl --node node1 blob info 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf # Delete a specific blob meroctl --node node1 blob delete 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf ``` Commands: - `list` List all blobs - `info` Get information about a specific blob - `delete` Delete a specific blob ### Manage Contexts **Syntax:** ```bash title="Terminal" meroctl --node context ``` **Commands:** - `list` - List all contexts - `create --application-id ` - Create a new context - `join ` - Join an existing context - `get ` - Get context details - `watch ` - Watch context events - `delete ` - Delete a context - `update ` - Update app in context - `identity ` - Manage context identities - `alias ` - Manage context aliases - `use ` - Set the default context - `proposals ` - Manage proposals within a context - `sync ` - Explicitly request a sync **Real Examples:** ```bash # List all contexts meroctl --node node1 context list # Create a new context meroctl --node node1 context create --application-id 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf # Join an existing context meroctl --node node1 context join 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf # Get context details meroctl --node node1 context get 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf # Watch context events meroctl --node node1 context watch 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf # Delete a context meroctl --node node1 context delete 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf ``` Commands: - `alias` Create an alias for a context - `list` List all contexts - `create` Create a new context - `join` Join an application context - `invite` Create invitation to a context for a invitee - `get` Fetch details about the context - `delete` Delete a context - `watch` Watch events from a context and optionally execute commands - `update` Update app in context - `identity` Manage context identities - `use` Set the default context - `proposals` Manage proposals within a context - `sync` Explicitly request a sync `get` subcommands- `info` Get context information - `client-keys` Get client keys - `storage` Get storage information - `identity` Create or list a context identity or it's alias - `delete` Delete a context - `watch` Watch events from a context - `update` Update app in context - `peers` Show a number of connected peers ### Managing Context Identities and Aliases The `context identity` command supports alias management to simplify working with public keys across contexts: ```bash meroctl --node context identity ``` Commands: - `list` List identities in a context - Use `--owned` to get only owned identities ```bash meroctl --node context identity list --owned ``` - `alias` Manage identity aliases - `add --context ` Create new alias for an identity - `remove --context ` Remove an identity alias - `get --context ` Look up an identity's hash by alias ### Context Management with Aliases The `context` command includes alias management as a subcommand to simplify working with context IDs: ```bash meroctl --node context ``` The alias subcommand structure: ```bash meroctl --node context alias ``` - `add ` Create new alias for a context - `remove ` Remove a context alias - `get ` Look up a context by alias Additional context commands support using aliases: - `create --as ...` Create context with an identity alias - `get ` Get context info using alias - `invite --as ` Create invitation using aliases - And more... :::tip See the Invitations and Joinings guide for detailed examples of using aliases to streamline context management. ::: ### Manage Nodes **Syntax:** ```bash title="Terminal" meroctl --node node ``` **Commands:** - `list` - List all configured nodes - `add ` - Add or connect to a node - `remove ` - Remove a node connection - `use ` - Set a node as active (default for commands) **Real Examples:** ```bash # List all configured nodes meroctl --node node1 node list # Add a new local node meroctl --node node1 node add node2 /path/to/node2 # Add a remote node meroctl --node node1 node add remote-node http://remote-node.com # Set a node as active meroctl --node node1 node use node2 # Remove a node connection meroctl --node node1 node remove node2 ``` Commands: - `add` Add or connect to a node - `remove` Remove a node connection - `use` Set a node as active (default for commands) - `list` List all configured nodes ### Manage Identities **Syntax:** ```bash title="Terminal" meroctl --node identity ``` **Commands:** - `generate` - Generate public/private key pair used for context identity **Real Examples:** ```bash # Generate a new key pair meroctl --node node1 identity generate # List identities in a context meroctl --node node1 context identity list 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf # List only owned identities meroctl --node node1 context identity list 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf --owned ``` Commands: - `generate` Generate public/private key pair used for context identity ### Show connected peers **Syntax:** ```bash title="Terminal" meroctl --node peers ``` **Real Examples:** ```bash # Show peers for default node meroctl --node node1 peers # Show peers with JSON output meroctl --node node1 peers --output-format json ``` ### Call a method on a context **Syntax:** ```bash title="Terminal" meroctl --node call [OPTIONS] --as ``` **Arguments:** - `` - ContextId of the context - `` - Method to call on the context **Options:** - `--args ` - JSON arguments to pass to the method - `--as ` - Public key or alias of the executor - `--id ` - Id of the RPC execute call [default: dontcare] **Real Examples:** ```bash # Call a simple method meroctl --node node1 call --as 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf get_value # Call with arguments meroctl --node node1 call --as 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf set_value --args '{"key": "name", "value": "John"}' # Call with specific ID meroctl --node node1 call --as 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf create_user --id user-123 ``` Arguments: - `` ContextId of the context - `` Method to call on the context Options: - `--args ` JSON arguments to pass to the method - `--as ` Public key of the executor - `--id ` Id of the RPC execute call [default: dontcare] --- ## Protocol SDK The Protocol SDK within the Calimero Network serves as a foundational tool for developers, enabling them to design, develop, and deploy the specific protocols that govern the operation of their decentralized applications (DApps). This SDK is particularly crucial in a network like Calimero, where privacy, security, and decentralized communication are paramount. Here's an overview of the Protocol SDK, highlighting its features, functionalities, and its role in the development lifecycle of DApps: ### Features and Functionalities - **Protocol Definition**: Allows developers to define the rules and behaviors of their application networks, including communication protocols, data formats, and interaction patterns among nodes. - **Code Generation**: Automates the generation of boilerplate code required to implement the defined protocols, significantly speeding up the development process and reducing the potential for errors. - **Interoperability Support**: Facilitates the creation of protocols that can interact with various blockchains and external systems, ensuring that Calimero-based DApps can operate within the broader blockchain ecosystem. - **Security Focus**: Provides tools and libraries to incorporate advanced security features into protocols, such as end-to-end encryption, secure key management, and privacy-preserving data sharing mechanisms. - **Performance Optimization**: Includes optimization tools and best practices to ensure that the protocols are efficient in terms of resource usage, suitable for decentralized networks where performance can be a critical concern. ### Role in DApp Development - **Protocol Development**: At the core of any DApp on the Calimero Network is a protocol that dictates how the application functions, how nodes within the application's network communicate, and how data is handled and stored. The Protocol SDK is the primary tool for developing these protocols. - **Testing and Deployment**: The SDK provides an environment for thorough testing of the protocols in simulated conditions before they are deployed on the live network. This ensures that any issues can be identified and resolved in a controlled setting, minimizing risks. - **Versioning and Updates**: Supports protocol versioning, enabling developers to iterate on their protocols and roll out updates in a structured manner. This is crucial for maintaining compatibility and ensuring the longevity and scalability of DApps. ### Integration with Other Calimero Components - **Client Nodes**: Protocols developed with the Protocol SDK are deployed on client nodes, which act as the runtime environment for the DApps built on these protocols. - **Client SDKs**: These SDKs interact with the protocols at a higher level, providing interfaces for end-users to interact with the DApps. The seamless integration between the Protocol SDK and Client SDKs ensures a smooth user experience. - **Specialized Nodes**: Some protocols may require specialized computational resources or functionalities. The Protocol SDK allows for the integration of these services, enabling DApps to leverage the specialized nodes within the Calimero Network. ### Use Cases - **Decentralized Messaging**: For a messaging app, the Protocol SDK could be used to define the encryption protocols, message delivery mechanisms, and peer discovery protocols. - **Decentralized Finance (DeFi)**: In a DeFi application, the SDK could define the protocols for executing smart contracts, handling transactions, and interacting with external blockchains for asset transfers. The Protocol SDK is a critical component of the Calimero Network, empowering developers to build sophisticated, secure, and efficient decentralized applications. By abstracting much of the complexity associated with protocol development, the SDK enables developers to focus on the unique features and functionalities of their DApps, fostering innovation and growth within the Calimero ecosystem. --- ## Rust Protocol SDK # Calimero Rust SDK This guide provides a comprehensive reference of the essential macros and functionality provided by the Calimero SDK for building P2P Rust applications. ## Core Macros ### #[app::state] Marks a struct as the application state. The state struct must implement `BorshSerialize` and `BorshDeserialize`. This macro automatically generates the `Self::external()` method for external proposal functionality. ```rust use calimero_sdk::borsh::{BorshDeserialize, BorshSerialize}; #[app::state(emits = for MyEvent)] #[derive(Default, BorshSerialize, BorshDeserialize)] #[borsh(crate = "calimero_sdk::borsh")] struct MyAppState { // Your state fields here users: calimero_storage::collections::UnorderedMap, } ``` **Key Features:** - Automatically generates `Self::external()` method for proposal management - Integrates with Calimero's storage system - Supports event emission specification ### #[app::logic] Marks an implementation block as containing the application logic. All public methods in this block become available as application endpoints. ```rust #[app::logic] impl MyAppState { // Your methods here pub fn add_user(&mut self, username: String, profile: UserProfile) -> app::Result { // Implementation Ok(()) } } ``` ### #[app::init] Marks a method as the initializer, which is called when the application is first deployed. ```rust #[app::logic] impl MyAppState { #[app::init] pub fn init() -> Self { Self { users: calimero_storage::collections::UnorderedMap::new(), } } // Example with init parameters #[app::init] pub fn init_with_params( initial_admin: String, max_users: u32, is_public: bool, ) -> Self { Self { users: calimero_storage::collections::UnorderedMap::new(), admin: initial_admin, max_users, is_public, } } } ``` ### #[app::event] Defines an event type that can be emitted by your application. Events support lifetime parameters for efficient string handling. ```rust #[app::event] pub enum MyEvent { ValueUpdated { key: &'a str, value: &'a str }, ValueRemoved { key: &'a str }, UserAdded { username: &'a str }, } ``` ## Utility Macros ### app::emit! Emit events from your application. Events are only emitted if the transaction succeeds. ```rust app::emit!(MyEvent::ValueUpdated { key: &key, value: &new_value }); app::emit!(MyEvent::UserAdded { username: &username }); ``` ### app::log! Log messages for debugging and monitoring. These appear in the application logs. ```rust app::log!("Setting key: {:?} to value: {:?}", key, value); app::log!("User {} added successfully", username); ``` ### app::bail! Return an error and exit the current function early. ```rust if !self.users.contains_key(&username) { app::bail!("User not found: {}", username); } ``` ### app::err! Create an error value for returning from functions. ```rust return app::err!("Invalid input: {}", input); ``` ## Return Types The SDK provides a convenient `app::Result` type alias: ```rust use calimero_sdk::app; pub fn my_function(&self) -> app::Result { // Your implementation Ok("success".to_string()) } ``` ## Complete Example Here's a complete example showing how these macros work together: ```rust use calimero_sdk::borsh::{BorshDeserialize, BorshSerialize}; use calimero_sdk::{app, env}; use calimero_storage::collections::UnorderedMap; #[app::event] pub enum StoreEvent { ValueSet { key: &'a str, value: &'a str }, ValueRemoved { key: &'a str }, } #[app::state(emits = for StoreEvent)] #[derive(Default, BorshSerialize, BorshDeserialize)] #[borsh(crate = "calimero_sdk::borsh")] struct Store { values: UnorderedMap, } #[app::logic] impl Store { #[app::init] pub fn init() -> Self { Self { values: UnorderedMap::new(), } } pub fn set(&mut self, key: String, value: String) -> app::Result { app::log!("Setting key: {:?} to value: {:?}", key, value); if self.values.contains(&key)? { app::emit!(StoreEvent::ValueSet { key: &key, value: &value }); } else { app::emit!(StoreEvent::ValueSet { key: &key, value: &value }); } self.values.insert(key, value)?; Ok(()) } pub fn get(&self, key: &str) -> app::Result> { app::log!("Getting key: {:?}", key); self.values.get(key).map_err(Into::into) } pub fn remove(&mut self, key: &str) -> app::Result> { app::log!("Removing key: {:?}", key); if let Some(value) = self.values.remove(key)? { app::emit!(StoreEvent::ValueRemoved { key }); Ok(Some(value)) } else { Ok(None) } } } ``` ## Important Notes State changes are atomic - if a method fails, all changes are rolled back Events are only emitted if the transaction succeeds Read-only operations have no network overhead All public methods in the `#[app::logic]` block become available as application endpoints Collections return `Result` for most operations - handle errors appropriately The `Self::external()` method is automatically generated by `#[app::state]` for proposal management ## Next Steps - Learn about [Calimero Collections](./calimero-collections) for data storage - See [Rust SDK Deep Dive](./rust-sdk-deep-dive) for advanced patterns --- ## Rust SDK Deep Dive # Rust SDK Deep Dive This comprehensive guide covers advanced usage of the Calimero Rust SDK, including collections, events, macros, and best practices for building production applications. ## Table of Contents - [Collections](#collections) - [Events and Event Handling](#events-and-event-handling) - [Advanced Macros](#advanced-macros) - [State Management Patterns](#state-management-patterns) - [Performance Optimization](#performance-optimization) - [Testing and Debugging](#testing-and-debugging) - [Common Patterns and Anti-patterns](#common-patterns-and-anti-patterns) ## Collections Calimero provides a rich set of collections optimized for WebAssembly and decentralized applications. These collections are designed to work efficiently with the Calimero storage system and return `Result` for robust error handling. ### Core Collections #### UnorderedMap A hash map implementation optimized for Wasm with O(1) average case operations: ```rust use calimero_storage::collections::UnorderedMap; #[app::state] #[derive(Default, BorshSerialize, BorshDeserialize)] #[borsh(crate = "calimero_sdk::borsh")] struct MyApp { users: UnorderedMap, settings: UnorderedMap, } #[app::logic] impl MyApp { pub fn add_user(&mut self, username: String, profile: UserProfile) -> app::Result { if self.users.contains_key(&username)? { return app::bail!("User already exists"); } self.users.insert(username, profile)?; Ok(()) } pub fn get_user(&self, username: &str) -> app::Result> { self.users.get(username).map_err(Into::into) } pub fn remove_user(&mut self, username: &str) -> app::Result> { self.users.remove(username) } pub fn get_user_count(&self) -> app::Result { self.users.len() } } ``` #### UnorderedSet A hash set for unique elements: ```rust use calimero_storage::collections::UnorderedSet; #[app::state] #[derive(Default, BorshSerialize, BorshDeserialize)] #[borsh(crate = "calimero_sdk::borsh")] struct AccessControl { admins: UnorderedSet, moderators: UnorderedSet, } #[app::logic] impl AccessControl { pub fn add_admin(&mut self, username: String) -> app::Result { self.admins.insert(username)?; Ok(()) } pub fn is_admin(&self, username: &str) -> app::Result { self.admins.contains(username) } pub fn remove_admin(&mut self, username: &str) -> app::Result { self.admins.remove(username) } pub fn get_admin_count(&self) -> app::Result { self.admins.len() } } ``` #### Vector A dynamic array implementation: ```rust use calimero_storage::collections::Vector; #[app::state] #[derive(Default, BorshSerialize, BorshDeserialize)] #[borsh(crate = "calimero_sdk::borsh")] struct TaskManager { tasks: Vector, completed_tasks: Vector, } #[app::logic] impl TaskManager { pub fn add_task(&mut self, task: Task) -> app::Result { self.tasks.insert(None, task)?; Ok(()) } pub fn complete_task(&mut self, index: u32) -> app::Result { if index >= self.tasks.len()? { return app::bail!("Invalid task index"); } let task = self.tasks.remove(index as usize)?; self.completed_tasks.insert(None, task)?; Ok(()) } pub fn get_task(&self, index: u32) -> app::Result> { if index >= self.tasks.len()? { return Ok(None); } self.tasks.get(index as usize) } pub fn get_all_tasks(&self) -> app::Result> { self.tasks.entries()?.collect() } } ``` ### Advanced Collection Patterns #### Nested Collections ```rust #[app::state] #[derive(Default, BorshSerialize, BorshDeserialize)] #[borsh(crate = "calimero_sdk::borsh")] struct SocialNetwork { // User -> List of friends friendships: UnorderedMap>, // User -> List of posts user_posts: UnorderedMap>, // Post -> List of likes post_likes: UnorderedMap>, } #[app::logic] impl SocialNetwork { pub fn add_friend(&mut self, user: String, friend: String) -> app::Result { // Add bidirectional friendship self.friendships .entry(user.clone()) .or_insert_with(|| UnorderedSet::new()) .insert(friend.clone())?; self.friendships .entry(friend) .or_insert_with(|| UnorderedSet::new()) .insert(user)?; Ok(()) } pub fn get_friends(&self, user: &str) -> app::Result> { Ok(self.friendships .get(user)? .map(|friends| friends.entries()?.collect()) .unwrap_or_default()) } pub fn add_post(&mut self, user: String, post: Post) -> app::Result { self.user_posts .entry(user.clone()) .or_insert_with(|| Vector::new()) .insert(None, post)?; Ok(()) } } ``` #### Collection Iterators ```rust #[app::logic] impl MyApp { pub fn get_all_users(&self) -> app::Result> { self.users.entries()?.collect() } pub fn search_users(&self, query: &str) -> app::Result> { Ok(self.users .entries()? .filter(|(username, _)| username.contains(query)) .map(|(_, profile)| profile) .collect()) } pub fn get_active_users(&self) -> app::Result> { Ok(self.users .entries()? .filter(|(_, profile)| profile.is_active) .map(|(_, profile)| profile) .collect()) } } ``` ## Events and Event Handling ### Event Definition Events in Calimero use lifetime parameters for efficient string handling: ```rust #[app::event] pub enum AppEvent { UserCreated { username: &'a str, timestamp: u64 }, UserUpdated { username: &'a str, field: &'a str }, UserDeleted { username: &'a str }, SettingsChanged { key: &'a str, old_value: &'a str, new_value: &'a str }, } ``` ### Event Emission Events are emitted using the `app::emit!` macro and are only recorded if the transaction succeeds: ```rust #[app::logic] impl MyApp { pub fn create_user(&mut self, username: String, profile: UserProfile) -> app::Result { app::log!("Creating user: {}", username); if self.users.contains_key(&username)? { return app::bail!("User already exists"); } self.users.insert(username.clone(), profile)?; app::emit!(AppEvent::UserCreated { username: &username, timestamp: env::block_timestamp(), }); Ok(()) } pub fn update_user(&mut self, username: String, new_profile: UserProfile) -> app::Result { app::log!("Updating user: {}", username); let old_profile = self.users.get(&username)? .ok_or_else(|| app::err!("User not found"))?; self.users.insert(username.clone(), new_profile)?; app::emit!(AppEvent::UserUpdated { username: &username, field: "profile", }); Ok(()) } } ``` ### Event Patterns #### Batch Events ```rust #[app::logic] impl MyApp { pub fn bulk_create_users(&mut self, users: Vec) -> app::Result { let mut created_count = 0; for (username, profile) in users { if !self.users.contains_key(&username)? { self.users.insert(username.clone(), profile)?; created_count += 1; app::emit!(AppEvent::UserCreated { username: &username, timestamp: env::block_timestamp(), }); } } app::log!("Created {} new users", created_count); Ok(()) } } ``` #### Conditional Events ```rust #[app::logic] impl MyApp { pub fn update_user_if_changed(&mut self, username: String, new_profile: UserProfile) -> app::Result { let old_profile = match self.users.get(&username)? { Some(profile) => profile, None => return Ok(false), }; if old_profile == new_profile { return Ok(false); // No change, no event } self.users.insert(username.clone(), new_profile)?; app::emit!(AppEvent::UserUpdated { username: &username, field: "profile", }); Ok(true) // Changed } } ``` ## Advanced Macros ### Macro Composition Macros can be combined for complex functionality: ```rust #[app::state(emits = for AppEvent)] #[derive(Default, BorshSerialize, BorshDeserialize)] #[borsh(crate = "calimero_sdk::borsh")] struct AdvancedApp { users: UnorderedMap, audit_log: Vector, } #[app::logic] impl AdvancedApp { #[app::init] pub fn init() -> Self { Self { users: UnorderedMap::new(), audit_log: Vector::new(), } } pub fn perform_action(&mut self, action: Action) -> app::Result { // Log the action let entry = AuditEntry { action: action.clone(), timestamp: env::block_timestamp(), user: env::predecessor_account_id().to_string(), }; self.audit_log.insert(None, entry)?; // Perform the action match action { Action::CreateUser { username, profile } => { self.create_user(username, profile)?; } Action::UpdateUser { username, profile } => { self.update_user(username, profile)?; } Action::DeleteUser { username } => { self.delete_user(username)?; } } Ok(()) } } ``` ### Custom Error Types ```rust #[derive(Debug, thiserror::Error)] pub enum AppError { #[error("User not found: {0}")] UserNotFound(String), #[error("Invalid operation: {0}")] InvalidOperation(String), #[error("Insufficient permissions")] InsufficientPermissions, } impl From for calimero_sdk::types::Error { fn from(err: AppError) -> Self { calimero_sdk::types::Error::msg(err.to_string()) } } #[app::logic] impl MyApp { pub fn admin_only_operation(&mut self, username: String) -> app::Result { if !self.is_admin(&env::predecessor_account_id().to_string())? { return Err(AppError::InsufficientPermissions.into()); } // Perform admin operation Ok(()) } } ``` ### Advanced Macro Features > **Note**: Some of these advanced macro features may be planned for future > releases or available in different versions of the SDK. Check the latest SDK > documentation for current availability. #### Custom State Configuration Advanced state configuration with metadata and versioning: ```rust #[app::state( emits = for AppEvent, version = "1.0.0", name = "UserManagementApp" )] #[derive(Default, BorshSerialize, BorshDeserialize)] #[borsh(crate = "calimero_sdk::borsh")] struct UserManagementApp { users: UnorderedMap, settings: AppSettings, } #[derive(BorshSerialize, BorshDeserialize, Default)] struct AppSettings { max_users: u32, allow_public_profiles: bool, maintenance_mode: bool, } #[app::logic] impl UserManagementApp { pub fn create_user(&mut self, username: String, profile: UserProfile) -> app::Result { // Check maintenance mode if self.settings.maintenance_mode { return app::bail!("Application is in maintenance mode"); } // Check user limit if self.users.len()? >= self.settings.max_users { return app::bail!("Maximum user limit reached"); } if self.users.contains_key(&username)? { return app::bail!("User already exists"); } self.users.insert(username.clone(), profile)?; app::emit!(AppEvent::UserCreated { username: &username, timestamp: env::block_timestamp(), }); Ok(()) } pub fn update_settings(&mut self, new_settings: AppSettings) -> app::Result { // Validate settings if new_settings.max_users **Note**: While these specific macros may not be implemented, the patterns > shown here demonstrate how to implement access control using the available SDK > features. ```rust #[app::logic] impl UserManagementApp { // Admin-only operations pub fn admin_only_operation(&mut self, username: String) -> app::Result { if !self.is_admin(&env::predecessor_account_id().to_string())? { return app::bail!("Admin access required"); } // Perform admin operation Ok(()) } // Moderator or admin operations pub fn moderator_or_admin_operation(&mut self, username: String) -> app::Result { let caller = env::predecessor_account_id().to_string(); if !self.is_admin(&caller)? && !self.is_moderator(&caller)? { return app::bail!("Moderator or admin access required"); } // Perform operation Ok(()) } // Role-based access control pub fn role_based_operation(&mut self, username: String, required_role: UserRole) -> app::Result { let caller = env::predecessor_account_id().to_string(); let caller_role = self.get_user_role(&caller)?; if !self.has_permission(caller_role, required_role)? { return app::bail!("Insufficient permissions for this operation"); } // Perform operation Ok(()) } fn has_permission(&self, user_role: UserRole, required_role: UserRole) -> app::Result { match (user_role, required_role) { (UserRole::Admin, _) => Ok(true), // Admin can do everything (UserRole::Moderator, UserRole::Moderator | UserRole::User) => Ok(true), (UserRole::User, UserRole::User) => Ok(true), _ => Ok(false), } } } ``` #### Conditional Logic and State Management Advanced patterns for conditional operations and state transitions: ```rust #[app::logic] impl UserManagementApp { pub fn conditional_user_operation(&mut self, username: String, operation: UserOperation) -> app::Result { // Check application state if self.settings.maintenance_mode { return app::bail!("Application is in maintenance mode"); } // Check user state let user = self.users.get(&username)? .ok_or_else(|| app::err!("User not found"))?; if !user.is_active { return app::bail!("User account is not active"); } // Perform operation based on user role match operation { UserOperation::UpdateProfile(profile) => { if !self.can_update_profile(&username, &profile)? { return app::bail!("Cannot update profile with current permissions"); } self.users.insert(username, profile)?; } UserOperation::ChangeRole(new_role) => { if !self.can_change_role(&username, new_role)? { return app::bail!("Cannot change role with current permissions"); } let mut updated_user = user.clone(); updated_user.role = new_role; self.users.insert(username, updated_user)?; } UserOperation::Suspend(reason) => { if !self.can_suspend_user(&username)? { return app::bail!("Cannot suspend user with current permissions"); } let mut updated_user = user.clone(); updated_user.status = UserStatus::Suspended(reason); self.users.insert(username, updated_user)?; } } Ok(()) } fn can_update_profile(&self, username: &str, profile: &UserProfile) -> app::Result { let caller = env::predecessor_account_id().to_string(); // Users can update their own profile if caller == *username { return Ok(true); } // Admins can update any profile if self.is_admin(&caller)? { return Ok(true); } // Check if profile is public and caller has permission if profile.is_public && self.settings.allow_public_profiles { return Ok(true); } Ok(false) } } ``` ## State Management Patterns ### Immutable State Access ```rust #[app::logic] impl MyApp { pub fn get_user_stats(&self) -> app::Result { let mut stats = UserStats::default(); for (_, profile) in self.users.entries()? { stats.total_users += 1; if profile.is_active { stats.active_users += 1; } if profile.is_premium { stats.premium_users += 1; } } Ok(stats) } } ``` ### Mutable State with Validation ```rust #[app::logic] impl MyApp { pub fn transfer_user(&mut self, from: String, to: String) -> app::Result { // Validate input if from == to { return app::bail!("Cannot transfer user to same account"); } // Check source user exists let profile = self.users.get(&from)? .ok_or_else(|| app::err!("Source user not found"))?; // Check destination doesn't exist if self.users.contains_key(&to)? { return app::bail!("Destination user already exists"); } // Perform transfer atomically self.users.remove(&from)?; self.users.insert(to, profile)?; app::emit!(AppEvent::UserTransferred { from: &from, to: &to, }); Ok(()) } } ``` ### State Initialization Patterns ```rust #[app::logic] impl MyApp { #[app::init] pub fn init() -> Self { Self { users: UnorderedMap::new(), settings: UnorderedMap::new(), metadata: AppMetadata { version: "1.0.0".to_string(), created_at: env::block_timestamp(), }, } } #[app::init] pub fn init_with_config(config: AppConfig) -> Self { let mut app = Self::init(); // Apply configuration for (key, value) in config.default_settings { app.settings.insert(key, value).expect("Failed to set default"); } app } } ``` ## Performance Optimization ### Efficient Iteration ```rust #[app::logic] impl MyApp { // ✅ Good: Use references to avoid cloning pub fn get_active_users(&self) -> app::Result> { Ok(self.users .entries()? .filter(|(_, profile)| profile.is_active) .map(|(_, profile)| profile) .collect()) } // ❌ Bad: Unnecessary cloning pub fn get_active_users_bad(&self) -> app::Result> { Ok(self.users .entries()? .filter(|(_, profile)| profile.is_active) .map(|(_, profile)| profile.clone()) // Unnecessary clone .collect()) } } ``` ### Batch Operations ```rust #[app::logic] impl MyApp { pub fn bulk_update_users(&mut self, updates: Vec) -> app::Result { for (username, profile) in updates { self.users.insert(username, profile)?; } Ok(()) } pub fn bulk_remove_users(&mut self, usernames: Vec) -> app::Result { let mut removed_count = 0; for username in usernames { if self.users.remove(&username)?.is_some() { removed_count += 1; } } Ok(removed_count) } } ``` ### Memory Management ```rust #[app::logic] impl MyApp { pub fn cleanup_old_data(&mut self, cutoff_timestamp: u64) -> app::Result { let mut to_remove = Vec::new(); // Collect items to remove for (key, profile) in self.users.entries()? { if profile.last_activity app::Result { app::log!("Starting complex operation with input: {:?}", input); // Step 1: Validation app::log!("Step 1: Validating input"); self.validate_input(&input)?; // Step 2: Processing app::log!("Step 2: Processing input"); let intermediate = self.process_input(input)?; // Step 3: Finalization app::log!("Step 3: Finalizing operation"); let result = self.finalize_operation(intermediate)?; app::log!("Complex operation completed successfully"); Ok(result) } } ``` ### Error Context ```rust #[app::logic] impl MyApp { pub fn get_user_with_context(&self, username: &str) -> app::Result { self.users.get(username) .map_err(|e| { app::log!("Failed to get user '{}': {:?}", username, e); e })? .ok_or_else(|| { app::log!("User '{}' not found", username); app::err!("User not found: {}", username) }) } } ``` ### Unit Testing > **Note**: While the Calimero SDK doesn't provide built-in testing macros, you > can use standard Rust testing approaches. Here are examples of how to > structure tests for your Calimero applications. ```rust #[cfg(test)] mod tests { use super::*; use calimero_storage::collections::UnorderedMap; // Mock storage for testing struct MockStorage; impl StorageAdaptor for MockStorage { // Implement mock storage methods for testing } #[test] fn test_user_creation() { let mut app = UserManagementApp::default(); let profile = UserProfile::default(); let result = app.create_user("test_user".to_string(), profile); assert!(result.is_ok()); assert!(app.users.contains_key("test_user").unwrap()); } #[test] fn test_duplicate_user_creation() { let mut app = UserManagementApp::default(); let profile = UserProfile::default(); app.create_user("test_user".to_string(), profile.clone()).unwrap(); let result = app.create_user("test_user".to_string(), profile); assert!(result.is_err()); } #[test] fn test_user_update() { let mut app = UserManagementApp::default(); let profile = UserProfile::default(); let updated_profile = UserProfile { name: "Updated Name".to_string(), ..profile.clone() }; app.create_user("test_user".to_string(), profile).unwrap(); let result = app.update_user("test_user".to_string(), updated_profile); assert!(result.is_ok()); } } ``` ### Debug Logging and State Inspection ```rust #[app::logic] impl MyApp { pub fn debug_state(&self) -> app::Result { let user_count = self.users.len()?; let active_users = self.users.entries()? .filter(|(_, profile)| profile.is_active) .count(); let debug_info = format!( "Users: {}, Active: {}, Settings: {:?}", user_count, active_users, self.settings ); app::log!("Debug state: {}", debug_info); Ok(debug_info) } pub fn validate_integrity(&self) -> app::Result> { let mut issues = Vec::new(); // Check for orphaned references for (username, profile) in self.users.entries()? { if let Some(referenced_user) = &profile.referenced_by { if !self.users.contains_key(referenced_user)? { issues.push(format!("User '{}' references non-existent user '{}'", username, referenced_user)); } } } // Check for circular references // ... implementation details ... if !issues.is_empty() { app::log!("Integrity check found {} issues: {:?}", issues.len(), issues); } Ok(issues) } } ``` ### Performance Monitoring ```rust #[app::logic] impl MyApp { pub fn performance_metrics(&self) -> app::Result { let start_time = std::time::Instant::now(); let user_count = self.users.len()?; let storage_size = self.estimate_storage_size()?; let operation_count = self.get_operation_count()?; let metrics = PerformanceMetrics { user_count, storage_size, operation_count, response_time: start_time.elapsed().as_millis() as u64, }; app::log!("Performance metrics: {:?}", metrics); Ok(metrics) } fn estimate_storage_size(&self) -> app::Result { // Estimate storage size based on collection contents let mut total_size = 0; for (_, profile) in self.users.entries()? { total_size += std::mem::size_of_val(&profile); } Ok(total_size) } } #[derive(Debug)] struct PerformanceMetrics { user_count: usize, storage_size: usize, operation_count: u64, response_time: u64, } ``` ## Common Patterns and Anti-patterns ### ✅ Good Patterns #### Atomic Operations ```rust #[app::logic] impl MyApp { pub fn atomic_user_update(&mut self, username: String, updates: UserUpdates) -> app::Result { // All operations are atomic - if any fail, all changes are rolled back let mut profile = self.users.get(&username)? .ok_or_else(|| app::err!("User not found"))?; // Apply updates if let Some(name) = updates.name { profile.name = name; } if let Some(email) = updates.email { profile.email = email; } // Save updated profile self.users.insert(username, profile)?; app::emit!(AppEvent::UserUpdated { username: &username, field: "profile", }); Ok(()) } } ``` #### Proper Error Handling ```rust #[app::logic] impl MyApp { pub fn safe_user_operation(&mut self, username: &str) -> app::Result { // Handle all possible error cases match self.users.get(username)? { Some(profile) => { if profile.is_active { // Perform operation Ok(()) } else { app::bail!("User is not active") } } None => app::bail!("User not found"), } } } ``` ### ❌ Anti-patterns #### Ignoring Errors ```rust // ❌ Bad: Ignoring collection operation results pub fn bad_user_operation(&mut self, username: String) { self.users.insert(username, UserProfile::default()); // Ignoring Result } // ✅ Good: Handle errors properly pub fn good_user_operation(&mut self, username: String) -> app::Result { self.users.insert(username, UserProfile::default())?; Ok(()) } ``` #### Unnecessary Cloning ```rust // ❌ Bad: Cloning when references would suffice pub fn bad_get_users(&self) -> app::Result> { Ok(self.users.entries()? .map(|(k, _)| k.clone()) // Unnecessary clone .collect()) } // ✅ Good: Use references pub fn good_get_users(&self) -> app::Result> { Ok(self.users.entries()? .map(|(k, _)| k) // No clone needed .collect()) } ``` #### Complex Nested Operations ```rust // ❌ Bad: Deep nesting makes error handling complex pub fn bad_nested_operation(&mut self, user_id: &str) -> app::Result { if let Some(user) = self.users.get(user_id)? { if let Some(profile) = user.profile { if let Some(settings) = profile.settings { // Deep nesting makes this hard to read and maintain settings.update()?; } } } Ok(()) } // ✅ Good: Early returns for cleaner code pub fn good_nested_operation(&mut self, user_id: &str) -> app::Result { let user = self.users.get(user_id)? .ok_or_else(|| app::err!("User not found"))?; let profile = user.profile .ok_or_else(|| app::err!("User profile not found"))?; let settings = profile.settings .ok_or_else(|| app::err!("User settings not found"))?; settings.update()?; Ok(()) } ``` ## Next Steps - Learn about [Calimero Collections](./calimero-collections) for data storage - See [Rust Protocol SDK](./protocol-rs-sdk) for basic macro usage --- ## Calimero Collections Reference # Calimero Collections Reference This comprehensive reference covers all available collections in the Calimero SDK, their performance characteristics, and best practices for using them effectively. ## Table of Contents - [Overview](#overview) - [Core Collections](#core-collections) - [Specialized Collections](#specialized-collections) - [Performance Characteristics](#performance-characteristics) - [Collection Patterns](#collection-patterns) - [Best Practices](#best-practices) - [Migration from Standard Collections](#migration-from-standard-collections) ## Overview Calimero collections are specifically designed for WebAssembly and decentralized applications. They provide: - **Wasm Optimization**: Designed for efficient serialization and memory usage - **Persistent Storage**: Automatic integration with Calimero's storage system - **Type Safety**: Full Rust type safety with compile-time guarantees - **Performance**: Optimized for common operations in decentralized apps - **Error Handling**: All operations return `Result` for robust error handling ## Core Collections ### UnorderedMap A hash map implementation optimized for Wasm with O(1) average case operations. All operations return `Result`. #### Basic Usage ```rust use calimero_storage::collections::UnorderedMap; #[app::state] #[derive(Default, BorshSerialize, BorshDeserialize)] #[borsh(crate = "calimero_sdk::borsh")] struct UserRegistry { users: UnorderedMap, user_count: u32, } #[app::logic] impl UserRegistry { pub fn add_user(&mut self, username: String, profile: UserProfile) -> app::Result { if self.users.contains_key(&username)? { return app::bail!("User already exists"); } self.users.insert(username, profile)?; self.user_count += 1; Ok(()) } pub fn get_user(&self, username: &str) -> app::Result> { self.users.get(username).map_err(Into::into) } pub fn update_user(&mut self, username: String, profile: UserProfile) -> app::Result { if !self.users.contains_key(&username)? { return app::bail!("User not found"); } self.users.insert(username, profile)?; Ok(()) } pub fn remove_user(&mut self, username: &str) -> app::Result> { let user = self.users.remove(username)?; if user.is_some() { self.user_count -= 1; } Ok(user) } } ``` #### Advanced Operations ```rust #[app::logic] impl UserRegistry { // Batch operations pub fn add_multiple_users(&mut self, users: Vec) -> app::Result { for (username, profile) in users { if self.users.contains_key(&username)? { return app::bail!("User {} already exists", username); } self.users.insert(username, profile)?; self.user_count += 1; } Ok(()) } // Iteration pub fn get_all_users(&self) -> app::Result> { self.users.entries()?.collect() } // Search pub fn search_users(&self, query: &str) -> app::Result> { Ok(self.users .entries()? .filter(|(username, _)| username.contains(query)) .map(|(_, profile)| profile) .collect()) } // Statistics pub fn get_user_count(&self) -> app::Result { self.users.len() } } ``` ### UnorderedSet A hash set for unique elements with similar performance characteristics to UnorderedMap. ```rust use calimero_storage::collections::UnorderedSet; #[app::state] #[derive(Default, BorshSerialize, BorshDeserialize)] #[borsh(crate = "calimero_sdk::borsh")] struct AccessControl { admins: UnorderedSet, moderators: UnorderedSet, } #[app::logic] impl AccessControl { pub fn add_admin(&mut self, username: String) -> app::Result { self.admins.insert(username)?; Ok(()) } pub fn is_admin(&self, username: &str) -> app::Result { self.admins.contains(username) } pub fn remove_admin(&mut self, username: &str) -> app::Result { self.admins.remove(username) } pub fn get_admin_count(&self) -> app::Result { self.admins.len() } pub fn get_all_admins(&self) -> app::Result> { self.admins.entries()?.collect() } } ``` ### Vector A dynamic array implementation optimized for sequential access. ```rust use calimero_storage::collections::Vector; #[app::state] #[derive(Default, BorshSerialize, BorshDeserialize)] #[borsh(crate = "calimero_sdk::borsh")] struct TaskManager { tasks: Vector, completed_tasks: Vector, } #[app::logic] impl TaskManager { pub fn add_task(&mut self, task: Task) -> app::Result { self.tasks.insert(None, task)?; Ok(()) } pub fn complete_task(&mut self, index: u32) -> app::Result { if index >= self.tasks.len()? { return app::bail!("Invalid task index"); } let task = self.tasks.remove(index as usize)?; self.completed_tasks.insert(None, task)?; Ok(()) } pub fn get_task(&self, index: u32) -> app::Result> { if index >= self.tasks.len()? { return Ok(None); } self.tasks.get(index as usize) } pub fn get_all_tasks(&self) -> app::Result> { self.tasks.entries()?.collect() } } ``` ## Specialized Collections > **Note**: Some of these specialized collections may be planned for future > releases or available in different versions of the SDK. Check the latest SDK > documentation for current availability. ### LookupMap A map optimized for cases where you need to iterate over keys or values frequently. This collection provides better iteration performance at the cost of slightly higher memory usage. ```rust use calimero_storage::collections::LookupMap; #[app::state] #[derive(Default, BorshSerialize, BorshDeserialize)] #[borsh(crate = "calimero_sdk::borsh")] struct Analytics { // User -> Page views page_views: LookupMap, // Page -> Total views total_views: LookupMap, } #[app::logic] impl Analytics { pub fn record_page_view(&mut self, user: String, page: String) -> app::Result { // Increment user's page views let user_views = self.page_views.get(&user)?.unwrap_or(0); self.page_views.insert(user, user_views + 1)?; // Increment total page views let total = self.total_views.get(&page)?.unwrap_or(0); self.total_views.insert(page, total + 1)?; Ok(()) } pub fn get_user_stats(&self, user: &str) -> app::Result { Ok(self.page_views.get(user)?.unwrap_or(0)) } pub fn get_page_stats(&self, page: &str) -> app::Result { Ok(self.total_views.get(page)?.unwrap_or(0)) } pub fn get_top_pages(&self, limit: usize) -> app::Result> { let mut pages: Vec = self.total_views .entries()? .map(|(page, views)| (page, views)) .collect(); pages.sort_by(|a, b| b.1.cmp(&a.1)); pages.truncate(limit); Ok(pages) } } ``` ### TreeMap A sorted map implementation for cases where you need ordered keys. This collection maintains keys in sorted order, enabling efficient range queries and ordered iteration. ```rust use calimero_storage::collections::TreeMap; #[app::state] #[derive(Default, BorshSerialize, BorshDeserialize)] #[borsh(crate = "calimero_sdk::borsh")] struct Leaderboard { // Score -> List of users with that score score_rankings: TreeMap>, // User -> Current score user_scores: UnorderedMap, } #[app::logic] impl Leaderboard { pub fn update_score(&mut self, user: String, new_score: u32) -> app::Result { // Remove user from old score ranking if let Some(old_score) = self.user_scores.get(&user)? { if let Some(users_at_score) = self.score_rankings.get(&old_score)? { let mut updated_users = users_at_score.clone(); updated_users.remove(&user)?; if updated_users.is_empty() { self.score_rankings.remove(&old_score)?; } else { self.score_rankings.insert(old_score, updated_users)?; } } } // Add user to new score ranking let mut users_at_score = self.score_rankings .get(&new_score)? .unwrap_or_else(|| UnorderedSet::new()); users_at_score.insert(user.clone())?; self.score_rankings.insert(new_score, users_at_score)?; // Update user's score self.user_scores.insert(user, new_score)?; Ok(()) } pub fn get_top_players(&self, limit: usize) -> app::Result> { let mut top_players = Vec::new(); // Iterate in reverse order (highest scores first) for (score, users) in self.score_rankings.entries()?.rev() { for user in users.entries()? { top_players.push((user, score)); if top_players.len() >= limit { break; } } if top_players.len() >= limit { break; } } Ok(top_players) } pub fn get_user_rank(&self, user: &str) -> app::Result> { let user_score = match self.user_scores.get(user)? { Some(score) => score, None => return Ok(None), }; let mut rank = 1; // Count users with higher scores for (score, users) in self.score_rankings.entries()?.rev() { if score > user_score { rank += users.len()? as u32; } else { break; } } Ok(Some(rank)) } } ``` ### When to Use Each Collection #### Use UnorderedMap when: - You need fast key-value lookups - Order doesn't matter - Memory efficiency is important #### Use UnorderedSet when: - You need to track unique elements - Fast membership testing is required - Order doesn't matter #### Use Vector when: - You need ordered elements - Sequential access is common - You frequently add/remove from the end #### Use LookupMap when: - You frequently iterate over keys or values - Memory overhead is acceptable - You need both map and iteration performance #### Use TreeMap when: - You need sorted keys - Range queries are common - Order matters for your use case ## Performance Characteristics ### Operation Complexity | Operation | UnorderedMap | UnorderedSet | Vector | | --------- | ------------ | ------------ | ------ | | Insert | O(1) avg | O(1) avg | O(1) | | Get | O(1) avg | O(1) avg | O(1) | | Remove | O(1) avg | O(1) avg | O(n) | | Iterate | O(n) | O(n) | O(n) | | Contains | O(1) avg | O(1) avg | O(n) | ### Memory Usage - **UnorderedMap/UnorderedSet**: Hash table overhead + key-value storage - **Vector**: Dynamic array with growth factor - All collections automatically handle serialization/deserialization ## Collection Patterns ### Entry API for Conditional Operations ```rust #[app::logic] impl UserRegistry { pub fn get_or_create_user(&mut self, username: String) -> app::Result { // This pattern doesn't work with current API - collections return Result // Use conditional logic instead if !self.users.contains_key(&username)? { let default_profile = UserProfile::default(); self.users.insert(username.clone(), default_profile)?; } // Note: We can't return &mut due to Result wrapper // Consider returning the value or using a different pattern Ok(()) } } ``` ### Batch Operations ```rust #[app::logic] impl UserRegistry { pub fn bulk_update_users(&mut self, updates: Vec) -> app::Result { for (username, profile) in updates { self.users.insert(username, profile)?; } Ok(()) } pub fn bulk_remove_users(&mut self, usernames: Vec) -> app::Result { let mut removed_count = 0; for username in usernames { if self.users.remove(&username)?.is_some() { removed_count += 1; } } Ok(removed_count) } } ``` ## Best Practices ### Error Handling Always handle the `Result` returned by collection operations: ```rust // ❌ Bad - ignoring errors let value = self.users.get(&key).unwrap(); // ✅ Good - proper error handling let value = match self.users.get(&key)? { Some(v) => v, None => return app::bail!("User not found"), }; ``` ### Memory Management ```rust // ❌ Bad - creating unnecessary clones let all_users: Vec = self.users.entries()? .map(|(k, _)| k.clone()) .collect(); // ✅ Good - avoid unnecessary cloning let all_users: Vec = self.users.entries()? .map(|(k, _)| k) .collect(); ``` ### Transaction Safety ```rust #[app::logic] impl UserRegistry { pub fn transfer_user(&mut self, from: String, to: String) -> app::Result { // All operations are atomic - if any fail, all changes are rolled back let profile = self.users.remove(&from)? .ok_or_else(|| app::err!("Source user not found"))?; self.users.insert(to, profile)?; Ok(()) } } ``` ## Migration from Standard Collections ### Key Differences | Standard Collection | Calimero Collection | Key Changes | | ------------------- | -------------------- | ------------------------------- | | `HashMap` | `UnorderedMap` | Returns `Result` | | `HashSet` | `UnorderedSet` | Returns `Result` | | `Vec` | `Vector` | Returns `Result` | ### Migration Example ```rust // Before (standard collections) use std::collections::HashMap; struct OldApp { users: HashMap, } impl OldApp { pub fn add_user(&mut self, username: String, profile: UserProfile) { self.users.insert(username, profile); } } // After (Calimero collections) use calimero_storage::collections::UnorderedMap; #[app::state] #[derive(Default, BorshSerialize, BorshDeserialize)] #[borsh(crate = "calimero_sdk::borsh")] struct NewApp { users: UnorderedMap, } #[app::logic] impl NewApp { pub fn add_user(&mut self, username: String, profile: UserProfile) -> app::Result { self.users.insert(username, profile)?; Ok(()) } } ``` ## Common Pitfalls ### Forgetting Error Handling ```rust // ❌ This will not compile let user = self.users.get(&username).unwrap(); // ✅ Handle the Result let user = self.users.get(&username)?; ``` ### Assuming Standard Collection Methods ```rust // ❌ Standard collections don't have these methods let count = self.users.len(); // Standard collections return usize directly // ✅ Calimero collections return Result let count = self.users.len()?; // Returns Result ``` ### Ignoring Storage Errors ```rust // ❌ Bad - storage errors can indicate serious issues if let Ok(user) = self.users.get(&username) { // Handle user } // ✅ Good - propagate errors up let user = self.users.get(&username)?; ``` ## Next Steps - Learn about [Rust SDK Macros](./protocol-rs-sdk) for application structure - See [Rust SDK Deep Dive](./rust-sdk-deep-dive) for advanced patterns --- ## External Functions and Proposals # External Functions and Proposals This guide covers how to use Calimero's external proposal system to interact with external smart contracts, create proposals, and manage the governance system in Calimero applications. ## Table of Contents - [Overview](#overview) - [External Proposal System](#external-proposal-system) - [Two Ways to Access External Functions](#two-ways-to-access-external-functions) - [Proposal Actions](#proposal-actions) - [Creating Proposals](#creating-proposals) - [Approving Proposals](#approving-proposals) - [Complete Examples](#complete-examples) - [Best Practices](#best-practices) ## Overview Calimero provides an external proposal system that allows your application to: - **Create proposals** for external blockchain actions - **Approve proposals** through governance - **Execute external function calls** to other contracts - **Manage transfers** and [context modifications](/core-concepts/contexts) (context refers to Calimero Application Networks) - **Implement governance systems** with approval workflows The system is built around the `External` struct and `DraftProposal` builder pattern, accessible through two different approaches. ## External Proposal System ### Core Components The external proposal system consists of several key components: ```rust use calimero_sdk::env::ext::{External, DraftProposal, ProposalAction, ProposalId}; // External interface for proposal management let external = External; // Create a draft proposal let draft = external.propose(); // Build and send the proposal let proposal_id = draft .external_function_call("contract.near", "increment", "{}", 0) .send(); ``` ### System Functions The system provides low-level WASM functions for proposal management: ```rust use calimero_sdk::sys; // Send a proposal (returns proposal ID) let proposal_id = unsafe { sys::send_proposal(actions_buffer, result_buffer) }; // Approve a proposal unsafe { sys::approve_proposal(proposal_id_buffer) }; ``` ## Two Ways to Access External Functions Calimero provides **two equivalent ways** to access external functionality: ### 1. Via `Self::external()` (Macro-Generated) When you use `#[app::state]` on your struct, the macro system automatically generates a `Self::external()` method: ```rust #[app::state] #[derive(Default, BorshSerialize, BorshDeserialize)] #[borsh(crate = "calimero_sdk::borsh")] struct MyApp { // ... your app state } #[app::logic] impl MyApp { pub fn create_proposal(&mut self) -> ProposalId { // ✅ Self::external() is automatically generated by #[app::state] macro Self::external() .propose() .external_function_call("contract.near", "increment", "{}", 0) .send() } pub fn approve_proposal(&mut self, proposal_id: ProposalId) { // ✅ Self::external() works for approval too Self::external().approve(proposal_id); } } ``` ### 2. Via Direct `External` Usage You can also use the `External` struct directly: ```rust use calimero_sdk::env::ext::External; #[app::logic] impl MyApp { pub fn create_proposal(&mut self) -> ProposalId { // ✅ Direct External usage External .propose() .external_function_call("contract.near", "increment", "{}", 0) .send() } pub fn approve_proposal(&mut self, proposal_id: ProposalId) { // ✅ Direct External usage External::approve(proposal_id); } } ``` ### 3. Both Approaches Are Equivalent ```rust // These two lines do exactly the same thing: Self::external().approve(proposal_id); External::approve(proposal_id); // The macro generates this implementation: impl MyApp { fn external() -> ::calimero_sdk::env::ext::External { ::calimero_sdk::env::ext::External {} } } ``` **Real-world example**: The [demo-blockchain-integrations](https://github.com/calimero-network/demo-blockchain-integrations) repository uses `Self::external()` extensively: ```rust // From demo-blockchain-integrations/logic/src/lib.rs Self::external() .propose() .external_function_call( receiver_id.to_string(), method_name.to_string(), args.to_string(), deposit, ) .send() ``` ## Proposal Actions Calimero supports several types of proposal actions: ### 1. External Function Call Execute a function on an external contract: ```rust use calimero_sdk::env::ext::{ProposalAction, AccountId}; let action = ProposalAction::ExternalFunctionCall { receiver_id: AccountId("contract.near".to_string()), method_name: "increment".to_string(), args: "{}".to_string(), deposit: 0, }; ``` ### 2. Transfer Transfer tokens to an account: ```rust let action = ProposalAction::Transfer { receiver_id: AccountId("user.near".to_string()), amount: 1_000_000_000_000_000_000_000_000, // 1 NEAR }; ``` ### 3. Context Configuration Modify application context settings: ```rust // Set number of required approvals let action = ProposalAction::SetNumApprovals { num_approvals: 3, }; // Set active proposals limit let action = ProposalAction::SetActiveProposalsLimit { active_proposals_limit: 10, }; // Set context value let action = ProposalAction::SetContextValue { key: b"config_key".to_vec().into_boxed_slice(), value: b"config_value".to_vec().into_boxed_slice(), }; ``` ### 4. Proposal Management Delete existing proposals: ```rust let action = ProposalAction::DeleteProposal { proposal_id: ProposalId([0u8; 32]), }; ``` ## Creating Proposals ### Using DraftProposal Builder The recommended way to create proposals is using the `DraftProposal` builder pattern: ```rust use calimero_sdk::env::ext::{External, DraftProposal, AccountId}; #[app::logic] impl MyApp { // Using Self::external() (macro-generated) pub fn create_transfer_proposal_via_self(&mut self, receiver: String, amount: u128) -> ProposalId { Self::external() .propose() .transfer(AccountId(receiver), amount) .send() } // Using External directly pub fn create_transfer_proposal_via_external(&mut self, receiver: String, amount: u128) -> ProposalId { External .propose() .transfer(AccountId(receiver), amount) .send() } pub fn create_function_call_proposal( &mut self, receiver_id: String, method_name: String, args: String, deposit: u128, ) -> ProposalId { // Both approaches work identically Self::external() .propose() .external_function_call(receiver_id, method_name, args, deposit) .send() } pub fn create_context_modification_proposal( &mut self, key: Vec, value: Vec, ) -> ProposalId { Self::external() .propose() .set_context_value(key.into_boxed_slice(), value.into_boxed_slice()) .send() } } ``` ### Advanced Proposal Creation with Multiple Actions Create complex proposals with multiple actions: ```rust #[app::logic] impl MyApp { pub fn create_complex_proposal(&mut self) -> ProposalId { Self::external() .propose() .transfer(AccountId("treasury.near".to_string()), 1_000_000_000_000_000_000_000_000) .external_function_call( "governance.near".to_string(), "propose_upgrade".to_string(), r#"{"new_code_hash": "abc123"}"#.to_string(), 0, ) .set_num_approvals(5) .set_active_proposals_limit(20) .send() } } ``` ### Manual Proposal Creation For advanced use cases, you can create proposals manually: ```rust use calimero_sdk::env::ext::{ProposalAction, ProposalId}; use calimero_sdk::sys; #[app::logic] impl MyApp { pub fn create_manual_proposal(&mut self, actions: Vec) -> ProposalId { // Serialize actions to Borsh let actions_data = borsh::to_vec(&actions) .expect("Failed to serialize actions"); // Create buffer for result let mut result_buffer = [0u8; 32]; // Send proposal using system function unsafe { sys::send_proposal( calimero_sdk::sys::Buffer::from(&actions_data), calimero_sdk::sys::BufferMut::new(&mut result_buffer) ) } ProposalId(result_buffer) } } ``` ## Approving Proposals ### Basic Approval Both approaches work identically: ```rust use calimero_sdk::env::ext::{External, ProposalId}; #[app::logic] impl MyApp { // Using Self::external() (macro-generated) pub fn approve_proposal_via_self(&mut self, proposal_id: ProposalId) { Self::external().approve(proposal_id); } // Using External directly pub fn approve_proposal_via_external(&mut self, proposal_id: ProposalId) { External::approve(proposal_id); } } ``` ### Approval with Validation ```rust #[app::logic] impl MyApp { pub fn approve_proposal_with_validation(&mut self, proposal_id: ProposalId) -> Result { // Check if user can approve if !self.can_approve_proposals(&env::predecessor_account_id()) { return Err("Insufficient permissions to approve proposals".to_string()); } // Check if proposal exists and is not already approved if let Some(proposal) = self.proposals.get(&proposal_id) { if proposal.approved { return Err("Proposal already approved".to_string()); } // Mark as approved locally if let Some(mut proposal) = self.proposals.get_mut(&proposal_id) { proposal.approved = true; proposal.approver = Some(env::predecessor_account_id()); } // Approve externally - both approaches work Self::external().approve(proposal_id); // OR: External::approve(proposal_id); Ok(()) } else { Err("Proposal not found".to_string()) } } fn can_approve_proposals(&self, user: &str) -> bool { self.governance_members.contains(user) } } ``` ## Complete Examples ### Governance DAO Example This example implements a simple DAO where members can create treasury transfer proposals and approve them; proposal data is stored locally while creation/approval is executed via the external proposal system (`Self::external()` / `External`). ```rust use calimero_sdk::{app, env, env::ext::{External, DraftProposal, ProposalAction, AccountId, ProposalId}}; use calimero_storage::collections::{UnorderedMap, UnorderedSet}; #[app::state] #[derive(Default, BorshSerialize, BorshDeserialize)] #[borsh(crate = "calimero_sdk::borsh")] struct DAO { members: UnorderedSet, proposals: UnorderedMap, treasury_balance: u128, } #[derive(BorshSerialize, BorshDeserialize)] pub struct LocalProposal { pub id: ProposalId, pub creator: String, pub actions: Vec, pub approvals: UnorderedSet, pub executed: bool, pub created_at: u64, } #[app::logic] impl DAO { #[app::init] pub fn init(initial_members: Vec) -> Self { let mut members = UnorderedSet::new(); for member in initial_members { members.insert(member); } Self { members, proposals: UnorderedMap::new(), treasury_balance: 0, } } pub fn create_treasury_transfer_proposal( &mut self, receiver_id: String, amount: u128, ) -> Result { if !self.members.contains(&env::predecessor_account_id()) { return Err("Only members can create proposals".to_string()); } if amount > self.treasury_balance { return Err("Insufficient treasury balance".to_string()); } // Create external proposal using Self::external() (macro-generated) let proposal_id = Self::external() .propose() .transfer(AccountId(receiver_id.clone()), amount) .send(); // Store proposal locally let local_proposal = LocalProposal { id: proposal_id, creator: env::predecessor_account_id(), actions: vec![ProposalAction::Transfer { receiver_id: AccountId(receiver_id), amount, }], approvals: UnorderedSet::new(), executed: false, created_at: env::block_timestamp(), }; self.proposals.insert(proposal_id, local_proposal); Ok(proposal_id) } pub fn approve_proposal(&mut self, proposal_id: ProposalId) -> Result { if !self.members.contains(&env::predecessor_account_id()) { return Err("Only members can approve proposals".to_string()); } let mut proposal = self.proposals.get_mut(&proposal_id) .ok_or("Proposal not found")?; if proposal.executed { return Err("Proposal already executed".to_string()); } let approver = env::predecessor_account_id(); if proposal.approvals.contains(&approver) { return Err("Already approved this proposal".to_string()); } // Add approval locally proposal.approvals.insert(approver.clone()); // Approve externally - both approaches work identically Self::external().approve(proposal_id); // OR: External::approve(proposal_id); Ok(()) } pub fn get_proposal_status(&self, proposal_id: ProposalId) -> Option { self.proposals.get(&proposal_id).map(|proposal| { if proposal.executed { ProposalStatus::Executed } else if proposal.approvals.len() >= 3 { // Assuming 3 approvals required ProposalStatus::ReadyToExecute } else { ProposalStatus::PendingApproval { current_approvals: proposal.approvals.len() as u32, required: 3, } } }) } } #[derive(BorshSerialize, BorshDeserialize)] pub enum ProposalStatus { PendingApproval { current_approvals: u32, required: u32 }, ReadyToExecute, Executed, } ``` ### Cross-Chain Bridge Example This example outlines a governance-controlled bridge flow: a local bridge proposal is created and persisted, while an external proposal triggers an on-chain bridge contract (initiate_bridge); approvals are tracked locally, and in production you would map and approve the corresponding external ProposalId to execute the bridge. ```rust use calimero_sdk::{app, env, env::ext::{External, DraftProposal, ProposalAction, AccountId}}; #[app::state] #[derive(Default, BorshSerialize, BorshDeserialize)] #[borsh(crate = "calimero_sdk::borsh")] struct CrossChainBridge { bridge_operators: UnorderedSet, pending_bridges: UnorderedMap, bridge_counter: u64, } #[derive(BorshSerialize, BorshDeserialize)] pub struct BridgeProposal { pub id: u64, pub source_chain: String, pub target_chain: String, pub amount: u128, pub recipient: String, pub status: BridgeStatus, } #[derive(BorshSerialize, BorshDeserialize)] pub enum BridgeStatus { Pending, Approved, Executed, Failed, } #[app::logic] impl CrossChainBridge { pub fn create_bridge_proposal( &mut self, source_chain: String, target_chain: String, amount: u128, recipient: String, ) -> Result { let proposal_id = self.bridge_counter; self.bridge_counter += 1; let bridge_proposal = BridgeProposal { id: proposal_id, source_chain, target_chain, amount, recipient, status: BridgeStatus::Pending, }; self.pending_bridges.insert(proposal_id, bridge_proposal); // Create external proposal for the bridge operation // Both approaches work identically let _external_proposal_id = Self::external() .propose() .external_function_call( "bridge.near".to_string(), "initiate_bridge".to_string(), serde_json::to_string(&bridge_proposal).unwrap(), 0, ) .send(); Ok(proposal_id) } pub fn approve_bridge(&mut self, proposal_id: u64) -> Result { if !self.bridge_operators.contains(&env::predecessor_account_id()) { return Err("Only bridge operators can approve bridges".to_string()); } let mut bridge = self.pending_bridges.get_mut(&proposal_id) .ok_or("Bridge proposal not found")?; if bridge.status != BridgeStatus::Pending { return Err("Bridge is not in pending status".to_string()); } bridge.status = BridgeStatus::Approved; // Note: In a real implementation, you would need to map the local proposal ID // to the external proposal ID for approval // For now, we'll just update the local status Ok(()) } } ``` ## Best Practices ### 1. Always Validate Proposals ```rust pub fn create_proposal(&mut self, actions: Vec) -> Result { // Validate proposal parameters self.validate_proposal_actions(&actions)?; // Check permissions if !self.can_create_proposals(&env::predecessor_account_id()) { return Err("Insufficient permissions".to_string()); } // Create proposal - both approaches work let proposal_id = Self::external().propose(); // OR: let proposal_id = External::propose(); // Add actions one by one with validation for action in actions { proposal_id = self.add_validated_action(proposal_id, action)?; } Ok(proposal_id.send()) } fn validate_proposal_actions(&self, actions: &[ProposalAction]) -> Result { for action in actions { match action { ProposalAction::ExternalFunctionCall { deposit, .. } => { if *deposit > self.max_proposal_deposit { return Err("Deposit too high".to_string()); } } ProposalAction::Transfer { amount, .. } => { if *amount > self.max_transfer_amount { return Err("Transfer amount too high".to_string()); } } _ => {} } } Ok(()) } ``` ### 2. Use Events for Tracking ```rust #[app::event] pub enum GovernanceEvent { ProposalCreated { id: &'a ProposalId, creator: &'a str }, ProposalApproved { id: &'a ProposalId, approver: &'a str }, ProposalExecuted { id: &'a ProposalId }, } #[app::logic] impl GovernanceApp { pub fn approve_proposal(&mut self, proposal_id: ProposalId) -> Result { // ... approval logic ... app::emit!(GovernanceEvent::ProposalApproved { id: &proposal_id, approver: &env::predecessor_account_id(), }); Ok(()) } } ``` ### 3. Implement Proper Error Handling ```rust pub fn approve_proposal(&mut self, proposal_id: ProposalId) -> Result { let proposal = self.proposals.get(&proposal_id) .ok_or("Proposal not found")?; if proposal.executed { return Err("Proposal already executed".to_string()); } if !self.can_approve_proposals(&env::predecessor_account_id()) { return Err("Insufficient permissions".to_string()); } // Approve with error handling - both approaches work Self::external().approve(proposal_id); // OR: External::approve(proposal_id); // Update local state if let Some(mut proposal) = self.proposals.get_mut(&proposal_id) { proposal.approvals.insert(env::predecessor_account_id()); } Ok(()) } ``` ### 4. Use Batch Operations for Efficiency ```rust pub fn approve_multiple_proposals(&mut self, proposal_ids: Vec) -> Result { for proposal_id in proposal_ids { self.approve_proposal(proposal_id)?; } Ok(()) } ``` ## Important Notes ### ✅ **Both APIs Are Valid and Equivalent** 1. **`Self::external()`** - Generated automatically by `#[app::state]` macro 2. **`External::`** - Direct usage of the External struct 3. **Both work identically** - Choose the style that fits your codebase ### 🔄 **How It Actually Works** 1. **Macro Generation**: `#[app::state]` automatically generates `Self::external()` method 2. **Proposal Creation**: Use either `Self::external().propose()` or `External::propose()` (they are equivalent - `Self::external()` returns an `External` instance) 3. **Action Building**: Use the `DraftProposal` builder methods to add actions 4. **Proposal Submission**: Call `.send()` to submit the proposal and get a `ProposalId` 5. **Approval**: Use either `Self::external().approve()` or `External::approve()` 6. **External Execution**: Proposals are executed by the external proxy contract system ### 📚 **Next Steps** This guide covers the essential aspects of Calimero's external proposal system. For more advanced topics, explore: - [Building Complex Applications](./rust-sdk-deep-dive) - [Calimero Collections Reference](./calimero-collections) Remember to always test your governance systems thoroughly and implement proper security measures for production deployments. --- ## TypeScript Client SDK ## Getting Started with Calimero SDK for Typescript Our TypeScript Client SDK is a powerful tool designed to simplify the process of interacting with decentralized peer-to-peer applications installed on the node. Install the SDK using npm: ```bash npm install @calimero-network/calimero-client ``` ## Core Components The TypeScript Client SDK consists of several key components: ### 1. RpcClient (JsonRpcClient) Handles communication with the node's RPC server for executing queries and mutations. ### 2. SubscriptionsClient (WsSubscriptionsClient) Manages WebSocket connections for real-time updates and event subscriptions. ### 3. HttpClient (AxiosHttpClient) Provides HTTP communication capabilities for admin API interactions. ### 4. Authentication Components - `AccessTokenWrapper`: Manages JWT token lifecycle - `ClientLogin`: Handles user authentication flow - `SetupModal`: Configures node URL and application settings ### 5. Storage Utilities Manages local storage for: - Access tokens - Refresh tokens - Application URLs - Context identities ## Authentication Setup ### 1. Client Login Component ```typescript import { ClientLogin } from '@calimero-network/calimero-client'; const LoginPage = () => { return ( process.env.NEXT_PUBLIC_API_URL} getApplicationId={() => process.env.NEXT_PUBLIC_APPLICATION_ID} successRedirect={() => navigate('/dashboard')} /> ); }; ``` ### 2. Token Management ```typescript import { AccessTokenWrapper } from '@calimero-network/calimero-client'; const App = () => { return ( process.env.NEXT_PUBLIC_API_URL} > ); }; ``` ### 3. Initial Setup ```typescript import { SetupModal } from '@calimero-network/calimero-client'; const Setup = () => { return ( navigate('/dashboard')} getNodeUrl={() => localStorage.getItem('nodeUrl')} setNodeUrl={(url) => localStorage.setItem('nodeUrl', url)} getApplicationId={() => localStorage.getItem('appId')} setApplicationId={(id) => localStorage.setItem('appId', id)} /> ); }; ``` ### 4. Token Storage ```typescript import { setAccessToken, getAccessToken, setRefreshToken, getRefreshToken, clearJWT, } from '@calimero-network/calimero-client'; // Store tokens setAccessToken(accessToken); setRefreshToken(refreshToken); // Retrieve tokens const accessToken = getAccessToken(); const refreshToken = getRefreshToken(); // Clear tokens on logout clearJWT(); ``` ## Using RpcClient Important: Always use the `execute` method instead of deprecated `query` or `mutate` methods. ```typescript import { JsonRpcClient } from '@calimero-network/calimero-client'; // Initialize client const rpcClient = new JsonRpcClient( process.env.NEXT_PUBLIC_API_URL, '/jsonrpc', ); // Define interfaces interface CreatePost { title: string; text: string; } interface Post { id: string; title: string; text: string; } // Execute a method const response = await rpcClient.execute({ applicationId: process.env.NEXT_PUBLIC_APPLICATION_ID, method: 'create_post', argsJson: { title: 'My First Post', text: 'Hello Calimero!', }, }); ``` ## Using SubscriptionsClient ```typescript import { WsSubscriptionsClient } from '@calimero-network/calimero-client'; // Initialize client const wsClient = new WsSubscriptionsClient( process.env.NEXT_PUBLIC_API_URL, '/ws', ); // Connect to WebSocket await wsClient.connect(); // Subscribe to application events wsClient.subscribe([process.env.NEXT_PUBLIC_APPLICATION_ID]); // Handle incoming events wsClient.addCallback((event) => { switch (event.type) { case 'StateMutation': console.log('State updated:', event.data.newRoot); break; case 'ExecutionEvent': console.log('Event:', event.data); break; } }); // Cleanup wsClient.disconnect(); ``` ## Contract API Integration ```typescript import { ContractApiDataSource } from '@calimero-network/calimero-client'; const contractApi = new ContractApiDataSource(httpClient); // Get proposals const proposals = await contractApi.getContractProposals(); // Get proposal details const details = await contractApi.getProposalDetails(proposalId); // Get number of approvals const approvals = await contractApi.getNumberOfApprovals(proposalId); ``` ## Error Handling ```typescript interface ErrorResponse { code?: number; message: string; } try { const result = await rpcClient.execute(params); if (result.error) { if (result.error.code === 401) { // Handle authentication error clearJWT(); redirectToLogin(); } console.error('RPC Error:', result.error.message); } } catch (error) { console.error('Unexpected error:', error); } ``` ## Admin API Endpoints ### Protected Endpoints (Requires Authentication) #### Application Management - `POST /admin-api/root-key` - Create root key - `POST /admin-api/install-application` - Install application - `POST /admin-api/uninstall-application` - Uninstall application - `GET /admin-api/applications` - List applications - `GET /admin-api/applications/:app_id` - Get application details #### Context Management - `POST /admin-api/contexts` - Create context - `GET /admin-api/contexts` - List contexts - `GET /admin-api/contexts/:context_id` - Get context details - `DELETE /admin-api/contexts/:context_id` - Delete context - `GET /admin-api/contexts/:context_id/users` - Get context users - `GET /admin-api/contexts/:context_id/client-keys` - Get context client keys - `GET /admin-api/contexts/:context_id/storage` - Get context storage - `GET /admin-api/contexts/:context_id/identities` - Get context identities - `POST /admin-api/contexts/invite` - Invite to context - `POST /admin-api/contexts/join` - Join context - `POST /admin-api/contexts/:context_id/capabilities/grant` - Grant capabilities to context identities - `POST /admin-api/contexts/:context_id/capabilities/revoke` - Revoke capabilities from context identities #### Identity Management - `POST /admin-api/identity/context` - Generate context identity - `DELETE /admin-api/identity/keys` - Delete auth keys - `POST /admin-api/generate-jwt-token` - Generate JWT token - `GET /admin-api/did` - Fetch DID - `DELETE /admin-api/did` - Delete DID #### Alias Management ##### Create routes ```http title="POST /admin-api/alias/create/context" { "alias": "string", "value": { "contextId": "ContextId" } } ``` - Generate context identity ```http title="POST /admin-api/alias/create/identity/{context_id}" { "alias": "string", "value": { "identity": "PublicKey" } } ``` - Creates an alias for an identity within a specific context. ```http title="POST /admin-api/alias/create/application" { "alias": "string", "value": { "applicationId": "ApplicationId" } } ``` - Creates an alias for an application ID. ##### Capabilities Management ```http title="POST /admin-api/contexts/:context_id/capabilities/grant" { "capabilities": [["ContextIdentity", "Capability"]], "signer_id": "PublicKey" } ``` - Grants specific capabilities to context identities. ```http title="POST /admin-api/contexts/:context_id/capabilities/revoke" { "capabilities": [["ContextIdentity", "Capability"]], "signer_id": "PublicKey" } ``` - Revokes specific capabilities from context identities. ##### Lookup routes - `POST /admin-api/alias/lookup/context/:name` - Generate context identity - `POST /admin-api/alias/lookup/application/:name` - Generate context identity - `POST /admin-api/alias/lookup/identity/:context/:name` - Generate context identity ##### Delete routes - `POST /admin-api/alias/delete/context/:name` - Generate context identity - `POST /admin-api/alias/delete/application/:name` - Generate context identity - `POST /admin-api/alias/delete/identity/:context/:name` - Generate context identity ### Unprotected Endpoints #### Authentication - `GET /admin-api/health` - Health check - `GET /admin-api/certificate` - Get certificate - `POST /admin-api/request-challenge` - Request challenge - `POST /admin-api/add-client-key` - Add client key - `POST /admin-api/refresh-jwt-token` - Refresh JWT token #### Proposal Management - `GET /admin-api/contexts/:context_id/proposals/:proposal_id/approvals/count` - Get proposal approvals count - `GET /admin-api/contexts/:context_id/proposals/:proposal_id/approvals/users` - Get proposal approvers - `GET /admin-api/contexts/:context_id/proposals/count` - Get active proposals count - `POST /admin-api/contexts/:context_id/proposals` - Get proposals - `GET /admin-api/contexts/:context_id/proposals/:proposal_id` - Get proposal details - `POST /admin-api/contexts/:context_id/proposals/get-context-value` - Get context value - `POST /admin-api/contexts/:context_id/proposals/context-storage-entries` - Get context storage entries - `GET /admin-api/contexts/:context_id/proxy-contract` - Get proxy contract ## Using Admin API Any endpoints not exposed throught the SDK you can use like this: ```typescript import { AxiosHttpClient } from '@calimero-network/calimero-client'; const httpClient = new AxiosHttpClient(axios); // Example: Get context details const contextDetails = await httpClient.get( `${nodeUrl}/admin-api/contexts/${contextId}`, ); // Example: Create context const createContext = await httpClient.post( `${nodeUrl}/admin-api/contexts`, contextData, ); ``` ## Important Notes 1. Always use the `execute` method for RPC calls instead of deprecated `query` or `mutate` 2. Implement proper error handling for both RPC and WebSocket operations 3. Ensure proper cleanup of WebSocket connections when they're no longer needed 4. Store sensitive information securely using the provided storage utilities 5. Use the built-in token refresh mechanism to maintain sessions 6. All protected endpoints require valid JWT authentication --- ## Python Client SDK # Python Client SDK The Calimero Python Client SDK provides a comprehensive, high-performance interface for interacting with Calimero Network APIs. Built with Rust and PyO3, it offers native performance with the ease of Python development. ## Overview The Python client library is designed for developers who want to build Calimero applications using Python, whether for web development, data analysis, automation, or integration with existing Python ecosystems. ### Key Features - **🚀 High Performance**: Built with Rust and PyO3 for optimal performance. - **🔒 Type Safety**: Strongly typed Python bindings with full IDE support. - **⚡ Synchronous Interface**: Clean synchronous API with Rust-powered backend. - **🌐 Multi-Protocol**: Support for NEAR, Ethereum, ICP, and Starknet networks. - **📦 Easy Installation**: Simple `pipx install` with no complex dependencies. - **🔧 Comprehensive API**: Full access to all Calimero Network functionality. ## Supported Protocols - ✅ **NEAR Protocol** - Account-based architecture with optimized storage. - ✅ **Ethereum** - Full EVM compatibility with gas optimization. - ✅ **Internet Computer Protocol (ICP)** - Canister-based execution model. - ✅ **Starknet** - Cairo-based smart contracts with account abstraction. ## Quick Start ### Installation Install the Python client from PyPI: ```bash pipx install calimero-client-py pipx ensurepath ``` ### Basic Usage ```python import calimero def main(): # Create a connection connection = calimero.create_connection( api_url="http://localhost:2428", node_name="example-node" ) # Create a client client = calimero.create_client(connection) # List available contexts contexts = client.list_contexts() print(f"Found {len(contexts)} contexts") # List applications apps = client.list_applications() print(f"Found {len(apps)} applications") if __name__ == "__main__": main() ``` ### CLI Usage The Python client includes a command-line interface: ```bash # Install the CLI pipx install calimero-client-py pipx ensurepath # Use the CLI (requires a running Calimero node) calimero-client-py --help calimero-client-py --version calimero-client-py --base-url http://localhost:2428 list-contexts ``` **Note**: The CLI requires a running Calimero node to function properly. If no node is running, you'll get connection errors. ## Architecture ### Synchronous Interface The client provides a clean synchronous interface with high performance: ```python import calimero def example(): connection = calimero.create_connection("http://localhost:2428") client = calimero.create_client(connection) # All operations are synchronous contexts = client.list_contexts() apps = client.list_applications() return contexts, apps ``` ## Core Components ### Connection Management ```python import calimero # Basic connection connection = calimero.create_connection( api_url="http://localhost:2428", node_name="my-node" ) # Create client client = calimero.create_client(connection) ``` ### Client Operations The client provides comprehensive access to Calimero functionality: #### Connection Management - `get_api_url()`: Get the API URL for this client - `get_peers_count()`: Get the number of connected peers #### Application Management - `get_application(app_id: str)`: Get information about a specific application - `list_applications()`: List all available applications - `install_application(url: str, hash: Optional[str], metadata: Optional[bytes])`: Install application from URL - `install_dev_application(path: str, metadata: Optional[bytes])`: Install development application from local path - `uninstall_application(app_id: str)`: Uninstall an application #### Context Management - `get_context(context_id: str)`: Get information about a specific context - `list_contexts()`: List all available contexts - `create_context(application_id: str, protocol: str, params: Optional[str])`: Create a new context - `delete_context(context_id: str)`: Delete a context - `sync_context(context_id: str)`: Sync a specific context - `sync_all_contexts()`: Sync all contexts #### Context Operations - `get_context_storage(context_id: str)`: Get context storage information - `get_context_identities(context_id: str)`: Get identities associated with a context - `get_context_client_keys(context_id: str)`: Get client keys for a context - `invite_to_context(context_id: str, inviter_id: str, invitee_id: str)`: Invite someone to a context - `join_context(context_id: str, invitee_id: str, invitation_payload: str)`: Join a context using invitation - `update_context_application(context_id: str, application_id: str, executor_public_key: str)`: Update context application #### Function Execution - `execute_function(context_id: str, method: str, args: str, executor_public_key: str)`: Execute a function call via JSON-RPC #### Permission Management - `grant_permissions(context_id: str, permissions: str)`: Grant permissions to users in a context - `revoke_permissions(context_id: str, permissions: str)`: Revoke permissions from users in a context #### Proposal Management - `get_proposal(context_id: str, proposal_id: str)`: Get proposal information - `get_proposal_approvers(context_id: str, proposal_id: str)`: Get proposal approvers - `list_proposals(context_id: str, args: Optional[str])`: List proposals in a context #### Identity Management - `generate_context_identity()`: Generate a new context identity #### Blob Management - `list_blobs()`: List all blobs - `get_blob_info(blob_id: str)`: Get information about a specific blob - `delete_blob(blob_id: str)`: Delete a blob #### Alias Management - `create_context_identity_alias(context_id: str, alias: str, public_key: str)`: Create context identity alias - `create_context_alias(alias: str, context_id: str)`: Create context alias - `create_application_alias(alias: str, application_id: str)`: Create application alias - `delete_context_alias(alias: str)`: Delete context alias - `delete_context_identity_alias(alias: str, context_id: str)`: Delete context identity alias - `delete_application_alias(alias: str)`: Delete application alias - `list_context_aliases()`: List context aliases - `list_context_identity_aliases(context_id: str)`: List context identity aliases - `list_application_aliases()`: List application aliases - `lookup_context_alias(alias: str)`: Lookup context alias - `lookup_context_identity_alias(alias: str, context_id: str)`: Lookup context identity alias - `lookup_application_alias(alias: str)`: Lookup application alias - `resolve_context_alias(alias: str)`: Resolve context alias - `resolve_context_identity_alias(alias: str, context_id: str)`: Resolve context identity alias - `resolve_application_alias(alias: str)`: Resolve application alias ## Error Handling The client provides comprehensive error handling: ```python import calimero try: result = client.execute_function( context_id="context-123", method="transfer", args='{"amount": 100}', executor_public_key="key-123" ) except RuntimeError as e: print(f"Client error: {e}") except Exception as e: print(f"Unexpected error: {e}") ``` ## Development Workflow ### Local Development ```bash # Clone the repository git clone https://github.com/calimero-network/calimero-client-py cd calimero-client-py # Install in development mode pip install -e . # Run tests pytest # Run example python example_usage.py ``` ### Building from Source ```bash # Install maturin pip install maturin # Build the package maturin build --release # Install the built package pip install target/wheels/calimero_client_py-*.whl ``` ## Installation ### Prerequisites Before installing the Python client, ensure you have: - **Python 3.9+** (3.11+ recommended for best performance). - **pipx** (recommended) or **pip** package manager. - **Rust toolchain** (only needed for building from source). ### PyPI Installation (Recommended) The easiest way to install the Python client is from PyPI: ```bash pipx install calimero-client-py pipx ensurepath ``` #### Verify Installation ```python # Test the installation python -c "import calimero; print('Installation successful!')" # Check version python -c "import calimero; print(calimero.__version__)" ``` ### Virtual Environment (Recommended for Development) Create an isolated environment for your project: ```bash # Create virtual environment python -m venv calimero-env # Activate virtual environment # On Windows: calimero-env\Scripts\activate # On macOS/Linux: source calimero-env/bin/activate # Install the client pip install calimero-client-py ``` **Alternative**: Use `pipx` for a cleaner installation: ```bash pipx install calimero-client-py pipx ensurepath ``` ### Building from Source For development or if you need the latest features: ```bash # Install Rust toolchain curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh source ~/.cargo/env # Install maturin (Python-Rust build tool) pip install maturin # Clone and build git clone https://github.com/calimero-network/calimero-client-py cd calimero-client-py maturin build --release pip install target/wheels/calimero_client_py-*.whl ``` ## API Reference ### Connection Management ```python import calimero # Basic connection connection = calimero.create_connection( api_url="http://localhost:2428", node_name="my-node" ) # Create client client = calimero.create_client(connection) ``` ### Client Operations The client provides comprehensive access to Calimero functionality: #### Connection Management - `get_api_url()`: Get the API URL for this client - `get_peers_count()`: Get the number of connected peers #### Application Management - `get_application(app_id: str)`: Get information about a specific application - `list_applications()`: List all available applications - `install_application(url: str, hash: Optional[str], metadata: Optional[bytes])`: Install application from URL - `install_dev_application(path: str, metadata: Optional[bytes])`: Install development application from local path - `uninstall_application(app_id: str)`: Uninstall an application #### Context Management - `get_context(context_id: str)`: Get information about a specific context - `list_contexts()`: List all available contexts - `create_context(application_id: str, protocol: str, params: Optional[str])`: Create a new context - `delete_context(context_id: str)`: Delete a context - `sync_context(context_id: str)`: Sync a specific context - `sync_all_contexts()`: Sync all contexts #### Context Operations - `get_context_storage(context_id: str)`: Get context storage information - `get_context_identities(context_id: str)`: Get identities associated with a context - `get_context_client_keys(context_id: str)`: Get client keys for a context - `invite_to_context(context_id: str, inviter_id: str, invitee_id: str)`: Invite someone to a context - `join_context(context_id: str, invitee_id: str, invitation_payload: str)`: Join a context using invitation - `update_context_application(context_id: str, application_id: str, executor_public_key: str)`: Update context application #### Function Execution - `execute_function(context_id: str, method: str, args: str, executor_public_key: str)`: Execute a function call via JSON-RPC #### Permission Management - `grant_permissions(context_id: str, permissions: str)`: Grant permissions to users in a context - `revoke_permissions(context_id: str, permissions: str)`: Revoke permissions from users in a context #### Proposal Management - `get_proposal(context_id: str, proposal_id: str)`: Get proposal information - `get_proposal_approvers(context_id: str, proposal_id: str)`: Get proposal approvers - `list_proposals(context_id: str, args: Optional[str])`: List proposals in a context #### Identity Management - `generate_context_identity()`: Generate a new context identity #### Blob Management - `list_blobs()`: List all blobs - `get_blob_info(blob_id: str)`: Get information about a specific blob - `delete_blob(blob_id: str)`: Delete a blob #### Alias Management - `create_context_identity_alias(context_id: str, alias: str, public_key: str)`: Create context identity alias - `create_context_alias(alias: str, context_id: str)`: Create context alias - `create_application_alias(alias: str, application_id: str)`: Create application alias - `delete_context_alias(alias: str)`: Delete context alias - `delete_context_identity_alias(alias: str, context_id: str)`: Delete context identity alias - `delete_application_alias(alias: str)`: Delete application alias - `list_context_aliases()`: List context aliases - `list_context_identity_aliases(context_id: str)`: List context identity aliases - `list_application_aliases()`: List application aliases - `lookup_context_alias(alias: str)`: Lookup context alias - `lookup_context_identity_alias(alias: str, context_id: str)`: Lookup context identity alias - `lookup_application_alias(alias: str)`: Lookup application alias - `resolve_context_alias(alias: str)`: Resolve context alias - `resolve_context_identity_alias(alias: str, context_id: str)`: Resolve context identity alias - `resolve_application_alias(alias: str)`: Resolve application alias ### Error Handling ```python import calimero try: result = client.execute_function( context_id="context-123", method="transfer", args='{"amount": 100}', executor_public_key="key-123" ) except RuntimeError as e: print(f"Client error: {e}") except Exception as e: print(f"Unexpected error: {e}") ``` ## Examples ### Basic Usage ```python import calimero def main(): # Create a connection connection = calimero.create_connection( api_url="http://localhost:2428", node_name="example-node" ) # Create a client client = calimero.create_client(connection) # List available contexts contexts = client.list_contexts() print(f"Found {len(contexts)} contexts") # List applications apps = client.list_applications() print(f"Found {len(apps)} applications") if __name__ == "__main__": main() ``` ## Resources - **GitHub Repository**: [calimero-client-py](https://github.com/calimero-network/calimero-client-py) - **PyPI Package**: [calimero-client-py](https://pypi.org/project/calimero-client-py/) - **API Documentation**: Complete method reference - **Examples Repository**: [calimero-examples](https://github.com/calimero-network/examples) ## Getting Help - **Documentation**: Comprehensive guides and examples - **GitHub Issues**: [Report bugs and request features](https://github.com/calimero-network/calimero-client-py/issues) - **Community Discord**: [Join the conversation](https://discord.gg/calimero) - **Email Support**: [team@calimero.network](mailto:team@calimero.network) ## Next Steps Ready to get started with the Python client? 1. **Follow the Installation section above** to set up your environment 2. **Explore the API Reference section** to understand available methods 3. **Check out the Examples section** for practical usage patterns The Python client brings the power and performance of Calimero Network to the Python ecosystem, enabling you to build sophisticated decentralized applications with the tools you know and love. --- ## Client SDKs # Client SDKs Client SDKs enable frontend and backend development for Calimero applications in different programming languages. ## Available SDKs ### TypeScript Client [TypeScript Client](/developer-tools/SDK/client-sdk/client-ts-sdk) enables frontend development in TypeScript, supporting wallet authentication and context data handling via JSON-RPC and WebSockets. ### Python Client [Python Client](/developer-tools/SDK/client-sdk/calimero-client-py) is a high-performance Python client library built with Rust and PyO3, perfect for backend services, data analysis, and automation workflows. ## Getting Started Choose the SDK that best fits your development needs: - **Frontend Development**: Use the [TypeScript Client](/developer-tools/SDK/client-sdk/client-ts-sdk) for web applications - **Backend Services**: Use the [Python Client](/developer-tools/SDK/client-sdk/calimero-client-py) for server-side applications - **Data Analysis**: Use the [Python Client](/developer-tools/SDK/client-sdk/calimero-client-py) for data processing and analysis - **Automation**: Use the [Python Client](/developer-tools/SDK/client-sdk/calimero-client-py) for workflow automation ## Features Both SDKs provide: - **Full API Coverage**: Complete access to all Calimero Network functionality - **Type Safety**: Strongly typed interfaces for better development experience - **Error Handling**: Comprehensive error handling and recovery - **Documentation**: Extensive documentation and examples - **Community Support**: Active community and support channels ## Next Steps - **Choose your SDK**: Select the appropriate SDK for your use case - **Follow the Installation Guide**: Set up your development environment - **Explore Examples**: Check out practical usage examples - **Read the API Reference**: Understand all available methods and features --- ## Admin Dashboard Node is gated with authentication. In order to interact with the node from any app, you need to register a decentralized identity. We have build an Admin Dashboard which is a web application designed to streamline the management of node states within your system. Connected with the API provided by the Admin Client API, this user interface offers a user-friendly platform for overseeing and controlling various aspects of your node infrastructure. ### Access Admin Dashboard The node operates as a server for the Admin Dashboard and can be accessed directly through a web browser. The Admin Dashboard is locally available at `http://localhost:NODE_PORT/admin-dashboard/` but you can also use application deployed on Github Pages at `https://calimero-network.github.io/admin-dashboard/`. > **_NOTE:_** Update `NODE_PORT` in the placeholder `http://localhost:NODE_PORT` > with the `--server-port` value defined during node setup in > [Getting-started](../../getting-started/setup) ### Setup After the initial setup you will see the setup page where you need to enter node URL. ![SetupPage](/admin-dashboard-functions/1setup-node-url.png) > **NOTE**: Error occurs if the entered URL is not valid node url or the node is > not running ![SetupPageError](/admin-dashboard-functions/setup-node-url-error.png) ### Login After button "Set node url" is clicked you will be redirected to select wallet page. ![WalletSelector](/admin-dashboard-functions/2select-wallet.png) In this example we will be using Metamask with Ethereum Mainnet. Press "Connect wallet" button to invoke Metamask extension which is needed to continue login process. ![ConnectWallet](/admin-dashboard-functions/3connect-wallet.png) When wallet is connected you will be shown the next screen with displayed connected wallet address. ![ConnectedAccount](/admin-dashboard-functions/4logged-in-acc.png) To finish login process press "Sign authentication transaction" which will envoke Metamask to sign transaction. ![SignLoginTxn](/admin-dashboard-functions/5metamask-sign-msg-login.png) > **NOTE**: Displayed "BNB 2" text is just a wallet name and this is configued > in Metamask. You might have default wallet name "Account" or "Account 1" On the first login, your selected wallet will be used as a root key to use further functionalities. > **NOTE**: If you logout and try to login with different wallet that is not the > root key you will get the next error ![SinginError](/admin-dashboard-functions/7no-root-key-login.png) After this request is confimed you will be redirected to the Identity page of the Admin Dashboard where you can see your root key. ![IdentityPage](/admin-dashboard-functions/6identity.png) ### Add new root key To add new root key press the "Add new root key" button and you will go through the same process. In this example we will be using Metamask again but this time we will use BNB Chain network with different wallet. ![AddNewRootKey](/admin-dashboard-functions/8add-new-root-key.png) Again, sign the transaction. ![ConfirmNewRootKey](/admin-dashboard-functions/9add-root-key-txn.png) You will be redirected back the Identity page and in the list you can see that the new root key with BNB network was added. ![NewRootkeyList](/admin-dashboard-functions/10new-root-key-added.png) ### Applications Navigate to "Applications" tab to view table of available, owned or installed applications. ![Applications](/admin-dashboard-functions/12applications.png) #### Publish new application To publish new application click on the "Publish new application" button and you will be redirected to next page. > **NOTE**: For this step you will need a NEAR blockchain account because the > Package Manager application is a smart contract deployed on the NEAR > blockchain. > This is needed because from your account you interact with the smart contract > deployed on the NEAR blockchain > You can create NEAR blockchain account with > [MyNEARWallet](https://testnet.mynearwallet.com/) > Package Manager contains metadata of applications used by the node Metadata for each application contains: - Package name `String` - Description `String` - optional and can be set to "-" - Repository URL `String` - optional and can be set to "-" - Path `String` - IPFS URL to the application binary, filled automatically when application wasm is uploaded - Version `String` - has "X.X.X" format (e.g. 0.0.1 or 1.0.0) - Noted `String` - optional and can be set to "-" - Hash `String` - Hash value of application wasm, filled automatically when application wasm is uploaded ![PublishApp](/admin-dashboard-functions/13upload-new-app.png) After you fill in the Package name, Description, Repository URL click on the "Upload wasm" button and select your application `.wasm` file (e.g. only-peers application wasm). ![SelectWasmFile](/admin-dashboard-functions/13select-wasm.png) After file was uploaded to IPFS, Path and Hash field will be filled automatically and only thing left is to fill in version and notes. ![UploadedWasm](/admin-dashboard-functions/13filled-in-fields.png) After you click the "Publish" button, application will be published in the Package Manager contract and it will be available in the available applications list. You will get a status popup message. ![ApplicationPublished](/admin-dashboard-functions/13published-application.png) #### Install available application To install one of the available application from the Package Manager click on the "install new application" button. ![Applications](/admin-dashboard-functions/12applications.png) You will be redirected to install page where you can browse available applications or upload a new one. ![InstallApplication](/admin-dashboard-functions/14install-application.png) On "Browse" button click you will get a available application list where you can select application you want to install. ![BrowseApplications](/admin-dashboard-functions/14browse-applications.png) On selection the application Metadata will be displayed and you can press "install application" button to install the application. ![BrowseApplications](/admin-dashboard-functions/15install-application.png) On sucessfull installation you will get status popup. ![ApplicationInstalled](/admin-dashboard-functions/16application-installed.png) ### Context Preview context which you have already joined or invited. You can also create a new context. ![ContextList](/admin-dashboard-functions/18context-list.png) #### Start new context To start new context press the "Start new context" button on previous page. You can select application with which you want to create a new context or you can upload a new application. ![StartContextSelect](/admin-dashboard-functions/19start-context-select.png) If your application requires initialization parameters, you can select the 'Initialize application state' option during startup. > **NOTE**: To understand this lets take two examples. > > 1. Consider a JavaScript class MyClass. In some cases, you can create an > instance using `new MyClass()` if there are no initialization parameters > required. > 2. However, if your class requires specific parameters for initialization, you > need to pass them like this: `new MyClass({ application_name: name })`. > This is similar to your WASM application's requirement for initialization > parameters during its startup process. ![StartContextSelected](/admin-dashboard-functions/20start-context-selected.png) > **NOTE**: You must select a protocol for your context. Currently supported > protocols are: NEAR, Starknet, ICP, Stellar, Ethereum On button "Start" click the new context will be created and you will get status popup. ![Context installed](/admin-dashboard-functions/21context-installed.png) You can now see created context in the context list. ![ContextListInstalled](/admin-dashboard-functions/22context-list.png) ### Export Allows you to export you identity on current device and import it on new device. ![ContextList](/admin-dashboard-functions/23export-identity.png) --- ## Desktop App Calimero Node Manager Desktop app is a simple desktop application that allows you to manage and interact with Calimero nodes without needing any technical knowledge. ## Download and Install the App 1. Go to the [Calimero Node Manager download page](https://github.com/calimero-network/node-multiplatform-tauri/releases). 2. Select the latest version and download the app for your machine from the assets list: Download and run the installer. Follow the instructions to install the app. > **_NOTE:_** Supported OS > > - Linux: Works on modern distributions like Fedora, Red Hat, CentOS, Ubuntu, > and Debian (64-bit systems) > - Mac: Supports macOS for both Intel and Apple Silicon Supported ### Key Features - Initialize Node: Create and initialize a new node with specified parameters. - Update Node: Modify the configuration of an existing node. - Start Node: Start a specified node. - Stop Node: Stop a running node. - Get Node Log: Retrieve the log file for a specified node. - Delete Node: Remove a specified node from the application. - Open Dashboard: Open the admin dashboard for a specified node. If you'd like to run the app locally or contribute to its development, you can access the source code on our GitHub repository: [Node Multiplatform Tauri](https://github.com/calimero-network/node-multiplatform-tauri) Feel free to explore, fork the repository, and contribute by submitting pull requests or reporting issues. --- ## Admin API Reference The Calimero Admin API provides programmatic access to manage various aspects of a Calimero node. This API is essential for building administrative tools, monitoring systems, and integrating Calimero nodes into existing infrastructure. ## Base URL All admin API endpoints are served under the `/admin-api` path: ```bash http://your-node-address:port/admin-api ``` ## Authentication The admin API supports session-based authentication. When enabled, endpoints require valid authentication credentials. ## Response Format All API responses follow a consistent JSON format: ```json { "payload": { // Response data here } } ``` Error responses include an error message: ```json { "error": "Error description" } ``` ## Endpoints ### Health Check #### GET `/admin-api/health` Check if the admin API is running and healthy. **Response:** ```json { "payload": { "data": { "status": "alive" } } } ``` ### Authentication Status #### GET `/admin-api/is-authed` Check if the current request is authenticated. **Response:** ```json { "payload": { "data": { "is_authed": true } } } ``` ### SSL Certificate #### GET `/admin-api/certificate` Retrieve the SSL certificate for the node. **Response:** ```json { "payload": { "data": { "certificate": "-----BEGIN CERTIFICATE-----\n..." } } } ``` ## Application Management :::note The admin API endpoints for application management provide the same functionality as the `meroctl` CLI tool. Both support installing applications from local file paths and remote URLs. ::: ### Install Application #### POST `/admin-api/install-application` Install a new application on the node. This endpoint requires a **publicly accessible HTTP/HTTPS URL** for the WASM file. **Request Body:** **Example with descriptive placeholders:** ```json { "url": "https://example.com/application.wasm", "metadata": [], "hash": "optional-hash-value" } ``` **Example with realistic dummy values:** ```json { "url": "https://github.com/calimero/example-app/releases/v1.0.0/app.wasm", "metadata": [], "hash": "8BtADbnT4DyX6P7rYjKh2trR5zk3nk5L5zSp4G7rHArc" } ``` **Required Fields:** - `url`: **HTTP/HTTPS URL** to the WASM file (local file paths are not supported) - `metadata`: Array of bytes (can be empty `[]` for basic installation) **Optional Fields:** - `hash`: Optional hash verification for the WASM file **Response:** ```json { "data": { "applicationId": "HiHvjUJwQEAJKYr9yaTUPcNQi8BfBdnBWEVGqVFNHFF" } } ``` :::note **Important:** The `url` field must be a valid HTTP/HTTPS URL. Local file paths (`/path/to/file.wasm`) and file protocol URLs (`file:///path/to/file.wasm`) are not supported for security reasons. **Testing with Local Files:** To test with local WASM files, you need to serve them via a local HTTP server first: ```bash # Start a local HTTP server in the directory containing your WASM file cd /path/to/your/wasm/files python3 -m http.server 8000 # Then use the HTTP URL in your request curl -X POST http://localhost:2428/admin-api/install-application \ -H "Content-Type: application/json" \ -d '{ "url": "http://localhost:8000/your-app.wasm", "metadata": [] }' ``` ::: ### Install Development Application #### POST `/admin-api/install-dev-application` Install a development version of an application. This endpoint supports both local file paths and remote URLs. **Request Body:** **Example with descriptive placeholders:** ```json { "path": "/path/to/dev-application.wasm", "metadata": [] } ``` **Example with realistic dummy values:** ```json { "path": "/Users/developer/projects/my-app/target/wasm32-unknown-unknown/release/app.wasm", "metadata": [] } ``` **Alternative Request Body (URL-based):** **Example with descriptive placeholders:** ```json { "path": "https://example.com/dev-application.wasm", "metadata": [] } ``` **Example with realistic dummy values:** ```json { "path": "https://github.com/calimero/example-app/releases/nightly/app.wasm", "metadata": [] } ``` :::note Similar to the main install endpoint, the `path` field accepts both local file paths and remote URLs for development applications. ::: ### Working Examples #### Example 1: Install Application from Local HTTP Server ```bash # Step 1: Start a local HTTP server cd /path/to/your/wasm/files python3 -m http.server 8000 # Step 2: Install the application curl -X POST http://localhost:2428/admin-api/install-application \ -H "Content-Type: application/json" \ -d '{ "url": "http://localhost:8000/your-app.wasm", "metadata": [] }' ``` **Expected Response:** ```json { "data": { "applicationId": "HiHvjUJwQEAJKYr9yaTUPcNQi8BfBdnBWEVGqVFNHFF" } } ``` #### Example 2: Install Application from Remote URL ```bash curl -X POST http://localhost:2428/admin-api/install-application \ -H "Content-Type: application/json" \ -d '{ "url": "https://example.com/application.wasm", "metadata": [] }' ``` #### Example 3: Install with Metadata (Advanced) ```bash # If you need to include metadata, encode it as bytes curl -X POST http://localhost:2428/admin-api/install-application \ -H "Content-Type: application/json" \ -d '{ "url": "http://localhost:8000/your-app.wasm", "metadata": [99, 117, 114, 98], "hash": "optional-hash-here" }' ``` ### Understanding Metadata The `metadata` field allows you to attach additional information to your application installation. While optional, it's useful for tracking application details and displaying information in the admin dashboard. #### Metadata Structure The admin dashboard uses this metadata structure: ```typescript interface AppMetadata { applicationUrl: string; // URL where the WASM file is hosted applicationName: string; // Human-readable name of the application applicationOwner: string; // Name of the developer/owner applicationVersion: string; // Version number (e.g., "1.0.0") description: string; // Brief description of the application contractAppId: string; // Optional contract identifier repositoryUrl: string; // Link to source code repository } ``` #### How Metadata Gets Encoded The metadata must be provided as an array of bytes (0-255 values). The admin dashboard automatically handles this encoding: ```typescript function createAppMetadata(application: AppMetadata): Uint8Array { return new TextEncoder().encode(JSON.stringify(application)); } ``` #### Metadata Examples **Example 1: Empty Metadata (Simplest)** ```json { "url": "http://localhost:8000/app.wasm", "metadata": [] } ``` **Example 2: Basic Metadata (Name and Version)** ```json { "url": "http://localhost:8000/app.wasm", "metadata": [ 123, 34, 110, 97, 109, 101, 34, 58, 34, 67, 117, 114, 98, 32, 65, 112, 112, 34, 44, 34, 118, 101, 114, 115, 105, 111, 110, 34, 58, 34, 49, 46, 48, 46, 48, 34, 125 ] } ``` This represents: `{"name":"Curb App","version":"1.0.0"}` **Example 3: Full Metadata Structure** ```json { "url": "http://localhost:8000/app.wasm", "metadata": [ 123, 34, 97, 112, 112, 108, 105, 99, 97, 116, 105, 111, 110, 78, 97, 109, 101, 34, 58, 34, 67, 117, 114, 98, 32, 67, 104, 97, 116, 34, 44, 34, 97, 112, 112, 108, 105, 99, 97, 116, 105, 111, 110, 79, 119, 110, 101, 114, 34, 58, 34, 65, 110, 116, 111, 110, 34, 44, 34, 97, 112, 112, 108, 105, 99, 97, 116, 105, 111, 110, 86, 101, 114, 115, 105, 111, 110, 34, 58, 34, 49, 46, 48, 46, 48, 34, 44, 34, 100, 101, 115, 99, 114, 105, 112, 116, 105, 111, 110, 34, 58, 34, 76, 111, 99, 97, 108, 32, 116, 101, 115, 116, 32, 105, 110, 115, 116, 97, 108, 108, 97, 116, 105, 111, 110, 34, 125 ] } ``` This represents: `{"applicationName":"Curb Chat","applicationOwner":"Anton","applicationVersion":"1.0.0","description":"Local test installation"}` #### How to Generate Metadata Bytes **Option 1: Use JavaScript/Node.js** ```javascript const metadata = { applicationName: 'My App', applicationVersion: '1.0.0', }; const bytes = Array.from(new TextEncoder().encode(JSON.stringify(metadata))); console.log(bytes); // [123, 34, 97, 112, 112, 108, 105, 99, 97, 116, 105, 111, 110, 78, 97, 109, 101, 34, 58, 34, 77, 121, 32, 65, 112, 112, 34, 44, 34, 97, 112, 112, 108, 105, 99, 97, 116, 105, 111, 110, 86, 101, 114, 115, 105, 111, 110, 34, 58, 34, 49, 46, 48, 46, 48, 34, 125] ``` **Option 2: Use Online Tools** - Convert your JSON to bytes using online text-to-bytes converters - Or use the admin dashboard which handles encoding automatically #### When to Use Metadata - **Empty `[]`**: For quick testing and basic installations - **Basic metadata**: When you want to track app name and version - **Full metadata**: For production deployments where you need complete app information #### Metadata Best Practices 1. **Keep it simple**: Start with empty metadata for testing 2. **Use consistent naming**: Follow the same structure across applications 3. **Include essential info**: At minimum, include `applicationName` and `applicationVersion` 4. **Avoid sensitive data**: Don't include API keys, passwords, or private information ### List Applications #### GET `/admin-api/applications` Get a list of all installed applications. ### Troubleshooting Common Issues #### Issue: "Failed to deserialize the JSON body into the target type: missing field `url`" **Cause:** Using `path` instead of `url` field **Solution:** Use `url` field for the install endpoint: ```json { "url": "http://example.com/app.wasm", "metadata": [] } ``` #### Issue: "Failed to deserialize the JSON body into the target type: metadata: invalid type: map, expected a sequence" **Cause:** Metadata should be an array, not an object **Solution:** Use array format: ```json { "url": "http://example.com/app.wasm", "metadata": [] } ``` #### Issue: "Failed to deserialize the JSON body into the target type: metadata[0]: invalid type: string, expected u8" **Cause:** Metadata array should contain numbers (0-255), not strings **Solution:** Use empty array or encode strings as byte values: ```json { "url": "http://example.com/app.wasm", "metadata": [] } ``` #### Issue: "HTTP 404 Not Found" on `/admin-api/install-dev-application` **Cause:** The dev endpoint may not be available on your node version **Solution:** Use the regular install endpoint with an HTTP URL instead #### Issue: "relative URL without a base" or "file:// protocol not supported" **Cause:** Local file paths and file:// URLs are not supported **Solution:** Serve your WASM file via HTTP server: ```bash # Start local HTTP server cd /path/to/wasm/files python3 -m http.server 8000 # Use HTTP URL curl -X POST http://localhost:2428/admin-api/install-application \ -H "Content-Type: application/json" \ -d '{ "url": "http://localhost:8000/app.wasm", "metadata": [] }' ``` #### Issue: "error sending request for url" or "builder error for url" **Cause:** The URL is not accessible or the web server is not running **Solution:** 1. Verify the web server is running 2. Test URL accessibility: `curl http://localhost:8000/app.wasm` 3. Check firewall/network settings #### Issue: App installed but not visible in admin dashboard **Cause:** Dashboard may be pointing to different node or have connection issues **Solution:** 1. Verify app exists via CLI: `meroctl --node node1 app list` 2. Check dashboard node configuration 3. Verify dashboard can access the admin API **Response:** ```json { "payload": { "data": [ { "id": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf", "name": "My Application", "version": "1.0.0", "installed_at": "2024-01-01T00:00:00Z" } ] } } ``` ### Get Application #### GET `/admin-api/applications/:application_id` Get details of a specific application. **Response:** ```json { "payload": { "data": { "id": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf", "name": "My Application", "version": "1.0.0", "installed_at": "2024-01-01T00:00:00Z", "contexts_count": 5 } } } ``` ### Uninstall Application #### DELETE `/admin-api/applications/:application_id` Remove an application from the node. **Response:** ```json { "payload": { "data": { "success": true, "message": "Application uninstalled successfully" } } } ``` ## Context Management ### Create Context #### POST `/admin-api/contexts` Create a new context. **Request Body:** **Example with descriptive placeholders:** ```json { "application_id": "application_identity_hash", "name": "context_name", "protocol": "near" } ``` **Example with realistic dummy values:** ```json { "application_id": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf", "name": "My Context", "protocol": "near" } ``` **Response:** ```json { "payload": { "data": { "context_id": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf", "name": "My Context", "application_id": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf", "created_at": "2024-01-01T00:00:00Z" } } } ``` ### List Contexts #### GET `/admin-api/contexts` Get a list of all contexts. **Response:** ```json { "payload": { "data": [ { "id": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf", "name": "My Context", "application_id": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf", "created_at": "2024-01-01T00:00:00Z", "members_count": 3 } ] } } ``` ### Get Context #### GET `/admin-api/contexts/:context_id` Get details of a specific context. **Response:** ```json { "payload": { "data": { "id": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf", "name": "My Context", "application_id": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf", "created_at": "2024-01-01T00:00:00Z", "members": [ { "identity": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf", "alias": "alice", "capabilities": ["read", "write"] } ] } } } ``` ### Delete Context #### DELETE `/admin-api/contexts/:context_id` Delete a context and all its data. **Response:** ```json { "payload": { "data": { "success": true, "message": "Context deleted successfully" } } } ``` ### Invite to Context #### POST `/admin-api/contexts/invite` Invite a peer to join a context. **Request Body:** **Example with descriptive placeholders:** ```json { "context_id": "context_identity_hash", "invitee_identity": "invitee_public_key", "inviter_identity": "inviter_identity_hash" } ``` **Example with realistic dummy values:** ```json { "context_id": "A4NyeJydZYKeSDiDV4CHYADVGywVekWEubi47Syf6FQy", "invitee_identity": "57kiq6jgag3iL7vp4D5iwFLnecB4sig9sP6ELJLv87tL", "inviter_identity": "3aipBNQaNzkWeMrSTzWwAQRdfWvwvFXqSKCZ1CBT8d4k" } ``` **Note:** This endpoint corresponds to the CLI command `meroctl context invite {invitee_identity} --as {inviter_identity} -c {context_id}`. The `invitee_identity` is the public key of the person being invited, `inviter_identity` is the identity of the person doing the inviting, and `context_id` specifies which context to invite them to. ### Join Context #### POST `/admin-api/contexts/join` Accept an invitation to join a context. **Request Body:** **Example with descriptive placeholders:** ```json { "invitation_payload": "base64_encoded_invitation_payload" } ``` **Example with realistic dummy values:** ```json { "invitation_payload": "maWT6kDxpR1EMXb2YSXzdHkegU76QVcf3yXUAyDUEQ27nuHJkpYQbyota6BpZbaouxDn1biAywcj2GnUSRurDn2mvjMwuo6WrjvvgvvAUGDKmAtzN7oqhtWVt5cGEucgrZaei7S255e1KXikmFHWN8UPKMBkuWVFq" } ``` **Note:** This endpoint corresponds to the CLI command `meroctl context join {invitation_payload}`. The `invitation_payload` is the result of a successful context invite operation and contains all the necessary information to join the context, including the context ID and invitation details. ### Update Context Application #### POST `/admin-api/contexts/:context_id/application` Update the application associated with a context. **Request Body:** ```json { "application_id": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf" } ``` ### Get Context Storage #### GET `/admin-api/contexts/:context_id/storage` Get the raw storage data for a context. **Response:** ```json { "payload": { "data": { "entries": [ { "key": "user:alice", "value": "encrypted_data_here" } ], "total_size": 1024 } } } ``` ### Get Context Identities #### GET `/admin-api/contexts/:context_id/identities` Get all identities in a context. **Response:** ```json { "payload": { "data": [ { "identity": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf", "alias": "alice", "capabilities": ["read", "write"], "joined_at": "2024-01-01T00:00:00Z" } ] } } ``` ### Get Owned Context Identities #### GET `/admin-api/contexts/:context_id/identities-owned` Get only the identities owned by the current user in a context. **Response:** ```json { "payload": { "data": [ { "identity": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf", "alias": "alice", "capabilities": ["read", "write"], "joined_at": "2024-01-01T00:00:00Z" } ] } } ``` ### Grant Capabilities #### POST `/admin-api/contexts/:context_id/capabilities/grant` Grant capabilities to an identity in a context. **Request Body:** ```json { "identity": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf", "capabilities": ["read", "write", "admin"] } ``` ### Revoke Capabilities #### POST `/admin-api/contexts/:context_id/capabilities/revoke` Revoke capabilities from an identity in a context. **Request Body:** ```json { "identity": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf", "capabilities": ["admin"] } ``` ## Identity Management ### Generate Context Identity #### POST `/admin-api/identity/context` Generate a new identity for use in a context. **Request Body:** ```json { "context_id": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf" } ``` **Response:** ```json { "payload": { "data": { "identity": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf", "private_key": "base64_encoded_private_key" } } } ``` ## Proposal Management ### Get Active Proposals Count #### GET `/admin-api/contexts/:context_id/proposals/count` Get the number of active proposals in a context. **Response:** ```json { "payload": { "data": { "count": 5 } } } ``` ### Get Proposals #### POST `/admin-api/contexts/:context_id/proposals` Get a list of proposals in a context. **Request Body:** ```json { "limit": 10, "offset": 0, "status": "active" } ``` **Response:** ```json { "payload": { "data": [ { "id": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf", "title": "Update Protocol", "status": "active", "created_by": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf", "created_at": "2024-01-01T00:00:00Z" } ] } } ``` ### Get Proposal #### GET `/admin-api/contexts/:context_id/proposals/:proposal_id` Get details of a specific proposal. **Response:** ```json { "payload": { "data": { "id": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf", "title": "Update Protocol", "description": "Update to latest protocol version", "status": "active", "created_by": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf", "created_at": "2024-01-01T00:00:00Z", "approvals_count": 3, "required_approvals": 5 } } } ``` ### Get Proposal Approvals Count #### GET `/admin-api/contexts/:context_id/proposals/:proposal_id/approvals/count` Get the number of approvals for a proposal. **Response:** ```json { "payload": { "data": { "count": 3, "required": 5 } } } ``` ### Get Proposal Approvers #### GET `/admin-api/contexts/:context_id/proposals/:proposal_id/approvals/users` Get the list of users who approved a proposal. **Response:** ```json { "payload": { "data": [ { "identity": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf", "alias": "alice", "approved_at": "2024-01-01T00:00:00Z" } ] } } ``` ### Get Context Value #### POST `/admin-api/contexts/:context_id/proposals/get-context-value` Get a specific value from the context storage. **Request Body:** ```json { "key": "user:alice" } ``` ### Get Context Storage Entries #### POST `/admin-api/contexts/:context_id/proposals/context-storage-entries` Get multiple storage entries from a context. **Request Body:** ```json { "keys": ["user:alice", "user:bob"], "limit": 100 } ``` ### Get Proxy Contract #### GET `/admin-api/contexts/:context_id/proxy-contract` Get the proxy contract information for a context. **Response:** ```json { "payload": { "data": { "contract_address": "0x1234...", "network": "ethereum", "status": "active" } } } ``` ## Context Synchronization ### Sync Context #### POST `/admin-api/contexts/sync` Synchronize a context with the network. **Request Body:** ```json { "context_id": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf" } ``` #### POST `/admin-api/contexts/sync/:context_id` Synchronize a specific context. ## Network Information ### Get Peers Count #### GET `/admin-api/peers` Get the number of connected peers. **Response:** ```json { "payload": { "data": { "count": 15, "active": 12 } } } ``` ## Blob Management ### Upload Blob #### PUT `/admin-api/blobs` Upload a new blob to the node. **Request Body:** Binary data **Headers:** ``` Content-Type: application/octet-stream ``` **Response:** ```json { "payload": { "data": { "blob_id": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf", "size": 1024, "uploaded_at": "2024-01-01T00:00:00Z" } } } ``` ### List Blobs #### GET `/admin-api/blobs` Get a list of all blobs. **Response:** ```json { "payload": { "data": [ { "id": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf", "size": 1024, "uploaded_at": "2024-01-01T00:00:00Z" } ] } } ``` ### Download Blob #### GET `/admin-api/blobs/:blob_id` Download a blob by ID. **Response:** Binary data **Headers:** ``` Content-Type: application/octet-stream Content-Disposition: attachment; filename="blob.bin" ``` ### Get Blob Info #### HEAD `/admin-api/blobs/:blob_id` Get metadata about a blob without downloading it. **Response Headers:** ``` Content-Length: 1024 Content-Type: application/octet-stream X-Uploaded-At: 2024-01-01T00:00:00Z ``` ### Delete Blob #### DELETE `/admin-api/blobs/:blob_id` Delete a blob from the node. **Response:** ```json { "payload": { "data": { "success": true, "message": "Blob deleted successfully" } } } ``` ## Alias Management ### Create Alias #### POST `/admin-api/alias/create/context` Create an alias for a context. **Request Body:** ```json { "name": "my-context", "target": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf" } ``` #### POST `/admin-api/alias/create/application` Create an alias for an application. **Request Body:** ```json { "name": "my-app", "target": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf" } ``` #### POST `/admin-api/alias/create/identity/:context` Create an alias for an identity in a context. **Request Body:** ```json { "name": "alice", "target": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf" } ``` ### Lookup Alias #### POST `/admin-api/alias/lookup/context/:name` Look up a context by alias. **Response:** ```json { "payload": { "data": { "target": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf", "created_at": "2024-01-01T00:00:00Z" } } } ``` #### POST `/admin-api/alias/lookup/application/:name` Look up an application by alias. #### POST `/admin-api/alias/lookup/identity/:context/:name` Look up an identity by alias in a specific context. ### Delete Alias #### POST `/admin-api/alias/delete/context/:name` Delete a context alias. #### POST `/admin-api/alias/delete/application/:name` Delete an application alias. #### POST `/admin-api/alias/delete/identity/:context/:name` Delete an identity alias. ### List Aliases #### GET `/admin-api/alias/list/context` List all context aliases. #### GET `/admin-api/alias/list/application` List all application aliases. #### GET `/admin-api/alias/list/identity/:context` List all identity aliases in a context. ## Error Handling The admin API returns appropriate HTTP status codes and error messages: - **400 Bad Request**: Invalid request parameters - **401 Unauthorized**: Authentication required - **403 Forbidden**: Insufficient permissions - **404 Not Found**: Resource not found - **500 Internal Server Error**: Server-side error ## Rate Limiting The admin API may implement rate limiting to prevent abuse. Check response headers for rate limit information. ## WebSocket Support Some endpoints support WebSocket connections for real-time updates. Check the specific endpoint documentation for WebSocket support details. ## Examples ### Complete Workflow: Create Context and Invite Users ```bash # 1. Create a context curl -X POST http://localhost:2428/admin-api/contexts \ -H "Content-Type: application/json" \ -d '{ "application_id": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf", "name": "Team Collaboration", "protocol": "near" }' # 2. Generate an identity for the context curl -X POST http://localhost:2428/admin-api/identity/context \ -H "Content-Type: application/json" \ -d '{ "context_id": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf" }' # 3. Invite a peer to the context curl -X POST http://localhost:2428/admin-api/contexts/invite \ -H "Content-Type: application/json" \ -d '{ "context_id": "A4NyeJydZYKeSDiDV4CHYADVGywVekWEubi47Syf6FQy", "invitee_identity": "57kiq6jgag3iL7vp4D5iwFLnecB4sig9sP6ELJLv87tL", "inviter_identity": "3aipBNQaNzkWeMrSTzWwAQRdfWvwvFXqSKCZ1CBT8d4k" }' # 4. Accept the invitation (using the payload from step 3) curl -X POST http://localhost:2428/admin-api/contexts/join \ -H "Content-Type: application/json" \ -d '{ "invitation_payload": "maWT6kDxpR1EMXb2YSXzdHkegU76QVcf3yXUAyDUEQ27nuHJkpYQbyota6BpZbaouxDn1biAywcj2GnUSRurDn2mvjMwuo6WrjvvgvvAUGDKmAtzN7oqhtWVt5cGEucgrZaei7S255e1KXikmFHWN8UPKMBkuWVFq" }' ``` ### Monitoring Node Health ```bash # Check API health curl http://localhost:2428/admin-api/health # Get peer count curl http://localhost:2428/admin-api/peers # List all contexts curl http://localhost:2428/admin-api/contexts ``` ### Understanding Metadata The `metadata` field allows you to attach additional information to your application installation. While optional, it's useful for tracking application details and displaying information in the admin dashboard. #### Metadata Structure The admin dashboard uses this metadata structure: ```typescript interface AppMetadata { applicationUrl: string; // URL where the WASM file is hosted applicationName: string; // Human-readable name of the application applicationOwner: string; // Name of the developer/owner applicationVersion: string; // Version number (e.g., "1.0.0") description: string; // Brief description of the application contractAppId: string; // Optional contract identifier repositoryUrl: string; // Link to source code repository } ``` #### How Metadata Gets Encoded The metadata must be provided as an array of bytes (0-255 values). The admin dashboard automatically handles this encoding: ```typescript function createAppMetadata(application: AppMetadata): Uint8Array { return new TextEncoder().encode(JSON.stringify(application)); } ``` #### Metadata Examples **Example 1: Empty Metadata (Simplest)** ```json { "url": "http://localhost:8000/app.wasm", "metadata": [] } ``` **Example 2: Basic Metadata (Name and Version)** ```json { "url": "http://localhost:8000/app.wasm", "metadata": [ 123, 34, 110, 97, 109, 101, 34, 58, 34, 67, 117, 114, 98, 32, 65, 112, 112, 34, 44, 34, 118, 101, 114, 115, 105, 111, 110, 34, 58, 34, 49, 46, 48, 46, 48, 34, 125 ] } ``` This represents: `{"name":"Curb App","version":"1.0.0"}` **Example 3: Full Metadata Structure** ```json { "url": "http://localhost:8000/app.wasm", "metadata": [ 123, 34, 97, 112, 112, 108, 105, 99, 97, 116, 105, 111, 110, 78, 97, 109, 101, 34, 58, 34, 67, 117, 114, 98, 32, 67, 104, 97, 116, 34, 44, 34, 97, 112, 112, 108, 105, 99, 97, 116, 105, 111, 110, 79, 119, 110, 101, 114, 34, 58, 34, 65, 110, 116, 111, 110, 34, 44, 34, 97, 112, 112, 108, 105, 99, 97, 116, 105, 111, 110, 86, 101, 114, 115, 105, 111, 110, 34, 58, 34, 49, 46, 48, 46, 48, 34, 44, 34, 100, 101, 115, 99, 114, 105, 112, 116, 105, 111, 110, 34, 58, 34, 76, 111, 99, 97, 108, 32, 116, 101, 115, 116, 32, 105, 110, 115, 116, 97, 108, 108, 97, 116, 105, 111, 110, 34, 125 ] } ``` This represents: `{"applicationName":"Curb Chat","applicationOwner":"Anton","applicationVersion":"1.0.0","description":"Local test installation"}` #### How to Generate Metadata Bytes **Option 1: Use JavaScript/Node.js** ```javascript const metadata = { applicationName: 'My App', applicationVersion: '1.0.0', }; const bytes = Array.from(new TextEncoder().encode(JSON.stringify(metadata))); console.log(bytes); // [123, 34, 97, 112, 112, 108, 105, 99, 97, 116, 105, 111, 110, 78, 97, 109, 101, 34, 58, 34, 77, 121, 32, 65, 112, 112, 34, 44, 34, 97, 112, 112, 108, 105, 99, 97, 116, 105, 111, 110, 86, 101, 114, 115, 105, 111, 110, 34, 58, 34, 49, 46, 48, 46, 48, 34, 125] ``` **Option 2: Use Online Tools** - Convert your JSON to bytes using online text-to-bytes converters - Or use the admin dashboard which handles encoding automatically #### When to Use Metadata - **Empty `[]`**: For quick testing and basic installations - **Basic metadata**: When you want to track app name and version - **Full metadata**: For production deployments where you need complete app information #### Metadata Best Practices 1. **Keep it simple**: Start with empty metadata for testing 2. **Use consistent naming**: Follow the same structure across applications 3. **Include essential info**: At minimum, include `applicationName` and `applicationVersion` 4. **Avoid sensitive data**: Don't include API keys, passwords, or private information ### Troubleshooting Common Issues ## Next Steps - **Build Admin Tools**: Use these endpoints to create custom administrative interfaces - **Monitor Nodes**: Implement monitoring and alerting based on API responses - **Automate Operations**: Create scripts for common administrative tasks - **Integration**: Connect Calimero nodes to existing infrastructure management systems --- ## Advanced Configuration # Advanced Configuration This guide covers advanced features and configuration options for Merobox. ## Overview Merobox provides several advanced configuration options organized into specialized areas: - **[Environment Variables](./environment-variables)** - Configure Merobox behavior and Docker settings - **[Docker Image Management](./docker-image-management)** - Manage Docker images and containers - **[Network Configuration](./network-configuration)** - Basic network setup and port management - **[Authentication Service Integration](./auth-service-integration)** - Auth service configuration - **[Workflow Advanced Features](./workflow-advanced-features)** - Conditional execution and parallel steps - **[Testing Framework Integration](./testing-framework-integration)** - Basic testing configuration - **[Resource Management](./resource-management)** - Basic resource limits and monitoring - **[Security Configuration](./security-configuration)** - Basic security settings ## Quick Reference ### Common Configuration Patterns ```yaml # Basic workflow configuration nodes: count: 3 image: ghcr.io/calimero-network/merod:latest chain_id: testnet-1 prefix: calimero-node ``` ### CLI Configuration ```bash # Core configuration merobox run --count 3 --image ghcr.io/calimero-network/merod:edge # Network settings merobox run --count 3 --base-port 3000 --base-rpc-port 4000 ``` ## Configuration Best Practices ### 1. Environment-Specific Configuration Use different configurations for different environments: ```yaml # Development development: nodes: count: 2 image: ghcr.io/calimero-network/merod:edge # Production production: nodes: count: 5 image: ghcr.io/calimero-network/merod:latest ``` ### 2. Resource Planning Plan resources based on your workload: - **CPU**: 0.5-2.0 cores per node depending on workload - **Memory**: 1-4GB per node for typical applications - **Storage**: 10-100GB depending on data requirements - **Network**: Consider bandwidth and latency requirements ### 3. Security Considerations Implement security best practices: - Use non-root users - Enable read-only filesystems where possible - Drop unnecessary capabilities - Implement network segmentation - Use secrets management - Enable audit logging ### 4. Monitoring and Observability Set up basic monitoring: ```yaml # Basic monitoring monitoring: enabled: true metrics: - cpu_usage - memory_usage - disk_usage - network_io ``` ## Troubleshooting Advanced Issues ### Debug Mode Enable comprehensive debugging: ```bash # Enable debug logging export LOG_LEVEL=DEBUG # Run with verbose output merobox bootstrap run workflow.yml --verbose # Enable Docker debug export DOCKER_BUILDKIT=0 export DOCKER_CLI_EXPERIMENTAL=enabled ``` ### Performance Profiling Profile Merobox performance: ```bash # Monitor resource usage docker stats $(docker ps -q --filter "name=calimero-") # Analyze logs merobox logs calimero-node-1 | grep -E "(ERROR|WARN|PERF)" ``` ### Network Diagnostics Diagnose network issues: ```bash # Check Docker networks docker network ls docker network inspect calimero-web # Test connectivity docker exec calimero-node-1 ping calimero-node-2 # Check port binding netstat -tulpn | grep -E "(2428|2528)" ``` ## Best Practices for Advanced Usage ### Production Deployment 1. **Resource Planning**: Allocate appropriate resources for your workload 2. **Security**: Implement proper security measures and access controls 3. **Monitoring**: Set up comprehensive monitoring and alerting 4. **Backup**: Implement regular backup and recovery procedures 5. **Documentation**: Maintain detailed documentation of your configuration ### Development Workflow 1. **Environment Isolation**: Use separate environments for different stages 2. **Version Control**: Track configuration changes in version control 3. **Testing**: Implement comprehensive testing at all levels 4. **CI/CD Integration**: Integrate Merobox into your CI/CD pipeline 5. **Code Review**: Review configuration changes before deployment ### Performance Optimization 1. **Resource Tuning**: Optimize resource allocation based on usage patterns 2. **Caching**: Implement appropriate caching strategies 3. **Parallelization**: Use parallel execution where possible 4. **Monitoring**: Continuously monitor and optimize performance 5. **Scaling**: Implement auto-scaling based on demand ## Next Steps Now that you understand advanced configuration: - [Python Customization](./python-customization) - Basic Python integration - [Troubleshooting](./troubleshooting) - Common issues and solutions --- ## Advanced Examples # Advanced Examples These examples demonstrate complex workflows and advanced features for power users who need sophisticated Merobox capabilities. :::info Available Step Types For a complete reference of all available step types and their configuration options, see the [Workflow System documentation](./workflows#step-types). The following step types are supported: - **`install_application`** - Install WASM applications on nodes - **`create_context`** - Create blockchain contexts for applications - **`create_identity`** - Generate cryptographic identities - **`invite_identity`** - Invite identities to join contexts - **`join_context`** - Join contexts using invitations - **`call`** - Execute smart contract functions - **`wait`** - Add delays between steps - **`repeat`** - Execute steps multiple times - **`script`** - Execute custom scripts - **`assert`** - Validate conditions and outputs - **`json_assert`** - Validate JSON data structures - **`parallel`** - Execute multiple steps in parallel Each step type has specific configuration parameters and output options. Refer to the [Workflow System documentation](./workflows#step-types) for detailed examples and configuration options. ::: ## Example 1: Custom Script Integration Use custom scripts for complex setup and validation. ```yaml description: Custom script integration example name: Script Integration nodes: chain_id: testnet-1 count: 1 image: ghcr.io/calimero-network/merod:edge prefix: script-node steps: - name: Pre-Setup Script type: script script: | echo "Setting up test environment..." mkdir -p /tmp/test-data echo "test_data" > /tmp/test-data/sample.txt echo "Pre-setup complete" - name: Install Application type: install_application node: script-node-1 path: ./my-app.wasm dev: true outputs: app_id: applicationId - name: Post-Installation Script type: script script: | echo "Application installed: {{app_id}}" echo "Verifying installation..." # Add custom verification logic here echo "Installation verified" - name: Custom Validation type: script script: | echo "Running custom validation..." # Add custom validation logic here echo "Validation complete" stop_all_nodes: true ``` ## Example 2: Error Handling and Recovery Implement robust error handling in your workflows. ```yaml description: Error handling example name: Error Handling nodes: chain_id: testnet-1 count: 1 image: ghcr.io/calimero-network/merod:edge prefix: error-node steps: - name: Install Application type: install_application node: error-node-1 path: ./my-app.wasm dev: true outputs: app_id: applicationId - name: Create Context type: create_context node: error-node-1 application_id: '{{app_id}}' outputs: context_id: contextId member_key: memberPublicKey - name: Test Error Handling type: call node: error-node-1 context_id: '{{context_id}}' executor_public_key: '{{member_key}}' method: test_error_handling args: should_fail: true outputs: error_result: result - name: Validate Error Response type: assert statements: - "contains({{error_result}}, 'error')" - name: Recovery Test type: call node: error-node-1 context_id: '{{context_id}}' executor_public_key: '{{member_key}}' method: test_recovery args: should_succeed: true outputs: recovery_result: result - name: Validate Recovery type: assert statements: - "contains({{recovery_result}}, 'success')" stop_all_nodes: true ``` ## Example 3: Conditional Workflow Execution Execute different workflow paths based on conditions. ```yaml description: Conditional workflow execution name: Conditional Workflow nodes: chain_id: testnet-1 count: 2 image: ghcr.io/calimero-network/merod:edge prefix: conditional-node steps: - name: Check Environment type: script script: | if [ "$ENVIRONMENT" = "production" ]; then echo "production" > /tmp/env_type echo "true" > /tmp/use_ssl elif [ "$ENVIRONMENT" = "staging" ]; then echo "staging" > /tmp/env_type echo "true" > /tmp/use_ssl else echo "development" > /tmp/env_type echo "false" > /tmp/use_ssl fi outputs: env_type: output use_ssl: output - name: Production Setup type: script condition: "{{env_type}} == 'production'" script: | echo "Setting up production environment..." # Production-specific setup echo "Production setup complete" - name: Staging Setup type: script condition: "{{env_type}} == 'staging'" script: | echo "Setting up staging environment..." # Staging-specific setup echo "Staging setup complete" - name: Development Setup type: script condition: "{{env_type}} == 'development'" script: | echo "Setting up development environment..." # Development-specific setup echo "Development setup complete" - name: SSL Configuration type: script condition: "{{use_ssl}} == 'true'" script: | echo "Configuring SSL for {{env_type}} environment..." # SSL configuration logic echo "SSL configuration complete" stop_all_nodes: true ``` ## Example 4: Parallel Processing Execute multiple operations in parallel for improved performance. ```yaml description: Parallel processing example name: Parallel Processing nodes: chain_id: testnet-1 count: 5 image: ghcr.io/calimero-network/merod:edge prefix: parallel-node steps: - name: Install Applications in Parallel type: parallel max_concurrent: 3 steps: - name: Install App 1 type: install_application node: parallel-node-1 path: ./app1.wasm dev: true outputs: app_id_1: applicationId - name: Install App 2 type: install_application node: parallel-node-2 path: ./app2.wasm dev: true outputs: app_id_2: applicationId - name: Install App 3 type: install_application node: parallel-node-3 path: ./app3.wasm dev: true outputs: app_id_3: applicationId - name: Create Contexts in Parallel type: parallel steps: - name: Create Context 1 type: create_context node: parallel-node-1 application_id: '{{app_id_1}}' outputs: context_id_1: contextId member_key_1: memberPublicKey - name: Create Context 2 type: create_context node: parallel-node-2 application_id: '{{app_id_2}}' outputs: context_id_2: contextId member_key_2: memberPublicKey - name: Create Context 3 type: create_context node: parallel-node-3 application_id: '{{app_id_3}}' outputs: context_id_3: contextId member_key_3: memberPublicKey - name: Test All Applications type: parallel steps: - name: Test App 1 type: call node: parallel-node-1 context_id: '{{context_id_1}}' executor_public_key: '{{member_key_1}}' method: test_functionality args: test_data: 'app1_test' - name: Test App 2 type: call node: parallel-node-2 context_id: '{{context_id_2}}' executor_public_key: '{{member_key_2}}' method: test_functionality args: test_data: 'app2_test' - name: Test App 3 type: call node: parallel-node-3 context_id: '{{context_id_3}}' executor_public_key: '{{member_key_3}}' method: test_functionality args: test_data: 'app3_test' stop_all_nodes: true ``` ## Example 5: Dynamic Workflow Generation Generate workflows dynamically based on configuration. ```yaml description: Dynamic workflow generation name: Dynamic Workflow nodes: chain_id: testnet-1 count: 3 image: ghcr.io/calimero-network/merod:edge prefix: dynamic-node steps: - name: Generate Workflow Configuration type: script script: | # Generate dynamic configuration echo "Generating workflow configuration..." # Create configuration file cat > /tmp/workflow_config.json << EOF { "applications": [ {"name": "app1", "path": "./app1.wasm"}, {"name": "app2", "path": "./app2.wasm"}, {"name": "app3", "path": "./app3.wasm"} ], "test_cases": [ {"name": "test1", "method": "test_method1"}, {"name": "test2", "method": "test_method2"}, {"name": "test3", "method": "test_method3"} ] } EOF echo "Configuration generated" outputs: config_file: '/tmp/workflow_config.json' - name: Install Applications Dynamically type: repeat count: 3 outputs: iteration: iteration steps: - name: Install Application {{iteration}} type: install_application node: dynamic-node-{{iteration}} path: ./app{{iteration}}.wasm dev: true outputs: app_id: applicationId - name: Create Contexts Dynamically type: repeat count: 3 outputs: iteration: iteration steps: - name: Create Context {{iteration}} type: create_context node: dynamic-node-{{iteration}} application_id: '{{app_id}}' outputs: context_id: contextId member_key: memberPublicKey - name: Run Tests Dynamically type: repeat count: 3 outputs: iteration: iteration steps: - name: Run Test {{iteration}} type: call node: dynamic-node-{{iteration}} context_id: '{{context_id}}' executor_public_key: '{{member_key}}' method: test_method{{iteration}} args: test_data: 'test_{{iteration}}' stop_all_nodes: true ``` ## Example 6: Workflow Composition Compose complex workflows from smaller, reusable components. ```yaml description: Workflow composition example name: Workflow Composition nodes: chain_id: testnet-1 count: 3 image: ghcr.io/calimero-network/merod:edge prefix: compose-node steps: # Common setup steps - name: Common Setup type: script script: | echo "Common setup logic" # Common setup steps # Application-specific steps - name: Install Application type: install_application node: compose-node-1 path: ./my-app.wasm dev: true outputs: app_id: applicationId - name: Create Context type: create_context node: compose-node-1 application_id: '{{app_id}}' outputs: context_id: contextId member_key: memberPublicKey # Testing steps - name: Run Tests type: parallel steps: - name: Test 1 type: call node: compose-node-1 context_id: '{{context_id}}' executor_public_key: '{{member_key}}' method: test1 - name: Test 2 type: call node: compose-node-1 context_id: '{{context_id}}' executor_public_key: '{{member_key}}' method: test2 # Common cleanup steps - name: Common Cleanup type: script script: | echo "Common cleanup logic" # Common cleanup steps stop_all_nodes: true ``` ## Best Practices for Advanced Examples ### 1. Error Handling ```yaml # Always include comprehensive error handling steps: - name: Risky Operation type: call node: calimero-node-1 method: risky_method retry: attempts: 3 delay: 5 backoff: exponential on_error: - name: Log Error type: script script: echo "Operation failed: {{error}}" - name: Cleanup type: script script: echo "Cleaning up..." ``` ### 2. Performance Optimization ```yaml # Use parallel execution where possible steps: - name: Parallel Operations type: parallel max_concurrent: 3 steps: - name: Operation 1 type: call # ... operation details - name: Operation 2 type: call # ... operation details ``` ### 3. Resource Management ```yaml # Set appropriate resource limits nodes: count: 3 # Scale based on needs ``` ### 4. Monitoring and Validation ```yaml # Always validate results steps: - name: Validate Results type: assert statements: - '{{result}} != null' - "contains({{result}}, 'expected_value')" ``` ## Next Steps Now that you understand advanced examples: - [Best Practices](./best-practices) - Guidelines for effective Merobox usage --- ## Authentication Service Integration # Authentication Service Integration This guide covers basic authentication service configuration for Merobox, including enabling the auth service and basic configuration options. ## Basic Auth Service Configuration Merobox provides basic authentication service support with Traefik proxy: ### Enable Auth Service Enable authentication service when starting nodes: ```bash # Start nodes with auth service merobox run --count 2 --auth-service # Use custom auth service image merobox run --count 2 --auth-service --auth-image ghcr.io/calimero-network/mero-auth:latest ``` ### Workflow Configuration Enable auth service in workflow files: ```yaml # workflow.yml auth_service: true auth_image: ghcr.io/calimero-network/mero-auth:edge nodes: chain_id: testnet-1 count: 2 image: ghcr.io/calimero-network/merod:edge prefix: calimero-node ``` ### Environment Variables Configure auth service using environment variables: ```bash # Auth service is configured through CLI flags # merobox run --auth-service --auth-image ghcr.io/calimero-network/mero-auth:edge ``` ## Auth Service Features When auth service is enabled, Merobox provides: ### Node Access - **Authenticated Access**: Nodes are accessible via `http://node1.127.0.0.1.nip.io` - **Admin Dashboard**: Available at `http://node1.127.0.0.1.nip.io/admin-dashboard` - **API Authentication**: All API endpoints require authentication ### Traefik Integration The auth service uses Traefik proxy for: - **Automatic HTTPS**: SSL termination and certificate management - **Load Balancing**: Distribute requests across multiple nodes - **Authentication**: Handle authentication for all requests ## Managing Auth Service ### Start Auth Service ```bash # Start nodes with auth service merobox run --count 2 --auth-service # Stop auth service merobox stop --auth-service # Stop all nodes and auth service merobox stop --all ``` ### Check Auth Service Status ```bash # List running services merobox list # Check health status merobox health --verbose ``` ## Troubleshooting ### Common Issues Check auth service logs and status: ```bash # Check auth service logs docker logs calimero-auth-service # Check Traefik logs docker logs calimero-traefik # Verify auth service is running docker ps | grep auth ``` ### Debug Mode Enable debug logging for troubleshooting: ```bash # Enable debug logging export LOG_LEVEL=DEBUG # Run with verbose output merobox run --count 2 --auth-service --verbose ``` ## Best Practices ### Security 1. **Use HTTPS**: Always use HTTPS in production environments 2. **Secure Images**: Use specific image tags instead of `latest` 3. **Environment Variables**: Store sensitive configuration in environment variables 4. **Regular Updates**: Keep auth service images updated ### Performance 1. **Resource Limits**: Set appropriate memory and CPU limits 2. **Monitoring**: Monitor auth service performance 3. **Scaling**: Consider multiple nodes for high availability ## Next Steps Now that you understand basic authentication service integration: - [Node Management](./node-management) - Complete node management guide - [Workflows](./workflows) - Workflow system and automation - [Environment Variables](./environment-variables) - Configuration options - [Troubleshooting](./troubleshooting) - Common issues and solutions --- ## Best Practices # Best Practices This guide provides comprehensive best practices for using Merobox effectively, from workflow design to testing integration and performance optimization. ## Workflow Design ### 1. Start Simple Begin with basic workflows and gradually add complexity: ```yaml # Start with simple workflows description: Simple workflow name: Basic Workflow nodes: chain_id: testnet-1 count: 1 image: ghcr.io/calimero-network/merod:edge steps: - name: Basic Step type: script script: echo "Hello, Merobox!" stop_all_nodes: true ``` ### 2. Modular Steps Break complex operations into smaller, focused steps: ```yaml # Good: Modular approach steps: - name: Setup Environment type: script script: | echo "Setting up environment..." # Setup logic - name: Install Application type: install_application node: calimero-node-1 path: ./my-app.wasm - name: Configure Application type: script script: | echo "Configuring application..." # Configuration logic - name: Test Application type: call node: calimero-node-1 method: test_functionality ``` ### 3. Clear Naming Use descriptive names for steps and variables: ```yaml # Good: Descriptive names steps: - name: Install User Management Application type: install_application node: calimero-node-1 path: ./user-mgmt.wasm outputs: user_app_id: applicationId - name: Create User Management Context type: create_context node: calimero-node-1 application_id: '{{user_app_id}}' outputs: user_context_id: contextId admin_key: memberPublicKey ``` ### 4. Error Handling Include validation and error checking steps: ```yaml steps: - name: Risky Operation type: call node: calimero-node-1 method: risky_method retry: attempts: 3 delay: 5 backoff: exponential on_error: - name: Log Error type: script script: echo "Operation failed: {{error}}" - name: Cleanup type: script script: echo "Cleaning up..." - name: Validate Results type: assert statements: - '{{result}} != null' - "contains({{result}}, 'expected_value')" ``` ## Testing Integration ### 1. Isolated Tests Use separate node prefixes for different test suites: ```python # Use different prefixes for isolation @pytest.fixture def unit_test_cluster(): with cluster(count=1, prefix="unit") as env: yield env @pytest.fixture def integration_test_cluster(): with cluster(count=3, prefix="integration") as env: yield env @pytest.fixture def performance_test_cluster(): with cluster(count=5, prefix="performance") as env: yield env ``` ### 2. Resource Cleanup Always clean up resources after tests: ```python @pytest.fixture def test_cluster(): with cluster(count=2, prefix="test") as env: try: yield env finally: # Cleanup logic print("Cleaning up test resources...") ``` ### 3. Parallel Testing Use different prefixes for parallel test execution: ```python # Enable parallel testing pytestmark = pytest.mark.parallel def test_parallel_1(): with cluster(count=1, prefix="parallel-1") as env: # Test logic pass def test_parallel_2(): with cluster(count=1, prefix="parallel-2") as env: # Test logic pass ``` ### 4. Environment Setup Use workflows for complex test environment setup: ```yaml # workflows/test-setup.yml description: Test environment setup name: Test Setup nodes: chain_id: testnet-1 count: 3 image: ghcr.io/calimero-network/merod:edge prefix: test-node steps: - name: Install Application type: install_application node: test-node-1 path: ./test-app.wasm dev: true outputs: app_id: applicationId - name: Create Context type: create_context node: test-node-1 application_id: '{{app_id}}' outputs: context_id: contextId member_key: memberPublicKey - name: Setup Test Data type: call node: test-node-1 context_id: '{{context_id}}' executor_public_key: '{{member_key}}' method: setup_test_data args: test_cases: 100 stop_all_nodes: false # Keep nodes running for tests ``` ## Performance Considerations ### 1. Minimal Waits Use appropriate wait times, not excessive delays: ```yaml # Good: Minimal wait times steps: - name: Wait for Node type: wait seconds: 5 # Reasonable wait time message: 'Waiting for node to start...' # Bad: Excessive wait times steps: - name: Wait for Node type: wait seconds: 300 # Too long! ``` ### 2. Resource Management Stop nodes when not needed: ```yaml # Good: Stop nodes when done stop_all_nodes: true # Or stop specific nodes steps: - name: Stop Unused Nodes type: script script: | merobox stop calimero-node-2 merobox stop calimero-node-3 ``` ### 3. Efficient Operations Design workflows to minimize sequential dependencies: ```yaml # Good: Parallel operations steps: - name: Parallel Setup type: parallel steps: - name: Setup Node 1 type: script script: echo "Setting up node 1" - name: Setup Node 2 type: script script: echo "Setting up node 2" - name: Setup Node 3 type: script script: echo "Setting up node 3" # Bad: Sequential operations steps: - name: Setup Node 1 type: script script: echo "Setting up node 1" - name: Setup Node 2 type: script script: echo "Setting up node 2" - name: Setup Node 3 type: script script: echo "Setting up node 3" ``` ### 4. Monitoring Include health checks and monitoring in long-running workflows: ```yaml steps: - name: Health Check type: script script: | echo "Checking node health..." curl -f http://calimero-node-1:2428/health echo "Node is healthy" - name: Monitor Performance type: script script: | echo "Monitoring performance..." # Add monitoring logic ``` ## Security Best Practices ### 1. Use Non-Root Users ```yaml nodes: security: user: '1000:1000' # Non-root user read_only: true no_new_privileges: true ``` ### 2. Drop Unnecessary Capabilities ```yaml nodes: security: capabilities: drop: ['ALL'] add: ['NET_BIND_SERVICE'] # Only add what's needed ``` ### 3. Use Secrets Management ```yaml secrets: - name: api-key environment: API_KEY required: true nodes: secrets: - api-key environment: API_KEY_FILE: /run/secrets/api-key ``` ### 4. Network Segmentation ```yaml networks: - name: calimero-internal driver: bridge options: com.docker.network.bridge.enable_icc: 'false' com.docker.network.bridge.enable_ip_masquerade: 'true' ``` ## Configuration Management ### 1. Environment-Specific Configuration ```yaml # Use different configurations for different environments development: nodes: count: 1 resources: memory: '1G' cpus: '0.5' security: read_only: false production: nodes: count: 5 resources: memory: '4G' cpus: '2.0' security: read_only: true no_new_privileges: true ``` ### 2. Version Control Track configuration changes in version control: ```bash # Track workflow files git add workflows/ git commit -m "Add new workflow configuration" # Track environment files git add .env.example git commit -m "Update environment configuration" ``` ### 3. Configuration Validation Validate configuration before deployment: ```yaml steps: - name: Validate Configuration type: script script: | echo "Validating configuration..." # Add validation logic echo "Configuration is valid" ``` ## Monitoring and Observability ### 1. Comprehensive Monitoring ```yaml monitoring: enabled: true metrics: - cpu_usage - memory_usage - disk_usage - network_io - application_metrics alerts: - metric: memory_usage threshold: 80 action: restart_node ``` ### 2. Logging Include comprehensive logging: ```yaml steps: - name: Log Important Events type: script script: | echo "Starting important operation..." # Operation logic echo "Operation completed successfully" ``` ### 3. Health Checks Include health checks in workflows: ```yaml steps: - name: Health Check type: script script: | echo "Performing health check..." curl -f http://calimero-node-1:2428/health echo "Health check passed" ``` ## Troubleshooting ### 1. Debug Mode Enable debug mode for troubleshooting: ```bash # Enable debug logging export LOG_LEVEL=DEBUG # Run with verbose output merobox bootstrap run workflow.yml --verbose ``` ### 2. Resource Monitoring Monitor resource usage: ```bash # Check resource usage docker stats $(docker ps -q --filter "name=calimero-") # Check logs merobox logs calimero-node-1 ``` ### 3. Network Diagnostics Diagnose network issues: ```bash # Check network connectivity docker network ls docker network inspect calimero-web # Test connectivity docker exec calimero-node-1 ping calimero-node-2 ``` ## Documentation ### 1. Document Workflows Include comprehensive documentation: ```yaml description: | This workflow demonstrates how to set up a multi-node Calimero cluster with authentication service integration. It includes error handling, monitoring, and cleanup. name: Multi-Node Auth Setup # ... workflow steps ``` ### 2. Document Configuration Document configuration options: ```yaml # Configuration documentation nodes: # Number of nodes to create count: 3 # Docker image to use image: ghcr.io/calimero-network/merod:edge # Resource limits resources: memory: '2G' # Memory limit per node cpus: '1.0' # CPU limit per node ``` ### 3. Document Dependencies Document external dependencies: ```yaml # External dependencies # - Docker must be running # - Ports 2428, 2528, 2628 must be available # - At least 4GB RAM and 2 CPU cores recommended ``` ## Common Pitfalls ### 1. Resource Exhaustion **Problem**: Not enough resources for the workload. **Solution**: Monitor resource usage and scale appropriately. ```yaml # Monitor resources monitoring: enabled: true metrics: - memory_usage - cpu_usage alerts: - metric: memory_usage threshold: 80 action: scale_up ``` ### 2. Network Issues **Problem**: Nodes can't communicate with each other. **Solution**: Check network configuration and connectivity. ```yaml # Proper network configuration networks: - name: calimero-web driver: bridge options: com.docker.network.bridge.enable_icc: 'true' ``` ### 3. Timing Issues **Problem**: Operations fail due to timing issues. **Solution**: Use appropriate wait times and retry logic. ```yaml # Proper timing steps: - name: Wait for Node type: wait seconds: 10 message: 'Waiting for node to start...' - name: Retry Operation type: call node: calimero-node-1 method: operation retry: attempts: 3 delay: 5 ``` ## Next Steps Now that you understand best practices: - [Quick Start Tutorials](./quick-start-tutorials) - Get started with Merobox - [Real-World Examples](./real-world-examples) - Practical examples for common scenarios - [Testing Integration Examples](./testing-integration-examples) - Integration with testing frameworks - [Advanced Examples](./advanced-examples) - Complex workflows and advanced features --- ## Docker Image Management # Docker Image Management This guide covers basic Docker image management for Merobox deployments. ## Basic Image Configuration Merobox uses Docker images for running Calimero nodes: ```yaml # workflow.yml nodes: image: ghcr.io/calimero-network/merod:edge auth_image: ghcr.io/calimero-network/mero-auth:edge ``` ### Image Selection Choose the right image for your use case: ```yaml # Production - stable release nodes: image: ghcr.io/calimero-network/merod:latest # Development - edge features nodes: image: ghcr.io/calimero-network/merod:edge # Specific version nodes: image: ghcr.io/calimero-network/merod:v0.7.0 ``` ## Basic Image Management ### Pulling Images ```bash # Pull the default Calimero image docker pull ghcr.io/calimero-network/merod:edge # Pull the authentication service image docker pull ghcr.io/calimero-network/mero-auth:edge # List available images docker images | grep calimero ``` ### Image Verification ```bash # Check image details docker inspect ghcr.io/calimero-network/merod:edge # Test image locally docker run --rm ghcr.io/calimero-network/merod:edge --version # Check image layers docker history ghcr.io/calimero-network/merod:edge ``` ## Basic Docker Commands ### Container Management ```bash # List running containers docker ps # List all containers docker ps -a # Stop containers docker stop calimero-node-1 # Remove containers docker rm calimero-node-1 # Clean up unused containers docker container prune ``` ### Image Management ```bash # List images docker images # Remove unused images docker image prune # Clean up all unused resources docker system prune ``` ## Troubleshooting Image Issues ### Common Problems ```bash # Check image availability docker pull ghcr.io/calimero-network/merod:edge # Inspect image details docker inspect ghcr.io/calimero-network/merod:edge # Check image layers docker history ghcr.io/calimero-network/merod:edge # Test image locally docker run --rm ghcr.io/calimero-network/merod:edge --version ``` ### Debug Image Pull Issues ```bash # Enable debug logging export DOCKER_BUILDKIT=0 export DOCKER_CLI_EXPERIMENTAL=enabled # Check registry connectivity docker login ghcr.io docker pull ghcr.io/calimero-network/merod:edge # Verify image integrity docker run --rm ghcr.io/calimero-network/merod:edge sh -c "echo 'Image is working'" ``` ## Best Practices ### Image Selection 1. **Use specific tags**: Avoid `latest` in production 2. **Regular updates**: Keep images up to date with security patches 3. **Version pinning**: Pin to specific versions for reproducibility ### Performance Optimization 1. **Pre-pull images**: Pull images before running workflows 2. **Clean up unused images**: Regularly clean up unused Docker resources 3. **Use appropriate image sizes**: Choose the right image for your use case ### Security Considerations 1. **Use official images**: Prefer official Calimero images 2. **Regular updates**: Keep images updated with security patches 3. **Verify image integrity**: Check image signatures when possible ## Next Steps Now that you understand basic Docker image management: - [Node Management](./node-management) - Complete node management guide - [Workflows](./workflows) - Workflow system and automation - [Environment Variables](./environment-variables) - Configuration options - [Troubleshooting](./troubleshooting) - Common issues and solutions --- ## Environment Variables # Environment Variables Merobox supports several environment variables for configuration, allowing you to customize Docker images, logging, networking, and other aspects of the system. ## Core Configuration Basic environment variables for Merobox operation: ```bash # Docker daemon connection (standard Docker environment variable) export DOCKER_HOST="unix:///var/run/docker.sock" # Logging level (if supported by Merobox) export LOG_LEVEL="INFO" # DEBUG, INFO, WARNING, ERROR ``` ### Docker Configuration Standard Docker environment variables: ```bash # Docker daemon settings export DOCKER_HOST="unix:///var/run/docker.sock" export DOCKER_TLS_VERIFY="1" export DOCKER_CERT_PATH="/path/to/certs" # Docker build settings export DOCKER_BUILDKIT="1" # Enable BuildKit export DOCKER_CLI_EXPERIMENTAL="enabled" # Enable experimental features ``` ## Authentication Service The authentication service is configured through CLI flags, not environment variables: ```bash # Enable authentication service merobox run --auth-service # Use custom auth service image merobox run --auth-service --auth-image ghcr.io/calimero-network/mero-auth:latest ``` ## Network Configuration Network settings are configured through CLI flags: ```bash # Custom base ports merobox run --base-port 3000 --base-rpc-port 4000 # Use different port ranges merobox run --count 3 --base-port 3000 --base-rpc-port 4000 ``` ## Development and Testing For development and testing, use CLI flags instead of environment variables: ```bash # Development mode (if supported) merobox run --count 1 --verbose # Testing with custom configuration merobox run --count 2 --prefix test-nodes ``` ## Production Configuration For production deployments, use CLI flags and workflow configuration: ```bash # Production deployment merobox run --count 5 --image ghcr.io/calimero-network/merod:latest # With authentication service merobox run --count 5 --auth-service --image ghcr.io/calimero-network/merod:latest ``` ## Best Practices ### Environment Management 1. **Use CLI flags**: Prefer CLI flags over environment variables when possible 2. **Documentation**: Document all custom configurations 3. **Testing**: Test configurations before deployment ### Security Considerations 1. **Secrets**: Never store secrets in version control 2. **Access control**: Limit access to production configurations 3. **Auditing**: Log configuration changes for audit purposes ## Next Steps Now that you understand environment variables: - [Docker Image Management](./docker-image-management) - Managing Docker images and containers - [Network Configuration](./network-configuration) - Advanced network setup - [Authentication Service Integration](./auth-service-integration) - Auth service configuration - [Advanced Configuration](./advanced-configuration) - Other advanced features --- ## Examples and Tutorials # Examples and Tutorials This section provides comprehensive examples and tutorials to help you get started with Merobox and understand its capabilities across different use cases. ## Overview Merobox examples are organized into specialized areas to help you find exactly what you need: - **[Quick Start Tutorials](./quick-start-tutorials)** - Step-by-step tutorials to get you started quickly - **[Real-World Examples](./real-world-examples)** - Practical examples for common real-world scenarios - **[Testing Integration Examples](./testing-integration-examples)** - Integration with testing frameworks and CI/CD - **[Advanced Examples](./advanced-examples)** - Complex workflows and advanced features for power users - **[Best Practices](./best-practices)** - Guidelines and best practices for effective Merobox usage ## Quick Start If you're new to Merobox, start with the [Quick Start Tutorials](./quick-start-tutorials) to learn the basics: 1. **Your First Workflow** - Create and run your first Merobox workflow 2. **Application Deployment** - Deploy a simple WASM application 3. **Multi-Node Setup** - Work with multiple nodes 4. **Authentication Service** - Set up auth service integration 5. **Custom Scripts** - Use custom scripts in workflows ## Common Use Cases ### Development and Testing - **Local Development**: Set up isolated development environments - **Unit Testing**: Test individual components in isolation - **Integration Testing**: Test component interactions - **Performance Testing**: Test application performance under load ### Production Scenarios - **Multi-Node Deployments**: Deploy across multiple nodes - **Authentication Integration**: Integrate with authentication services - **Monitoring and Alerting**: Set up comprehensive monitoring - **Load Balancing**: Test applications behind load balancers ### CI/CD Integration - **Automated Testing**: Integrate with CI/CD pipelines - **Parallel Test Execution**: Run tests in parallel for faster feedback - **Test Data Management**: Manage test data effectively - **Reporting and Coverage**: Generate comprehensive test reports ## Example Categories ### Basic Examples Start with these fundamental examples: ```yaml # Simple workflow description: My first Merobox workflow name: First Workflow nodes: chain_id: testnet-1 count: 1 image: ghcr.io/calimero-network/merod:edge steps: - name: Wait for Node type: wait seconds: 5 - name: Check Health type: script script: echo "Node is running!" stop_all_nodes: true ``` ### Intermediate Examples Build on the basics with these examples: - Multi-node communication - Application deployment and testing - Authentication service integration - Performance testing - Database integration ### Advanced Examples For power users, explore these advanced features: - Custom step types - Dynamic workflow generation - Workflow composition - Advanced monitoring - Error handling and recovery ## Getting Help ### Documentation - **Workflow System**: Learn about workflow configuration and step types - **Node Management**: Understand node operations and management - **Advanced Configuration**: Explore advanced configuration options - **Python Customization**: Extend Merobox with Python ### Community - **GitHub Issues**: Report bugs and request features - **Discussions**: Ask questions and share experiences - **Examples Repository**: Find more examples and templates ### Support - **Troubleshooting**: Common issues and solutions - **Best Practices**: Guidelines for effective usage - **Performance Tips**: Optimize your workflows ## Next Steps Choose your path based on your needs: ### For Beginners 1. Start with [Quick Start Tutorials](./quick-start-tutorials) 2. Try [Real-World Examples](./real-world-examples) 3. Learn [Best Practices](./best-practices) ### For Developers 1. Explore [Testing Integration Examples](./testing-integration-examples) 2. Study [Advanced Examples](./advanced-examples) 3. Follow [Best Practices](./best-practices) ### For DevOps Engineers 1. Review [Real-World Examples](./real-world-examples) 2. Implement [Testing Integration Examples](./testing-integration-examples) 3. Apply [Best Practices](./best-practices) ## Contributing Examples We welcome contributions to improve our examples: 1. **Fork the repository** 2. **Create your example** following our guidelines 3. **Test thoroughly** to ensure it works 4. **Submit a pull request** with clear documentation ### Example Guidelines - **Clear Documentation**: Include comprehensive comments and explanations - **Working Code**: Ensure all examples are tested and functional - **Best Practices**: Follow established patterns and conventions - **Error Handling**: Include proper error handling and validation - **Resource Cleanup**: Always clean up resources when done ## Resources ### Documentation - [Installation and Setup](./installation) - Get started with Merobox - [Node Management](./node-management) - Manage Calimero nodes - [Workflow System](./workflows) - Configure and run workflows - [Advanced Configuration](./advanced-configuration) - Advanced features ### Tools - [Python Customization](./python-customization) - Extend Merobox with Python - [Troubleshooting](./troubleshooting) - Common issues and solutions ### Community - [GitHub Repository](https://github.com/calimero-network/merobox) - Source code and issues - [Discussions](https://github.com/calimero-network/merobox/discussions) - Community discussions - [Examples Repository](https://github.com/calimero-network/merobox-examples) - More examples and templates --- ## Merobox Overview # Merobox Overview Merobox is a command-line interface (CLI) tool designed to streamline the development, testing, and deployment of Calimero applications. It provides a robust framework for managing Calimero nodes in Docker, orchestrating complex workflows with YAML configurations, and integrating with popular testing frameworks. ## Key Features - **Node Management**: Easily run, stop, list, and manage Calimero nodes in isolated Docker containers. - **Workflow Automation**: Define and execute multi-step workflows using declarative YAML files for tasks like application installation, context creation, identity management, and function calls. - **Application Lifecycle**: Simplify the process of deploying and interacting with Calimero applications. - **Context Orchestration**: Manage blockchain contexts, including creation, invitations, and joining. - **Identity Management**: Generate and manage cryptographic identities for testing and development. - **Testing Integration**: Seamlessly integrate with Python testing frameworks like Pytest for ephemeral node and workflow setups. - **Dynamic Value Capture**: Capture and reuse outputs from previous steps within your workflows. - **Authentication Service Integration**: Configure and interact with Calimero's authentication service. ## Getting Started To get started with Merobox, refer to the [Installation Guide](./installation) and the [Quick Start Tutorials](./quick-start-tutorials). ## Further Reading - [Installation](./installation) - [Node Management](./node-management) - [Workflows](./workflows) - [Examples](./examples) - [Advanced Configuration](./advanced-configuration) - [Python Customization](./python-customization) - [Troubleshooting](./troubleshooting) For more developer-focused instructions and the latest updates, please refer to the [Merobox GitHub README](https://github.com/calimero-network/merobox/blob/main/README.md). --- ## Installation and Setup # Installation and Setup This guide covers installing Merobox CLI and setting up your development environment for managing Calimero nodes in Docker containers. ## Prerequisites Before installing Merobox, ensure you have the following prerequisites installed on your system: ### Required Software - **Python 3.8 or higher**: Merobox is a Python CLI tool - **pipx**: For installing Python applications in isolated environments (required for Calimero development) - **Docker 20.10 or higher**: For running Calimero nodes in containers - **Docker Compose**: For managing multi-container applications - **Git**: For cloning repositories and version control ### System Requirements - **Operating System**: Linux, macOS, or Windows - **Memory**: At least 4GB RAM (8GB recommended for multiple nodes) - **Storage**: At least 10GB free disk space - **Network**: Internet connection for downloading Docker images ### Verify Prerequisites Check that all prerequisites are installed and working: ```bash # Check Python version python3 --version # Should output: Python 3.8.x or higher # Check pipx installation pipx --version # Should output: pipx version 1.x.x or higher # Check Docker installation docker --version # Should output: Docker version 20.10.x or higher # Check Docker Compose docker compose version # Should output: Docker Compose version 2.x.x or higher # Check Git git --version # Should output: git version 2.x.x or higher ``` ## Installation Methods ### Method 1: Install with pipx (Recommended) For Calimero development, install Merobox using pipx to ensure proper isolation: 1. **Install pipx** using a package manager for your OS (see [pipx installation guide](https://github.com/pypa/pipx?tab=readme-ov-file#install-pipx)) 2. **Install Merobox and dependencies:** ```bash # Install maturin (required dependency) pipx install maturin # Install Merobox pipx install merobox ``` ### Method 3: Install from Source (For Merobox Development Only) If you want to contribute to Merobox itself or use the latest development version: ```bash # Clone the repository git clone https://github.com/calimero-network/merobox.git cd merobox # Install in development mode using pip (only for Merobox development) pip install -e . ``` :::warning Development Context The `pip` installation method above is **only** for developing Merobox itself. For Calimero app development, always use `pipx` as shown in Method 1. ::: ## Verify Installation After installation, verify that Merobox is working correctly: ```bash # Check Merobox version merobox --version # Should output: merobox, version 0.1.20 # Check available commands merobox --help # Should display the help message with available commands ``` ## Docker Setup Merobox relies on Docker to run Calimero nodes. Ensure Docker is properly configured: ### Start Docker Service **Linux (systemd):** ```bash sudo systemctl start docker sudo systemctl enable docker ``` **macOS:** Start Docker Desktop from Applications or: ```bash open -a Docker ``` **Windows:** Start Docker Desktop from Start Menu ### Verify Docker is Running ```bash # Check Docker daemon status docker info # Should show Docker system information # Test Docker with a simple container docker run hello-world # Should download and run the hello-world container ``` ### Docker Permissions (Linux) If you get permission errors, add your user to the docker group: ```bash # Add user to docker group sudo usermod -aG docker $USER # Log out and log back in, or run: newgrp docker # Verify permissions docker run hello-world ``` ## Configuration ### Environment Variables Merobox supports several environment variables for configuration: ```bash # Docker image is set through CLI flags # merobox run --image ghcr.io/calimero-network/merod:edge # Set Docker daemon connection export DOCKER_HOST="unix:///var/run/docker.sock" # Set logging level export LOG_LEVEL="INFO" # DEBUG, INFO, WARNING, ERROR ``` ### Docker Image Management Merobox automatically manages Docker images, but you can pre-pull images for faster startup: ```bash # Pull the default Calimero image docker pull ghcr.io/calimero-network/merod:edge # Pull the authentication service image docker pull ghcr.io/calimero-network/mero-auth:edge # List available images docker images | grep calimero ``` ## Quick Test Test your installation with a simple workflow: ```bash # Start a single Calimero node merobox run --count 1 # Check if the node is running merobox list # Check node health merobox health # Stop the node merobox stop ``` ## Troubleshooting Installation ### Common Issues #### Python Not Found ```bash # Error: python3: command not found ``` **Solution:** - Install Python 3.8+ from [python.org](https://python.org) - On Ubuntu/Debian: `sudo apt install python3 python3-pip` - On macOS: `brew install python3` - On Windows: Download from Microsoft Store or python.org #### pipx Not Found ```bash # Error: pipx: command not found ``` **Solution:** - Install pipx using a package manager for your OS (see [pipx installation guide](https://github.com/pypa/pipx?tab=readme-ov-file#install-pipx)) - Restart your terminal after installation #### Docker Not Running ```bash # Error: Cannot connect to the Docker daemon ``` **Solution:** - Start Docker service: `sudo systemctl start docker` (Linux) - Start Docker Desktop (macOS/Windows) - Check Docker status: `docker info` #### Permission Denied ```bash # Error: Permission denied while trying to connect to Docker daemon ``` **Solution:** - Add user to docker group: `sudo usermod -aG docker $USER` - Log out and log back in - Or run: `newgrp docker` #### Port Already in Use ```bash # Error: Port 2528 already in use ``` **Solution:** - Find process using port: `lsof -ti:2528` - Kill process: `lsof -ti:2528 | xargs kill` - Or use different ports: `merobox run --count 1` #### Network Issues ```bash # Error: Failed to pull image ``` **Solution:** - Check internet connection - Verify Docker registry access: `docker pull hello-world` - Check firewall settings - Try with different DNS: `echo "nameserver 8.8.8.8" | sudo tee /etc/resolv.conf` ### Getting Help If you encounter issues not covered here: 1. **Check Merobox help**: `merobox --help` 2. **Check command help**: `merobox --help` 3. **Enable debug logging**: `LOG_LEVEL=DEBUG merobox ` 4. **Check Docker logs**: `docker logs ` 5. **GitHub Issues**: [Report issues on GitHub](https://github.com/calimero-network/merobox/issues) ## Next Steps Now that Merobox is installed and configured: - [Basic Node Management](./node-management) - Learn how to start, stop, and manage nodes - [Workflow Configuration](./workflows) - Create and run complex workflows - [Examples and Tutorials](./examples) - Practical examples and tutorials - [Advanced Configuration](./advanced-configuration) - Advanced setup options ## Development Setup :::warning Merobox Development Only This section is **only** for contributing to Merobox itself. For Calimero development, use the pipx installation method above. ::: If you plan to contribute to Merobox or run it from source: ### Clone and Setup ```bash # Clone the repository git clone https://github.com/calimero-network/merobox.git cd merobox # Create virtual environment python3 -m venv venv source venv/bin/activate # On Windows: venv\Scripts\activate # Install development dependencies pip install -r requirements.txt # Install in development mode pip install -e . ``` ### Development Dependencies The development environment includes: - **Black**: Code formatting - **Ruff**: Linting and code quality - **Pytest**: Testing framework - **Pre-commit**: Git hooks for code quality ### Running Tests ```bash # Run all tests pytest # Run specific test file pytest tests/test_specific.py # Run with verbose output pytest -v ``` ### Code Quality ```bash # Format code black merobox/ # Check formatting black --check merobox/ # Lint code ruff merobox/ ``` --- ## Network Configuration # Network Configuration This guide covers basic network configuration for Merobox, including port management and network settings. ## Port Configuration Merobox provides basic port configuration for Calimero nodes: ### Basic Port Settings Configure ports in workflow files: ```yaml # workflow.yml nodes: chain_id: testnet-1 count: 2 image: ghcr.io/calimero-network/merod:edge prefix: calimero-node base_port: 2428 # Base P2P port (optional) base_rpc_port: 2528 # Base RPC port (optional) ``` ### CLI Port Configuration Set ports using command-line flags: ```bash # Custom base ports merobox run --count 2 --base-port 2428 --base-rpc-port 2528 # Different port ranges merobox run --count 3 --base-port 3000 --base-rpc-port 4000 ``` ### Automatic Port Detection Merobox automatically detects available ports: - **P2P Ports**: Starting from 2428, increments by 1 for each node - **RPC Ports**: Starting from 2528, increments by 1 for each node - **Conflict Resolution**: Automatically finds next available ports ## Environment Variables Configure network settings using environment variables: ### Network Configuration ```bash # Network configuration is done through CLI flags # merobox run --base-port 3000 --base-rpc-port 4000 ``` ### Advanced Network Settings ```bash # Advanced network configuration is done through Docker # Create custom Docker networks if needed docker network create calimero-custom ``` ## Docker Networks Merobox uses Docker's default networking: ### Default Network Behavior - **Bridge Network**: Merobox uses Docker's default bridge network - **Automatic Management**: Networks are created and managed automatically - **Node Communication**: Nodes can communicate with each other by default ### Network Management ```bash # List Docker networks docker network ls # Inspect Merobox networks docker network inspect bridge # Check node network connectivity docker exec calimero-node-1 ping calimero-node-2 ``` ## Troubleshooting ### Common Network Issues Check network connectivity and port availability: ```bash # Check port binding netstat -tulpn | grep -E "(2428|2528)" # Test connectivity between nodes docker exec calimero-node-1 ping calimero-node-2 # Check Docker networks docker network ls docker network inspect bridge # Find processes using ports lsof -ti:2528 ``` ### Port Conflicts If ports are already in use: ```bash # Kill process using port lsof -ti:2528 | xargs kill # Or use different ports merobox run --base-rpc-port 3000 # Check all ports netstat -tulpn | grep -E "(2428|2528)" ``` ### Network Debugging Enable debug logging for troubleshooting: ```bash # Enable debug logging export LOG_LEVEL=DEBUG # Run with verbose output merobox run --count 2 --verbose # Check node logs merobox logs calimero-node-1 ``` ## Best Practices ### Port Management 1. **Use Default Ports**: Stick to default ports (2428, 2528) when possible 2. **Avoid Conflicts**: Check for port conflicts before starting nodes 3. **Document Ports**: Keep track of custom port configurations 4. **Test Connectivity**: Verify node communication after configuration changes ### Network Configuration 1. **Environment Variables**: Use environment variables for consistent configuration 2. **Port Ranges**: Use appropriate port ranges for different environments 3. **DNS Settings**: Configure DNS for better name resolution 4. **Network Isolation**: Use network isolation for security when needed ### Troubleshooting 1. **Check Logs**: Always check node logs first when issues occur 2. **Verify Ports**: Ensure ports are available and not conflicting 3. **Test Connectivity**: Use ping and other tools to test connectivity 4. **Docker Networks**: Check Docker network status and configuration ## Next Steps Now that you understand basic network configuration: - [Node Management](./node-management) - Complete node management guide - [Workflows](./workflows) - Workflow system and automation - [Environment Variables](./environment-variables) - Configuration options - [Troubleshooting](./troubleshooting) - Common issues and solutions --- ## Node Management # Node Management Merobox provides comprehensive commands for managing Calimero nodes in Docker containers. This guide covers all the essential node management operations. ## Overview Merobox's node management system allows you to: - **Start and Stop Nodes**: Launch single or multiple Calimero nodes - **Monitor Health**: Check node status and connectivity - **View Logs**: Access real-time and historical logs - **Manage Data**: Clean up node data and reset environments - **Authentication**: Enable authentication services with Traefik proxy ## Starting Nodes ### Basic Node Startup Start a single Calimero node with default settings: ```bash merobox run ``` This creates a node named `calimero-node-1` with: - Chain ID: `testnet-1` - Auto-detected ports - Default Docker image: `ghcr.io/calimero-network/merod:edge` ### Multiple Nodes Start multiple nodes for testing multi-node scenarios: ```bash # Start 3 nodes merobox run --count 3 # Start 5 nodes with custom prefix merobox run --count 5 --prefix my-nodes ``` ### Custom Configuration Configure nodes with specific settings: ```bash # Custom chain ID and ports merobox run --count 2 --chain-id mainnet-1 --base-port 2428 --base-rpc-port 2528 # Custom Docker image merobox run --image ghcr.io/calimero-network/merod:latest # Force pull latest image merobox run --force-pull --image ghcr.io/calimero-network/merod:edge ``` ### Authentication Service Enable authentication service with Traefik proxy: ```bash # Start nodes with auth service merobox run --count 2 --auth-service # Custom auth service image merobox run --auth-service --auth-image ghcr.io/calimero-network/mero-auth:latest ``` When auth service is enabled: - Nodes are accessible via `http://node1.127.0.0.1.nip.io` - Admin dashboard: `http://node1.127.0.0.1.nip.io/admin-dashboard` - API endpoints require authentication ### Single Node with Custom Data Directory For development with persistent data: ```bash merobox run --data-dir /path/to/custom/data ``` ## Command Reference ### `merobox run` Start Calimero nodes in Docker containers. **Syntax:** ```bash merobox run [OPTIONS] ``` **Options:** | Option | Short | Default | Description | | ----------------- | ----- | ------------------ | ---------------------------------------- | | `--count` | `-c` | `1` | Number of nodes to start | | `--base-port` | `-p` | Auto-detect | Base P2P port for first node | | `--base-rpc-port` | `-r` | Auto-detect | Base RPC port for first node | | `--chain-id` | | `testnet-1` | Blockchain chain ID | | `--prefix` | | `calimero-node` | Node name prefix | | `--data-dir` | | | Custom data directory (single node only) | | `--image` | | Default image | Custom Docker image | | `--force-pull` | | `False` | Force pull Docker image | | `--auth-service` | | `False` | Enable authentication service | | `--auth-image` | | Default auth image | Custom auth service image | **Examples:** ```bash # Basic usage merobox run # Multiple nodes merobox run --count 3 # Custom configuration merobox run --count 2 --chain-id mainnet-1 --prefix my-nodes # With authentication merobox run --count 2 --auth-service # Force pull latest image merobox run --force-pull --image ghcr.io/calimero-network/merod:edge ``` ## Stopping Nodes ### Stop Specific Node Stop a single node by name: ```bash merobox stop calimero-node-1 ``` ### Stop All Nodes Stop all running Calimero nodes: ```bash merobox stop --all ``` ### Stop Authentication Service Stop only the authentication service stack: ```bash merobox stop --auth-service ``` ### Command Reference ### `merobox stop` Stop Calimero nodes and services. **Syntax:** ```bash merobox stop [NODE_NAME] [OPTIONS] ``` **Arguments:** - `NODE_NAME`: Specific node name to stop (optional) **Options:** | Option | Description | | ---------------- | --------------------------------------------- | | `--all` | Stop all running nodes and auth service stack | | `--auth-service` | Stop auth service stack only (Traefik + Auth) | **Examples:** ```bash # Stop specific node merobox stop calimero-node-1 # Stop all nodes merobox stop --all # Stop auth service only merobox stop --auth-service ``` ## Monitoring Nodes ### List Running Nodes View all currently running Calimero nodes: ```bash merobox list ``` Output shows: - Node names - Container status - Port mappings - Image information ### Check Node Health Monitor node health and connectivity: ```bash # Check all running nodes merobox health # Check specific node merobox health --node calimero-node-1 # Verbose output with detailed information merobox health --verbose ``` Health check includes: - **Health Status**: Node operational status - **Authentication**: Admin API authentication status - **Peers**: Number of connected peers - **API Connectivity**: Admin API endpoint availability ### View Node Logs Access real-time and historical logs: ```bash # Show last 100 lines merobox logs calimero-node-1 # Show last 500 lines merobox logs calimero-node-1 --tail 500 # Follow logs in real-time merobox logs calimero-node-1 --follow ``` ### Command Reference #### `merobox list` List all running Calimero nodes. **Syntax:** ```bash merobox list ``` #### `merobox health` Check node health status. **Syntax:** ```bash merobox health [OPTIONS] ``` **Options:** | Option | Short | Default | Description | | ----------- | ----- | --------- | -------------------------------------- | | `--node` | `-n` | All nodes | Specific node name to check | | `--timeout` | | `10` | Timeout in seconds for health check | | `--verbose` | `-v` | `False` | Show verbose output with raw responses | #### `merobox logs` View node logs. **Syntax:** ```bash merobox logs NODE_NAME [OPTIONS] ``` **Arguments:** - `NODE_NAME`: Name of the node to view logs for **Options:** | Option | Default | Description | | -------- | ------- | --------------------------- | | `--tail` | `100` | Number of log lines to show | ## Data Management ### Complete Reset Remove all node data and containers for a fresh start: ```bash # Dry run to see what would be deleted merobox nuke --dry-run # Actually delete all data (requires confirmation) merobox nuke # Force deletion without confirmation merobox nuke --force ``` The `nuke` command: - Stops all running nodes - Deletes all data directories - Removes containers - Shows space freed ### Command Reference #### `merobox nuke` Delete all Calimero node data for complete reset. **Syntax:** ```bash merobox nuke [OPTIONS] ``` **Options:** | Option | Short | Description | | ----------- | ----- | ---------------------------------------------------- | | `--dry-run` | | Show what would be deleted without actually deleting | | `--force` | `-f` | Force deletion without confirmation prompt | | `--verbose` | `-v` | Show verbose output | ## Port Management ### Automatic Port Detection Merobox automatically detects available ports: - **P2P Ports**: Starting from 2428, increments by 1 for each node - **RPC Ports**: Starting from 2528, increments by 1 for each node - **Conflict Resolution**: Automatically finds next available ports ### Custom Port Configuration Specify custom ports for your nodes: ```bash # Custom base ports merobox run --count 3 --base-port 3000 --base-rpc-port 4000 # Results in: # Node 1: P2P=3000, RPC=4000 # Node 2: P2P=3001, RPC=4001 # Node 3: P2P=3002, RPC=4002 ``` ### Port Conflicts If ports are already in use: ```bash # Check what's using a port lsof -ti:2528 # Kill process using port lsof -ti:2528 | xargs kill # Or use different ports merobox run --base-rpc-port 3000 ``` ## Docker Image Management ### Default Images Merobox uses these default Docker images: - **Node Image**: `ghcr.io/calimero-network/merod:edge` - **Auth Service**: `ghcr.io/calimero-network/mero-auth:edge` ### Custom Images Use custom Docker images: ```bash # Custom node image merobox run --image ghcr.io/calimero-network/merod:latest # Custom auth image merobox run --auth-service --auth-image ghcr.io/calimero-network/mero-auth:latest ``` ### Force Pull Images Always use the latest images: ```bash # Force pull node image merobox run --force-pull --image ghcr.io/calimero-network/merod:edge # Force pull in workflow # Set force_pull_image: true in workflow YAML ``` ## Network Configuration ### Docker Networks Merobox automatically creates and manages Docker networks: - **Default Network**: `bridge` for basic node communication - **Auth Networks**: `calimero_web` and `calimero_internal` when auth service is enabled ### Authentication Service Networks When `--auth-service` is enabled: - **calimero_web**: External communication (Traefik ↔ Internet) - **calimero_internal**: Secure backend communication (Auth ↔ Nodes) ### Network Troubleshooting Check network configuration: ```bash # List Docker networks docker network ls # Inspect network docker network inspect calimero_web # Check container networking docker inspect calimero-node-1 ``` ## Best Practices ### Development Workflow 1. **Start Fresh**: Use `merobox nuke` to clean up between development sessions 2. **Use Prefixes**: Use descriptive prefixes for different test scenarios 3. **Monitor Health**: Regularly check `merobox health` during development 4. **View Logs**: Use `merobox logs` to debug issues ### Production-like Testing 1. **Multiple Nodes**: Test with multiple nodes to simulate real scenarios 2. **Authentication**: Use `--auth-service` for production-like testing 3. **Custom Images**: Test with specific image versions 4. **Port Management**: Use custom ports to avoid conflicts ### Resource Management 1. **Clean Up**: Regularly use `merobox nuke` to free disk space 2. **Monitor Resources**: Use `docker stats` to monitor resource usage 3. **Stop Unused Nodes**: Stop nodes when not needed to save resources ## Troubleshooting ### Common Issues #### Node Won't Start ```bash # Check Docker is running docker ps # Check port conflicts netstat -tulpn | grep :2528 # Clean up and retry merobox nuke merobox run ``` #### Health Check Fails ```bash # Check node logs merobox logs calimero-node-1 # Check specific node health merobox health --node calimero-node-1 --verbose # Restart node merobox stop calimero-node-1 merobox run --count 1 ``` #### Authentication Issues ```bash # Check auth service status docker ps | grep -E "(proxy|auth)" # Restart auth service merobox stop --auth-service merobox run --auth-service ``` ### Debug Commands ```bash # Enable debug logging LOG_LEVEL=DEBUG merobox run # Verbose health check merobox health --verbose # Check Docker containers docker ps -a | grep calimero # Check Docker logs docker logs calimero-node-1 ``` ## Next Steps Now that you understand node management: - [Workflow Configuration](./workflows) - Create and run complex workflows - [Examples and Tutorials](./examples) - Practical examples and tutorials - [Advanced Configuration](./advanced-configuration) - Advanced setup options - [Troubleshooting](./troubleshooting) - Common issues and solutions --- ## Python Customization # Python Customization This guide covers basic Python integration and testing capabilities for Merobox CLI. ## Overview Merobox provides basic Python integration for testing and automation: - **Testing Integration**: Use Merobox in your Python test suites - **Basic Automation**: Automate Merobox workflows with Python scripts - **Environment Management**: Manage test environments programmatically ## Basic Python Integration ### Using Merobox in Python Scripts ```python # basic_integration.py import subprocess import json def run_merobox_command(command): """Run a Merobox command and return the result.""" try: result = subprocess.run( ['merobox'] + command, capture_output=True, text=True, check=True ) return result.stdout except subprocess.CalledProcessError as e: print(f"Error running merobox command: {e}") return None # Example usage def start_nodes(count=2): """Start Merobox nodes.""" output = run_merobox_command(['run', '--count', str(count)]) print(f"Started {count} nodes: {output}") def check_health(): """Check node health.""" output = run_merobox_command(['health']) print(f"Health status: {output}") def stop_nodes(): """Stop all nodes.""" output = run_merobox_command(['stop', '--all']) print(f"Stopped nodes: {output}") ``` ## Testing Integration ### Basic Testing Setup ```python # test_merobox.py import pytest import subprocess import time class MeroboxTestHelper: def __init__(self): self.nodes = [] def start_nodes(self, count=2): """Start Merobox nodes for testing.""" result = subprocess.run( ['merobox', 'run', '--count', str(count)], capture_output=True, text=True ) if result.returncode == 0: self.nodes = [f"calimero-node-{i+1}" for i in range(count)] return True return False def stop_nodes(self): """Stop all test nodes.""" subprocess.run(['merobox', 'stop', '--all']) self.nodes = [] def check_health(self): """Check health of all nodes.""" result = subprocess.run( ['merobox', 'health'], capture_output=True, text=True ) return result.returncode == 0 # Test fixture @pytest.fixture def merobox_helper(): helper = MeroboxTestHelper() yield helper helper.stop_nodes() def test_basic_functionality(merobox_helper): """Test basic Merobox functionality.""" assert merobox_helper.start_nodes(2) assert merobox_helper.check_health() merobox_helper.stop_nodes() ``` ## Workflow Automation ### Running Workflows from Python ```python # workflow_automation.py import subprocess import yaml import tempfile import os def create_workflow_file(workflow_config): """Create a temporary workflow file.""" with tempfile.NamedTemporaryFile(mode='w', suffix='.yml', delete=False) as f: yaml.dump(workflow_config, f) return f.name def run_workflow(workflow_config): """Run a Merobox workflow from Python.""" workflow_file = create_workflow_file(workflow_config) try: result = subprocess.run( ['merobox', 'bootstrap', 'run', workflow_file], capture_output=True, text=True, check=True ) return result.stdout except subprocess.CalledProcessError as e: print(f"Workflow failed: {e}") return None finally: os.unlink(workflow_file) # Example workflow configuration def create_test_workflow(): """Create a simple test workflow.""" return { 'description': 'Test workflow from Python', 'name': 'Python Test Workflow', 'nodes': { 'chain_id': 'testnet-1', 'count': 1, 'image': 'ghcr.io/calimero-network/merod:edge' }, 'steps': [ { 'name': 'Test Step', 'type': 'script', 'script': 'echo "Hello from Python workflow!"' } ], 'stop_all_nodes': True } # Run the workflow workflow_config = create_test_workflow() result = run_workflow(workflow_config) print(f"Workflow result: {result}") ``` ## Best Practices ### Error Handling ```python # Always handle errors properly def safe_merobox_command(command): """Run Merobox command with proper error handling.""" try: result = subprocess.run( ['merobox'] + command, capture_output=True, text=True, check=True ) return result.stdout except subprocess.CalledProcessError as e: print(f"Command failed: {e}") print(f"Error output: {e.stderr}") return None except FileNotFoundError: print("Merobox not found. Please install it first.") return None ``` ### Resource Cleanup ```python # Always clean up resources def test_with_cleanup(): """Test with proper cleanup.""" try: # Start nodes subprocess.run(['merobox', 'run', '--count', '2']) # Run tests result = subprocess.run(['merobox', 'health']) assert result.returncode == 0 finally: # Always cleanup subprocess.run(['merobox', 'stop', '--all']) ``` ## Next Steps Now that you understand basic Python integration with Merobox: - [Node Management](./node-management) - Complete node management guide - [Workflows](./workflows) - Workflow system and automation - [Environment Variables](./environment-variables) - Configuration options - [Troubleshooting](./troubleshooting) - Common issues and solutions --- ## Quick Start Tutorials # Quick Start Tutorials These tutorials will help you get up and running with Merobox quickly, from your first workflow to deploying applications. ## Tutorial 1: Your First Merobox Workflow This tutorial walks you through creating and running your first Merobox workflow. ### Step 1: Install Merobox ```bash pip install merobox ``` ### Step 2: Create a Simple Workflow Create a file called `my-first-workflow.yml`: ```yaml description: My first Merobox workflow name: First Workflow nodes: chain_id: testnet-1 count: 1 image: ghcr.io/calimero-network/merod:edge prefix: calimero-node steps: - name: Wait for Node Startup type: wait seconds: 5 - name: Check Node Health type: script script: | echo "Node is running!" echo "Checking health..." stop_all_nodes: true ``` ### Step 3: Run the Workflow ```bash merobox bootstrap run my-first-workflow.yml ``` ### Step 4: Verify Results ```bash # Check if nodes are running merobox list # Check node health merobox health ``` ## Tutorial 2: Application Deployment Deploy a simple WASM application using Merobox. ### Step 1: Prepare Your Application Create a simple WASM file or use the provided example: ```bash # Download example WASM file curl -o kv-store.wasm https://example.com/kv-store.wasm ``` ### Step 2: Create Deployment Workflow Create `deploy-app.yml`: ```yaml description: Deploy and test a WASM application name: Application Deployment nodes: chain_id: testnet-1 count: 1 image: ghcr.io/calimero-network/merod:edge prefix: calimero-node steps: - name: Install Application type: install_application node: calimero-node-1 path: ./kv-store.wasm dev: true outputs: app_id: applicationId - name: Create Context type: create_context node: calimero-node-1 application_id: '{{app_id}}' outputs: context_id: contextId member_key: memberPublicKey - name: Test Application type: call node: calimero-node-1 context_id: '{{context_id}}' executor_public_key: '{{member_key}}' method: set args: key: hello value: world outputs: set_result: result - name: Verify Data type: call node: calimero-node-1 context_id: '{{context_id}}' executor_public_key: '{{member_key}}' method: get args: key: hello outputs: get_result: result - name: Validate Results type: assert statements: - "contains({{get_result}}, 'world')" stop_all_nodes: true ``` ### Step 3: Run the Deployment ```bash merobox bootstrap run deploy-app.yml ``` ## Tutorial 3: Multi-Node Setup Learn how to set up and work with multiple nodes. ### Step 1: Create Multi-Node Workflow Create `multi-node.yml`: ```yaml description: Multi-node setup tutorial name: Multi-Node Tutorial nodes: chain_id: testnet-1 count: 2 image: ghcr.io/calimero-network/merod:edge prefix: tutorial-node steps: - name: Wait for All Nodes type: wait seconds: 10 message: 'Waiting for all nodes to start...' - name: Check Node 1 Health type: script script: | echo "Checking node 1 health..." curl -f http://tutorial-node-1:2428/health || echo "Node 1 not ready yet" - name: Check Node 2 Health type: script script: | echo "Checking node 2 health..." curl -f http://tutorial-node-2:2428/health || echo "Node 2 not ready yet" - name: List All Nodes type: script script: | echo "All nodes are running:" merobox list stop_all_nodes: true ``` ### Step 2: Run Multi-Node Workflow ```bash merobox bootstrap run multi-node.yml ``` ## Tutorial 4: Authentication Service Set up Merobox with authentication service for production-like testing. ### Step 1: Create Auth Service Workflow Create `auth-tutorial.yml`: ```yaml description: Authentication service tutorial name: Auth Service Tutorial # Enable authentication service auth_service: true nodes: chain_id: testnet-1 count: 1 image: ghcr.io/calimero-network/merod:edge prefix: auth-tutorial steps: - name: Wait for Auth Service type: wait seconds: 15 message: 'Waiting for authentication service to start...' - name: Check Auth Service type: script script: | echo "Authentication service is running!" echo "Node URL: http://auth-tutorial-1.127.0.0.1.nip.io" echo "Auth Login: http://auth-tutorial-1.127.0.0.1.nip.io/auth/login" echo "Admin Dashboard: http://auth-tutorial-1.127.0.0.1.nip.io/admin-dashboard" stop_all_nodes: true ``` ### Step 2: Run Auth Service Tutorial ```bash merobox bootstrap run auth-tutorial.yml ``` ## Tutorial 5: Custom Scripts Learn how to use custom scripts in your workflows. ### Step 1: Create Script Tutorial Create `script-tutorial.yml`: ```yaml description: Custom scripts tutorial name: Script Tutorial nodes: chain_id: testnet-1 count: 1 image: ghcr.io/calimero-network/merod:edge prefix: script-tutorial steps: - name: Pre-Setup Script type: script script: | echo "Setting up test environment..." mkdir -p /tmp/merobox-tutorial echo "Merobox Tutorial" > /tmp/merobox-tutorial/readme.txt echo "Pre-setup complete" - name: Main Script type: script script: | echo "Running main tutorial script..." echo "Current directory: $(pwd)" echo "Available files:" ls -la /tmp/merobox-tutorial/ echo "Script execution complete" - name: Cleanup Script type: script script: | echo "Cleaning up..." rm -rf /tmp/merobox-tutorial echo "Cleanup complete" stop_all_nodes: true ``` ### Step 2: Run Script Tutorial ```bash merobox bootstrap run script-tutorial.yml ``` ## Common Issues and Solutions ### Issue: Node Not Starting **Problem**: Node fails to start or takes too long. **Solution**: ```yaml # Add longer wait times steps: - name: Wait for Node type: wait seconds: 30 # Increase wait time message: 'Waiting for node to start...' ``` ### Issue: Application Installation Fails **Problem**: Application installation fails with errors. **Solution**: ```yaml # Use dev mode for local development steps: - name: Install Application type: install_application node: calimero-node-1 path: ./my-app.wasm dev: true # Enable dev mode ``` ### Issue: Context Creation Fails **Problem**: Context creation fails or times out. **Solution**: ```yaml # Add validation and error handling steps: - name: Create Context type: create_context node: calimero-node-1 application_id: '{{app_id}}' outputs: context_id: contextId member_key: memberPublicKey - name: Validate Context type: assert statements: - '{{context_id}} != null' - '{{member_key}} != null' ``` ## Next Steps Now that you've completed the quick start tutorials: - [Real-World Examples](./real-world-examples) - Practical examples for common scenarios - [Testing Integration Examples](./testing-integration-examples) - Integration with testing frameworks - [Advanced Examples](./advanced-examples) - Complex workflows and advanced features - [Best Practices](./best-practices) - Guidelines for effective Merobox usage --- ## Real-World Examples # Real-World Examples These examples demonstrate how to use Merobox for common real-world scenarios, from multi-node testing to basic performance testing. ## Example 1: Multi-Node Testing Test your application across multiple nodes to simulate real-world scenarios. ```yaml description: Multi-node application testing name: Multi-Node Test nodes: chain_id: testnet-1 count: 3 image: ghcr.io/calimero-network/merod:edge prefix: test-node steps: # Install application on first node - name: Install Application type: install_application node: test-node-1 path: ./my-app.wasm dev: true outputs: app_id: applicationId # Create context on first node - name: Create Context type: create_context node: test-node-1 application_id: '{{app_id}}' outputs: context_id: contextId member_key: memberPublicKey # Create identities on other nodes - name: Create Identity on Node 2 type: create_identity node: test-node-2 outputs: node2_key: publicKey - name: Create Identity on Node 3 type: create_identity node: test-node-3 outputs: node3_key: publicKey # Invite nodes to context - name: Invite Node 2 type: invite_identity node: test-node-1 context_id: '{{context_id}}' grantee_id: '{{node2_key}}' granter_id: '{{member_key}}' capability: member outputs: invite2: invitation - name: Invite Node 3 type: invite_identity node: test-node-1 context_id: '{{context_id}}' grantee_id: '{{node3_key}}' granter_id: '{{member_key}}' capability: member outputs: invite3: invitation # Join contexts - name: Join from Node 2 type: join_context node: test-node-2 context_id: '{{context_id}}' invitee_id: '{{node2_key}}' invitation: '{{invite2}}' - name: Join from Node 3 type: join_context node: test-node-3 context_id: '{{context_id}}' invitee_id: '{{node3_key}}' invitation: '{{invite3}}' # Test cross-node communication - name: Set Data from Node 1 type: call node: test-node-1 context_id: '{{context_id}}' executor_public_key: '{{member_key}}' method: set args: key: shared_data value: 'set from node 1' - name: Read from Node 2 type: call node: test-node-2 context_id: '{{context_id}}' executor_public_key: '{{node2_key}}' method: get args: key: shared_data outputs: result2: result - name: Read from Node 3 type: call node: test-node-3 context_id: '{{context_id}}' executor_public_key: '{{node3_key}}' method: get args: key: shared_data outputs: result3: result # Validate cross-node consistency - name: Validate Cross-Node Data type: assert statements: - "contains({{result2}}, 'set from node 1')" - "contains({{result3}}, 'set from node 1')" - '{{result2}} == {{result3}}' stop_all_nodes: true ``` ## Example 2: Authentication Service Integration Use Merobox with authentication services for production-like testing. ```yaml description: Workflow with authentication service name: Auth Service Example # Enable authentication service auth_service: true nodes: chain_id: testnet-1 count: 2 image: ghcr.io/calimero-network/merod:edge prefix: auth-node steps: - name: Wait for Auth Service type: wait seconds: 10 message: 'Waiting for authentication service to start...' - name: Install Application type: install_application node: auth-node-1 path: ./my-app.wasm dev: true outputs: app_id: applicationId - name: Create Context type: create_context node: auth-node-1 application_id: '{{app_id}}' outputs: context_id: contextId member_key: memberPublicKey - name: Test with Auth Service type: script script: | echo "Testing with authentication service..." echo "Node 1 URL: http://auth-node-1.127.0.0.1.nip.io" echo "Node 2 URL: http://auth-node-2.127.0.0.1.nip.io" echo "Auth Login: http://auth-node-1.127.0.0.1.nip.io/auth/login" echo "Admin Dashboard: http://auth-node-1.127.0.0.1.nip.io/admin-dashboard" stop_all_nodes: true ``` ## Example 3: Performance Testing Test your application's performance with repeated operations. ```yaml description: Performance testing workflow name: Performance Test nodes: chain_id: testnet-1 count: 1 image: ghcr.io/calimero-network/merod:edge prefix: perf-node steps: - name: Install Application type: install_application node: perf-node-1 path: ./my-app.wasm dev: true outputs: app_id: applicationId - name: Create Context type: create_context node: perf-node-1 application_id: '{{app_id}}' outputs: context_id: contextId member_key: memberPublicKey - name: Performance Test type: repeat count: 100 outputs: iteration: iteration steps: - name: Set Data type: call node: perf-node-1 context_id: '{{context_id}}' executor_public_key: '{{member_key}}' method: set args: key: 'perf_key_{{iteration}}' value: 'perf_value_{{iteration}}' - name: Get Data type: call node: perf-node-1 context_id: '{{context_id}}' executor_public_key: '{{member_key}}' method: get args: key: 'perf_key_{{iteration}}' outputs: result: result - name: Validate Performance type: assert statements: - "contains({{result}}, 'perf_value_{{iteration}}')" - name: Performance Summary type: script script: | echo "Performance test completed!" echo "Executed 100 set/get operations successfully" stop_all_nodes: true ``` ## Example 4: Basic Error Handling Implement basic error handling in your workflows. ```yaml description: Error handling example name: Error Handling Test nodes: chain_id: testnet-1 count: 1 image: ghcr.io/calimero-network/merod:edge prefix: error-node steps: - name: Install Application type: install_application node: error-node-1 path: ./my-app.wasm dev: true outputs: app_id: applicationId - name: Create Context type: create_context node: error-node-1 application_id: '{{app_id}}' outputs: context_id: contextId member_key: memberPublicKey - name: Test Error Handling type: call node: error-node-1 context_id: '{{context_id}}' executor_public_key: '{{member_key}}' method: test_error_handling args: should_fail: true outputs: error_result: result - name: Validate Error Response type: assert statements: - "contains({{error_result}}, 'error')" - name: Recovery Test type: call node: error-node-1 context_id: '{{context_id}}' executor_public_key: '{{member_key}}' method: test_recovery args: should_succeed: true outputs: recovery_result: result - name: Validate Recovery type: assert statements: - "contains({{recovery_result}}, 'success')" stop_all_nodes: true ``` ## Example 5: Parallel Operations Execute multiple operations in parallel for improved performance. ```yaml description: Parallel operations example name: Parallel Operations nodes: chain_id: testnet-1 count: 3 image: ghcr.io/calimero-network/merod:edge prefix: parallel-node steps: - name: Install Applications in Parallel type: parallel steps: - name: Install App 1 type: install_application node: parallel-node-1 path: ./app1.wasm dev: true outputs: app_id_1: applicationId - name: Install App 2 type: install_application node: parallel-node-2 path: ./app2.wasm dev: true outputs: app_id_2: applicationId - name: Install App 3 type: install_application node: parallel-node-3 path: ./app3.wasm dev: true outputs: app_id_3: applicationId - name: Create Contexts in Parallel type: parallel steps: - name: Create Context 1 type: create_context node: parallel-node-1 application_id: '{{app_id_1}}' outputs: context_id_1: contextId member_key_1: memberPublicKey - name: Create Context 2 type: create_context node: parallel-node-2 application_id: '{{app_id_2}}' outputs: context_id_2: contextId member_key_2: memberPublicKey - name: Create Context 3 type: create_context node: parallel-node-3 application_id: '{{app_id_3}}' outputs: context_id_3: contextId member_key_3: memberPublicKey - name: Test All Applications type: parallel steps: - name: Test App 1 type: call node: parallel-node-1 context_id: '{{context_id_1}}' executor_public_key: '{{member_key_1}}' method: test_functionality args: test_data: 'app1_test' - name: Test App 2 type: call node: parallel-node-2 context_id: '{{context_id_2}}' executor_public_key: '{{member_key_2}}' method: test_functionality args: test_data: 'app2_test' - name: Test App 3 type: call node: parallel-node-3 context_id: '{{context_id_3}}' executor_public_key: '{{member_key_3}}' method: test_functionality args: test_data: 'app3_test' stop_all_nodes: true ``` ## Best Practices for Real-World Examples ### 1. Environment Configuration ```yaml # Use environment-specific settings nodes: image: ghcr.io/calimero-network/merod:edge environment: RUST_LOG: info ``` ### 2. Error Handling ```yaml # Include basic error handling steps: - name: Risky Operation type: call node: calimero-node-1 method: risky_method retry: attempts: 3 delay: 5 on_error: - name: Log Error type: script script: echo "Operation failed: {{error}}" ``` ### 3. Validation ```yaml # Always validate results steps: - name: Validate Results type: assert statements: - '{{result}} != null' - "contains({{result}}, 'expected_value')" ``` ## Next Steps Now that you've seen real-world examples: - [Testing Integration Examples](./testing-integration-examples) - Integration with testing frameworks - [Advanced Examples](./advanced-examples) - Complex workflows and advanced features - [Best Practices](./best-practices) - Guidelines for effective Merobox usage --- ## Resource Management # Resource Management This guide covers basic resource configuration for Merobox nodes, including memory and CPU limits for Docker containers. ## Basic Resource Configuration Merobox supports basic resource limits for Docker containers running Calimero nodes: ### Memory and CPU Limits ```yaml nodes: count: 2 image: ghcr.io/calimero-network/merod:edge resources: memory: '1G' # Memory limit cpus: '0.5' # CPU limit ``` ### Environment Variables Configure resource-related environment variables: ```yaml nodes: count: 2 image: ghcr.io/calimero-network/merod:edge environment: RUST_LOG: 'info' resources: memory: '2G' cpus: '1.0' ``` ## Node Configuration ### Basic Node Setup Configure Calimero nodes with basic settings: ```yaml nodes: chain_id: testnet-1 count: 2 image: ghcr.io/calimero-network/merod:edge prefix: calimero-node base_port: 2428 base_rpc_port: 2528 resources: memory: '1G' cpus: '0.5' ``` ### Multiple Node Configuration Set up multiple nodes with different resource allocations: ```yaml nodes: chain_id: testnet-1 count: 3 image: ghcr.io/calimero-network/merod:edge prefix: calimero-node resources: memory: '2G' cpus: '1.0' ``` ## Environment Variables Configure Merobox behavior using environment variables: ### Core Configuration ```bash # Docker daemon connection export DOCKER_HOST="unix:///var/run/docker.sock" # Logging level (if supported) export LOG_LEVEL="INFO" # DEBUG, INFO, WARNING, ERROR ``` ### Docker Configuration ```bash # Docker daemon settings export DOCKER_HOST="unix:///var/run/docker.sock" # Docker build settings export DOCKER_BUILDKIT="1" export DOCKER_CLI_EXPERIMENTAL="enabled" ``` ## Authentication Service Configure authentication service for Merobox nodes: ### Basic Auth Service ```yaml # Enable authentication service auth_service: true auth_image: ghcr.io/calimero-network/mero-auth:edge nodes: count: 2 image: ghcr.io/calimero-network/merod:edge resources: memory: '1G' cpus: '0.5' ``` ### Auth Service Environment Variables ```bash # Auth service is configured through CLI flags # merobox run --auth-service --auth-image ghcr.io/calimero-network/mero-auth:edge ``` ## Troubleshooting ### Common Resource Issues Check resource usage and node status: ```bash # Check running Merobox nodes merobox list # Check node health merobox health # View node logs merobox logs calimero-node-1 # Check Docker container resource usage docker stats $(docker ps -q --filter "name=calimero-") ``` ### Resource Debugging Enable debug logging for troubleshooting: ```bash # Enable debug logging export LOG_LEVEL=DEBUG # Run workflow with verbose output merobox bootstrap run workflow.yml --verbose # Check Docker container details docker inspect calimero-node-1 ``` ## Best Practices ### Resource Planning 1. **Start Small**: Begin with minimal resources and scale up as needed 2. **Monitor Usage**: Use `docker stats` to monitor actual resource consumption 3. **Test Workloads**: Test your applications with realistic data loads 4. **Plan for Growth**: Consider future scaling requirements ### Performance Tips 1. **Memory Allocation**: Start with 1-2GB memory per node for most workloads 2. **CPU Allocation**: Use 0.5-1.0 CPU cores per node for basic operations 3. **Environment Variables**: Use appropriate `RUST_LOG` levels for debugging 4. **Docker Images**: Use specific image tags instead of `latest` for stability ### Troubleshooting Tips 1. **Check Logs**: Always check node logs first when issues occur 2. **Resource Limits**: Ensure Docker has sufficient resources available 3. **Network Issues**: Verify port availability and network connectivity 4. **Image Updates**: Keep Docker images updated for latest features ## Next Steps Now that you understand basic resource management: - [Node Management](./node-management) - Complete node management guide - [Workflows](./workflows) - Workflow system and automation - [Environment Variables](./environment-variables) - Configuration options - [Troubleshooting](./troubleshooting) - Common issues and solutions --- ## Security Configuration # Security Configuration This guide covers basic security configuration for Merobox deployments and best practices for secure usage. ## Basic Security Settings Merobox provides basic security configuration through environment variables and Docker settings: ### Environment Variables ```bash # Basic security environment variables export RUST_LOG="info" export DOCKER_HOST="unix:///var/run/docker.sock" ``` ### Docker Security Basic Docker security practices for Merobox: ```bash # Run containers as non-root user when possible docker run --user 1000:1000 ghcr.io/calimero-network/merod:edge # Use read-only filesystem for additional security docker run --read-only ghcr.io/calimero-network/merod:edge # Drop unnecessary capabilities docker run --cap-drop=ALL --cap-add=NET_BIND_SERVICE ghcr.io/calimero-network/merod:edge ``` ## Network Security Basic network security practices for Merobox: ### Port Management ```bash # Check which ports are in use netstat -tulpn | grep -E "(2428|2528)" # Use different ports if conflicts exist merobox run --base-port 3000 --base-rpc-port 3001 ``` ### Network Isolation ```bash # Create isolated Docker network docker network create calimero-isolated # Run nodes on isolated network docker run --network calimero-isolated ghcr.io/calimero-network/merod:edge ``` ## Secrets Management Basic secrets management for Merobox: ### Environment Variables ```bash # Use environment variables for sensitive data (if supported) export API_KEY="your-api-key" export SECRET_KEY="your-secret-key" # Pass to Merobox (if supported) merobox run --count 1 ``` ### File-based Secrets ```bash # Store secrets in files with proper permissions echo "your-secret" > secret.txt chmod 600 secret.txt # Mount secrets as volumes docker run -v $(pwd)/secret.txt:/app/secret.txt ghcr.io/calimero-network/merod:edge ``` ## Authentication Basic authentication practices for Merobox: ### Auth Service Integration ```bash # Enable authentication service merobox run --auth-service # Access nodes through auth service # Node URLs will be available at: http://node-name.127.0.0.1.nip.io ``` ### Basic Access Control ```bash # Use different node prefixes for different users merobox run --prefix user1-nodes merobox run --prefix user2-nodes # Stop nodes when not needed merobox stop --all ``` ## Basic Security Best Practices ### 1. Use Non-Root Users ```bash # Run containers as non-root user docker run --user 1000:1000 ghcr.io/calimero-network/merod:edge ``` ### 2. Read-Only Filesystems ```bash # Use read-only filesystem when possible docker run --read-only ghcr.io/calimero-network/merod:edge ``` ### 3. Drop Unnecessary Capabilities ```bash # Drop all capabilities and add only what's needed docker run --cap-drop=ALL --cap-add=NET_BIND_SERVICE ghcr.io/calimero-network/merod:edge ``` ### 4. Resource Limits ```bash # Set memory and CPU limits docker run --memory=1g --cpus=0.5 ghcr.io/calimero-network/merod:edge ``` ## Troubleshooting Security Issues ### Common Security Issues ```bash # Check container security settings docker inspect calimero-node-1 | jq '.[0].HostConfig.SecurityOpt' # Check user permissions docker exec calimero-node-1 id # Check file permissions docker exec calimero-node-1 ls -la /calimero/ # Check network connectivity docker exec calimero-node-1 netstat -tulpn ``` ### Security Debugging ```bash # Enable security debugging export RUST_LOG=debug # Check security logs docker logs calimero-node-1 | grep -i security # Test authentication curl -H "Authorization: Bearer " http://localhost:2528/health ``` ## Security Best Practices ### General Security 1. **Principle of Least Privilege**: Grant minimum necessary permissions 2. **Regular Updates**: Keep all components updated 3. **Monitoring**: Monitor for security events 4. **Documentation**: Document security procedures ### Container Security 1. **Non-root Users**: Run containers as non-root users 2. **Read-only Filesystems**: Use read-only filesystems where possible 3. **Capability Dropping**: Drop unnecessary capabilities 4. **Resource Limits**: Set appropriate resource limits ### Network Security 1. **Port Management**: Use appropriate ports and check for conflicts 2. **Network Isolation**: Use isolated Docker networks 3. **Access Control**: Control network access appropriately ### Data Protection 1. **Environment Variables**: Use environment variables for sensitive data 2. **File Permissions**: Set proper file permissions for secrets 3. **Cleanup**: Clean up sensitive data when done ## Next Steps Now that you understand basic security configuration: - [Node Management](./node-management) - Complete node management guide - [Workflows](./workflows) - Workflow system and automation - [Environment Variables](./environment-variables) - Configuration options - [Troubleshooting](./troubleshooting) - Common issues and solutions --- ## Testing Framework Integration # Testing Framework Integration This guide covers basic testing configuration for Merobox and integration with Python testing frameworks. ## Basic Testing Setup Configure Merobox for testing scenarios: ### Basic Testing Setup ```python # conftest.py import pytest import subprocess import time @pytest.fixture(scope="session") def merobox_cluster(): """Basic Merobox cluster for testing.""" # Start nodes subprocess.run(['merobox', 'run', '--count', '2']) time.sleep(10) # Wait for nodes to start yield # Cleanup subprocess.run(['merobox', 'stop', '--all']) @pytest.fixture(scope="function") def test_environment(): """Test environment for each test.""" # Start fresh nodes for each test subprocess.run(['merobox', 'run', '--count', '1']) time.sleep(5) yield # Cleanup subprocess.run(['merobox', 'stop', '--all']) ``` ### Basic Test Examples ```python # test_basic.py import pytest import subprocess def test_merobox_health(merobox_cluster): """Test that Merobox nodes are healthy.""" result = subprocess.run(['merobox', 'health'], capture_output=True, text=True) assert result.returncode == 0 assert "healthy" in result.stdout.lower() def test_merobox_list(merobox_cluster): """Test that we can list running nodes.""" result = subprocess.run(['merobox', 'list'], capture_output=True, text=True) assert result.returncode == 0 assert "calimero-node" in result.stdout def test_workflow_execution(test_environment): """Test basic workflow execution.""" # Create a simple workflow workflow_content = """ description: Test workflow name: Test Workflow nodes: chain_id: testnet-1 count: 1 image: ghcr.io/calimero-network/merod:edge steps: - name: Test Step type: script script: echo "Hello from test workflow" stop_all_nodes: true """ with open("test_workflow.yml", "w") as f: f.write(workflow_content) try: result = subprocess.run( ['merobox', 'bootstrap', 'run', 'test_workflow.yml'], capture_output=True, text=True ) assert result.returncode == 0 finally: import os os.remove("test_workflow.yml") ``` ## Test Helper Classes ### Basic Test Helper ```python # test_helpers.py import subprocess import time import tempfile import os class MeroboxTestHelper: def __init__(self): self.nodes = [] self.workflow_files = [] def start_nodes(self, count=2): """Start Merobox nodes for testing.""" result = subprocess.run( ['merobox', 'run', '--count', str(count)], capture_output=True, text=True ) if result.returncode == 0: self.nodes = [f"calimero-node-{i+1}" for i in range(count)] time.sleep(10) # Wait for nodes to start return True return False def stop_nodes(self): """Stop all test nodes.""" subprocess.run(['merobox', 'stop', '--all']) self.nodes = [] def check_health(self): """Check health of all nodes.""" result = subprocess.run( ['merobox', 'health'], capture_output=True, text=True ) return result.returncode == 0 def create_workflow(self, workflow_config): """Create a temporary workflow file.""" with tempfile.NamedTemporaryFile(mode='w', suffix='.yml', delete=False) as f: import yaml yaml.dump(workflow_config, f) self.workflow_files.append(f.name) return f.name def run_workflow(self, workflow_file): """Run a Merobox workflow.""" result = subprocess.run( ['merobox', 'bootstrap', 'run', workflow_file], capture_output=True, text=True ) return result def cleanup(self): """Clean up all resources.""" self.stop_nodes() for file in self.workflow_files: if os.path.exists(file): os.unlink(file) self.workflow_files = [] # Usage in tests @pytest.fixture def merobox_helper(): helper = MeroboxTestHelper() yield helper helper.cleanup() def test_with_helper(merobox_helper): """Test using the helper class.""" assert merobox_helper.start_nodes(2) assert merobox_helper.check_health() # Create and run a workflow workflow_config = { 'description': 'Test workflow', 'name': 'Test Workflow', 'nodes': { 'chain_id': 'testnet-1', 'count': 1, 'image': 'ghcr.io/calimero-network/merod:edge' }, 'steps': [ { 'name': 'Test Step', 'type': 'script', 'script': 'echo "Hello from test"' } ], 'stop_all_nodes': True } workflow_file = merobox_helper.create_workflow(workflow_config) result = merobox_helper.run_workflow(workflow_file) assert result.returncode == 0 ``` ## Best Practices ### Error Handling ```python # Always handle errors properly def safe_merobox_command(command): """Run Merobox command with proper error handling.""" try: result = subprocess.run( ['merobox'] + command, capture_output=True, text=True, check=True ) return result.stdout except subprocess.CalledProcessError as e: print(f"Command failed: {e}") print(f"Error output: {e.stderr}") return None except FileNotFoundError: print("Merobox not found. Please install it first.") return None ``` ### Resource Cleanup ```python # Always clean up resources def test_with_cleanup(): """Test with proper cleanup.""" try: # Start nodes subprocess.run(['merobox', 'run', '--count', '2']) # Run tests result = subprocess.run(['merobox', 'health']) assert result.returncode == 0 finally: # Always cleanup subprocess.run(['merobox', 'stop', '--all']) ``` ### Test Isolation ```python # Use different prefixes for different tests def test_isolated(): """Test with isolated environment.""" try: # Start nodes with unique prefix subprocess.run(['merobox', 'run', '--count', '1', '--prefix', 'test-isolated']) # Run test result = subprocess.run(['merobox', 'health']) assert result.returncode == 0 finally: # Cleanup subprocess.run(['merobox', 'stop', '--all']) ``` ## Running Tests ### Basic Test Execution ```bash # Run all tests pytest # Run specific test file pytest test_merobox.py # Run with verbose output pytest -v # Run with coverage pytest --cov=my_app ``` ### Continuous Integration ```yaml # .github/workflows/test.yml name: Test with Merobox on: [push, pull_request] jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Set up Python uses: actions/setup-python@v4 with: python-version: '3.9' - name: Install dependencies run: | pip install -r requirements.txt pip install merobox - name: Run tests run: | pytest tests/ -v ``` ## Next Steps Now that you understand basic testing integration: - [Node Management](./node-management) - Complete node management guide - [Workflows](./workflows) - Workflow system and automation - [Environment Variables](./environment-variables) - Configuration options - [Troubleshooting](./troubleshooting) - Common issues and solutions --- ## Testing Integration Examples # Testing Integration Examples These examples show how to integrate Merobox with popular testing frameworks using subprocess calls for automated testing. ## Example 1: Python Testing with Merobox Use Merobox in your Python test suite with subprocess calls. ### Basic Test Setup ```python # test_my_app.py import pytest import subprocess import time import tempfile import os class MeroboxTestHelper: def __init__(self): self.nodes = [] self.workflow_files = [] def start_nodes(self, count=2, prefix="test"): """Start Merobox nodes for testing.""" result = subprocess.run( ['merobox', 'run', '--count', str(count), '--prefix', prefix], capture_output=True, text=True ) if result.returncode == 0: self.nodes = [f"{prefix}-{i+1}" for i in range(count)] time.sleep(10) # Wait for nodes to start return True return False def stop_nodes(self): """Stop all test nodes.""" subprocess.run(['merobox', 'stop', '--all']) self.nodes = [] def check_health(self): """Check health of all nodes.""" result = subprocess.run( ['merobox', 'health'], capture_output=True, text=True ) return result.returncode == 0 def create_workflow(self, workflow_config): """Create a temporary workflow file.""" with tempfile.NamedTemporaryFile(mode='w', suffix='.yml', delete=False) as f: import yaml yaml.dump(workflow_config, f) self.workflow_files.append(f.name) return f.name def run_workflow(self, workflow_file): """Run a Merobox workflow.""" result = subprocess.run( ['merobox', 'bootstrap', 'run', workflow_file], capture_output=True, text=True ) return result def cleanup(self): """Clean up all resources.""" self.stop_nodes() for file in self.workflow_files: if os.path.exists(file): os.unlink(file) self.workflow_files = [] @pytest.fixture(scope="function") def merobox_helper(): """Create a Merobox test helper for each test.""" helper = MeroboxTestHelper() yield helper helper.cleanup() def test_basic_functionality(merobox_helper): """Test basic Merobox functionality.""" assert merobox_helper.start_nodes(2) assert merobox_helper.check_health() # Test basic operations result = subprocess.run(['merobox', 'list'], capture_output=True, text=True) assert result.returncode == 0 assert "test-1" in result.stdout def test_workflow_execution(merobox_helper): """Test workflow execution.""" workflow_config = { 'description': 'Test workflow', 'name': 'Test Workflow', 'nodes': { 'chain_id': 'testnet-1', 'count': 1, 'image': 'ghcr.io/calimero-network/merod:edge' }, 'steps': [ { 'name': 'Test Step', 'type': 'script', 'script': 'echo "Hello from test workflow"' } ], 'stop_all_nodes': True } workflow_file = merobox_helper.create_workflow(workflow_config) result = merobox_helper.run_workflow(workflow_file) assert result.returncode == 0 ``` ## Example 2: CI/CD Pipeline Integration Integrate Merobox into CI/CD pipelines. ### GitHub Actions ```yaml # .github/workflows/test.yml name: Test with Merobox on: [push, pull_request] jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Set up Python uses: actions/setup-python@v4 with: python-version: '3.9' - name: Install dependencies run: | pip install -r requirements.txt pip install merobox - name: Start Merobox nodes run: | merobox run --count 2 sleep 10 merobox health - name: Run tests run: | pytest tests/ -v - name: Cleanup if: always() run: | merobox stop --all ``` ### GitLab CI ```yaml # .gitlab-ci.yml stages: - test test: stage: test image: python:3.9 before_script: - pip install -r requirements.txt - pip install merobox script: - merobox run --count 2 - sleep 10 - merobox health - pytest tests/ -v after_script: - merobox stop --all ``` ## Example 3: Docker-based Testing Use Docker for isolated testing environments. ### Dockerfile for Testing ```dockerfile # Dockerfile.test FROM python:3.9 # Install system dependencies RUN apt-get update && apt-get install -y \ docker.io \ && rm -rf /var/lib/apt/lists/* # Install Python dependencies COPY requirements.txt . RUN pip install -r requirements.txt RUN pip install merobox # Copy test files COPY tests/ /app/tests/ COPY . /app/ WORKDIR /app # Run tests CMD ["pytest", "tests/", "-v"] ``` ## Best Practices ### 1. Resource Management ```python # Always clean up resources def test_with_cleanup(): helper = MeroboxTestHelper() try: helper.start_nodes(2) # Run tests assert helper.check_health() finally: helper.cleanup() ``` ### 2. Error Handling ```python # Handle errors gracefully def test_with_error_handling(): helper = MeroboxTestHelper() try: result = helper.start_nodes(2) if not result: pytest.skip("Failed to start nodes") # Continue with test except Exception as e: pytest.fail(f"Test failed: {e}") finally: helper.cleanup() ``` ### 3. Parallel Testing ```python # Use different prefixes for parallel tests def test_parallel_1(): helper = MeroboxTestHelper() helper.start_nodes(1, prefix="test1") # Run test helper.cleanup() def test_parallel_2(): helper = MeroboxTestHelper() helper.start_nodes(1, prefix="test2") # Run test helper.cleanup() ``` ## Next Steps Now that you understand testing integration: - [Best Practices](./best-practices) - Guidelines for effective Merobox usage - [Troubleshooting](./troubleshooting) - Common issues and solutions --- ## Troubleshooting and Best Practices # Troubleshooting and Best Practices This guide helps you resolve common issues and provides best practices for using Merobox effectively in different scenarios. ## Common Issues ### Installation Problems #### Python Not Found ```bash # Error: python3: command not found ``` **Solutions:** - Install Python 3.8+ from [python.org](https://python.org) - On Ubuntu/Debian: `sudo apt install python3 python3-pip` - On macOS: `brew install python3` - On Windows: Download from Microsoft Store or python.org #### Permission Denied ```bash # Error: Permission denied while trying to connect to Docker daemon ``` **Solutions:** - Add user to docker group: `sudo usermod -aG docker $USER` - Log out and log back in - Or run: `newgrp docker` - On Windows/macOS: Ensure Docker Desktop is running #### Module Not Found ```bash # Error: ModuleNotFoundError: No module named 'merobox' ``` **Solutions:** - Install Merobox: `pip install merobox` - Use virtual environment: `python3 -m venv venv && source venv/bin/activate` - Check Python path: `which python3` and `pip3 list | grep merobox` ### Docker Issues #### Docker Not Running ```bash # Error: Cannot connect to the Docker daemon ``` **Solutions:** - Start Docker service: `sudo systemctl start docker` (Linux) - Start Docker Desktop (macOS/Windows) - Check Docker status: `docker info` - Restart Docker: `sudo systemctl restart docker` (Linux) #### Container Creation Fails ```bash # Error: Failed to create container ``` **Solutions:** - Check Docker daemon: `docker info` - Verify image exists: `docker images | grep calimero` - Check disk space: `df -h` - Check Docker logs: `journalctl -u docker.service` (Linux) #### Port Conflicts ```bash # Error: Port 2528 already in use ``` **Solutions:** - Find process using port: `lsof -ti:2528` - Kill process: `lsof -ti:2528 | xargs kill` - Use different ports: `merobox run --base-rpc-port 3000` - Check all ports: `netstat -tulpn | grep -E "(2428|2528)"` ### Node Management Issues #### Node Won't Start ```bash # Error: Failed to start Calimero node ``` **Solutions:** - Check Docker is running: `docker ps` - Verify port availability: `netstat -tulpn | grep :2528` - Check Docker permissions: `docker run hello-world` - Clean up existing containers: `merobox nuke` - Check node logs: `merobox logs calimero-node-1` #### Node Health Check Fails ```bash # Error: Health check failed ``` **Solutions:** - Check node logs: `merobox logs calimero-node-1` - Verify node is ready: `merobox health --verbose` - Restart node: `merobox stop calimero-node-1 && merobox run --count 1` - Check network connectivity: `docker exec calimero-node-1 ping 8.8.8.8` #### Memory Issues ```bash # Error: Container killed due to memory limit ``` **Solutions:** - Increase memory limit: `docker run --memory=2g ...` - Check system memory: `free -h` - Monitor memory usage: `docker stats` - Optimize application memory usage ### Workflow Issues #### Variable Resolution Errors ```yaml # Error: Variable '{{missing_var}}' not found ``` **Solutions:** - Check variable names for typos - Ensure previous steps export the required variables - Use `merobox bootstrap validate workflow.yml` - Check variable naming consistency - Verify step outputs are properly configured #### Step Validation Failures ```yaml # Error: Required field 'node' missing ``` **Solutions:** - Validate workflow: `merobox bootstrap validate workflow.yml` - Check step configuration and required fields - Verify field types and values - Review step documentation for required fields #### API Call Failures ```yaml # Error: API request failed ``` **Solutions:** - Check node health: `merobox health` - Verify nodes are ready before making API calls - Check network connectivity and port availability - Review API endpoint configuration - Check authentication if using auth service ### Authentication Service Issues #### Cannot Access Node via nip.io URL ```bash # Error: ERR_CONNECTION_TIMED_OUT at http://node1.127.0.0.1.nip.io ``` **Solutions:** - Check if auth services are running: `docker ps | grep -E "(proxy|auth)"` - Verify DNS resolution: `nslookup node1.127.0.0.1.nip.io` - Check Traefik dashboard: `http://localhost:8080/dashboard/` - Restart auth services: `merobox stop --auth-service && merobox run --auth-service` #### 404 Errors on Auth URLs ```bash # Error: 404 Not Found at http://node1.127.0.0.1.nip.io/auth/login ``` **Solutions:** - Verify auth container is running: `docker logs auth` - Check Traefik routing: `curl http://localhost:8080/api/http/routers` - Restart the node: `merobox stop node-name && merobox run --auth-service` - Check auth service configuration #### Network Connection Problems ```bash # Error: Could not connect to auth networks ``` **Solutions:** - Check Docker networks: `docker network ls | grep calimero` - Recreate networks: `merobox stop --all && merobox run --auth-service` - Check Docker daemon: `docker system info` - Restart Docker networking ## Debugging Techniques ### Enable Debug Logging ```bash # Set debug log level export LOG_LEVEL=DEBUG # Run with verbose output merobox bootstrap run workflow.yml --verbose # Check specific command merobox health --verbose ``` ### Docker Debugging ```bash # Check Docker system info docker system info # Check Docker networks docker network ls docker network inspect calimero-web # Check container logs docker logs calimero-node-1 docker logs auth docker logs proxy # Inspect containers docker inspect calimero-node-1 docker exec -it calimero-node-1 /bin/sh ``` ### Network Diagnostics ```bash # Check port binding netstat -tulpn | grep -E "(2428|2528)" # Test connectivity docker exec calimero-node-1 ping calimero-node-2 docker exec calimero-node-1 curl http://localhost:2528/admin-api/health # Check DNS resolution nslookup node1.127.0.0.1.nip.io dig node1.127.0.0.1.nip.io ``` ### Workflow Debugging ```bash # Validate workflow before running merobox bootstrap validate workflow.yml # Run with step-by-step output merobox bootstrap run workflow.yml --verbose # Check specific step merobox bootstrap run workflow.yml --step "Install Application" ``` ## Performance Issues ### Slow Workflow Execution **Symptoms:** - Workflows taking longer than expected - Timeouts on API calls - High resource usage **Solutions:** - Check node resources: `docker stats` - Monitor system resources: `htop`, `iotop` - Optimize workflow steps - Use appropriate wait times - Check network latency - Review Docker resource limits ### High Memory Usage **Symptoms:** - Container using excessive memory - System running out of memory - OOM (Out of Memory) errors **Solutions:** - Check memory limits: `docker stats` - Monitor memory usage: `free -h` - Restart nodes if needed - Check for memory leaks - Optimize application memory usage - Increase system memory if needed ### Network Performance **Symptoms:** - Slow API responses - Timeout errors - Connection failures **Solutions:** - Check network latency: `ping` - Monitor network usage: `iftop`, `nethogs` - Check Docker network configuration - Review firewall settings - Optimize network topology ## Best Practices ### Development Workflow 1. **Start Simple**: Begin with basic workflows and gradually add complexity 2. **Use Version Control**: Track workflow files and configuration changes 3. **Test Incrementally**: Test each step before adding the next 4. **Clean Up**: Use `merobox nuke` to clean up between development sessions 5. **Monitor Resources**: Keep an eye on Docker resource usage ### Production Deployment 1. **Resource Planning**: Allocate appropriate resources for your workload 2. **Security**: Implement proper security measures and access controls 3. **Monitoring**: Set up comprehensive monitoring and alerting 4. **Backup**: Implement regular backup and recovery procedures 5. **Documentation**: Maintain detailed documentation of your configuration ### Testing Strategy 1. **Isolated Tests**: Use separate node prefixes for different test suites 2. **Resource Cleanup**: Always clean up resources after tests 3. **Parallel Testing**: Use different prefixes for parallel test execution 4. **Environment Setup**: Use workflows for complex test environment setup 5. **Assertions**: Include comprehensive assertions in your workflows ### Workflow Design 1. **Modular Steps**: Break complex operations into smaller, focused steps 2. **Clear Naming**: Use descriptive names for steps and variables 3. **Error Handling**: Include validation and error checking steps 4. **Documentation**: Add descriptions to explain workflow purpose 5. **Reusability**: Design workflows to be reusable across different scenarios ### Resource Management 1. **Clean Up**: Regularly use `merobox nuke` to free disk space 2. **Monitor Resources**: Use `docker stats` to monitor resource usage 3. **Stop Unused Nodes**: Stop nodes when not needed to save resources 4. **Optimize Images**: Use appropriate Docker images for your use case 5. **Scale Appropriately**: Use the right number of nodes for your workload ## Performance Optimization ### Workflow Optimization 1. **Minimal Waits**: Use appropriate wait times, not excessive delays 2. **Parallel Operations**: Design workflows to minimize sequential dependencies 3. **Efficient Steps**: Use the most efficient step types for your use case 4. **Resource Reuse**: Reuse nodes and contexts where possible 5. **Batch Operations**: Group related operations together ### Docker Optimization 1. **Image Management**: Use appropriate base images and layers 2. **Resource Limits**: Set appropriate resource limits for containers 3. **Network Optimization**: Use efficient network configurations 4. **Storage Optimization**: Use appropriate storage drivers and volumes 5. **Cleanup**: Regularly clean up unused images and containers ### System Optimization 1. **Resource Allocation**: Allocate appropriate system resources 2. **Network Configuration**: Optimize network settings for your use case 3. **Storage Performance**: Use fast storage for Docker data 4. **Memory Management**: Optimize memory usage and swap configuration 5. **CPU Optimization**: Use appropriate CPU allocation and scheduling ## Monitoring and Alerting ### Health Monitoring ```bash # Regular health checks merobox health --verbose # Monitor resource usage docker stats # Check system resources htop iotop ``` ### Log Monitoring ```bash # Monitor node logs merobox logs calimero-node-1 --follow # Check Docker logs docker logs calimero-node-1 # Monitor system logs journalctl -u docker.service -f ``` ### Alerting Setup ```yaml # monitoring.yml monitoring: enabled: true metrics: - cpu_usage - memory_usage - disk_usage - network_io alerts: - metric: memory_usage threshold: 80 action: restart_node - metric: cpu_usage threshold: 90 action: scale_up ``` ## Getting Help ### Self-Help Resources 1. **Documentation**: Review relevant sections of this documentation 2. **Command Help**: Use `merobox --help` or `merobox --help` 3. **Validate Workflows**: Use `merobox bootstrap validate` to check configuration 4. **Check Logs**: Review node and application logs for error messages 5. **Community Resources**: Check GitHub issues and discussions ### Community Support 1. **GitHub Issues**: [Report issues on GitHub](https://github.com/calimero-network/merobox/issues) 2. **Discussions**: Use GitHub Discussions for questions and ideas 3. **Documentation**: Contribute to documentation improvements 4. **Examples**: Share your workflow examples with the community ### Professional Support 1. **Enterprise Support**: Contact Calimero for enterprise support 2. **Consulting**: Get professional help with complex deployments 3. **Training**: Attend Merobox training sessions 4. **Custom Development**: Request custom features or integrations ## Common Solutions ### Quick Fixes ```bash # Restart everything merobox stop --all merobox nuke merobox run --count 1 # Check Docker docker system prune -f docker volume prune -f # Restart Docker sudo systemctl restart docker # Check permissions sudo usermod -aG docker $USER newgrp docker ``` ### Reset Environment ```bash # Complete reset merobox stop --all merobox nuke docker system prune -af docker volume prune -f merobox run --count 1 ``` ### Debug Mode ```bash # Enable debug logging export LOG_LEVEL=DEBUG merobox bootstrap run workflow.yml --verbose # Check specific issues merobox health --verbose merobox logs calimero-node-1 ``` ## Next Steps Now that you understand troubleshooting: - [Advanced Configuration](./advanced-configuration) - Advanced setup options --- ## Workflow Advanced Features # Workflow Advanced Features This guide covers advanced workflow features in Merobox, including parallel step execution and custom scripts. ## Parallel Step Execution Execute multiple steps simultaneously for improved performance: ### Basic Parallel Execution ```yaml steps: - name: Parallel Operations type: parallel steps: - name: Install App 1 type: install_application node: calimero-node-1 path: ./app1.wasm dev: true outputs: app_id_1: applicationId - name: Install App 2 type: install_application node: calimero-node-2 path: ./app2.wasm dev: true outputs: app_id_2: applicationId - name: Create Identity type: create_identity node: calimero-node-3 outputs: public_key: publicKey ``` ### Parallel with Repeat ```yaml steps: - name: Parallel Repeat Operations type: parallel steps: - name: Install Multiple Apps type: repeat count: 3 outputs: iteration: iteration steps: - name: Install App {{iteration}} type: install_application node: calimero-node-{{iteration}} path: ./app{{iteration}}.wasm dev: true outputs: app_id: applicationId ``` ## Custom Scripts Use custom scripts for complex operations: ### Basic Script Execution ```yaml steps: - name: Custom Setup Script type: script script: | echo "Setting up custom environment..." mkdir -p /tmp/custom-data echo "custom_value" > /tmp/custom-data/sample.txt echo "Setup complete" outputs: setup_result: output - name: Use Script Output type: call node: calimero-node-1 context_id: '{{context_id}}' method: process_data args: data: '{{setup_result}}' ``` ### Script with Variables ```yaml steps: - name: Process Data Script type: script script: | echo "Processing data for context: {{context_id}}" echo "Using public key: {{public_key}}" # Process the data echo "processed_data" > /tmp/result.txt echo "Processing complete" outputs: processed_data: output ``` ## Error Handling Implement basic error handling in workflows: ### Try-Catch Pattern ```yaml steps: - name: Risky Operation type: call node: calimero-node-1 context_id: '{{context_id}}' method: risky_method args: should_fail: false outputs: result: result - name: Handle Success type: script script: | echo "Operation succeeded: {{result}}" condition: '{{result}} != null' - name: Handle Error type: script script: | echo "Operation failed, continuing..." condition: '{{result}} == null' ``` ### Validation Steps ```yaml steps: - name: Validate Input type: assert statements: - 'is_set({{context_id}})' - 'is_set({{public_key}})' - name: Execute Operation type: call node: calimero-node-1 context_id: '{{context_id}}' method: safe_method args: key: '{{public_key}}' ``` ## Performance Optimization ### Resource Management ```yaml # Use appropriate node counts nodes: count: 3 # Scale based on needs image: ghcr.io/calimero-network/merod:edge steps: - name: Efficient Operations type: parallel max_concurrent: 2 # Limit concurrent operations steps: - name: Operation 1 type: call # ... operation details - name: Operation 2 type: call # ... operation details ``` ### Caching and Reuse ```yaml steps: - name: Create Reusable Data type: script script: | echo "Creating reusable data..." echo "shared_data" > /tmp/shared.txt outputs: shared_data: output - name: Use Shared Data type: call node: calimero-node-1 method: use_shared_data args: data: '{{shared_data}}' - name: Use Shared Data Again type: call node: calimero-node-2 method: use_shared_data args: data: '{{shared_data}}' ``` ## Best Practices ### 1. Use Parallel Execution ```yaml # Good: Use parallel execution for independent operations steps: - name: Parallel Setup type: parallel steps: - name: Setup Node 1 type: install_application # ... setup details - name: Setup Node 2 type: install_application # ... setup details ``` ### 2. Implement Error Handling ```yaml # Good: Include error handling steps: - name: Validate Before Execution type: assert statements: - 'is_set({{required_variable}})' - name: Execute Operation type: call # ... operation details ``` ### 3. Use Appropriate Resources ```yaml # Good: Scale resources appropriately nodes: count: 3 # Use appropriate number of nodes image: ghcr.io/calimero-network/merod:edge ``` ## Troubleshooting ### Common Issues #### Parallel Execution Failures ```bash # Check node health merobox health --verbose # Check logs merobox logs calimero-node-1 ``` #### Script Execution Errors ```bash # Run with verbose output merobox bootstrap run workflow.yml --verbose # Check script output merobox logs calimero-node-1 | grep "script" ``` ## Next Steps Now that you understand advanced workflow features: - [Best Practices](./best-practices) - Guidelines for effective Merobox usage - [Troubleshooting](./troubleshooting) - Common issues and solutions --- ## Workflow System # Workflow System Merobox's workflow system allows you to define complex, multi-step operations using YAML configuration files. Workflows can orchestrate node management, application deployment, context creation, identity management, and contract execution in a declarative way. ## Overview Workflows in Merobox provide: - **Declarative Configuration**: Define complex operations in simple YAML files - **Dynamic Value Management**: Capture and reuse values between steps - **Step Orchestration**: Chain multiple operations together - **Error Handling**: Built-in validation and error reporting - **Testing Integration**: Perfect for automated testing scenarios ## Workflow Structure ### Basic Workflow Format ```yaml description: 'Workflow description' name: 'Workflow Name' # Node configuration nodes: chain_id: testnet-1 count: 2 image: ghcr.io/calimero-network/merod:edge prefix: calimero-node # Workflow steps steps: - name: 'Step Name' type: step_type # Step-specific parameters outputs: variable_name: field_name # Global configuration stop_all_nodes: false restart: false wait_timeout: 60 ``` ### Node Configuration Define the Calimero nodes for your workflow: ```yaml nodes: chain_id: testnet-1 # Blockchain chain ID count: 2 # Number of nodes to create image: ghcr.io/calimero-network/merod:edge # Docker image prefix: calimero-node # Node name prefix base_port: 2428 # Base P2P port (optional) base_rpc_port: 2528 # Base RPC port (optional) ``` ### Global Configuration Control workflow behavior: ```yaml # Force pull Docker images even if they exist locally force_pull_image: true # Enable authentication service with Traefik proxy auth_service: true auth_image: ghcr.io/calimero-network/mero-auth:edge # Stop all nodes at the end of workflow stop_all_nodes: false # Restart nodes at the beginning of workflow restart: false # Timeout for waiting operations wait_timeout: 60 ``` ## Step Types Merobox supports various step types for different operations: ### Application Management #### `install_application` Install WASM applications on Calimero nodes. ```yaml - name: Install Application type: install_application node: calimero-node-1 path: ./my-app.wasm # Local file path # OR url: https://example.com/app.wasm # Remote URL dev: true # Development mode outputs: app_id: applicationId # Export application ID ``` #### `create_context` Create blockchain contexts for applications. ```yaml - name: Create Context type: create_context node: calimero-node-1 application_id: '{{app_id}}' # Use captured value params: # Optional context parameters param1: value1 outputs: context_id: contextId member_public_key: memberPublicKey ``` ### Identity Management #### `create_identity` Generate cryptographic identities. ```yaml - name: Create Identity type: create_identity node: calimero-node-2 outputs: public_key: publicKey ``` #### `invite_identity` Invite identities to join contexts. ```yaml - name: Invite Identity type: invite_identity node: calimero-node-1 context_id: '{{context_id}}' grantee_id: '{{public_key}}' granter_id: '{{member_public_key}}' capability: member outputs: invitation: invitation ``` #### `join_context` Join contexts using invitations. ```yaml - name: Join Context type: join_context node: calimero-node-2 context_id: '{{context_id}}' invitee_id: '{{public_key}}' invitation: '{{invitation}}' ``` ### Contract Execution #### `call` Execute smart contract functions. ```yaml - name: Execute Contract Call type: call node: calimero-node-1 context_id: '{{context_id}}' executor_public_key: '{{member_public_key}}' method: set # Contract method name args: # Method arguments key: hello value: world outputs: result: result ``` ### Control Flow #### `wait` Add delays between steps. ```yaml - name: Wait for Propagation type: wait seconds: 5 ``` #### `repeat` Execute steps multiple times. ```yaml - name: Repeat Operations type: repeat count: 3 outputs: current_iteration: iteration steps: - name: Set Value type: call node: calimero-node-1 context_id: '{{context_id}}' method: set args: key: 'key_{{current_iteration}}' value: 'value_{{current_iteration}}' ``` #### `script` Execute custom scripts. ```yaml - name: Run Custom Script type: script script: | echo "Hello from script" echo "Context ID: {{context_id}}" outputs: script_output: output ``` ### Validation and Testing #### `assert` Validate conditions and outputs. ```yaml - name: Validate Results type: assert statements: - 'is_set({{context_id}})' - "contains({{result}}, 'expected_value')" - '{{count}} >= 1' - 'regex({{public_key}}, ^[1-9A-HJ-NP-Za-km-z]+$)' ``` #### `json_assert` Validate JSON data structures. ```yaml - name: Validate JSON type: json_assert statements: - 'json_equal({{result}}, {"output": "expected"})' - 'json_subset({{result}}, {"output": "expected"})' ``` ## Dynamic Value Management ### Variable Capture Capture values from step outputs for use in subsequent steps: ```yaml steps: - name: Install App type: install_application node: calimero-node-1 path: ./app.wasm outputs: app_id: applicationId # Capture 'applicationId' as 'app_id' - name: Create Context type: create_context node: calimero-node-1 application_id: '{{app_id}}' # Use captured value outputs: context_id: contextId ``` ### Variable Resolution Variables are resolved using `{{variable_name}}` syntax: ```yaml args: key: 'user_{{user_id}}_data_{{iteration}}' message: 'Processing {{context_id}} with {{public_key}}' ``` ### Available Variables - **Step Outputs**: Variables exported by previous steps - **Workflow Context**: Global workflow variables - **Environment Variables**: System environment variables - **Repeat Iteration**: Current iteration number in repeat steps ## Workflow Execution ### Running Workflows Execute workflows using the bootstrap command: ```bash # Run a workflow merobox bootstrap run workflow.yml # Run with authentication service merobox bootstrap run workflow.yml --auth-service # Run with custom auth image merobox bootstrap run workflow.yml --auth-service --auth-image ghcr.io/calimero-network/mero-auth:latest # Run with verbose output merobox bootstrap run workflow.yml --verbose ``` ### Validating Workflows Validate workflow configuration before execution: ```bash # Validate workflow syntax merobox bootstrap validate workflow.yml # Create sample workflow merobox bootstrap create-sample ``` ### Command Reference #### `merobox bootstrap run` Execute a workflow from a YAML file. **Syntax:** ```bash merobox bootstrap run CONFIG_FILE [OPTIONS] ``` **Arguments:** - `CONFIG_FILE`: Path to the workflow YAML file **Options:** | Option | Description | | ------------------- | ------------------------------------------------ | | `--auth-service` | Enable authentication service with Traefik proxy | | `--auth-image TEXT` | Custom Docker image for the auth service | | `--verbose, -v` | Enable verbose output | | `--help` | Show help message | #### `merobox bootstrap validate` Validate workflow configuration. **Syntax:** ```bash merobox bootstrap validate CONFIG_FILE ``` #### `merobox bootstrap create-sample` Create a sample workflow file. **Syntax:** ```bash merobox bootstrap create-sample ``` ## Example Workflows ### Basic Application Deployment ```yaml description: Deploy and test a simple application name: Basic App Deployment nodes: chain_id: testnet-1 count: 1 image: ghcr.io/calimero-network/merod:edge prefix: calimero-node steps: - name: Install Application type: install_application node: calimero-node-1 path: ./my-app.wasm dev: true outputs: app_id: applicationId - name: Create Context type: create_context node: calimero-node-1 application_id: '{{app_id}}' outputs: context_id: contextId member_key: memberPublicKey - name: Test Application type: call node: calimero-node-1 context_id: '{{context_id}}' executor_public_key: '{{member_key}}' method: test args: input: 'test_data' stop_all_nodes: true ``` ### Multi-Node Context Sharing ```yaml description: Create context on one node and join from another name: Multi-Node Context Sharing nodes: chain_id: testnet-1 count: 2 image: ghcr.io/calimero-network/merod:edge prefix: calimero-node steps: # Install application on first node - name: Install Application type: install_application node: calimero-node-1 path: ./shared-app.wasm dev: true outputs: app_id: applicationId # Create context on first node - name: Create Context type: create_context node: calimero-node-1 application_id: '{{app_id}}' outputs: context_id: contextId member_key: memberPublicKey # Create identity on second node - name: Create Identity type: create_identity node: calimero-node-2 outputs: public_key: publicKey # Invite second node to context - name: Invite Node type: invite_identity node: calimero-node-1 context_id: '{{context_id}}' grantee_id: '{{public_key}}' granter_id: '{{member_key}}' capability: member outputs: invitation: invitation # Join context from second node - name: Join Context type: join_context node: calimero-node-2 context_id: '{{context_id}}' invitee_id: '{{public_key}}' invitation: '{{invitation}}' # Test cross-node communication - name: Set Value from Node 1 type: call node: calimero-node-1 context_id: '{{context_id}}' executor_public_key: '{{member_key}}' method: set args: key: shared_data value: 'set from node 1' - name: Get Value from Node 2 type: call node: calimero-node-2 context_id: '{{context_id}}' executor_public_key: '{{public_key}}' method: get args: key: shared_data outputs: result: result - name: Validate Cross-Node Communication type: assert statements: - "contains({{result}}, 'set from node 1')" stop_all_nodes: true ``` ### Testing with Assertions ```yaml description: Comprehensive testing workflow with assertions name: Testing with Assertions nodes: chain_id: testnet-1 count: 1 image: ghcr.io/calimero-network/merod:edge prefix: calimero-node steps: - name: Install Application type: install_application node: calimero-node-1 path: ./test-app.wasm dev: true outputs: app_id: applicationId - name: Create Context type: create_context node: calimero-node-1 application_id: '{{app_id}}' outputs: context_id: contextId member_key: memberPublicKey - name: Set Test Data type: call node: calimero-node-1 context_id: '{{context_id}}' executor_public_key: '{{member_key}}' method: set args: key: test_key value: test_value outputs: set_result: result - name: Get Test Data type: call node: calimero-node-1 context_id: '{{context_id}}' executor_public_key: '{{member_key}}' method: get args: key: test_key outputs: get_result: result # Basic assertions - name: Validate Basic Conditions type: assert statements: - 'is_set({{context_id}})' - 'is_set({{member_key}})' - "{{context_id}} != ''" - "contains({{get_result}}, 'test_value')" # JSON assertions - name: Validate JSON Response type: json_assert statements: - 'json_equal({{get_result}}, {"output": "test_value"})' - 'json_subset({{get_result}}, {"output": "test_value"})' # String operations - name: Validate String Operations type: assert statements: - 'regex({{member_key}}, ^[1-9A-HJ-NP-Za-km-z]+$)' - "not_contains({{member_key}}, 'invalid')" stop_all_nodes: true ``` ## Best Practices ### Workflow Design 1. **Modular Steps**: Break complex operations into smaller, focused steps 2. **Clear Naming**: Use descriptive names for steps and variables 3. **Error Handling**: Include validation steps to catch issues early 4. **Documentation**: Add descriptions to explain workflow purpose ### Variable Management 1. **Consistent Naming**: Use consistent naming conventions for variables 2. **Minimal Scope**: Only capture variables you actually need 3. **Clear Mapping**: Use descriptive names in outputs sections 4. **Validation**: Validate captured values before using them ### Testing Integration 1. **Assertions**: Include assertions to validate expected outcomes 2. **Edge Cases**: Test both success and failure scenarios 3. **Cleanup**: Use `stop_all_nodes: true` for test workflows 4. **Isolation**: Use unique prefixes for parallel test runs ### Performance 1. **Minimal Waits**: Use appropriate wait times, not excessive delays 2. **Resource Management**: Stop nodes when not needed 3. **Image Management**: Use `force_pull_image: true` sparingly 4. **Parallel Operations**: Design workflows to minimize sequential dependencies ## Troubleshooting ### Common Issues #### Variable Resolution Errors ```yaml # Error: Variable '{{missing_var}}' not found ``` **Solutions:** - Check variable names for typos - Ensure previous steps export the required variables - Use `merobox bootstrap validate` to check configuration #### Step Validation Failures ```yaml # Error: Required field 'node' missing ``` **Solutions:** - Validate workflow: `merobox bootstrap validate workflow.yml` - Check step configuration and required fields - Verify field types and values #### API Call Failures ```yaml # Error: API request failed ``` **Solutions:** - Check node health: `merobox health` - Verify nodes are ready before making API calls - Check network connectivity and port availability ### Debug Commands ```bash # Validate workflow before running merobox bootstrap validate workflow.yml # Run with verbose output merobox bootstrap run workflow.yml --verbose # Check node status merobox health --verbose # View node logs merobox logs calimero-node-1 ``` ## Next Steps Now that you understand workflows: - [Examples and Tutorials](./examples) - Practical examples and tutorials - [Advanced Configuration](./advanced-configuration) - Advanced setup options - [Python Customization](./python-customization) - Advanced Python customization and extensions - [Troubleshooting](./troubleshooting) - Common issues and solutions --- ## Overview Welcome to the Tutorials section of the Calimero Network documentation! Here, you’ll find step-by-step guides to help you build, deploy, and integrate applications using Calimero's features. Whether you're a beginner setting up your first environment or an experienced developer exploring advanced use cases, these tutorials are designed to provide practical insights and hands-on experience. Let's get started! --- ## Near Wallet ## Logging in with NEAR Wallet Calimero allows users to authenticate via their NEAR wallet, ensuring secure access to the Admin Dashboard. The process of logging in with a NEAR wallet involves the following steps: ### 1. User Initiates Login - The user navigates to the Admin Dashboard and selects the option to "Login with NEAR Wallet." ### 2. Server Issues Challenge - Upon selecting the login option, the server generates a challenge. This challenge is then presented to the user for signing. ### 3. Redirect to MyNearWallet - The user is redirected to the [MyNearWallet](https://mynearwallet.com/) page, where they will be asked to sign the server-issued challenge using their NEAR wallet. ### 4. Signature Sent to Backend - After the user signs the challenge, the signature is sent back to the server for verification. ### 5. Signature Verification - The backend verifies the signature to ensure its authenticity. If the signature is valid, the user's public key is extracted from the signature and added as the user's unique identifier. - This public key serves as the user's Decentralized Identifier (DID), not as a context-specific identity. The DID enables decentralized authentication, ensuring that the user can interact with the platform securely across different contexts and applications. ### 6. Successful Login - Once verified, the user is successfully logged into the Admin Dashboard, with their public key (DID) serving as their user ID for future interactions. This login flow ensures a secure and seamless authentication process leveraging NEAR's robust blockchain-based security. --- ## Starknet Wallet ## Logging in with Starknet Calimero allows users to authenticate via their Starknet wallet, ensuring secure access to the Admin Dashboard. The process of logging in with a Starknet wallet involves the following steps: ### 1. User Initiates Login - The user navigates to the Admin Dashboard and selects the option to "Login with Starknet." - They are presented with two wallet options: ArgentX and Metamask Snap. ### 2. Wallet Selection - The user chooses their preferred wallet: ArgentX or Metamask Snap. ### 3. Wallet Connection #### For ArgentX: - If ArgentX is installed, the user is prompted to connect their wallet. - If not installed, the user is directed to install the ArgentX browser extension. #### For Metamask Snap: - If Metamask with Starknet Snap is installed, the user is prompted to connect. - If not installed, the user is guided to install Metamask and the Starknet Snap. ### 4. Server Issues Challenge - Upon selecting the wallet, the server generates a challenge. This challenge is then presented to the user for signing. ### 5. User Signs Challenge - The user is prompted to sign the server-issued challenge using their selected Starknet wallet. ### 6. Signature Sent to Backend - After the user signs the challenge, the signature is sent back to the server for verification. ### 7. Signature Verification The backend verifies the signature to ensure its authenticity using a process tailored to Starknet's cryptographic standards. This verification process differs depending on whether the user is using ArgentX or Metamask Snap. #### For ArgentX Wallets: 1. **On-Chain Verification**: - The signature is initially verified on-chain using the Starknet network. - A JSON-RPC call is made to the Starknet node, invoking the `isValidSignature` function on the user's wallet contract. - This on-chain verification ensures the signature's validity according to the wallet's implementation. 2. **Message Hash Verification**: - If the on-chain verification succeeds, an additional verification of the message hash is performed off-chain. - This step ensures the integrity of the signed message and prevents any tampering. #### For Metamask Snap: 1. **Off-Chain Verification**: - The signature is verified off-chain using Starknet's cryptographic libraries. - This involves using the `verify` function from the `starknet_crypto` library to check the signature against the provided public key. 2. **Message Hash Verification**: - If the signature is valid, an additional verification of the message hash is performed. - This ensures the integrity of the signed message, similar to the ArgentX process. #### Final Steps: - If the signature and hash are valid, the user's Starknet address is extracted from the wallet address used in the verification process. - This address serves as the user's Decentralized Identifier (DID), enabling decentralized authentication across different contexts and applications. ### 8. Successful Login - Once verified, the user is successfully logged into the Admin Dashboard, with their Starknet address (DID) serving as their user ID for future interactions. This login flow ensures a secure and seamless authentication process leveraging Starknet's robust blockchain-based security. --- ## ICP Wallet ## Logging in with Internet Identity The login process with Internet Identity involves these key steps: ### 1. User Initiates Login - The user navigates to the Admin Dashboard and selects the option to "Login with Internet Identity." ### 2. Challenge Preparation - The system prepares a challenge (encoded as a session public key) before initiating the authentication process. ### 3. Internet Identity Authentication - A popup window opens, connecting to the Internet Identity service. - The user authenticates through this popup. - The challenge (session public key) is sent to the II service. ### 4. Delegation Chain Retrieval - Upon successful authentication, the II service creates and returns a delegation chain. - This delegation chain includes cryptographic proofs of the user's authentication. ### 5. Processing the Authentication Result - The application receives the delegation chain from the II service. - It processes this data, extracting necessary information like the user's public key. ### 6. Login Request Preparation - The application prepares a login request including: - The serialized delegation chain - The original challenge payload - Wallet metadata (ICP canister ID and wallet name) ### 7. Server-side Verification - The login request is sent to the Calimero server. - The server verifies the delegation chain and challenge. ### 8. Successful Login - Upon successful verification, the user is logged in and directed to the identity page. - The user's Internet Identity becomes their unique identifier for future interactions within the Calimero ecosystem. This process leverages ICP's delegation chain mechanism for secure, decentralized authentication. --- ## Ethereum Wallet ## Logging in with Ethereum Wallet Calimero allows users to authenticate via their Ethereum wallet (such as Metamask), ensuring secure access to the Admin Dashboard. The authentication process leverages Ethereum's ECDSA signature verification through the authentication service. ## Authentication Flow The Ethereum authentication in Calimero follows the same challenge-response pattern as other supported wallets. Here's how it works: ### 1. User Initiates Login - The user initiates login and is redirected to the authentication page - They are presented with wallet options, typically Metamask for Ethereum authentication ### 2. Server Issues Authentication Challenge - The authentication service generates a unique challenge for the user to sign - This challenge typically includes a timestamp and unique identifier to prevent replay attacks ### 3. User Signs Authentication Challenge - The user is prompted to sign the server-issued structured challenge using their Ethereum wallet. - The wallet extension displays the challenge data and requests user confirmation. - The user approves the signature request in their wallet. ### 4. Signature Sent to Backend - After the user signs the challenge, the signature is sent back to the server for verification. ### 5. Signature Verification - The backend verifies the Ethereum signature to ensure its authenticity using standard ECDSA signature verification. - The signature is verified against the structured challenge that was signed. - If the signature is valid, the user's Ethereum address is extracted and stored as the user's unique identifier. ### 6. Successful Authentication - Once verified, the user is successfully authenticated. - The user's identifier is the wallet's public key. ## Authentication Implementation Calimero's Ethereum authentication uses structured challenges with ECDSA signature verification: - **Structured Challenge Generation**: The server generates a unique structured challenge for each login attempt using the Request format - **Signature Verification**: The backend verifies the ECDSA signature against the structured challenge - **Identity Storage**: The user's Ethereum wallet public key serves as their unique identifier ## Supported Ethereum Wallets Calimero supports MetaMask for Ethereum authentication: - **Metamask**: The most popular Web3 wallet extension ## Security Features - **Structured Challenge-Response Authentication**: Each login attempt uses a unique server-generated structured challenge with specific fields for enhanced security - **ECDSA Signature Verification**: Uses Ethereum's standard cryptographic methods for signature validation - **Public Key-based Identity**: User identity is tied to their Ethereum wallet public key, ensuring decentralized authentication - **Standard Wallet Security**: Leverages the security features of established Ethereum wallets like Metamask This authentication flow leverages Ethereum's robust ECDSA signature verification to provide secure, decentralized access to Calimero's features while maintaining consistency with the blockchain's native security model. --- ## Create Context import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import ContextCreateBinary from '../shared/context-create-binary.mdx'; import ContextCreateInteractiveCli from '../shared/context-create-interactive-cli.mdx'; import ContextCreateAdminDashboard from '../shared/context-create-admin-dashboard.mdx'; import InitNode from '../shared/node-init-binary.mdx'; import RunNode from '../shared/node-run-binary.mdx'; ### Initialize node ### Run node ### Create a New Context You have now created a new context. To create an alias for this context, use: ```bash title="Terminal" meroctl --node node1 context alias add my_context 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf ``` Next step is to invite users to join your context. Continue with [invitations and joinings](./invitations-and-joinings). --- ## Invitations and Joinings ## Invite to context with Admin Dashboard > Currently not available ## Invite node to context using interactive CLI Inviting to the context consists of 3 steps: 1. Identity Exchange: The peer wishing to join (invitee) shares their identity with the existing member (invitor). 2. Create Invitation: The invitor generates an invitation payload for the context. 3. Accept Invitation: The invitee signs the invitation payload to confirm their identity and join the context. In the example below, Node 2 is the invitee, and Node 1 is the invitor. :::tip If you create an Context Alias for your context using ```bash title="Context Alias (Node1)" meroctl --node node1 context alias add my_context 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf ``` And then create an Identity Alias for your identity in the context using ```bash title="Identity Alias (Node1)" meroctl --node node1 context identity alias add bob 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf --context my_context ``` Everywhere in the tutorial, on the node which you have created the aliases (aliases are node specific), you can use `my_context` and `bob` instead of the contextID and the invitorContextIdentity ::: ### Step-by-Step Process 1. Invitee Generates and Shares Identity To join the context, the invitee must first create and share their identity with the invitor. This can be done using the following command: ```bash title="Node 2" meroctl --node node1 context identity generate ``` This generates a private and public key pair, where the public key is the inviteeContextIdentity. Share this public key with the invitor. ```bash title="Example" meroctl --node node1 context identity generate │ Private Key: 6i52ykNTqSqCgfDPYiKktListssSP9YwxLMzApDZe5U7 │ Public Key: 3yBQQvn2G8ghWk2ZwmLFYxJQgM1YmDtiigi84nGpZBb9 ``` ![Create Identity](/cli/3-create-identity.png) 2. Invitor Creates an Invitation Once the invitee's public key is shared, the invitor generates an invitation to the context using the following command: ```bash title="Node 1" meroctl --node node1 context invite 3yBQQvn2G8ghWk2ZwmLFYxJQgM1YmDtiigi84nGpZBb9 --as DaSkad6DK7f6fUhjz1CvNW7L4TkWZmWqAcwysZHG3Xs5 ``` ```bash title="Example" meroctl --node node1 context invite 3yBQQvn2G8ghWk2ZwmLFYxJQgM1YmDtiigi84nGpZBb9 --as DaSkad6DK7f6fUhjz1CvNW7L4TkWZmWqAcwysZHG3Xs5 │ Invited 3yBQQvn2G8ghWk2ZwmLFYxJQgM1YmDtiigi84nGpZBb9 to context 567C5Gg4mxHMPKy2wLJ4uvb3DHsbcpVDYUsuAWgTPgXn │ Invitation Payload: axDfZcWCw7jc3i7MeG9JqhsYrNXRSvg6hXVWYpdsxNRhAjKD35S5FruCzyRWHYHzQyN1QbSVyRKRiTwz8Kbq4aj2dSdmi7HNxwcjTS5JkK1xwJMA8ogYuWHhDj4jfLdukPZb2SavC8cq3npFydinVZ ``` ![Context Invite](/cli/4-context-invite.png) To check the invitor's identity, use this command: ```bash title="Node 1" meroctl --node node1 context identity list --context 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf ``` ```bash title="Example" meroctl --node node1 context identity list --context 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf │ Identity | Owned │ | DaSkad6DK7f6fUhjz1CvNW7L4TkWZmWqAcwysZHG3Xs5 | * ``` Listed identity is invitorContextIdentity used above. 3. Invitee Accepts the Invitation After receiving the invitation payload from the invitor, the invitee can accept the invitation by signing it with their private key: ```bash title="Node 2" meroctl --node node1 context join 6i52ykNTqSqCgfDPYiKktListssSP9YwxLMzApDZe5U7 axDfZcWCw7jc3i7MeG9JqhsYrNXRSvg6hXVWYpdsxNRhAjKD35S5FruCzyRWHYHzQyN1QbSVyRKRiTwz8Kbq4aj2dSdmi7HNxwcjTS5JkK1xwJMA8ogYuWHhDj4jfLdukPZb2SavC8cq3npFydinVZ ``` ```bash title="Example" meroctl --node node1 context join 6i52ykNTqSqCgfDPYiKktListssSP9YwxLMzApDZe5U7 axDfZcWCw7jc3i7MeG9JqhsYrNXRSvg6hXVWYpdsxNRhAjKD35S5FruCzyRWHYHzQyN1QbSVyRKRiTwz8Kbq4aj2dSdmi7HNxwcjTS5JkK1xwJMA8ogYuWHhDj4jfLdukPZb2SavC8cq3npFydinVZ 2024-10-07T12:18:51.809231Z INFO calimero_context: Subscribed to context context_id=567C5Gg4mxHMPKy2wLJ4uvb3DHsbcpVDYUsuAWgTPgXn 2024-10-07T12:18:51.811102Z INFO calimero_context: Joined context with pending catchup context_id=567C5Gg4mxHMPKy2wLJ4uvb3DHsbcpVDYUsuAWgTPgXn │ Joined context 567C5Gg4mxHMPKy2wLJ4uvb3DHsbcpVDYUsuAWgTPgXn as 3yBQQvn2G8ghWk2ZwmLFYxJQgM1YmDtiigi84nGpZBb9, waiting for catchup to complete... ``` ![Context Join](/cli/5-context-join.png) 4. Verifying Context Membership To confirm successful joining of the context, the invitee can list their contexts with the following command: ```bash title="Node 2" meroctl --node node1 context list ``` ```bash title="Example" meroctl --node node1 context list │ Context ID | Application ID | Last Transaction │ 567C5Gg4mxHMPKy2wLJ4uvb3DHsbcpVDYUsuAWgTPgXn | Ahe2vLWLgswJARv5LsafXp7uJyb2Ba9GjzUSeLc71gUF | b3ipivssRRm1ehRTSpUD3GKdUpvi3vq311pCT4iLvui ``` ![Context List](/cli/8-context-list.png) 5. The invitee can also list identities in the context ```bash title="Node 2" meroctl --node node1 context identity list --context 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf ``` ```bash title="Example" meroctl --node node1 context identity list --context 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf │ Identity | Owned │ 3yBQQvn2G8ghWk2ZwmLFYxJQgM1YmDtiigi84nGpZBb9 | * │ DaSkad6DK7f6fUhjz1CvNW7L4TkWZmWqAcwysZHG3Xs5 | ``` ![Context Identity List](/cli/6-identity-list.png) --- ## Key-Value Store tutorial ## Building a Key-Value Store with Calimero SDK The Calimero SDK for Rust empowers developers to build applications that compile to WebAssembly (Wasm) and run securely within the Calimero virtual machine. This guide will walk you through creating a complete key-value store application and preparing it for deployment. ### Prerequisites Before you begin, ensure you have Rust installed on your system. If not, follow the official Rust installation guide for your platform: [Rust Installation Guide](https://www.rust-lang.org/tools/install). You should ensure you have the `wasm32-unknown-unknown` target installed. Run the following command in your terminal to install the target: ```bash title="Terminal" rustup target add wasm32-unknown-unknown ``` ### Setting Up Your Project To create a new project, initialize a Rust library project using Cargo. Run the following command in your terminal: ```bash title="Terminal" cargo new --lib kv-store ``` You should have a tree that looks like this: ```bash title="Terminal" $ tree kv-store kv-store ├── Cargo.toml └── src └── lib.rs 2 directories, 2 files ``` At this point, we can `cd` into the `kv-store` directory: ```bash title="Terminal" cd kv-store ``` Next, you need to specify the crate-type as `cdylib` in your `Cargo.toml` file to generate a dynamic library that can be compiled to Wasm: ```toml title="File: Cargo.toml" [lib] crate-type = ["cdylib"] ``` You can now configure your project to use the Calimero SDK by adding it as a dependency in your `Cargo.toml` file: ```toml title="File: Cargo.toml" [dependencies] calimero-sdk = { git = "https://github.com/calimero-network/core" } calimero-storage = { git = "https://github.com/calimero-network/core" } ``` Then, we need to specify a custom build profile for the most compact Wasm output: ```toml title="File: Cargo.toml" [profile.app-release] inherits = "release" codegen-units = 1 opt-level = "z" lto = true debug = false panic = "abort" overflow-checks = true ``` Your `Cargo.toml` file should now look like this ```toml title="File: Cargo.toml" showLineNumbers [package] name = "kv-store" version = "0.1.0" edition = "2021" # highlight-start [lib] crate-type = ["cdylib"] # highlight-end # highlight-start [dependencies] calimero-sdk = { git = "https://github.com/calimero-network/core" } calimero-storage = { git = "https://github.com/calimero-network/core" } # highlight-end # highlight-start [profile.app-release] inherits = "release" codegen-units = 1 opt-level = "z" lto = true debug = false panic = "abort" overflow-checks = true # highlight-end ``` And finally, create a `build.sh` script to compile your application into Wasm format, for example: ```bash title="File: build.sh" showLineNumbers #!/bin/bash set -e cd "$(dirname $0)" TARGET="${CARGO_TARGET_DIR:-../../target}" rustup target add wasm32-unknown-unknown cargo build --target wasm32-unknown-unknown --profile app-release mkdir -p res cp $TARGET/wasm32-unknown-unknown/app-release/kv_store.wasm ./res/ ``` You can optionally choose to install and use [`wasm-opt`](https://github.com/WebAssembly/binaryen), for an additional optimization step in the build script. This step is not required but can help reduce the size of the generated Wasm file: ```bash title="File: build.sh" if command -v wasm-opt > /dev/null; then wasm-opt -Oz ./res/kv_store.wasm -o ./res/kv_store.wasm fi ``` Don't forget to make the `build.sh` script executable: ```bash title="Terminal" chmod +x build.sh ``` At this point, your project structure should look like this: ```bash title="Terminal" $ tree . ├── Cargo.toml ├── build.sh └── src └── lib.rs 2 directories, 3 files ``` ### Writing Your Application Now, let's create a simple key-value store application using the Calimero SDK. Start by defining your application logic in `lib.rs`: ```rust title="File: src/lib.rs" showLineNumbers use calimero_sdk::borsh::{BorshDeserialize, BorshSerialize}; use calimero_sdk::app; #[app::state] #[derive(Default, BorshSerialize, BorshDeserialize)] #[borsh(crate = "calimero_sdk::borsh")] struct KvStore {} #[app::logic] impl KvStore { #[app::init] pub fn init() -> KvStore { KvStore {} } } ``` The `KvStore` struct represents the state of your application, which will be borsh-encoded in the app-scoped state partition on the node's storage. The `#[app::state]` attribute macro marks the struct as the application state, permitting its use by Calimero SDK. The `#[app::logic]` attribute macro marks the implementation block as the application logic, allowing you to define the methods that interact with the application state. An initializer method (named `init`) is denoted by the `#[app::init]` attribute macro, which is called when the application is executed against a freshly created context. Consider a method like `get` that retrieves a value from the key-value store: ```rust pub fn get(&self, key: &str) -> Result, Error> { // Snip... } ``` The inputs must be deserializable from the transaction data, and the output must be serializable to the response data. The `Option` type is used to represent the possibility of the key not being present in the store. The `Error` type is used to represent the possible error conditions that may occur during the execution of the method. And now, here's a complete example of a key-value store application: ```rust title="File: src/lib.rs" showLineNumbers use calimero_sdk::borsh::{BorshDeserialize, BorshSerialize}; use calimero_sdk::types::Error; use calimero_sdk::{app, env}; use calimero_storage::collections::UnorderedMap; #[app::state] #[derive(Default, BorshSerialize, BorshDeserialize)] #[borsh(crate = "calimero_sdk::borsh")] struct KvStore { entries: UnorderedMap, } #[app::logic] impl KvStore { #[app::init] pub fn init() -> KvStore { // highlight-start KvStore { entries: UnorderedMap::new(), } // highlight-end } // highlight-start pub fn set(&mut self, key: String, value: String) -> Result { env::log(&format!("Setting key: {:?} to value: {:?}", key, value)); self.entries.insert(key, value)?; Ok(()) } // highlight-end // highlight-start pub fn entries(&self) -> Result, Error> { env::log("Getting all entries"); Ok(self.entries.entries()?.collect()) } // highlight-end // highlight-start pub fn get(&self, key: &str) -> Result, Error> { env::log(&format!("Getting key: {:?}", key)); self.entries.get(key).map_err(Into::into) } // highlight-end // highlight-start pub fn remove(&mut self, key: &str) -> Result, Error> { env::log(&format!("Removing key: {:?}", key)); self.entries.remove(key).map_err(Into::into) } // highlight-end } ``` ### Building Your Application Once your application logic is defined, run the `./build.sh` script to compile your application into Wasm format. This script will generate `kv_store.wasm` in the `res` folder of your application. ```bash title="Terminal" $ ./build.sh info: component 'rust-std' for target 'wasm32-unknown-unknown' is up to date # Snip... Compiling calimero-sdk v0.1.0 Compiling calimero-storage v0.1.0 Compiling kv-store v0.1.0 (/apps/kv-store) Finished `app-release` profile [optimized] target(s) in 1.20s $ tree . ├── Cargo.toml ├── build.sh ├── res │   └── kv_store.wasm └── src └── lib.rs 3 directories, 4 files ``` ### Deploying Your Application After successfully building your application, you can upload the compiled `kv_store.wasm` to the registry for use by a live Calimero node. ### Writing Efficient Code with Calimero SDK In the following code snippet: ```rust title="File: src/lib.rs" pub fn get(&self, key: &str) -> Result, Error> { // Snip... } ``` you'll notice that we prioritize using references instead of owned values. This approach optimizes performance and memory usage by minimizing unnecessary data copying. For input parameters, such as `&str` and `&[u8]`, utilizing references allows you to avoid unnecessary copying of data. Similarly, for output values, you can return references to data that live as long as `&self` or any of the input parameters. By doing so, you reduce memory overhead and improve the overall efficiency of your application. ### Handling Errors with Calimero SDK When designing methods that may potentially fail, it's recommended to return a `Result` type with an error variant representing the possible failure cases. This enables you to handle errors more effectively and communicate error conditions to users of your application. This is recommended over using the `Error` type exported from `calimero_sdk` and over panicking. Both of which only return a string message. #### Error Report Comparison Let's take the following cases (all of which fail when the key does not exist); 1. Using `calimero_sdk::types::Error`: This is provided for convenience, since most errors already don't implement `Serialize`, and so they cannot be immediately returned. This first converts the error to a string and then returns it. Which then JSON-encodes the string representation. ```rust title="File: src/lib.rs" use calimero_sdk::types::Error; pub fn get(&self, key: &str) -> Result { self.items.get(key)?.ok_or_else(|| Error::msg("key not found")) } ``` This failure will result in this outcome: ```rust ExecutionError([ 107, 75, 101, 121, 32, 110, 111, 116, 32, 102, 111, 117, 110, 100, 34 ]) ``` which can be decoded to ```json "key not found" ``` This `Error` can be constructed with `?` so long as the source error implements `std::error::Error`. Behaviourally similar to [`anyhow::Error`](https://docs.rs/anyhow/latest/anyhow/struct.Error.html) or [`eyre::Report`](https://docs.rs/eyre/latest/eyre/struct.Report.html). 2. Using a custom error type (recommended): For structured error handling, we recommend defining a custom error type that encodes all the possible error variants for that method. This allows you to provide more context about the error condition and handle different error scenarios more effectively. As opposed to string parsing. ```rust title="File: src/lib.rs" use calimero_sdk::serde::Serialize; #[derive(Debug, Serialize)] #[serde(crate = "calimero_sdk::serde")] #[serde(tag = "kind", content = "data")] pub enum Error { NotFound(&'a str), } ``` ```rust title="File: src/lib.rs" pub fn get(&self, key: &'a str) -> Result> { // Snip... Err(Error::NotFound(key)) } ``` This failure will result in this outcome: ```rust ExecutionError([ 123, 34, 107, 105, 110, 100, 34, 58, 34, 78, 111, 116, 70, 111, 117, 110, 100, 34, 44, 34, 100, 97, 116, 97, 34, 58, 34, 116, 104, 105, 110, 103, 34, 125 ]) ``` which can be decoded to ```json { "kind": "NotFound", "data": "thing" } ``` As will most likely be the case, you may need to work with storage errors while you've defined a custom error type. In this case, you can pull in [`thiserror`](https://docs.rs/thiserror/latest/thiserror/) to help. ```rust title="File: src/lib.rs" use thiserror::Error; #[derive(Debug, Error, Serialize)] #[serde(crate = "calimero_sdk::serde")] #[serde(tag = "kind", content = "data")] pub enum Error { #[error("key not found: {0}")] NotFound(&'a str), #[error("store error: {0}")] StoreError(#[from] StoreError), } ``` ```rust title="File: src/lib.rs" pub fn get(&self, key: &'a str) -> Result> { // Snip... self.items.get(key)?.ok_or_else(|| Error::NotFound(key)) } ``` An example store error would then be represented as: ```rust ExecutionError( [ 123, 34, 107, 105, 110, 100, 34, 58, 34, 83, 116, 111, 114, 101, 69, 114, 114, 111, 114, 34, 44, 34, 100, 97, 116, 97, 34, 58, 34, 73, 110, 100, 101, 120, 32, 110, 111, 116, 32, 102, 111, 117, 110, 100, 32, 102, 111, 114, 32, 73, 68, 58, 32, 57, 51, 49, 53, 97, 98, 101, 49, 101, 97, 101, 48, 102, 102, 53, 98, 48, 48, 52, 53, 51, 97, 100, 97, 102, 99, 99, 53, 102, 101, 102, 50, 49, 100, 55, 52, 49, 51, 57, 55, 101, 50, 49, 99, 53, 49, 53, 51, 55, 99, 51, 54, 52, 52, 50, 52, 50, 48, 56, 55, 52, 57, 99, 57, 34, 125, ], ) ``` which can be decoded to ```json { "kind": "StoreError", "data": "Index not found for ID: 9315abe1eae0ff5b00453adafcc5fef21d741397e21c51537c364424208749c9" } ``` 3. Panic (ideally, development only) ```rust title="File: src/lib.rs" pub fn get(&self, key: &str) -> String { self.items.get(key).expect("store error").expect("key not found") } ``` A non-existent key would then lead to this outcome: ```rust HostError( Panic { context: Guest, message: "key not found", location: At { file: "apps/kv-store/src/lib.rs", line: 98, column: 14, }, }, ) ``` And a storage error, would produce this: ```rust HostError( Panic { context: Guest, message: "store error: StorageError(IndexNotFound(Id { bytes: [123, 240, 135, 21, 77, 143, 81, 169, 15, 202, 99, 210, 167, 165, 188, 156, 87, 146, 7, 211, 100, 92, 169, 189, 124, 115, 200, 242, 240, 73, 68, 123] }))", location: At { file: "apps/kv-store/src/lib.rs", line: 98, column: 14, }, }, ) ``` By following the second (recommended) approach, you can handle errors more gracefully and provide meaningful feedback to users of your Calimero application. And the first approach, if you want a hassle-free method of dealing with errors. ### Emitting Events with Calimero SDK To facilitate real-time monitoring of state transitions within your Calimero application, you can emit events using the `app::emit!` macro provided by the Calimero SDK. Event emission is particularly useful for handling live state transitions triggered by other actors, allowing subscribed clients to receive immediate updates about relevant actions. Let's focus on emitting events for mutating calls, specifically `set` and `remove` methods: First, define your custom events using the `#[app::event]` proc macro. In this example, we'll define events for setting a new key-value pair (`Inserted`), updating an existing value (`Updated`), and removing a key-value pair (`Removed`): ```rust title="File: src/lib.rs" use calimero_sdk::app; #[app::event] pub enum Event { Inserted { key: &'a str, value: &'a str }, Updated { key: &'a str, value: &'a str }, Removed { key: &'a str }, } ``` Each event variant can carry additional data to provide context about the event. Now, you need to associate the event with the application logic by annotating the application state. ```rust title="File: src/lib.rs" // highlight-start #[app::state(emits = for Event)] // highlight-end #[derive(Default, BorshSerialize, BorshDeserialize)] #[borsh(crate = "calimero_sdk::borsh")] struct KvStore { // Snip... } ``` And finally, within your application logic methods, emit events using the `app::emit!` macro: ```rust title="File: src/lib.rs" pub fn set(&mut self, key: String, value: String) -> Result { if self.items.contains(&key)? { app::emit!(Event::Updated { key: &key, value: &value, }); } else { app::emit!(Event::Inserted { key: &key, value: &value, }); } self.items.insert(key, value)?; Ok(()) } pub fn remove(&mut self, key: &str) -> Result { app::emit!(Event::Removed { key }); self.entries.remove(key)?.ok_or_else(|| Error::msg("key not found")) } ``` In each method, we emit the corresponding event with relevant data. This allows external observers to react to these events and take appropriate actions. By emitting events, you can ensure connected clients receive real-time updates about state transitions within your Calimero application, enabling them to respond to changes as they occur. ### Ensuring Atomicity and Event Reliability in Calimero Applications In Calimero applications, ensuring atomicity of state changes and reliability of event emission is crucial for maintaining data consistency and facilitating reliable communication between actors. Here's how atomicity and event reliability are enforced: #### Atomic State Changes When a method call fails, whether due to panics or returning an `Err`, all state changes made up to that point are discarded. This ensures that if an operation cannot be completed successfully, the application's state remains consistent and unaffected by partial updates. By enforcing atomicity, Calimero promotes data integrity and prevents inconsistencies that may arise from incomplete transactions. #### Reliable Event Emission Similarly, event emission in Calimero applications is tied to the successful execution of transactions. Events are only relayed when a transaction has been successfully executed, ensuring that external observers receive updates about state changes reliably. By linking event emission to transaction execution, Calimero guarantees that event notifications accurately reflect the application's current state, enhancing the reliability and consistency of communication between actors. This also means it doesn't matter if the event emission is done before or after the state change, as the event will only be emitted if the state change is successful. By adhering to these principles of atomicity and event reliability, Calimero applications maintain data integrity and enable robust interaction between different components, facilitating the development of secure and dependable decentralized systems. ### Local-First Efficiency Read-only operations (like `get`) have no network overhead, as they don't modify state and can be executed locally. ### Conclusion You've now learned how to set up a Rust project using the Calimero SDK, write a simple application, build it into Wasm, and prepare it for deployment. Experiment with different features and functionalities to create powerful and secure applications with Calimero. Happy coding! 🚀 --- ## Publish App After you have built your application, you can publish it to the network. This will make it available for users to interact with. ## Publishing application Navigate to [Admin Dashboard](../developer-tools/apps/admin-dashboard) and select `Applications` tab. ![Publish application](/admin-dashboard/publish-new-application.png) Enter the required data and publish the application. After publishing, your application will be available for users to interact with in new contexts. ## Application preview Good practice allows users to try an app before they decide to use it. You can deploy an app to any host provider. We have used GitHub Pages to preview our [example app](./install-application). You can do the same by following the steps below. ## Example app preview First you need to enable GitHub Pages by following [Creating a GitHub Pages](https://docs.github.com/en/pages/getting-started-with-github-pages/creating-a-github-pages-site) Our example app is written in next.js and code is available in [only-peers-client](https://github.com/calimero-network/only-peers-client) After enabling GitHub Pages, a few changes are required in your next.js app. `next.config.mjs` should contain output: "export" field GitHub Actions is a platform used to automate the deployment process. You can find example in [github workflow](https://github.com/calimero-network/only-peers-client/tree/master/.github/workflows) --- ## Install application import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; We have created simple and easy to use example application called `only-peers`. Application enables writing posts and leaving comments. To try out application you need to create new context where application will be installed. ### Create new context To create new context, node CLI has to be used. If the node doesn't have application installed, you need to install the application first. If the application is already installed, you can skip this step. ```bash meroctl --node node1 app install --path /path/to/app ``` :::note If you want to install an application that is published in the registry, check `meroctl --node node1 app install -h` for options ::: After the application is installed, you can create new context: ```bash title="Terminal" meroctl --node node1 context create --application-id 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf --protocol near ``` ```bash title="Terminal" meroctl --node node1 context create --application-id 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf --protocol starknet ``` ```bash title="Terminal" meroctl --node node1 context create --application-id 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf --protocol icp ``` ```bash title="Terminal" meroctl --node node1 context create --application-id 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf --protocol stellar ``` ```bash title="Terminal" meroctl --node node1 context create --application-id 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf --protocol ethereum ``` You are now part of the context and can start using the application. ### Try out Only-peers application We have built and deployed a demo app so you can try it out immediately. Navigate to https://calimero-network.github.io/only-peers-client/ to access app frontend. You will be asked to setup the app by the adding the node url. It is the same url you used while starting the node in [Getting-started](../getting-started/setup) `http://localhost:NODE_PORT` After setting up node url, you will be asked to login. > **_NOTE:_** Use your wallet which you have already added as root key in > [Admin Dashboard](../developer-tools/apps/admin-dashboard) You are now ready to use the app. Enjoy! --- ## Only Peers Only Peers is a decentralized social networking app that empowers you to share your thoughts and engage with your community without compromising your privacy. It allows you to write posts, leave comments, and interact with friends. However, unlike traditional social networks, Only Peers ensures your data remains under your control, protected from central authorities and prying eyes. Experience the freedom of a decentralized platform where your voice cannot be censored and your privacy is prioritized. Explore the source code on [GitHub repository](https://github.com/calimero-network/only-peers-client). --- ## Private DAO An example application that connects to Calimero Proxy Contract and allows creating proposals, voting on them, and executing the approved proposals. ```mermaid graph TD A[Application] -->|Connects to| B[Calimero Proxy Contract] B -->|Handles| C[Proposals] C -->|Create| D[New Proposals] C -->|Approve| E[Proposal Voting] C -->|Delete| J[Delete Proposal] E -->|Check Votes| F{Required Votes Met?} F -->|Yes| G[Execute Proposal] G -->|Perform Actions| H[Blockchain Operations] F -->|No| I[Store Vote Count] ``` Full source code is available in our GitHub [repository](https://github.com/calimero-network/demo-blockchain-integrations). --- ## Building with Internet Computer (ICP) import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; # Building with Internet Computer (ICP) Step-by-Step Guide to Building an Application for Calimero with Internet Computer (ICP) This tutorial will cover the following topics: 1. **Starting a Local Devnet** - Setting and configuring a local devnet using dfx through a script. 2. **Configuring Nodes** - Setting and configuring nodes, installing demo application, creating context and inviting nodes into the context. 3. **Installing a Blockchain Demo Application** - Setup demo application that contains all functionalities for interaction with smart contracts. 4. **Creating a Proposal** - Creating a proposal to execute cross-contract call to "Greet" smart contract deployed in step 3. :::note For easier guidance, we have named each terminal block by their purpose. ::: ## Requirements To follow this tutorial, you'll need the following: - Calimero ICP Devnet - [Repository](https://github.com/calimero-network/icp-devnet) - Calimero Core - Installation instructions - [Instructions](/getting-started/packaged#installation) - Demo Blockchain Integrations - [Repository](https://github.com/calimero-network/demo-blockchain-integrations) You also need following tools: - **Cargo**: Version `1.86.0-nightly` used for tutorial - [url](https://www.rust-lang.org/tools/install) - **DFX** (Dfinity SDK): Version `0.24.3` used for tutorial - [url](https://internetcomputer.org/docs/current/developer-docs/getting-started/install) - **Candid Extractor**: Version `0.1.5` used for tutorial - [url](https://crates.io/crates/candid-extractor) - **Pnpm**: Version `9.6.0` used for tutorial - [url](https://pnpm.io/installation) ## Configuring a Local Devnet We have prepared a script that will deploy the contracts and create needed accounts on local environment. The script does the following: - Creates accounts needed for ledger deployment. - Deploys the context, ledger and example external contract. - Funds the deployed context contract for initial usage. - Calls required initial methods on deployed contracts. Scripts are located in Calimero ICP Devnet repository. First clone the [repository](https://github.com/calimero-network/icp-devnet). ```bash title="ICP Setup Terminal" git clone https://github.com/calimero-network/icp-devnet cd icp-devnet chmod +x ./deploy_devnet_addon.sh ``` The repository contains two scripts: `deploy_devnet_fresh.sh` - Starts fresh dfx environment and deploys the contracts and needed accounts. :::caution This script will delete the current dfx environment and create a new one. ::: `deploy_devnet_addon.sh` - Deploys the contracts and creates needed account on already existing dfx environment. :::note dfx environment needs to be started before running this script. ::: For this tutorial we will be using `deploy_devnet_fresh.sh` script. ```bash title="ICP Setup Terminal" ./deploy_devnet_fresh.sh ``` ```bash title="ICP Setup Terminal | Output of successful deployment" === Deployment Summary === Context Contract ID: bkyz2-fmaaa-aaaaa-qaaaq-cai Ledger Contract ID: bd3sg-teaaa-aaaaa-qaaba-cai Demo External Contract ID: be2us-64aaa-aaaaa-qaabq-cai Account Information: Minting Account: 8b768d662eeebfcbe55b180a7ac0ccb46e2ccd59cacd0b4ec3404f0c8d8b8086 Initial Account: 670183527b941adeae9e1552525853af7812d9441758c668b2e3b8553dead7a0 Archive Principal: kbwrg-ggsyr-4zl47-3z7by-owf4g-draak-xj2ni-mcvwv-6wqxc-nkjam-gae Recipient Principal: mfefj-dsyoh-rb3b2-3yagk-rvb2p-wcb2v-fhu2n-fel2f-wqzjn-yhtxx-hqe Deployment completed successfully! ``` :::note Leave this terminal open as you will need values in later steps. ::: ## Context and Proxy Contract ### Context contract Calimero context contract is used to create and manage contexts and their members. More information about context contract can be found [here](/core-concepts/blockchains-integration#context-contract). To see the context contract implementation, refer to the [calimero-network/core](https://github.com/calimero-network/core/tree/master/contracts/icp/context-config) repository. ### Proxy contract Proxy contract is used to handle blockchain operations such as cross-contract calls and trasnfers as well as storing variables. More information about proxy contract can be found [here](/core-concepts/blockchains-integration#functionality-details). To see the proxy contract implementation, refer to the [calimero-network/core](https://github.com/calimero-network/core/tree/master/contracts/icp/context-proxy) repository. ## Setting up and configuring nodes :::info By deafult proposal needs 3 approvals before it is executed so we will setup 3 nodes. ::: - Initialize and start 3 nodes - Install [blockchain demo application](https://github.com/calimero-network/demo-blockchain-integrations/tree/master/logic) - Create context for blockchain demo application - Invite all nodes in the same context #### Initialize and Start Nodes Open 3 terminals side by side. ```bash title="Node1 Terminal" merod --node-name node1 init --server-port 2427 --swarm-port 2527 ``` ```bash title="Node2 Terminal" merod --node-name node2 init --server-port 2428 --swarm-port 2528 ``` ```bash title="Node3 Terminal" merod --node-name node3 init --server-port 2429 --swarm-port 2529 ``` The output should look like this: You can verify nodes initialization files by looking into `~/.calimero` folder. More on node initialization can be found [here](/getting-started/setup/). Now we are going to start the nodes with the commands: ```bash title="Node1 Terminal" merod --node-name node1 run ``` ```bash title="Node2 Terminal" merod --node-name node2 run ``` ```bash title="Node3 Terminal" merod --node-name node3 run ``` After running the nodes you should see the similar to this one: #### Install application Inside the node1 terminal we are going to install blockchain demo application. 1. Clone the [Demo Blockchain Integrations](https://github.com/calimero-network/demo-blockchain-integrations) repository. 2. Navigate to the `logic` directory. 3. Build the application using the `build.sh` script, script will compile wasm file in /res directory. 4. Copy the `blockchain.wasm` file path Applications are installed with the following command: ```bash title="Node1 Terminal" application install file >Installed application: ``` After installing the application we can create context: ```bash title="Node1 Terminal" context create --protocol icp >Created context with identity ``` After creating context we need to add node2 and node3 to it. To be able to do that we need to create identities for node2 and node3 with whom they will join to the created context. ```bash title="Node2 Terminal" identity new >> Private key: >> Public key: ``` ```bash title="Node3 Terminal" identity new >> Private key: >> Public key: ``` The output should look like this: After creating identities we can invite node2 and node3 to join the created context. ```bash title="Node1 Terminal" context invite --as >> Invitation payload: ``` This will generate a invitation payload which can node2 use to join the context. Copy the invitation payload and do the following command in node2 terminal: ```bash title="Node2 Terminal" context join >> Joined context ``` Repeat the same steps for Node3. ```bash title="Node1 Terminal" context invite --as ``` ```bash title="Node3 Terminal" context join >> Joined context ``` The output in terminal after inviting and joining nodes should look like this: More on invitations and joining the context can be found [here](https://calimero-network.github.io/tutorials/invitations-and-joinings) Lastly we can check if all nodes are connected in context with: ```bash title="Node1 Terminal" context ls > ``` ```bash title="Node1 Terminal" identity ls ``` We can see there are 3 nodes connected in context. #### Fund Proxy Contract We have now installed the application and created context. To be able to fully use proxy contract we need to fund it. To get proxy contract address we need to use API GET call to retrieve it from the node. The 3 running nodes are located on: - https://localhost:2427 - https://localhost:2428 - https://localhost:2429. Any of them can be queried to get the value of proxy contract id. API endpoint to fetch proxy contract id is: `http://localhost:2428/admin-api/contexts/CONTEXT_ID/proxy-contract` > CONTEXT_ID can be copied from previous steps And for our case it is: `http://localhost:2428/admin-api/contexts/7krojDziAKRWP8KrUD8aYGL3z5peScNxPXaBWTKCFr2h/proxy-contract` Example of request in postman: With proxy contract id we can now fund it using dfx native commands. Reopen the terminal you used to execute devnet deployment script and do the following commands: Switch to initial identity. ```bash title="ICP Setup Terminal Terminal" dfx identity use initial >Using identity: "initial". ``` To fund the contract we are going to use command: ```bash dfx canister call icrc1_transfer '( record { to = record { owner = principal ""; subaccount = null; }; amount = 1_000_000_000; fee = null; memo = null; from_subaccount = null; created_at_time = null; } )' ``` > LEDGER_CONTRACT_ID - this value can be viewed when ./deploy_devnet.sh script > finishes. > > In our case value its: `bd3sg-teaaa-aaaaa-qaaba-cai` > > PROXY_CONTRACT_ID - value we got in previous step. > > In our case value its: `b77ix-eeaaa-aaaaa-qaada-cai` ```bash title="ICP Setup Terminal Terminal" dfx canister call bd3sg-teaaa-aaaaa-qaaba-cai icrc1_transfer '( record { to = record { owner = principal "b77ix-eeaaa-aaaaa-qaada-cai"; subaccount = null; }; amount = 1_000_000_000; fee = null; memo = null; from_subaccount = null; created_at_time = null; } )' > (variant { Ok = 1 : nat }) ``` And verify the contract balance: ```bash title="ICP Setup Terminal Terminal" dfx canister call bd3sg-teaaa-aaaaa-qaaba-cai icrc1_balance_of '( record { owner = principal "b77ix-eeaaa-aaaaa-qaada-cai"; subaccount = null; } )' > (1_000_000_000 : nat) ``` ## Blockchain Demo Application This application is used to demonstrate how to interact with Calimero Proxy contract through creating, approving and executing proposals. The proxy contract supports the following types of proposals: - **External function call** - Enables cross-contract call execution. - **Transfer** - Allows transferring funds. - **Set approval threshold** - Specifies the number of approvals required for proposal execution. - **Set active proposal limit** - Defines the maximum number of active proposals. - **Set context variables** - Allows setting key-value pairs for context variables. While the demo focuses on specific functionality, you can customize the application to suit your needs. Instead of building a complete frontend, this demo serves as a foundation that you can extend to create a fully-featured frontend application tailored to your requirements. Navigate to frontend directory and install dependencies. ```bash title="Terminal" cd demo-blockchain-integrations/app && pnpm install ``` Since we have 3 nodes we will need to setup frontend for each of them. Open 3 terminals side by side and run the following commands in each of them from /app directory: ```bash title="Terminal 1 | Terminal 2 | Terminal 3" pnpm run dev ``` After running the frontend you should see the following output in each of the terminals: Open the applications in by clicking on the link in each of the terminals and you will see form with input fields for node URL and application ID. To get application ID you can use following command from any of node terminals: ```bash title="Node1 Terminal" application ls > ``` The node URL of the nodes are respectively: - http://localhost:2427 - http://localhost:2428 - http://localhost:2429 After submitting the form you will be redirected to admin dashboard where you will see the login page with wallet selector. Select a wallet you wish to login with and afterwards you will be prompted to select the context and context identity you wish to use. After selecting both you will be logged in and automatically redirected to the application. ### Change Approval Threshold Proposal First we are going to create an approval threshold change proposal so future proposals can be executed by only 1 node. As mentioned earlier each proposal by default needs 3 approvals to be executed. 1. Click on "Create Proposal" button and select "Change number of approvals needed" from dropdown. 2. In the input field set the number of approvals to 1 (this means that proposal will be executed on creation). 3. Click on "Create Proposal" button to create the proposal. After proposal is created you will get the alert that proposal is created and it will be shown in other frontend applications by selecting it in select field. After selecting proposal on other frontend applications (node2 and node3) you can see that it is created and can be approved by clicking on "Approve Proposal" button. ### Cross-Contract Call Proposal At this point proposal approval threshold is 1, meaning proposal will be executed on creation. This means when we create cross-contract call proposal it will insantly be executed. We have created a demo external contract that contains following methods: - **test_method_no_transfer** - This method does not require a deposit for execution, it sets sent arguments in its storage. - **test_method** - This method requires a deposit for execution, it first transfers the funds to the contract from proxy contract and then sets sent arguments in its storage, if the transfer is completed successfully. - **get_calls** - This method returns all calls stored in the contract. - **clear_state** - This method clears the storage of the contract. You can see the contract itself and its implementation [here](https://github.com/calimero-network/core/tree/master/contracts/icp/context-proxy/mock/external). When creating cross-contract call proposal we need to provide following values: - **Contract ID**: br5f7-7uaaa-aaaaa-qaaca-cai - Address of the external contract - shown in terminal output of dfx deployment script for demo external contract - **Method Name**: test_method_no_transfer - Name of the method that will be called - **Deposit**: 0 - Deposit for transfer, in this case we are not transferring any funds so we can set it to 0 - **Arguments**: someKey: someValue - Key and value pairs for arguments that are sent to the external contract After we create proposal it will be executed, and the success of the execution can be checked by running following command in terminal: ```bash title="DFX terminal" dfx canister call get_calls ``` :::caution It's important to note that in this tutorial, we use a function call that does not require a deposit for execution. On the other hand, there are examples where cross-contract calls require a deposit. For instance, in the case of an ERC20 token implementation, the following steps typically occur: 1. The contract detects that an attached deposit is present. 2. It calls the deployed ledger contract to enable the external contract (the contract being called) to withdraw funds. Before doing so, it sets the allowance to 0 to prevent vulnerabilities related to the Attack Vector. 3. It then calls the ledger contract again to set the allowance equal to the deposit attached to the call. 4. The external contract is called, and it must contain logic to withdraw the funds that have been allocated for it in the ledger contract. 5. After the cross-contract call is completed, the proxy contract sets the allowance back to 0 to ensure security. ::: ## Conclusion This tutorial has covered all the essential steps developers need to build an application for Calimero and ICP, from setting up the environment to deploying a fully functional smart contract. By following this guide, you should now have a solid understanding of the development workflow and how to build your own ICP application for Calimero. Feel free to use our demo applications as a starting point to build and customize your own application. If you have any questions or need assistance, don't hesitate to reach out to us on Socials. We're here to help! --- ## Building with Stellar import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; # Building with Stellar Step-by-Step Guide to Building an Application for Calimero with Stellar This tutorial will cover the following topics: 1. **Smart contract overview** - Overview of the smart contracts and their functionality. 2. **Configuring Nodes** - Setting and configuring nodes, installing demo application, creating context and inviting nodes into the context. 3. **Installing a Blockchain Demo Application** - Setup demo application that contains all functionalities for interaction with smart contracts. 4. **Creating a Proposal** - Creating a proposal to execute cross-contract call to "Greet" smart contract deployed in step 3. :::note For easier guidance, we have named each terminal block by their purpose. ::: ## Requirements To follow this tutorial, you'll need the following: - Calimero Core - Installation instructions - [Instructions](/getting-started/packaged#installation) - Demo Blockchain Integrations - [Repository](https://github.com/calimero-network/demo-blockchain-integrations) You also need following tools: - **Cargo**: Version `1.86.0-nightly` used for tutorial - [url](https://www.rust-lang.org/tools/install) - **Pnpm**: Version `9.6.0` used for tutorial - [url](https://pnpm.io/installation) - **Stellar CLI** - Installation instructions - [Instructions](https://developers.stellar.org/docs/build/smart-contracts/getting-started/setup#install-the-stellar-cli) ## Smart contract overview ### Contracts overview Both contracts are written in Rust, you can find their implementations in the following repositories: - [Context contract](https://github.com/calimero-network/core/tree/master/contracts/stellar/context-config) - [Proxy contract](https://github.com/calimero-network/core/tree/master/contracts/stellar/context-proxy) ## Context and Proxy Contract ### Context contract Context contract is deployed on Stellar testnet and implemented into Calimero. It is used to create and manage contexts and their members. More information about context contract can be found [here](/core-concepts/blockchains-integration#context-contract). ### Proxy contract With each context creation a proxy contract for that context is deployed. Proxy contract is used to handle blockchain operations such as cross-contract calls and transfers as well as storing variables. More information about proxy contract can be found [here](/core-concepts/blockchains-integration#functionality-details). ## Setting up and configuring nodes :::info By deafult proposal needs 3 approvals before it is executed so we will setup 3 nodes. ::: - Initialize and start 3 nodes - Install [blockchain demo application](https://github.com/calimero-network/demo-blockchain-integrations/tree/master/logic) - Create context for blockchain demo application - Invite all nodes in the same context #### Initialize and Start Nodes Open 3 terminals side by side. ```bash title="Node1 Terminal" merod --node-name node1 init --server-port 2427 --swarm-port 2527 ``` ```bash title="Node2 Terminal" merod --node-name node2 init --server-port 2428 --swarm-port 2528 ``` ```bash title="Node3 Terminal" merod --node-name node3 init --server-port 2429 --swarm-port 2529 ``` The output should look like this: You can verify nodes initialization files by looking into `~/.calimero` folder. :::note Self signer vs. Relayer signer In `config.toml` file you can see default configuration: ``` [context.config.stellar] signer = "self" protocol = "stellar" network = "testnet" contract_id = ``` If you use `self` for signer then you also need to change `public` and `secret` key. ``` [context.config.signer.self.stellar.testnet] rpc_url = "https://soroban-testnet.stellar.org/" public_key = secret_key = ``` The values auto-generated only represent the format needed for self signing feature and are not real Stellar accounts. To create your own Stellar account with CLI follow the Stellar [Docs](https://developers.stellar.org/docs/build/guides/cli) If signer is changed to `signer = "relayer"` you don't need to changed anything as relayer will handle all the blockchain operations. ::: More on node initialization can be found [here](/getting-started/setup/). Now we are going to start the nodes with the commands: ```bash title="Node1 Terminal" merod --node-name node1 run ``` ```bash title="Node2 Terminal" merod --node-name node2 run ``` ```bash title="Node3 Terminal" merod --node-name node3 run ``` After running the nodes you should see the similar to this one: #### Install application Inside the node1 terminal we are going to install blockchain demo application. 1. Clone the [Demo Blockchain Integrations](https://github.com/calimero-network/demo-blockchain-integrations) repository. 2. Navigate to the `logic` directory. 3. Build the application using the `build.sh` script, script will compile wasm file in /res directory. 4. Copy the `blockchain.wasm` file path Applications are installed with the following command: ```bash title="Node1 Terminal" application install file >Installed application: ``` After installing the application we can create context: ```bash title="Node1 Terminal" context create --protocol stellar >Created context with identity ``` After creating context we need to add node2 and node3 to it. To be able to do that we need to create identities for node2 and node3 with whom they will join to the created context. ```bash title="Node2 Terminal" identity new >> Private key: >> Public key: ``` ```bash title="Node3 Terminal" identity new >> Private key: >> Public key: ``` The output should look like this: After creating identities we can invite node2 and node3 to join the created context. ```bash title="Node1 Terminal" context invite --as >> Invitation payload: ``` This will generate a invitation payload which can node2 use to join the context. Copy the invitation payload and do the following command in node2 terminal: ```bash title="Node2 Terminal" context join >> Joined context ``` Repeat the same steps for Node3. ```bash title="Node1 Terminal" context invite --as ``` ```bash title="Node3 Terminal" context join >> Joined context ``` The output in terminal after inviting and joining nodes should look like this: More on invitations and joining the context can be found [here](https://calimero-network.github.io/tutorials/invitations-and-joinings) Lastly we can check if all nodes are connected in context with: ```bash title="Node1 Terminal" context ls > ``` ```bash title="Node1 Terminal" identity ls ``` We can see there are 3 nodes connected in context. #### Fund Proxy Contract We have now installed the application and created context. To be able to fully use proxy contract we need to fund it. To get proxy contract address we need to use API GET call to retrieve it from the node. The 3 running nodes are located on: - https://localhost:2427 - https://localhost:2428 - https://localhost:2429. Any of them can be queried to get the value of proxy contract id. API endpoint to fetch proxy contract id is: `http://localhost:2428/admin-api/contexts/CONTEXT_ID/proxy-contract` > CONTEXT_ID can be copied from previous steps And for our case it is: `http://localhost:2428/admin-api/contexts/7krojDziAKRWP8KrUD8aYGL3z5peScNxPXaBWTKCFr2h/proxy-contract` Example of request in postman: With proxy contract id we can now fund it using Stellar CLI. To be able to fund the contract we need to create a testnet account and seed it with tokens. You can create a testnet account and automatically fund it by using Stellar CLI: ```bash stellar keys generate --global --network testnet --fund ``` Where: - `` is the name of the account you want to create. To get the account address you can use the following command: ```bash stellar keys address > ``` Now you can fund the contract by using the following command: ```bash stellar contract invoke \ --id CDLZFC3SYJYDZT7K67VZ75HPJVIEUVNIXF47ZG2FB2RMQQVU2HHGCYSC \ --source \ --network testnet \ -- transfer \ --from \ --to \ --amount 100000000 ``` Where: - `` is the name of the account for signing the transaction. - `CDLZFC3SYJYDZT7K67VZ75HPJVIEUVNIXF47ZG2FB2RMQQVU2HHGCYSC` is the address of the Stellar testnet ledger contract. - `` is the address of the account you created. - `` is the address of the proxy contract you want to fund. - `100000000` is the amount of lumens you want to fund the contract with. (1 XLM = 10000000) To verify contract balance you can check it on Stellar Explorer by visiting: `https://stellar.expert/explorer/testnet/contract/` ## Blockchain Demo Application This application is used to demonstrate how to interact with Calimero Proxy contract through creating, approving and executing proposals. The proxy contract supports the following types of proposals: - **External function call** - Enables cross-contract call execution. - **Transfer** - Allows transferring funds. - **Set approval threshold** - Specifies the number of approvals required for proposal execution. - **Set active proposal limit** - Defines the maximum number of active proposals. - **Set context variables** - Allows setting key-value pairs for context variables. While the demo focuses on specific functionality, you can customize the application to suit your needs. Instead of building a complete frontend, this demo serves as a foundation that you can extend to create a fully-featured frontend application tailored to your requirements. Navigate to frontend directory and install dependencies. ```bash title="Terminal" cd demo-blockchain-integrations/app && pnpm install ``` Since we have 3 nodes we will need to setup frontend for each of them. Open 3 terminals side by side and run the following commands in each of them from /app directory: ```bash title="Terminal 1 | Terminal 2 | Terminal 3" pnpm run dev ``` After running the frontend you should see the following output in each of the terminals: Open the applications in by clicking on the link in each of the terminals and you will see form with input fields for node URL and application ID. To get application ID you can use following command from any of node terminals: ```bash title="Node1 Terminal" application ls > ``` The node URL of the nodes are respectively: - http://localhost:2427 - http://localhost:2428 - http://localhost:2429 After submitting the form you will be redirected to admin dashboard where you will see the login page with wallet selector. Select a wallet you wish to login with and afterwards you will be prompted to select the context and context identity you wish to use. After selecting both you will be logged in and automatically redirected to the application. ### Change Approval Threshold Proposal First we are going to create an approval threshold change proposal so future proposals can be executed by only 1 node. As mentioned earlier each proposal by default needs 3 approvals to be executed. 1. Click on "Create Proposal" button and select "Change number of approvals needed" from dropdown. 2. In the input field set the number of approvals to 1 (this means that proposal will be executed on creation). 3. Click on "Create Proposal" button to create the proposal. After proposal is created you will get the alert that proposal is created and it will be shown in other frontend applications by selecting it in select field. After selecting proposal on other frontend applications (node2 and node3) you can see that it is created and can be approved by clicking on "Approve Proposal" button. ### Cross-Contract Call Proposal At this point proposal approval threshold is 1, meaning proposal will be executed on creation. This means when we create cross-contract call proposal it will instantly be executed. We have created a `` and deployed it on Stellar testnet. Its address is: ``` CAJF5BPK4AT7UNVF4NDXMUXDKXCBFV56H2TNMUJIX6WUSNFP4UMH7W46 ``` It contains following methods: - **no_deposit** - This method does not require a deposit for execution, it sets sent arguments in its storage. - **deposit** - This method requires a deposit for execution, it first transfers the funds to the contract from proxy contract and then sets sent arguments in its storage, if the transfer is completed successfully. - **get_calls** - This method returns all calls stored in the contract. - **clear_state** - This method clears the storage of the contract. You can see the contract itself and its implementation [here](https://github.com/calimero-network/core/tree/master/contracts/stellar/context-proxy/mock_external). When creating cross-contract call proposal we need to provide following values: - **Contract ID**: `{DEMO_EXTERNAL_CONTRACT_ID}` - Address of the external contract - **Method Name**: `deposit` - Name of the contract method that will be called - **Deposit**: 10000000 - Deposit for transfer (1 XLM) - **Arguments**: The following arguments correspond to the deposit method parameters: | Type | Value | | ------- | --------------------- | | address | `{PROXY_CONTRACT_ID}` | | i128 | 10000000 | | string | testKEY | | string | testVALUE | Supported Soroban argument types: - `i32` - 32-bit signed integer - `u32` - 32-bit unsigned integer - `i64` - 64-bit signed integer - `u64` - 64-bit unsigned integer - `i128` - 128-bit signed integer - `u128` - 128-bit unsigned integer - `bool` - Boolean value - `string` - Text string - `address` - Contract or account address - `symbol` - Symbol value - `bytes` - Byte array After we create proposal it will be executed, and the success of the execution can be checked calling demo external contract method **get_value** with Stellar CLI: ```bash stellar contract invoke \ --id \ --network testnet \ --source \ -- \ get_value \ --key "testKEY" ``` To check if transfer was successful we can use Stellar Explorer by visiting: `https://stellar.expert/explorer/testnet/contract/` There you will see the current funds attached to the contract. :::caution The implementation of cross-contract call with deposit is based on the following steps: 1. The contract detects that an attached deposit is present. 2. It calls the ledger contract to enable the external contract (the contract being called) to withdraw funds. 3. The external contract is called, and it must contain logic to withdraw the funds that have been allocated for it in the ledger contract. 4. After the cross-contract call is completed, the proxy contract sets the allowance back to 0 to ensure security. ::: ## Conclusion This tutorial has covered all the essential steps developers need to build an application for Calimero and Stellar, from environment overview to deploying a fully functional smart contract. By following this guide, you should now have a solid understanding of the development workflow and how to build your own Stellar application for Calimero. Feel free to use our demo applications as a starting point to build and customize your own application. If you have any questions or need assistance, don't hesitate to reach out to us on Socials. We're here to help! --- ## Building with Ethereum import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; # Building with Ethereum Step-by-Step Guide to Building an Application for Calimero with Ethereum This tutorial will cover the following topics: 1. **Smart contract overview** - Overview of the smart contracts and their functionality. 2. **Configuring Nodes** - Setting and configuring nodes, installing demo application, creating context and inviting nodes into the context. 3. **Installing a Blockchain Demo Application** - Setup demo application that contains all functionalities for interaction with smart contracts. 4. **Creating a Proposal** - Creating a proposal to execute cross-contract call to an external smart contract deployed in step 3. :::note For easier guidance, we have named each terminal block by their purpose. ::: ## Requirements To follow this tutorial, you'll need the following: - Calimero Core - Installation instructions - [Instructions](/getting-started/packaged#installation) - Demo Blockchain Integrations - [Repository](https://github.com/calimero-network/demo-blockchain-integrations) You also need following tools: - **Cargo**: Version `1.86.0-nightly` used for tutorial - [url](https://www.rust-lang.org/tools/install) - **Pnpm**: Version `9.6.0` used for tutorial - [url](https://pnpm.io/installation) - **Foundry**: For local Ethereum development - [Instructions](https://book.getfoundry.sh/getting-started/installation) ## Smart contract overview ### Contracts overview Both contracts are written in Solidity, you can find their implementations in the following repositories: - [Context contract](https://github.com/calimero-network/contracts/tree/master/contracts/ethereum/context-config) - [Proxy contract](https://github.com/calimero-network/contracts/tree/master/contracts/ethereum/context-proxy) ## Context and Proxy Contract ### Context contract Context contract is deployed on Ethereum sepolia network and implemented into Calimero. It is used to create and manage contexts and their members. More information about context contract can be found [here](/core-concepts/blockchains-integration#context-contract). ### Proxy contract With each context creation a proxy contract for that context is deployed. Proxy contract is used to handle blockchain operations such as cross-contract calls and transfers as well as storing variables. More information about proxy contract can be found [here](/core-concepts/blockchains-integration#functionality-details). ## Setting up and configuring nodes :::info By deafult proposal needs 3 approvals before it is executed so we will setup 3 nodes. ::: - Initialize and start 3 nodes - Install [blockchain demo application](https://github.com/calimero-network/demo-blockchain-integrations/tree/master/logic) - Create context for blockchain demo application - Invite all nodes in the same context #### Initialize and Start Nodes Open 3 terminals side by side. ```bash title="Node1 Terminal" merod --node-name node1 init --server-port 2427 --swarm-port 2527 ``` ```bash title="Node2 Terminal" merod --node-name node2 init --server-port 2428 --swarm-port 2528 ``` ```bash title="Node3 Terminal" merod --node-name node3 init --server-port 2429 --swarm-port 2529 ``` The output should look like this: You can verify nodes initialization files by looking into `~/.calimero` folder. :::note Self signer vs. Relayer signer In `config.toml` file you can see default configuration: ``` [context.config.ethereum] signer = "relayer" protocol = "ethereum" network = "sepolia" contract_id = ``` If you use `self` for signer then you also need to change `public` and `secret` key. ``` [context.config.signer.self.ethereum.testnet] rpc_url = account_id = secret_key = ``` The values auto-generated only represent the format needed for self signing feature and are not real Ethereum accounts. To create your own Ethereum account with CLI follow the Foundry [Docs](https://book.getfoundry.sh/reference/cast/cast-wallet-new) ::: More on node initialization can be found [here](/getting-started/setup/). Now we are going to start the nodes with the commands: ```bash title="Node1 Terminal" merod --node-name node1 run ``` ```bash title="Node2 Terminal" merod --node-name node2 run ``` ```bash title="Node3 Terminal" merod --node-name node3 run ``` After running the nodes you should see the similar to this one: #### Install application Inside the node1 terminal we are going to install blockchain demo application. 1. Clone the [Demo Blockchain Integrations](https://github.com/calimero-network/demo-blockchain-integrations) repository. 2. Navigate to the `logic` directory. 3. Build the application using the `build.sh` script, script will compile wasm file in /res directory. 4. Copy the `blockchain.wasm` file path Applications are installed with the following command: ```bash title="Node1 Terminal" application install file >Installed application: ``` After installing the application we can create context: ```bash title="Node1 Terminal" context create --protocol ethereum >Created context with identity ``` After creating context we need to add node2 and node3 to it. To be able to do that we need to create identities for node2 and node3 with whom they will join to the created context. ```bash title="Node2 Terminal" identity new >> Private key: >> Public key: ``` ```bash title="Node3 Terminal" identity new >> Private key: >> Public key: ``` The output should look like this: After creating identities we can invite node2 and node3 to join the created context. ```bash title="Node1 Terminal" context invite --as >> Invitation payload: ``` This will generate a invitation payload which can node2 use to join the context. Copy the invitation payload and do the following command in node2 terminal: ```bash title="Node2 Terminal" context join >> Joined context ``` Repeat the same steps for Node3. ```bash title="Node1 Terminal" context invite --as ``` ```bash title="Node3 Terminal" context join >> Joined context ``` The output in terminal after inviting and joining nodes should look like this: More on invitations and joining the context can be found [here](https://calimero-network.github.io/tutorials/invitations-and-joinings) Lastly we can check if all nodes are connected in context with: ```bash title="Node1 Terminal" context ls > ``` ```bash title="Node1 Terminal" identity ls ``` We can see there are 3 nodes connected in context. #### Fund Proxy Contract We have now installed the application and created context. To be able to fully use proxy contract we need to fund it. To get proxy contract address we need to use API GET call to retrieve it from the node. The 3 running nodes are located on: - https://localhost:2427 - https://localhost:2428 - https://localhost:2429. Any of them can be queried to get the value of proxy contract id. API endpoint to fetch proxy contract id is: `http://localhost:2428/admin-api/contexts/CONTEXT_ID/proxy-contract` > CONTEXT_ID can be copied from previous steps And for our case it is: `http://localhost:2428/admin-api/contexts/7krojDziAKRWP8KrUD8aYGL3z5peScNxPXaBWTKCFr2h/proxy-contract` Example of request in postman: Now you can fund the proxy contract by using the following command: ```bash cast send \ --value 1eth \ --private-key \ ``` Where: - `` is the account from which to transfer the funds. - `` is the address of the proxy contract you want to fund. - `1eth` is the amount of eth you want to fund the contract with. For more information about transferring funds checkout this [link](https://book.getfoundry.sh/reference/cast/cast-send) To verify contract balance you can check it on Etherscan Explorer by visiting: `https://sepolia.etherscan.io/address/` ## Blockchain Demo Application This application is used to demonstrate how to interact with Calimero Proxy contract through creating, approving and executing proposals. The proxy contract supports the following types of proposals: - **External function call** - Enables cross-contract call execution. - **Transfer** - Allows transferring funds. - **Set approval threshold** - Specifies the number of approvals required for proposal execution. - **Set active proposal limit** - Defines the maximum number of active proposals. - **Set context variables** - Allows setting key-value pairs for context variables. While the demo focuses on specific functionality, you can customize the application to suit your needs. Instead of building a complete frontend, this demo serves as a foundation that you can extend to create a fully-featured frontend application tailored to your requirements. Navigate to frontend directory and install dependencies. ```bash title="Terminal" cd demo-blockchain-integrations/app && pnpm install ``` Since we have 3 nodes we will need to setup frontend for each of them. Open 3 terminals side by side and run the following commands in each of them from /app directory: ```bash title="Terminal 1 | Terminal 2 | Terminal 3" pnpm run dev ``` After running the frontend you should see the following output in each of the terminals: Open the applications in by clicking on the link in each of the terminals and you will see form with input fields for node URL and application ID. To get application ID you can use following command from any of node terminals: ```bash title="Node1 Terminal" application ls > ``` The node URL of the nodes are respectively: - http://localhost:2427 - http://localhost:2428 - http://localhost:2429 After submitting the form you will be redirected to admin dashboard where you will see the login page with wallet selector. Select a wallet you wish to login with and afterwards you will be prompted to select the context and context identity you wish to use. After selecting both you will be logged in and automatically redirected to the application. ### Change Approval Threshold Proposal First we are going to create an approval threshold change proposal so future proposals can be executed by only 1 node. As mentioned earlier each proposal by default needs 3 approvals to be executed. 1. Click on "Create Proposal" button and select "Change number of approvals needed" from dropdown. 2. In the input field set the number of approvals to 1 (this means that proposal will be executed on creation). 3. Click on "Create Proposal" button to create the proposal. After proposal is created you will get the alert that proposal is created and it will be shown in other frontend applications by selecting it in select field. After selecting proposal on other frontend applications (node2 and node3) you can see that it is created and can be approved by clicking on "Approve Proposal" button. ### Cross-Contract Call Proposal At this point proposal approval threshold is 1, meaning proposal will be executed on creation. This means when we create cross-contract call proposal it will instantly be executed. We have created a `` and deployed it on Ethereum sepolia. Its address is: ``` 0x2c42DADf80b1a76Db4038027D7DB24863a9d0d5D ``` It contains following methods: - **setValueNoDeposit** - This method does not require a deposit for execution, it sets sent arguments in its storage. - **deposit** - This method requires a deposit for execution, it first transfers the funds to the contract from proxy contract and then sets sent arguments in its storage, if the transfer is completed successfully. - **getValue** - This method returns all calls stored in the contract. You can see the contract itself and its implementation [here](https://github.com/calimero-network/calimero-evm-contracts/tree/master/context-proxy/mock). When creating cross-contract call proposal we need to provide following values: - **Contract ID**: `{DEMO_EXTERNAL_CONTRACT_ID}` - Address of the external contract - **Method Name**: `deposit(string, string)` - Name of the contract method that will be called with arguments types - **Deposit**: 10000000 - Deposit for transfer in WEI (10¹⁸ WEI = 1 ETH) - **Arguments**: The following arguments correspond to the deposit method parameters: | Type | Value | | ------ | --------- | | string | testKEY | | string | testVALUE | Supported Ethereum argument types: - `bool` - Boolean value, representing true or false. - `string` - Text string, used for storing sequences of characters. - `address` - Contract or account address, a 20-byte value used to identify accounts. - `bytes` - Byte array, used for arbitrary-length binary data. - `int256` - 256-bit signed integer, used for storing large integer values. - `uint256` - 256-bit unsigned integer, used for storing large non-negative integer values. - `array(bool)` - Array of boolean values. - `array(string)` - Array of strings. - `array(address)` - Array of addresses. - `array(bytes)` - Array of byte arrays. - `array(int256)` - Array of 256-bit signed integers. - `array(uint256)` - Array of 256-bit unsigned integers. - `tuple` - A fixed-size collection of values, which can be of different types. These types are used to define the parameters and return values of smart contract functions in Ethereum-based blockchains. After we create proposal it will be executed, and the success of the execution can be checked calling demo external contract method **get_value** with Forge CLI: ```bash cast call \ \ "getValue(string)" \ "testKEY" ``` To check if transfer was successful we can use Etherscan Explorer by visiting: `https://sepolia.etherscan.io/address/` There you will see the current funds attached to the contract. :::caution The implementation of cross-contract call with deposit is based on the following steps: 1. The contract detects that an attached deposit is present. 2. It calls the ledger contract to enable the external contract (the contract being called) to withdraw funds. 3. The external contract is called, and it must contain logic to withdraw the funds that have been allocated for it in the ledger contract. 4. After the cross-contract call is completed, the proxy contract sets the allowance back to 0 to ensure security. ::: ## Conclusion This tutorial has covered all the essential steps developers need to build an application for Calimero and Ethereum, from environment overview to deploying a fully functional smart contract. By following this guide, you should now have a solid understanding of the development workflow and how to build your own Ethereum application for Calimero. Feel free to use our demo applications as a starting point to build and customize your own application. If you have any questions or need assistance, don't hesitate to reach out to us on Socials. We're here to help! --- ## Rock Paper Scissors A popular game we all know and love, but this time played online and enforcing complete security using the commit - reveal scheme. After you pick your choice of either rock, paper or scissors, you encrypt your choice with a password. This ensures the other player can’t see your choice. After your opponent has also decided, it is time to reveal your password and determine the winner. You cannot change your mind between choosing and revealing because your choice is locked with the special password. This prevents any bad actors from changing their minds in the middle of the game.Have fun with a classic game that is completely fair and private! Explore the source code on [GitHub repository](https://github.com/calimero-network/rock-paper-scissors) --- ## Blockchain interaction with Proxy Contract ## Overview The Calimero proxy contract enables cross-chain interactions through a proposal-based system. The integration consists of: 1. Backend Application (Rust) 2. Frontend Application (TypeScript/React) :::tip More info on blockchain integrations can be found in the [Blockchains Integration](../core-concepts/blockchains-integration) section. ::: ## Backend Implementation The backend contract handles proposal creation and management. Here's the core functionality: ```rust #[app::logic] impl AppState { pub fn create_new_proposal(&mut self, request: CreateProposalRequest) -> Result { // Implementation handles different types of proposals: // 1. ExternalFunctionCall // 2. Transfer // 3. SetContextValue // 4. SetNumApprovals // 5. SetActiveProposalsLimit } pub fn approve_proposal(&self, proposal_id: ProposalId) -> Result { Self::external().approve(proposal_id); env::emit(&Event::ApprovedProposal { id: proposal_id }); Ok(()) } } ``` ## API Types Define your types for the integration: ```typescript // types.ts export enum ProposalActionType { ExternalFunctionCall = 'ExternalFunctionCall', Transfer = 'Transfer', SetNumApprovals = 'SetNumApprovals', SetActiveProposalsLimit = 'SetActiveProposalsLimit', SetContextValue = 'SetContextValue', } export interface CreateProposalRequest { action_type: ProposalActionType; params: { receiver_id?: string; method_name?: string; args?: string; deposit?: string; gas?: string; amount?: string; num_approvals?: number; active_proposals_limit?: number; key?: string; value?: string; }; } ``` ## API Implementation Create a data source to handle API calls: ```typescript // LogicApiDataSource.ts export class LogicApiDataSource implements ClientApi { async createProposal( request: CreateProposalRequest, ): ApiResponse { const params: RpcQueryParams = { contextId: getContextId(), method: ClientMethod.CREATE_PROPOSAL, argsJson: { request }, executorPublicKey: jwtObject.executor_public_key, }; return await getJsonRpcClient().execute(params, config); } async approveProposal( request: ApproveProposalRequest, ): ApiResponse { // Implementation } } ``` ## Creating Proposals Here are examples of creating different types of proposals: ### 1. External Function Call ```typescript const request: CreateProposalRequest = { action_type: 'ExternalFunctionCall', params: { receiver_id: 'contract.near', method_name: 'transfer', args: JSON.stringify({ amount: '100' }), deposit: '1000000000000000000000', // 1 NEAR gas: '30000000000000', // 30 TGas }, }; ``` ### 2. Token Transfer ```typescript const request: CreateProposalRequest = { action_type: 'Transfer', params: { receiver_id: 'recipient.near', amount: '1000000000000000000000', // 1 NEAR }, }; ``` ### 3. Set Context Value ```typescript const request: CreateProposalRequest = { action_type: 'SetContextValue', params: { key: 'my_key', value: 'my_value', }, }; ``` ### 4. Set Number of Approvals ```typescript const request: CreateProposalRequest = { action_type: 'SetNumApprovals', params: { num_approvals: 3, }, }; ``` ### 5. Set Active Proposals Limit ```typescript const request: CreateProposalRequest = { action_type: 'SetActiveProposalsLimit', params: { active_proposals_limit: 10, }, }; ``` ## Approving Proposals To approve a proposal: ```typescript const approvalRequest: ApproveProposalRequest = { proposal_id: 'proposal-id-here', }; await logicApiDataSource.approveProposal(approvalRequest); ``` ## Error Handling The implementation includes comprehensive error handling: ```typescript try { const result = await logicApiDataSource.createProposal(request); if (result?.error) { console.error('Error:', result.error); // Handle error appropriately } } catch (error) { console.error('Unexpected error:', error); // Handle unexpected errors } ``` ## Best Practices 1. **Input Validation** - Validate all parameters before sending - Use appropriate types for amounts (strings for large numbers) - Format JSON strings properly 2. **Error Handling** - Implement proper error handling - Log important operations - Handle all possible error cases 3. **Gas Management** - Use appropriate gas limits - Default to 30 TGas for NEAR - Monitor gas usage Need help? Join our [Discord](https://discord.gg/calimero) or check our [GitHub](https://github.com/calimero-network/docs). --- ## Python Client Integration # Python Client Integration This tutorial walks you through integrating the Calimero Python client SDK into your applications, from basic setup to advanced patterns and real-world use cases. ## Prerequisites Before starting this tutorial, ensure you have: - **Python 3.9+** installed - **pipx** package manager (recommended) or **pip** - **Basic Python knowledge** (error handling, function calls) - **Calimero node** running locally or remotely - **Basic understanding** of Calimero concepts (contexts, applications, functions) ## Step 1: Installation and Setup ### Install the Python Client ```bash pipx install calimero-client-py pipx ensurepath ``` ### Verify Installation ```python # test_installation.py import calimero def test_installation(): """Test that the Python client is properly installed.""" try: connection = calimero.create_connection("http://localhost:2428") client = calimero.create_client(connection) print("✓ Python client installed successfully!") return True except Exception as e: print(f"✗ Installation test failed: {e}") return False if __name__ == "__main__": test_installation() ``` Run the test: ```bash python test_installation.py ``` ## Step 2: Basic Connection and Operations ### Create a Connection ```python # basic_connection.py import calimero def basic_connection_example(): """Basic connection and operation example.""" # Create connection connection = calimero.create_connection( api_url="http://localhost:2428", node_name="example-node" ) # Create client client = calimero.create_client(connection) # Test basic operations print("Testing basic operations...") # Get peers count peers_count = client.get_peers_count() print(f"Connected peers: {peers_count}") # List contexts contexts = client.list_contexts() print(f"Found {len(contexts)} contexts") # List applications apps = client.list_applications() print(f"Found {len(apps)} applications") return client if __name__ == "__main__": basic_connection_example() ``` ### Error Handling ```python # error_handling.py import calimero def error_handling_example(): """Example of proper error handling.""" try: connection = calimero.create_connection("http://localhost:2428") client = calimero.create_client(connection) # This might fail if the context doesn't exist context = client.get_context("non-existent-context") print(f"Context: {context}") except RuntimeError as e: print(f"Client error: {e}") except Exception as e: print(f"Unexpected error: {e}") if __name__ == "__main__": error_handling_example() ``` ## Step 3: Application Management ### Install and Manage Applications ```python # application_management.py import calimero def application_management_example(): """Example of managing applications.""" connection = calimero.create_connection("http://localhost:2428") client = calimero.create_client(connection) print("=== Application Management Example ===") # Install an application print("Installing application...") app = client.install_application( url="https://example.com/my-app.wasm", metadata=b'{"name": "My Test App", "version": "1.0.0"}' ) app_id = app["application_id"] print(f"✓ Installed app: {app_id}") # Get application details app_info = client.get_application(app_id) print(f"✓ App details: {app_info}") # List all applications apps = client.list_applications() print(f"✓ Total applications: {len(apps)}") # Uninstall the application print("Uninstalling application...") result = client.uninstall_application(app_id) print(f"✓ Uninstall result: {result}") return app_id if __name__ == "__main__": application_management_example() ``` ### Development Application Installation ```python # dev_application.py import calimero def dev_application_example(): """Example of installing development applications.""" connection = calimero.create_connection("http://localhost:2428") client = calimero.create_client(connection) print("=== Development Application Example ===") # Install development application from local path app = client.install_dev_application( path="/path/to/your/app.wasm", metadata=b'{"name": "Dev App", "version": "0.1.0"}' ) app_id = app["application_id"] print(f"✓ Installed dev app: {app_id}") # Clean up client.uninstall_application(app_id) print("✓ Cleaned up dev app") if __name__ == "__main__": dev_application_example() ``` ## Step 4: Context Management ### Create and Manage Contexts ```python # context_management.py import calimero def context_management_example(): """Example of managing contexts.""" connection = calimero.create_connection("http://localhost:2428") client = calimero.create_client(connection) print("=== Context Management Example ===") # First, install an application print("Installing application...") app = client.install_application( url="https://example.com/my-app.wasm", metadata=b'{"name": "Context Test App"}' ) app_id = app["application_id"] try: # Create a context print("Creating context...") context = client.create_context( application_id=app_id, protocol="near", params='{"network": "testnet"}' ) context_id = context["context_id"] print(f"✓ Created context: {context_id}") # Get context details context_info = client.get_context(context_id) print(f"✓ Context info: {context_info}") # Sync the context print("Syncing context...") sync_result = client.sync_context(context_id) print(f"✓ Sync result: {sync_result}") # Get context identities identities = client.get_context_identities(context_id) print(f"✓ Context identities: {identities}") # Get context storage storage = client.get_context_storage(context_id) print(f"✓ Context storage: {storage}") return context_id finally: # Clean up print("Cleaning up...") client.delete_context(context_id) client.uninstall_application(app_id) print("✓ Cleanup completed") if __name__ == "__main__": context_management_example() ``` ## Step 5: Function Execution ### Execute Smart Contract Functions ```python # function_execution.py import calimero def function_execution_example(): """Example of executing smart contract functions.""" connection = calimero.create_connection("http://localhost:2428") client = calimero.create_client(connection) print("=== Function Execution Example ===") # Install application app = client.install_application( url="https://example.com/my-app.wasm", metadata=b'{"name": "Function Test App"}' ) app_id = app["application_id"] try: # Create context context = client.create_context(app_id, "near") context_id = context["context_id"] print(f"✓ Created context: {context_id}") # Generate executor identity executor_identity = client.generate_context_identity() executor_public_key = executor_identity["public_key"] print(f"✓ Generated executor identity") # Execute a simple function print("Executing function...") result = client.execute_function( context_id=context_id, method="initialize", args='{}', executor_public_key=executor_public_key ) print(f"✓ Function result: {result}") # Execute function with complex arguments print("Executing function with arguments...") transfer_result = client.execute_function( context_id=context_id, method="transfer", args='{"amount": 100, "to": "alice.near", "memo": "Payment"}', executor_public_key=executor_public_key ) print(f"✓ Transfer result: {transfer_result}") return context_id finally: # Clean up client.delete_context(context_id) client.uninstall_application(app_id) print("✓ Cleanup completed") if __name__ == "__main__": function_execution_example() ``` ## Step 6: Best Practices ### Connection Management ```python # connection_management.py import calimero class CalimeroClientWrapper: """Wrapper class for better connection management.""" def __init__(self, api_url: str, node_name: str = None): self.connection = calimero.create_connection(api_url, node_name) self.client = calimero.create_client(self.connection) def get_connection_info(self): """Get connection information.""" return { "api_url": self.client.get_api_url(), "peers_count": self.client.get_peers_count() } def health_check(self): """Perform a health check.""" try: contexts = self.client.list_contexts() return {"status": "healthy", "contexts_count": len(contexts)} except Exception as e: return {"status": "unhealthy", "error": str(e)} # Usage example def connection_management_example(): """Example of proper connection management.""" wrapper = CalimeroClientWrapper("http://localhost:2428", "my-node") # Get connection info info = wrapper.get_connection_info() print(f"Connection info: {info}") # Health check health = wrapper.health_check() print(f"Health status: {health}") if __name__ == "__main__": connection_management_example() ``` ### Error Handling Patterns ```python # error_patterns.py import calimero import time def robust_client_operation(client, operation, *args, max_retries=3, **kwargs): """Execute operation with retry logic.""" last_error = None for attempt in range(max_retries): try: return operation(*args, **kwargs) except RuntimeError as e: last_error = e if attempt < max_retries - 1: print(f"Attempt {attempt + 1} failed, retrying in 2s: {e}") time.sleep(2) else: print(f"All {max_retries} attempts failed") break raise last_error def error_patterns_example(): """Example of robust error handling patterns.""" connection = calimero.create_connection("http://localhost:2428") client = calimero.create_client(connection) # Use robust operation wrapper try: contexts = robust_client_operation(client, client.list_contexts) print(f"Successfully retrieved {len(contexts)} contexts") except RuntimeError as e: print(f"Failed to retrieve contexts: {e}") if __name__ == "__main__": error_patterns_example() ``` ## Step 7: Testing ### Basic Testing ```python # test_basic.py import calimero def test_basic_operations(): """Test basic client operations.""" connection = calimero.create_connection("http://localhost:2428") client = calimero.create_client(connection) # Test connection api_url = client.get_api_url() assert api_url == "http://localhost:2428" # Test peers count peers = client.get_peers_count() assert isinstance(peers, (int, dict)) # Test contexts listing contexts = client.list_contexts() assert isinstance(contexts, (list, dict)) print("✓ All basic tests passed") if __name__ == "__main__": test_basic_operations() ``` ### Integration Testing ```python # test_integration.py import calimero def test_full_workflow(): """Test complete workflow.""" connection = calimero.create_connection("http://localhost:2428") client = calimero.create_client(connection) print("=== Integration Test ===") # Install application app = client.install_application( url="https://example.com/test.wasm", metadata=b'{"name": "Test App"}' ) app_id = app["application_id"] print(f"✓ Installed app: {app_id}") try: # Create context context = client.create_context(app_id, "near") context_id = context["context_id"] print(f"✓ Created context: {context_id}") # Generate identity identity = client.generate_context_identity() print(f"✓ Generated identity") # Execute function result = client.execute_function( context_id, "test", "{}", identity["public_key"] ) print(f"✓ Function executed: {result}") print("✓ Integration test completed successfully") finally: # Cleanup client.delete_context(context_id) client.uninstall_application(app_id) print("✓ Cleanup completed") if __name__ == "__main__": test_full_workflow() ``` ## Resources - **GitHub Repository**: [calimero-client-py](https://github.com/calimero-network/calimero-client-py) - **PyPI Package**: [calimero-client-py](https://pypi.org/project/calimero-client-py/) - **Community Discord**: [Join the conversation](https://discord.gg/calimero) - **Email Support**: [team@calimero.network](mailto:team@calimero.network) Happy coding with the Calimero Python client! 🐍✨ --- ## The Data Sovereignty Manifesto ## Vision & Mission We envision a digital world where users decide how and when their data is stored, processed, and accessed. In such a future, individuals will be compensated with a share of the profit, if they provide consent for their data to be utilized by a specific entity for a specific context. Our framework will ensure that data sharing is consensual, secure, and transparent, allowing users to reclaim their privacy and digital autonomy. In the digital age, the concept of privacy has evolved significantly. Traditional notions of privacy, often centred around the control of personal information, have been challenged by the pervasive nature of digital technologies. In a world where your data flows freely across the internet, and is processed by centralized servers, the control over your personal information is often lost. A future dominated by AI, where your digital personalities can be replicated and exploited, calls for urgent safeguarding of contextual integrity and data privacy. We aim to empower individuals to take charge of their digital identities. Contextual integrity is a theory developed by Helen Nissenbaum, that suggests privacy is preserved when information flows appropriately according to social norms within specific contexts. Rather than focusing solely on the individual's control over their data, contextual integrity considers the context-specific norms that govern information sharing and usage. Different social settings or spheres (e.g., healthcare, financial records, personal relationships and similar), different actors, attributes and conditions under which information is shared, each have their own expectations for information flow. By examining these parameters, contextual integrity provides a nuanced understanding of privacy that is sensitive to the specific social contexts in which information is used. The digital environment involves numerous actors, including users, service providers, third-party advertisers, and data brokers. This complexity increases the difficulty of tracking and managing information flows. Bad actors exploit data streams, leading to breaches of privacy and trust. We believe that every individual should have the power to control their personal data. Our mission is to build a peer-to-peer ecosystem that enables users to dictate who they share their data with and how data access is managed. Digital platforms often merge multiple contexts, making it difficult to distinguish appropriate information flows. For example, social media combines personal, professional and public spheres, challenging traditional privacy norms. Digital technologies enable the collection and analysis of vast amounts of diverse data, often beyond what individuals explicitly share. This includes metadata, behavioral data, and similar, which complicates consent and control and often gives away much more about the user than the user actually understands or wants to share. The principles governing data transmission in the digital world are often non-transparent and dynamic. Terms of service and privacy policies frequently change, and the mechanisms of data sharing and processing are not always transparent to users. ## Rationale and Addressing the Problem ◦ Legal frameworks should consider the specific contexts in which data is used. This means creating sector-specific privacy regulations that account for the unique norms and expectations of each context. ◦ Digital platforms must be transparent about their data practices and accountable for maintaining appropriate information flows. This includes clear communication about how data is collected, used, and shared, as well as mechanisms for users to hold platforms accountable. ◦ Empowering users with tools to manage their privacy in context-specific ways is crucial. This can involve granular privacy settings, consent mechanisms that reflect contextual norms, and educational initiatives to inform users about their privacy rights and risks. ◦ Designers and developers of digital technologies should incorporate contextual integrity principles into the design of systems and services. This means considering the social contexts in which their technologies will be used and ensuring that information flows align with context-specific norms. ◦ Regulators have tried to address this issue by implementing data protection laws. However, in practice, there are no effective tools to solve the problem easily, and the regulations are prone to human error. This results in unnecessary bureaucracy, without solving the problem at its core. ## Core Values and Beliefs **Privacy** We believe that privacy is a fundamental human right. **Ownership & Control** Data should be owned and managed by the user whose data it is, not by large corporations, big tech, or the government. Individuals should have complete control over their digital footprint. **Transparency & Security** Data management processes must be transparent and understandable. Personal data should be protected against unauthorized access and misuse. ## Goals and Objectives The need for data sovereignty is urgent. Daily data breaches compromise medical records, putting lives at risk, financial records are disclosed, leading to theft, and personal secrets can be exposed without consent. In a world where data is a valuable asset, it is essential to create a fair and secure environment. Contextual integrity, which addresses data privacy in the digital age, must be upheld to protect individual rights and foster trust in digital interactions. Our goals include: ◦ Creating a robust, secure, and user-friendly peer-to-peer protocol for data access control. ◦ Providing tools and resources to enable builders to develop Self Sovereign Apps, where users can manage their data independently. ◦ Target developers to raise awareness, join the movement, and build products within this new framework. ◦ Educating the public about the importance of data sovereignty and how to protect their digital identity. ◦ Developing user economy models, for the users to be compensated when their data is used by profit-making entities. ## Join the Movement The future of our digital world depends on the actions we take today. By reclaiming control over our personal data, we can ensure privacy, security, and fairness for all. We call on developers to build innovative solutions that prioritize user-controlled data management. We urge individuals to take greater care of their digital identity and privacy, advocating for a future where personal data is respected and protected. Join us in creating a fair and secure digital world. Let us rise to the challenge and create a digital environment where data sovereignty is the norm, not the exception. Because it is YOUR data, and YOU should control what happens to it! --- ## ELI5 ## Explain Like I'm Five ![ELI5](/eli5withtrademark.png) --- ## _02 Changelog --- ## _03 Versioning --- ## General This section offers guidance on resolving common issues with Calimero Network features. As the platform evolves, additional troubleshooting topics will be added to help address various challenges. For further assistance, consult our community forums or reach out through official support channels. --- ## SSL/TLS ### SSL/TLS Support To be able to access the the node from external source on the same network you will need to install the generated self-signed certificate. > **_NOTE:_** Installing the SSL certificate is only necessary if you plan to > access the node from an external source on the same network. If you are > running the application locally, you do not need to install the certificate. ### Steps to Add the Certificate to Your Device 1. **Locate the Certificate**: - Download the certificate from `http://localhost:/admin-api/certificate`. - The `` is the port number used as an argument in the `--server-port` flag in the section [Initialize and start your node (separate terminal)](../getting-started/setup). - For example: `bash http://localhost:2428/admin-api/certificate` 2. **Add the Certificate to Trusted Certificates**: - **For Windows**: 1. Open the `Run` dialog (Win + R) and type `mmc` to open the Microsoft Management Console. 2. Go to `File` -> `Add/Remove Snap-in...`. 3. Select `Certificates` and click `Add`. 4. Choose `Computer account`, then `Next` and `Finish`. 5. Expand `Certificates (Local Computer)` -> `Trusted Root Certification Authorities`. 6. Right-click `Certificates`, then `All Tasks` -> `Import...`. 7. Follow the prompts to import the certificate file. - **For macOS**: 1. Double-click the certificate file. 2. This will open the `Keychain Access` application. 3. Choose `System` from the list of keychains. 4. Drag and drop the certificate into the `System` keychain. 5. Authenticate with your administrator password if prompted. 6. Right-click the certificate and select `Get Info`. 7. Expand the `Trust` section and select `Always Trust` from the `When using this certificate` dropdown. - **For Linux**: 1. Copy the certificate to `/usr/local/share/ca-certificates/` (or `/etc/pki/ca-trust/source/anchors/` depending on your distribution). 2. Run `sudo update-ca-certificates` (or `sudo update-ca-trust extract` for Red Hat-based distributions). 3. **Restart Your Browser**: - Close and reopen your web browser to ensure it recognizes the newly added certificate. ### Rules for Generating SSL Certificates - If a certificate doesn't exist, a new one will be generated based on your current local IP address. - If a certificate exists for the current IP address, it will be used. - If a certificate exists but is not configured for the current IP address, a new certificate will be created. > **_NOTE:_** Every time a new certificate is generated (e.g., on the first > start of the server or when the IP address changes), you will need to add it > to your device's trusted certificates. --- ## Limitations --- ## GitHub All work on Calimero happens directly on GitHub. Both core team members and external contributors send pull requests which go through the same review process. ## How to Contribute We appreciate your interest in contributing to our project! To get started with contributing, please follow the instructions outlined in our [CONTRIBUTING](https://github.com/calimero-network/core/blob/master/CONTRIBUTING.md) file. ## Additional Ways to Contribute - **Show Your Support:** If you find this project helpful, please star it on GitHub. Your stars help the project grow and reach more developers like you. - **Spread the Word:** Share the project with your friends and colleagues who might be interested in contributing or using it. - **Join Discussions:** Participate in discussions on GitHub [Issues](https://github.com/calimero-network/core/issues) or [Discussions](https://github.com/orgs/calimero-network/discussions) to share your ideas and provide feedback. - **Write Tutorials or Blog Posts:** Create tutorials or blog posts about how to use the project or your experience contributing to it. --- ## Hackathons # Hackathons At Calimero, we love innovation and collaboration, which is why we plan to regularly organize hackathons that include our product. We have exciting events planned for the future. Stay tuned by following us on our social media channels to get the latest updates. ## What is a Hackathon? A hackathon is an event where developers, designers, and other tech enthusiasts come together to create innovative solutions in a short amount of time. Participants work in teams to build projects, often centered around a specific theme or technology. Hackathons are a great opportunity to: - **Learn New Skills:** Whether you’re a beginner or an expert, hackathons provide a platform to learn new technologies and tools. - **Collaborate:** Work with other passionate individuals, share ideas, and collaborate on projects. - **Showcase Your Talent:** Present your project to a panel of judges and other participants, winning prizes and recognition. - **Network:** Meet like-minded individuals, industry experts, and potential employers or collaborators. ## How to Get Involved When we announce a new hackathon, here’s how you can participate: 1. **Register for the Event:** Sign up through the event link provided in our announcements. 2. **Form a Team:** Join with friends or team up with other participants. 3. **Build Your Project:** Use our product and other tools to create something amazing. 4. **Submit Your Work:** Present your project at the end of the hackathon for a chance to win prizes and gain recognition. ### Stay Updated Follow us on social media to be the first to know about our upcoming hackathons and other events. We’ll share all the details, including how to register and participate. Thank you for your interest in contributing to our project through hackathons. We look forward to seeing your innovative ideas and solutions! --- ## Bounty Program # Bounty Program We're excited to announce that Calimero will soon be launching a bounty program! This program is designed to reward contributors who help us tackle various issues and enhance our project. ## What is a Bounty Program? A bounty program is an initiative where contributors can earn rewards for completing specific tasks or resolving issues. These tasks can range from fixing bugs to adding new features, improving documentation, or optimizing performance. The rewards can vary and may include monetary compensation, swag, exclusive access to new features, or public recognition. ## How it Works While we are still in the process of finalizing the details of our bounty program, here’s an overview of how it will function: 1. **Identify Bounty Issues:** We will mark specific [issues](https://github.com/calimero-network/core/issues) in our repository with a "bounty" label. These issues will be eligible for rewards. 2. **Contribute:** Choose a bounty-labeled issue that interests you, solve it, and submit a pull request. 3. **Review:** Our team will review your contribution. If your solution meets the requirements and quality standards, it will be merged. 4. **Reward:** Once the issue is resolved and your contribution is accepted, you will receive a reward. The exact nature of the reward will be specified in the issue description. ### Exceptional Contributions While not all issues may be marked with a bounty label, we recognize that exceptional solutions deserve recognition. If you provide a particularly outstanding solution to an issue not marked as a bounty, we may still decide to reward your contribution. ## Stay Tuned We are currently refining the details of our bounty program, including the reward system. Follow us on social media and keep an eye on our announcements to be the first to know when the bounty program goes live. Thank you for your patience and continued support. We look forward to collaborating with you and rewarding your valuable contributions! --- ## Community and Support # Community and Support We value our community and are here to support you. Here are the best ways to get in touch and find help: ## GitHub - **Issues:** For development-related queries, bug reports, and feature requests, please use our [GitHub Issues](https://github.com/calimero-network/core/issues). - **Discussions:** For more extensive discussions on various topics, join our [GitHub Discussions](https://github.com/orgs/calimero-network/discussions). ## Discord For news, casual conversations, and random questions, join our [Discord Server](https://discord.gg/jxAeJd2E). It’s a great place to connect with other contributors, share ideas, and stay informed about the latest updates. ## Social Media Stay connected and get the latest news and updates by following us on social media: - **LinkedIn:** Follow us on [LinkedIn](https://www.linkedin.com/company/calimero-network/) for professional updates and networking opportunities. - **Twitter:** Stay up-to-date with our latest tweets by following us on [Twitter](https://twitter.com/CalimeroNetwork). Thank you for being a part of the Calimero community. Your contributions and engagement help us grow and improve! --- ## Learning # Learning Expand your knowledge and stay informed by exploring the following resources: ## Learn More - **Website:** Visit our [website](https://www.calimero.network/) for comprehensive information about the project, including features and documentation. - **Blog:** Check out our [articles on X](https://x.com/CalimeroNetwork/articles) for insightful articles, tutorials, and the latest news about our project. We provide a variety of resources to help you learn more about our project and stay up-to-date with the latest developments. Dive into our documentation, read our blog, and explore our website to get the most out of our project. --- ## Context Create Admin Dashboard ## Create a context using the Admin Dashboard 1. Navigate to the "Contexts" tab and click the "Start New Context" button. 2. Use the "Browse" button to select an application from the list. 3. Choose application. 4. Press "Start" to initiate the context creation process with the selected application. Visit [Admin Dashboard Page](../../developer-tools/apps/admin-dashboard) to view detailed instructions with images. After the context is created, you will be redirected to the context dashboard. Keep this page open as you will need the Context ID later. --- ## Context Create Binary import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; ```bash title="Terminal" meroctl --node node1 context create --application-id 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf --protocol near ``` ```bash title="Terminal" meroctl --node node1 context create --application-id 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1brV1RKEXtfvfsf --protocol starknet ``` ```bash title="Terminal" meroctl --node node1 context create --application-id 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf --protocol icp ``` ```bash title="Terminal" meroctl --node node1 context create --application-id 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf --protocol stellar ``` ```bash title="Terminal" meroctl --node node1 context create --application-id 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf --protocol ethereum ``` :::tip - `--application-id` flag specifies the application ID to attach to the context - `--protocol` flag specifies the protocol to use for the context - `--as` flag creates an alias for your identity in your context, you can use it instead of the PublicKey ::: --- ## Context Create Interactive Cli import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; ### Install the Application Run the following command to install the application: ```bash title="Node Terminal" application install url {url} {metadata} ``` :::note metadata example: ```bash '{"contractAppId": "3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf"}' ``` ::: ### Create a New Context ```bash title="Node Terminal" context create --application-id 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf --protocol near ``` ```bash title="Node Terminal" context create --application-id 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf --protocol starknet ``` ```bash title="Node Terminal" context create --application-id 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf --protocol icp ``` ```bash title="Node Terminal" context create --application-id 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf --protocol stellar ``` ```bash title="Node Terminal" context create --application-id 3Hfk2VekXQ58vYHW3hUtA3mh2Rwtb1brV1RKEXtfvfsf --protocol ethereum ``` :::tip `--application-id` flag specifies the application ID to attach to the context. `--protocol` flag specifies the protocol to use for the context. ::: ```bash title="Output" 2024-10-07T11:55:36.658396Z INFO calimero_context: Subscribed to context context_id=567C5Gg4mxHMPKy2wLJ4uvb3DHsbcpVDYUsuAWgTPgXn Created context 567C5Gg4mxHMPKy2wLJ4uvb3DHsbcpVDYUsuAWgTPgXn with identity DaSkad6DK7f6fUhjz1CvNW7L4TkWZmWqAcwysZHG3Xs5 ``` ![Create context](/cli/2-create-context.png) --- ## Install Meroctl You can choose to install `meroctl` using either the installation script or Homebrew. import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; If you use Homebrew, you can install `meroctl` directly with the following steps: #### Steps 1. Add the Calimero Homebrew tap: ```bash brew tap calimero-network/homebrew-tap ``` 2. Install `merod`: ```bash brew install meroctl ``` 3. Verify the installation: ```bash meroctl --version ``` If the version number appears, the installation was successful. The installation script supports multiple terminal environments and will automatically configure your PATH based on your shell. #### Steps 1. Open your terminal and run: ```bash curl -sSf https://raw.githubusercontent.com/calimero-network/install-sh/master/install-meroctl.sh | bash ``` 2. Source the updated PATH: ```bash source ``` Replace `` with `.bashrc`, `.zshrc`, or the relevant configuration file for your shell. 3. Verify the installation: ```bash meroctl --version ``` If the version number appears, the installation was successful. #### Notes for Shell Environments The script automatically updates your PATH based on your shell configuration: - **Bash**: Updates `.bashrc`. - **Zsh**: Updates `.zshrc`. - **Fish**: Modifies `~/.config/fish/config.fish`. - **Csh/Tcsh**: Updates `.cshrc`. To apply the changes immediately, use the `source` command as described in step 2 above. --- ## Install Merod You can choose to install `merod` using either the installation script or Homebrew. import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; If you use Homebrew, you can install `merod` directly with the following steps: #### Steps 1. Add the Calimero Homebrew tap: ```bash brew tap calimero-network/homebrew-tap ``` 2. Install `merod`: ```bash brew install merod ``` 3. Verify the installation: ```bash merod --version ``` If the version number appears, the installation was successful. The installation script supports multiple terminal environments and will automatically configure your PATH based on your shell. #### Steps 1. Open your terminal and run: ```bash curl -sSf https://raw.githubusercontent.com/calimero-network/install-sh/master/install-merod.sh | bash ``` 2. Source the updated PATH: ```bash source ``` Replace `` with `.bashrc`, `.zshrc`, or the relevant configuration file for your shell. 3. Verify the installation: ```bash merod --version ``` If the version number appears, the installation was successful. #### Notes for Shell Environments The script automatically updates your PATH based on your shell configuration: - **Bash**: Updates `.bashrc`. - **Zsh**: Updates `.zshrc`. - **Fish**: Modifies `~/.config/fish/config.fish`. - **Csh/Tcsh**: Updates `.cshrc`. To apply the changes immediately, use the `source` command as described in step 2 above. --- ## Node Init Binary ```bash title="Terminal" merod --node-name node1 init --server-port 2428 --swarm-port 2528 ``` Node configuration file contains protocol defined metada and is located at `~/.calimero/your_node_name/config.toml`. --- ## Node Init Source ```bash title="Terminal" cargo run -p merod -- --node-name node1 init --server-port 2428 --swarm-port 2528 ``` Node configuration file contains protocol defined metada and is located at `~/.calimero/your_node_name/config.toml`. --- ## Node Run Binary ```bash title="Terminal" merod --node-name node1 run ``` Wait for a few moments and node logs should appear. --- ## Node Run Source ```bash title="Terminal" cargo run -p merod -- --node-name node1 run ``` Wait for a few moments and node logs should appear.