--- title: How it works description: The sync daemon that owns the DuckDB writer lock, the three ingest paths, the localhost query server, and how the dashboard reads without contending for it. keywords: ClawMetry architecture, DuckDB agent telemetry, agent observability design eyebrow: Get started --- # How it works Three processes, one database, three ways data gets in. Understanding this layout explains most of the behaviour you will encounter, including why reading a raw log file inside a request handler works on your laptop and returns nothing in the cloud. ## The processes **The sync daemon** (`clawmetry sync`, or `python -m clawmetry.sync`) is where the work happens. It polls every runtime it can find, reads what is new, normalises it, and writes it into the local store. It owns the DuckDB **writer lock** — exclusively. It also runs the detectors, evaluates Guard policies, and, when cloud sync is on, encrypts and pushes a snapshot. **The dashboard** (`clawmetry`) is a Flask app serving the UI on port 8900. It holds no writer lock and, in normal operation, does not open DuckDB at all: the process marks itself `CLAWMETRY_ROLE=dashboard` precisely so the store layer refuses to hand it a writer. **The enforcement proxy** (`clawmetry proxy start`) is optional. It sits in front of model calls on port 4100 and applies budget limits, loop detection and model routing. Nothing else depends on it — except OpenClaw's advisory pause, which is only enforced when the proxy is running. ## Getting data in Everything lands in the same store, through three paths. ### 1. The filesystem Most runtimes write transcripts to disk. The daemon reads them where they are, read-only, and never modifies an agent's directory. Each runtime has its own format — JSONL, SQLite, a directory of JSON files, a Markdown transcript, a Postgres database, even an undocumented protobuf blob — and its own adapter. See [How detection works](/docs/runtimes/detection/) and the per-runtime [reference pages](/docs/runtimes/overview/). ### 2. The gateway WebSocket OpenClaw and NemoClaw expose a JSON-RPC gateway on port 18789. Subscribing to it gives sub-second liveness that filesystem polling cannot: you see a tool call while it is running, not after the transcript is flushed. ### 3. OpenTelemetry The dashboard exposes OTLP receivers at `/v1/metrics`, `/v1/traces` and `/v1/logs`. Anything that speaks OTLP can push into ClawMetry, which is the supported path for runtimes with no local store — n8n on Postgres, a hosted CI agent, your own service. See [OpenTelemetry ingest](/docs/runtimes/opentelemetry/). There is also a fourth, explicitly opt-in path for code you control: the [HTTP ingest API](/docs/runtimes/custom-ingest/) and the [interceptor](/docs/runtimes/sdk-agents/). ## The store One DuckDB file, on your machine, holding events, sessions, spans, rollups and Guard state. It is the single data layer every feature reads and writes; there is no second source of truth. Because the daemon holds the writer lock, the dashboard cannot simply open the file. Instead the daemon runs a **localhost query server** and advertises it in a discovery file: ```bash title="~/.clawmetry/local_query.json" {"port": 51843, "token": "…", "pid": 4711} ``` Anything that needs to read — the dashboard, the MCP server, the cloud relay — reads the discovery file, checks the pid is alive, and posts a query to `http://127.0.0.1:/api/local/query` with the bearer token. One writer, many readers, no lock contention. :::warning The rule this implies A request handler must read from the store, not from raw files. Opening `~/.openclaw/**.jsonl` inside a handler works on a laptop and returns empty in the cloud, where the container has no agent directories at all. Everything goes through the [query API](/docs/data/query-api/). ::: ## The query contract The read surface is a declared contract, `q/1`, with a fixed set of *shapes* (`sessions`, `events`, `transcript`, `aggregates`, `spans`, `traces`, `models`, `runtimes`, `search`, `health`, and more). Inside `q/1`, evolution is additive only: new shapes and new optional arguments may appear, but renaming or removing one requires a new contract version. The same shapes back the HTTP API, the MCP server and the cloud relay, so all three agree by construction. Full list and argument reference: [Query API](/docs/data/query-api/). ## Cloud, when you turn it on The daemon builds a snapshot, encrypts the content-bearing parts with **AES-256-GCM** using a key that never leaves your machine, and pushes it to the ingest endpoint. The server stores ciphertext; your browser decrypts it locally when you open the cloud dashboard. Aggregate counters — cost totals, session counts, health status — travel in plaintext so the cloud can render a summary and a device display without holding your key. Which fields are which is documented in [End-to-end encryption](/docs/cloud/encryption/). ## Where intervention fits Nothing in the ingest path can touch an agent. There are exactly four surfaces that can, and adding a fifth means adding it to that list with locks of the same strength: 1. **Manual control** — you press Pause, Stop or Kill in the Guard tab. 2. **Autonomous policy** — a Guard policy fires, which requires all three locks open: the policy's action must not be `monitor`, `CLAWMETRY_POLICY_ENFORCE=1` must be set on the node, and an entitlement check that fails closed must pass. 3. **Pre-tool gates** — where a runtime exposes a hook, ClawMetry can hold an action before the tool runs. 4. **Cron management** via gateway RPC. Both control paths end in the same actuator, so an automatic pause and a hand-pressed pause are identical to the agent process. See [Pause, stop, kill](/docs/guard/process-control/). ## The frontend The dashboard UI is served from static files and templates — no npm, no bundler, no build step. That is a deliberate constraint: a dashboard you can `pip install` and run on a Raspberry Pi should not need a JavaScript toolchain.