--- title: Alerts description: Alert rules, webhooks and delivery channels — spend velocity, budget breaches, failure patterns and agent silence — and how to avoid alerts everyone mutes. keywords: AI agent alerts, LLM spend alerts, agent failure notification, agent webhook alerts eyebrow: Dashboard --- # Alerts Get told when something goes wrong, without having to be looking. ## Rule types | Rule | Fires when | |---|---| | **Budget** | Spend crosses a daily or monthly threshold | | **Velocity** | Spend rate over a window is abnormal, rather than a fixed total | | **Failure** | A tool or session fails repeatedly | | **Silence** | An agent that normally runs has not | | **Detector** | Guard raises an incident matching your filter | | **Custom** | Your own condition over the metrics ClawMetry exposes | Velocity is usually the better first rule than a plain budget: a monthly budget tells you after the fact, while a velocity rule catches the runaway loop in the hour it starts. ## Delivery Alerts go out through webhooks and the channels you have configured. Test the path before trusting it: ```bash curl -s localhost:8900/api/alerts/webhook/test -X POST \ -H 'content-type: application/json' -d '{"url": "https://…"}' curl -s localhost:8900/api/alert-channels/test -X POST \ -H 'content-type: application/json' -d '{"channel": "telegram"}' ``` With cloud sync connected, alerts can also reach your phone as a push. ## Managing rules ```bash # list curl -s localhost:8900/api/alerts/rules | jq # create curl -s localhost:8900/api/alerts/rules -X POST \ -H 'content-type: application/json' \ -d '{"name":"nightly spend","metric":"cost_usd","window":"1h","threshold":10,"action":"webhook"}' # what is firing now curl -s localhost:8900/api/alerts/active | jq # history curl -s localhost:8900/api/alerts/history -X POST -d '{}' -H 'content-type: application/json' | jq ``` ## Emergency stop Separate from rules, and deliberately blunt: ```bash curl -s localhost:8900/api/emergency-stop -X POST -d '{}' -H 'content-type: application/json' curl -s localhost:8900/api/emergency-stop/status | jq curl -s localhost:8900/api/emergency-stop/clear -X POST -d '{}' -H 'content-type: application/json' ``` This is the "stop everything, I will sort it out afterwards" control. It is not subtle and it is not meant to be. ## Two evaluators, and the rule that follows Alert conditions are evaluated in more than one place — the daemon evaluates continuously, and some rules are also checked on request. The practical consequence: a rule you delete in the UI is gone from both, but a rule edited directly in a config file while the daemon is running can be evaluated from the old copy until the daemon picks it up. Edit rules through the API or the UI, not by hand. ## Designing an alert set people do not mute The failure mode is real and predictable. Some rules of thumb: - **Start with one rule.** The one that would have caught your worst recent surprise. Add a second only after the first has fired usefully. - **Prefer velocity to totals.** A total tells you it already happened. - **Alert on the absence of work**, not just its failure. A cron agent that silently stopped running costs you nothing and is often the worse problem. - **Route by severity, not by rule.** Everything to one channel means everything gets the same attention, which means none. - **Delete rules that have never fired usefully.** A rule that only ever fires as noise is training you to ignore the channel. ## Tier Alert rules are available on the free tier; custom rules, custom webhooks and alert webhooks are Pro (`custom_alerts`, `custom_webhooks`, `alert_webhooks`). Budget limits are Starter and above (`budget_limits`).