The Slow Burn: Three Agent Failures That Take Days (Not Minutes) to Notice

I’ve been writing about agent failures that announce themselves: the overnight cost spike, the deadlocked subprocess, the loop that runs to dawn. Those are the ones that burn at 3 AM and leave a phone notification in their wake.

But there’s a second class of failures that’s harder and, in some ways, worse. These don’t spike. They drift. Your agent runs, exits clean, produces output, posts to Slack, sends the email. And somewhere inside that clean surface, something has been quietly wrong for three days, or eleven days, or three weeks. You find out when a human reviewer happens to look closely enough.

Three patterns keep showing up in community reports and in my own inbox. I want to walk through each one: what the developer saw, what actually happened, and what ClawMetry surfaces — specifically, why you need historical data, not just real-time alerts, to catch these.

Incident 1: The Weekly Digest That Went Quiet

A developer builds an agent that runs every Sunday at 8 AM. It reads from a data pipeline output directory, aggregates token usage and session counts across the week, and posts a digest to Slack. Works great for six weeks.

In week seven, the upstream data pipeline team moves their output from /data/processed/ to /data/v2/processed/. Nobody tells the agent. The agent reads the old path, finds an empty directory, and generates a summary: “Usage appeared flat this week — nothing notable to report.” It posts the message to Slack. The cron shows “ran successfully.” Exit code 0.

Three weeks pass. July 4th week falls in there — genuinely quiet. The empty summaries don’t raise alarms. It takes a quarterly planning meeting, where someone asks why July token spend looks flat when three new agents were deployed, for anyone to pull the numbers and find nothing there.

Pattern: reported across multiple developer communities discussing scheduled agent reliability, 2026. The upstream path-change scenario is one of the most common silent-failure triggers in pipeline-dependent agents.

What ClawMetry surfaces

The fix is trivial once you see it: validate that the directory has records before proceeding, and fail loud if it doesn’t. But the harder problem is finding out when it broke. Without historical session data, you’re looking at 5 identical Slack messages and a hunch. With it, you have a precise timestamp and a verbatim tool response.

Cron run history with per-step token counts. The cliff is unmistakable once you can see across weeks.

Incident 2: The Memory Poisoning Spiral

This one is subtle in a way that makes it hard to explain to someone who hasn’t lived it. Agents with persistent memory — SOUL.md, MEMORY.md, or equivalent — read that file at the start of each session as ground truth. Whatever is in there shapes everything that follows.

A developer is debugging a staging environment issue and tells the agent: “Use the staging gateway at gateway-staging.internal:18789 for all tests.” The agent, being helpful, writes this to SOUL.md so it won’t forget. The debugging session completes. The developer closes the terminal.

The next session opens. It reads SOUL.md. It sees: “Use staging gateway at gateway-staging.internal:18789.” It uses the staging gateway. The session completes cleanly — staging is live, it responds, tool calls return data. Every one of the next 14 sessions does the same thing, against a staging environment that nobody is watching.

The failure surfaces when someone checks production logs for the automated task results and finds nothing. Two weeks of expected production writes: missing.

Pattern: documented in Claude Code community discussions on persistent memory management, 2026. The session-level context bleeds into multi-session memory, which is a design challenge for any agent with write access to its own memory files.

What ClawMetry surfaces

The fix is clear: tighten the agent’s write policy for environment-sensitive values in persistent memory, and add a session-start check that validates the gateway URL against an allowlist. But first you have to find the change. The memory diff timeline is the only way to do that without manual log archaeology.

Incident 3: The API Deprecation That Looked Like Success

Of the three, this one is most dangerous because it degrades the model’s output quality without touching any of the signals you’re watching. No errors. No cost spikes. No hung sessions. Just subtly wrong answers, delivered with confidence, formatted beautifully.

A developer runs an agent nightly to pull pricing data from an external API and compute cost projections. On August 5, the API provider ships v2.1.0. They rename unit_price_usd to unit_price and move currency handling to a separate field. They consider this a minor, non-breaking change. The old field still exists in responses — it just returns null now.

The model receives the response, sees unit_price_usd: null, and gracefully handles it the way models do: it defaults to zero and continues. The projection math runs. The output is formatted cleanly. Each nightly session exits with code 0 and sends the email.

For eleven days, every line item in the projections shows $0.00. The failure surfaces on day twelve when a human reviewer, preparing for a budget meeting, opens the report and notices that eleven days of costs are zero across the board.

Pattern: described in agent observability engineering blogs and developer forums, 2026. Vendors describe this category as “silent semantic degradation” — failures where HTTP 200 and correct output format coexist with meaningless content. See: Laminar.sh, April 2026.

What ClawMetry surfaces

I want to be precise about what we can and can’t do here. We can’t tell you that null is the wrong value; we don’t know the API’s contract. What we can do is show you the verbatim response from each day, so a human reviewer can spot the change in seconds instead of filing a support ticket with the API provider and waiting. The forensics go from days to a single query.

The real problem: these failures don’t need alerts, they need archaeology

Look at what the three incidents have in common. In each one:

• No cost spike. The cron ran the same amount of work (or less). Budget alerts would stay silent.
• No stalled session. All sessions completed promptly. The “stalled” detector doesn’t fire.
• No error rate. HTTP 200s throughout. All exit codes were 0.

Traditional real-time monitoring is designed for point-in-time anomalies: a spike, a stall, an error. Slow burns don’t produce anomalies in the moment. They produce a normal-looking present and an abnormal past, and you only see the abnormal past by comparing today to last week.

This is why the DuckDB store matters. Every session event, tool call request and response, memory file write, and cron output is ingested into a local column store. You can query across all of it. “Show me every session where the directory tool call returned an empty list.” “Show me the SOUL.md diff history for the last 30 days.” “Compare the pricing API response from last Tuesday vs this Tuesday.”

Those queries run locally, in milliseconds, against data that’s been accumulating since you installed ClawMetry. No external service. No egress cost. The data is yours.

What to build into your agents

Beyond observability, each incident points to a concrete defensive pattern:

For empty-dataset crons

Fail loud on empty input. Don’t generate a “nothing this week” summary from zero records — raise an error. An agent that says “I found 0 records and will summarize nothing” should exit non-zero, not post to Slack. Reserve the graceful “quiet week” message for weeks where you actively verified data was available and it really was flat.

For persistent memory

Separate durable facts from session-specific instructions. Environment configuration — which gateway, which API endpoint, which tenant — should live in a config file with an explicit update policy, not in a free-form memory file that any session can overwrite. Session-level notes go in the session; multi-session truths require review before persisting.

For external API responses

Validate schema, not just status. After the tool call returns 200, assert that the fields you depend on are non-null before passing the response downstream. This won’t catch every semantic drift, but it turns an eleven-day silent failure into a same-day tool call error that shows up in ClawMetry’s session timeline immediately.

The honest part

Real-time alerts are necessary but not sufficient for agents. I’ve shipped budget alerts, stalled-session detection, and cost-spike notifications, and they catch real problems — the overnight fires, the runaway loops. But the three incidents above would have sailed right past all of them.

The gap they expose isn’t in alerting — it’s in forensics. When a developer asks “when did this break?” they need a data store that goes back far enough to answer. That’s what we built with the DuckDB layer: not just a real-time view, but a queryable history of every session event since installation.

Slow burns hide in the past. The tool that catches them is the one that can look back.