Watching an AWS AgentCore fleet with ClawMetry (Pydantic AI and LangChain come free)

This week an enterprise architect described his setup to me: about a hundred agents in Dev, forty in Test, twenty approved in Prod, all running on Amazon Bedrock AgentCore, deployed through Terraform and OIDC GitHub workflows. His question was simple: what does agent observability look like for a fleet like that, without buying yet another per-trace SaaS bill?

The answer turned out to be one Terraform change, and it generalizes far beyond AgentCore. This post walks the whole path end to end: prove the pipeline on your laptop in five minutes with no cloud account and no API key, then roll it out to a real AgentCore fleet, then reuse the exact same wiring for Pydantic AI and LangChain apps.

The trick: AgentCore already speaks OpenTelemetry

AgentCore auto-instruments every agent with ADOT, the AWS distribution of OpenTelemetry, and exports spans that follow the GenAI semantic conventions: model, token usage, tool calls, session ids. By default those spans go to CloudWatch. But the exporter is just OTLP, and AWS documents re-pointing it at partner tools as a first-class pattern.

ClawMetry ships a native OTLP receiver on the dashboard port: /v1/traces, /v1/metrics, /v1/logs. Point the exporter at it and the fleet appears. No ClawMetry SDK in your agent, no code change to agent.py, nothing new for a security review beyond one egress rule inside your own VPC.

Prove it in five minutes, on your laptop, for free

Do not start with the cloud rollout. Start with a terminal:

pip install 'clawmetry[otel]' && clawmetry     # dashboard + OTLP receiver on :8900

# in a second terminal
pip install pydantic-ai-slim opentelemetry-sdk opentelemetry-exporter-otlp-proto-http
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:8900
curl -LO https://raw.githubusercontent.com/vivekchand/clawmetry/main/examples/bring-your-own-agent/pydantic_ai_agent.py
python pydantic_ai_agent.py

That script is a real Pydantic AI agent with a real tool call. With no ANTHROPIC_API_KEY set it uses Pydantic AI’s built-in TestModel, so the run costs zero and needs zero accounts. Open http://localhost:8900 and you will find:

• Tracing: the run as a span tree, invoke_agent → chat → execute_tool → chat, with per-span tokens
• Sessions: one row for the conversation, with title, token total, and a dollar cost derived from the model’s pricing (spans almost never carry cost; token counts are enough)
• The runtime switcher and Agent Inventory: your app listed as invoice-copilot (OTel), next to any local runtimes ClawMetry already watches

Everything below is this same pipeline with different senders. That is the point of standing on OpenTelemetry instead of a vendor SDK.

The AgentCore rollout

Run ClawMetry inside your VPC (it is one Docker container with an embedded database, so telemetry never leaves your network), and fan out through an OpenTelemetry Collector so CloudWatch keeps working:

AgentCore Runtime (ADOT)  ->  OTel Collector (your VPC)  ->  ClawMetry (self-hosted)
                                      \->  CloudWatch / X-Ray (unchanged)

The integration itself is environment variables on the AgentCore runtime resource, which means it is a reviewed pull request in your Terraform repo, rolled out by the pipeline you already trust:

environment_variables = {
  AGENT_OBSERVABILITY_ENABLED = "true"
  OTEL_EXPORTER_OTLP_PROTOCOL = "http/protobuf"
  OTEL_EXPORTER_OTLP_ENDPOINT = "https://clawmetry.observability.internal:8900"
  OTEL_EXPORTER_OTLP_HEADERS  = "Authorization=Bearer ${var.clawmetry_token}"
  OTEL_RESOURCE_ATTRIBUTES    = "service.name=${var.agent_name},deployment.environment=${var.environment}"
}

Two attributes carry the fleet’s legibility. service.name is the agent’s identity: every app gets its own entry in the switcher and the inventory. deployment.environment keeps Dev, Test, and Prod separable, so the twenty approved production agents are never mixed into the hundred and forty running upstream. AgentCore stamps a session id on each runtime session’s telemetry, and ClawMetry groups spans into sessions by it.

Want a working open-source project to try this on? The awslabs/agentcore-samples repo deploys real Strands agents to AgentCore, and its partner-observability tutorials use exactly this exporter re-pointing. ClawMetry’s examples/bring-your-own-agent directory carries the same agent shape as a single runnable file.

Pydantic AI and LangChain, same wire

Pydantic AI instruments itself with OpenTelemetry natively; one line turns it on for every agent in the process:

Agent.instrument_all(InstrumentationSettings(tracer_provider=provider))

LangChain and LangGraph go through the OpenLLMetry instrumentation package, which hooks the callback system both run on:

LangchainInstrumentor().instrument(tracer_provider=provider)

Both runnable examples, the full attribute mapping table, and the troubleshooting list (no sessions? your spans lack a conversation id; app missing from the switcher? set service.name) live in docs/BRING_YOUR_OWN_AGENT.md.

Every OTel-speaking app becomes sessions, traces, tokens and cost in the same dashboard that watches your local runtimes.

What this path is, and what it is not

Two honest boundaries, because trust in an observability tool is the whole product:

• Observation only. ClawMetry can pause, stop, or gate runtimes it reaches as a process or a hook. A span stream is not that; an OTLP fleet is watched, not controlled.
• Self-host for privacy. OTLP arrives at the receiver in plaintext, so the deployment that keeps content inside your network is the one drawn above: the receiver runs in your VPC and nothing leaves it.

If you are paying per-trace for this today

Hosted agent-observability platforms mostly price on ingest: per trace, per gigabyte, per seat. An agent fleet is the worst case for that model, because span volume scales with agent count times steps per task, and a hundred and sixty agents emit all day. The bill grows with exactly the thing you are trying to scale.

ClawMetry’s answer is structural rather than a discount: the observability layer is open source, the deployment is self-hosted on your infrastructure, and the ingest volume is yours to keep. I wrote an honest comparison with Arize Phoenix earlier, including the cases where Phoenix is the better pick; the short version is that if your traces already flow over OpenTelemetry, trying the alternative is one environment variable, fully reversible, and you can run both during the evaluation.