D1 · Agentic Architecture › Cluster 7 · Performance

7.6 · Observability & Agent Metrics

You can't tune what you don't measure

CONCEPT What to instrument

Every lever in this cluster — model choice, effort, caching, streaming, state — is only tunable if you can see its effect. Observability for agents has two halves: log tool calls and trace each run (the structure of what happened), and read the usage object (the token economics of each call).

usage fieldMeaning
input_tokensUncached input processed this call.
output_tokensTokens generated (the pricey side, ~5×).
cache_creation_input_tokensTokens written to cache (~1.25×).
cache_read_input_tokensTokens served from cache (~0.1×) — your cache-hit signal.
🔑 Also capture response._request_id on every call. It's what Anthropic support needs to investigate a specific request — log it alongside the usage record.

CONCEPT The metrics that matter

Aggregate the per-call data into run-level metrics. These are the numbers an architect reports on:

MetricWhy it matters
Latency p50 / p95 / p99 Tail latency (p95/p99), not the average, is what users feel — track all three.
Cost per run Sum token cost across every call in a run — the true price of the agent, not per-call.
Error rate Failed calls / runs — catches regressions, bad tools, and timeouts.
Resolution rate Fraction of runs that actually solved the task — the quality metric that justifies cost.
💡 Optimize resolution rate ÷ cost per run, not raw cost. A cheaper model that lowers resolution rate can raise total cost through retries — the trap from 7.1.

DIAGRAM Trace + usage per run

flowchart TD R([Agent run · trace_id]) --> C1["LLM call → log usage + _request_id"] C1 --> TC["🟢 tool call → log name, args, latency, ok/err"] TC --> C2["LLM call → log usage + _request_id"] C2 --> D{"done?"} D -->|no| TC D -->|yes| AGG["🔵 aggregate the run:
cost/run · latency p50/p95/p99
error rate · resolution rate"] AGG --> OBS([Dashboards / alerts]) classDef tool fill:#d4edda,stroke:#28a745,color:#155724; classDef agg fill:#d1ecf1,stroke:#17a2b8,color:#0c5460; class TC tool; class AGG agg;

CODE Logging usage & a trace record

Wrap each call: pull the usage fields and _request_id, emit one structured record per call tagged with the run's trace_id, and record each tool call too.

import time, uuid

def traced_call(trace_id, messages, tools):
    t0 = time.perf_counter()
    resp = client.messages.create(
        model="claude-opus-4-8", max_tokens=4096,
        tools=tools, messages=messages,
    )
    u = resp.usage
    log.info("llm_call", extra={
        "trace_id":      trace_id,
        "request_id":    resp._request_id,          # for Anthropic support
        "input_tokens":  u.input_tokens,
        "output_tokens": u.output_tokens,
        "cache_read":    u.cache_read_input_tokens,   # cache-hit signal (7.3)
        "cache_write":   u.cache_creation_input_tokens,
        "stop_reason":   resp.stop_reason,
        "latency_ms":    round((time.perf_counter() - t0) * 1000),
    })
    return resp

def traced_tool(trace_id, name, args, fn):
    t0 = time.perf_counter()
    ok = True
    try:
        return fn(**args)
    except Exception:
        ok = False; raise
    finally:
        log.info("tool_call", extra={
            "trace_id": trace_id, "tool": name, "ok": ok,
            "latency_ms": round((time.perf_counter() - t0) * 1000),
        })

trace_id = str(uuid.uuid4())      # one id ties the whole run together

EXAM TRAPS

Trap: Reporting average latency only. → Tail latency (p95/p99) is what users experience; the mean hides it.
Trap: Measuring cost per call instead of cost per run. → An agent run is many calls; per-run cost is the number that matters.
Trap: Ignoring resolution rate while chasing cheaper tokens. → A cheaper model that lowers resolution rate can raise total cost via retries. Track quality, not just spend.
Trap: Not capturing _request_id. → Without it, Anthropic support can't investigate a specific failing request.

PRACTICE

Q1 · Your dashboard shows flat average latency but users complain of slowness. What are you missing?

The tail — track p95/p99, not just p50/mean. A small fraction of very slow runs (retries, big generations) drives the felt experience while the average stays calm.

Q2 · You need to prove a caching change actually helped. Which usage field, and where does it come from?

cache_read_input_tokens from the usage object (7.3). Log it per call and confirm it's > 0 across identical-prefix requests; a run-level rise means the cache is being reused.

Q3 · What single identifier lets you reconstruct an entire agent run from your logs, and what ties one call to Anthropic support?

A per-run trace_id you generate stitches all calls and tool invocations together; the per-call response._request_id is what support uses to look up a specific request.