CONCEPT Every response carries a usage object
You cannot manage what you don't measure. Every Messages API response returns a usage
object — non-streaming and streaming alike — and it is the ground truth for cost and cache health.
The four fields you must know:
| Field | What it counts | Relative price |
|---|---|---|
input_tokens | Uncached prompt tokens processed at full price | 1× input rate |
output_tokens | Tokens the model generated (incl. thinking) | 1× output rate |
cache_creation_input_tokens | Tokens written to the cache this request | ~1.25× (5m TTL) / 2× (1h TTL) |
cache_read_input_tokens | Tokens served from the cache this request | ~0.1× input rate |
input_tokens is the uncached remainder only, not the total prompt.
The full prompt size = input_tokens + cache_creation_input_tokens + cache_read_input_tokens. If an
agent ran for hours but input_tokens shows 4K, the rest was served from cache — sum the three,
don't read one field.CONCEPT Deriving cost and cache hit-rate
The usage object is the raw material for the two numbers finance and SRE actually care about.
Cost is a weighted sum of the four fields against the model's per-token prices — cache reads at ~0.1× and cache writes at the 1.25×/2× premium, so caching only pays off after repeated reads (break-even is two requests on the 5-minute TTL, three on the 1-hour TTL).
Cache hit-rate tells you whether your prefix is actually being reused. Compute it as
cache_read / (cache_read + cache_creation + input_tokens) across requests. If
cache_read_input_tokens is zero across repeated identical-prefix requests, a silent
invalidator is at work — a timestamp or UUID in the system prompt, unsorted JSON, or a varying tool set.
usage object per call and aggregate downstream. Storing derived
cost only throws away the detail you need when prices change or you want to attribute spend by tier.DIAGRAM From usage to the two numbers that matter
(uncached)"] R --> O["output_tokens"] R --> CW["cache_creation_input_tokens"] R --> CR["cache_read_input_tokens"] I --> COST["💰 Cost = Σ tokens × price
(reads ~0.1×, writes 1.25–2×)"] O --> COST CW --> COST CR --> COST I --> HIT["📊 Hit-rate = read /
(read + create + input)"] CW --> HIT CR --> HIT HIT --> AL{"read == 0
on repeat?"} AL -->|"yes"| BAD["🔴 Silent invalidator
audit the prefix"] AL -->|"no"| OK["🟢 Cache working"] classDef ok fill:#d4edda,stroke:#28a745,color:#155724; classDef gate fill:#ffe8cc,stroke:#ff9f40,color:#9a4a0a; classDef bad fill:#f8d7da,stroke:#dc3545,color:#721c24; class OK ok; class COST gate; class BAD bad;
CODE Read usage, compute cost + hit-rate
The response object exposes all four fields directly. Streaming works too: the final message from
stream.get_final_message() carries the same usage.
# Prices per 1M tokens (Opus 4.8 example): $5 input / $25 output. # Cache read ~0.1x input; cache write ~1.25x input (5-minute TTL). IN, OUT, C_READ, C_WRITE = 5.0, 25.0, 0.5, 6.25 resp = client.messages.create(model="claude-opus-4-8", max_tokens=1024, system=SYSTEM, messages=msgs) u = resp.usage def cost_usd(u): return (u.input_tokens*IN + u.output_tokens*OUT + u.cache_creation_input_tokens*C_WRITE + u.cache_read_input_tokens*C_READ) / 1_000_000 prompt_total = u.input_tokens + u.cache_creation_input_tokens + u.cache_read_input_tokens hit_rate = u.cache_read_input_tokens / prompt_total if prompt_total else 0 emit_metric("llm.cost_usd", cost_usd(u)) emit_metric("llm.cache_hit_rate", hit_rate) emit_metric("llm.output_tokens", u.output_tokens) # feeds OTPM tracking (8.3)
TRADE-OFFS Log raw usage vs. log derived cost
✅ Log the raw usage object
- Survives price changes — recompute cost anytime
- Cache hit-rate and token throughput both derivable
- Attribute spend by tenant, route, or model later
⚠️ Log only a computed dollar figure
- Smaller records, one number to chart
- But: hard-codes today's prices into history
- Can't reconstruct hit-rate or ITPM/OTPM after the fact
EXAM TRAPS
input_tokens as the whole prompt. → It's the
uncached remainder. Total = input_tokens + cache_creation_input_tokens +
cache_read_input_tokens.cache_creation_input_tokens and
cache_read_input_tokens — not cache_write, cached_tokens, or
prompt_tokens. Don't invent usage fields.message_start (input) and cumulatively on message_delta (output); the final message
carries the complete object.PRACTICE
Q1 · Name the four token fields on the usage object.
input_tokens, output_tokens,
cache_creation_input_tokens (tokens written to cache), and cache_read_input_tokens
(tokens served from cache). The two cache fields are how you compute cache hit-rate.
Q2 · cache_read_input_tokens is 0 on every repeated request. What's wrong?
A silent cache invalidator is changing the prefix each time — a
datetime.now() or UUID in the system prompt, unsorted json.dumps(), or a tool set
that varies per request. Diff the rendered prompt bytes between two requests to find it.
Q3 · How do you compute total prompt size from the usage object?
Sum the three input-side fields:
input_tokens + cache_creation_input_tokens + cache_read_input_tokens. Reading
input_tokens alone undercounts because it excludes everything served from or written to cache.