D5 · Context & Reliability › Cluster 7 · Cost & Performance Optimization

7.1 · Cost Levers (Model, Cache, Batch)

The four dials that move spend

CONCEPT Four dials, and you turn them in order

On the Messages API you pay per input token and per output token, and the rate depends on the model. Almost every cost win comes from turning one of four dials: right-size the model tier, cache the stable prompt prefix, push non-urgent work through the Batch API, and reduce the tokens you send and receive. They compound — cheaper tokens (model), fewer paid tokens (cache), a 50% discount (batch), and fewer tokens overall (concision) all multiply together.

DialWhat it changesRough savingBest when
Model tierPer-token rate (Haiku < Sonnet < Opus)Up to ~5× cheaper input on Haiku vs OpusTask is easy enough for a smaller model
Prompt cachingCached prefix billed at ~0.1×~90% off the cached tokensLarge stable prefix reused across calls
Batch API50% off all token usageHalf priceNon-interactive / bulk / evals
Fewer tokensTotal billable tokensLinear with what you cutAlways — concise prompts & outputs
🔑 Published rates (per million tokens, input / output): Haiku 4.5 $1 / $5 · Sonnet $3 / $15 · Opus 4.8 $5 / $25. The model dial is the biggest single move — a Haiku call costs ~1/5 the input of the same call on Opus.

CONCEPT Right-size the model to the task difficulty

Reach for the smallest model that reliably clears the task. Haiku for classification, extraction, routing, short summaries — high-volume, speed-critical work. Sonnet for balanced production workloads. Opus for the hardest reasoning, long-horizon agentic runs, and correctness-critical output. Don't over-provision "to be safe": a task that Haiku handles at $1/MTok input is pure waste on Opus at $5. Don't under-provision either — a model that fails and forces a retry (or ships a wrong answer) is the most expensive option of all.

Route by difficulty: cheap model for the easy 80%, escalate to Opus only for the hard cases. A small classifier that decides which model to call often pays for itself many times over.
Anti-pattern: sending every request to Opus "for quality." You pay the top rate on work a smaller tier would clear — and usually slower, too.

DIAGRAM Turn the dials in order

flowchart TD A["A request to serve"] --> M{"Hardest model needed?"} M -->|"easy: classify / extract"| HAIKU["🟢 Haiku tier
$1 / $5 per MTok"] M -->|"balanced"| SON["🔵 Sonnet tier
$3 / $15"] M -->|"hard / agentic"| OPUS["🟠 Opus tier
$5 / $25"] HAIKU --> C{"Large stable prefix?"} SON --> C OPUS --> C C -->|"yes"| CACHE["🟢 Cache the prefix
reads ~0.1× (~90% off)"] C -->|"no"| U{"Needs an answer now?"} CACHE --> U U -->|"no: bulk / evals"| BATCH["🟢 Batch API
50% off, async ≤24h"] U -->|"yes: interactive"| LIVE["🔵 Live call
keep it concise"] classDef ok fill:#d4edda,stroke:#28a745,color:#155724; classDef mid fill:#cce5ff,stroke:#3399ff,color:#004085; classDef warn fill:#ffe8cc,stroke:#ff9f40,color:#9a4a0a; class HAIKU ok; class CACHE ok; class BATCH ok; class SON mid; class LIVE mid; class OPUS warn;

CODE Same call, three tiers — pick by difficulty

The model string is the dial. Add cache_control to reuse a stable prefix, and note where the cheaper tier is simply correct.

# Easy, high-volume task → Haiku (cheapest tokens, fastest).
label = client.messages.create(
    model="claude-haiku-4-5",          # $1 / $5 per MTok
    max_tokens=16,
    messages=[{"role": "user", "content": f"Sentiment (one word): {review}"}],
)

# Hard reasoning task → Opus, with the big shared prefix CACHED.
ans = client.messages.create(
    model="claude-opus-4-8",           # $5 / $25 per MTok
    max_tokens=1024,
    system=[{
        "type": "text",
        "text": big_policy_doc,          # stable across many calls
        "cache_control": {"type": "ephemeral"},  # reads ~0.1× base
    }],
    messages=[{"role": "user", "content": question}],
)
# Reused prefix shows up as cache_read_input_tokens on later calls.
print(ans.usage.cache_read_input_tokens)

TRADE-OFFS Cheaper tokens vs. capability

✅ Turning the dials

  • Right-sized model: up to ~5× cheaper input on easy work
  • Cache: ~90% off the reused prefix
  • Batch: flat 50% off non-urgent jobs
  • Concision: linear savings, every call

⚠️ Pushing too hard

  • Too-small a model → retries or wrong answers (net more expensive)
  • Caching a prefix that changes every call → pays the write premium, never reads
  • Batching a latency-sensitive path → users wait minutes
🔑 Optimize spend after the workload is correct, not before. A cheap wrong answer is the worst outcome — measure quality first, then squeeze cost.

EXAM TRAPS

Trap: "Use Opus everywhere for quality." → Right-size the model. Haiku input is ~1/5 the price of Opus and handles classification/extraction/routing well. Escalate to Opus only for hard or correctness-critical tasks.
Trap: Treating the four levers as either/or. → They stack. A cached prefix in a batched request on a right-sized model gets all three discounts at once.
Trap: "Prompt caching always saves money." → Only when the prefix is stable and reused. Cache writes cost ~1.25× (5-min) or 2× (1-hour); if you never get a read, you've paid a premium for nothing.
Trap: Reaching for the Batch API on an interactive path. → Batch is asynchronous (up to a 24-hour window). It's for evals and bulk jobs, never for a user waiting on a reply.

PRACTICE

Q1 · Name the four cost dials and one sentence on each.

Model tier — right-size to task difficulty (Haiku/Sonnet/Opus); prompt caching — reuse a stable prefix at ~0.1× (≈90% off); Batch API — 50% off asynchronous bulk work; fewer tokens — concise prompts and capped outputs cut billable tokens linearly.

Q2 · A task classifies support tickets by category at high volume. Which tier and why?

Haiku — classification is easy and speed/volume-critical, so the cheapest tier ($1/$5 per MTok) that does it reliably is correct. Sending this to Opus is ~5× the input cost for no quality gain.

Q3 · Why can "use a cheaper model" sometimes increase total cost?

If the model is too weak it fails or answers wrong, forcing retries, human correction, or a re-run on a bigger model — you pay twice. Right-size to the smallest model that reliably clears the task, not the smallest one that ever does.