D5 · Context & Reliability › Cluster 2 · Prompt Caching

2.1 · What Prompt Caching Is & Why

Pay once, reuse the prefix

CONCEPT Cache the static prefix, skip re-processing it

Prompt caching stores the result of processing a static prefix of your prompt so that repeated requests carrying that same prefix skip re-processing it. On the Messages API, every request is stateless — you resend the whole prompt each time. If a large, unchanging chunk sits at the front (a long system prompt, a tool catalog, a retrieved document, a few-shot block), you normally pay full input price to re-tokenize and re-process it on every call. Caching lets you pay to write it once, then read it back at a fraction of the cost on subsequent calls.

🔑 It caches a prefix, not the whole prompt. Everything up to a cache breakpoint is stored; whatever comes after (the volatile question, per-request data) is processed fresh every time.

CONCEPT Why it's worth it — cost and latency

The win is two-fold. Cost: a cache read costs about 0.1× the base input-token price — roughly a 90% discount on the cached portion. Latency: cached tokens are served from the cache instead of being processed, so time-to-first-token on a big shared prefix drops sharply. The one-time cost is the write: a 5-minute cache write is 1.25× base input price (a 1-hour write is 2× — covered in 2.4). So a cached prefix breaks even after just two requests: 1.25× + 0.1× = 1.35× versus 2× if you never cached.

Token treatmentPrice vs. base inputWhen
Uncached input1×Content after the breakpoint, or no caching
Cache write (5-min)1.25×First request — prefix written to cache
Cache write (1-hour)2×Opt-in longer TTL (see 2.4)
Cache read (hit)~0.1×Later requests reuse the identical prefix
Ideal when the same large context is reused across many calls: agent system prompts + tool definitions, a document you ask many questions about, long few-shot examples, or a multi-turn conversation whose history keeps growing.

DIAGRAM First call writes, later calls read

flowchart TD A["Large static prefix
(system + tools + docs)"] --> B{"Prefix already
in cache?"} B -->|"no — first call"| WRITE["🟠 Cache WRITE
process once · 1.25× price"] B -->|"yes — identical bytes"| READ["🟢 Cache READ
skip processing · ~0.1× price"] WRITE --> SUF["Process volatile suffix
(the new question) · 1×"] READ --> SUF SUF --> OUT["Model response"] classDef ok fill:#d4edda,stroke:#28a745,color:#155724; classDef gate fill:#ffe8cc,stroke:#ff9f40,color:#9a4a0a; classDef neutral fill:#e7f1ff,stroke:#4a90d9,color:#1a4971; class READ ok; class WRITE gate; class SUF neutral;

CODE One breakpoint, two requests

Mark the end of the reusable prefix with cache_control. The first call writes; the second, with a byte-identical system prompt, reads it back. Inspect usage to confirm.

# First request — writes the prefix to cache
r1 = client.messages.create(
    model="claude-opus-4-8", max_tokens=1024,
    system=[{
        "type": "text",
        "text": LARGE_STATIC_PROMPT,       # same bytes every call
        "cache_control": {"type": "ephemeral"},  # breakpoint here
    }],
    messages=[{"role": "user", "content": "First question"}],
)
print(r1.usage.cache_creation_input_tokens)  # > 0  → written (paid 1.25×)

# Second request — identical prefix, different question
r2 = client.messages.create(
    model="claude-opus-4-8", max_tokens=1024,
    system=[{"type": "text", "text": LARGE_STATIC_PROMPT,
             "cache_control": {"type": "ephemeral"}}],
    messages=[{"role": "user", "content": "Second question"}],
)
print(r2.usage.cache_read_input_tokens)      # > 0  → hit (paid ~0.1×)

TRADE-OFFS When caching pays off — and when it doesn't

✅ Cache it

  • Large prefix reused across many requests
  • Prefix is stable — same bytes each call
  • Prefix clears the model's minimum token floor (2.3)
  • Break-even after ~2 requests within the TTL window

⚠️ Don't bother

  • Prefix changes from the first token every call
  • Prefix is tiny (below the cacheable minimum → silently uncached)
  • Content is used once and never repeated
  • You'd pay the 1.25× write premium for zero reads

EXAM TRAPS

Trap: "Caching stores the model's output / the answer." → No. It caches the processed input prefix so it needn't be re-processed. The completion is generated fresh every call.
Trap: "It caches the whole prompt." → It caches a prefix up to a breakpoint. Volatile content after the breakpoint is always processed at full price.
Trap: "Caching is free." → The first write costs 1.25× base input (2× for the 1-hour TTL). You only come out ahead once reads outnumber that premium — roughly two requests.
Trap: Assuming caching is stateful/persistent like a session. → The API is stateless; the cache is an ephemeral, prefix-keyed optimization with a short TTL (2.4), not a stored conversation.

PRACTICE

Q1 · What exactly does prompt caching store, and what does it save?

It stores the processed static prefix of the prompt (system, tools, leading messages). On a later request with the identical prefix, the model skips re-processing it — saving both cost (cache reads ≈ 0.1× base input) and latency (fewer tokens to process).

Q2 · You cache a prefix but only ever send one request with it. Good idea?

No. You pay the 1.25× write premium and get zero reads back, so it costs more than not caching. Caching pays off only when the same prefix is reused across multiple requests within the TTL.

Q3 · Roughly how many reuses to break even on a 5-minute cache?

About two: the write costs 1.25× and one read costs 0.1× (1.35× total) versus 2× if you paid full price for the prefix twice. Beyond that, every additional read is nearly free.