D5 · Context & Reliability › Cluster 2 · Prompt Caching

2.4 · Cache TTL (5m / 1h) & Hits/Misses

The cache is ephemeral

CONCEPT Two TTLs — 5-minute default, 1-hour opt-in

Cache entries are ephemeral — they expire. There are two time-to-live options:

Each hit refreshes the TTL. Every time cached content is read, its expiry clock resets from that moment. So a prefix that's used at least once inside the window stays warm indefinitely — continuous traffic keeps the cache alive without any re-warming. The TTL only matters during gaps in usage.

🔑 Refresh-on-hit means the "5 minutes" is a 5-minute idle timeout, not a hard 5-minute lifetime. Steady traffic keeps it warm; a >5-minute quiet spell lets it lapse.

CONCEPT Hit, miss, and the fallback to a write

On each request the API looks for a live cache entry matching the prefix (walking back up to 20 content blocks from the breakpoint):

A miss is never an error — it just costs a write. The danger is a silent miss where you expected a hit (a TTL lapse, or a prefix-invalidating byte change from 2.3).

usage fieldMeaningPrice paid
cache_read_input_tokensServed from cache (hit)~0.1×
cache_creation_input_tokensWritten to cache (miss → write)1.25× (5m) / 2× (1h)
input_tokensUncached remainder (after last breakpoint)1×
⚠️ input_tokens is only the uncached tail. Total prompt size = input_tokens + cache_creation_input_tokens + cache_read_input_tokens. A tiny input_tokens after a long run means the rest was served from cache — check the sum.

DIAGRAM Lookup decides read vs. write

flowchart TD A["Request with cached prefix"] --> Q{"Live matching
entry in cache?"} Q -->|"yes"| HIT["🟢 HIT · read ~0.1×
refresh TTL"] Q -->|"no: expired / first call /
prefix byte changed"| MISS["🟠 MISS → WRITE
re-cache · 1.25× (or 2× for 1h)"] HIT --> NEXT["next request within TTL
→ another hit"] MISS --> NEXT2["next request within TTL
→ now a hit"] IDLE["> TTL idle gap"] -.->|"entry lapses"| Q classDef ok fill:#d4edda,stroke:#28a745,color:#155724; classDef gate fill:#ffe8cc,stroke:#ff9f40,color:#9a4a0a; classDef neutral fill:#e7f1ff,stroke:#4a90d9,color:#1a4971; class HIT ok; class MISS gate; class NEXT neutral; class NEXT2 neutral; class IDLE neutral;

CODE Choosing a TTL and verifying hits

Use the 1-hour TTL only for bursty traffic with gaps longer than 5 minutes — the doubled write cost needs more reads to pay off. Then confirm hits via usage.

resp = client.messages.create(
    model="claude-opus-4-8", max_tokens=1024,
    system=[{"type": "text", "text": LARGE_PROMPT,
             # 1-hour TTL: survives longer idle gaps, costs 2× to write
             "cache_control": {"type": "ephemeral", "ttl": "1h"}}],
    messages=[{"role": "user", "content": question}],
)

u = resp.usage
if u.cache_read_input_tokens:
    print("HIT — served from cache at ~0.1x")
elif u.cache_creation_input_tokens:
    print("MISS → WRITE — re-cached this request")

# Zero reads across identical-prefix calls? Either the TTL lapsed
# between calls, or a silent invalidator changed the prefix bytes (2.3).

TRADE-OFFS 5-minute vs. 1-hour TTL

✅ 1-hour TTL

  • Survives idle gaps up to an hour without re-writing
  • Fewer cold misses in bursty, spaced-out traffic
  • Still refreshes on every hit

⚠️ Its cost

  • Write is 2× base input (vs 1.25× for 5-min)
  • Needs ≥3 reuses to break even (2× + 0.2× vs 3× uncached)
  • Wasteful under continuous traffic — the 5-min default stays warm anyway

EXAM TRAPS

Trap: "The 5-minute cache always dies after 5 minutes." → It's an idle timeout — each hit refreshes the TTL. Continuous use keeps it alive; only a >5-min gap lets it expire.
Trap: "A cache miss throws an error." → A miss just falls back to a write (re-caches at 1.25×/2×). No error — the risk is a silent, unexpected miss.
Trap: Reading input_tokens as the total prompt size. → It's only the uncached tail. Add cache_read_input_tokens + cache_creation_input_tokens for the full input.
Trap: Reaching for the 1-hour TTL by default. → It doubles the write cost. Under steady traffic the 5-minute default stays warm on its own; use 1h only for gappy, bursty patterns.

PRACTICE

Q1 · Does a cache entry expire exactly 5 minutes after it's written?

No — each hit refreshes the TTL, so the 5-minute clock resets on every read. It expires only after a >5-minute idle gap with no hits. Continuous traffic keeps it warm indefinitely.

Q2 · What happens on a cache miss?

The request falls back to a write: it re-processes and re-caches the prefix at the write price (1.25× for 5-min, 2× for 1h), reported in cache_creation_input_tokens. It's not an error.

Q3 · When is the 1-hour TTL worth its 2× write cost?

For bursty traffic with gaps longer than 5 minutes, where the 5-minute entry would lapse between bursts. It needs at least three reuses to break even, so under continuous traffic the 5-minute default is cheaper.