D5 · Context & Reliability › Cluster 5 · Reliability: Errors & Retries

5.4 · Rate Limits & Handling 429

RPM · ITPM · OTPM

CONCEPT Three limits, measured per minute, per model

The Messages API meters your organization on three independent limits, each measured per minute and enforced separately for each model class:

LimitCountsExceed it →
RPM — requests per minuteNumber of API calls429 rate_limit_error naming which limit tripped, with a retry-after header
ITPM — input tokens per minuteUncached input tokens (input_tokens + cache_creation_input_tokens)
OTPM — output tokens per minuteActual generated output tokens, counted in real time

You hit a 429 the moment any one of the three trips. Because limits are per model, you can run different models up to their respective ceilings at the same time.

🔑 On most models, cache_read_input_tokens do NOT count toward ITPM — cached prefix tokens are effectively free against your rate limit. A high cache-hit rate raises your effective input throughput far above the raw ITPM number. And max_tokens does not factor into OTPM — only tokens actually generated count, so a generous max_tokens carries no rate-limit penalty.

CONCEPT Token bucket — continuous, not a fixed reset

Limits use a token-bucket algorithm: your capacity is continuously replenished up toward your maximum, rather than resetting all at once on a clock boundary. So you don't get a full budget at the top of each minute — capacity trickles back steadily. This is why short bursts can trip a 429 even when your per-minute average is under the limit (a 60 RPM ceiling behaves roughly like 1 request/second). Your usage tier — Start, Build, Scale, or Custom — sets the actual numbers; organizations start on a tier and can move up over time.

⚠️ A sharp usage spike can also trigger 429s from acceleration limits, separate from your steady-state ceiling. Ramp traffic up gradually and keep usage patterns consistent.

CONCEPT Read the anthropic-ratelimit-* headers

Every response carries headers so you can steer before hitting a wall. Watch the *-remaining values and slow down as they approach zero; read *-reset (RFC 3339 timestamps) to know when capacity returns.

HeaderTells you
retry-afterSeconds to wait after a 429 before retrying
anthropic-ratelimit-requests-limit / -remaining / -resetYour RPM ceiling, requests left, and reset time
anthropic-ratelimit-input-tokens-limit / -remaining / -resetITPM ceiling, input tokens left, reset time
anthropic-ratelimit-output-tokens-limit / -remaining / -resetOTPM ceiling, output tokens left, reset time
anthropic-ratelimit-tokens-limit / -remaining / -resetCombined view of the most restrictive token limit in effect

DIAGRAM Which ceiling did I hit?

flowchart TD A["Send request"] --> B{"429 rate_limit_error?"} B -->|"no"| OK["🟢 Proceed"] B -->|"yes"| W["Read retry-after"] W --> R{"Which limit? (error msg)"} R -->|"RPM"| F["🟠 Fewer calls:
batch / dedupe"] R -->|"ITPM"| I["🟠 Fewer input tokens:
prompt caching"] R -->|"OTPM"| O["🟠 Fewer output tokens:
trim max_tokens / responses"] F --> RETRY["Wait retry-after,
retry w/ backoff"] I --> RETRY O --> RETRY classDef ok fill:#d4edda,stroke:#28a745,color:#155724; classDef gate fill:#ffe8cc,stroke:#ff9f40,color:#9a4a0a; class OK ok; class RETRY ok; class F gate; class I gate; class O gate;

CODE React to a 429 using the headers

Honor retry-after, and proactively throttle when remaining gets low.

import anthropic

try:
    resp = client.messages.create(model="claude-opus-4-8", max_tokens=1024,
                                   messages=[{"role":"user","content":"…"}])
    h = resp._request_id  # (headers via with_raw_response)
except anthropic.RateLimitError as e:
    hdrs = e.response.headers
    wait = int(hdrs.get("retry-after", "60"))       # server-told wait
    print(hdrs.get("anthropic-ratelimit-input-tokens-remaining"))
    backoff_and_retry(wait)                       # wait, then retry (5.3)

# Proactive: read remaining from a good response and slow down early
raw = client.messages.with_raw_response.create(...)
rem = int(raw.headers.get("anthropic-ratelimit-requests-remaining", "999"))
if rem < 5:
    throttle()                                    # ease off before the 429

TRADE-OFFS React to 429 vs. steer by headers

✅ Read headers + honor retry-after

  • Throttle before the 429 using *-remaining
  • retry-after gives the exact recovery wait
  • Prompt caching cuts ITPM — cached reads don't count

⚠️ Only react after the 429 fires

  • Simpler — no header bookkeeping
  • But: you always take the failure first
  • Bursts against a token bucket trip limits you could have avoided

EXAM TRAPS

Trap: "There's one tokens-per-minute limit." → There are two token limits — ITPM (input) and OTPM (output) — plus RPM. Any one tripping returns a 429.
Trap: Thinking cached tokens count against ITPM. → On most models cache_read_input_tokens do not count toward ITPM — prompt caching raises effective input throughput.
Trap: "Limits reset at the top of each minute." → They use a token bucket — continuous replenishment, not a fixed reset. A burst can 429 even when your minute average is under the limit.
Trap: Setting a huge max_tokens and expecting an OTPM hit. → OTPM counts only tokens actually generated; max_tokens doesn't factor in. No rate-limit penalty for headroom.

PRACTICE

Q1 · Name the three Messages API rate limits and what each counts.

RPM = requests per minute; ITPM = uncached input tokens per minute; OTPM = output tokens generated per minute. They're enforced separately per model; any one tripping returns a 429.

Q2 · You keep getting 429s tagged ITPM. Cheapest fix without a tier bump?

Prompt caching. Cached-read tokens don't count toward ITPM on most models, so caching a large stable prefix (system prompt, docs, tool defs) sharply raises effective input throughput.

Q3 · Which header tells you how long to wait after a 429, and which tells you when to throttle early?

retry-after gives the seconds to wait after a 429. The anthropic-ratelimit-*-remaining headers on good responses let you throttle proactively before you hit the wall.