D5 · Context & Reliability › Cluster 7 · Cost & Performance Optimization

7.2 · The Batch API (async, ~50% off)

Half price for non-urgent work

CONCEPT Trade immediacy for a 50% discount

The Message Batches API (POST /v1/messages/batches) processes many Messages requests asynchronously at 50% of standard token prices — input, output, and cache usage all halved. You submit a bundle of requests, poll for completion, then retrieve results later. You are trading immediacy (results come back within a window, not in a few seconds) for cost. It is the right tool for evals, bulk classification, offline enrichment, dataset generation — anything where "in the next hour or so" is soon enough.

PropertyValue
Discount50% off standard pricing (all token types)
ProcessingAsynchronous; most batches finish in < 1 hour
WindowResults ready when all requests finish or after 24 hours, whichever comes first; unfinished requests expire at 24h
Result retentionDownloadable for 29 days after creation
Batch sizeUp to 100,000 requests or 256 MB, whichever comes first
Statusprocessing_status: in_progress → ended
Per-request resultsucceeded · errored · canceled · expired
🔑 Two clocks: the 24-hour processing window (past it, unfinished requests expire and aren't billed) and the 29-day result window (after that you can still see the batch, but not download results).

CONCEPT Async by design — results arrive in any order

The batch is not a stream. You get back a batch id, poll processing_status until it reads ended, then pull results from the results_url. Results come back in any order, so every request carries a custom_id you set — key results by custom_id, never by position. Each per-request result has its own type (succeeded / errored / canceled / expired): one bad request doesn't sink the batch, and expired requests aren't billed. Nearly all Messages features work inside a batch — vision, tools, prompt caching — with two exceptions: stream: true is meaningless (results are one file), and max_tokens: 0 cache pre-warming isn't supported.

Not for interactive paths. A chat reply, a voice turn, a "generate this now" button — none of these can wait a batch window. Batch is for work that has no user watching the clock.

DIAGRAM Submit → poll → retrieve

flowchart TD A["Bundle N requests
each with a custom_id"] --> B["POST /v1/messages/batches"] B --> C{"processing_status?"} C -->|"in_progress"| C C -->|"ended (≤24h)"| D["GET results_url
(.jsonl, any order)"] D --> E{"per-request result"} E -->|"succeeded"| OK["🟢 read message, key by custom_id"] E -->|"errored"| ERR["🟠 inspect error; maybe resubmit"] E -->|"expired"| EXP["🔴 24h window passed
not billed → resubmit"] classDef ok fill:#d4edda,stroke:#28a745,color:#155724; classDef warn fill:#ffe8cc,stroke:#ff9f40,color:#9a4a0a; classDef bad fill:#f8d7da,stroke:#dc3545,color:#721c24; class OK ok; class ERR warn; class EXP bad;

CODE Create, poll, retrieve — keyed by custom_id

Submit a bundle, wait for ended, then match results back by custom_id (not order).

import time
from anthropic.types.message_create_params import MessageCreateParamsNonStreaming
from anthropic.types.messages.batch_create_params import Request

batch = client.messages.batches.create(requests=[
    Request(
        custom_id=f"ticket-{i}",                # YOUR key — results come back unordered
        params=MessageCreateParamsNonStreaming(
            model="claude-haiku-4-5",
            max_tokens=16,
            messages=[{"role": "user", "content": f"Category: {t}"}],
        ),
    )
    for i, t in enumerate(tickets)
])

# Poll until the whole batch has ended (async — can take up to 24h).
while client.messages.batches.retrieve(batch.id).processing_status != "ended":
    time.sleep(30)

results = {}
for r in client.messages.batches.results(batch.id):
    if r.result.type == "succeeded":
        results[r.custom_id] = r.result.message.content[0].text  # key by custom_id

TRADE-OFFS Batch vs. live request

✅ Batch API

  • 50% off every token — the single biggest flat discount
  • High throughput; up to 100k requests per submission
  • Full feature set (tools, vision, caching)
  • Per-request failures are isolated

⚠️ Live request

  • Immediate response — required for anything interactive
  • Streaming for low time-to-first-token
  • Full price, but no waiting
  • Use for chat, agents, user-facing calls
🔑 Decision rule: Can this wait up to 24 hours? Yes → batch (half price). No → live request (7.3 for making it feel fast).

EXAM TRAPS

Trap: "Batch results stream back as they finish." → No. You poll processing_status until ended, then retrieve a results file. Nearly all finish under an hour, but the window is up to 24 hours.
Trap: Matching results to inputs by position. → Results return in any order. Set a custom_id on every request and key results by it.
Trap: "Batch is a different, weaker API." → It runs the same Messages requests with the same features (tools, vision, caching). The differences: async delivery, 50% price, no live streaming, no max_tokens: 0 pre-warming.
Trap: Confusing the two clocks. → 24 hours is the processing window (unfinished requests expire, unbilled). 29 days is how long finished results stay downloadable.

PRACTICE

Q1 · What discount does the Batch API give, and on what?

50% off standard token pricing — applied to input, output, and cache token usage — in exchange for asynchronous processing instead of an immediate response.

Q2 · A batch request is still processing at the 24-hour mark. What happens, and are you billed?

It expires (result type expired) — the processing window closed. You are not billed for expired requests; resubmit them if you still need them.

Q3 · Why must you set a custom_id on every batched request?

Because results come back in any order. The custom_id is how you map each result to the input it answers — keying by array position would misalign them.