D4 · Prompt Engineering › Cluster 7 · Structured Outputs & JSON

7.5 · Streaming & Parsing Structured Output

Incremental JSON safely

CONCEPT Partial JSON is not parseable JSON

When you stream a response, JSON arrives one fragment at a time. You cannot json.loads() a half-finished object — you must accumulate every delta first, then parse once the block is complete. Two delta types carry structured content:

SourceDelta typeField to concatenate
Structured output (7.4) — JSON in the texttext_deltadelta.text
Tool use (7.3) — args in a tool_use blockinput_json_deltadelta.partial_json
🔑 The SSE order is message_start → content_block_start → content_block_delta* → content_block_stop → message_delta → message_stop. For a tool block, content_block_start carries an empty input: {}; the real arguments stream as input_json_delta.partial_json string fragments that concatenate to the full JSON.

CONCEPT Let the SDK helpers do the accumulation

You rarely hand-assemble deltas. The SDK helpers accumulate for you and hand back a complete, parsed object:

⚠️ Streaming is required for large max_tokens (above ~16K) to avoid SDK HTTP timeouts. Use .get_final_message() / .finalMessage() to get the complete response even when you don't handle individual events — you get timeout protection for free.

DIAGRAM Accumulate, then parse — never mid-stream

flowchart TD S["content_block_start (input: {})"] --> D["content_block_delta ×N"] D -->|"tool_use → partial_json"| ACC["Concatenate fragments
into a buffer"] D -->|"structured output → text"| ACC ACC --> MID{"Parse a fragment now?"} MID -->|"❌ tempting"| BAD["🔴 json.loads() on partial
→ JSONDecodeError"] MID -->|"✅ wait"| STOP["content_block_stop"] STOP --> PARSE["🟢 json.loads(buffer) once complete
(or use get_final_message())"] classDef ok fill:#d4edda,stroke:#28a745,color:#155724; classDef bad fill:#f8d7da,stroke:#dc3545,color:#721c24; class PARSE ok; class BAD bad;

CODE Helper path vs. manual accumulation

# ✅ Recommended — let the helper accumulate & parse the tool input for you
with client.messages.stream(
    model="claude-opus-4-8", max_tokens=1024,
    tools=[record_contact_tool], tool_choice={"type": "tool", "name": "record_contact"},
    messages=[{"role": "user", "content": "Jane Doe, jane@co.com, enterprise."}],
) as stream:
    for text in stream.text_stream:
        render(text)                       # show progress live if you want
    final = stream.get_final_message()      # fully assembled + parsed

block = next(b for b in final.content if b.type == "tool_use")
data  = block.input                         # already a dict — no manual parsing

# ── Manual accumulation (low-level events): buffer, THEN parse ──
buf = []
for event in client.messages.create(
        model="claude-opus-4-8", max_tokens=1024, stream=True,
        tools=[record_contact_tool],
        tool_choice={"type": "tool", "name": "record_contact"},
        messages=[{"role": "user", "content": "Jane Doe, jane@co.com, enterprise."}]):
    if event.type == "content_block_delta" and event.delta.type == "input_json_delta":
        buf.append(event.delta.partial_json)   # fragments of the JSON args
    elif event.type == "content_block_stop":
        data = json.loads("".join(buf))       # parse ONLY when the block is done
Never call json.loads() on the running buffer mid-stream — a partial object raises JSONDecodeError. Parse only after content_block_stop (or read the helper's final message).

TRADE-OFFS SDK helper vs. raw event loop

✅ Helper (stream() + final message)

  • Accumulation, parsing, and timeout handling done for you
  • Tool inputs arrive as parsed dicts
  • Still lets you render partial text live

⚠️ Raw stream=True event loop

  • Lower memory, full control over each event
  • You must buffer partial_json / text yourself
  • Easy to mis-parse mid-stream — the classic bug

EXAM TRAPS

Trap: Parsing JSON on each delta. → Deltas are fragments; a partial object won't parse. Accumulate all of them, then json.loads() after content_block_stop.
Trap: Reading tool args from content_block_start. → Its input is {}; the real arguments stream as input_json_delta.partial_json fragments.
Trap: Confusing the delta types. → Structured-output JSON streams as text_delta (delta.text); tool arguments stream as input_json_delta (delta.partial_json).
Trap: Non-streaming request with a large max_tokens. → Above ~16K the SDK can hit an HTTP timeout. Stream and use get_final_message() / finalMessage().

PRACTICE

Q1 · Why can't you parse structured output while it's still streaming?

Because each delta is a fragment of the JSON — a partial object isn't valid and json.loads() raises. You must accumulate every text_delta / input_json_delta and parse only after content_block_stop.

Q2 · Which field carries streamed tool-call arguments, and which carries structured-output JSON?

Tool arguments stream in input_json_delta.partial_json (concatenate the fragments); structured-output JSON streams in text_delta.text. The tool block's initial content_block_start.input is empty ({}).

Q3 · What's the simplest safe way to get the parsed result when streaming?

Use the SDK helper — client.messages.stream(...) then stream.get_final_message() (Python) or await stream.finalMessage() (TypeScript). It accumulates the deltas, hands back the complete message with tool inputs already parsed, and gives timeout protection for free.