D2 · Tool Design & MCP › Cluster 4 · Advanced Tool Use

4.3 · Streaming & Fine-Grained Tool Use

Stream partial tool args

CONCEPT Tool arguments stream as input_json_delta — accumulate before you parse

When you stream a response that contains a tool call, the arguments don't arrive whole. The content_block_start event carries a placeholder input: {} (an empty object — a slot marker). The real input arrives as a series of content_block_delta events of type input_json_delta, each carrying a partial_json string fragment. Your job is the accumulation contract:

  1. On content_block_start (type tool_use) → start an empty string.
  2. On each input_json_delta → append delta.partial_json.
  3. On content_block_stop → now parse the accumulated string with a JSON parser.
⚠️ The type mismatch is by design: input starts as an empty object, the deltas are strings. Never json.loads() a single fragment — a lone partial_json like {"loc is not valid JSON. Concatenate first, parse last.

CONCEPT Fine-grained tool streaming: skip the buffer for big params

By default, the API buffers and validates each parameter value server-side before streaming it to you — so nothing appears for a large parameter (a document, a code block) until Claude finishes generating it. Fine-grained tool streaming removes that buffer: fragments arrive as Claude generates them, cutting time-to-first-fragment for large params.

🔑 Same event shape either way (input_json_delta / partial_json). eager_input_streaming only changes when fragments arrive and whether they're pre-validated — not the accumulation contract.

DIAGRAM Fragments in, one guarded parse out

flowchart TD S["content_block_start
tool_use, input: {}"] --> ACC["buf = ''"] ACC --> D1["input_json_delta
partial_json: '{\"file'"] D1 --> D2["input_json_delta
partial_json: 'name\":\"poem'"] D2 --> D3["input_json_delta
partial_json: '.txt\"}'"] D3 --> STOP["content_block_stop"] STOP --> P{"json.loads(buf)"} P -->|"valid"| OK["🟢 run the tool"] P -->|"invalid / truncated"| ERR["🟠 tool_result is_error:true
{\"INVALID_JSON\": buf}"] classDef ok fill:#d4edda,stroke:#28a745,color:#155724; classDef gate fill:#ffe8cc,stroke:#ff9f40,color:#9a4a0a; class OK ok; class ERR gate;

CODE Enable eager streaming, accumulate, guard the parse

import json
tool_json = {}   # block index -> accumulated string

with client.messages.stream(
    model="claude-opus-4-8", max_tokens=16000,
    tools=[{
        "name": "make_file",
        "description": "Write text to a file",
        "eager_input_streaming": True,   # fine-grained: no server buffer/validate
        "input_schema": {"type": "object",
            "properties": {"filename": {"type": "string"},
                           "body": {"type": "string"}},
            "required": ["filename", "body"]},
    }],
    messages=[{"role": "user", "content": "Write a long poem to poem.txt"}],
) as stream:
    for ev in stream:
        if ev.type == "content_block_start" and ev.content_block.type == "tool_use":
            tool_json[ev.index] = ""                          # 1) start empty
        elif ev.type == "content_block_delta" and ev.delta.type == "input_json_delta":
            tool_json[ev.index] += ev.delta.partial_json      # 2) append fragment
        elif ev.type == "content_block_stop" and ev.index in tool_json:
            try:
                args = json.loads(tool_json[ev.index])          # 3) parse at STOP
            except json.JSONDecodeError:
                # invalid/truncated → hand it back as an error tool_result
                args = {"INVALID_JSON": tool_json[ev.index]}   # is_error: True

TRADE-OFFS Default buffered vs. fine-grained streaming

✅ eager_input_streaming: true

  • Fragments arrive as generated — low time-to-first-token for big params
  • Longer chunks, fewer mid-word breaks; great for live-rendering a file/code
  • Latency-sensitive UIs (write-as-you-watch)

⚠️ What you give up

  • No server-side validation → partial / invalid JSON possible
  • max_tokens can truncate a parameter mid-value
  • You must guard the parse and return is_error on failure
Turn it on per-tool only where a param is large and latency matters. Small, validation-sensitive args are fine on the default buffered path.

EXAM TRAPS

Trap: Parsing each partial_json fragment individually. → Fragments are not standalone JSON. Accumulate across input_json_delta events, parse once at content_block_stop.
Trap: Reading the tool args from the content_block_start input. → That's a placeholder empty {}; the real value is built from the deltas.
Trap: Reaching for the fine-grained-tool-streaming-2025-05-14 beta header. → Legacy. The per-tool eager_input_streaming: true field replaces it; explicit false keeps buffered streaming even if the old header is still sent.
Trap: Trusting eager-streamed input is valid JSON. → It isn't guaranteed. Guard the parse; on failure return a tool_result with is_error: true wrapping the raw string.

PRACTICE

Q1 · Walk the three-step accumulation contract for streamed tool input.

(1) On content_block_start with type tool_use, init an empty string. (2) On each input_json_delta, append delta.partial_json. (3) On content_block_stop, parse the accumulated string. The initial input:{} is only a placeholder.

Q2 · What does eager_input_streaming: true change, and what's the risk?

It disables server-side buffering/validation so large-parameter fragments stream out as Claude generates them (lower latency). The risk is that the accumulated input can be partial or invalid JSON — including max_tokens truncation — so you must guard the parse and return is_error when it fails.

Q3 · Your accumulated string won't parse. What do you send back?

A tool_result block with is_error: true whose content wraps the raw string, e.g. {"INVALID_JSON": "<the unparseable input>"} — so Claude sees the failure and can retry.