CCA-F › Domain 5 › Silent Failures — Passes but Wrong

🕳️ Silent Failures — the request passes but the result is wrong

HTTP 200 · stop_reason:"end_turn" · no error flag · and still wrong — the dangerous quadrant

WHY THIS PAGE The failures the API can't tell you about

The companion page covered the loud failures — the system tells you something is wrong: a 400, a refusal, an is_error:true, a max_tokens cut-off. Those are the safe ones — a signal fires and your code can branch on it.

This page is about the silent class: the request is well-formed, validation passes, the model runs, and you get back HTTP 200 with stop_reason:"end_turn" and no error anywhere — but the content is wrong, fabricated, or logically broken. Nothing in the API surfaces it. It's billed, it looks successful, and downstream systems trust it. That's why it's more dangerous than a 400: a 400 stops the pipeline; a wrong-but-valid answer flows through it.

The core danger: a 400 fails closed (nothing happens). A silent-wrong result fails open — a hallucinated SQL filter, a fabricated citation, or a wrong tool argument sails downstream and, in an agent loop, compounds across turns. No status code will ever catch it.

THE QUADRANT Everything is green, the answer is still wrong

flowchart TD R["Your request"] --> V{"Validation"} V -->|"invalid"| E400["400 — caught, loud"]:::loud V -->|"valid"| M["MODEL INFERENCE"] M --> S{"stop_reason"} S -->|"refusal"| REF["refusal — caught, loud"]:::loud S -->|"max_tokens"| MT["truncation — caught, loud"]:::loud S -->|"end_turn ✅"| OK["200 OK · looks perfect"]:::ok OK --> Q{"Is the content actually correct?"} Q -->|"yes"| DONE["✔ genuinely done"]:::ok Q -->|"NO — but nothing flags it"| SILENT["🕳️ SILENT FAILURE
hallucination · logic error · ungrounded
wrong tool arg · schema-valid-but-wrong"]:::silent SILENT -.->|"only caught by"| GUARD["YOUR verification layer:
evals · verifier pass · grounding ·
deterministic checks · human review"]:::guard classDef loud fill:#fef3c7,stroke:#b45309,color:#7c2d12; classDef ok fill:#dcfce7,stroke:#15803d,color:#14532d; classDef silent fill:#fee2e2,stroke:#b91c1c,color:#7f1d1d; classDef guard fill:#e7ecff,stroke:#3b5bdb,color:#1e3a8a;

Every gate says pass. The correctness check on the right does not exist unless you build it.

TAXONOMY The ways a valid request returns a wrong answer

1 · Hallucination (fabrication)

The model invents something that doesn't exist — a fake citation, a non-existent API parameter, a table/column that isn't in the schema, a plausible-but-wrong number. Well-formed, confident, and false.

2 · Logical / reasoning error

Right approach, wrong execution: an arithmetic slip, an inverted condition (> vs <), a mis-ordered step, an off-by-one. The prose reasoning looks sound; the conclusion doesn't follow.

3 · Ungrounded / unfaithful to context

You supplied documents (RAG), and the answer contradicts or ignores them — a faithfulness failure. The retrieval was fine; the model didn't stay anchored to it.

4 · Instruction drift / partial compliance

The model quietly drops one of your constraints — "only rows after 2024", "answer in JSON", "don't include PII" — and satisfies the rest. Looks complete; silently violates the spec.

5 · Schema-valid but semantically wrong

The trap of structured outputs. strict:true / output_config.format guarantees the shape — every field present, right types — not the values. A perfectly-validated object can carry a wrong amount, a swapped ID, or a mislabelled category. Schema conformance ≠ correctness.

6 · Wrong tool argument (the agent-loop killer)

The model emits a well-formed tool_use with wrong values — the right function, bad arguments (queries last month instead of last week, filters the wrong column). Your tool runs fine, returns a real tool_result with no is_error, and the wrong data feeds the next turn. Nothing errored; everything is wrong.

7 · Sycophancy / anchoring to a wrong premise

The user asserts something false ("since our fiscal year starts in March…") and the model agrees and builds on it rather than correcting it. The output is internally consistent with a bad assumption.

8 · Prompt-injection-induced output

Retrieved content or a tool result contains instructions ("ignore previous rules and…") and the model follows them. The request was valid; the data hijacked the answer. Security failure wearing a 200 OK.

WHO CATCHES WHAT Loud vs silent

FailureHTTPDoes the API flag it?Who must catch it
Invalid request400Yes — error typefix request construction
Refusal200Yes — stop_reason:"refusal"branch on stop_reason
Tool failure you reported200Yes — is_error:true (you set it)the model self-corrects
Truncation200Yes — stop_reason:"max_tokens"continue / raise cap
Hallucination / logic / ungrounded / wrong values200NO — nothing. end_turn, no flag.YOU — evals, verifier, grounding, deterministic checks, human review
The whole bottom row shares one property: the API is working exactly as designed. There is no status code for "correct". Correctness is an application-layer guarantee — it is never something the Messages API returns.

DEFENCES How to turn silent failures into loud ones

You can't ask the API for correctness, so the job is to add a check that fires — convert an invisible wrong answer into a signal you can branch on, retry, or escalate.

Failure modeDefence that makes it loud
Hallucinated facts / entitiesGrounding + citations (answer only from supplied docs); reject if a claim has no source
Logic / reasoning slipsVerifier pass in a fresh context (LLM-as-judge with a clean window); self-consistency (sample N, take the majority)
Ungrounded / unfaithfulFaithfulness eval — does every sentence trace to a retrieved chunk?
Instruction driftGolden-set evals asserting each constraint; automated constraint checks post-generation
Schema-valid but wrong valuesDeterministic validation beyond the schema — range checks, referential integrity, business rules on the parsed object
Wrong tool argumentValidate before executing — e.g. dry-run / EXPLAIN a query, confirm the arg is in range, or an approval hook for risky calls
Sycophancy / bad premiseSystem prompt to challenge assumptions; a verifier that checks the premise, not just the answer
Prompt injectionRuntime guardrails, treat tool/retrieved content as untrusted data, least-privilege tools, isolate instructions from data
We already built two of these in the nl2sql-uc-agent project: verifier.py runs a fresh-context verdict on the generated SQL (a second, clean pair of eyes), and validate_sql() runs an EXPLAIN to make a wrong-but-well-formed query fail loudly before it ever executes. That's the pattern: a deterministic check where one exists, an independent LLM judge where one doesn't.
Layer them by cost: deterministic checks first (free, exact — schema, ranges, EXPLAIN, referential integrity), then an LLM verifier (catches reasoning/grounding), then offline evals on a golden set (catches drift over time), then human-in-the-loop for the highest-stakes actions. Cheap-and-exact before expensive-and-probabilistic.

WHY MORE DANGEROUS than a 400 — the one-liner

• A 400 fails closed: loud, unbilled, nothing happens downstream. Annoying, safe.
• A silent-wrong result fails open: HTTP 200, billed, looks perfect, and flows into whatever consumes it — a dashboard, a transaction, the next tool call.
• In an agent loop it compounds: turn 3's hallucination becomes turn 4's premise. By the time it's visible, the trajectory is far off course and expensive to unwind.

EXAM ONE-LINERS

• There is no status code for "correct" — a wrong answer returns 200 + end_turn, identical to a right one.
• Structured outputs / strict:true guarantee shape, not truth — validate the values yourself.
• A wrong tool argument produces a valid tool_result with no is_error — validate args before executing.
• Correctness is an application-layer guarantee: evals + a verification pass + deterministic checks, not an API feature.
• Prefer deterministic checks where they exist (EXPLAIN, ranges, schema+rules); use an LLM verifier / self-consistency only where they don't.