D4 · Prompt Engineering › Cluster 8 · Reliability, Hallucination & Iteration

8.1 · Reducing Hallucinations

Give Claude an out

CONCEPT Lower the pressure to always answer

A hallucination is confident text that is factually wrong or unsupported by the context you gave. The root cause is often social pressure baked into the prompt: if the only acceptable-looking output is a complete answer, the model will manufacture one rather than appear unhelpful. The single most effective fix is to explicitly permit uncertainty — tell Claude it is allowed, and expected, to say "I don't know" or "I don't have enough information to confidently assess this."

🔑 Anthropic's #1 documented technique: allow Claude to say "I don't know." A single sentence granting permission to admit uncertainty drastically reduces false information.

The other three basic strategies all work by grounding the answer in the source text rather than the model's parametric memory:

TechniqueWhat you ask Claude to doWhy it reduces hallucination
Allow "I don't know"Admit uncertainty instead of guessingRemoves the pressure that forces a fabricated answer
Answer only from contextUse only the provided documents, not general knowledgeCuts off the parametric-memory path where hallucinations originate
Extract quotes firstPull word-for-word quotes before analyzing (esp. docs >20k tokens)Anchors the response to the actual text
Verify with citationsCite a supporting quote per claim; retract if none existsMakes each claim auditable and self-checking
⚠️ These techniques reduce hallucinations; they do not eliminate them. Always validate critical information for high-stakes decisions.

CONCEPT Advanced verification techniques

Beyond the basics, four techniques catch hallucinations that slip through:

DIAGRAM Route the request through a grounding gate

flowchart TD Q["User question + source docs"] --> P{"Prompt gives Claude
an out?"} P -->|"no — 'you must answer'"| H["🔴 Model invents an answer
(hallucination risk)"] P -->|"yes — 'say I don't know'"| G["Answer only from context"] G --> E{"Supporting quote
found in source?"} E -->|"yes"| A["🟢 Claim, cited to the quote"] E -->|"no"| I["🟠 'I don't know' /
retract the claim"] classDef ok fill:#d4edda,stroke:#28a745,color:#155724; classDef gate fill:#ffe8cc,stroke:#ff9f40,color:#9a4a0a; classDef bad fill:#f8d7da,stroke:#dc3545,color:#721c24; class A ok; class I gate; class H bad;

CODE BAD → GOOD prompt contrast

The bad prompt demands an answer with no escape hatch and no grounding. The good prompt grants permission to be uncertain, restricts Claude to the source, and requires quote-backed claims.

# ❌ BAD — no out, no grounding: invites confident fabrication
As our M&A advisor, analyze this report on the acquisition of AcmeCo.
<report>{{REPORT}}</report>
Give me the financial projections, integration risks, and regulatory hurdles.

# ✅ GOOD — permission to be uncertain + answer only from context + quote-first
As our M&A advisor, analyze this report on the acquisition of AcmeCo.
<report>{{REPORT}}</report>

1. First, extract exact word-for-word quotes from the report that are most
   relevant to financials, integration risk, and regulatory hurdles.
   If you cannot find relevant quotes, state "No relevant quotes found."
2. Base your analysis ONLY on the extracted quotes, referencing each by number.
   Use only the report — do not use outside knowledge.
3. If the report lacks the information, say
   "I don't have enough information to confidently assess this."

TRADE-OFFS Grounding vs. fluency

✅ Grounded, quote-first prompting

  • Auditable — every claim traces to a quote
  • Far fewer fabrications on long documents
  • "I don't know" is an acceptable, honest output

⚠️ Costs you pay

  • Longer outputs and more tokens (quotes + analysis)
  • More reserved, less fluent-sounding answers
  • Extra latency from CoT / best-of-N verification
Reserve the heaviest verification (best-of-N, iterative refinement) for high-stakes, accuracy-critical tasks; a simple "you may say I don't know" line is cheap enough to add everywhere.

EXAM TRAPS

Trap: "Raise the stakes — tell it accuracy is CRITICAL and it MUST answer." → This increases hallucination by removing the escape hatch. The fix is the opposite: give Claude permission to say "I don't know."
Trap: Assuming lower temperature eliminates hallucinations. → Temperature affects variance, not grounding. A confident wrong answer is still wrong at temp 0. Ground in the source and permit uncertainty instead. (On current models temperature isn't even a tunable parameter.)
Trap: Treating quote extraction as optional formatting. → For long docs (>20k tokens), extract-quotes-first is the mechanism that anchors the answer to real text — it's the technique, not decoration.
Trap: Believing these techniques make output trustworthy enough to skip review. → They reduce but never eliminate hallucination; validate critical facts before acting on them.

PRACTICE

Q1 · A summarizer keeps inventing statistics not in the source. What is the first, cheapest fix?

Add an explicit out and a grounding restriction: tell Claude to use only the provided document and to say "I don't have enough information" when the source doesn't cover something. Granting permission to admit uncertainty is the single most effective, lowest-cost technique.

Q2 · Why ask Claude to extract quotes before analyzing a 50-page contract?

Quote-first grounds the analysis in the actual text rather than the model's memory, which is where hallucinations originate. Referencing each claim back to a numbered quote also makes the output auditable and lets Claude retract claims it can't support.

Q3 · You run the same factual prompt 5 times and get 3 different answers. What does that tell you, and what is the technique called?

The inconsistency across runs is a strong signal of hallucination — the model is guessing, not retrieving. This is best-of-N verification: comparing multiple outputs of the same prompt to surface unreliable claims.