D5 · Context & Reliability › Cluster 4 · Long Context & Retrieval

4.2 · 'Lost in the Middle' & Attention

The middle gets less attention

CONCEPT Attention isn't uniform across a long window

"Lost in the middle" is a widely-observed phenomenon for long-context language models: a model attends most reliably to content near the start and end of its context, and least reliably to content buried in the middle. A fact placed halfway through a 200K-token dump can be present, correct, and still under-used — the window held it, but the model didn't weight it. A 1M-token window does not make this go away; it just gives you more room to bury things.

🔑 "In the context window" ≠ "attended to." Retrieval recall (was the fact present?) and attention (did the model use it?) are two different failures. A huge window fixes the first, not the second.

CONCEPT The mitigations are placement + explicit retrieve-then-answer

Anthropic's long-context best practices are, in effect, "lost in the middle" countermeasures — and they line up exactly with 4.1:

Think of quote-first as manually pulling the middle out to the edge: the model copies the relevant lines into a fresh <quotes> block at the end, then reasons over that focused, well-attended set.
If a specific fact must land, don't rely on it surviving in the middle of a huge dump. Either retrieve it into a small prompt (4.3) or place it deliberately near the query (4.5).

DIAGRAM Attention across a long window

flowchart TD W["Long context window"] --> TOP["🟢 Top: well attended
→ put long docs here"] W --> MID["🔴 Middle: attention sags
→ facts get 'lost'"] W --> BOT["🟢 Bottom: well attended
→ put the query here"] MID -.->|"quote relevant
passages first"| BOT BOT --> ANS["Answer grounded in
re-surfaced quotes"] classDef ok fill:#d4edda,stroke:#28a745,color:#155724; classDef gate fill:#ffe8cc,stroke:#ff9f40,color:#9a4a0a; classDef bad fill:#f8d7da,stroke:#dc3545,color:#721c24; class TOP ok; class BOT ok; class MID bad; class ANS ok;

CODE Retrieve-then-answer in one prompt

Force the model to extract quotes into a tagged block before answering — the buried middle gets copied to the recent tail, where attention is strong.

# Long docs on top (high-attention edge)…
corpus = wrap_documents(records, history)   # <documents>…</documents>

# …retrieve-then-answer instruction at the bottom (the other edge).
instruction = """
First, find quotes from the records and appointment history that are
relevant to the reported symptoms. Place them in <quotes> tags.
Then, based ONLY on those quotes, list what would help diagnose the
symptoms in <info> tags. If a claim has no supporting quote, say so.
"""

resp = client.messages.create(
    model="claude-opus-4-8", max_tokens=4096,
    messages=[{"role": "user",
               "content": corpus + "\n" + instruction}],
)
# The <quotes> block is the model's own retrieval pass over the middle.

TRADE-OFFS Stuff-and-hope vs. surface-then-reason

✅ Placement + quote-first

  • Load-bearing material sits at attention edges
  • Buried facts get re-surfaced to the tail
  • Answer is grounded in explicit extracted quotes

⚠️ Stuff the whole window, query on top

  • Middle facts present but under-weighted
  • No re-surfacing step → silent omissions
  • Bigger window ≠ better recall of the middle

EXAM TRAPS

Trap: "A 1M-token window means I can dump everything and it'll all be used equally." → No. Attention still sags in the middle; a bigger window just gives more room to lose things.
Trap: Conflating "in context" with "attended to." → A fact can be present and correct yet under-used. Recall and attention are different failure modes.
Trap: Fixing lost-in-the-middle by re-ordering nothing but adding "pay attention!" → The documented fixes are structural: top/bottom placement and quote-relevant-passages-first, not louder instructions.
Trap: Putting the single most critical fact in the exact middle of a huge dump. → Place it near the query, or retrieve it into a small prompt instead (4.3).

PRACTICE

Q1 · What is "lost in the middle"?

Long-context models attend most to content near the start and end of the window and least to content in the middle. A fact buried mid-prompt may be present but under-used — so answers silently miss it even though it was "in context."

Q2 · Two documented mitigations?

(1) Placement — long docs on top, query at the end (the query-at-the-end lift is up to 30%). (2) Explicit retrieve-then-answer — ask Claude to quote the relevant passages first, which re-surfaces buried evidence into the high-attention tail before it reasons.

Q3 · Does a larger context window solve this?

No. A larger window improves how much you can include (recall), but attention is still uneven — the middle stays weakly attended. Good placement and retrieval matter more, not less, as the window grows.