CONCEPT Attention isn't uniform across a long window
"Lost in the middle" is a widely-observed phenomenon for long-context language models: a model attends most reliably to content near the start and end of its context, and least reliably to content buried in the middle. A fact placed halfway through a 200K-token dump can be present, correct, and still under-used — the window held it, but the model didn't weight it. A 1M-token window does not make this go away; it just gives you more room to bury things.
CONCEPT The mitigations are placement + explicit retrieve-then-answer
Anthropic's long-context best practices are, in effect, "lost in the middle" countermeasures — and they line up exactly with 4.1:
- Placement: put long documents near the top and the query at the end, so the load-bearing material sits at the high-attention edges of the window (query-at-the-end is worth up to 30% on complex multi-document inputs).
- Explicit retrieve-then-answer: ask Claude to quote the relevant passages first, then answer from those quotes. This re-surfaces buried evidence into the recent, high-attention tail before the model reasons — it "cuts through the noise."
- Structure: wrap each doc in
<document>tags with a<source>, so boundaries and metadata are unambiguous.
<quotes> block at the end, then reasons over that focused, well-attended set.DIAGRAM Attention across a long window
→ put long docs here"] W --> MID["🔴 Middle: attention sags
→ facts get 'lost'"] W --> BOT["🟢 Bottom: well attended
→ put the query here"] MID -.->|"quote relevant
passages first"| BOT BOT --> ANS["Answer grounded in
re-surfaced quotes"] classDef ok fill:#d4edda,stroke:#28a745,color:#155724; classDef gate fill:#ffe8cc,stroke:#ff9f40,color:#9a4a0a; classDef bad fill:#f8d7da,stroke:#dc3545,color:#721c24; class TOP ok; class BOT ok; class MID bad; class ANS ok;
CODE Retrieve-then-answer in one prompt
Force the model to extract quotes into a tagged block before answering — the buried middle gets copied to the recent tail, where attention is strong.
# Long docs on top (high-attention edge)… corpus = wrap_documents(records, history) # <documents>…</documents> # …retrieve-then-answer instruction at the bottom (the other edge). instruction = """ First, find quotes from the records and appointment history that are relevant to the reported symptoms. Place them in <quotes> tags. Then, based ONLY on those quotes, list what would help diagnose the symptoms in <info> tags. If a claim has no supporting quote, say so. """ resp = client.messages.create( model="claude-opus-4-8", max_tokens=4096, messages=[{"role": "user", "content": corpus + "\n" + instruction}], ) # The <quotes> block is the model's own retrieval pass over the middle.
TRADE-OFFS Stuff-and-hope vs. surface-then-reason
✅ Placement + quote-first
- Load-bearing material sits at attention edges
- Buried facts get re-surfaced to the tail
- Answer is grounded in explicit extracted quotes
⚠️ Stuff the whole window, query on top
- Middle facts present but under-weighted
- No re-surfacing step → silent omissions
- Bigger window ≠ better recall of the middle
EXAM TRAPS
PRACTICE
Q1 · What is "lost in the middle"?
Long-context models attend most to content near the start and end of the window and least to content in the middle. A fact buried mid-prompt may be present but under-used — so answers silently miss it even though it was "in context."
Q2 · Two documented mitigations?
(1) Placement — long docs on top, query at the end (the query-at-the-end lift is up to 30%). (2) Explicit retrieve-then-answer — ask Claude to quote the relevant passages first, which re-surfaces buried evidence into the high-attention tail before it reasons.
Q3 · Does a larger context window solve this?
No. A larger window improves how much you can include (recall), but attention is still uneven — the middle stays weakly attended. Good placement and retrieval matter more, not less, as the window grows.