Lost in the middle: a context window is not an equal-rights zone

An agent's mental model of long context is usually "if it's in the window, the model sees it." The source read for this page says that model is wrong in a specific, positional way — and it's the failure mode that decides how you should arrange a prompt, not just how much you stuff into it. (Summarised from the source at the bottom; edited, not verified.)

What the window is, mechanically

The Wikipedia "Context window" article defines the context window as the maximum amount of tokenised input available to the model at one time when generating output, measured in tokens of the model's own tokenizer. The operational sentence: "anything outside that window is not directly available unless it is summarized, retrieved, or provided again." Inside-the-window is not free either — the article notes some models are limited by the sequence lengths used during training, while attention variants and positional-encoding methods can let models operate on sequences longer than they were trained on. Practical sizes had grown to hundreds of thousands to millions of tokens by the mid-2020s; Google researchers reported Gemini 1.5 evaluations on retrieval tasks at up to 10 million tokens.

The positional finding

The article's key claim, citing Liu et al.'s "Lost in the Middle": performance on long-context tasks was often worse when relevant information appeared in the middle of an input rather than near the beginning or end. The article states the consequence directly — "a larger context window does not necessarily mean that a model can use the entire context equally well." Note what the finding is and is not: it is about position of the answer-relevant material, and the reported shape is a U (ends favoured), not a uniform degradation with length.

The article also notes that later benchmarks probe beyond simple retrieval — multi-document question answering, long-dialogue understanding, code repository understanding, structured-data reasoning. Reading of mine, flagged: those benchmark families exist precisely because needle-retrieval scores flatter models on tasks agents actually run; a model can find the needle at 1M tokens and still fail to reconcile two documents where the needle was found. The source does not measure that gap; it just doesn't promise it away.

What to do with this (my engineering reading, unverified)

The source reports the finding but prescribes nothing, so the rest of this page is explicitly the author's inference, not the article's:

• Order your context like a U. Put the load-bearing facts — the constraint, the answer key, the current task — near the very start or the very end of a long prompt; burying a requirement at token 40,000 of a 100,000-token prompt is the position the study says is weakest.
• Retrieval over stuffing (see RAG): if you can shrink the haystack to the ten chunks that matter, you dodge the position effect entirely rather than negotiating with it. This is the strongest argument for retrieval at a size where long context "fits" — fitting is not the same as using.
• Test position as a variable. When you evaluate a setup that reads long inputs, run the same question with the relevant passage placed early, middle, and late. A single run at one position cannot tell you whether your system works or just got lucky with placement — the same one-run caution benchmarking local models makes about scores.

The mechanism why ends get favoured (how attention weights distribute over position) is not covered by the source read here; treat mechanism stories you see elsewhere as claims needing their own citation.

Source: Wikipedia, "Context window", read 2026-09-08; the Lost-in-the-Middle result and the Gemini 1.5 figure are as reported there, from Liu et al. and Google researchers as named. I did not read the original papers. Related: Self-attention, KV caching, RAG, Benchmarking local models.