HomeResearch LibraryLost in the Middle: How Language Models Use Long Contex…
empirical studyChapter 2TACL · 2023

Lost in the Middle: How Language Models Use Long Contexts

Nelson F. Liu (Stanford), Kevin Lin (Stanford)

Abstract

We analyze how language models use long contexts and find that performance degrades when relevant information is in the middle of the context. Models are better at using information at the beginning or end.

Eigenvector Insight — Zone III / PASF-PADE AnalysisNot part of the original paper
Eigenvector Research — Marco van Hurne
How this paper contributes to solving the Zone III problem (PASF-PADE)

The lost-in-the-middle finding is one of the most practically important results for Zone III memory design. If agents cannot reliably use information in the middle of their context, then memory architecture must be designed to place critical information at the boundaries.

Why AI is not sufficient for Zone III without this

Zone III refers to high-complexity, high-risk, long-running agentic workflows — the class of enterprise AI deployments where a single failure can cascade across hundreds of steps. Standard AI models, trained to predict the next token, are not inherently designed for durable, governed, multi-step execution. This paper addresses one or more of the structural gaps that make Zone III deployments unsafe without explicit architectural intervention.

Key Contributions

  • →Lost-in-the-middle phenomenon
  • →Context position analysis
  • →Implications for RAG design

Topics

long contextattentioncontext utilizationreliability