You paste a 40-page contract into an AI assistant and ask it to pull the liability clauses. The AI responds confidently — but something feels off. It cited the clause on page 22 but missed the carve-out on page 8. This isn't a bug in the traditional sense. It's what researchers call the "lost in the middle" problem — and understanding it changes how you use AI on anything that matters.
What a context window is. Every AI model has a context window — the total amount of text it can actively hold in mind during a single conversation. Think of it like a whiteboard with a hard size limit. The numbers have grown dramatically over the past year: GPT-4o handles 128,000 tokens (about 95,000 words — roughly a full novel). Claude Sonnet 4.6 and Opus 4.6 both support 1,000,000 tokens — about 750,000 words — a milestone that became generally available in March 2026. GPT-4.1 and Gemini 2.5 Pro are also at 1M. For most everyday tasks, you'll never get close to the limit. Where it matters: long contracts, full policy manuals, extended research sessions.
Bigger doesn't mean it read all of it. Researchers at Stanford and UC Berkeley found a U-shaped pattern in how language models use their context window. Models do best with information near the very beginning or the very end. When the key fact sits in the middle, accuracy can drop by more than 20%. A model with a 1M token window isn't giving equal weight to every token in that window. It's weighted toward what came first and last. The middle is where things go missing — which is exactly where page 8 of a 40-page document lives.
What this means in practice. For high-stakes document analysis, structure your prompts deliberately. Put the most critical context at the beginning, not buried deep in a paste. Break long documents into sections and analyze each separately before asking for a synthesis. If you're looking for something specific in a long document, be explicit — don't assume the model scanned everything with equal attention.
The honest caveat. "I gave it everything" doesn't mean it processed everything carefully. For contracts, compliance documents, or financial summaries — anything with real consequences — AI is a first-pass scanner, not a final reader. Verify its output against the source, particularly for information that would have been in the middle of a long input.
The trend. Context windows will keep expanding. The harder problem is improving attention distribution across those windows — models that reliably surface information from anywhere in a long document, not just the edges. That's where the meaningful progress is still happening, and it's a better thing to watch than raw token counts.