You paste a 40-page contract into an AI assistant and ask it to pull the liability clauses. The AI responds confidently — but something feels off. It cited the clause on page 22 but missed the carve-out on page 8. This isn't a bug in the traditional sense. It's what researchers call the "lost in the middle" problem — and understanding it changes how you use AI on anything that matters.
What a context window is. Every AI model has a context window — the total amount of text it can actively hold in mind during a single conversation. Think of it like a whiteboard with a hard size limit. The numbers have grown dramatically over the past year: GPT-4o handles 128,000 tokens (about 95,000 words — roughly a full novel). Claude Sonnet 4.6 and Opus 4.6 both support 1,000,000 tokens — about 750,000 words — a milestone that became generally available in March 2026. GPT-4.1 and Gemini 2.5 Pro are also at 1M. For most everyday tasks, you'll never get close to the limit. Where it matters: long contracts, full policy manuals, extended research sessions.
Bigger doesn't mean it read all of it. Stanford researchers found that language models follow a U-shaped attention pattern across their context window. Content at the very beginning gets strong attention. Content near the end gets strong attention. Content buried in the middle gets up to 30% less reliable recall. A model with a 1M token window isn't giving equal weight to every token in that window. It's weighted toward what came first and last. The middle is where things go missing — which is exactly where page 8 of a 40-page document lives.
What this means in practice. For high-stakes document analysis, structure your prompts deliberately. Put the most critical context at the beginning, not buried deep in a paste. Break long documents into sections and analyze each separately before asking for a synthesis. If you're looking for something specific in a long document, be explicit — don't assume the model scanned everything with equal attention.
The honest caveat. "I gave it everything" doesn't mean it processed everything carefully. For contracts, compliance documents, or financial summaries — anything with real consequences — AI is a first-pass scanner, not a final reader. Verify its output against the source, particularly for information that would have been in the middle of a long input.
The trend. Context windows will keep expanding. The harder problem is improving attention distribution across those windows — models that reliably surface information from anywhere in a long document, not just the edges. That's where the meaningful progress is still happening, and it's a better thing to watch than raw token counts.