What is a context window?
The ceiling on how much text a model can read at once, instructions, documents and its own reply included.
Context window — A context window is the maximum amount of text — measured in tokens — a language model can consider at once, including the instructions, the documents it is shown and its own reply. It bounds how much of a record or thread a model can reason over.
Why it matters
A model does not have unlimited attention. Everything it reads for a given request, the system instructions, any documents or record history it is shown, the conversation so far, and the reply it is about to write, all count against one limit measured in tokens, roughly pieces of words. Go past that limit and something has to give: either older content gets dropped, or the request fails outright.
A large context window sounds like it solves every problem, feed the model everything and let it figure out what matters, but that is not how it plays out in practice. Beyond a point, more context makes answers worse, not better, because the model has to search a longer haystack for the relevant needle and can lose track of it. It also costs more, since providers charge by the token for both what goes in and what comes out. The better approach for a record with a long history is usually to retrieve just the relevant passages, the same idea behind retrieval-augmented generation, rather than pasting the entire timeline in every time.
How a context window fills up
- 1
System instructions are counted first
The rules and role given to the model before anything else.
- 2
Retrieved or shown documents are added
Whatever passages or record history the model is given for this request.
- 3
The conversation so far counts too
Every prior turn in a multi-step exchange.
- 4
The model's own reply uses the remaining space
A near-full window leaves little room to answer in.
- 5
Content is trimmed or the request fails
Once the token limit is reached, something has to be cut.
The mistake to watch for
Questions
- How is a context window different from memory?
- A context window is the limit on one request: everything the model reads and writes in that single call. Memory usually refers to information deliberately carried across separate requests, stored and reintroduced rather than kept in one continuous window.
- What happens when a request exceeds the context window?
- Depending on the system, older content is dropped to make room, or the request is rejected outright, which is why long records are usually summarised or retrieved selectively rather than sent in full.
- Does a bigger context window always give better answers?
- No. Very long context can dilute a model's focus on the relevant part, so retrieving just the passages that matter often outperforms sending everything.