What is a context window?
The cap on how much text a model can read at once, counting your instructions, your documents and its own reply.
Want to see it first? Book a demo
Context window — A context window is the maximum amount of text, measured in tokens, that a language model can consider at once, including the instructions, the documents it is shown and its own reply. It bounds how much of a record or thread a model can reason over.
Why it matters
A model can't pay attention to everything. For each request, it reads the system instructions, any documents or record history you show it, and the chat so far. Then it writes a reply. All of that counts against one limit, measured in tokens, which are roughly pieces of words. Go past the limit and something has to give. Either older text gets dropped, or the request fails outright.
A large context window sounds like it solves everything: feed the model it all and let it work out what matters. In practice it doesn't play out that way. Past a point, more context makes answers worse, not better. The model has to search a longer haystack for the one needle, and it can lose track of it. It also costs more, since providers charge by the token for what goes in and what comes out. For a record with a long history, you usually do better to fetch just the passages that matter. That's the idea behind retrieval-augmented generation. Don't paste the whole timeline in every time.
How a context window fills up
- 1
System instructions are counted first
The rules and role given to the model before anything else.
- 2
Retrieved or shown documents are added
Whatever passages or record history the model is given for this request.
- 3
The conversation so far counts too
Every prior turn in a multi-step exchange.
- 4
The model's own reply uses the remaining space
A near-full window leaves little room to answer in.
- 5
Content is trimmed or the request fails
Once the token limit is reached, something has to be cut.
The mistake to watch for
Questions
- How is a context window different from memory?
- A context window is the limit on one request: all the model reads and writes in that single call. Memory usually means facts carried from one request to the next on purpose. They're stored and fed back in, not kept in one long window.
- What happens when a request exceeds the context window?
- It depends on the system. Older text is dropped to make room, or the request is turned down outright. That's why long records are usually summed up, or only the key parts are fetched, instead of sending them in full.
- Does a bigger context window always give better answers?
- No. A very long context can pull the model's focus away from the part that matters. Fetching just the key passages often beats sending everything.