context window

In depth

What it covers

The window is measured in tokens and covers everything that goes into a call: the standing instructions, the history carried over, the attached documents, the tool results, and not forgetting the answer to be produced, which takes up room as well. It is a characteristic of the model, settled during its training: you do not enlarge it from the product, you change model. It weighs on cost and on response time, since every call pays again for the whole of what is sent, and not for the new message alone. A long conversation therefore costs more and more with each turn, even though the user has the feeling of carrying on the same discussion.

Large is not well used

A large window does not guarantee good use of its content. The quality of the attention paid to an element depends on its place: beginnings and ends are put to better use than middles, and a decisive detail buried in the centre of a very long document can be ignored without anything signalling it. Filling the window because it is large is therefore a poor strategy: selecting the right extracts remains more accurate and cheaper than sending everything. When the limit is exceeded, the behaviour is not a clean error either but a truncation, most often a silent one.

The useful distinction

Telling the window apart from the context changes the way a problem is solved. If an assistant forgets an instruction given ten messages earlier, the cause is almost never the size of the window: it is the strategy of the harness, which summarised, truncated or simply failed to send that instruction back. Moving to a model with a wider window is expensive and corrects nothing if the selection stays poor, because you do not lose what you never sent. The useful question is not how much the model can see, but what you decide to show it, and by what rule.