Context windoworContext
The context window is a capacity of the model: the largest amount of text it can have in front of it at once, measured in tokens. The context is what the harness actually placed inside it for one request, and that is a choice.
Point by point
| Criterion | Floor 1Floor 1 · The Modela solid block on its own: the prediction machineContext window | Floor 2Floor 2 · The Harnessthe block and its bolted-on plates: what gets added to itContext |
|---|---|---|
| What it is | A limit | A content |
| Who sets it | The model’s provider | Whoever builds the system |
| Changes when | You change model | At every request |
| Measured in | Tokens available | Tokens consumed |
| When it is too small | You need another model | You need to choose better what goes in |
| What you pay for | Nothing: it is a capacity | Everything: the context is billed |
On the ground four situations
A model announces that it accepts two hundred thousand tokens.
Floor 1 · The Modela solid block on its own: the prediction machineContext windowThat is an advertised capacity, independent of any use. It costs nothing as long as you do not fill it.
Your system sends, with every request, a standing instruction, five documents and the history of the conversation.
Floor 2 · The Harnessthe block and its bolted-on plates: what gets added to itContextThat is an assembly you decided on, rebuilt at every call, and billed every time.
A long conversation loses an instruction given at the beginning.
BothThe context has outgrown the window: the content overflowed the capacity. The remedy is on the harness side, summarise or reinject, not on the model side.
A system systematically fills the window to the brim “so as not to lose anything”.
Floor 2 · The Harnessthe block and its bolted-on plates: what gets added to itContextThe useful counter-example: available capacity is not a target to reach. A cluttered context costs more and degrades the quality, what matters drowning in the rest.
The test that settles it
Ask whether the figure changes when you edit your code without changing model. What moves is the context, the content you assembled. What stays identical is the context window, the capacity the model gave you.
The trap
Treating a large window as the answer to problems of relevance. More room does not mean better chosen: past a certain level of filling, models are less good at finding a piece of information buried in the middle, and the bill follows the length.
Check click your answer
Level 1 · Recognise
When you call a model, what is billed at every request?
Level 2 · Distinguish
Your system reaches the limit. What is the first question to ask?
Level 2 · Distinguish
Does a larger window improve the quality of the answers?