GlossaryFloor 1 · The Modela solid block on its own: the prediction machineFloor 1 · The Model
inference
No. 051 · v2026-08FR: inférenceInference is the moment when an already trained model produces an answer: it reads what it is given and writes the continuation, without learning anything along the way. Like a musician playing a piece, as opposed to the years spent working on the instrument.
What it is not
Inference is not training, and the two bear no resemblance to each other: training modifies the model and is counted in weeks of computation, inference does not modify it and is counted in seconds. Nor is it a search: nothing is retrieved from a database, everything is reconstructed. And the word denotes no logical deduction, despite what its ordinary sense suggests: it comes from the vocabulary of statistics, where to infer means to estimate from a model.
In depth
Step by small step
Inference proceeds by small steps. The model does not compose an answer in one block: it predicts the most probable fragment of text, adds it to what it already has, then starts again with this continuation as its new input. It is this mechanism that explains the word-by-word display, but also a deeper property: every fragment produced influences all the following ones, so much so that a bad start propagates. It is what makes effective the instructions that ask the model to reason before concluding, since they lengthen the path before the answer.
The economics
This is where the economics of the system plays out. Training is a single and colossal investment; inference is a recurring cost, proportional to usage, which ends up far exceeding it for a heavily used service. Two quantities count: the length of the input, which has to be read again on every request, and that of the output, produced fragment by fragment and therefore more expensive. Many real optimisations consist simply in giving less to read and asking for less to be written.
What you actually control
The inference phase is also the only one an organisation really controls. It does not choose what the model has learned, but it chooses what it gives it to read, with which instructions, with which tools, and what it does with the output. In other words, the whole harness operates at inference. That is why the question “should we retrain?” comes up far less often than people think: most problems are settled by changing what surrounds the call, not the model called.
Relations where the neighbours live
Check 3 questions · click your answer
Level 1 · Recognise
What happens inside the model during an inference?
Level 2 · Distinguish
For a heavily used service, which cost ends up dominating the other?
Level 2 · Distinguish
Why does asking the model to “set out its reasoning” often change the quality of the answer?
Who works with this 2 roles
The roles for which this term is part of the ordinary work.
Lexigraph, "Inference", v2026-08, https://www.lexigraph.org/en/inference/, CC BY 4.0.