hallucination

In depth

The mechanism

The model assigns a probability to every possible continuation and draws one: nothing in this mechanism tells a true continuation from a merely plausible one. When the information asked for is absent, rare or contradictory in what the model has learned, the most plausible continuation is still a well-formed sentence, and it comes out with the assurance of all the others. The most dangerous inventions are therefore the most credible ones: a reference in the right format, a court decision in the name of a real jurisdiction, an existing author paired with a book they never wrote. A confident tone measures nothing: it is a trait of style, not an indicator of reliability.

Not every error is one

Not every false answer is a hallucination, and the distinction dictates the fix. Information that is accurate but out of date is a matter of the date at which the model’s knowledge stops; a calculation error is a matter of reasoning; an answer founded on a faulty document is a matter of source quality. A hallucination in the strict sense is the assertion of content with no support: it is handled by giving the model something to lean on and by checking that what it puts forward does appear in that support. Filing every failure under the same word leads to treating a source problem with a change of model, which corrects nothing and costs a quarter.

What reduces without removing

No known technique removes the phenomenon: they reduce its frequency and, above all, its reach. Supplying the relevant documents and requiring every assertion to point back to an extract moves the problem to the right side, since an assertion without a source becomes detectable before publication. Asking the model whether it is sure of its answer brings nothing reliable, that answer being produced by the same process as the previous one. Reducing the variability of the sampling makes outputs more stable, not more true: a model can be wrong in a perfectly reproducible way. The decisive guardrail remains organisational: deciding which assertions are binding, and imposing a human check at those precise points rather than everywhere or nowhere.