GlossaryFloor 1 · The Modela solid block on its own: the prediction machineFloor 1 · The Model
transformer
No. 057 · v2026-08FR: transformeurThe transformer is the network architecture that equips almost every current model: at each step, each fragment of text looks at all the others and keeps those that shed light on it. Like a reader who, at every word, goes back to the ones that fix its meaning.
What it is not
The transformer is not a model, it is a construction plan: two models built on it can differ in everything, in size, in data, in behaviour. Nor is it a learning method: it describes how information circulates, not how the parameters are adjusted. And the word has nothing to do with electrical transformers, or with the idea of transforming a text: it comes from the title of the founding paper.
In depth
Attention
The innovation has a name, attention. Earlier architectures read text from left to right while keeping a memory of what came before, which made them slow to train and forgetful over long distances. The transformer replaces this sequential reading with an operation where each fragment directly computes its relevance to all the others. A pronoun can thus be linked to its antecedent five hundred words earlier in a single step, and not in five hundred.
Two consequences
Two consequences follow, and they explain the history of the last ten years. The first is that the computation becomes massively parallelisable: all the fragments are processed at the same time, which suits graphics accelerators exactly and made training on gigantic corpora possible. The second is a cost: comparing each fragment with all the others grows with the square of the length, which explains why context windows took years to lengthen and why an input twice as long costs more than twice as much.
Beyond text
The architecture is today so dominant that it has spilled beyond text. The same principles handle an image cut into squares, sound cut into slices, video, code or molecular structures: it is enough that the input can be cut into a sequence of elements. That is what makes multimodal models possible, where several kinds of input live together in the same mechanism. In other words, the architecture has nothing specifically linguistic about it, which is no doubt the reason for its success.
Relations where the neighbours live
Check 3 questions · click your answer
Level 1 · Recognise
What does attention bring compared with a sequential reading?
Level 2 · Distinguish
Why does an input twice as long cost more than twice as much?
Level 2 · Distinguish
Why does the same architecture serve text, image and sound?
Who works with this 1 role
The roles for which this term is part of the ordinary work.
Lexigraph, "Transformer", v2026-08, https://www.lexigraph.org/en/transformer/, CC BY 4.0.