The Model

A machine that predicts what comes next in a text, trained once and for all, with no memory and no access to the world.

The terms on this floor

29 entries

alignmentNo. 060Alignment is the work that makes a model behave as one wishes: following an instruction, refusing certain requests, adopting a tone. Like the induction that follows a hire, which does not change what a person knows but what they do with it.
artificial intelligenceNo. 045Artificial intelligence is the name of an entire field, not of a technique: everything that makes a machine perform tasks once associated with human judgement. Like the word “medicine”, which covers a vaccine as much as a hip replacement.
benchmarkNo. 061A benchmark is a standardised test that scores models on a common set of questions, in order to compare them with one another. Like a national examination: it ranks the candidates, it does not say which one will do the job in your team.
biasNo. 062A bias is a systematic and reproducible deviation in a model’s outputs, inherited from its data or its training. Like a badly calibrated scale: it does not get things wrong at random, it always gets them wrong in the same direction.
context windowNo. 007The context window is the amount of text a model can have in front of it at once: your messages and the documents supplied. Like a worktop: past a certain size, something has to come off before anything else goes on.
deep learningNo. 047Deep learning is the branch of machine learning that stacks up many layers of computation, each one describing the data a notch more abstractly than the one before. Like a chain of reviewers where the first sees strokes, the next shapes, the last a face.
determinismNo. 089A system is deterministic when the same input always gives exactly the same output. A language model is not: like an orchestra playing the same score again, the request is identical and the performance never quite is.
distillationNo. 063Distillation consists in training a small model to imitate the answers of a large one, in order to obtain a far lighter object that keeps the essentials of the behaviour. Like a workshop manual drawn from the experience of a master craftsman: shorter, and sufficient for most cases.
embeddingNo. 053An embedding is the translation of a text into a long list of numbers, arranged so that two texts of similar meaning give neighbouring lists. Like coordinates on a map, where nearby towns talk about nearby things.
fine-tuningNo. 014Fine-tuning means continuing the training of an existing model on your own examples, in order to change the way it answers: like an experienced professional you spend a few weeks training in your in-house practices, until they become reflexes.
frontier modelNo. 087A frontier model is one of the most capable models in existence at a given moment: the category is defined in relation to what the field can do, never by a size or a figure. Like a world record: it is not a performance fixed once and for all, it is the one nobody has beaten yet.
generative AINo. 017Generative AI refers to the systems that produce new content, text, image, sound or code, rather than classifying or measuring what already exists. Like a draughtsman composing a scene never seen before, where a stamp always reproduces the same imprint.
hallucinationNo. 011A hallucination is a false answer stated with the same assurance as a true one: an invented date, a quotation that does not exist. Like someone who, rather than admit a gap in their memory, fills it with whatever sounds right.
inferenceNo. 051Inference is the moment when an already trained model produces an answer: it reads what it is given and writes the continuation, without learning anything along the way. Like a musician playing a piece, as opposed to the years spent working on the instrument.
large language modelNo. 005A large language model is a machine for predicting what comes next in a text: from what precedes, it proposes the next fragment, then starts again. Like someone who had read everything and would finish your sentences, without checking what they claim.
machine learningNo. 046Machine learning consists in no longer writing the rules of a program but giving it examples so that it derives them itself. Like learning to recognise an edible mushroom by seeing a thousand of them, rather than by reading their description.
modelNo. 004A model is the frozen result of long learning on masses of examples: it keeps from them a way of continuing what it is given. Like an engine out of the factory: it does not start on its own and never changes once built.
multimodalNo. 058A multimodal model handles several kinds of input in the same computation: text, images, sound, without first converting them into a single format. Like a person looking at a chart while listening to the commentary on it, rather than reading its written description.
neural networkNo. 048A neural network is a long sequence of multiplications and additions, organised in layers, whose coefficients are adjusted until the outputs match the examples. Like a mixing desk with billions of faders, set automatically by ear.
open sourceNo. 056Open source names a licensing regime that grants four freedoms: to use, to study, to modify and to redistribute, with no discrimination as to use or as to person. Like a recipe published with its list of ingredients, which anyone can cook, change and publish again.
open weightNo. 055An open weight model is a model whose parameters can be downloaded: you can run it on your own machines, adapt it, integrate it. Like an engine delivered to you that you fit wherever you want, without the plans of the factory that built it.
parameter (weight)No. 049A parameter is one of the numerical coefficients adjusted during training: together they make up everything the model has retained. Like an instrument being tuned: the settings remain, the tunes played to tune it are nowhere inside.
pre-trainingNo. 052Pre-training is the first phase, the longest and the most expensive: the model reads immense quantities of text, learning only to predict what comes next. Like years of reading that build a general culture, before anyone teaches you to answer a question politely.
reasoning modelNo. 059A reasoning model first produces, for itself, a long series of intermediate steps, then writes its answer from them. Like someone covering a sheet with rough calculations before writing the line with the result.
temperatureNo. 054Temperature sets the degree of chance at the moment the model picks the next fragment: low, it almost always takes the most probable one; high, it allows itself less expected choices. Like a cook who follows the recipe to the letter, or departs from it a little at every move.
tokenNo. 006A token is the small piece of text that the model handles: not a letter, not quite a word, rather a chunk of a common word. What you write reaches it cut up in this way, like a text read syllable by syllable.
tokenisationNo. 088Tokenisation is the operation that rewrites any incoming text as a sequence of fragments taken from a fixed vocabulary, the only one the model can read. Like a printer’s type case: whatever word is to be set, it has to be set with the sorts that are in the case, and a rare word takes more pieces than a common one.
training dataNo. 050Training data is the set of examples from which a model was formed: what it can say and what it is ignorant of both depend on their content. Like the library where someone learned everything, and whose gaps become their own.
transformerNo. 057The transformer is the network architecture that equips almost every current model: at each step, each fragment of text looks at all the others and keeps those that shed light on it. Like a reader who, at every word, goes back to the ones that fix its meaning.

The other floors

The grid

The Elevator is the infographic that maps these four floors: one and the same request, read again at every level.