ParametersorTraining data
Parameters are what is left of a model once training is over: numerical coefficients. Training data is what served to set them, and then disappeared from the model.
Point by point
| Criterion | Floor 1Floor 1 · The Modela solid block on its own: the prediction machineParameters | Floor 1Floor 1 · The Modela solid block on its own: the prediction machineTraining data |
|---|---|---|
| Present in the model shipped | Yes, they are the model | No, they are not in it |
| What gets counted | Billions of coefficients | Trillions of tokens read |
| Published | Sometimes: that is open weight | Almost never |
| What they determine | Memory use and cost to run | What the model knows and ignores |
| Can an item be removed | Not applicable | No: it would mean retraining |
| Subject of legal dispute | Rarely | That is where it all plays out |
On the ground four situations
A provider announces a seventy-billion model that runs on two graphics cards.
Floor 1 · The Modela solid block on its own: the prediction machineParametersThat figure describes the size of the coefficient file, hence the memory required. It says nothing about what the model read, nor about how good it is.
A model knows nothing of an event that happened after a certain date.
Floor 1 · The Modela solid block on its own: the prediction machineTraining dataThe cut-off date is a property of the corpus that was read, not of the coefficients. No later touch-up will fill that hole.
Someone asks for their data to be deleted from a model that has already been trained.
Floor 1 · The Modela solid block on its own: the prediction machineTraining dataThe useful counter-example: the data is nowhere inside the model, it contributed to setting coefficients. There is nothing to delete, and that is precisely the problem.
A model reproduces word for word a protected passage from its corpus.
BothThe passage is not stored, and yet out it comes: coefficients can have memorised a heavily repeated wording. This is the point where the distinction becomes legally delicate.
The test that settles it
Ask what is inside the file you download. If the answer is “numbers”, those are the parameters. The data stayed with whoever ran the training.
The trap
Believing that a model “contains” its data. That image leads to two symmetrical mistakes: hoping to extract a precise piece of information from it, and believing a document can be taken out of it.
Check click your answer
Level 1 · Recognise
What do you download when a model is released?
Level 2 · Distinguish
The number of parameters or the volume of data read: which better predicts the quality of a model?
Level 2 · Distinguish
Why does telling parameters apart from training data matter legally?