Fine-tuningorRAG
Fine-tuning modifies the model itself, by continuing its training on your examples: it changes the way it answers. RAG does not touch the model: it fetches the right documents and places them in the context at the moment of the question.
Point by point
| Criterion | Floor 1Floor 1 · The Modela solid block on its own: the prediction machineFine-tuning | Floor 2Floor 2 · The Harnessthe block and its bolted-on plates: what gets added to itRAG |
|---|---|---|
| What changes | The weights of the model | What it is given to read |
| Updating a fact | Retrain | Edit a document |
| Where the cost goes | Upfront, in training | On every request |
| Traceable source | None: the knowledge is diluted | The source can be cited |
| Good for | A style, a format, a trade vocabulary | Facts, above all changing ones |
| Bad for | Facts that change | Changing the way it answers |
On the ground four situations
Your answers must adopt the house tone and its turns of phrase, on subjects that barely move.
Floor 1 · The Modela solid block on its own: the prediction machineFine-tuningThis is a way of answering, not a subject matter. It is learned, it is not looked up.
An internal assistant has to answer on the leave policy, revised every year and binding.
Floor 2 · The Harnessthe block and its bolted-on plates: what gets added to itRAGThe information changes and must be citable: it belongs to the documents, not to the weights.
You want the output to be, every single time, JSON that conforms to your schema.
Structured outputNeither one as a first move: constraining a format is a harness function. Fine-tuning comes only afterwards, if the format stays unstable.
Your trade uses a term nobody else uses, and the model gets it wrong.
BothRAG hands it the glossary to read at the right moment; fine-tuning makes it a reflex if the mistake shows up everywhere. Doing both is the ordinary answer here.
The test that settles it
Ask yourself whether the answer has to be able to cite its source. If so, it is RAG. If what you want to change is the way of answering and not the matter answered on, it is fine-tuning.
The trap
You do not fix a factual error with fine-tuning: you only teach the model to be wrong with more confidence. The two approaches often combine, they do not replace each other.
Check click your answer
Level 1 · Recognise
What exactly does fine-tuning modify?
Level 2 · Distinguish
Your prices change every month and the system must always quote the right one. What do you do?
Level 2 · Distinguish
After fine-tuning on two thousand product sheets, the model still recommends a product withdrawn from the catalogue. Why is that hard to fix?