few-shot prompting

In depth

Zero, one, few

A “shot” is one example supplied inside the request, and the word comes in degrees. Zero-shot: the instruction alone, with no examples. One-shot: a single example. Few-shot: a handful, in practice two to five. All three terms come from machine learning, where they described a system’s ability to generalise from very little training data. Applied to a language model that has already been trained, they no longer describe learning at all but the contents of a context, and it is from that shift that the confusion with fine-tuning grows: in one case the weights move once and for all, in the other nothing moves and everything is replayed on every call.

Why the example works

What makes an example more effective than a description is that it carries several things at once without any of them needing to be named: the format, the length, the tone, the level of detail, the handling of edge cases. A description has to enumerate them, and it always leaves one out. The most profitable move is to add a counter-example, together with one sentence saying what is wrong with it: two correct examples give a direction, one wrong example shows where the boundary runs. It is then worth asking what rule was inferred from the examples supplied: the answer is the real brief, the one nobody could have written, and it often contains a clause you never intended.

The cost and its flaws

The cost is real and is paid on every call: examples occupy the context window and are billed in tokens, which prompt caching softens when the block stays stable at the head of the request. Three failure modes come up regularly. Examples that are too uniform produce a pastiche, the model imitating their surface rather than their logic. Order matters, the last example weighing more than the first, which makes a comparison unreliable if you move the examples around between two attempts. And when the task demands stable behaviour across a large volume of calls and the block of examples becomes unwieldy, fine-tuning becomes a serious question: that is the point at which you would rather pay once than on every request.