prompt engineering

In depth

The principles that work

What produces real gains comes down to a few principles, and none of them is esoteric. Describe the task, the role and the audience rather than leaving them to be guessed. Provide two or three examples of input and expected output, which is worth more than ten lines of description. Impose an explicit output format, and constrain it with a schema when the output is consumed by code. Say what to do in case of doubt, failing which the model will invent rather than abstain. Finally, break a complex request into several calls, because a prompt that does four things does at least two of them badly.

Context before wording

The shift of recent years is clear: the wording matters less than the context supplied. A mediocre prompt with the right documents beats a finely chiselled prompt without them. This is why the useful work has moved towards assembling what the model has in front of it, and towards defining the tools it can call. Recent models also follow instructions far better, which has made obsolete contortions that were once necessary.

Measurement

The discipline that separates serious practice from tinkering is measurement. A change of prompt that improves one case can degrade ten others, and no one will see it without a test set. Prompts therefore have to be versioned like code, a set of representative cases including past failures has to be kept, and that set replayed at every change, including when it is the model that changes. Without this, you optimise blind a system whose performance you do not know.