orchestration

In depth

The questions to settle

As soon as a system puts more than one piece to work, someone has to answer questions that have nothing to do with the model: in what order to call, what to do when a call fails or drags, how many times to retry, how to avoid doing the same thing twice, and where to resume if everything stops in the middle. These are the questions of any distributed system, and they arise identically whether the coordinated piece is a model or a database. This is why orchestration is a matter of ordinary engineering, and not of anything specific to AI.

What the model changes

What changes with a model is that the calls are slow, expensive and non-deterministic. A badly designed retry does not only cost time: it pays for the whole context again, and it can produce a different answer from the first attempt. Orchestration must therefore know what has already been done, what can be replayed without harm, and what must on no account be replayed, such as a payment or a message sent. This distinction is worked out before the incident, not during it.

The common confusion

The frequent confusion consists in believing that adding a layer of orchestration makes a system agentic, or the reverse, that an agentic system does without orchestration. Both are false. A sequence written in advance and perfectly orchestrated remains a workflow, since nobody chooses the next step at execution time. And an agent needs orchestration all the more because it chains unpredictable turns: it is orchestration that sets the limits on time, on spending and on the number of attempts.