error recovery

In depth

What the tool returns

Everything depends on what the tool returns. A silent error, a bare code or a truncated technical message, gives the model no purchase: it will conclude what it can and change action at random. A message that says what did not work, why, and what would be expected allows a targeted correction from the next turn onwards. Designing the error messages of tools as text meant to be read by a model is one of the most profitable pieces of work on a harness, and one of the most neglected.

The four kinds of failure

Failures are handled according to their nature, and confusing them is expensive. A passing unavailability calls for a spaced retry. An invalid input calls for a correction of the parameter, not a repetition. A missing permission calls for a stop, because no attempt will make it appear. An empty result is not an error but a piece of information, and an agent that treats it as a failure sets off searching again where there is nothing. Telling these four cases apart in the returned message radically changes the behaviour observed.

Bounding stubbornness

What remains is to bound stubbornness. An agent left free insists, because nothing in its training pushes it to give up: consecutive failures therefore have to be counted per tool, the repetition of the same action detected, and the way out decided explicitly. That way out can be a change of strategy, a request for help from a human, or a documented abandonment. It is also what has to appear in the trace: a system where tasks that succeeded first time are not distinguished from those that succeeded after five recoveries has no means of improving.