guardrails

In depth

The three places

Guardrails sit in three places, and confusing those places loses the greater part of the benefit. Upstream, you limit what comes in: which sources are readable, which requests are admissible, which data must never join the context. At the moment of acting, you limit what the system can trigger: authorised tools, rights reduced to the strict minimum, ceilings, human confirmation for anything irreversible. Downstream, you check what comes out before it reaches a person or another piece of software: expected shape, sources present, absence of information that was not meant to circulate.

The principle

The principle that governs them fits in one sentence: a guardrail assumes that the model has failed. The point is not to make it wiser, but to make sure that a false, hijacked or absurd output produces no lasting damage. That is why an effective guardrail lives outside the text: a check written into the harness applies whatever happens, whereas an instruction inserted into the context shares the fate of everything else there and can be contradicted by a document read along the way. The right question is therefore never whether the model will obey, but what happens on the day it does not.

The traps

The first trap is excess: limits that are too tight produce a system that refuses everything, which teams get around by going to work elsewhere, out of any supervision. The second is the decorative guardrail, that confirmation request nobody pays attention to any more after the hundredth time: an approval you cannot refuse is not one. The third is believing that they are written once and for all, whereas they are chosen according to the damage feared, tested like the rest of the system, and revisited every time the perimeter of action widens. The last is confusing them with regulatory compliance, which imposes obligations of result without saying how to restrain a particular tool.