May 18, 20266 min

Agents fail on authority, not intelligence

The interesting failure is never a wrong answer. It is a correct answer applied somewhere nobody agreed to.

Every agent demo that falls over in production falls over the same way. The model reasons well enough, picks a tool, and writes to a place the person watching never approved. Nothing about that is a reasoning problem. It is a permissions problem wearing a reasoning costume.

The fix is boring and it works. Give the model a way to propose, not to act. Put a deterministic gate between the proposal and the world, hand it a grant that burns the moment it is used, and refuse to mark anything done until you have read the world back and confirmed it changed.

Once the runtime holds authority, the model gets to be creative again. It can guess, backtrack, try a strange route, and the worst case is a denied proposal in a log instead of a deleted table.

The second thing that goes wrong is drift. A long run slowly stops being about the thing it started as. Measuring similarity between the current working context and the original intent gives you a number, and a number gives you a place to stop.

Most of the work in this area is not model work. It is the same work as any system that touches money or records: small permissions, written trails, and a way to prove afterward what happened.

agents, runtime

Leave a note