Why the Rules That Constrain an Agent Should Live in a Deterministic Layer It Never Sees
Ronni Holmvig Strøm · 2026-06-21
For two years the field treated agent safety as a property of the model. We wrote longer system prompts. We fine-tuned for refusal. We red-teamed the weights and shipped the ones that behaved. The implicit theory was that a sufficiently well-trained model would, when handed a tool and a goal, choose
For two years the field treated agent safety as a property of the model. We wrote longer system prompts. We fine-tuned for refusal. We red-teamed the weights and shipped the ones that behaved. The implicit theory was that a sufficiently well-trained model would, when handed a tool and a goal, choose correctly on its own. A quieter line of work through 2025 and into this year has reached a different conclusion, and it has reached it from several directions at once.
You do not govern an agent by asking the model to behave. You govern it with a deterministic layer that sits outside the model, evaluates what the agent is about to do against a set of obligations and prohibitions, and renders a verdict the model cannot argue with. The rules live where the model cannot reach them. That is the design decision that actually matters for anyone shipping agents that invoke tools and cross organizational boundaries, and a cluster of recent papers and releases now points squarely at it.
The Permit/Prohibit Engine Was Never Enough
Start with the gap, because the most useful of the new work begins by naming it precisely. In Deontic Policies for Runtime Governance of Agentic AI Systems, Anupam Joshi, Tim Finin, Karuna Pande Joshi, and Lalana Kagal argue that the policy engines we already trust for access control were built for the wrong question (Joshi et al., 2026). XACML, Rego, and Cedar answer "is this action permitted or prohibited?" That covers a real and important subset of governance. It does not cover the rest of it.
Enterprise governance has a richer structure. It specifies what an agent is obliged to do after it acts, for instance notify the CISO once a particular threshold is crossed. It specifies the conditions under which a standing obligation may be waived. It specifies which rule wins when two policies collide. None of that fits inside permit-or-prohibit, which is why the authors reach for deontic logic, the formal logic of obligation, permission, and prohibition that ethics and law have used for a long time before software borrowed it.
Their system, AgenticRei, expresses these policies in OWL on top of the Rei framework and evaluates them at runtime through a high-performance logic engine that runs, in their words, "entirely outside the LLM." The same pipeline governs both the agent's tool calls and the messages it sends to other agents. The choice of vocabulary is the tell. Obligation, dispensation, precedence: these are the categories of an institution, not of a code review.
Why the Rules Live Where the Model Cannot Reach
The phrase "entirely outside the LLM" carries the weight of the whole argument, and the load-bearing word is outside.
Anything you place inside the model inherits the model's surface. It is probabilistic, it is sensitive to phrasing, and it can be talked out of its commitments by a sufficiently clever context. A guardrail expressed as a system prompt is a guardrail the next prompt can erode. A refusal learned in training is a refusal that some distribution of inputs will fail to trigger. We have spent considerable effort discovering, empirically, that the model's own judgment is exactly the wrong place to put a constraint you actually need to hold.
A deterministic engine does not have that surface. It does not weigh the agent's justification, it does not find the request reasonable under the circumstances, and it cannot be flattered. It checks the proposed action against the policy and returns a decision. AgentSpec, from Haoyu Wang, Christopher M. Poskitt, and Jun Sun, makes the case in numbers (Wang et al., 2025). Their lightweight rule language, where each rule is a triggering event, a predicate, and an enforcement mechanism, prevented unsafe executions in over 90% of code-agent cases, eliminated every hazardous action in their embodied-agent tasks, and enforced full compliance from simulated autonomous vehicles. The overhead was measured in milliseconds. A rule that lives outside the model costs almost nothing to check and does not negotiate.
There is a subtlety worth keeping. AgentSpec also used an LLM to generate candidate rules, and there the numbers loosened, with recall around 71% for embodied agents. Writing the policy is a model-assisted, fallible act. Enforcing it is not. The discipline is to keep those two jobs apart.