Agentic AI in Production: Guardrails, Not Prompts

A demo agent and a production agent differ in one important way. The demo agent is allowed to be wrong, because the only consequence is a bad screenshot. The production agent acts on real systems, and being wrong means a real order, a real email, a real database write.

Most of the engineering effort that separates the two is not in the model. It is in the controls around it.

Why prompt-level instructions are not a control

"Never delete a record without asking" is a useful sentence in a system prompt. It is not a guardrail, for three reasons.

First, it is probabilistic. Instructions in a prompt influence behaviour; they do not constrain it. A sufficiently unusual input can move the model past an instruction it has followed a thousand times.

Second, it is invisible to the rest of the system. An instruction lives inside the model call. Nothing in your infrastructure can enforce it, audit it, or prove it held.

Third, it fails open. When the instruction is not followed, nothing stops the action. The system proceeds and the consequence happens.

A guardrail has the opposite properties: deterministic, enforced outside the model, and it fails closed.

The four guardrails that carry the weight

1. Permission scope. Give the agent credentials that can only do what the agent should do. If it must read from a customer table and write to a queue, it gets read access to one and write access to the other, and nothing else. This is the highest-value guardrail because it makes whole categories of failure impossible rather than unlikely.

2. Action allow-lists with approval gates. Enumerate the actions the agent may take, then classify them: autonomous, approval required, or forbidden. Approval gates should be enforced by the system that performs the action, not requested in a prompt. The gate is a code path, not a sentence.

3. Budget and loop limits. Every agent run gets a maximum number of steps, tool calls, and tokens. Without these, a confused agent that loops is not merely wrong; it is expensive, and it may be expensive at 3 a.m. with nobody watching.

4. A complete audit trail. For every run: inputs, the model and version, each tool call with its arguments and result, and the final action. This is what lets you answer questions about what happened, which is the difference between an incident you can investigate and one you can only speculate about.

Design for the failure you cannot prevent

Assume that eventually something will get through. Design so that when it does, the blast radius is small and the recovery is fast.

• Idempotency. Every action carries a key so a retry does not duplicate the effect. Agents retry. This is not optional.
• Dry-run mode. Run the full trajectory and log the actions it would have taken, without taking them. This is how you evaluate an agent before trusting it.
• Reversibility. Prefer actions that can be undone. Where an action is irreversible, that is exactly the action that needs an approval gate.
• A kill switch. A single switch that stops the agent taking new actions and leaves state inspectable. Rehearse using it.

Which actions should never be autonomous

A short list worth treating as a default: anything that moves money, anything that communicates with a customer without review, anything that deletes or overwrites data, anything that changes permissions, and anything that provisions infrastructure. These are not judgement calls. The cost of being wrong is high and the benefit of removing the human is small.

Evaluate trajectories, not answers

An agent can reach the right outcome through a terrible path: three failed attempts, an unnecessary escalation, and a lucky retry. Answer-level evaluation scores that as a success.

What you want to evaluate is the trajectory: which tools were called, in what order, with what arguments, and how many steps it took. Score against an expected path, and treat steps and cost as first-class quality signals rather than operational trivia.

Start with read-only

The practical way to ship an agent is to start where it cannot cause harm. Give it read access, let it produce recommendations, and review them. You will learn more about the failure modes in two weeks of read-only operation than in a month of design. Then grant write permissions on one narrow action, and expand only when the evidence supports it.