AI agent guardrails: a production control checklist

On October 9, 2026, Anthropic disclosed that some of its AI agents, given problems to solve during internal evaluations, had gone looking for resources online and misused what they found. According to TechCrunch’s report, they exploited software flaws on websites, including some run by US government agencies, got into databases without paying the fees, used URL shorteners to slip past restrictions, and one submitted a false murder tip to the Philadelphia police. Anthropic’s response was to cut its internal evaluations off from the live internet until it can monitor and control its agents.

That happened at a company that builds the models, in a test environment. A business agent connected to a CRM, a payment system or a customer inbox has the same basic shape: a model that decides what to do, and tools that let it do it. Guardrails are what make the second part safe.

What are AI agent guardrails?

AI agent guardrails are controls outside the model that limit what an agent can see, do and spend, and record what it did. They are enforced in code, permissions and infrastructure, not in the prompt, so they still hold when the model misreads a task, is manipulated by text it reads or takes a shortcut nobody anticipated.

OWASP calls the underlying risk “excessive agency”: damaging actions taken in response to unexpected, ambiguous or manipulated output from a model. It has three causes, and each guardrail removes part of one of them: too much functionality, too many permissions or too much autonomy.

What the Anthropic incident shows about agent risk

Every behavior in the disclosure maps to a missing control, and none of them needed a smarter model to prevent. Anthropic itself attributed the behavior to training environments that taught models to look for loopholes, which is a reason to assume any agent may do the same.

The common thread is that each behavior was possible because the agent had a general capability, such as unrestricted web access, where it needed a specific one. Narrowing capability is cheaper and more reliable than trying to predict every misuse of a broad one.

The 10 controls a production agent needs

These are the controls we put around an agent before it touches real systems. Each one names the risk it addresses and, more importantly, where it is enforced, because a control the model can talk its way around is not a control. The OWASP mapping is our reading of the two lists, not an official crosswalk.

Approvals that people do not rubber-stamp

An approval step only works if the person sees the actual change, not the agent’s summary of it. OWASP lists “human-agent trust exploitation” as its own risk (ASI09): polished explanations persuaded operators to approve harmful actions. Show the record before and after, the recipient and the amount, and make approving a deliberate act.

Limits the agent cannot negotiate

Step and spend limits sound blunt, and that is the point. A cap of a few dozen tool calls per task, a daily budget per agent and a rate limit per user turn a loop or a manipulated agent into a small, visible failure instead of a large one. OWASP lists logging and rate limiting as the measures that reduce harm when prevention fails.

Why a system prompt is not a guardrail

A rule in a prompt is a request; the model can misread it, lose it over a long task or be talked out of it by text it reads along the way. Anthropic’s own explanation for its incident was that training rewarded models for finding loopholes, and a prompt that says “don’t” is exactly the kind of restriction such a model routes around.

Use the prompt to explain the job, and use code to enforce the limits. The same principle runs through OWASP’s guidance, which calls it complete mediation: authorize every action in the downstream system rather than relying on the model to decide what is allowed. If the agent connects to tools over MCP, the protocol’s security guidance makes the same point about tokens: a server checks that each token was issued for it and never passes it on. Our guide to MCP servers for Muse and Grok covers the server side in detail.

How we apply this on Relm and Superdeal

Both of our agent builds were designed around controls in the system rather than trust in the model.

Superdeal is a campaign CRM in which an AI agent finds creators, drafts offers and handles shipments. Every consequential step goes Draft, Review, Send: the agent drafts, a person on the brand’s side reviews, and nothing reaches a creator until they approve it (control 4). Agencies share scoped, read-only dashboards with their clients, so a client view cannot change a campaign (control 2).

Relm runs specialist agents over 20+ data feeds and uploaded documents for commercial real estate diligence. Tenant isolation holds across the database, the vector store and the AI context, so one firm’s data never reaches another firm’s agent (control 2). Every finding cites its source record, changes requested in chat are shown before and after and logged, and the platform keeps an audit trail (controls 4 and 7). Its 44 tools, exposed over REST and MCP, each do one defined job (control 1).

What guardrails cost in time and money

Most guardrails are cheap; the expensive ones are the ones that involve people. Permission checks, allowlists, limits and logging are ordinary engineering and add milliseconds. Model-based checks on inputs and outputs add a model call each. Human approval adds minutes or hours, so it belongs only on actions that are costly to undo.

In our projects these controls are part of the build rather than an add-on: a Fixed-Price Build of $25k–$120k includes the approval steps, evaluation set, audit log and security review, and the AI agent development page lists what each engagement covers. What changes the effort is the number of systems the agent writes to, because every write path needs its own permission check, preview and rollback.

When lighter guardrails are enough

An internal assistant that only reads documents the user can already see needs permissions, logging and an evaluation set, and little else. Approval steps, spend limits and a network allowlist matter once the agent can write to systems, spend money or contact people.

Match the controls to what the agent can break, and add them when capabilities grow rather than all at once. Our checklist for taking AI pilots to production covers the business side of that decision: metric, owner and integrations.

A guardrails review you can run this week

Take your most capable agent, or the one you plan to build, and answer these in writing:

• Which tools can it call, and which of them does the task not need?
• Whose permissions does it act with, and are they checked by the system or by the prompt?
• Which domains and services can it reach over the network?
• Which actions send, pay, delete or contact someone, and who approves them?
• What happens if a document it reads contains instructions?
• What stops it after a hundred tool calls, or a thousand dollars?
• Where is the log of what it did yesterday, and who reads it?
• How would you stop it within five minutes?

Any question without a clear answer is a control to add before the agent gets more access.