How do you audit what an AI agent did in production?

You need a trail that records every action the agent took or was blocked from taking, who or what approved it, and when, in a format that ties back to your actual policy. A general-purpose trace log shows what happened; an audit trail specifically needs to show what was allowed to happen and who is accountable for that decision.

What is an approval hold for AI agents?

An approval hold pauses an agent action that exceeds a defined risk threshold, such as a refund over a dollar limit, instead of letting it proceed automatically. A named human reviewer approves or rejects it, and that decision is recorded against them rather than against the agent.

Does SOC 2 compliance require AI agent audit evidence?

SOC 2 doesn't name AI agents specifically, but its control objectives around access control, change management, and monitoring apply to any system making consequential decisions, agents included. If an agent can approve a transaction or take an action with financial or customer impact, an auditor will expect evidence of what it was authorized to do and how that authorization was enforced.

Who is accountable when an AI agent makes a wrong decision?

Without an audit trail that separates actor types, accountability defaults to whoever is easiest to blame, usually the team that shipped the agent, regardless of whether a human approved the specific action, an API key triggered it automatically, or the system acted within a pre-approved policy. Attributing each action to a specific actor, human, key, or system, is what makes accountability answerable rather than assumed.

How to Govern AI Agents: Proving What They Were Allowed to Do

Name: Zespan
Brand: Zespan
Availability: InStock

A compliance reviewer asks: what was this agent allowed to do, and who approved the decision that led to a $4,000 refund on an account that should have been capped at $2,000?

If the honest answer is "the agent decided it was fine, and we trust the prompt to have gotten it right," that's not an answer a reviewer, a regulator, or a customer disputing a charge will accept. This post covers what actually governing an AI agent — as opposed to hoping it behaves — requires.

Why "we trust the prompt" doesn't hold up

A system prompt describing what an agent should and shouldn't do is a design intention, not a control. It's not enforced outside the model's own discretion, it can be bypassed by a phrasing the prompt wasn't tuned against (see how to stop an AI agent from making the same mistake twice), and — critically for governance — it leaves no record of what the agent was actually authorized to do at the moment it acted. If someone asks six weeks later why a specific refund went through, "the prompt said not to approve refunds over $2,000" isn't evidence. It's a description of an intention that may or may not have held for that specific request.

Governing an agent means having three things a prompt alone can't give you: a hard limit that's actually enforced, a point where a risky action stops for a human, and a record of what happened that names who's accountable.

What governing an agent in production actually requires

1. Enforcement that runs where the agent runs, not as a suggestion to the model. A governance rule has to be a check the agent's action is evaluated against, not an instruction it's asked to follow. This is the same mechanism covered in AI agent guardrails for production: a rule compiled into a bundle the SDK evaluates in-process, refused inside your own service if it violates policy, regardless of how the model arrived at the decision.

2. A hold for actions that exceed a risk threshold. Not every action needs a human in the loop — most don't, and requiring approval for everything defeats the purpose of automation. But an action with real consequences above a defined threshold — a refund over a dollar amount, a commitment above a certain value, anything that would need sign-off if a person did it — should pause instead of executing, route to a named reviewer, and wait. The agent doesn't get to decide it's fine on its own past that line.

3. An audit trail that names the actor, not just the event. "Refund issued" is an event. "Refund issued, blocked by policy, held for finance-oncall, approved by [specific person] with a reason on the record" is evidence. The difference matters because compliance and legal review isn't asking what happened; it's asking who is accountable for it. A trail that doesn't distinguish a human decision from an automated one from a scheduled job effectively has no accountable actor at all, no matter how detailed the individual log lines are.

What this looks like as one action

09:42:10  policy.blocked        refund.issue — $4,000 over the ceiling
                                 evaluated in-process
09:42:10  approval.requested    held for review — finance-oncall
                                 agent paused
09:51:03  approval.decided      approved, with a reason on the record
                                 actor: user
09:51:03  audit.recorded        one entry, attributable
                                 SOC 2 · CC8.1

Four things happened in this sequence, and each one is the answer to a different question a reviewer will actually ask. What was blocked and why: policy.blocked, evaluated in-process, against the actual $2,000 ceiling. Who was asked to decide: approval.requested, a named reviewer, not "the system." What was decided and by whom: approval.decided, actor type user, with a reason attached. That the decision is on the permanent record: audit.recorded, mapped to a real SOC 2 control (CC8.1, change management).

Compare this to a trace log that shows the refund API call succeeded. That tells you the transaction happened. It tells a reviewer nothing about whether it was authorized, who authorized it, or what would have happened if no one had.

Why the actor type matters more than it looks

A subtle failure in a lot of "audit logging" is recording that something happened without recording what kind of actor caused it. A refund approved by a person under a documented exception is a different thing than a refund approved automatically because it happened to fall inside policy, which is different again from a scheduled batch job re-running a previously approved action. If your log format doesn't distinguish actor: user from actor: api_key from actor: system, a scheduled job's actions get implicitly attributed to whoever happened to trigger it — usually not accurately, and not in a way that holds up when someone asks specifically who decided this.

Why this is a Team-plan capability, not a Free-tier checkbox

Approval holds, audit trails mapped to compliance controls, and agent governance at this level are meaningfully more than turning on a guardrail — they require a review workflow, actor attribution across every action type, and an evidence trail built to be handed to an external auditor. In Zespan, this is governance, available on the Team plan and above. If you're on an earlier plan and don't yet need SOC 2-mapped audit evidence, the guardrail enforcement layer covered in AI agent guardrails for production is available starting on Solo. See pricing for exactly what's included at each tier.

The takeaway

If your answer to "what was this agent allowed to do" is a description of what the prompt says it should do, you don't have governance — you have a hope that held so far. Governing an agent means the constraint is enforced where the agent runs, the risky decisions stop for a specific named person, and the record shows who is accountable, not just what occurred. That's the difference between an incident you can explain in one sentence and one that takes a week of log archaeology to reconstruct, badly.