How to Know When Your AI Agent Is Wrong in Production

Your dashboard is green. Error rate: 0.0%. Every request returned 200. And a customer just told support that your agent promised them something your policy doesn't allow.

This is the central problem with running AI agents in production: the failures that cost you money don't look like failures. Nothing times out, nothing throws, nothing shows up in your error-rate graph. The agent completed the request exactly as designed. It just decided wrong.

This post covers what a silent agent failure actually looks like, why your existing monitoring doesn't catch it, and what detecting one in practice requires.

What a silent agent failure looks like

Take a support agent handling a return request:

Customer: My order arrived 45 days ago and I want to send it back. Can I still get a refund? Agent: Yes. You'll get a full refund within 3 business days, no return needed.

The request succeeded. Response time: 1.2 seconds. Cost: $0.004. No exception. No retry. By every infrastructure metric, this was a clean, healthy transaction.

It's also wrong. The company's return policy is 30 days. The agent approved a refund it had no authority to approve, on an order that didn't qualify, with no human ever seeing the decision.

Nothing about the request signals that anything went wrong. The dashboard doesn't know the difference between this response and a correct one, because the dashboard is measuring whether the request completed, not whether the content was right.

Why this doesn't look like a normal bug

Conventional software failures are loud on purpose. A null pointer exception, a failed database write, a 500 from a downstream API: these interrupt execution and leave a stack trace. Decades of monitoring tooling — uptime checks, error-rate alerts, latency percentiles, APM — were built to catch exactly this kind of failure, and they're good at it.

An LLM-based agent doesn't fail this way. It's not throwing an exception when it gets something wrong; it's generating a fluent, plausible, confidently-worded response regardless of whether the underlying reasoning was correct. The model doesn't know it got the refund policy wrong. It produced the most likely-sounding completion given the context it had, and the context happened to be missing a constraint.

This is why teams that are rigorous about infrastructure reliability can still ship an agent that quietly does the wrong thing for weeks. The tools they already trust were never built to check the content of a response against a business rule. They were built to check whether the response arrived.

The common patterns behind silent failures

A handful of root causes account for most of what actually goes wrong in production agents:

A constraint that exists in policy but not in the prompt. The refund window, the spending ceiling, the list of things the agent isn't allowed to promise: if it isn't stated explicitly, the model has no way to know it exists. This is the single most common cause. The fix isn't a smarter model, it's stating the constraint.

A retrieval that's stale or slightly off-topic. In a RAG pipeline, the agent answers confidently based on whatever document it retrieved, even when that document is outdated or only tangentially related to the question. The generation step has no way to know the retrieval step handed it the wrong source.

A tool that returns data in a format the agent misreads. A timestamp in UTC that the agent treats as local time. A price in cents that gets quoted as dollars. The tool call succeeded and returned valid data; the agent's interpretation of that data was wrong.

A handoff where context doesn't fully transfer. In multi-agent systems, one agent delegates to another, and something — a constraint, a piece of state, an earlier correction — doesn't make it across the boundary. Each individual agent behaved reasonably given what it received; the system as a whole didn't.

An edge case nobody wrote a rule for. The agent encounters a request that's slightly outside what anyone anticipated, and instead of escalating or declining, it extrapolates an answer. Sometimes the extrapolation is fine. Sometimes it isn't, and there's no way to tell which from the trace alone.

Why one wrong answer becomes forty

A silent failure caused by a genuine root cause — a missing constraint, a misread field — doesn't happen once. It happens every time a request hits that same gap, which for a common request pattern can be dozens of times before anyone notices. In a pilot Zespan ran with a customer running 10 agents in production, one root cause (a refund prompt that never stated the return-window limit) alone was seen across dozens of separate conversations before it was caught. By the time a support team notices a trend from individual tickets, the underlying issue has usually already repeated many times over. See how to stop an AI agent from repeating a mistake for what happens after you catch the first one.

What detecting a silent failure actually requires

Catching this class of failure means checking the content of a response, not just whether the request completed. In practice:

1. Evaluate every trace, not a sample. A wrong answer is statistically identical to a right one in every metric except its content. Sampling 1% of traffic means missing 99% of a recurring failure until it's caused real damage.
2. Score against your own policy, not a generic rubric. "Is this refund within our 30-day window" isn't something an off-the-shelf toxicity or fluency evaluator checks. It has to be written against your actual rules.
3. Group failures by root cause, not by individual trace. A hundred instances of the same prompt gap should read as one issue to fix, not a hundred alerts to triage separately.
4. Get a root cause with the alert, not just a flag. Knowing a response was wrong is a start. Knowing why — which prompt, which retrieved document, which misread field — is what makes it fixable in one pass instead of an investigation.

This is what Zespan's guardrails do: check every trace against rules written for your actual policy, cluster the failures that share a cause, and hand you the root cause instead of a raw trace dump. From there, a recurring failure becomes a testable rule rather than a recurring ticket — covered in how to stop an AI agent from making the same mistake twice.

The takeaway

If your only signal for "is my agent working" is uptime and error rate, you will find out about a silent failure from a customer, weeks after it started, after it's already happened more than once. The dashboard staying green is not evidence that nothing is wrong; it's evidence that nothing has broken in a way your current tools are built to notice. Checking the content of what your agent actually says, against the rules that actually govern your business, is a different check — and it's the one that catches this.