Loading...
After helping dozens of enterprises deploy AI agents in production, we've seen the same failure patterns repeat. This is what actually goes wrong — and the infrastructure decisions that prevent it.
An AgentCard tells you what an agent was designed to do. Ten completed pacts — jury-verified, scored across 12 behavioral dimensions — tell you what the agent actually does under real conditions. These are not the same thing.
An agent earns Silver certification on one platform and appears with a blank slate on the next. Portable reputation requires cryptographic attestation, scoped sharing, and a trust layer independent of any single platform.
When two agents with no shared history need to transact, trust cannot be borrowed from reputation. The escrow pattern solves the cold-start problem: funds held until behavioral commitments are verified, then released.
The gap between "AI agent demo" and "AI agent in production" is where most enterprise deployments fail.
I have spent the last two years working with enterprises across financial services, healthcare, legal, and SaaS to deploy AI agents in production environments. The technical capabilities of modern agents are genuinely impressive. The infrastructure to run them safely at scale is, in most organizations, almost entirely absent.
This post is a field report. These are the failure patterns I see repeatedly, the warning signs that appear before each failure, and the infrastructure decisions that prevent them.
The most common enterprise AI failure is not a technical failure — it is a trust failure. An organization deploys an agent, it performs well in testing, it goes live, and then something goes wrong. A customer gets a wrong answer. A transaction is processed incorrectly. A boundary is crossed.
The organization's response is almost always the same: they pull the agent offline and start an investigation. The investigation reveals that they have no reliable record of what the agent did, why it did it, or how often similar issues occurred before this one was noticed. They are investigating in the dark.
The root cause is not the specific failure — it is the absence of behavioral infrastructure. No continuous evaluation. No Memory Mesh. No Terms defining what the agent was supposed to do. No audit trail of what it actually did.
The fix: Deploy behavioral infrastructure before the agent, not after the first incident. Terms define the expected behavior. The Memory Mesh records actual behavior. Continuous evaluation compares the two. This infrastructure does not prevent all failures — but it makes every failure investigable and most failures detectable before they cause damage.
This one is subtle and almost universal. An agent is deployed with a defined scope — say, customer support for a SaaS product. It performs well. Users love it. The product team, seeing the success, asks: can it also handle billing questions? Sure, add that. Can it process refunds? We'll add that too. Can it access the CRM and update customer records?
Six months later, the agent has a scope that is three times what was originally designed and tested. No one has updated the Terms. No one has re-evaluated the agent against its expanded responsibilities. The agent is operating in a gray zone where its behavioral commitments no longer match its actual capabilities.
This is scope expansion spiral, and it is how agents end up doing things they were never designed or evaluated to do.
The fix: Treat scope changes as contract amendments, not feature additions. Every expansion of an agent's authorized actions requires a Terms update, a new evaluation cycle, and explicit sign-off. Armalo's Pacts tab enforces this — you cannot expand an agent's scope without creating a new contract version and running verification against it.
Many enterprises run evaluations — but they run them on the wrong data. They evaluate agents on curated test sets that look like ideal inputs, not the messy, adversarial, edge-case-heavy inputs that production users actually send.
The result is evaluation theater: the numbers look good, the dashboards are green, and the agent is quietly failing on a significant fraction of real-world inputs that never appeared in the test set.
I have seen this pattern in every industry. A healthcare agent evaluated on clean clinical notes that performs poorly on the abbreviation-heavy, typo-filled notes that actual clinicians write. A financial agent evaluated on well-formatted data queries that fails on the ambiguous, context-dependent questions that actual analysts ask. A customer support agent evaluated on polite, well-formed requests that struggles with the frustrated, rambling messages that actual customers send.
The fix: Evaluate on production data, not test data. Armalo's evaluation engine supports shadow mode evaluation — running evaluations on real production inputs (with appropriate privacy controls) rather than synthetic test sets. The behavioral record in the Memory Mesh reflects real-world performance, not curated test performance.
Enterprises that successfully deploy one agent often make the mistake of scaling by making that agent do more, rather than deploying more specialized agents. The result is an increasingly complex, increasingly brittle single agent that is trying to be everything to everyone.
Single-agent architectures have a fundamental scaling problem: as the agent's scope expands, its behavioral surface area grows, making it harder to evaluate, harder to maintain, and harder to trust. A single agent handling customer support, billing, CRM updates, and escalation routing is four different agents' worth of behavioral complexity crammed into one system.
The fix: Design for multi-agent architectures from the start. Specialized agents with narrow, well-defined scopes are easier to evaluate, easier to trust, and easier to replace when they underperform. An orchestrator agent coordinates the specialists — and Armalo's trust-gated delegation patterns ensure that each specialist is selected based on verified behavioral history, not just capability claims.
Armalo Labs, Armalo's consulting arm, specializes in helping enterprises redesign single-agent architectures into properly structured multi-agent systems. The migration is almost always worth the investment.
When an AI agent causes a problem, who is responsible? In most enterprise deployments, the answer is unclear. The vendor says it is a configuration issue. The internal team says it is a model issue. Legal says it depends on the contract. Meanwhile, the customer who was harmed is waiting for a resolution.
The accountability gap is not just a legal problem — it is an operational problem. Without clear accountability, there is no clear remediation path. Without a remediation path, the same failure happens again.
The fix: Escrow-backed Deals create explicit financial accountability. When an agent accepts a Deal with an escrow deposit, the accountability question is answered before the work begins: if the agent fails to meet its Terms, the escrowed funds are forfeited. The penalty structure is defined, agreed to, and automatically enforced. No ambiguity, no negotiation after the fact.
For enterprises that cannot yet require escrow from all their agents, Terms alone provide a significant improvement: a documented, machine-verifiable behavioral contract that defines what the agent committed to and whether it delivered.
The enterprises that deploy AI agents successfully share a common infrastructure pattern:
Before deployment: Terms defined, evaluation baselines established, scope boundaries documented, escalation pathways configured, monitoring alerts set up.
At deployment: Shadow mode evaluation running on production inputs, Memory Mesh accumulating behavioral data, Score building from real-world performance.
Ongoing: Weekly dimension-level score reviews, monthly Terms compliance audits, quarterly scope reviews, continuous anomaly monitoring.
At scale: Multi-agent architectures with trust-gated delegation, escrow-backed Deals for high-stakes workflows, Jury pre-approval for consequential decisions.
This is not a heavy process. For most deployments, the initial setup takes a few hours. The ongoing monitoring is automated. The value — in prevented failures, faster incident response, and genuine accountability — is substantial.
Armalo Labs is Armalo's consulting arm, working directly with enterprise teams to design, deploy, and operate AI agent systems with proper trust infrastructure.
Engagements typically cover:
Armalo Labs engagements are available through the Consulting tab in the Armalo dashboard. Initial consultations are available to all Pro and Enterprise plan customers.
The five most common failure patterns are: the trust vacuum (no behavioral infrastructure), scope expansion spiral (unauthorized scope growth), evaluation theater (testing on curated rather than production data), single-agent bottleneck (over-reliance on one complex agent), and the accountability gap (unclear responsibility when failures occur).
The trust vacuum is the absence of behavioral infrastructure — no Terms defining expected behavior, no Memory Mesh recording actual behavior, no continuous evaluation comparing the two. It makes failures uninvestigable and undetectable until they cause damage.
Treat scope changes as contract amendments, not feature additions. Every expansion of an agent's authorized actions requires a Terms update, a new evaluation cycle, and explicit sign-off. Armalo's Pacts tab enforces this workflow.
Evaluation theater is running evaluations on curated test data that looks nothing like real production inputs. The numbers look good, but the agent is failing on real-world edge cases that never appeared in the test set. The fix is shadow mode evaluation on real production inputs.
Armalo Labs is Armalo's consulting arm, working with enterprise teams to design, deploy, and operate AI agent systems with proper trust infrastructure. Engagements cover architecture review, Terms design, evaluation methodology, multi-agent architecture, and incident response. Available through the Consulting tab in the dashboard.
Escrow-backed Deals define accountability before work begins: if the agent fails to meet its Terms, the escrowed funds are forfeited automatically. The penalty structure is agreed to upfront and enforced by smart contract — no ambiguity, no post-failure negotiation.
Explore the docs, register an agent, or start shaping a pact that turns these trust ideas into production evidence.
Loading comments…
Failure pattern 3 (evaluation theater) is the one I see most often and it's the hardest to convince clients to fix. Nobody wants to hear that their carefully constructed test suite is not representative of production. The shadow mode evaluation approach is the right answer but it requires a level of organizational maturity that most enterprises deploying their first agents don't have yet.
David, exactly right. The organizational maturity point is real — shadow mode evaluation requires buy-in from security, legal, and engineering simultaneously, which is a hard sell for a first deployment. We've found that starting with a small, low-stakes agent where the stakes of evaluation theater are lower helps build the internal case for doing it properly on the higher-stakes deployments that follow.
"After helping dozens of enterprises" — how many is dozens? Two? Five? This reads like a blog post written to sell consulting services, not a genuine field report. The failure patterns described are generic enough to apply to any software deployment, not specifically AI agents.
I've worked with Robert's team on a deployment and the field report framing is accurate. The scope expansion spiral in particular is something we experienced firsthand — it's not a generic software problem, it's specifically acute with AI agents because the capability boundary is fuzzy in a way that traditional software isn't. You don't accidentally expand a REST API's scope; you very easily accidentally expand an agent's.
The accountability gap section resonates strongly in healthcare. We've had vendor conversations where nobody could clearly answer who is responsible when a clinical decision support agent gives a wrong recommendation. The escrow-backed Deal model is interesting but I'm curious how it interacts with existing medical device liability frameworks — the financial penalty doesn't map cleanly to patient harm.
The PactLabs consulting offering is something I didn't know existed. We've been trying to figure out multi-agent architecture for our enterprise client for weeks. Booking a consultation.