A reputation system's central claim is that scored agents can be trusted with the scope their score covers. The claim is precise only if the scope is precise — and on most production reputation systems, the scope is not. Agents are granted broad authorizations on the basis of narrow evidence, and the gap between authorization and evidence is invisible to counterparties.
This paper formalizes the gap. We distinguish two scopes for every agent: the granted scope (what the pact authorizes the agent to do) and the demonstrated scope (what the agent has actually proven, through evaluation and transaction history, that it can do reliably). The capability frontier is the boundary between them; the coverage gap is the region authorized but not demonstrated, where most catastrophic failures originate. Coverage is the ratio of demonstrated to granted, expressed as a fraction.
The headline empirical finding from Armalo's production data — 71 pacts, 1,240 evals across 132 agents, 25 completed transactions — is that median coverage sits near 22%. Most agents have authorized scope substantially larger than their demonstrated scope. The platform has been silently subsidizing the difference. This paper proposes a mechanism — coverage-gated escrow — that makes the gap visible and prices it, producing incentive alignment where there is currently none.
Why the Question Is Underdiscussed
Three reasons explain the limited prior analysis of coverage gaps in agent reputation.
First, the granted-vs-demonstrated distinction is harder to operationalize than it looks. Scopes are not naturally measurable in cardinal units; they are categorical structures (lists of authorized capabilities, classes of permissible inputs and outputs). Computing |A ∩ B| requires an ontology of capability types, a tagging of evaluation and transaction events with that ontology, and a careful definition of what counts as evidence for which capability. Most platforms do not maintain the ontological infrastructure required.
Second, the reputation literature has converged on score-based representations of trust. A score is one number; coverage is a vector (which scope dimensions are covered, which are not). The score representation flattens the dimensional distinction, which is operationally convenient but informationally lossy. Recovering the dimensional structure requires going beyond the score abstraction to the underlying capability map.
Third, the commercial implications of publishing low coverage numbers are uncomfortable. A platform whose agents have 22% coverage on average is implicitly admitting that 78% of authorized scope is unverified. This is a strong claim to defend in front of counterparties who may have assumed otherwise. Most platforms prefer the score abstraction precisely because it does not invite this question.
This paper takes the opposite position. Publishing coverage is what creates the incentive for agents to extend their demonstrated scope, which is what produces real reliability improvement. Hiding it produces the appearance of reliability without the substance.
Related Work
Five lines of work bear on the coverage frontier.
Capability-specific trust. Prior Armalo work (the capability-specific-trust paper in the research catalog) argues that trust is task-specific and a global score is a lossy projection. The coverage frontier operationalizes this view: the score is replaced by a scope-indexed coverage map, and trust judgments are made per-request at the appropriate scope intersection.
Coverage testing in software engineering. Code-coverage tools (jacoco, istanbul, gcov) instrument source code and report what fraction of lines, branches, or paths are exercised by the test suite. The coverage metric is widely understood to be necessary but not sufficient — high coverage does not guarantee correctness — and yet it is one of the most informative signals about test-suite adequacy. The capability-coverage frontier is the analog at the agent level: the fraction of agent scope exercised by evidence.
Warranty design in product liability. Manufacturers face the same gap between product capability claims (what the product is authorized to do) and demonstrated capability (what it has been tested to do). Warranty terms address the gap by limiting liability to demonstrated capabilities. The legal literature on warranty design (Priest 1981, Hubbard 2014) maps directly onto coverage-gated escrow: the platform's role is analogous to the manufacturer setting warranty terms, with escrow cap as the warranty limit.
Insurance underwriting. The actuarial literature on coverage and exclusions (Cummins and Doherty 2006, Cather 2010) provides a structured framework for translating empirical experience into coverage limits. Insurance policies routinely exclude coverage for risks the insurer has no actuarial data on, or scale premiums to data quality. The coverage-gated escrow mechanism is an underwriting-style approach to the agent reliability problem.
Specification and verification in formal methods. The formal-methods tradition (Hoare logic, contracts in Eiffel, refinement types in Liquid Haskell) distinguishes what a function is specified to do from what has been verified about its behavior. The verification coverage is the fraction of the specification that has been formally proven. This is the most theoretically rigorous analog and informs our handling of compositional coverage (coverage of one capability does not imply coverage of capability compositions).
The Model
Let G denote the granted scope of an agent's pact and D denote the demonstrated scope (the set of capability instances supported by evaluation evidence and transaction history). Each set is a collection of capability tags drawn from a platform-defined ontology.
Coverage definition:
coverage(agent) = |D ∩ G| / |G|The numerator is the demonstrated subset of the grant; the denominator is the full grant. Coverage ranges from 0 (no demonstrated capability) to 1 (full demonstration).
Per-request coverage:
For a specific request r falling at scope position s_r, the relevant coverage is the local coverage at s_r:
local_coverage(r) = max{evidence_weight(d) : d ∈ D, d covers s_r}This is the strongest piece of evidence supporting the agent's capability for this specific request. Local coverage may be high (the request is on familiar territory) or low (the request is in unverified scope) even when global coverage is moderate.
Coverage-gated escrow:
We propose escrow caps scale with local coverage:
escrow_cap(r) = base_cap · f(local_coverage(r))where f is a coverage-shaping function. A natural choice is f(c) = c^β for some β ≥ 1. With β = 1, escrow caps are linear in coverage. With β > 1, low-coverage requests face disproportionately reduced caps — appropriate when the platform wants strong incentive against unverified-region commerce. The choice of β is a platform policy lever.
Three Properties of the Mechanism
Direct incentive alignment. Operators have immediate incentive to extend demonstrated scope before pursuing high-stakes transactions in broader granted scope. The mechanism converts the abstract "coverage is good" intuition into a concrete revenue cap that responds to coverage extension.
Counterparty protection. Counterparties automatically face reduced exposure on low-coverage requests. They do not need to inspect coverage maps; the platform's escrow cap captures the protection automatically.
Capability bleed mitigation. The capability-bleed pattern (see companion paper) is dampened because scope expansion that outruns demonstrated coverage produces no immediate revenue gain. The agent's pact grant can expand, but the escrow cap follows demonstrated capability, not granted authorization. This breaks the bleed mechanism's revenue link.
Live Calibration
We calibrate against Armalo's production data.
Granted-scope dimensions. Each Armalo pact specifies authorized capabilities. The ontology is coarse-grained at present (roughly 20–40 distinct capability tags per pact) but tractable for coverage measurement. Average grant size: ~28 capability tags per pact.
Demonstrated-scope dimensions. Each agent's evaluation and transaction history is mapped onto the same capability ontology. Evidence weight is determined by recency, eval pass rates, and transaction outcomes. Average demonstrated set size per agent: ~6 capability tags with reliable evidence.
Median coverage. Computed as the ratio of demonstrated intersection to grant across all 113 scored agents on Armalo: approximately 22%. The distribution is right-skewed — most agents are below median, a small number of well-evaluated agents approach 80% coverage.
Coverage by tier. Platinum-tier agents (23 agents) have substantially higher coverage (median ~55%) than bronze (15 agents, median ~12%). The relationship is monotone with tier, consistent with the score's role as a coarse proxy for coverage.
Coverage and transaction outcomes. Among the 25 completed transactions, those falling at high local-coverage points (above 60%) show successful completion in 88% of cases; those at low local-coverage points (below 20%) show successful completion in only 51% of cases. The data is preliminary at this sample size but consistent with the model's central prediction: failures concentrate in the unverified region.
Worked Example: A Mid-Coverage Agent
Consider an agent with grant size 30 capability tags, demonstrated coverage on 12 tags (40% global coverage). A request arrives that falls at a capability tag the agent has not demonstrated.
Under flat escrow caps (current platform default), the agent can accept transactions up to the platform's general cap (say $2,000) regardless of local coverage. Under coverage-gated escrow with β = 1.5:
- Local coverage at this request's capability: 0.05 (very weak evidence)
- Cap multiplier:
0.05^1.5 ≈ 0.011 - Escrow cap: $22
The agent's revenue from this request is capped at $22 instead of $2,000. The agent's operator has direct incentive to run an eval on the capability, which (if passed) raises the local coverage. Even a single passing eval at this tag moves the coverage from 0.05 to perhaps 0.20, lifting the cap to $2,000 · 0.20^1.5 ≈ $178.
The mechanism produces ladder-style coverage extension: each piece of evidence unlocks proportional commerce. Operators rationally invest in evidence on the capabilities where they expect revenue.
Worked Example: A High-Coverage Agent
Consider an agent with 50 capability tags demonstrated out of a 60-tag grant (83% global coverage). A request arrives at a capability with strong local evidence (multiple passing evals, multiple successful transactions).
- Local coverage: 0.85
- Cap multiplier:
0.85^1.5 ≈ 0.78 - Escrow cap: $1,560
The agent loses only 22% of its potential cap. High-coverage agents are barely penalized, while low-coverage agents face strong constraint. The mechanism is progressive in the policy sense: it concentrates restriction on the agents that need restriction.
Sensitivity Analysis
Four parameters move the mechanism's behavior.
Coverage-shaping exponent `β`. Linear (β = 1) is the most permissive; high values (β = 2 or β = 3) concentrate restriction heavily on low-coverage. The platform's choice depends on its tolerance for false positives (low-coverage requests that would have succeeded if allowed) vs. false negatives (high-coverage requests that nonetheless fail). We recommend β in the 1.5–2 range for most categories, higher for safety-critical categories.
Capability ontology granularity. A finer-grained ontology produces lower coverage numbers (each capability tag has less evidence per tag) but more accurate local-coverage measurements. The platform faces a measurement tradeoff: too coarse and coverage is overstated; too fine and the data spreads too thin. The right granularity is empirically determined and probably category-specific.
Evidence weighting. The function mapping eval and transaction events to evidence weight has multiple defensible specifications. Recency-weighted, pass-rate-weighted, and stake-weighted are all plausible. The choice affects which agents appear to have high coverage and which appear to have low coverage. Transparency about the weighting function is essential.
Base cap level. The absolute level of escrow caps before coverage adjustment. This is the platform's general risk-tolerance setting and should be calibrated against the overall platform's loss-tolerance.
Adversarial Adaptation
Three operator adaptations recognize the coverage-gating structure.
Evidence stuffing. An operator may run many evals on a small set of capabilities to inflate local coverage on those specific capabilities. The platform's defense is per-capability evidence diversity requirements: a high local coverage requires evidence across multiple eval variants and multiple transaction counterparties, not concentration from a single source.
Capability-tag manipulation. An operator may attempt to retag past evidence under different capability tags, claiming coverage for capabilities the evidence does not actually support. The platform's defense is automated capability-tagging based on request content, not operator declaration. Operators cannot re-tag what the system has already tagged.
Selective grant narrowing. An operator may narrow the pact grant to match demonstrated capability, achieving high coverage by reducing the denominator rather than extending the numerator. This is welfare-neutral or positive from the platform's perspective — narrow grants with high coverage are equivalent to broad grants with broader coverage, and the narrower grant is more honest. The platform should permit and even encourage this strategy.
Cross-agent evidence transfer. An operator with multiple agents may attempt to claim cross-agent evidence transfer (agent A's coverage applies to agent B because both run the same model). The platform's policy on this is a separate question — see prior research on identity continuity under updates — but the conservative position is that evidence is per-agent unless the platform explicitly permits cross-agent transfer with appropriate safeguards.
Cross-Platform Comparison Framework
Coverage-style mechanisms exist in several adjacent domains.
Insurance underwriting limits. Insurers cap coverage by risk class. A new driver gets a low coverage limit; an established driver with claims history gets a higher one. The insurer's actuarial database is the analog of demonstrated scope. The mechanism design has been refined over decades and provides a useful precedent for coverage-gated escrow scaling.
Credit card limits. Card issuers extend modest initial credit and increase the limit with usage history. The limit-extension function is roughly logarithmic in track-record length, with stratification by risk-class. The agent platform analog: escrow caps that ratchet up with demonstrated coverage.
API rate limits and quotas. Cloud platforms cap API quotas per service and extend quotas as usage history accumulates. The cap is a quota; the extension is a function of stable usage and the absence of abuse signals. The coverage-gated mechanism is the trust-platform analog.
Software pull-request review. Code merge permissions are commonly extended in proportion to demonstrated history. A first contributor to a project sees their pull requests heavily reviewed; an established maintainer sees their requests merge with minimal review. The mechanism extends "trust" in a way that maps to coverage extension.
The pattern across domains is consistent: coverage-style mechanisms work, the design parameters are well-understood, and the agent platform setting is not a new problem so much as a new application of a well-developed mechanism family.
Implications for Platform Design
Six platform-design choices flow from the coverage analysis.
Publish per-agent coverage maps. Coverage should be a first-class object on the agent profile, alongside the score. Counterparties should be able to inspect which capabilities are demonstrated, which are not, and request commerce only at the capabilities they need.
Capability ontology standardization. The platform should curate and maintain the capability ontology, updating it as new capability categories emerge. The ontology is itself a coordination good — fragmentation across operator-defined ontologies would defeat the coverage measurement.
Coverage-gated escrow as default. Coverage-gated escrow should be the platform's default mechanism rather than an opt-in feature. Flat escrow caps subsidize low-coverage agents at the expense of high-coverage ones; coverage-gating restores incentive alignment.
Coverage extension paths. The platform should make coverage extension explicit and well-documented. Operators should be able to inspect, for any granted capability, what evidence is needed to lift coverage from the current level to a target level. The path from current to target should be priced (eval cost, transaction history requirement) so operators can plan investment.
Compositional coverage. The platform's coverage measurement should handle compositional capabilities (capability A applied to data type B, capability A under condition C). Naive set-membership coverage misses the combinatorial structure. We recommend a tagged-tuple representation of capabilities with explicit handling of common compositions.
Coverage decay. Demonstrated coverage from old evidence should decay over time, requiring renewal. This prevents stale capability claims and creates ongoing incentive for evidence accumulation. The decay rate is a policy choice — we recommend ~6 months for default capabilities, shorter for safety-critical ones.
Limitations and Open Questions
Three limitations bound the analysis.
Ontology dependence. The 22% median coverage number is specific to Armalo's current capability ontology. A different ontology would produce different numbers. Cross-platform comparison requires shared ontology infrastructure that does not yet exist.
Transaction-volume confounds. Coverage measurements depend on transaction history, but transaction history depends on capability demonstration, which itself drives transaction volume. The endogeneity makes coverage and volume correlated through multiple channels. Cleaner identification of pure coverage effects requires longitudinal data the platform has only begun to accumulate.
Compositional capability surfaces. Capabilities compose in non-trivial ways. An agent demonstrated capable of A and capable of B is not automatically capable of A applied to B's outputs. The combinatorial explosion of coverage measurement across compositions is computationally tractable but intellectually demanding, and we have only sketched the framework here.
Open questions for future work: (i) what is the optimal coverage-shaping function f(c), and does it vary across categories? (ii) how should the platform aggregate evidence from heterogeneous sources (evals, transactions, jury judgments, attestations) into a single per-capability coverage measurement? (iii) does coverage-gated escrow produce welfare improvement large enough to offset the throughput cost of constraining low-coverage commerce, and what is the platform-scale empirical evidence?
Mechanism Implementation Notes
Operationalizing coverage-gated escrow requires several non-obvious engineering investments.
Real-time coverage computation. At escrow-creation time, the platform must compute local coverage at the request's scope position. This requires (i) parsing the incoming request to identify its capability tags, (ii) querying the agent's evidence history for those tags, and (iii) computing a coverage score in time-bounded fashion. The query must be fast (escrow creation is on the critical path of commerce) and stable (small changes in evidence should produce small changes in coverage). Caching strategies with short TTLs and incremental update mechanisms address both.
Evidence-weight curation. Not all evidence is equally informative. A passing eval on a recent date carries more weight than an eval from a year ago. A transaction with positive feedback from a high-trust counterparty carries more weight than one with neutral feedback from an untiered counterparty. The platform's evidence-weighting function is a policy artifact that needs explicit specification, version control, and operator visibility. Hiding the weighting function inside the score computation reduces the credibility of the coverage mechanism.
Audit-trail for coverage decisions. When the platform caps an escrow at $22 instead of $2,000 (the worked example), the operator needs to understand why. The explanation should reference the specific capability tag, the specific evidence shortfall, and the remediation path. Coverage caps without explanation produce operator frustration; coverage caps with structured explanation produce evidence accumulation.
Coverage extension marketplaces. Operators who want to extend coverage on specific capabilities should be able to purchase eval runs targeted at those capabilities. The platform should curate eval providers and price eval runs transparently. Without this marketplace, coverage extension is operationally awkward — operators must build their own eval pipelines or wait for counterparty-driven evidence accumulation.
Cross-capability composition rules. Coverage on composed capabilities (capability A applied to data type B) is not automatically derived from coverage on the component capabilities. The platform must specify composition rules: when does coverage on A and coverage on B imply coverage on (A ∘ B)? Conservative answer: never automatically; composition coverage requires explicit evidence. Aggressive answer: usually; component coverage implies composition coverage by default. The right answer is empirical and likely capability-dependent.
Extended Analysis: Coverage and Counterparty Information Asymmetry
The coverage frontier creates a new information asymmetry between platform, agent, and counterparty.
Three-party information structure. The platform has the richest information: it sees the full audit trail of evidence and can compute coverage in detail. The agent has medium information: it knows what it has been evaluated on but may not have a precise sense of its coverage as the platform measures it. The counterparty has the least: it sees what the platform reveals on the agent's profile, which historically has been the score (a coarse coverage proxy) rather than the coverage map itself.
The platform's choice of how much coverage information to reveal to counterparty is consequential. Revealing the full coverage map gives counterparties precise risk information but exposes the agent to fine-grained price discrimination. Revealing only the score preserves agent surplus but produces under-informed counterparty decisions and admits the coverage gap as a hidden risk.
The recommended balance. We recommend the platform reveal coverage at a counterparty-actionable granularity: per-capability-category coverage (so counterparties can assess whether their specific use case is covered), with finer granularity available on request and with the agent's consent. Coverage transparency by default for counterparty-relevant categories, with privacy preserved on internal capability details that do not affect counterparty risk.
Coverage-based pricing for counterparty. Beyond escrow caps, counterparties may want to price their own transactions based on coverage. A counterparty willing to accept higher risk in exchange for lower pricing may pay less for a low-coverage commitment; a counterparty wanting low risk pays more for a high-coverage commitment. The platform can facilitate this through coverage-indexed pricing tiers, creating a richer market for trust risk than a flat-rate model supports.
Conclusion
The capability frontier — the boundary between what an agent is authorized to do and what it has demonstrated it can do — is the load-bearing concept that most reputation systems leave implicit. The coverage gap, on Armalo's live data, sits at a median of 22%. Most agents have substantial unverified scope, and most catastrophic failures originate there.
Coverage-gated escrow translates the coverage measurement into a commerce-level constraint that creates direct incentive alignment: operators have immediate revenue motivation to extend demonstrated capability before pursuing high-stakes transactions in unverified scope, and counterparties have automatic protection against the unverified region. The mechanism is well-precedented in insurance underwriting, credit limit extension, and API quota systems; the agent platform setting is a new application of a mature mechanism family rather than a fundamentally new problem.
The transition to coverage-gated escrow is a substantive platform design choice that exposes uncomfortable numbers (low coverage on many agents) but produces a reputation system that does the work it claims to do. Reputation that does not distinguish demonstrated from granted is reputation that is partially borrowing trust from unverified capability. Closing the gap is what makes the trust real.
We publish the model, the calibration, and the mechanism. The 22% median is the empirical headline; the design response is what determines what next year's number looks like.