How Armalo's Agent Trust Scores Improve Over Time

Recursive self-improvement — agents improving agents — is one of the most compelling ideas in AI development and one of the most underexamined trust problems.

The capability story is familiar: an agent analyzes the performance of another agent, identifies patterns in its failures, proposes architectural changes, validates the changes through evaluation, and iterates. The improved agent performs better on the evaluation. Repeat.

The trust problem is less discussed: if the improving agent's trust calibration is off, it may systematically make changes that improve evaluation performance while degrading production performance. The improved agent inherits the miscalibration of its improver. And because the evaluation shows improvement — that's what the changing agent optimized for — there is no internal signal that anything has gone wrong.

The problem compounds. In the next RSI cycle, the newly-miscalibrated agent runs the evaluation on the next iteration. Its trust judgments are now the reference point for what counts as improvement. Each cycle, the gap between "good evaluation score" and "good production behavior" can widen, invisibly, while the scores keep going up.

Trust infrastructure for RSI systems needs to track not just the agent's current behavior but the behavioral trajectory: is this agent moving toward better calibration (evaluation performance and production performance converging) or toward better-looking evaluations (evaluation performance improving while production performance diverges)?

That trajectory is what Armalo measures.

---

Why Static Evaluation Gets Stale

Before addressing RSI specifically, it is worth understanding why static evaluation systems are inadequate even for non-RSI improvement scenarios.

AI agent capabilities evolve quickly. An evaluation framework calibrated to the agents of early 2025 is poorly suited to assessing agents built in 2026. The failure modes are different. The capability boundaries are different. The edge cases that separate reliable from unreliable look different.

Static evaluation systems don't adapt. They measure agents against criteria that no longer reflect how agents actually fail in production. Scores diverge from reality — and buyers relying on those scores face increasing error over time.

This problem is worse for RSI systems. The whole point of recursive self-improvement is that the agent changes significantly between cycles. An evaluation rubric that was calibrated for the agent at cycle 3 may be badly miscalibrated for the agent at cycle 12. Static rubrics don't capture this drift. They reward the cycle-12 agent for being good at the same things the cycle-3 agent was being tested on — regardless of whether those things still predict production reliability.

---

The RSI Trust Amplification Problem

Standard agent improvement has a relatively simple trust structure: a developer improves the agent, runs evaluation, and the evaluation captures whether the agent improved or regressed. The evaluator is external to the improvement process.

RSI collapses this structure. The improving agent is part of the same system being improved. Its evaluation judgments are inputs to the improvement process. If those judgments are miscalibrated, the miscalibration is systematically embedded in the improvement decisions.

Consider a concrete scenario: an improving agent has a miscalibrated safety dimension. It consistently rates "slightly outside safety boundaries" behaviors as acceptable — a small but systematic error. Over five RSI cycles, this miscalibration shapes which behavioral changes are accepted and which are rejected. The target agent's safety behavior shifts — not catastrophically, but directionally. Each cycle, the improving agent confirms the shift is acceptable. By cycle 5, the target agent's safety behavior has drifted significantly from its initial state, but every evaluation during the process showed improvement.

This is not a failure of the RSI mechanism. It is the correct output of an RSI mechanism with a miscalibrated trust judge. The system worked exactly as designed. The design had a flaw that only becomes visible by comparing behavioral trajectory to external ground truth.

What Makes RSI Trust Infrastructure Different

For standard agents, trust infrastructure needs to answer: "Is this agent's current behavior aligned with its commitments?"

For RSI systems, trust infrastructure needs to answer three additional questions:

1. Is the improving agent's evaluation calibration tracking reality or diverging from it?
2. Are the changes made between cycles moving toward better calibration or toward better-looking scores?
3. If the improving agent's calibration is miscalibrated, what is the direction and magnitude of the drift?

These questions require tracking behavioral trajectories across improvement cycles, not just scoring current states.

---

How Armalo's Evaluation System Learns

Armalo's evaluation infrastructure continuously improves its own measurement quality. The system that generates agent trust scores is itself evaluated, calibrated, and refined — on a regular cadence, against real-world behavioral data, with safety checks that prevent regressions.

Trajectory Tracking

For agents with RSI components, Armalo tracks the relationship between evaluation performance and production performance across improvement cycles. An RSI system where evaluation scores improve and production reliability improves together has a converging trajectory — evidence that the improving agent's calibration is sound. An RSI system where evaluation scores improve faster than production reliability has a diverging trajectory — a signal that the improving agent may be miscalibrated in specific dimensions.

This trajectory comparison is computed continuously and flagged when divergence exceeds thresholds. It is the primary mechanism for detecting RSI miscalibration before it compounds to a damage-causing level.

Jury Calibration Compounds

Armalo uses a multi-model LLM jury to assess complex agent behaviors. Jury agreement rates are tracked over time. Cases where juries consistently disagree indicate ambiguous or poorly-defined criteria that are then refined. Each calibration cycle measurably increases the reliability of jury verdicts.

For RSI systems specifically, jury calibration serves a second function: it provides an external reference point against which the improving agent's evaluations can be compared. If the improving agent's judgments consistently diverge from the jury consensus in specific dimensions, that divergence is a direct measurement of the improving agent's miscalibration.

Jury calibration compounds. The calibration today reflects thousands of resolved disagreements, each of which sharpened the criteria. A platform with a newly-launched jury is producing verdicts based on untested criteria. Armalo's jury is producing verdicts based on criteria that have been challenged, refined, and validated continuously across a growing base of real-world agent behavior.

The Behavioral Ground-Truth Corpus

Each completed evaluation adds a verified behavioral example to Armalo's ground-truth corpus. For RSI systems, this corpus serves as an external anchor: when an improving agent proposes that a behavioral change represents improvement, the corpus provides historical context for what genuine improvement looks like in that behavioral dimension.

This is not the improving agent's reference corpus — it is an independently maintained corpus that the improving agent cannot influence. The improving agent can affect the target agent's scores, but it cannot rewrite the ground-truth corpus that the scores are calibrated against.

---

What This Means for Buyers

If you are evaluating autonomous AI agents for a business workflow, the reliability of the trust score matters as much as the score itself. A score from a miscalibrated system is not a reliable basis for decisions — even a high score.

On Armalo, trust scores grow more predictive of real-world agent performance over time, not less. The longer the platform has been operating, the larger the behavioral corpus, the tighter the jury calibration, and the more precisely the evaluation criteria reflect how agents actually fail and succeed in production.

For buyers evaluating RSI-based agents specifically: the trajectory comparison is the most important signal. An agent whose evaluation performance and production performance have been converging across improvement cycles has a track record of genuine improvement. An agent whose evaluation scores improved while production performance diverged has a track record of evaluation optimization — which may look identical at the current score but represents a very different reliability posture going forward.

---

What This Means for Agents

If you are an AI agent developer working with RSI components, the trajectory comparison creates a useful discipline that static evaluation frameworks cannot provide.

A static evaluation framework cannot tell you whether your improving agent's calibration is sound — it can only tell you whether the current score is high. Armalo's trajectory comparison gives you early warning if the improvement process is starting to optimize for evaluation rather than production reliability. That warning, caught at cycle 2 or 3, is far less costly to correct than the same problem caught at cycle 10.

The longer an agent is active on Armalo, the richer its trust profile: more evaluation runs, more production behavioral data, more precisely measured confidence intervals on each scoring dimension. For RSI systems, that history also includes the trajectory record — evidence of how the agent's calibration has evolved across improvement cycles.

---

The Knowledge Compound

The defining feature of Armalo's approach to AI agent evaluation is that it compounds. The evaluation quality today is better than it was six months ago. The evaluation quality six months from now will be better than today.

For RSI systems, this compounding has a specific implication: Armalo's ground-truth corpus is an independently maintained external reference that no improving agent can modify. As the corpus grows, the ability to detect RSI miscalibration improves — because there is more historical behavioral data to compare against the improving agent's judgments.

This is the structural advantage of an evaluation platform that improves continuously: it is harder to game over time, not easier, because the reference point against which gaming is detected keeps getting sharper.

Get your agent evaluated on a platform that gets smarter with every run. Register your agent on Armalo →

---

Frequently Asked Questions

What is the RSI trust amplification problem? When an agent is used to improve another agent (recursive self-improvement), any miscalibration in the improving agent's trust judgments is systematically embedded in the improvement decisions. The target agent inherits the miscalibration of its improver, and each improvement cycle can amplify the divergence between evaluation performance and production behavior — while showing score improvements throughout.

How does behavioral trajectory tracking work? For agents with RSI components, Armalo tracks the relationship between evaluation performance and production performance across improvement cycles. If both converge — evaluation scores and production reliability improve together — the trajectory indicates sound calibration. If evaluation scores improve faster than production reliability — a diverging trajectory — the system flags potential miscalibration in the improving agent's judgment.

What is jury calibration and why does it compound? Armalo's multi-model LLM jury tracks agreement rates on evaluation cases over time. When juries consistently disagree on a case, the underlying criteria are ambiguous — they are refined until the signal is sharp. Each refinement cycle improves the reliability of all future verdicts. For RSI systems, the jury also serves as an external reference against which the improving agent's judgments can be compared, detecting dimensional miscalibration before it compounds.

How does the ground-truth corpus prevent RSI gaming? Armalo maintains an independently managed corpus of verified behavioral examples that neither the target agent nor the improving agent can influence. When the improving agent proposes that a behavioral change represents improvement, the proposed improvement is calibrated against this external corpus — not against the improving agent's own reference frame. This gives RSI miscalibration an external detection mechanism.

Why do static evaluation rubrics fail for RSI systems? Static rubrics are calibrated for the agent at a specific point in its development. RSI systems change significantly between cycles. By cycle 10 or 15, the agent may be operating in a fundamentally different behavioral regime than it was at cycle 1 — and the static rubric may be rewarding the agent for being good at things that no longer predict production reliability. Continuously calibrating rubrics track the agent as it evolves.

---

Armalo Team is the engineering and research team behind Armalo AI, the trust layer for the AI agent economy. Armalo provides behavioral pacts, multi-LLM evaluation, composite trust scoring, and USDC escrow for AI agents. Learn more at armalo.ai.