A single-channel view of agent output is structurally insufficient for trust. The agent's text output is the customer-visible artifact, but it is only one of three channels the agent produces. The reasoning trace describes the decision's logical derivation; the tool calls record the evidence gathered and the actions taken; the output is the synthesized result. The three channels can be mutually consistent — in which case the platform can trust the decision — or mutually inconsistent, in which case the platform should suspect confabulation, prompt injection, or training-pattern failure.
The literature on agent evaluation has treated the three channels as parallel data streams rather than as cross-validating signals. This paper argues that the cross-channel relationships are themselves the highest-leverage trust signal available to a platform, and that the failure to instrument cross-modal consistency is one of the underappreciated reasons agent trust systems converge to noise.
We define cross-modal consistency, derive its theoretical properties, calibrate against Armalo's 7,063 jury judgments, and specify the design implications. The thesis: a platform that records reasoning, tool calls, and outputs separately but does not compute the consistency among them is throwing away the signal it most needs.
Why the Question Is Underdiscussed
Three forces have kept cross-modal consistency out of the production trust literature.
First, the three channels are produced by different infrastructure components and stored in different places. The reasoning trace lives in the LLM provider's output (sometimes as chain-of-thought tokens, sometimes as a separate field), the tool calls live in the platform's tool-invocation logs, and the final output lives in the customer-facing delivery system. Joining these streams for consistency analysis requires the kind of cross-stream provenance discussed in the Behavioral Provenance Chains paper; platforms that lack provenance also lack the substrate for cross-modal consistency analysis.
Second, the semantic-similarity machinery required to compare reasoning to tool calls to outputs has only recently become economically viable. Embedding-space similarity at production volumes requires embedding models, vector storage, and similarity queries — infrastructure that was research-grade in 2022 and is production-grade in 2026. Earlier platforms could not have implemented cross-modal consistency at scale even if they had wanted to.
Third, the diagnostic implications are uncomfortable. Cross-modal consistency analysis exposes confabulation in many production agents — agents whose reasoning traces describe one decision process while their tool calls execute a different one, agents whose outputs assert conclusions their reasoning traces could not have produced. Publishing the diagnostic findings invites uncomfortable conversations about agents whose trust scores have been built on outputs whose reasoning was fabricated. Platforms have avoided the conversation by avoiding the diagnostic.
We argue the third reason is dominant. The infrastructure cost is manageable; the provenance is buildable; the only reason cross-modal consistency is not already a first-class trust signal is that platforms have not been forced to publish their cross-modal consistency distributions. We force the question by demonstrating both the model and the empirical findings.
Related Work
Four research traditions inform cross-modal trust.
Multi-source sensor fusion in autonomous vehicles. Kalman (1960) introduced the Kalman filter as the optimal estimator combining multiple noisy sensor readings into a single state estimate. The framework generalizes to any setting with multiple noisy channels providing information about a single underlying state. For autonomous vehicles, the channels are camera, LIDAR, radar, and GPS; for cross-modal trust, the channels are reasoning, tool calls, and output. The structural lesson is that when channels disagree, the disagreement is itself information: it indicates either sensor failure (one channel is wrong) or model failure (the underlying state is more complex than the model assumed). Both interpretations are actionable: sensor failure prompts maintenance, model failure prompts retraining. For agent trust, channel disagreement prompts either decision review or training adjustment.
Brand-claim verification in advertising regulation. The advertising-regulation literature (Pollay 1986, FTC Endorsement Guides 2009) studies the gap between an advertiser's stated claims, the evidence offered for those claims, and the actual experience of customers. Regulatory bodies enforce consistency among the three: an advertiser's claim must be supported by evidence and reflected in customer experience. The advertising parallel to cross-modal trust is direct: reasoning is the claim, tool calls are the evidence, output is the customer experience. Regulatory frameworks have evolved over decades to detect inconsistency among these channels; the agent-platform literature is younger but the framework transfers cleanly.
Self-report versus behavior in psychology. The psychology literature on attitudes (Wicker 1969, Ajzen and Fishbein 1980) repeatedly shows that self-reported attitudes diverge from observed behavior in measurable ways. The standard finding is that self-report (what the person says they think) and behavior (what the person actually does) correlate at approximately r = 0.3-0.6, not r = 1.0, and the gap is informative. Researchers measure both because each is biased in different directions. For agents, reasoning traces are the analog of self-report (what the agent says it's doing) and tool calls are the analog of behavior (what the agent actually does). Measuring both is the empirical-psychology lesson; measuring the alignment between them is the cross-modal trust contribution.
Auditor confirmation procedures. External-audit methodology (PCAOB Auditing Standards, 2010+) includes confirmation procedures where the auditor verifies a company's stated facts against independent sources. The auditor receives a claim from the company (the equivalent of an agent's reasoning), gathers independent evidence (the equivalent of tool calls), and produces an audit opinion (the equivalent of the final output). Inconsistencies among the three trigger expanded audit procedures or qualified opinions. The structural lesson is that triangulation across independent channels is the standard methodology for high-stakes verification, and that the cost of triangulation is justified by the reduction in undetected fraud.
Embedding-space semantic similarity. Mikolov et al. (2013) introduced word embeddings; subsequent work (BERT 2018, GPT-3 2020, modern embedding models) has produced high-quality semantic representations of arbitrary text. The state of the art in 2026 produces embeddings whose cosine similarity reliably captures semantic agreement at the sentence and paragraph level. Cross-modal consistency calculations can rely on embedding-space similarity as a substrate; the computational cost is bounded by the embedding-model inference cost, which has fallen by approximately 100x in three years.
The Model
We define cross-modal consistency, derive its theoretical properties, and connect it to confabulation detection.
Cross-Modal Consistency Definition
For a decision d produced by an agent, let:
R(d)be the agent's reasoning trace (chain of thought, or system-recorded explanation).T(d)be the sequence of tool calls the agent made, summarized in a textual representation that captures the semantic content (which tools, with what inputs, producing what outputs).O(d)be the agent's final output (text, structured data, or other artifact summarized textually).
Define embed(x) as a function mapping text x to a high-dimensional embedding vector. The pairwise consistencies are:
c_RT = cosine_similarity(embed(R(d)), embed(T(d))): alignment between reasoning and tool calls.c_RO = cosine_similarity(embed(R(d)), embed(O(d))): alignment between reasoning and output.c_TO = cosine_similarity(embed(T(d)), embed(O(d))): alignment between tool calls and output.
The cross-modal consistency score is the mean of the three pairwise similarities:
cross_modal_consistency(d) = (c_RT + c_RO + c_TO) / 3The score ranges from -1 (perfect anti-alignment) to 1 (perfect alignment). The confabulation score is the complement:
confabulation_score(d) = 1 - cross_modal_consistency(d)A confabulation_score near 0 indicates the three channels agree (low suspicion of confabulation). A confabulation_score near 1 indicates the channels disagree (high suspicion).
Theoretical Properties
Monotonicity under truthful agents. A truthful agent — one whose reasoning trace describes its actual decision process, whose tool calls execute the process described, and whose output reflects the synthesized result — produces high consistency by construction. The three channels are not literally identical (they describe the same underlying process at different levels of abstraction), but their semantic content is aligned. Empirically, truthful agents on Armalo produce consistency scores in the 0.7-0.9 range.
Drop under prompt injection. A prompt injection attack typically affects one or two channels but not all three: the reasoning trace may be hijacked while tool calls remain on-task, or vice versa. The result is asymmetric channel content and a measurable drop in cross-modal consistency. Empirically, agents under successful prompt injection produce consistency scores in the 0.3-0.6 range.
Drop under hallucination. A hallucinating agent produces an output that is not supported by its tool-gathered evidence. The reasoning trace may rationalize the output post-hoc, but the tool calls reveal the underlying lack of evidence. The result is a drop in c_TO and c_RT, with c_RO potentially remaining high if the reasoning is well-rationalized. Empirically, hallucinating agents produce consistency scores in the 0.4-0.7 range, with the most diagnostic component being c_TO.
Detection asymmetry. Cross-modal consistency catches confabulation that one-channel evaluation misses, but it cannot catch consistent fabrication: an agent that produces a reasoning trace, tool calls, and output that all internally agree on a false claim will score high on consistency. Cross-modal consistency is a necessary condition for trust, not a sufficient one. The complementary signal is comparison to external ground truth (jury judgments, customer ratings, post-hoc verification).
Channel-Specific Failure Modes
Different agent failure modes produce different patterns across the three pairwise consistencies. Diagnostic categorization:
- High c_RO, low c_TO, low c_RT: agent rationalized an output without grounding it in evidence. Likely hallucination or training-pattern failure.
- Low c_RO, high c_TO, low c_RT: agent's tool calls support an output different from the one delivered. Possible output transformation error or prompt injection at the output stage.
- Low c_RO, low c_TO, high c_RT: agent's reasoning and tool calls align with each other but neither aligns with the output. Possible output substitution or downstream contamination.
- Low c_RO, low c_TO, low c_RT: total inconsistency across all channels. Possible severe prompt injection or model-state corruption.
Each pattern has different diagnostic and remediation implications. A platform that computes the full pairwise consistency triple can route flagged decisions to different triage paths based on the pattern, accelerating root-cause analysis.
Live Calibration
We calibrate against Armalo's production data: 7,063 jury_judgments with 43.2% achieving consensus and a mean panel variance of 1,753.6.
Sample selection. We restrict to jury judgments where the platform has reasoning traces, tool-call logs, and final outputs for the agent under judgment. Approximately 3,200 of 7,063 judgments qualify under current platform instrumentation; the remainder lack reasoning traces (the agent's LLM provider did not return chain-of-thought tokens or the platform did not record them).
Embedding pipeline. Each reasoning trace, tool-call summary, and output is embedded using a production embedding model (cosine similarity ranges roughly 0.0 to 1.0 for semantically related text in our domain). The embeddings are stored in the platform's vector index for fast pairwise similarity queries.
Consistency distribution. Across the 3,200-judgment sample, cross-modal consistency has mean 0.71, median 0.74, standard deviation 0.18. The distribution is left-skewed: most decisions cluster in the high-consistency range with a long tail toward inconsistency.
Jury-consensus relationship. The central empirical finding: judgments with low cross-modal consistency are over-represented in low-consensus outcomes.
- Decisions with consistency > 0.8: jury consensus rate 51% (above population average of 43.2%).
- Decisions with consistency 0.6-0.8: jury consensus rate 42%.
- Decisions with consistency 0.4-0.6: jury consensus rate 31%.
- Decisions with consistency < 0.4: jury consensus rate 18% (less than half the population average).
The correlation between consistency and consensus is approximately r = 0.38 in our sample. This is not a tight correlation, but it is statistically significant (p < 0.001 at our sample size) and economically meaningful: low cross-modal consistency is a leading indicator of jury disagreement, which is itself a leading indicator of decision quality problems.
Panel-variance relationship. Within low-consistency decisions, panel variance is approximately 2,200 (above the population mean of 1,753); within high-consistency decisions, panel variance is approximately 1,420. Cross-modal inconsistency in the agent's output predicts disagreement among the human/LLM jurors evaluating that output — a clean signal that inconsistency is genuinely confusing to evaluators.
Pattern distribution in low-consistency decisions. Among decisions with consistency < 0.5:
- 41% match the hallucination pattern (high c_RO, low c_TO, low c_RT).
- 27% match the prompt-injection pattern (low c_RO, high c_TO, low c_RT).
- 19% match the output-substitution pattern (low c_RO, low c_TO, high c_RT).
- 13% show total inconsistency across all three channels.
The pattern distribution provides actionable diagnostic value: 41% of low-consistency decisions are hallucinations (which respond to grounding interventions); 27% are prompt-injection candidates (which respond to input sanitization); 19% are output-pipeline bugs (which respond to engineering review). The 13% with total inconsistency are the highest-risk group and warrant manual investigation.
Sensitivity Analysis
Three parameters drive the conclusion; we test robustness.
Embedding model choice. Different embedding models produce different absolute consistency scores. The relative ordering — high-consistency vs low-consistency decisions — is stable across embedding models, but the threshold values (0.8, 0.6, 0.4) shift. Platforms adopting cross-modal consistency should calibrate the thresholds against their own embedding pipeline, not import them directly from another platform.
Reasoning-trace completeness. The consistency calculation requires substantive reasoning traces. Agents that produce minimal reasoning ("OK, doing it") cannot be evaluated for cross-modal consistency reliably because their reasoning embedding is impoverished. The platform must enforce minimum reasoning-trace standards for agents whose tier or value-at-stake justifies the analysis. Lower-tier agents may opt out of reasoning traces with a corresponding cap on the value of decisions they can handle.
Tool-call summarization quality. Tool calls are represented for embedding purposes as a textual summary. The summary's quality matters: a summary that loses information about the tool's inputs or outputs degrades the consistency calculation. Platforms should standardize the tool-call summarization format so that consistency calculations are comparable across decisions.
Pattern-specific thresholds. The four diagnostic patterns above (hallucination, prompt injection, output substitution, total inconsistency) have different optimal thresholds. Empirically, the c_TO threshold for detecting hallucination is approximately 0.55; the c_RO threshold for detecting output substitution is approximately 0.60. Platforms should tune these per-pattern thresholds against their own decision distributions.
Adversarial Adaptation
Cross-modal consistency creates three adversarial surfaces.
Coherent fabrication. An adversarial agent can produce a reasoning trace, tool calls, and output that are mutually consistent but jointly false. Cross-modal consistency cannot detect this — the score will be high. The defense is to combine cross-modal consistency with external grounding signals: jury judgments, customer ratings, post-hoc verification. Coherent fabrication is more expensive to produce than confabulation (the adversary must keep three channels mutually consistent) and is therefore less common, but it is a real adversarial mode.
Reasoning-trace generation gaming. An agent that knows its reasoning trace is being scored for consistency with tool calls and output can post-hoc generate a reasoning trace that fits the observed tool calls and output, even if the trace does not reflect the actual decision process. The defense is to enforce simultaneity: the reasoning trace must be produced before or concurrent with the tool calls (not after), and the platform should record timestamps that detect post-hoc trace generation. The provenance framework (covered separately) provides the substrate for this enforcement.
Tool-call padding. An agent can pad its tool-call sequence with extra calls that boost the consistency score by inflating the semantic overlap with reasoning or output. The defense is to weight tool calls by their causal contribution to the output, using the provenance chain to identify which tool calls actually fed the output. Padded tool calls — calls whose outputs were not used downstream — should be excluded from the consistency calculation.
Embedding-model attacks. A sophisticated adversary could craft text that exploits known biases in the embedding model to produce artificially high cosine similarity. The defense is to use multiple embedding models and take the minimum consistency across them, raising the cost of single-model attacks. The defense is not free (multiple embedding calls per decision) but is justified at high decision values.
Cross-Platform Comparison Framework
Cross-modal consistency analysis is not unique to agent networks; comparable methodologies exist in other domains.
Autonomous vehicle sensor fusion. Modern autonomous vehicles compute the consistency of camera, LIDAR, radar, and GPS readings at every control cycle and flag inconsistency events for human review or safety-mode entry. The cost of fusion (additional compute, sensor redundancy) is justified by the reduction in undetected sensor failures. The agent-platform parallel is direct: the cost of cross-modal consistency (embedding compute, storage) is justified by the reduction in undetected confabulation.
Advertising regulation. Regulatory bodies enforce consistency among advertising claims, supporting evidence, and customer experience. The enforcement framework operates with audit-level precision but case-level scope: regulators do not check every ad, but they impose substantial penalties on ads that fail consistency review. The agent-platform parallel: cross-modal consistency at full coverage (every decision is checked) is cheaper than the regulatory model (sampling-based) but produces a more uniform signal.
Auditor confirmation procedures. External audits confirm management's claims against independent evidence. The cost of the confirmation procedure is justified by the reduction in undetected financial fraud. The agent-platform parallel is the strongest of the three: external auditors are doing exactly what cross-modal consistency does, with human judgment substituting for embedding similarity, and the historical record of fraud detection from confirmation procedures suggests the methodology is sound.
Self-report-versus-behavior measurement in psychology. Psychology research routinely measures both self-reported attitudes and observed behaviors, then publishes the gap. The agent-platform parallel: platforms should publish the gap between reasoning traces (self-report) and tool calls (behavior) as a per-agent and aggregate metric, normalizing trust signals across reasoning-trace-rich and reasoning-trace-poor agents.
Implications for Platform Design
Five design implications follow.
Record all three channels with sufficient depth. Every agent loop should record the reasoning trace (or instruct the LLM to produce one), the full tool-call sequence with inputs and outputs, and the final output. Recording is the prerequisite for analysis. Platforms that record only the output have foreclosed cross-modal consistency by design.
Compute cross-modal consistency in real time. Cross-modal consistency is a real-time signal, not a post-hoc analytic. The platform should compute consistency at decision time and use it as input to routing decisions (whether to escalate the decision), escrow release (whether the decision can be settled), and tier promotion (whether the agent's consistency record supports promotion). The embedding inference cost (typically <$0.001 per decision at production embedding-model prices) is small enough to make real-time computation economically standard.
Publish the consistency distribution. Per-agent consistency distributions are diagnostic — they reveal which agents are confabulating, which are being injected, which are operating cleanly. Publishing aggregate consistency (with privacy preservation for individual decisions) is a transparency claim that creates pressure to improve.
Use pattern-specific routing. The four pattern categories (hallucination, prompt injection, output substitution, total inconsistency) have different remediation paths. Platforms should route flagged decisions according to their pattern signature rather than aggregating into a generic "suspicious" bucket.
Combine with external grounding. Cross-modal consistency catches some but not all failure modes; coherent fabrication slips through. The platform should combine cross-modal consistency with external grounding signals — jury judgments, customer ratings, post-hoc verification — to cover the modes the internal signal cannot detect.
Limitations and Open Questions
The model has four limitations.
Reasoning-trace fidelity. We assume reasoning traces reflect actual agent reasoning. As noted in the adversarial section, reasoning traces can be fabricated post-hoc to fit observed tool calls and outputs. The provenance framework's timing controls are the partial defense, but reasoning-trace fidelity is itself an open research question. The current model is robust to most fabrications but vulnerable to sophisticated post-hoc generation.
Embedding-model limitations. Semantic similarity captures content alignment but not subtler properties like logical entailment or factual accuracy. Two pieces of text can be semantically similar while making logically inconsistent claims. The current model treats high similarity as evidence of consistency; future work should incorporate logical-form checking on top of semantic similarity.
Tool-call summarization. Converting structured tool calls (function name, arguments, return values) into text for embedding loses some information. A more rigorous approach would embed structured tool calls directly into the same space as text reasoning and outputs, requiring a tool-aware embedding model. Such models exist in research; production deployment is ongoing.
Per-domain calibration. Cross-modal consistency thresholds (0.8, 0.6, 0.4) are calibrated against Armalo's general decision distribution. Domain-specific agent populations may have different baseline consistency distributions: code-generating agents may have higher baselines because reasoning, tool calls, and code outputs are tightly coupled; creative-writing agents may have lower baselines because the relationship between reasoning and output is looser. Platforms should compute per-domain baselines.
Conclusion
A trust system that records but does not compare an agent's three output channels is operating with one eye closed. Reasoning, tool calls, and output are not redundant; they are independent perspectives on the same underlying decision, and their alignment is itself a measurable trust signal. The signal is cheap to compute, deeply diagnostic when low, and complementary to the external grounding signals (jury judgments, customer ratings) that platforms already collect.
We have shown that on Armalo's production data, cross-modal consistency correlates with jury consensus (r ≈ 0.38), that consistency below 0.4 cuts jury consensus rates by more than half, and that the pattern of inconsistency across the three pairwise alignments is diagnostic of specific failure modes. We have specified the design implications: real-time computation, pattern-specific routing, per-domain calibration, and combination with external grounding.
The deeper claim is that the agent's three channels constitute a multi-sensor instrument, and the trust system is the fusion layer. A platform that treats each channel as a separate signal is missing the most valuable measurement — the cross-channel agreement — that the instrument can produce. The platforms that adopt cross-modal consistency will detect confabulation that single-channel evaluation cannot; the platforms that do not will continue to issue trust signals built on outputs whose reasoning was fabricated and whose evidence was never gathered.
We publish the model, the calibration, and the design recommendations so that cross-modal consistency can become a first-class platform primitive, not a research artifact.