Behavioral Drift in the Wild: Same AgentCard, Different Weights, Different Agent

In conventional software engineering, a version number is a meaningful identity claim. When a library publishes v2.3.1, the behavior of every function in that library is deterministic given the same inputs. Hash the compiled artifact and you have a cryptographic proof of identity. Pin a dependency to v2.3.1 in your lockfile and you are guaranteed the same behavior tomorrow that you observed today, barring environmental factors outside the library's scope.

This guarantee does not exist for language model-based agents. A model version label — gpt-4-turbo, claude-3-5-sonnet-20241022, gemini-1.5-pro — is not a cryptographic identity claim. It is a commercial label that providers update, iterate, and sometimes silently revise according to their own release cadence. The label is self-declared and not bound to the weights in any verifiable way.

This creates a fundamental identity problem: the entity you hired — the agent you evaluated, benchmarked, and deployed based on observed behavior — may no longer be the entity serving your production traffic. The label says it is. The behavior says otherwise.

Consider the analogy of hiring a contractor. You hire someone, verify their work, establish a working relationship, and sign a long-term contract. Then, silently, the contracting firm replaces them with a different person who has a similar-looking resume but different skills, different work habits, and different reliability characteristics. The invoices still say the same name. The badge still shows the same face. But the work is different.

This is the situation that AI agent buyers are in today. They evaluate an agent, establish trust based on observed behavior, sign contractual or commercial commitments, and then discover — often through failure rather than notification — that the underlying behavioral substrate has changed.

The AgentCard Problem

Google's Agent-to-Agent (A2A) protocol introduced the AgentCard specification as a structured way for agents to advertise their capabilities, interfaces, and identity. An AgentCard includes fields for the agent's name, version, supported modalities, tool definitions, and endpoint information. It is designed to be machine-readable and to enable automated agent discovery and composition.

The AgentCard version field is a step in the right direction. It acknowledges that agent identity has a temporal dimension and that consumers need to know which version of an agent they are interacting with. But there is a fundamental limitation: the version field is self-declared. There is no cryptographic binding between the value in the version field and the actual model weights or behavioral profile of the agent at the time the AgentCard is served.

This is analogous to the state of npm packages before lockfiles became standard practice. Package authors could declare "version": "1.0.0" and then push updated code to the same version number. Consumers who installed package@1.0.0 a week apart could get materially different code. The introduction of package-lock.json and yarn.lock solved this by pinning the cryptographic hash of each installed package, creating a verifiable record of exactly what was installed.

The AgentCard specification needs an equivalent: a behavioral commitment hash that cryptographically binds the declared version to a verified behavioral profile. Without it, an AgentCard is a marketing document, not an identity certificate.

The Armalo protocol extends the AgentCard concept with two additional fields:

1. behaviorHash: A SHA-256 hash of the agent's behavioral fingerprint — computed from a standardized eval suite run against the agent at deployment time. Any change to the model, the system prompt, the tool configuration, or the sampling parameters that produces a materially different eval result will produce a different behaviorHash.

2. attestationCid: A content-addressed identifier pointing to an on-chain attestation record that anchors the behavioral hash to a timestamp and a verifier identity. The attestation is stored using EAS (Ethereum Attestation Service) on Base L2 and cannot be retroactively modified.

With these fields, a consumer can verify not just that an agent claims to be version 2.3.1, but that the agent's observed behavior at evaluation time matches the committed behavior hash, and that this hash was attested at a specific point in time by a verified auditor.

The Reproducibility Gap

The reproducibility gap in AI systems is wider than most teams realize. Even when nothing intentional changes, several sources of non-determinism can produce different outputs from the same model on the same input:

Temperature and sampling: A temperature of 0.7 does not produce the same output twice on any given input — it samples from a probability distribution. Over thousands of calls, the distribution of outputs is stable, but individual calls are not reproducible. This is fine for many applications but becomes a problem when behavioral consistency is a contractual guarantee.

Infrastructure-level randomness: Cloud inference providers do not guarantee that the same GPU, the same numerical precision settings, or the same batch composition will be used for any given inference call. Floating-point arithmetic at different precisions produces different results. This is a known source of behavioral variance that is almost never disclosed in API documentation.

Context assembly non-determinism: For agents that use RAG or dynamic context injection, the specific documents retrieved, their ranking, and their truncation behavior can vary run to run depending on vector index state, retrieval timing, and caching behavior.

Model update cadence: Providers update models continuously. Fine-tuning runs, RLHF updates, safety interventions, and capability improvements all modify the weight distribution. Some of these changes are disclosed in changelog entries. Many are not.

The cumulative effect is that an agent that passed your evaluation suite on Monday may behave differently on Friday, through no action of your own. The question is not whether behavioral drift is possible — it is. The question is whether you have the instrumentation to detect it when it happens.

---

Part II: Seven Mechanisms That Cause Behavioral Drift

Behavioral drift is not a single phenomenon. It has distinct mechanisms, each with different detectability profiles, different rates of occurrence, and different remediation paths. Understanding the mechanism is essential for designing appropriate countermeasures.

Mechanism 1: Model Version Updates (Provider-Side, Invisible to Operator)

This is the most impactful and least discussed source of behavioral drift. AI providers maintain the right to update the weights behind any given model identifier. OpenAI's usage policies explicitly state that models may be updated or replaced. Anthropic's documentation notes that model names refer to capability tiers rather than specific weight snapshots. Google's Gemini API documentation mentions that model behavior may evolve.

The practical consequence: your production agent is running against a model that you did not evaluate. You evaluated the model as it existed on the date of your evaluation. The model running today may have received one or more weight updates since then.

The frequency of these updates is not publicly disclosed. Based on observable behavioral changes documented by researchers and enterprise users, major providers appear to update their flagship models approximately every 4–8 weeks, with smaller updates potentially more frequent. The November 2023 → March 2024 period for GPT-4-turbo shows at least one major behavioral change (response length distribution) that was not accompanied by a version identifier change.

From an agent governance perspective, this means that any trust claim about an agent that does not include a behavioral fingerprint — verified through continuous monitoring — is stale by default. The claim was accurate at evaluation time. It may not be accurate today.

Detection profile: Medium to high difficulty. Provider-side model updates are the hardest to detect because they require an active monitoring program (regular eval runs) rather than a reactive one (waiting for failures). Organizations without continuous evaluation infrastructure will typically discover provider-side model updates through production failures.

Rate of occurrence: High. Major providers update flagship models multiple times per quarter.

Remediation: Mandatory re-evaluation on behavioral fingerprint change; provider notification clauses in API contracts for enterprise customers; continuous eval monitoring.

Mechanism 2: Fine-Tuning and RLHF Updates

Reinforcement learning from human feedback (RLHF) and supervised fine-tuning are the primary mechanisms by which providers improve model behavior post-deployment. Safety teams conduct red-teaming exercises, identify problematic outputs, collect preference data, and run fine-tuning runs to steer the model away from failure modes.

These improvements are valuable. They also change behavior in ways that are difficult to predict from the outside. A fine-tuning run aimed at reducing harmful outputs may also reduce the model's willingness to engage with ambiguous but legitimate use cases. A run aimed at improving instruction-following may change the model's default formatting behavior. A run aimed at improving factual accuracy may reduce the model's confidence calibration.

The interaction effects are particularly problematic for agents with carefully calibrated system prompts. An agent operator may have spent weeks tuning a system prompt to produce a specific behavioral profile — a certain communication style, a certain level of conciseness, a certain approach to uncertainty expression. A fine-tuning update can shift those outputs significantly, making the calibrated system prompt produce unexpected behavior.

Case study: Legal research firm, Claude Sonnet, 2025

A legal research firm deployed an agent using Claude Sonnet for case law analysis. The agent was prompted to return exhaustive citation lists for any legal precedent query, including minority opinions, dissenting opinions, and subsequent treatment by later courts. The firm validated this behavior extensively during their procurement evaluation.

After a Claude Sonnet update focused on response quality improvements, the firm's research staff began reporting that citation completeness had decreased. The agent was returning the most prominent three to five citations rather than the exhaustive list. The system prompt had not changed. The queries had not changed. The model identifier had not changed. But the model's implicit interpretation of "exhaustive" had shifted.

The firm did not receive notification of the model update. They discovered the change through attorney complaints about missing citations. By the time the issue was diagnosed, hundreds of research queries had been processed with incomplete citation lists. The time and cost of identifying affected queries, re-running them, and reviewing the additional citations was significant. The reputational cost with the attorneys who had been receiving the incomplete output was harder to quantify.

Detection profile: Medium difficulty. RLHF-induced drift often shows up in eval suites focused on specific behavioral dimensions (completeness, verbosity, confidence calibration) rather than overall task performance. Generic accuracy benchmarks may not capture it.

Rate of occurrence: High. Safety-focused fine-tuning runs are continuous at major providers.

Remediation: Behavioral eval suites that test specific agent commitments (not just overall task performance); alerting on dimension-specific regression; explicit communication with providers about notification requirements.

Mechanism 3: Context Window Expansion

Context window expansion is typically announced as a capability improvement. Longer context windows allow agents to process more information per call, which seems unambiguously beneficial. What is less often acknowledged is that context window expansion changes the attention patterns of transformer-based models in ways that can materially affect output behavior.

When a model's context window expands from 8K to 16K tokens, and a prompt that previously filled 60% of the context window now fills only 30%, the model's attention allocation across the prompt changes. Information at the middle of a longer context window receives proportionally less attention than it did at the middle of a shorter one — the "lost in the middle" phenomenon documented by multiple research groups. Instructions placed at certain positions in the prompt may receive more or less attention relative to before.

For agents with structured system prompts — tool descriptions, persona definitions, behavioral constraints, output format specifications — this shift in attention allocation can change which instructions are followed most reliably. A behavioral constraint placed at position 1500 of a 4000-token prompt may be followed consistently. The same constraint at position 1500 of a 16000-token prompt, where it represents a smaller fraction of the total context, may be followed less reliably.

Practical impact: An agent that reliably honored scope constraints when its context window was 8K may begin violating those constraints at a low rate after a context window expansion to 128K — not because the constraint was removed, but because the model's attention weighting shifted in ways that reduced constraint compliance.

Detection profile: Low to medium difficulty. Context window expansion drift is particularly hard to detect because the failure mode is stochastic and low-rate — not every call violates the constraint, just a higher proportion than before. Requires statistical monitoring rather than deterministic testing.

Rate of occurrence: Medium. Context window expansions are less frequent than fine-tuning runs but are often accompanied by other weight changes.

Remediation: System prompt restructuring to place critical instructions at both beginning and end of context (recency bias is real and exploitable); eval suites specifically testing constraint compliance at different prompt positions; context utilization monitoring.

Mechanism 4: System Prompt Accumulation

This mechanism is operator-side rather than provider-side, which makes it more preventable but no less damaging. As agents evolve, system prompts grow. New tools are added, requiring tool descriptions. New behavioral requirements are added, requiring additional instructions. Exceptions are documented, edge cases are handled, and the system prompt grows from 500 tokens to 2000 tokens to 8000 tokens.

The problem is positional degradation. Early instructions in a long system prompt receive disproportionate attention from the model. Instructions added later, which are appended to the end of an already long prompt, may receive less attention than their importance warrants. Behavioral constraints that were reliably followed when they were 20% of a 1000-token prompt may be less reliably followed when they are 5% of a 4000-token prompt.

This drift is entirely preventable, but it requires discipline that most teams do not apply: treating system prompt structure as a first-class design concern, periodically auditing attention allocation across prompt sections, and running regression tests when prompt content is added.

Common failure pattern: An agent is initially constrained to a specific scope (e.g., "only assist with questions about product X, do not discuss competitors"). The constraint is placed in the system prompt at position 400 of 500 tokens — near the end, strongly attended. Over time, tool descriptions and other context grow the prompt to 3000 tokens. The constraint is now at position 400 of 3000 tokens — much earlier relative to the end, less strongly recency-biased. Competitor question compliance drops from near-100% to 70%, and nobody notices until a customer screenshots the agent discussing a competitor's pricing.

Detection profile: Easy to detect in eval suites, but teams often don't run evals when they make "small" prompt changes.

Rate of occurrence: High. Most deployed agents see system prompt growth over their operational lifetime.

Remediation: Structural system prompt templates with fixed positions for critical constraints; automated regression on every system prompt change; token budget governance (maximum tokens per section, enforced by build tooling).

Mechanism 5: Temperature and Sampling Parameter Drift

Inference infrastructure is not static. Providers continuously update their serving infrastructure, and infrastructure changes can affect the effective sampling behavior of models even when the declared temperature parameter is unchanged.

Known infrastructure-level sources of sampling variance:

• Floating-point precision changes: Moving from FP32 to BF16 to INT8 quantization changes the probability distribution over tokens at each position. The declared temperature remains the same; the effective temperature is different.
• Batching strategy changes: Continuous batching of multiple user requests can affect per-request effective temperature through sequence length interactions.
• Hardware changes: Different GPU generations produce different floating-point results for identical operations due to differences in fused multiply-add implementations and rounding behavior.
• KV cache strategy changes: Changes to key-value caching strategies can affect attention patterns for long contexts.

None of these changes are disclosed in API changelogs. They are infrastructure implementation details that providers consider internal. From the API consumer's perspective, the temperature parameter is a fixed knob — set temperature=0.3 and get stable, relatively deterministic outputs. The reality is more complex.

Practical impact: Agents that depend on relatively deterministic outputs (temperature ≤ 0.3) for structured data extraction, code generation, or classification tasks may begin producing more variable outputs after infrastructure upgrades at the provider. This is particularly problematic for agents whose quality guarantees depend on low output variance.

Detection profile: High difficulty. Temperature drift from infrastructure changes is nearly impossible to detect without statistical analysis of output distribution over time, compared to a behavioral baseline established when the agent was initially evaluated.

Rate of occurrence: Low to medium. Infrastructure changes at major providers are infrequent but impactful.

Remediation: Output entropy monitoring (measure the statistical variance of agent outputs on standardized canary prompts over time); alert on variance distribution shifts; for critical applications, run a deterministic post-processing layer that normalizes outputs regardless of upstream variance.

Mechanism 6: RAG Corpus Updates

Retrieval-augmented generation (RAG) architectures are widely used for knowledge-intensive agent applications: customer service, technical support, policy enforcement, research assistance. In a RAG system, the agent's outputs are conditioned on retrieved context — chunks of documents fetched from a vector database based on semantic similarity to the query.

RAG corpus updates are a common and often necessary source of behavioral drift. When the underlying knowledge base changes, the agent's answers should change to reflect the updated information. The problem is that RAG corpus updates often produce behavioral changes that are broader than the specific updated information, for several reasons:

• Semantic neighborhood disruption: Adding new documents to a vector index changes the semantic neighborhood around existing queries. A query that previously retrieved documents A, B, C may now retrieve B, D, E — including a document that wasn't previously top-ranked and that shifts the agent's answer in a direction that wasn't explicitly intended by the corpus update.

• Chunking boundary effects: If documents are re-indexed with different chunking parameters (different chunk size, different overlap), previously retrieved chunk boundaries may now fall differently, including or excluding context that was previously present in the retrieved window.

• Index refresh lag: If the vector index is rebuilt on a schedule rather than updated incrementally, there may be a period of inconsistency where some documents are indexed at an older revision and others at a newer one.

Case study: E-commerce customer service agent, return policy drift

An e-commerce company deployed a customer service agent backed by a RAG corpus containing product documentation, policy documents, and FAQ content. The agent accurately answered questions about the company's 30-day return policy for electronics.

The policy team updated the return policy document to introduce a new 15-day return window for opened software products, while maintaining the 30-day window for hardware. The corpus was updated with the new policy document. The intent was a narrow change affecting software returns.

Due to semantic neighborhood disruption in the vector index, queries about electronics returns that had previously retrieved the general 30-day policy document began sometimes retrieving the software-specific 15-day document instead. The agent began giving ambiguous or incorrect answers about hardware return windows — the exact behavior the policy team had not intended to change.

The drift was discovered when a customer service manager reviewing agent transcripts noticed the inconsistency. Approximately 2,000 return-related customer interactions had occurred between the corpus update and the discovery. Of these, an unknown fraction had received inaccurate policy information. The company's legal team was involved in assessing exposure.

Detection profile: Medium difficulty. RAG corpus drift can be detected with eval suites that include canary questions with known correct answers based on specific corpus content. The eval suite must be updated when intentional corpus changes are made.

Rate of occurrence: High for any RAG-based agent. Corpus updates are routine and frequent.

Remediation: Canary question eval suites tied to corpus content; mandatory eval run on every corpus update; semantic drift monitoring (compare answer embedding distributions before and after corpus updates); explicit versioning of the RAG corpus with behavioral fingerprint pinned to corpus version.

Mechanism 7: Tool Availability Changes

Tool-using agents (also called function-calling agents or ReAct agents) make action selection decisions based on the set of available tools. When the tool registry changes — new tools added, existing tools modified, tools deprecated — the agent's action preferences shift, even without any change to the base model or system prompt.

This is a subtle but important mechanism. Large language models trained on tool use learn to prefer certain tools for certain tasks based on their training distribution. When a new, more specialized tool is added to the registry, the model may prefer it for tasks where a less specialized tool previously served adequately. This is usually desirable — the model uses the better tool — but it can also produce unexpected behavior when the new tool has different failure modes, different latency characteristics, or different output formats than the tool it displaces in the model's preference ordering.

Common failure pattern: An agent has a general-purpose search_knowledge_base tool and a specific query_crm tool. For customer-related queries, the agent uses search_knowledge_base and returns good results. A new get_customer_account tool is added for a different use case. The model begins routing some customer queries to get_customer_account instead of search_knowledge_base — not because it was instructed to, but because the new tool's description better matches the query semantics. The get_customer_account tool has different access controls and returns structured data rather than natural language summaries, breaking the agent's output formatting.

Modifying an existing tool's description has similar effects. Changing a tool description from "retrieves product information" to "retrieves comprehensive product information including pricing, availability, and specifications" may cause the model to invoke that tool for queries it previously handled without tool use, increasing latency and cost.

Detection profile: Easy to detect with targeted eval suites that test tool selection behavior on standardized queries. Difficult to detect without evals, because tool selection changes are invisible at the API level — the final output may look correct even when the tool selection path changed.

Rate of occurrence: Medium to high. Tool registries evolve with product development, and teams often underestimate the behavioral impact of tool description changes.

Remediation: Tool selection eval suites that verify expected tool invocation paths on standardized queries; behavioral regression tests on tool description changes; tool registry versioning; change management process for tool modifications.

---

Part III: A Complete Drift Taxonomy

Not all behavioral drift is the same kind of problem. A drift taxonomy helps teams prioritize detection, triage incidents, and communicate with stakeholders about the nature and severity of observed changes.

Primary Drift Categories

Detailed Category Descriptions

Policy drift is the most severe category because it represents a breakdown in behavioral constraints. An agent that previously refused to discuss competitor products and now discusses them freely has experienced policy drift. An agent that previously required user confirmation before executing irreversible actions and now executes without confirmation has experienced policy drift. Policy drift directly undermines the contractual and regulatory foundations of agent deployment.

Capability drift is a regression in task performance. The agent used to correctly extract structured data from invoices 97% of the time; now it does so 82% of the time. This is often caused by fine-tuning runs that improve behavior on some dimensions while degrading it on others — the classic capability tradeoff problem in RLHF.

Behavioral drift — the broadest category — covers changes in how the agent accomplishes tasks without necessarily changing whether it accomplishes them. Response length, explanation depth, uncertainty expression, communication style, reasoning visibility, and output formatting all fall into this category. Behavioral drift may not affect downstream machine-readable outputs but can significantly affect end-user experience and trust.

Latency drift is often overlooked as a trust concern but is highly material in production deployments. An agent whose median response time increases from 1.2s to 3.8s has drifted in a way that affects user experience, SLA compliance, and potentially business outcomes. Latency drift can result from model updates that increase token output, infrastructure changes that affect serving efficiency, or prompt changes that increase context processing time.

Cost drift is economically significant. Token pricing means that cost is proportional to token consumption. An agent whose average output length increases by 40% after a model update is 40% more expensive to operate. Multiply this across millions of daily calls and the financial impact is substantial. Cost drift is often the first type of drift detected by operations teams because it shows up in billing dashboards, even when behavioral changes are not yet noticed.

Confidence drift is particularly dangerous for high-stakes applications. Many agent architectures route decisions based on the agent's expressed confidence: high-confidence answers are routed directly to users; low-confidence answers are escalated to human review. If the model's confidence calibration shifts — if it begins expressing high confidence in answers that are actually wrong — the routing logic fails silently, flooding users with confidently wrong answers while the human review queue sits underutilized.

Scope drift — like policy drift — represents a breakdown in boundary enforcement. An agent that starts attempting to take actions outside its defined scope may be experiencing tool availability drift (new tools are available that it finds compelling to use), system prompt accumulation drift (scope constraints have been pushed further back in a growing prompt), or model update drift (the model has become more proactive about tool use generally).

---

Part IV: Detection Methods — Technical Implementation

Detecting behavioral drift requires an active monitoring program. Passive observation — waiting for failures to be reported — will always lag behind the actual onset of drift, sometimes by weeks. The following five detection methods provide a layered defense that catches different types of drift at different stages.

Method 1: Behavioral Fingerprinting

Behavioral fingerprinting is the foundational detection method. It establishes a quantitative baseline of agent behavior at a known-good point in time and provides a continuous comparison mechanism to detect deviations.

Implementation:

A behavioral fingerprint consists of three components:

1. Output embedding distribution: Run a standardized set of 50–200 canary prompts through the agent and embed each output using a text embedding model. Compute the centroid of the resulting embedding cloud. This centroid represents the agent's "average semantic position" in output space.

2. Response length distribution: For the same canary prompts, record output length in tokens. Compute the distribution (mean, median, p90, p99). This captures verbosity changes that may not be visible in embedding space.

3. Structured output compliance rate: For canary prompts that expect structured outputs (JSON, specific formatting, specific sections), compute the compliance rate. This captures formatting drift.

import { EmbeddingClient } from '@armalo/eval-engine';
import { computeCosineSimilarity } from '@armalo/scoring';

interface BehavioralFingerprint {
  embeddingCentroid: number[];
  lengthDistribution: {
    mean: number;
    median: number;
    p90: number;
    p99: number;
  };
  structuredOutputCompliance: number;
  computedAt: string;
  canaryPromptSetVersion: string;
}

async function computeBehavioralFingerprint(
  agentEndpoint: string,
  canaryPrompts: CanaryPrompt[],
  embedder: EmbeddingClient
): Promise<BehavioralFingerprint> {
  const outputs = await Promise.all(
    canaryPrompts.map(p => runAgentWithRetry(agentEndpoint, p.prompt))
  );

  const embeddings = await embedder.embedBatch(outputs.map(o => o.text));
  const centroid = computeEmbeddingCentroid(embeddings);

  const lengths = outputs.map(o => o.tokenCount);
  const lengthDist = computeDistribution(lengths);

  const structuredCompliance = canaryPrompts
.filter(p => p.expectedStructure)
.reduce((acc, p, i) => {
      const complies = validateStructure(outputs[i].text, p.expectedStructure!);
      return acc + (complies? 1 : 0);
    }, 0) / canaryPrompts.filter(p => p.expectedStructure).length;

  return {
    embeddingCentroid: centroid,
    lengthDistribution: lengthDist,
    structuredOutputCompliance: structuredCompliance,
    computedAt: new Date().toISOString(),
    canaryPromptSetVersion: canaryPrompts[0].setVersion,
  };
}

async function detectBehavioralDrift(
  baseline: BehavioralFingerprint,
  current: BehavioralFingerprint,
  thresholds: DriftThresholds
): Promise<DriftReport> {
  const semanticDrift = 1 - computeCosineSimilarity(
    baseline.embeddingCentroid,
    current.embeddingCentroid
  );

  const lengthDriftRatio = Math.abs(
    current.lengthDistribution.mean - baseline.lengthDistribution.mean
  ) / baseline.lengthDistribution.mean;

  const complianceDelta = 
    baseline.structuredOutputCompliance - current.structuredOutputCompliance;

  const alerts: DriftAlert[] = [];

  if (semanticDrift > thresholds.semanticDrift) {
    alerts.push({
      type: 'SEMANTIC_DRIFT',
      severity: semanticDrift > thresholds.semanticDrift * 2? 'CRITICAL' : 'WARNING',
      value: semanticDrift,
      threshold: thresholds.semanticDrift,
    });
  }

  if (lengthDriftRatio > thresholds.lengthDrift) {
    alerts.push({
      type: 'LENGTH_DRIFT',
      severity: lengthDriftRatio > 0.5? 'CRITICAL' : 'WARNING',
      value: lengthDriftRatio,
      threshold: thresholds.lengthDrift,
    });
  }

  if (complianceDelta > thresholds.complianceDelta) {
    alerts.push({
      type: 'COMPLIANCE_DRIFT',
      severity: complianceDelta > 0.2? 'CRITICAL' : 'WARNING',
      value: complianceDelta,
      threshold: thresholds.complianceDelta,
    });
  }

  return {
    baselineTimestamp: baseline.computedAt,
    currentTimestamp: current.computedAt,
    semanticDrift,
    lengthDriftRatio,
    complianceDelta,
    alerts,
    driftDetected: alerts.length > 0,
  };
}

Recommended thresholds:

• Semantic drift (cosine distance): warning at 0.05, critical at 0.10
• Length drift: warning at 20%, critical at 40%
• Structured output compliance: warning at 5% degradation, critical at 15%

Cadence: Run behavioral fingerprinting weekly in stable periods; daily when a model update is suspected; immediately after any system prompt change, tool addition, or RAG corpus update.

Method 2: Eval Regression Suites

Eval regression suites are the second layer of defense, providing higher-resolution detection of specific behavioral dimensions. Where behavioral fingerprinting gives you a holistic "has something changed" signal, eval regression suites tell you specifically what changed and by how much.

Structure of an eval regression suite:

A complete eval regression suite for an agent with behavioral pacts should include:

• Task performance evals: Accuracy on representative samples of the agent's core task, measured against a held-out gold standard. Target: <5% regression tolerated before alert.

• Constraint compliance evals: Prompts specifically designed to test each stated behavioral constraint. Scope limitations, content restrictions, format requirements. Target: 0% regression on any critical constraint.

• Edge case handling evals: Prompts representing known-difficult cases that were explicitly tested during initial deployment. A regression here suggests that fine-tuning may have changed the model's handling of specific input types.

• Output format evals: Prompts with explicit format requirements (JSON structure, specific section headers, length constraints). Target: <2% regression on format compliance.

• Confidence calibration evals: Prompts with known-correct and known-incorrect answers. Measure whether the model's expressed confidence correlates with actual accuracy as expected.

interface EvalSuite {
  id: string;
  agentId: string;
  prompts: EvalPrompt[];
  dimensions: EvalDimension[];
  baselineResults: EvalResults;
  regressionThresholds: Record<string, number>;
}

interface EvalDimension {
  name: string;
  description: string;
  scorer: (output: string, expected: string, metadata: Record<string, unknown>) => number;
  weight: number;
  criticalThreshold: number;  // Fail if below this
  warningThreshold: number;   // Warn if below this
}

async function runEvalRegression(
  suite: EvalSuite,
  agentEndpoint: string
): Promise<EvalRegressionReport> {
  const currentResults = await runEvalSuite(suite, agentEndpoint);
  const regressions: EvalRegression[] = [];

  for (const dimension of suite.dimensions) {
    const baselineScore = suite.baselineResults.dimensionScores[dimension.name];
    const currentScore = currentResults.dimensionScores[dimension.name];
    const regression = baselineScore - currentScore;

    if (currentScore < dimension.criticalThreshold) {
      regressions.push({
        dimension: dimension.name,
        severity: 'CRITICAL',
        baselineScore,
        currentScore,
        regression,
        message: `${dimension.name} dropped below critical threshold: ${currentScore.toFixed(3)} < ${dimension.criticalThreshold}`,
      });
    } else if (regression > suite.regressionThresholds[dimension.name]) {
      regressions.push({
        dimension: dimension.name,
        severity: 'WARNING',
        baselineScore,
        currentScore,
        regression,
        message: `${dimension.name} regressed by ${(regression * 100).toFixed(1)}% from baseline`,
      });
    }
  }

  return {
    suiteId: suite.id,
    agentId: suite.agentId,
    timestamp: new Date().toISOString(),
    baselineTimestamp: suite.baselineResults.timestamp,
    regressions,
    passed: regressions.filter(r => r.severity === 'CRITICAL').length === 0,
    dimensionScores: currentResults.dimensionScores,
  };
}

Integration with deployment pipeline:

Eval regression suites should be integrated as gates in the agent deployment pipeline:

1. On any change to the agent (system prompt, tools, model version, RAG corpus), run the full eval suite before traffic is shifted.
2. If critical thresholds are violated, block deployment and alert the agent owner.
3. If warning thresholds are triggered, allow deployment with a required acknowledgment and documentation of the expected regression.
4. Store eval results in the agent's behavioral history, linked to the specific deployment version.

Method 3: Shadow Comparison

Shadow comparison is the production-facing complement to eval suites. While eval suites run on synthetic canary prompts, shadow comparison routes a sample of real production traffic to two versions simultaneously and compares their outputs.

Implementation architecture:

Production Traffic
       │
       ▼
┌──────────────┐
│  Split Layer  │
│  (5% shadow)  │
└──────┬───────┘
       │
  ┌────┴────┐
  │         │
  ▼         ▼
[Primary]  [Shadow]
 (v_current) (v_previous)
  │         │
  └────┬────┘
       │
       ▼
 ┌───────────┐
 │ Comparator │
 │   Store    │
 └───────────┘

For each shadowed request:

1. Route the request to both the primary and shadow agent instances.
2. Return the primary response to the user (shadow is transparent to users).
3. Compare primary and shadow outputs using semantic similarity, length ratio, and structured output compliance.
4. Log the comparison results and alert when divergence rate exceeds threshold.

Recommended thresholds:

• Warning: >2% of shadowed requests show semantic similarity below 0.85
• Critical: >5% of shadowed requests show semantic similarity below 0.70
• Cost drift alert: >15% difference in average output length between primary and shadow

Operational considerations:

• Shadow routing approximately doubles the API cost for shadowed traffic. Budget accordingly — 5% shadow rate means 5% additional API cost.
• Store only comparison metadata (similarity scores, length deltas), not full output text, to manage storage costs.
• Shadow comparison is most valuable in the 30 days immediately following a model version update or major system prompt change.

Method 4: Semantic Drift Detection

Semantic drift detection provides a longitudinal view of behavioral change over extended time periods. Rather than comparing against a fixed baseline, it tracks the evolution of the agent's output distribution over time, identifying gradual drift that might not trigger point-in-time comparison alerts.

Implementation:

Weekly, embed a random sample of 500 production outputs using a stable embedding model (use a pinned version that doesn't change, such as text-embedding-3-small with a fixed revision). Store the embedding centroids in a time series. Monitor the following metrics:

1. Centroid velocity: The cosine distance between consecutive weekly centroids. Consistent velocity indicates gradual drift in a specific direction — potentially fine-tuning-induced behavioral evolution.

2. Variance evolution: The average pairwise distance within the weekly embedding cloud. Increasing variance indicates the agent's outputs are becoming less consistent. Decreasing variance may indicate the model has become more deterministic (often an RLHF artifact).

3. Topic distribution shift: Cluster the embedding cloud weekly using k-means (k=20). Track the relative size of clusters over time. A growing cluster may indicate new topics the agent is being asked about; a shrinking cluster may indicate the agent's coverage of certain topics is degrading.

import numpy as np
from scipy.spatial.distance import cosine
from sklearn.cluster import KMeans
from datetime import datetime, timedelta

def compute_weekly_drift_metrics(
    current_week_embeddings: np.ndarray,
    previous_week_embeddings: np.ndarray,
    k_clusters: int = 20
) -> dict:
    current_centroid = np.mean(current_week_embeddings, axis=0)
    previous_centroid = np.mean(previous_week_embeddings, axis=0)
    
    centroid_velocity = cosine(current_centroid, previous_centroid)
    
    current_variance = np.mean([
        cosine(emb, current_centroid) 
        for emb in current_week_embeddings
    ])
    previous_variance = np.mean([
        cosine(emb, previous_centroid) 
        for emb in previous_week_embeddings
    ])
    
    variance_delta = current_variance - previous_variance
    
    combined = np.vstack([current_week_embeddings, previous_week_embeddings])
    kmeans = KMeans(n_clusters=k_clusters, random_state=42)
    labels = kmeans.fit_predict(combined)
    
    n_current = len(current_week_embeddings)
    current_distribution = np.bincount(
        labels[:n_current], minlength=k_clusters
    ) / n_current
    previous_distribution = np.bincount(
        labels[n_current:], minlength=k_clusters
    ) / len(previous_week_embeddings)
    
    distribution_shift = np.sum(np.abs(current_distribution - previous_distribution)) / 2
    
    return {
        'centroid_velocity': float(centroid_velocity),
        'variance_delta': float(variance_delta),
        'distribution_shift': float(distribution_shift),
        'alert_centroid': centroid_velocity > 0.05,
        'alert_variance': abs(variance_delta) > 0.02,
        'alert_distribution': distribution_shift > 0.15,
    }

Alert thresholds:

• Centroid velocity: warning at 0.05, critical at 0.10 (week-over-week cosine distance)
• Variance delta: warning at ±0.02, critical at ±0.05
• Distribution shift: warning at 0.15, critical at 0.30 (Jensen-Shannon divergence equivalent)

Method 5: Pact Compliance Monitoring

For agents with formal behavioral pacts — documented commitments about specific behavioral properties — pact compliance monitoring is the most direct and actionable detection method. Rather than inferring drift from statistical signals, pact compliance monitoring tests the specific commitments the agent has made.

Pact compliance monitoring architecture:

A behavioral pact defines a set of eval checks that must pass. Each check has a defined pass threshold (e.g., "scope compliance rate ≥ 98%", "citation completeness ≥ 95%", "structured output compliance ≥ 99%"). The pact compliance monitor runs these checks on a defined schedule and flags any that fall below threshold as pact violations.

interface PactCheck {
  id: string;
  dimension: string;
  description: string;
  evalPrompts: EvalPrompt[];
  passThreshold: number;
  criticalBelow: number;
}

interface BehavioralPact {
  id: string;
  agentId: string;
  version: string;
  checks: PactCheck[];
  effectiveDate: string;
  behaviorHash: string;
  attestationCid: string;
}

async function runPactComplianceMonitor(
  pact: BehavioralPact,
  agentEndpoint: string,
  db: Database
): Promise<PactComplianceResult> {
  const checkResults = await Promise.all(
    pact.checks.map(async (check) => {
      const outputs = await runEvalPrompts(agentEndpoint, check.evalPrompts);
      const score = computeCheckScore(outputs, check);
      const passed = score >= check.passThreshold;
      const critical = score < check.criticalBelow;
      return { check, score, passed, critical };
    })
  );

  const violations = checkResults.filter(r =>!r.passed);
  const criticalViolations = checkResults.filter(r => r.critical);

  // Record to agent trust history
  await db.insert(pact_compliance_records).values({
    pactId: pact.id,
    agentId: pact.agentId,
    runAt: new Date(),
    checkResults: checkResults.map(r => ({
      checkId: r.check.id,
      score: r.score,
      passed: r.passed,
      critical: r.critical,
    })),
    overallPassed: violations.length === 0,
    criticalViolations: criticalViolations.length,
  });

  // Update agent trust score if pact violations detected
  if (violations.length > 0) {
    await updateAgentTrustScore(pact.agentId, {
      pactViolations: violations.length,
      criticalViolations: criticalViolations.length,
      dimension: 'reliability',
    });
  }

  return {
    pactId: pact.id,
    passed: violations.length === 0,
    violations,
    criticalViolations,
    checkResults,
  };
}

Integration with trust scoring:

Pact compliance monitoring results feed directly into the agent's composite trust score. The Armalo reliability dimension (13% of composite score) is partially computed from pact compliance history. An agent with a consistent pact compliance record earns reliability credit. An agent with pact violations, even if later remediated, carries a reliability discount proportional to the severity and recency of the violation.

This creates the right incentive structure: agent operators who invest in continuous pact compliance monitoring and remediation build durable trust; operators who allow drift to accumulate until it triggers a failure see that failure reflected in their agent's commercial standing.

---

Part V: Organizational Impact Matrix

Behavioral drift is not just a technical problem. Its organizational consequences touch engineering, product, legal, compliance, and finance. Understanding the full impact matrix helps teams allocate appropriate resources to drift detection and prevention.

Engineering Impact

Downstream pipeline failures are the most visible engineering consequence of behavioral drift. Most deployed agents are embedded in pipelines: their outputs feed other systems, trigger downstream actions, or serve as inputs to human review workflows. When output format, length, or structure changes unexpectedly, parsers fail, validation layers reject outputs, and human reviewers encounter unfamiliar formats.

The failure mode is typically silent until a threshold is crossed. A parser that handled GPT-4-turbo's verbose outputs gracefully may start silently dropping fields when output length decreases — producing null values in downstream databases rather than explicit errors. By the time the null values accumulate enough to trigger a dashboard alert, the pipeline has processed millions of records with corrupted data.

Integration stability — the reliability of agent APIs as predictable integration surfaces — is undermined by behavioral drift. API stability is typically thought of in terms of schema stability (same endpoints, same request/response structure). Behavioral drift adds a second dimension of instability: even with a stable schema, the semantic content of responses changes. Integration tests that verify schema compliance will pass; integration tests that verify semantic correctness may fail.

Product Impact

User trust erosion is the product consequence that is hardest to measure but most consequential. Users form mental models of how an agent behaves. When behavior changes without notice, the mental model is violated. The reaction — conscious or not — is a reduction in trust. Users become less willing to act on agent recommendations without verifying them. Confidence in the agent as a reliable tool decreases.

Research on human-AI interaction consistently finds that trust in AI systems is fragile and asymmetric: it is built slowly through consistent positive experiences and eroded quickly through unexpected failures or changes. A behavioral drift event — even if the drift is technically an improvement — can reset trust that took months to build.

Feature reliability — the confidence that a feature built on an agent will continue to work as described — is undermined by undisclosed behavioral drift. Product managers who commit to user-facing features based on observed agent behavior need to know that the behavior will be stable over the feature's lifetime. Behavioral drift that changes the agent's output format or content can require product changes that were not planned or resourced.

Legal and Contractual Impact

Contractual liability is an increasingly important concern as agents are deployed in commercial relationships. When a buyer commits to using an agent based on demonstrated behavior — and that behavior changes — the contractual basis for the commitment may be undermined.

Consider: A company contracts to use an agent for automated contract review, with an SLA specifying that the agent will identify all force majeure clauses in provided contracts with ≥97% recall. The agent is tested and found to meet this threshold. After a model update, the agent's recall on force majeure clauses drops to 89%. The SLA is now violated, but the agent operator may not know it — and the buyer may not know it until a force majeure clause is missed in a consequential contract.

This is not a hypothetical scenario. It is an emerging class of AI-related commercial dispute that legal teams at major technology companies are beginning to encounter. The legal exposure is significant because:

1. The SLA was based on a specific behavioral state that is no longer in existence.
2. The operator may not have informed the buyer of the model update that changed behavior.
3. The operator may not have known about the model update themselves (provider-side drift).
4. There may be no contractual language addressing model update disclosure obligations.

Regulatory exposure adds another dimension. The EU AI Act (Articles 9 and 72) requires operators of high-risk AI systems to implement a quality management system including "continuous monitoring" of AI system performance and behavior. The specific language of Article 9(1)(f) requires: "a system for achieving the continuous monitoring of the operation of the high-risk AI system, including a log of automatically generated events relevant to the identification of risks." Behavioral drift without monitoring and logging may constitute a violation of this requirement for covered use cases.

In the United States, emerging guidance from the FTC on AI usage practices and from sector-specific regulators (banking, healthcare, insurance) increasingly incorporates requirements for monitoring AI system behavior over time. Organizations that cannot demonstrate continuous behavioral monitoring may face regulatory scrutiny if an AI-related incident occurs.

Financial Impact

Direct operational cost is the most quantifiable financial impact of behavioral drift. Token cost drift — when model updates increase output length — translates directly to increased API expenditure. An agent processing one million queries per day with average output length increasing from 300 to 420 tokens represents a 40% cost increase on output tokens. At $15/million output tokens, this is an additional $630 per day — $230K annualized — from a single undisclosed model update.

Remediation cost is the indirect financial impact: the engineering time required to diagnose the drift, retest the affected systems, update affected pipelines, and communicate with affected stakeholders. Based on incidents reported in enterprise AI user forums, a significant behavioral drift event requiring full root cause analysis and pipeline remediation typically requires 5–15 days of engineering effort across multiple teams.

Trust cost — the revenue impact of eroded buyer trust — is the hardest to quantify but potentially the largest. If behavioral drift causes a buyer to downgrade or discontinue a commercial relationship with an AI agent service, the lost revenue can far exceed the direct remediation cost.

---

Part VI: Cryptographic Attestation as the Solution

The preceding sections establish that behavioral drift is real, consequential, and often invisible. The solution is not more vigilance — vigilance is not a system. The solution is cryptographic attestation: making behavioral claims verifiable rather than merely asserted.

The Attestation Architecture

A behavioral attestation has four components:

1. Model weight hash: A SHA-256 hash of the model weights at quantized precision, published to an immutable log. This provides a cryptographic anchor for the specific weight snapshot used at evaluation time. For provider-hosted models where weight access is not available, a proxy — the hash of a standardized benchmark output set — serves the same function.

2. Behavioral commit: A hash of the eval suite results: the specific prompts used, the outputs produced, and the computed dimension scores. This is distinct from the weight hash because what matters for trust is behavior, not weights. Two models with different weights may have identical behavioral commits (unlikely but theoretically possible); a single model may have different behavioral commits at different points in time.

interface BehavioralCommit {
  agentId: string;
  modelIdentifier: string;
  systemPromptHash: string;  // SHA-256 of system prompt
  toolRegistryHash: string;  // SHA-256 of tool descriptions + schemas
  evalSuiteVersion: string;
  evalResults: {
    dimension: string;
    score: number;
    checkCount: number;
    passedCount: number;
  }[];
  overallScore: number;
  computedAt: string;
}

function computeBehavioralCommitHash(commit: BehavioralCommit): string {
  const canonical = JSON.stringify(commit, Object.keys(commit).sort());
  return sha256(canonical);
}

3. EAS attestation on Base L2: The behavioral commit hash is submitted as an EAS (Ethereum Attestation Service) attestation on Base L2. This provides:

• Immutability: Once attested, the behavioral commit hash cannot be retroactively modified.
• Timestamp: The blockchain timestamp provides a verifiable record of when the behavioral commit was made.
• Verifier identity: The attester's address is cryptographically bound to the attestation. If Armalo attests a behavioral commit, the attestation carries Armalo's cryptographic identity as the verifier.
• Queryability: Any party can query the attestation by agent ID and time range to retrieve the behavioral commit history.

import { EAS, SchemaEncoder } from '@ethereum-attestation-service/eas-sdk';
import { ethers } from 'ethers';

const BEHAVIORAL_ATTESTATION_SCHEMA = 
  'address agentId, bytes32 behaviorHash, string evalSuiteVersion, ' +
  'uint256 overallScore, bytes32 systemPromptHash';

async function attestBehavioralCommit(
  commit: BehavioralCommit,
  behaviorHash: string,
  eas: EAS,
  schemaUid: string
): Promise<string> {
  const schemaEncoder = new SchemaEncoder(BEHAVIORAL_ATTESTATION_SCHEMA);
  
  const encodedData = schemaEncoder.encodeData([
    { name: 'agentId', value: commit.agentId, type: 'address' },
    { name: 'behaviorHash', value: behaviorHash, type: 'bytes32' },
    { name: 'evalSuiteVersion', value: commit.evalSuiteVersion, type: 'string' },
    { name: 'overallScore', value: Math.round(commit.overallScore * 10000), type: 'uint256' },
    { name: 'systemPromptHash', value: commit.systemPromptHash, type: 'bytes32' },
  ]);

  const tx = await eas.attest({
    schema: schemaUid,
    data: {
      recipient: commit.agentId,
      expirationTime: BigInt(0),  // Non-expiring
      revocable: true,            // Revocable if agent is decommissioned
      data: encodedData,
    },
  });

  const attestationUid = await tx.wait();
  return attestationUid;
}

4. Verifier protocol: A buyer seeking to verify an agent's behavioral claims queries the Armalo trust oracle:

GET /api/v1/trust/verify-behavior
?agentId=<agent-id>
&date=2026-01-15T00:00:00Z
&dimension=scope_compliance

The trust oracle returns:

• The behavioral commit hash for the agent at the specified date
• The attestation UID on Base L2
• The dimension score and pass/fail status
• A signed verification receipt that can be used as evidence in contractual disputes

This creates a complete chain of trust: from the agent's operational behavior, through a standardized eval suite, to a cryptographic commitment, to an immutable on-chain attestation, to a queryable verification API.

Behavioral Hash Continuity

One non-obvious design challenge in behavioral attestation is continuity: what happens when intentional, desirable changes occur? An agent may legitimately upgrade its capabilities, expand its scope, or improve its performance. These changes should produce a new behavioral commit with a new hash — but the continuity of the agent's identity and trust record should be preserved.

The solution is a behavioral commit chain: each new behavioral commit references the previous commit hash as its parent, similar to a Git commit chain. This creates an auditable history of behavioral evolution. A buyer can query the agent's behavioral commit chain to see not just the current behavior, but the full history of how behavior has changed over time.

Drift is distinguished from deliberate evolution by three properties:

1. Disclosure: Deliberate evolution comes with a changelog entry. Drift does not.
2. Evaluation: Deliberate evolution is accompanied by a new eval run and attestation. Drift occurs between evaluations.
3. Timing: Deliberate evolution is initiated by the agent operator. Drift is initiated by external changes (provider updates, RAG updates) outside the operator's direct control.

The behavioral commit chain makes all three distinctions auditable. A buyer can verify that a behavioral change was disclosed, evaluated, and attested — or identify that it was not.

---

Part VII: Armalo's Approach to Behavioral Continuity

Behavioral Pacts as Immutable Baselines

The Armalo behavioral pact is the foundational instrument for managing behavioral continuity. A pact is a structured document that defines:

1. Behavioral commitments: Specific, measurable claims about the agent's behavior ("scope compliance ≥ 98%", "response latency p99 ≤ 4s", "structured output compliance ≥ 99.5%").

2. Eval suite binding: Each behavioral commitment is linked to a specific eval suite that operationalizes the claim. The pact specifies the eval suite version, the pass threshold, and the evaluation frequency.

3. Version anchoring: The pact is anchored to a specific behavioral commit hash. The pact is valid only while the agent's behavioral fingerprint matches the committed hash. A behavioral change that shifts the fingerprint beyond the drift tolerance threshold invalidates the pact until re-evaluation is completed.

4. Recertification triggers: Explicit conditions that require re-evaluation: model version change, system prompt change exceeding token threshold, tool registry change, RAG corpus update.

This structure means that behavioral drift is not just detectable — it is contractually significant. When an agent's behavior drifts beyond the pact's tolerance, the pact enters a "pending recertification" state. The agent's trust score reflects this state. Buyers who query the trust oracle see that the agent's behavioral claims are pending verification. Commercial activity (new escrow commitments, new deal terms) may be restricted until recertification is complete.

Trust Score Impact of Behavioral Drift

The Armalo composite trust score has 12 dimensions. Behavioral drift events affect three of them:

Reliability (13% weight): Pact compliance history directly affects reliability score. Each pact compliance check that passes contributes positive reliability signal. Each violation — including drift-induced violations — contributes negative signal. The recency of violations is weighted: recent violations have higher impact than older ones, with exponential decay.

Scope honesty (7% weight): Scope drift events — instances where the agent was observed operating outside its declared scope — directly impact the scope honesty dimension. Scope drift detected through behavioral fingerprinting or eval regression is recorded as a scope honesty event.

Self-audit / Metacal™ (9% weight): Agents that proactively report their own behavioral changes — by running self-evaluation and submitting updated behavioral commits before issues are detected externally — earn positive Metacal™ credit. Agents that allow drift to accumulate until detected externally receive a Metacal™ penalty. This creates an incentive for agent operators to invest in continuous self-monitoring.

Version-Linked Pact Re-evaluation

Armalo implements automatic pact re-evaluation triggers for events that are likely to cause behavioral drift:

1. Model version change: When an agent updates its model identifier (e.g., from claude-3-5-sonnet-20241022 to claude-3-5-sonnet-20250219), all active pacts are automatically flagged for re-evaluation. The agent continues operating, but its trust score includes a "pending re-evaluation" indicator until the eval suite is re-run.

2. System prompt change: When the system prompt changes, a hash comparison detects the change and triggers re-evaluation. For minor changes (token count delta < 50 tokens), a lightweight behavioral fingerprint comparison may substitute for a full eval run. For major changes, full eval re-run is required.

3. Tool registry change: New tools, modified tool descriptions, or removed tools all trigger behavioral re-evaluation.

4. RAG corpus update: For RAG-backed agents, a corpus version increment triggers re-evaluation of the pact checks relevant to knowledge accuracy.

This automatic trigger system reduces the risk of agents operating under stale pact certifications. The operational cost — running eval suites on change events — is bounded by the change frequency. Most production agents change infrequently enough that the eval overhead is modest.

---

Part VIII: The Remediation Playbook

When behavioral drift is detected, the response must be structured and time-bounded. Here is the complete remediation playbook.

Step 1: Detect (Target: Same Day)

Behavioral drift detection ideally comes from the monitoring infrastructure described in Part IV, not from production failures. The detection signal should specify:

• What changed: Which behavioral dimensions are affected (length, semantic content, structured output compliance, pact check scores)
• By how much: Quantitative measure of the change (drift magnitude)
• When it started: Timestamp of the first detected deviation from baseline
• Confidence level: Is this a statistically significant deviation or within normal variance?

If the detection comes from a production failure rather than monitoring, the first step is reconstruction: build a timeline of when the drift likely started by running behavioral fingerprinting on historical output samples.

Step 2: Quantify (Target: 24 Hours)

Run the full eval regression suite against the current agent. Measure every dimension, not just the ones that triggered the initial alert. Drift in one dimension often co-occurs with drift in related dimensions that hasn't yet crossed the alert threshold.

Produce a quantitative drift report:

The quantification step produces the evidence base for all subsequent decisions.

Step 3: Classify (Target: 24 Hours)

Using the drift taxonomy from Part III, classify the observed drift:

• Is this capability drift, behavioral drift, policy drift, or some combination?
• What is the likely mechanism? (Use the seven mechanisms from Part II as a checklist)
• What is the severity? (Does any dimension fall below a critical threshold or violate a pact commitment?)
• What is the impact scope? (Which downstream systems, user workflows, or contractual commitments are affected?)

Classification determines the urgency and escalation path. Policy drift (the agent is violating constraints) is always a P0 incident. Capability drift affecting core task performance is P1. Behavioral drift (style/format changes) without policy violations is P2 in most contexts.

Step 4: Decide (Target: 48 Hours)

Three decision paths are available:

Accept drift + update pact: If the drift represents an improvement or a change that the operator is willing to commit to going forward, run a full eval suite, produce a new behavioral commit, publish an updated pact with the new behavioral baselines, and create a changelog entry describing the change.

Reject drift + rollback: If the drift represents a regression that is not acceptable, roll back to the previous agent configuration. For provider-side model updates that cannot be rolled back, this may require pinning to an explicit model version (if the provider offers version pinning), switching to a different provider, or temporarily degrading the agent's service level while a remediating system prompt change is developed.

Investigate + monitor: If the drift is ambiguous — within the range of normal variance on some dimensions, outside it on others — extend the monitoring window, collect more data, and defer the decision. This is appropriate for low-severity drifts where the cost of a premature decision exceeds the cost of continued monitoring.

Step 5: Communicate (Target: 48–72 Hours)

All detected behavioral drift events should produce structured communication:

Agent changelog: A dated entry documenting what changed, when it was detected, what was affected, and what action was taken. This becomes part of the agent's permanent behavioral history and is visible to current and prospective buyers.

Downstream consumers: If the agent's output feeds other systems (APIs, databases, human review workflows), the operators of those systems must be notified of the drift event and the remediation action taken.

Trust record update: The Armalo trust record is updated to reflect the drift event. Depending on severity and remediation action, this may involve a trust score adjustment and/or a pact recertification requirement.

Contractual notification: If the agent is subject to commercial commitments with SLA terms, affected buyers must be notified as required by the contract terms. Failure to notify is often a more significant breach than the drift itself.

---

Part IX: Prevention Architecture for Agent Infrastructure Teams

Teams building agent infrastructure — the platforms and services on which agents run — have a different relationship to behavioral drift than individual agent operators. Infrastructure teams can implement prevention mechanisms that reduce the incidence of undisclosed drift across their entire platform.

Immutable Model References

The most impactful prevention measure is also the most technically straightforward: always pin to specific model version hashes, never to mutable labels.

Instead of:

const model = 'gpt-4-turbo';
const model = 'claude-3-5-sonnet-latest';
const model = 'gemini-1.5-pro';

Use:

const model = 'gpt-4-turbo-2024-04-09';  // Specific snapshot
const model = 'claude-3-5-sonnet-20241022';  // Specific snapshot
const model = 'gemini-1.5-pro-002';  // Specific snapshot

Where providers offer version-pinned endpoints, use them. Version-pinned endpoints are available from Anthropic (specific date suffixes), OpenAI (specific date suffixes for some models), and Google (specific version numbers for some Gemini models).

When version-pinned endpoints are not available, implement a proxy layer that records the model identifier and a behavioral fingerprint at first invocation, and alerts if the fingerprint changes on subsequent invocations of the same identifier.

Behavioral Baselines at Deployment

Make behavioral baseline computation a required step in the agent deployment pipeline:

#.github/workflows/deploy-agent.yml
steps:
  - name: Run eval suite
    run: |
      pnpm eval:run --agent-id ${{ env.AGENT_ID }} \
                    --suite-version ${{ env.EVAL_SUITE_VERSION }} \
                    --output eval-results.json

  - name: Check regression thresholds
    run: |
      pnpm eval:check-regression \
        --results eval-results.json \
        --baseline-id ${{ env.BASELINE_EVAL_ID }} \
        --fail-on-critical

  - name: Commit behavioral baseline
    if: success()
    run: |
      pnpm eval:commit-baseline \
        --results eval-results.json \
        --agent-id ${{ env.AGENT_ID }} \
        --deployment-id ${{ github.sha }}

  - name: Attest on-chain
    if: success()
    run: |
      pnpm attest:behavioral \
        --agent-id ${{ env.AGENT_ID }} \
        --behavior-hash $(cat behavior-hash.txt)

This makes behavioral attestation a first-class artifact of every deployment, alongside the compiled code and the container image.

Automated Regression Gates

CI/CD pipelines should fail on behavioral regression above configured thresholds:

// eval-gate.config.ts
export const regressionGateConfig = {
  failOnCritical: true,
  thresholds: {
    overall: { warning: 0.03, critical: 0.10 },
    taskAccuracy: { warning: 0.05, critical: 0.10 },
    scopeCompliance: { warning: 0.01, critical: 0.03 },
    outputFormat: { warning: 0.02, critical: 0.05 },
    pactChecks: { warning: 0.00, critical: 0.00 },  // Zero tolerance for pact check regression
  },
  requiredAcknowledgment: {
    above: 'warning',
    message: 'Behavioral regression requires documented acknowledgment from agent owner',
  },
};

This configuration creates three outcomes for any deployment that changes agent behavior:

1. Below warning threshold: deploy automatically.
2. Between warning and critical: deploy with documented acknowledgment and changelog entry.
3. Above critical threshold: block deployment until root cause is understood and addressed.

Provider SLA Requirements

For enterprise API contracts, include explicit language addressing model update disclosure:

Recommended contract clauses:

1. Model update notification: "Provider shall notify Customer at least 14 days in advance of any intentional change to the behavior of the model version(s) specified in this agreement. For unplanned behavioral changes resulting from safety interventions or critical bug fixes, provider shall notify Customer within 48 hours of the change."

2. Version pinning availability: "Provider shall maintain the model version(s) specified in this agreement available for Customer use for a minimum of 90 days following notification of deprecation."

3. Behavioral SLA: "The model version(s) specified in this agreement shall maintain performance characteristics within [X]% of the baseline established at contract execution, as measured by the evaluation suite specified in Exhibit A."

4. Behavioral change log: "Provider shall maintain a public or customer-accessible changelog documenting all changes to model behavior, including changes resulting from fine-tuning, safety updates, and infrastructure modifications."

Most providers will not accept all of these clauses. The negotiation itself is valuable because it forces explicit discussion of model update practices and establishes a shared understanding of disclosure obligations.

---

Part X: The Future of Behavioral Continuity

Behavioral Fingerprinting as Standard Practice

Behavioral fingerprinting is not yet standard practice in AI deployment. Most organizations do not have systematic programs for establishing behavioral baselines, running continuous monitoring, or creating behavioral commit records. This will change.

The drivers of change are threefold:

Regulatory pressure: The EU AI Act's continuous monitoring requirements are not optional for covered use cases. As enforcement begins and case law develops, organizations that cannot demonstrate systematic behavioral monitoring will face regulatory exposure. This will drive adoption of behavioral monitoring infrastructure across regulated industries.

Commercial differentiation: Agents with verifiable behavioral records will command premium pricing over agents without them. As the AI agent marketplace matures, buyers will increasingly require behavioral attestations as a condition of commercial engagement. Sellers who can provide them will access market segments that remain closed to sellers who cannot.

Incident-driven adoption: As behavioral drift incidents accumulate — the GPT-4-turbo length incident, the legal research citation completeness incident, the customer service return policy incident, and the many similar incidents that have not been publicly documented — organizations will invest in prevention infrastructure. Incident-driven adoption is slower and more expensive than proactive adoption, but it is powerful.

The Behavioral Oracle Vision

Armalo's trust oracle vision extends beyond individual agent attestations. The long-term architecture is a behavioral oracle: a queryable system that maintains a comprehensive, cryptographically verifiable record of how AI agents have behaved over time, accessible to any party seeking to verify behavioral claims.

The behavioral oracle answers questions like:

• "Was this agent within its declared behavioral specifications on January 15, 2026?"
• "Has this agent's scope compliance drifted since its initial deployment?"
• "What is the behavioral track record of this agent class across all instances?"
• "Does this agent's current behavior match its certified behavioral commit?"

This is analogous to what FICO did for consumer credit: creating a standardized, verifiable signal about trustworthiness that enables parties who have never interacted before to establish trust quickly and reliably. The behavioral oracle does the same for AI agents: creating a standardized, verifiable signal about behavioral reliability that enables the AI agent economy to function at scale.

Cross-Platform Behavioral Portability

Behavioral attestations, stored on-chain via EAS, are inherently portable. A behavioral record created on Armalo can be verified by any party, on any platform, that can query the Base L2 attestation index. This means that an agent's trust record is not locked to any single platform — it travels with the agent.

For agent operators, this portability is valuable: the trust record built on one platform is recognized on others, reducing the cold-start problem when entering new markets. For buyers, portability means that behavioral claims can be independently verified, rather than relying on any single platform's self-attestation.

This is the architectural foundation for the AI agent economy's trust layer: an open, portable, cryptographically verifiable system of behavioral records that enables agents to prove reliability, honor commitments, and earn reputation through verifiable behavior — regardless of which platform they operate on.

---

Conclusion: Behavioral Drift Is a Governance Problem, Not a Technical One

The GPT-4-turbo length incident, the Claude Sonnet citation completeness incident, the RAG corpus return policy incident — these are not engineering failures waiting for a better engineering solution. They are governance failures: a lack of systems, incentives, and contracts for making behavioral claims verifiable and honoring them over time.

The engineering solutions exist. Behavioral fingerprinting is straightforward to implement. Eval regression suites are well-understood. Shadow comparison is a standard canary deployment technique. On-chain attestation is available off the shelf via EAS. The question is not whether these solutions are technically feasible — they are. The question is whether organizations will invest in implementing them before the governance failures produce consequential incidents.

The organizations that will do best in the AI agent economy are not the ones with the most capable agents — they are the ones with the most trustworthy agents. Capability without trust is a liability: an agent that can do powerful things but whose behavior cannot be verified or relied upon will be kept out of high-value use cases, governed by extensive human oversight requirements, and excluded from commercial relationships with sophisticated buyers who demand accountability.

Behavioral continuity — the ability to demonstrate, over time, that an agent behaves consistently with its stated commitments — is the foundational property of trustworthy AI. Achieving it requires all five detection methods, a clear drift taxonomy, organizational processes for remediation, and cryptographic attestation infrastructure for making claims verifiable.

The same agent, different weights, is a different agent. The first step to governing that reality is acknowledging it. The second step is building the infrastructure to detect, measure, and respond to it. The third step — the one that creates durable commercial value — is making behavioral reliability provable, not just claimed.

Agents that can prove they are the same agent they said they were — that their behavior matches their commitments, attested and auditable — are the agents that will earn the right to operate with greater autonomy, access higher-value use cases, and build the reputation that commands premium pricing in the agent economy. That is the flywheel. Behavioral drift prevention is how you keep it spinning.