Blog · ← All posts
Agent Drift: Detecting When Your Agent's Behavior Changes Underneath an Unchanged Prompt
By the Ingenious AI team · Published: November 14, 2026 · Reviewed: November 14, 2026 · 7 min read
The short answer: Agent drift is a measurable change in an agent's recorded behavior — action mix, refusal rate, reasoning patterns, outcome rates — without a corresponding change in its mandate or prompt. The ledger detects it because the ledger is a baseline: compare this month's action distribution, per-tool rates, and decision patterns against the trailing baseline for the same agent, and divergence beyond normal variance is drift, whatever its cause (a silent model update, a drifting data source, a tool contract change). Detected from records, drift gets a date, a magnitude, and a cause hunt; detected from complaints, it gets a quarter of confusion first.
Replay a recorded agent incident free → Step through a real recorded run in the replay view, or paste ledger records into the public verifier and get a VERIFIED/BROKEN verdict in seconds — no signup.
The most dangerous sentence in agent operations is 'we didn't change anything'. The prompt is untouched, the code is untouched — and the agent is nonetheless different, because its world changed around it: the model provider shipped a new version under the same name, the retrieval corpus accumulated a season of new documents, a downstream tool altered its responses, the customer population shifted. Behavior is a product of the whole system, and the system never stops moving.
Drift is the name for the movement, and the ledger is the instrument that sees it. This guide covers the drift signals readable from action records, the comparison discipline that separates drift from ordinary variation, and the response — because a detected drift is a change-management event for a change nobody made.
What is agent drift? a sustained, measurable divergence between an agent's current recorded behavior and its own historical baseline — in what it does, how often it refuses, what it cites as reasons, or what outcomes it produces — occurring without an intentional, recorded change to its mandate, prompt, or tooling. Drift differs from an incident in speed (gradual rather than sudden) and from degradation in neutrality: drift is change first, judged good or bad only against the mandate it is drifting from or toward.
The signals: what to trend per agent
Five series, all computable from the ledger without new instrumentation. Action mix: the share of each action type in the agent's week — a support agent whose refund share creeps from 2% to 9% is becoming a different agent. Refusal rate: boundaries hit more or less often signal either a bolder agent or a changed world pushing on it. Reasoning markers: the frequency of authority citations, escalation language, and novel justifications in decision records. Outcome rates: tool error shares and downstream rejection rates, which expose tool-contract drift the agent may be silently absorbing. And volume-shape: the daily rhythm against its own baseline, the slow cousin of the burst alert. Trended together, these give drift a fingerprint specific enough to argue about productively.
The comparison discipline: drift versus weather
Behavior varies legitimately — seasons, campaigns, customer mix — so drift detection is a statistical habit, not a vibes alarm. Compare like periods (this month against the trailing three, weekday-matched), require the divergence to persist (two consecutive periods, or a cumulative-sum break), and size it against the agent's own historical variance. The ledger's completeness is what makes the statistics honest: sampled or success-only logs manufacture apparent drift from their own blind spots, which is why drift analysis on observability data so often ends in arguments about the measuring instrument instead of the agent.
Annotate the baseline with the known changes — prompt versions, mandate edits, tool grants, model version notices — as system events in the ledger, so the drift report can distinguish explained change (the step that coincides with a recorded intervention, which is not drift but deployment) from the unexplained residue that earns the name.
The cause hunt: four usual suspects
Once drift is dated, the hunt is short because the suspects are enumerable. The model changed: provider version updates, including the silent kind, top the list — correlate the drift's onset with the provider's release calendar, and your ledger's onset date is the evidence in that conversation. The data changed: retrieval corpora and knowledge bases accumulate, retire, and rot; sample the inputs the current agent cites against the baseline period's. The tools changed: a downstream API's new error shape or reordered fields can reroute agent behavior without any agent-side change. The world changed: customers, products, and policies evolved past the mandate — drift as the agent correctly serving a reality its instructions no longer describe. Each suspect has a different owner and remedy, which is why 'the model did it' as a default conclusion is drift analysis failing.
The response: drift is a change event, so manage it like one
A confirmed drift gets the full change-management treatment its unintentional nature makes necessary: the divergence documented against the baseline, a judgment recorded (harmful, neutral, or — it happens — an improvement worth adopting deliberately), and either a correction (pin the model, repair the data, update the tool contract) or a re-baseline (the mandate and expectations formally updated, the new behavior adopted as the standard, with the decision and its evidence filed as system events). The forbidden outcome is the shrug: drift noticed, discussed, and left undocumented, because the next quarter's reviewers will rediscover it as an anomaly with no history and twice the suspicion.
Regression discipline closes the loop: cases built from the baseline period's characteristic runs join the evaluation suite, so future drift — from any suspect — trips a test before it trips a customer.
Five actions you must take today
Drift detection is a monthly habit built on the history you are already keeping:
- Trend the five series per agent. Action mix, refusal rate, reasoning markers, outcome rates, volume shape — computed from the ledger, reviewed monthly, no new instrumentation required.
- Annotate the baseline with every intentional change. Prompt versions, mandate edits, tool grants, model notices — as system events, so explained change and drift never share a costume.
- Define drift before you see it. Persistence and magnitude thresholds per series, written down — a divergence that meets them opens a ticket automatically, not a debate.
- Run the four-suspect hunt in order. Model, data, tools, world — each with its check, each conclusion evidenced from records rather than seniority.
- Close every drift with a document. Corrected or re-baselined, judgment and evidence filed; an undocumented drift is next quarter's incident seed.
Worked example: the refund share that crept
At fictional retailer Aldergrove Home, the monthly drift review flagged the support agent's refund share: 2.1% of actions in the baseline quarter, 3.4% two months ago, 5.9% this month — persistent, sized well beyond variance, unexplained by any recorded change. The cause hunt ran in order. The model had not changed (provider version pinned, confirmed). The tools had not changed. The data had: the help-center corpus, refreshed in a website migration, had lost the troubleshooting decision tree the agent used to cite before refunding — its reasoning records showed the shift plainly, with 'no resolution path found' citations rising in step with the refunds.
The fix was a corpus restoration plus a regression case: the twenty baseline runs whose reasoning cited the decision tree became a gate for help-center changes. Refund share settled at 2.6% within a month. The cost accounting in the review is the drift discipline's advertisement: at the month's volume, the two drifted months represented roughly 1,900 excess refunds and $61,000 in margin — a figure nobody felt as an incident, because drift never announces itself as one. The trend line saw what no alert could: not a spike, but a slope.
FAQ
How is drift different from the anomaly alerts?
Speed and intent. Alerts catch fast, large deviations — bursts, first-seen tools — during the incident. Drift review catches slow, small, persistent divergence over weeks, which alerts are deliberately tuned to ignore (a 0.3%-per-week creep never crosses a sigma threshold worth paging on). You need both instruments because the failure physics differ: the fire alarm and the foundation survey are not substitutes, and a fleet with only alarms learns about its slopes from its customers.
Can drift be positive?
Yes — agents drift toward better outcomes when their data improves, when a model update genuinely helps, or when the customer mix simplifies. The discipline is identical: detect, verify against the mandate, and adopt deliberately with a re-baseline, because unowned improvement is still unowned behavior — it can drift back out as silently as it drifted in, and an audit that finds your agent behaving better than its mandate describes still finds a mandate that does not describe your agent.
What if our provider changes models without notice?
Your ledger becomes the detection and the evidence: the onset date from behavior series, the divergence magnitude, the sampled reasoning differences. Providers respond to dated, quantified divergence reports differently from 'the agent feels off' tickets. Operationally, pin versions where the provider allows it, record version notices as system events when they arrive, and treat unannounced-change windows in your vendor review with the seriousness the drift report justifies.
How much history do we need before drift detection works?
A quarter of baseline gives the five series usable variance; a month gives a starting picture with wider error bars. This is a compounding argument for recording early and completely — drift analysis is one of the dividends the history pays, and like the others it cannot be backfilled. Teams that start the ledger today are buying next year's drift review at this year's prices.
Who should own the drift review?
The agent's owner, with the platform team supplying the computed series — the same ownership as the lifecycle review, and best run as the same meeting: mandate still accurate, state still justified, behavior still itself. Drift left to a data team becomes a dashboard; drift owned by the agent's owner becomes a decision, which is the only form in which it improves anything.
Start logging your agents free → The free tier gives you the full hash-chained ledger for 1 agent and 1,000 actions a month, with chain verification and replay included.