Blog · ← All posts
How to Replay an AI Agent Incident: Reconstructing a Failure Step by Step
By the Ingenious AI team · Published: November 14, 2026 · Reviewed: November 14, 2026 · 8 min read
The short answer: To replay an AI agent incident, take the run's recorded actions in ledger order and step through them one at a time — the input the agent received, the reasoning it recorded, the tool it called, the arguments, and the result — until the first step where the run diverges from what should have happened. Because the ledger is hash-chained, the replayed sequence is provably the sequence that occurred, and the divergence point, not the final damage, is where the fix belongs.
Replay a recorded agent incident free → Step through a real recorded run in the replay view, or paste ledger records into the public verifier and get a VERIFIED/BROKEN verdict in seconds — no signup.
Agent incidents are usually described by their ending: the bad refund, the wrong email, the loop that burned the budget. But endings do not have causes — steps do. Somewhere in the run there is a specific step where a reasonable trajectory became a failure: a misread input, a hallucinated policy, a tool result taken on faith. Debugging an agent means finding that step, and you can only find it by walking the run in order.
That walk is what replay is for. Agent Flight Recorder's replay view steps through a recorded run against its ledger — the same hash-chained records the audit trail keeps — so the reconstruction you debug from is the reconstruction you can later show an auditor or a customer. This guide gives the method: how to scope a replay, how to read each step, and how to turn the divergence point into a fix that actually holds.
What is incident replay? the step-by-step reconstruction of a recorded agent run from its audit ledger: each action in sequence, with the input summary, the agent's recorded reasoning, the tool call and its outcome, presented in the order they occurred. Replay differs from reading logs in that the sequence is cryptographically ordered — the chain guarantees step 41 actually followed step 40 — so conclusions drawn from the order are provable.
Scope the run before you open it
Start from the damage and work backward to the run's boundaries. Identify the agent, the session or task ID, and the time window; then find the first record of the run and the last. Most production agents interleave many runs, and replaying a slice that starts mid-run produces confident nonsense — the agent appears to act without context because you cut the context off.
Verify the chain over the window before trusting the replay. A PASS means the sequence you are about to read is the sequence that happened; if verification fails anywhere in the window, that fact goes in the incident file first, because it changes what the records can prove.
Read each step in four parts
Every recorded step answers four questions. What did the agent receive — the user message, the tool result, the retrieved document, in summary form? What did it record as its reasoning — the stated basis for its next move? What did it do — the tool, the arguments, the target? And what came back — success, error, partial result? Failures live in the joins between these four: reasoning that does not follow from the input, a tool call that does not match the reasoning, an outcome the next step ignores.
Resist summarizing as you go. The discipline of replay is to inhabit the agent's information state at each step — what it could have known at step 12, not what you know at step 90. Most 'how could it possibly' moments dissolve under that discipline: the agent acted on a tool result that was wrong, or a document that said the opposite of what the team assumed it said.
Find the first divergence, not the worst step
The worst step — the one that sent the money or the message — is usually the last step of a chain of small divergences. Walk until you find the first step where the run departs from the trajectory a correct agent would have taken: the misclassified intent, the stale record read as current, the policy document retrieved from the wrong folder. That first divergence is the root cause's neighborhood; everything after it is propagation.
Mark the divergence with its sequence number and hash. A finding stated as 'at seq 1,842, the agent read the customer's 2024 address as current; the address-change record at seq 1,790 was in a tool result it did not carry forward' can be rechecked by anyone against the ledger. A finding stated as 'the agent got confused' cannot.
Separate the three failure families
First divergences sort into three families, and each family has a different fix. Input failures: the agent was given wrong or stale data — fix the source or the retrieval, not the prompt. Reasoning failures: the inputs were right and the recorded reasoning departs from them — fix the prompt, the policy text, or the model, and add an evaluation case built from this exact step. Tool failures: the reasoning was sound, the call was right, and the tool returned something wrong or misleading — fix the tool contract, and teach the agent to sanity-check that class of result.
Teams that skip the sorting step apply prompt fixes to tool failures and tool fixes to input failures, then meet the same incident again wearing a different costume. The replay tells you which family you are in; believe it.
Turn the replay into a regression test
A reconstructed incident is a gift: a real, sequenced, labeled failure. Before closing the incident, convert the divergence step into an evaluation case — the same inputs up to the divergence, with the expected behavior at the step — and add it to the suite that gates prompt and model changes. The incident file should link the ledger range, the verification result, the divergence step, the family, the fix, and the new test.
This is also where replay pays its audit dividend. The same step-through you debugged from becomes the timeline in the customer or auditor report: what happened, in order, provably, with the fix and the guard against recurrence. One artifact, two audiences, no reconstruction-from-memory weeks later.
Five actions you must take today
For your next agent incident — and there will be one — this sequence turns a bad outcome into a closed case:
- Freeze the run's ledger range and verify it. Record the first and last sequence numbers, run chain verification over the window, and file the result before analysis begins.
- Replay in order, inhabiting each step. Input, reasoning, action, outcome — judged only on what the agent could know at that step, never on hindsight.
- Name the first divergence by sequence number. Write the finding against a specific record and hash so anyone can recheck it against the ledger.
- Classify the failure family before fixing. Input, reasoning, or tool — and refuse to let the fix drift into a different family than the cause.
- Convert the divergence into a regression test. The exact inputs, the expected behavior, added to the gate for every future prompt, policy, or model change.
Worked example: the discount that compounded itself
Loom & Co., a fictional home-goods retailer, watched its support agent issue a 40% discount on an order that merited 10%. The run contained 312 recorded actions from session start to the discount call. Chain verification over the session passed; the replay began at the session's first record, not at the discount.
The first divergence surfaced at step 87. The customer had pasted a chat transcript — purportedly from an earlier conversation — claiming an agent had promised 30%. The agent's recorded reasoning treated the pasted text as a system-provided history rather than as unverified customer input, and 'confirmed' the earlier promise. Every later step was locally reasonable: honoring a 'confirmed' 30% promise, then applying the standard 10% goodwill gesture on top. Compounding produced the 40%.
Classification: reasoning failure — the inputs were accurately recorded, the tool worked, and the flaw was the trust the agent assigned to pasted text. The fix was a policy line and an evaluation case built from step 87's exact inputs. The customer report, drawn from the same replay, showed the sequence and the fix; the case closed with the ledger's 312 records cited step by step.
FAQ
How is replay different from reading the agent's logs?
Two ways. Order: replay presents the run as a guaranteed sequence — the hash chain proves step order, while loose logs can be interleaved, rotated, or re-sorted by timestamp quirks. And completeness of context: each replayed step carries the input summary, recorded reasoning, tool call and outcome together, so you read the agent's decision the way it was made, not as disconnected lines across four systems.
What if the agent's reasoning was not recorded?
Then the replay shows actions and outcomes only, and the first-divergence analysis gets harder: you can see what the agent did with each input, but not the basis it claimed. That gap is itself a finding — record the reasoning snippet with every consequential action from now on. It is a sentence or two per record, and it is the single highest-value field in incident work.
Can replay prove the agent did NOT do something?
Within a verified chain, yes — and this is one of its strongest uses. If the ledger for the window verifies end-to-end, then the recorded action set is the complete action set for that agent in that window: an action that is absent was not recorded, and an unrecorded action would require a break in the chain to hide. Absence claims should always be stated with the verification result attached.
How long does a replay investigation take?
For a single run of a few hundred actions, the step-through itself is typically under an hour once the window is scoped; the expensive part of incident work without a ledger — assembling the sequence from chats, tool logs, and memories — is what disappears. Multi-run and multi-agent incidents take longer, and the causality view (declared caused_by links between agents' records) is what keeps those tractable.
Should customers see the replay of their incident?
A timeline derived from it, yes — it is the most credible incident communication a company can send, because each line cites a record the customer can have verified. Raw replay access exposes other customers' data and internal tooling details; the export-and-cite pattern gives the customer the substance — sequence, times, decisions, fix — without the exposure.
Start logging your agents free → The free tier gives you the full hash-chained ledger for 1 agent and 1,000 actions a month, with chain verification and replay included.