Blog · ← All posts
Omission Detection: Proving What an AI Agent Did NOT Do (and Catching What It Never Logged)
By the Ingenious AI team · Published: November 14, 2026 · Reviewed: November 14, 2026 · 7 min read
The short answer: Omission detection answers the hash chain's blind spot: a chain proves recorded actions were not altered, but it cannot see an action that was never recorded. The control is reconciliation — continuously comparing the ledger against independent downstream evidence (processor settlements, sent-mail logs, CRM change history, API gateway counts) and flagging actions that exist downstream with no matching ledger record. An unmatched downstream action is an omission, and its rate is the coverage metric your audit trail should be judged by.
Replay a recorded agent incident free → Step through a real recorded run in the replay view, or paste ledger records into the public verifier and get a VERIFIED/BROKEN verdict in seconds — no signup.
Every tamper-evidence conversation eventually reaches the honest objection: 'Your chain proves nobody edited the records. How do you know the agent recorded everything in the first place?' It is the right question, and a serious audit-trail system must answer it with mechanism, not reassurance. The answer is to stop asking the ledger to testify about itself and start cross-examining it against witnesses it does not control.
That cross-examination is omission detection, part of Agent Flight Recorder's evidence machinery. This guide explains the omission problem precisely, the reconciliation method that detects it, and how to operate coverage as a measured property of your fleet rather than a hope about your instrumentation.
What is an omission? a real-world action performed by an agent that has no corresponding record in the audit ledger — because instrumentation missed the path, the recorder was bypassed, or a component failed silently. Omissions are the completeness failure of an audit trail: distinct from tampering (altering records that exist), and detectable only by comparing the ledger against evidence outside itself.
Why chains cannot see their own gaps
A hash chain is a closed system of proof: it verifies that the records present form an unaltered sequence. An action that never entered the sequence leaves no hole in the mathematics — the chain of what was recorded still verifies beautifully. This is not a flaw in the design; it is the boundary of what the design claims. Integrity and completeness are separate properties, and conflating them is how audit trails acquire unearned confidence.
The failure modes that create omissions are mundane: a new tool added without instrumentation, a code path that calls the API directly instead of through the recording wrapper, a retry that executes in a worker the recorder does not wrap, an agent version that logs on success and forgets failures. None of these is malicious. All of them produce the same audit answer — a missing record where the world shows an action.
Reconciliation: the ledger meets its witnesses
Downstream systems keep their own books. The payment processor settles refunds whether or not your agent logged them; the mail server records sends; the CRM stamps changes with times and API keys; the API gateway counts calls. Reconciliation is the scheduled comparison: for each downstream action attributable to the agent, find the ledger record that matches it — by downstream ID where recorded, by tool, argument digest, and time window where not — and report the residue in both directions.
Ledger records with no downstream counterpart are usually benign (reads, internal decisions, failed attempts) and are interpreted by action type. Downstream actions with no ledger record are omissions by definition, and each one gets investigated as a small incident: which path produced it, and how long has that path been unrecorded? In AFR this cross-check is the omission detector's job, run continuously rather than at audit time.
Making matching strong: downstream IDs in the record
Reconciliation quality is decided at recording time. When the ledger record carries the downstream system's own identifier — the processor's refund ID, the mail server's message ID, the CRM's change ID — matching is exact and omissions stand out crisply. Where IDs are unavailable, matching falls back to fingerprints (tool + argument digest + tight time window), which is serviceable but noisier. The instrumentation rule follows: always capture and record the downstream ID; it is one field, and it converts your coverage audit from statistical to exact.
The same discipline helps in reverse during disputes: a ledger record whose downstream ID checks out in the processor's own system is corroborated by a witness neither party controls, which is the strongest form an agent record can take.
Operating coverage as a metric
Coverage — matched downstream actions divided by total downstream actions, per agent per period — belongs on the same dashboard as verification results. A healthy fleet runs at or near 100%, and the interesting event is any dip: a deploy that drops coverage to 97% has told you exactly where to look, days before an auditor's sample finds the gap and prices it as a finding. Alert on coverage decline the way you alert on errors; an unrecorded path is an error, merely a silent one.
Report coverage honestly in the audit packet. A stated, measured 99.7% with the residue investigated reads as a control environment; an unmeasured claim of 'we log everything' reads as a hope, and auditors are professionally obliged to test hopes.
Bypasses: when the agent routes around the recorder
The adversarial version of omission is the bypass: an agent (or a prompt injection steering it) that reaches a tool through a path the recorder does not wrap — a raw HTTP call instead of the tool client, a shell command instead of the API. Defense is architectural: tools should be reachable only through the recorded path (network and credential scoping that makes the unrecorded route fail), and the omission detector watches for the fingerprints of success downstream without a record.
This is where recording stops being observability and becomes part of the control plane. The ledger is not a camera watching the agent work; properly built, it is the doorway the agent works through — and a doorway counts everyone who passes.
Five actions you must take today
Completeness is engineered and then measured. Start both halves now:
- Inventory the witnesses. For every tool your agents use, name the downstream system whose own records can corroborate agent actions — processor, mail, CRM, gateway — and get read access for reconciliation.
- Record downstream IDs religiously. Every action record carries the downstream identifier it produced; audit your instrumentation for the tools that currently return one you throw away.
- Run reconciliation on a schedule. Weekly at minimum, continuous for money-moving agents; every unmatched downstream action opens a ticket, not a shrug.
- Put coverage on the dashboard. Per agent, per period, beside verification status — and alert on any decline, however small.
- Close the unrecorded routes. Scope credentials and network paths so tools are reachable only through the recording layer; a bypass that cannot execute cannot omit.
Worked example: the 3% that was a whole code path
Fictional insurer Claimline reconciled its document agent's ledger against its mail platform monthly. The first run matched 96.8% of agent-attributed sends. The residue was not random: every unmatched send was a denial-letter attachment generated through a legacy template service the agent called directly — a path added before the recorder existed, quietly handling roughly 90 letters a week, 3,900 over the reconciliation window's lookback.
Because the mail platform's records existed, the gap was boundable and repairable: the path was re-routed through the recording wrapper, coverage rose to 100% the following week, and the historical exposure was documented in the audit file with the mail platform's own logs cited as the interim record. Claimline's assessor later described the episode as the strongest item in the review — not because the gap existed, but because the company's own machinery found it, measured it, and closed it before the audit did.
The counterfactual is the lesson: without reconciliation, that path would have surfaced for the first time inside someone else's sample, framed as concealment rather than as a fixed defect.
FAQ
If an action was omitted, is the whole ledger untrustworthy?
No. The chain still proves the integrity of what is recorded, and the omission rate bounds the completeness question quantitatively. A ledger at measured 99.7% coverage with the residue explained is far stronger evidence than an unmeasured ledger claiming 100%. Trust in evidence is not binary; it is a set of measured properties, and coverage is the one teams most often leave unmeasured.
What downstream sources work for reconciliation?
Any system that independently records the effects of agent actions: payment processors and banks, email and messaging platforms, CRMs and ticketing systems, cloud and API gateways, database change-data-capture streams. You need one witness per consequential tool. Pure read actions often have no downstream witness — which is acceptable, because the consequential omissions are the writes, and those almost always leave a trace somewhere.
How tight must the time window be for fingerprint matching?
Tight enough to be unambiguous in context — seconds to a few minutes for most tool calls, given synchronized clocks. This is one more reason to record downstream IDs: fingerprint matching is a fallback for legacy tools, and every tool migrated to ID-carrying records removes a class of matching ambiguity permanently.
Can omission detection catch a deliberate bypass in real time?
It catches the effect — a downstream action with no record — on the reconciliation cadence, which for money-moving agents should be minutes, not months. Combined with architectural closure (tools reachable only through the recorded path), the bypass window shrinks to the reconciliation interval, and the attempt itself usually leaves fingerprints in gateway logs that identify the route used.
Should coverage appear in customer-facing reports?
Yes, in the audit packet and security documentation, with the same plainness as verification results. Sophisticated customers have begun asking the completeness question directly — 'how do you know it logged everything?' — and a measured coverage figure with a described method is the only answer that distinguishes a control from a claim.
Start logging your agents free → The free tier gives you the full hash-chained ledger for 1 agent and 1,000 actions a month, with chain verification and replay included.