Blog · ← All posts
What Exactly Should an AI Agent Log? Prompts, Tool Calls, Decisions and Outcomes Compared
By the Ingenious AI team · Published: November 14, 2026 · Reviewed: November 14, 2026 · 7 min read
The short answer: Log tool calls and decisions permanently; log prompts and full outputs briefly or not at all. The audit ledger should hold, for every step: the tool called with a masked argument summary and its outcome, the agent's stated reasoning for the decision, token counts, and a hash link to the previous record. Full prompt and completion text belongs — if anywhere — in a separate store that expires in days, because its audit value is low and its privacy and storage cost is the highest of any field you can record.
Replay a recorded agent incident free → Step through a real recorded run in the replay view, or paste ledger records into the public verifier and get a VERIFIED/BROKEN verdict in seconds — no signup.
'What should we log?' sounds like one question but it is four, because prompts, tool calls, decisions, and outcomes have different half-lives of usefulness and wildly different risk profiles. Teams that answer it once, globally — 'log everything' or 'log the minimum' — end up with a store that is simultaneously too dangerous to keep and too thin to investigate with.
This guide takes the four record types in turn, states what each is actually for, and assigns it a home: the permanent hash-chained ledger, the short-retention debug store, or nowhere. The assignments reflect one principle: the ledger exists to prove what the agent did and why, in a form that survives audits, disputes, and time. Anything that does not serve that purpose must justify itself against the risk of keeping it.
What is the decision record? the part of an agent log entry that captures why an action was taken: the agent's stated reasoning at the moment of decision, the policy or instruction version it was acting under, and any approval attached to the action. Decision records are what separate an audit trail from a call log — tool calls show what happened; decision records show what the agent thought it was doing, which is what incident reviews and auditors actually need.
Tool calls: the permanent spine
Every investigation, audit, and replay is built from tool calls, because tool calls are where the agent touches the world. Record each one in the ledger: tool name, timestamp, an argument summary with identifiers masked and the full payload represented by a digest, the outcome (success, failure, partial, refused), duration, and the downstream identifier where one exists. This is the non-negotiable layer — an agent run without its tool calls recorded is a story without verbs.
Argument summaries deserve craft. 'issue_refund $189.99 to order ••4217' reconstructs a decision; 'issue_refund {}' does not, and the raw payload with full card or account data is a liability the summary exists to avoid. Digest the full arguments so a later dispute can prove what the arguments were, without storing the sensitive values themselves in the audit store.
Decisions: the highest-value field per byte
The reasoning snippet — a sentence or two the agent produces at decision time — is routinely the field that resolves incidents. Tool calls show the refund; the decision record shows the agent believed the customer had been promised it by a manager, which redirects the entire investigation from code to conversation design. Auditors ask for it by another name: the basis on which the system acted.
Record it with the policy context that makes it interpretable: the policy or prompt version in force, the approval attached if the action crossed a threshold, and any guardrail verdict (allowed, refused, escalated) that shaped the step. Keep the snippet short by design; you are capturing the justification, not archiving the chain of thought, and a disciplined one-or-two-sentence rationale is more readable in a hearing than forty paragraphs of deliberation.
Prompts: debug material, not audit material
Full prompts feel like the complete record and are mostly a trap. They duplicate content owned by other systems (the conversation, the retrieved documents), they aggregate exactly the personal and confidential data your privacy review least wants concentrated, and in incident work they answer questions nobody is asking — the dispute is never about the forty pages the agent read; it is about the one claim it acted on.
Where prompts earn their keep is live debugging: tuning retrieval, diagnosing a misread, reproducing a bad step during development. That purpose needs days of retention, not years, and a store with tight access and automatic expiry. Many teams reasonably skip prompt logging in production entirely and rely on the decision record plus argument digests; if you keep prompts, keep them on the short clock and say so in the retention policy.
Outcomes and refusals: the half everyone forgets
A tool call without its outcome is half a record. Log what came back: success with the downstream ID, failure with the error class, partial results flagged as partial, and — the most forgotten category — refusals, where a tool or guardrail declined the agent's request. Refusal records are disproportionately valuable: they show boundaries working, and they show the agent's response to a boundary, which is where attempted workarounds become visible in a single sequence.
Outcomes also close the loop for lost-acknowledgement recovery: an attempt record without a matching outcome inside its window is precisely the ambiguous action the recovery sweeper hunts. Teams that log attempts and outcomes as separate, linkable records get that detection for free; teams that log only successes discover their ambiguities in reconciliation reports.
Token counts and timing: cheap signals, keep them in the ledger
Three numbers per step — input tokens, output tokens, wall-clock duration — cost almost nothing to store permanently and pay out of proportion. Cost incidents announce themselves in token counts before invoices arrive; a step that normally uses 800 tokens and suddenly uses 6,000 is a loop, a retrieval failure, or an essay nobody ordered. Duration exposes the slow tool that made the agent improvise. Both signals feed anomaly detection directly, and both belong in the ledger record rather than a metrics system that will be sampled away before the incident review.
Five actions you must take today
Audit your current logging against the four record types and fix the assignments:
- Guarantee the spine. Every tool call lands in the ledger with tool, masked argument summary, payload digest, outcome, and downstream ID — no sampling, no success-only logging.
- Add the decision record. A one-or-two-sentence stated rationale plus policy version and any approval, attached to every consequential step.
- Evict full prompts from the permanent store. Move them to a short-retention debug store with automatic expiry, or stop collecting them in production.
- Log refusals as first-class events. Every guardrail or tool refusal, with what the agent did next — the sequence is the evidence.
- Keep tokens and duration in the ledger. Three numbers per step, permanently; they are your cheapest anomaly signal and your cost incident's first draft.
Worked example: one refund, four record types
A fictional retailer, Alder & Vine, reconstructed a disputed refund from its ledger and could answer every question the customer's bank asked — from a single run of 96 records. The tool-call layer showed issue_refund called at 11:42 with a masked argument summary ($129.00, order ••8831) and a payload digest. The decision record carried the agent's stated basis — item arrived damaged, photo verified in the conversation, refund per policy v14 — and the guardrail verdict: allowed, under the $500 cap, no approval required.
The outcome record closed the loop: processor confirmation with the refund's downstream ID at 11:42:04, matching the processor's own settlement file when the bank checked. The prompt layer, by contrast, contributed nothing to the dispute and was not needed: the 30-plus pages of conversation and product data the agent had read stayed in the 14-day debug store, unmissed.
Total permanent evidence for the transaction: under two kilobytes, chain-verified as part of the day's 18,442 records. The bank's chargeback inquiry closed on the first exchange.
FAQ
Should we log the agent's full chain of thought?
No. Log the decision rationale — the short justification the agent states for the action it takes — not the extended deliberation. Long chains of thought are bulky, often contain echoed sensitive content, and are rarely coherent evidence; the stated basis at decision time is what investigators and auditors use, and it is what the agent can fairly be held to.
Are argument digests enough, or do auditors want raw arguments?
Auditors want to establish what the action was; a well-formed summary plus a digest does that, and lets you prove a claimed full record matches the digest later if one is produced from the operational system. Raw arguments in the audit store mostly add breach surface — full account numbers and personal data concentrated in the one store everyone wants to read. Mask, summarize, digest.
What about logging the model's confidence scores?
If your stack produces a meaningful confidence signal that gates decisions, record the score and the threshold it was judged against — that pair is decision context. If the score is decorative (never acted on, never thresholded), skip it; fields nobody uses in an investigation become noise that slows every future reader of the record.
How do we log multi-step plans the agent makes?
Record the plan as a decision event when it forms — the stated goal and intended steps in summary — then let the tool-call records show execution against it. Divergence between the recorded plan and the recorded actions is itself a diagnostic: plans that silently mutate mid-run are a known failure pattern, and you can only see the mutation if the original plan is on the record.
What is the minimum viable record for a small team?
Agent ID, timestamp, tool, masked argument summary, outcome, one-line reasoning, token counts, and the hash link — eight fields, one record per action, written as the action happens. That minimum reconstructs incidents, satisfies most audit samples, and fits in a spreadsheet export. Everything beyond it is refinement; everything less leaves a question you will eventually be asked and cannot answer.
Start logging your agents free → The free tier gives you the full hash-chained ledger for 1 agent and 1,000 actions a month, with chain verification and replay included.