Audit an AI agent by reconstructing a run from records of who initiated it, which permissions it used, what it called, and what changed. Connect those records with shared identifiers, preserve the relevant approvals, and compare the agent's reported outcome with evidence from the systems it acted on.
Suppose a sales agent says it updated an account after reading an email. You need to establish which account changed and whether someone approved the update. The conversation can help locate the event, but the investigation also needs the tool request and the CRM's record of the change.
Copy link to headingWhat is an AI agent audit trail?
An AI agent audit trail is a connected record of an agent's activity and the authority under which it acted. It should let someone examine a particular run after the event, including failed attempts and actions that only partly completed.
Application logs often record requests, errors, and timing. An agent may make several model calls and use several tools within one request, so a successful response alone does not establish what happened during the run.
Traces link the individual operations in a run through records called spans. OpenTelemetry's GenAI agent conventions are still in development and define spans for agent invocations and tool execution. Teams can use these conventions to describe activity consistently while application owners determine which actions and approvals an investigation must establish.
Keep a distinction between a record of an attempted action and evidence of its effect. Requests to update sales records can fail, time out after succeeding, or succeed against the wrong record. The audit trail should retain enough information to investigate each possibility.
Copy link to headingWhich records should you capture?
Choose records by working backward from the questions a reviewer would ask. For an agent that can change a business system, a useful starting point is:
These are recommended evidence categories, not a requirement to copy every payload into one log. Store references to protected source records where those references provide enough evidence. If a source can change, decide whether an authorized snapshot or version reference is needed.
Use stable identifiers to connect the pieces. Sessions can contain multiple runs, each with its own tool calls. Give each tool call a distinct identifier and retain the parent run identifier. When a downstream service provides its own request or transaction identifier, record that too.
Timestamps help establish order, but matching records by time alone becomes ambiguous when several employees use the same agent. Make the connection explicit wherever your systems allow it.
Copy link to headingHow do you connect an action to a person?
Record who initiated the task separately from the identity used to execute it. When an agent uses a service account to fulfill a sales employee's request, retain both identities and the application's authorization decision.
Human approval is another event. Capture who approved which proposed changes, then bind that decision to the execution. General messages such as "looks good" are difficult to interpret if the agent revises the proposal afterward.
Scheduled agents need an accountable owner even when no person is present to initiate a run. Record the job that triggered the activity and the identity it used. Do not label the owner as having personally approved every scheduled action unless the workflow requires and records that approval.
Shared credentials make this distinction more important. The destination system may record only the shared account, leaving your application responsible for connecting its activity to the employee's request. Credential storage alone does not create that connection.
Copy link to headingHow do you investigate a particular run?
Use the reported business outcome to find the relevant activity. In the sales example, start with the customer record and approximate time of the unexpected update.
Find the run and preserve the relevant records. Capture the run identifier, agent version, and links to the underlying evidence before routine retention removes them.
Establish the request and authority. Identify the initiating person or job, the executing identity, and the permissions that applied.
Compare the proposed action with any approval. Check that the reviewer saw the same account and field changes that the agent submitted.
Follow the tool call into the destination system. Compare its arguments and result with the CRM's change history. Check errors, retries, and partial completion.
Record the finding and its limits. State what the evidence establishes, any missing records, and the correction or further investigation required.
In this illustrative investigation, an agent reports that a CRM update timed out. The application sends a tool-call identifier that the CRM retains with its change record so a reviewer can connect the evidence below:
The evidence establishes that the approved change reached the CRM even though the agent did not receive confirmation. It does not explain what caused the response to fail. The reviewer should check the account's current state before deciding whether another action is needed; repeating the write could overwrite a change made since the first attempt. If a retry has already occurred, include that call and its downstream result in the investigation.
An agent-generated summary can help a reviewer navigate a long run. Check consequential conclusions against the underlying records, especially when the summary says an action succeeded or received approval.
Copy link to headingHow should you protect and retain audit records?
Audit records can contain the same confidential information as the task itself. Restrict who can inspect them, remove credentials from captured inputs, and decide which sensitive fields need redaction before collection. Avoid giving every person who can use an agent access to everyone else's conversations.
Choose a retention period based on the investigation window and the obligations that apply to the data. Record who owns the retention policy and whether an external storage destination is required. Your dashboard's default history may end before someone notices an incorrect business record.
Check collection coverage as well as retention. If a tracing system samples only some runs, an absent record may mean the run was not collected. Test a known action and confirm that you can retrieve all the evidence required for review. Protect the evidence against unauthorized changes and monitor export failures.
Article 12 of the EU AI Act addresses automatic logging capabilities for high-risk AI systems over their lifetime. This is a scoped requirement, not a rule that every internal agent must retain every prompt forever. Determine the applicable system classification and record obligations before choosing a retention policy.
Copy link to headingWhich Vercel records help reconstruct agent activity?
Different records answer different questions. Vercel Passport records successful access to a protected project as a passport-access-granted event, identifying the visitor and project context. That sign-in event does not establish permission for a later business action.
For agents built with eve, Agent Runs connects sessions and turns with tool arguments, results, timing, and token usage. Select a run to see its trigger and deployment, then inspect the tool calls within each turn. In the CRM example, those arguments help establish which account and field the agent attempted to change. You would still need the approval record and the CRM's change history to complete the investigation. For existing deployments, check tracing coverage before relying on this view.
Vercel Audit Logs cover team activity and are available to Enterprise team owners. Audit Log Drains can send those events to HTTP endpoints or Amazon S3. Exporting team events does not also establish that application approvals or downstream changes were captured.
Treat these as evidence sources to connect. Application-specific records must fill any gap between an employee entering the app, an agent calling a tool, and the destination recording a change.
Copy link to headingHow can you limit content sent to eve’s Agent Runs?
Configure Agent Runs export policies in agent/instrumentation/agent-runs.ts to retain an eve agent's spans while redacting their captured content:
import { agentRuns } from 'eve/instrumentation/otel';
export default agentRuns({ exportPolicy: { span: ({ audience }) => audience === 'public' ? { emit: true } : { redact: true, inputs: true, outputs: true }, },});The policy retains spans for private sessions but removes eve's known input and output content attributes, including tool arguments, tool results, and exception and status details. Public-session content passes through this policy without redaction, so choose the condition according to your data rules.
This redaction applies only to Agent Runs. It does not alter exports to other destinations or restore content excluded by eve's process-wide trace policy.
With arguments and results removed, the CRM investigation would rely on protected application records to show the proposed changes and their outcome. Before relying on the redacted trace, confirm that each of those records can still be matched to its span.
Copy link to headingFrequently asked questions
Copy link to headingAre application logs enough to audit an AI agent?
They can be if they capture the agent's operations and connect them to identities and outcomes. Ordinary request and error logs often omit that detail. Test whether your records let a reviewer reconstruct a known action.
Copy link to headingCan a trace prove an agent's reasoning?
No. Traces show the operations and content your instrumentation captures; they do not fully or reliably explain the model's internal reasoning. Evaluate the recorded actions against the task, permissions, and evidence of their results.
Copy link to headingMust every prompt be retained?
No. Decide which content is necessary for the audit purpose and which can be omitted, redacted, or referenced elsewhere. Retaining all prompt text can duplicate confidential data, so set collection rules before deployment.
Copy link to headingHow long should audit records be kept?
Choose a period that covers the relevant review needs and applicable recordkeeping obligations. Check the retention of every evidence source, including the destination system, before assuming a run can be reconstructed. Assign an owner to approve and maintain that policy.
Copy link to headingDoes a sign-in record prove who authorized a tool action?
No. Authentication establishes the user's identity at sign-in. The application checks whether an action is permitted, and a reviewer gives any required approval. Record those decisions separately and connect them to the tool operation they permitted.