The Agent Evidence Pack for a Security Review
The two documents an enterprise reviewer wants for your AI agent — the allow-list and the tool-call log — specified field by field, generated from code.
TL;DR: When an enterprise reviewer asks about your agent, they want two artifacts, not a conversation: a tool allow-list that shows what it can reach and who approved it, and a tool-call log that shows what it actually did. Most teams have a paragraph describing both and neither one as a real document. This is a field-by-field spec for both, plus the one thing that makes them survivable past week one — generating them from the code that already registers your tools, instead of hand-maintaining a spreadsheet that’s wrong by the second sprint.
What two documents actually satisfy an agent security review?
An allow-list that names every tool your agent can call, its credential scope, and who approved it — plus a tool-call log that records every invocation with enough detail to reconstruct what happened without you in the room. Everything else a reviewer asks — autonomy limits, data handling, incident response — gets answered faster once these two exist, because most of those questions are really “show me the allow-list” or “show me the log” phrased differently. I’ve written separately about the questionnaire itself, the specific lines InfoSec teams ask and what a credible early-stage answer sounds like, in why auto-filled security questionnaires kill AI deals — that post covers the conversation. This one covers the two documents you hand over when the conversation is done.
The tool allow-list: what actually goes in each row
A tool allow-list is a table with one row per tool: what that tool can reach, under which credential, at what risk class, whether it needs confirmation, and — the column most teams skip — who approved it and when. What it isn’t is a sentence on your security page saying “we restrict what our agent can access.” Below is the field set and why each one is there.
| Field | Why it’s on the row | What it proves |
|---|---|---|
| Tool name | Uniquely identifies the capability, not the code that implements it | The list maps to what the agent can do, not to file names |
| What it can reach | Named system, table, or endpoint — not “internal APIs” | The blast radius is bounded and specific |
| Credential and scope | The actual token or service account, and its permission set | The tool can’t do more than the row claims |
| Risk class | Read-only, reversible-write, irreversible-write, or spend | Reviewers can triage by consequence, not by tool count |
| Confirmation required | Yes/no — does a human gate this before it runs | Autonomy limits are a lookup, not a claim |
| Approved by, and when | A name and a date, not a team or a Slack thread | This is evidence a decision was made, not documentation of a default |
| Last reviewed | Date of the most recent re-check against current usage | The list reflects the agent as it runs today, not as it shipped |
That “approved by, and when” column is the whole difference between evidence and documentation. A list that says what the agent can do is a description of your intentions. A list that says a named person signed off on this exact scope on this exact date is a record a reviewer can act on — they can call that person, or at minimum they know someone was accountable for the decision instead of it accreting by default because a tool was easy to wire up.
A list that describes what the agent can do is documentation. A list that names who approved it and when is evidence.
The tool-call log: the field set for one record
A tool-call log record needs enough fields that someone who wasn’t in the room can reconstruct exactly what the agent did, to whom, and whether a human signed off — not a debug log line that happens to mention a tool name. Here’s the field set for a single record and why each field earns its place:
- Timestamp — when the call happened, not when the batch job that emitted the log ran.
- Actor and actor type — which agent, session, or automated job triggered the call, and whether the actor itself was a human-initiated request or a scheduled process. This is the field that answers “was a human even in this loop at all” without you having to ask.
- Tenant — which customer’s context the call ran under. In a multi-tenant product this is the single field that turns an incident investigation from a scan of everything into a filter.
- Tool invoked — the exact tool name, matching the allow-list row it came from. If the log and the allow-list use different names for the same tool, neither one is trustworthy.
- Arguments — the actual parameters passed, not a summary. This is what lets you answer “did it write to the record it was supposed to, or the one next to it.”
- Outcome — success, failure, or partial, plus the return value or error. Without this you know the agent tried something; you don’t know what happened.
- Human confirmation — whether a human approved this specific call, and who, if the tool’s allow-list row requires confirmation. This field is what makes the allow-list’s “confirmation required” column checkable against reality instead of aspirational.
- Provider request ID — where the call passed through a third-party LLM or API provider, their own request identifier, so you can correlate your log with theirs during an incident instead of guessing which of their calls was yours.
“The agent called the refund tool at 3:14pm” is a log line. A reviewer can’t do anything with it — they can’t tell whether that was a $12 refund a human approved or a $12,000 refund nobody looked at. The field set is the difference. It is also what an incident actually consumes — the postmortem of an agent failure turns on precisely this: the permissions delta between what the agent had and what it needed, and the gate that didn’t fire.
That last field has a second life on the storage side. The same record, written to a table an auditor samples, is a different problem — schema, retention clocks, whether you keep the raw prompt at all — and I take that up separately next week. This post stops at the field set, because a reviewer in a security review is asking what you can show them, not what your migration looks like.
How do you produce both without buying a governance platform?
You generate the allow-list from the code that registers your tools, and you get the log for free from the instrumentation point most agent frameworks already give you — a dedicated platform buys you a UI on top of data you should already be capturing, not the data itself. If your agent framework has a central place where tools get registered — a list of function definitions, an MCP server config, a router that dispatches tool calls — that registration is your allow-list’s source of truth. Building an MCP server in Phoenix is the worked version of this: the tools you deliberately expose there are the allow-list, in code. A short script that walks it and emits the table above, including pulling the credential scope from whatever secrets manager backs each tool, keeps the document honest because it can’t say anything the code doesn’t actually do.
The log is usually less work than teams expect, because the dispatch point that calls a tool is a single chokepoint in the code — every tool call already passes through it. Adding structured logging at that one function, with the field set above, gets you every record without touching the individual tools. If you’re already emitting application logs, the fix is making tool calls a distinct, queryable event type inside them rather than indistinguishable from every other log line — retained for a defined window, filterable by tenant and by request ID, so pulling the trail for one incident is a query, not a grep session.
The drift problem is the real reason evidence packs fail
A hand-maintained allow-list is wrong the first sprint nobody remembers to update it, because someone adds a tool and updates the spreadsheet never — the fix isn’t better discipline, it’s making the document impossible to drift from the code. This is the actual failure mode I’d flag first if you handed me your current security artifacts: not that the allow-list is missing, but that it was accurate the day someone wrote it and hasn’t been checked since. A reviewer who asks “when was this last reviewed” and gets a date from three tool-additions ago has found the same gap a hand-maintained document always produces — the list describes an agent that no longer exists.
Generating the list from source closes that gap structurally instead of procedurally. It can’t lag the code, because it is the code, read and rendered. The log has the same property for free — it can’t be stale, because it’s a record of what already happened, not a description someone has to remember to update. The only manual step left is the approval column, and that’s exactly the column that should stay manual: a person deciding to grant a tool a scope is a real decision, and automating it away would remove the one field that makes the whole document evidence instead of a config dump.
Where this fits with what you should have already built
An evidence pack documents a control layer; it doesn’t substitute for building one. I’ve made the case for the layer itself in running security at an AI-native company: per-action authorization, capability-scoped credentials, action-level audit logging, rate-limiting as a circuit breaker, and confirmation prompts on risky actions. If you haven’t built those yet, start there. The allow-list and the log are how you prove you did.
The threat model underneath them is OWASP’s LLM06:2025, Excessive Agency, which names three root causes: excessive functionality, excessive permissions, and excessive autonomy. Per-action authorization and capability scoping are the direct answers to the middle two.
Build it before the request lands
The teams that hand over a clean allow-list and a queryable log in the same call they got asked for them didn’t write either document that week — they built the agent so both fell out of the architecture, the allow-list from the tool registry and the log from the dispatch point every call already passes through. That’s the same posture a reviewer is actually grading in every other part of a security review: not whether you have zero gaps, but whether what you show them is real. If you’re heading into a review and want someone to read your agent the way the buyer’s security team will before they do, that’s the kind of engagement I run.