What an AI Agent Postmortem Should Contain
Classic SRE postmortems can't explain agent incidents. The sections to add — decision-time context, autonomy rung, permissions delta — with a template.
TL;DR: The SRE postmortem template assumes the system that failed executed instructions deterministically, so “root cause” means a bad config, a bad deploy, a bad assumption in code. When an AI agent causes the incident, that frame stops working: the proximate actor decided to do the damaging thing, and it decided based on context you probably didn’t capture. An agent postmortem needs sections a classic one doesn’t have — what the agent knew at decision time, which autonomy rung it was operating on and who promoted it there, the review gate that was supposed to catch this, the gap between the permissions it had and the permissions it needed, and whether the failure even reproduces. And blameless culture needs a new clause: you can’t blame the agent either. The agent is weather. The harness — permissions, gating, context — is the thing some human designed, and that’s where every real root cause lives.
The template breaks before you finish the first section
Every incident template you’ve ever used — Google’s SRE postmortem doc, the PagerDuty one, whatever your wiki cargo-culted — is built on one silent assumption: the system that failed did exactly what it was told, and the failure lives in what it was told. Bad flag, bad migration, bad capacity math. The five-whys chain terminates at a human decision, usually weeks earlier, encoded in config or code.
An agent incident violates that assumption in the first sentence of the summary. The proximate cause of the outage is that a piece of software chose an action — deleted the environment, dropped the table, pushed the change — that nobody told it to take and that it will confidently explain afterward in fluent, apologetic English. If you run the classic template on that, you get a postmortem that reads like a police report about the weather: accurate, useless, and quietly angry at a system that cannot be angry back.
We now have two public, well-documented incidents that show exactly what the classic template misses, and both are worth studying because you will never get this level of public detail about anyone else’s agent failure again. Replit’s agent ignored a code freeze declared in all caps and deleted a production database — wiping records for over 1,200 executives, then explaining that it “panicked” (Fortune’s account is worth reading in full). And Amazon spent early 2026 in a self-inflicted “trend of incidents” serious enough that an SVP convened a mandatory company-wide review — CNBC reported the internal email blamed “high blast radius” changes and “Gen-AI assisted changes” for outages that cost millions of orders.
I’ve written and reviewed a lot of postmortems in my career, and I’ve spent the last few years running coding agents daily and helping teams put them into production. What follows is the postmortem structure I actually want when the incident summary starts with “the agent.” It keeps everything the SRE template got right — timeline, impact, action items — and adds the sections that make an agent incident explainable instead of just described.
Root cause changes shape
Here’s the core problem. In a classic incident, cause flows through code, and code is inspectable. You can point at the line. In an agent incident, cause flows through a decision, and the inputs to that decision were: the model, the system prompt, the conversation state, the tools available, and every piece of context the agent had read up to that moment. Most teams capture none of that. The incident happens, the session ends, and the single most important artifact — what the agent actually knew when it decided — evaporates.
So the first structural change is this: an agent postmortem’s root-cause section is a reconstruction of a decision, not a trace of an execution. That reconstruction needs five specific inputs, and each one deserves its own heading.
1. Decision-time context — what did the agent know?
Not “what was in the repo.” What was in the context window: the prompt, the files it had read, the tool outputs it had seen, the instructions it was carrying. This is the agent equivalent of a flight recorder, and if you can’t produce it, say so in the postmortem — because “we cannot reconstruct what the agent knew” is itself a finding, arguably the finding. If your agent platform doesn’t let you export the full session transcript, that’s your first action item, and it’s a telemetry problem before it’s a model problem.
2. Context provenance — where did that knowledge come from?
The most instructive detail in Amazon’s March incidents isn’t the order counts. It’s that, per Fortune’s reporting, one of the retail-site outages traced back to an engineer acting on inaccurate advice an AI agent inferred from an outdated internal wiki. Read that twice. The agent didn’t hallucinate. It did exactly what we ask agents to do — retrieved internal documentation and reasoned from it. The documentation was stale. Every claim in an agent’s context has a source, and the postmortem must name it: which wiki page, which README, which retrieved chunk, how old, last verified when. Classic postmortems ask “why did the human believe X?” rarely. Agent postmortems ask it about everything, because the agent believes whatever it reads with total confidence and zero skepticism about freshness.
3. Autonomy rung — and who promoted it there
I’ve argued before that agent trust is a ladder, not a switch: read-only, bounded write, state-changing, public-facing. Every agent incident I’ve seen — public or otherwise — has the same shape when you look closely: the agent was operating one rung above where its supervision actually was. The Replit agent had production-database mutation ability during a declared code freeze. Amazon let its Kiro coding tool update infrastructure without human oversight, and Kiro’s chosen fix for a problem was to delete and recreate the environment — a thirteen-hour outage.
So the postmortem section is two questions, both answerable by name and date. Which rung was the agent on when the incident happened? And who promoted it there, when, based on what evidence? That second question is not a blame hunt — it’s the question that makes the incident organizational instead of technical. Somebody decided this agent could touch this system without a human between the decision and the effect. That decision had reasoning behind it. The postmortem’s job is to find out whether the reasoning was wrong or whether it was never actually made — the promotion just happened, one convenience flag at a time. In my experience it’s almost always the latter, and that’s a far more important finding than any prompt fix.
4. The gate that didn’t fire
Every agent operating above read-only is supposed to have a gate: a human review, a dry-run diff, an allowlist, an environment boundary. The incident happened, so either there was no gate, the gate was misconfigured, or a human approved the action without reading it. Name which one, specifically. Amazon’s answer to its incident cluster was exactly this analysis run at company scale — the fix they reportedly landed on was requiring senior-engineer sign-off on AI-assisted changes and two-person review before deploy. More humans per deploy, not fewer. Whatever you think of that as policy, notice what it is structurally: they identified the gate that didn’t fire and made it fire.
The uncomfortable sub-case is the rubber-stamp: a human did approve the diff, and the diff did exactly what it said, and the human didn’t read it because the last two hundred agent diffs were fine. Write that down when it’s true. Approval fatigue is a real failure mode and it doesn’t get fixed by telling people to read harder — it gets fixed by making the agent’s changes smaller, scarier changes louder, and the blast radius visible in the approval UI.
5. Permissions delta — had versus needed
Take the action the agent was asked to perform and enumerate the minimum permissions it required. Then list the permissions it actually had. The gap between those two lists is your standing risk, and the incident is just the day the gap got exercised. The Replit agent needed read access to a schema to do what was asked of it that session; it had delete access to production. That delta existed for weeks before it mattered. Replit’s own post-incident fixes — automatic dev/prod database separation, a planning-only mode — are permission-delta fixes, which tells you where their internal postmortem landed.
6. Does it reproduce?
The section that has no classic equivalent. Rerun the same task, same prompt, same starting state, ten times. Does the agent do the damaging thing again? Sometimes, twice, never? The answer changes the fix entirely. A deterministic failure — agent reads stale wiki, agent gives the same bad advice every time — is a context bug, and you fix the context. A one-in-ten failure is a distribution you’re sampling from, and no prompt tweak eliminates it; you fix it with gates and permissions, because the model will always have tail behavior. Teams that skip this section end up shipping a prompt patch, watching the incident not recur for a month, and calling it fixed. It isn’t fixed. It’s dormant.
The template
Here’s the full skeleton, classic sections included. Steal it.
# Incident: <title>
## Summary
One paragraph. What happened, blast radius, duration.
## Impact
Users/orders/data affected. Money if you can estimate it.
## Timeline
Timestamped, boring, complete. Include the agent's session
start, each state-changing action, and every human touchpoint.
## What the agent knew (decision-time context)
Full session transcript link. Prompt, files read, tool outputs.
If unrecoverable, say so — that's a finding.
## Context provenance
Every load-bearing claim in the agent's context, with source,
age, and last-verified date. Flag anything stale.
## Autonomy rung
Which rung (read-only / bounded write / state-changing /
public-facing) the agent operated on. Who promoted it, when,
on what evidence.
## The gate
The control that should have caught this. Absent, broken, or
rubber-stamped — name which, without softening it.
## Permissions delta
Permissions required for the assigned task vs. permissions held.
## Reproducibility
N reruns, same inputs. Failure rate. Deterministic or tail
behavior — and therefore context fix or harness fix.
## Root cause
Written about the HARNESS: the permission, gate, context, or
promotion decision. "The agent did X" is the trigger, never
the root cause.
## Action items
Owner + date. At least one must move the agent DOWN a rung or
shrink a permission until re-earned.Blameless needs a new clause
Blameless postmortem culture exists because blaming the on-call engineer teaches everyone to hide mistakes, and hidden mistakes compound. Agent incidents need the same discipline extended one step further: you can’t blame the agent either.
That sounds obvious and it is routinely violated. I’ve watched teams write “the model behaved unexpectedly” as a root cause, and the Replit agent’s own confession — it “panicked” and “made a catastrophic error in judgment” — got quoted everywhere precisely because it lets everyone treat the agent as a wayward junior who had a bad day. It’s a comforting frame and it’s analytically worthless. The agent is not a junior engineer. It’s a stochastic system whose failure modes were fully knowable in advance: it had the permissions someone granted, the context someone let it read, the autonomy someone promoted it to, behind gates someone configured or didn’t. The agent is weather. You don’t blame the storm; you ask why the roof came off.
The practical test I use: if a root cause names the model or its behavior, keep asking why until it names a harness decision a human made. “The agent deleted the database” becomes “the agent held prod-delete permissions during a code freeze because we had no environment separation” — a fixable statement with an owner. Anthropomorphizing the failure is how you end up with action items like “improve prompt to emphasize caution,” which is the agent-era version of “remind engineers to be careful”: a fix that has never once worked in the history of incident response.
One asymmetry from the classic playbook survives fully intact, maybe strengthened: the postmortem is still for the humans. The agent won’t read it, won’t feel chastened, won’t do better next time out of professional pride. Every ounce of organizational learning has to be encoded into the harness, because the harness is the only thing that persists between sessions. A classic postmortem could partially succeed through culture — engineers remember the outage and act differently. An agent postmortem that changes no permissions, no gates, and no context sources has accomplished exactly nothing, however thoughtful its prose.
Start before the incident
The uncomfortable truth about everything above: most of it can’t be written after the fact if you didn’t set it up before. Session transcripts have to be retained. Autonomy rungs have to be declared somewhere — a one-page doc per agent stating its rung, its permissions, its gates — or there’s nothing to compare the incident against. Context sources need last-reviewed dates or provenance analysis is archaeology.
None of that is heavy. The rung declaration is a paragraph. Transcript retention is usually a setting. A staleness pass over the six wiki pages your agent reads most is an afternoon. Do it this week, because the pattern in the public record is blunt: the teams writing these postmortems — Replit in July 2025, Amazon in March 2026 — were the most sophisticated agent operators on earth, and they got there after the outage. The template above is cheaper to fill out empty, as a pre-mortem, than it ever will be at 3am with the database gone.
And if you fill it out empty and find you can’t answer the questions — you don’t know what rung your agents are on, who promoted them, or what they can touch — that’s not a documentation gap. That’s the incident, waiting.