AI Agent Incident Response Runbook: Severity, Owners and Rollback

An AI agent incident response plan is a short written runbook that defines what counts as a failure, how serious it is, who acts first, and how to contain and reverse the damage. For a business agent that sends a wrong message or takes a wrong action, the plan should set four severity levels, assign owners before anything breaks, and list rollback steps in the order they must be performed. The sections below give you a working version you can adapt this week.

Key takeaways

• Write the runbook before the first incident, while nobody is under pressure to improvise.
• Classify failures by customer impact and reversibility, not by how technical the cause turns out to be.
• Name one incident owner and a separate approver for outbound corrections, so no one waits for a decision nobody owns.
• Contain first by pausing the outbound channel or removing write permission, then investigate.
• Rehearse rollback against test fixtures, and restore agent authority only after a documented review.

What counts as an agent incident

An agent incident is any event where an AI agent produces an effect that the business did not intend and cannot ignore. The most common forms are a wrong message sent to a customer, supplier or employee, a wrong change in a business system such as an order, booking, price field or refund, and a tool call that reaches data or actions outside the agent's approval boundary. The OWASP Top 10 for LLM Applications groups the last case under excessive agency, which is a useful label because it points to permissions as the root of most serious incidents.

Near misses belong in the same process. If a reviewer catches a draft before it leaves the outbox, or an approval step blocks an action, record it. These events cost little to handle and reveal which checks are working. They also become test cases later.

Four severity levels

Severity should describe what happened to people and money, not how much effort the fix takes. Four levels are enough for most small and mid-sized operations:

• Severity 1: an external party was harmed or an irreversible action was taken. Examples include a payment sent, personal data disclosed, a legal or safety commitment made, or a regulated notice sent. The incident owner contains the agent immediately and informs a senior leader the same hour.
• Severity 2: a wrong message or action reached one or more customers or suppliers and can still be corrected. Examples include a wrong delivery window, a wrong price quoted, or a booking moved without consent. Containment happens in the same working session, and corrections are approved before they go out.
• Severity 3: an internal error or a single external message with minor effect, caught by staff after it happened. Examples include a wrong status on an internal record or a reply with a misspelled product name that the customer did not act on. Fix it within the day and log the cause.
• Severity 4: a near miss blocked by a check, a reviewer or an approval gate. Log it, attach the input and output, and add it to the evaluation set.

When the level is unclear, choose the higher one. You can downgrade later with a written reason, but an incident that was under-classified at the start often gets less attention than it deserves.

Name the owners before the first incident

An incident runbook fails most often at the first decision, when someone asks who can pause the agent. Write the answer in advance. A small team can cover several roles with one person, but each role still needs a named holder and a named backup.

• Incident owner: the operations leader who decides on containment, severity changes and internal communication. This person has authority to pause the agent without further approval.
• Agent owner: the business lead of the process the agent supports. This person judges whether an output was wrong and what the correct outcome should have been.
• Technical operator: the person who maintains the agent's tools, credentials and integrations. This person executes the rollback and checks that permissions were actually removed.
• Outbound approver: the person who signs off on any correction sent to customers or suppliers. Keep this person separate from the agent owner so corrections are reviewed by someone with a different view.
• Managed operations contact: when a provider runs the agent under a managed arrangement, the provider's operations lead is listed here with the escalation path, so the runbook covers both sides of the handoff.

Also write the escalation rule for each level. The human escalation guide for AI agent failures describes how to send the decision to a person rather than a messy transcript, which saves time when a severity 1 call happens at an inconvenient hour.

Contain first, investigate second

Containment is the only step that must happen before understanding. The goal is to stop new harm while the facts are still being gathered. Use these steps in order:

• Pause the agent's outbound channel. Do this even when the cause is not yet known, since a paused agent can be resumed in minutes but an unpaused one can send more wrong messages in the meantime.
• Remove write permission for the affected tool or system. Revoke or rotate the credential if the agent may have been misused, and record the time of each change.
• Open the incident record with the timestamp, severity, agent version, prompt or configuration version in effect, and the name of the incident owner.
• Preserve evidence. Export the inputs, tool calls, outputs and approval records for the affected window. Do not edit or delete original records, and do not rely on the agent's own summary of what it did.
• Notify the people who must know internally, including the agent owner and the outbound approver, using the same incident record so nobody works from a different version of events.

Do not restart the agent during containment to see whether the problem returns. A restart changes the state you are trying to examine.

Roll back and correct, step by step

Rollback restores the business to the state it should be in, and correction tells affected parties the truth. They are separate tasks with separate owners. Work through the following sequence once containment is in place.

• Scope the damage. Use action identifiers and the time window from the logs to list every affected message, record and transaction. A list built from memory will miss items.
• Reverse reversible actions. Restore the previous field values, cancel the unintended booking or move it back, and check each reversal against the original record.
• Check for duplicates. If the agent retried a write, confirm that the system processed it once. Providers such as Stripe describe idempotency keys for exactly this purpose, and the guide on idempotent agent actions explains how to design actions so that a retry cannot repeat a refund.
• Draft corrections for affected parties. The outbound approver writes or approves each message by hand. Corrections should state what was wrong, what is now correct, and what the recipient needs to do, if anything.
• Restore the previous configuration or model version only after comparing the two in a test environment. Do not assume the earlier version was safe.
• Keep the agent in shadow mode after restoration. It drafts actions, a person reviews them, and authority returns in steps.

Illustrative scenario: an agent that confirms delivery windows to customers starts sending the previous week's window after a template change. Containment pauses outbound messages within the hour. Scoping uses the message log to find every confirmation sent since the change. The outbound approver sends one short correction per affected customer, the technical operator reverts the template, and the agent owner checks three sample deliveries before the agent returns to shadow mode.

Review, reopen authority and keep the runbook alive

A review should answer four questions in writing: what happened, why the checks did not stop it, which permission or boundary should change, and which test case now covers the failure. Keep the review blameless for individuals but specific about the system. If the agent had permission to send without a second look, the fix is usually a narrower permission, not a warning to the team.

Reopening authority is a decision, not a timer. The incident owner and agent owner agree on the evidence that the failure is fixed, usually a set of replayed cases that now pass, and the agent returns to supervised operation before it returns to full authority. The NIST AI Risk Management Framework is a helpful reference for framing this work around governance, measurement and management rather than around a single fix. For the operating notes that should stay current after the review, see the guide to AI agent runbook operations documentation.

Be Human designs these boundaries as part of a first engagement, defining the agent's memory, tools, channels and approval steps, and it can run monitoring, recurring evaluations, permission reviews and incident response as managed operations. The provider's role is to keep the runbook current with the agent as it changes, not to take the decision away from the business. If you want to know which task is the right first candidate for an agent with these controls in place, take the free Be Human AI diagnostic to find the first task worth handing to an agent.

Frequently asked questions

How is an AI agent incident different from an ordinary software bug? An agent can produce a wrong output that looks correct, and it can take several actions in sequence before anyone notices. Because of that, the runbook must cover messages and decisions as well as code, and it must rely on logs of inputs, tool calls and approvals. Containment usually means pausing the agent's authority rather than rolling back a release.

Should we shut the whole agent down during an incident? Not necessarily. Pause the outbound channel and remove write permission for the affected tool, which stops new harm while leaving read access for investigation. A full shutdown is appropriate when the failure affects every workflow the agent runs, or when the cause is unknown and the agent can reach sensitive data.

Who should be allowed to restart an agent after a Severity 1 incident? The incident owner should decide, based on a written review and replayed test cases showing the failure is fixed. The agent owner should confirm the business outcome is correct, and the technical operator should confirm the permissions are as intended. Restart in shadow mode first, then restore authority in steps.

Do we need to tell customers when an agent sends a wrong message? If the message affected a decision, a delivery, a payment or a commitment, a correction is almost always needed, and it should be written and sent by a person. Keep it short, state what was wrong and what is now correct, and tell the recipient what to do if anything. Record each correction in the incident file so the team can show what was sent and when.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.