Tabletop Exercise for a Superintelligent Agent Failure: A One-Hour Script
A one-hour tabletop script for practicing how your team detects, pauses and recovers from a wrong AI agent action, with roles, timings and decision points.

An AI incident tabletop exercise is a structured, one-hour discussion in which a small team works through a realistic wrong action by an agent and practices how to notice the problem, pause the agent's authority and restore the affected work. The point is not to produce a perfect report but to find where the warning signal, the pause control or the decision owner is missing before a real failure happens. The script below needs a facilitator, a note taker and the people who would actually act on the agent in production.
Key takeaways
- A useful tabletop exercise tests the path from first signal to paused agent, not the quality of the underlying model.
- Name the person who can pause the agent before the session starts, and confirm that the pause works in a test environment.
- Read injects during the hour that widen the damage, so the team has to make a second decision with less information.
- Record every decision with a time and a reason, so the review can separate what the team knew from what it assumed.
- Turn each gap into a named owner and a restart condition agreed before the agent runs again.
Why rehearse a wrong agent action before it happens
An agent that can read records, write messages or change data can be wrong in ways that look ordinary at first. It might send a correct template to the wrong group, create the same ticket twice, or report success on an action that never changed the record. In most teams, the path from "something looks off" to "the agent is paused and a named person owns the decision" is informal. People ask in a chat channel, someone checks a dashboard that nobody reads daily, and the agent keeps running while the question circulates.
A tabletop exercise makes that path explicit without touching live systems. It is a discussion around a fictional but plausible failure, and it forces the team to answer concrete questions in order: what would we notice, who would we tell, what do we stop, what do we check, and who decides when the agent may resume. The National Institute of Standards and Technology frames AI risk management as an ongoing practice across the system's life, not a one-time review before launch. A tabletop exercise is one of the cheapest ways to turn that idea into a habit that operations people can actually perform.
For the broader response process, including severity levels and rollback steps, the AI agent incident response playbook describes the controls that the exercise should test.
Before the session: scope, roles and the authority to pause
Choose one agent and one workflow. A support agent that drafts and sends replies, a bookkeeping agent that proposes entries, or a scheduling agent that books appointments are all good candidates. Write down what the agent is allowed to read, what it can write, and which actions require a human approval. If that list does not exist, the exercise will spend its time discovering the boundary, and that is a finding in itself.
Assign five roles before anyone meets:
- Facilitator: reads the injects, keeps time and does not defend the system or the people who built it.
- Agent owner: the business leader who decides on customer, money or reputation impact.
- Operator: the person who can pause the agent, disable a tool or revoke a credential without waiting for a developer.
- Observer: someone from support, finance or operations who would notice symptoms in daily work.
- Note taker: records timestamps, decisions, open questions and who was asked.
Before the session, confirm that the pause exists and that the operator knows how to use it. A pause should stop write actions while leaving read access and the activity log intact, so that the team can investigate. Test it in a sandbox, and write the steps down in one page. If the only way to stop the agent is to ask an engineer who may be unavailable, record that as the first gap to close. The human escalation guidance for agent failures explains how to send the decision to a person with enough context to act, rather than forwarding a raw log.
The one-hour script, minute by minute
The timings below assume a single group of five to eight people. Adjust the blocks if the group is larger, but keep the decision points, because they are where the exercise produces value. The facilitator should read each inject aloud and avoid giving answers.
- 0:00 to 0:05: State the ground rules. The scenario is fictional, nobody is being judged, and participants should speak as they would at work on a normal day.
- 0:05 to 0:15: Read inject one (see the next section). Each participant says what they would look at first and who they would contact.
- 0:15 to 0:25: The team chooses one of three actions: pause all write actions, limit the agent to drafting only, or continue with a specific check. The note taker records the choice and the time it was made.
- 0:25 to 0:35: Read inject two, which reveals that the agent acted on more items than the first signal suggested. The team identifies the affected items and how it would find them, preferably with a query rather than from memory.
- 0:35 to 0:45: Discuss recovery. Decide what to correct, what to reverse, whether customers or staff need a message, and who approves that message before it is sent.
- 0:45 to 0:55: Agree on restart conditions. What evidence is needed before the agent resumes, which actions stay disabled, and whether the agent returns with reduced authority for a period.
- 0:55 to 1:00: The note taker reads the decision log aloud. The team names three follow-ups, each with one owner.
The most common mistake in the first run is to skip the pause decision and move straight to fixing the cause. Insist on the pause first. A wrong action that is still repeating is more urgent than an explanation of why it happened.
Injects: scenarios to read aloud
The following scenarios are illustrative, invented for training, and do not describe any real company.
Illustrative scenario, inject one: a support agent drafts replies for a small online shop. After a template change, it begins adding a previous warehouse address to return instructions. The first signal is a customer who writes to say the address does not exist. The team must decide whether one complaint is enough to pause, and whom to ask first.
Illustrative scenario, inject two: the operator reports that the agent sent the corrected template to a group of customers who had already received the earlier version, and that a subset may have been given two sets of instructions. The team must identify the affected customers from the message log, not from the agent's own summary, which may be incomplete.
Illustrative scenario, inject three, optional: the tool that updates order notes returns success, but a spot check shows that the notes were not changed. The team must decide whether the agent can be trusted to report its own success, and what independent check would confirm an action.
Write your own injects using the workflow your team really runs. Good injects are specific enough to force a decision and vague enough that the team must investigate. Avoid injects that have an obvious villain or a single correct answer.
What to check in the run afterwards
The review matters as much as the hour itself. Within a day, the note taker and the facilitator should compare the decision log with the checklist below. Each "no" is a gap, not a verdict on the people involved.
- Was the first signal named within the first five minutes of the inject?
- Could the operator pause write actions without asking for engineering help?
- Was the person who decided to pause different from the person who built the agent?
- Were affected items found through a query or a log, rather than from memory?
- Did a human approve any message to customers or staff before it was sent?
- Did the team state a restart condition that someone can check?
The review also depends on evidence. Traces and logs are what allow a team to reconstruct what the agent saw, which tool it called and what came back. The OpenTelemetry documentation on observability concepts describes how traces, metrics and logs support that kind of reconstruction. If the team cannot answer "what did the agent do, in order, on the affected items" from its records, the most valuable follow-up is usually better logging, not a new prompt. The observability guidance on what to log lists the fields worth keeping for each action.
How Be Human handles this in managed operations
Be Human designs personalized agents with their memory, tools, channels and permissions defined in advance, and with approval boundaries written as operating rules rather than left to judgment in the moment. Managed operations include monitoring, incident response, permission review and controlled improvements, so the pause path and the write permissions are checked on a recurring basis, not only at launch. When a tabletop exercise reveals a gap, that gap can become a scheduled change with an owner, a test and a record of what changed. The monthly routine for agent support and maintenance shows how these checks fit into ordinary operations.
Turning findings into the next change
Every exercise should end with a short list of changes, and each change should have one owner and one condition for closing it. A missing pause becomes a documented procedure and a tested switch. A missing decision owner becomes a named role with an approval rule. A missing log field becomes an entry in the audit trail. Avoid vague items such as "improve monitoring" because nobody can tell when they are finished.
Keep the decision log with the agent's change history, so the next person can see why the agent's authority was reduced or restored. Run the exercise again after any major change to the agent's tools or permissions, and at a regular interval that your operations team can sustain. The first run usually reveals the obvious gaps. The second run tests whether the fixes hold under pressure.
If you are not sure which task in your business is ready for an agent, and which should wait until its boundaries are clearer, the free Be Human AI diagnostic can help you find the first task worth handing to an agent.
Frequently asked questions
How long does an AI agent incident tabletop exercise need? It needs about one hour for a single agent and one workflow, plus fifteen to thirty minutes afterwards for the review. Larger systems can be split into several sessions, each covering one agent or one decision path. Keep the session short enough that people can attend without rescheduling their daily work.
Who should attend the exercise? Include a facilitator, the business owner who makes decisions about the agent's impact, a person who can pause the agent or its tools, someone who would notice symptoms in daily work, and a note taker. Engineers can join, but the session should not depend on them to make business decisions. The aim is to test whether the organization can act without waiting for one specialist.
Does the exercise need access to live systems? No. The scenarios are read aloud and discussed, and the pause control should be tested separately in a sandbox or a staging environment before the session. Running the exercise against production data can create the very problem you are trying to rehearse. Keep the test in a controlled environment and record what was checked.
How often should a team run this exercise? Run it after a significant change to the agent's tools or permissions, and on a regular schedule that the operations team can sustain. Many teams find that a few runs per year is enough to keep the pause procedure current, but the right frequency depends on how often the agent's authority changes. Each run should produce a short list of follow-ups that someone is responsible for closing.
Your next step
See where an AI agent fits in your company first.
Take the free Be Human AI diagnostic. A few questions about your work show which task to hand to an agent first, what it needs to know and where a person approves.
Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.


