Personalized AI infrastructure
AI Pilot Exit Criteria: When to Scale, Pause or Stop an Agent
Set written exit criteria for an AI agent pilot: observable measures, clear thresholds and a dated review that decides to scale, pause or stop.

AI pilot exit criteria are a short, written set of conditions agreed before an agent starts work. They name the outcome the agent must produce, the threshold that counts as success, the safety evidence required, the date of the review, and what happens on each branch: scale, pause or stop. Criteria based on observable results settle the decision, while predictions about how capable future models will become do not.
Key takeaways
- Write the exit criteria before the first live task, with a named owner and a fixed review date.
- Measure what the agent did inside your own systems, such as tasks closed correctly, escalations handled and rework avoided, rather than how impressive its answers sounded.
- Define three branches in advance: scale when the evidence clears the bar, pause when results are close but unclear, and stop when fixing the problem costs more than the value it creates.
- Treat permissions, incidents and human overrides as exit conditions, not footnotes.
- Keep the working assets from a stopped pilot, including the test set, tool notes and decision record.
Why exit criteria belong before the first prompt
Most pilots end by drift. Someone says the agent seems useful, the team keeps using it, and nobody can say whether the next stage should be funded. The remedy is to decide how the pilot ends while the scope is still small and the decision is neutral. Once a few weeks of use have passed, each stakeholder holds an opinion shaped by their best or worst experience, and the criteria tend to bend toward whatever they already believed.
The NIST AI Risk Management Framework treats AI risk as an ongoing practice across the system's lifecycle rather than a single approval. A pilot is a good example of why that matters: the agent, its memory and its tools will change while the test runs, so the criteria need a schedule for re-checking, not only a finish line.
Pick measures you can observe in your own operations
A useful measure can be checked against a record your company already keeps or can easily produce. Measures that depend on how the agent sounds, or on a general sense that it is getting smarter, cannot be checked by anyone other than the person who likes it. Good candidates usually include:
- Tasks completed end to end without a person correcting the output
- Cases escalated to the right person, with the reason recorded
- Rework, meaning how often a staff member reopens or edits a result
- Complaints from customers or colleagues that trace back to the agent
- Time a staff member spends on the task, measured on a small sample before and during the pilot
Anthropic's guidance on evaluating agents and OpenAI's evals guide both describe the same discipline: run the agent against a fixed set of tasks, grade the outcomes the same way each time, and keep the results. Freeze a test set of real past cases before the pilot starts. Use live work to confirm the test set still reflects reality, and update the set only through a recorded change, so that scores from different review dates remain comparable.
Write the three branches: scale, pause and stop
Each branch should be stated in one sentence with its trigger, so that the review is a matching exercise rather than a debate. A workable version looks like this:
- Scale when the agent meets every agreed threshold on the fixed test set and on live work, no safety finding is open, and the owner can name the next role, channel or team that would be added.
- Pause when results are promising but one measure is short, the failures have a traceable cause, and the fix is a bounded change such as corrected memory content, a clearer tool description or a narrower permission.
- Stop when the threshold is still missed after the bounded fixes, when the agent depends on data the company cannot supply cleanly, or when the risk of an error exceeds what the leadership team has agreed to accept.
Illustrative scenario: a support team pilots an agent that drafts replies to billing questions. The criteria require that drafts be accepted without edits on a fixed set of past tickets, and that no draft quote a refund rule absent from the current policy. The agent passes the acceptance measure but fails the refund check in two cases, both traced to an outdated policy note in its memory. The decision is pause, not scale: the note is corrected, the test set is rerun, and the review date stays where it was set.
The rule for a result that falls between branches should be written down too. A common approach is to apply the stricter branch, so an unclear result counts as not met.
Treat safety and permissions as exit conditions
An agent that produces good results while exceeding its permissions has not passed the pilot. Safety evidence belongs in the same document as the business measures, with the same status of met, missed or unclear. The OWASP Top 10 for LLM Applications and its prompt injection guidance describe the classes of failure to check for, including instructions hidden in inputs that try to override the agent's own rules. Exit conditions can be simple:
- Every action traces to a named tool and a permission recorded in the access list.
- Any action above the approval threshold was reviewed by a person before it took effect.
- Inputs that tried to change the agent's instructions were logged and did not succeed.
- Each incident has an owner, a cause and a recorded fix before the next review.
If the company has not yet set approval limits, the approval thresholds guide for superintelligent agents explains how to raise them gradually, which keeps the safety criteria measurable. Permission checks should run on a schedule too, and the quarterly AI agent permission review gives a method and the evidence to keep.
A step-by-step plan for the exit review
The review itself should take a fixed sequence, so that the decision is reproducible and the evidence is complete:
- Step one: Write the criteria on one page, with the owner's name, each measure, its threshold, the branch rules and the review date.
- Step two: Freeze the test set of real past cases before the first live task, and grade it the same way at every review.
- Step three: Export the period's records: tasks, escalations, edits, permission events and incidents.
- Step four: Score each measure as met, missed or unclear, and write one line of evidence beside each score.
- Step five: Apply the branch the document names. If a result falls between branches, apply the stricter one.
- Step six: Record the decision, its reason, the people who approved it and the next review date in the same document.
Be Human runs this kind of review as part of managed operations. The company's agent role, memory, tools and approval boundaries are defined at the start, monitoring and recurring evaluations run during the pilot, and the evidence pack is prepared for the scale, pause or stop decision. Permission reviews and configuration changes are logged in the same record, so a pause does not erase the history of what the agent was allowed to do.
What a stop should keep, and when to revisit
Stopping well is a planned outcome, not a failure of the project. The company should keep the cleaned knowledge it gathered, the frozen test set, the tool descriptions, the permission record and the decision document. The guide to stopping an AI pilot without wasting the work walks through what to archive and what to delete. A stopped pilot can restart later, but only when the blocker named in the criteria has changed, such as a data cleanup, a clearer owner or a narrower task. Treat a new model release as a reason to re-run the test set, not as proof that the old decision was wrong. If you want to find the first task worth handing to an agent, the free Be Human AI diagnostic is a practical place to start.
Frequently asked questions
How long should an AI pilot run before the exit review? Set the date when the criteria are written, long enough to see a representative mix of work and short enough to keep the decision current. Choose the length from the volume and variety of the task, not from how quickly the technology appears to be moving. If the task arrives rarely, extend the window and record why in the document.
Can a pilot be judged on user satisfaction alone? Satisfaction is a useful secondary signal but a weak primary one, because people often like a fluent answer that turns out to be wrong. Pair it with graded outcomes on the fixed test set and with rework counts from live use. Treat any disagreement between these measures as a reason to pause and investigate.
What if the criteria are missed by a small margin? Apply the branch the document names, and do not move the threshold after the results are visible. If the miss is narrow and traceable to a bounded fix, pause, make the change and re-run the test set. If the cause is structural, stop, and keep the assets described in the earlier section.
Who should sign the exit decision? The operational owner should confirm the measures, a finance or risk representative should confirm the cost and exposure, and the executive sponsor should approve the branch. The role of an executive sponsor for an AI project explains what that person needs to review. Keep the signatures in the same record as the criteria so that the decision can be audited later.
Your next step
See where an AI agent fits in your company first.
Take the free Be Human AI diagnostic. A few questions about your work show which task to hand to an agent first, what it needs to know and where a person approves.
Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.


