Reviewing an AI Agent Like a Role: Goals, Evidence, Feedback, Changes

To run a performance review for an AI agent, treat the agent as a role with a named human reviewer. Compare what the role was meant to do against evidence from real work, turn your feedback into specific changes, and record who approved them. The review is a recurring business conversation, not a one-time technical test.

Key takeaways

• Review the role, not the model: the question is whether the work the agent does is still useful, safe and worth its cost.
• Name one human reviewer who owns the outcome of the role, and write that name down before the first review.
• Bring evidence from real work: sampled tasks, human corrections, escalations, incidents and usage, not impressions.
• Convert every finding into a concrete change to instructions, knowledge, tools, permissions or process, with an approver.
• Keep the rhythm short and regular so the review stays a habit and does not become an event that follows a failure.

Why does an AI agent need a performance review?

Most agents do not fail loudly. They drift. The business changes a policy, a supplier changes a form, a tool is updated, a new product line appears, and the agent keeps working from what it knew at launch. Nobody complains, because the output still looks fluent. Meanwhile people begin to quietly redo parts of the work, or stop sending the agent the harder cases.

A review makes that drift visible. It also settles a question that tends to stay vague: who is responsible for this agent being good at its job? With a human employee, the manager answers that question. With an agent, the answer is often a shrug between the person who requested it, the person who built it and the people who use it.

The familiar review format fits well because it asks four plain questions. What was this role meant to achieve? What actually happened? What should be said about it? What changes now? None of these needs a technical background. They need a calendar slot, access to evidence and someone with authority to decide.

If you are still defining the role itself, start with a job description for your first AI agent. A review is only as sharp as the description it is measured against.

Who should be the named reviewer?

Choose a person on the business side who owns the result the agent contributes to. For an agent that drafts replies to customer questions, that is usually the head of support, not the person who configured the tools. For an agent that prepares supplier paperwork, it is the operations lead who would feel the problem first.

The reviewer needs three things:

• Authority to approve changes to the role, within the limits the company has set.
• Enough familiarity with the work to judge whether an output is good, not just whether it is well written.
• Time reserved on a calendar, so the review does not depend on goodwill.

Around the reviewer sit supporting roles. A technical operator brings the logs, usage and incident history. One or two everyday users bring what it is like to work alongside the agent. Someone responsible for access rights confirms that permissions still match the job. In a small company one person may hold several of these seats, and that is fine as long as the reviewer is named.

Write the reviewer's name into the role description and into whatever channel the agent is announced in. If the person leaves, the reviewer seat should be reassigned the same week, which is the same continuity problem described in when your AI champion leaves.

How do you restate the goals before the review?

The first section of any review is goals, and for an agent they are often missing or too soft. Phrases such as "help the sales team" cannot be reviewed. Before the meeting, restate the role as a small set of goals, ideally three to five. For each goal, write four things:

• The task: what the agent does, in plain words.
• The quality bar: what a good result looks like to the person who receives it.
• The observable measure: something a reviewer can check in the record of work.
• The boundary: what the agent must not do, or must hand to a person.

Here is an example for an agent that triages a shared inbox. Task: sort incoming messages and draft a first reply for routine questions. Quality bar: the draft answers the actual question, uses the current policy and needs only light editing. Measure: the share of drafts a person sends with light or no edits, judged on a sampled set. Boundary: anything involving a refund decision, a legal complaint or a request to change account details goes to a person with a short summary.

Notice that the measure is something a person can observe, but it does not need to be a precise number to be useful. If you set success criteria when the agent was piloted, reuse them and ask whether they still describe the job. The article on success criteria for an AI agent pilot covers how to write them so they survive past launch.

Also add one goal that is easy to forget: the goal for the agent's relationship with people. Do colleagues trust it enough to use it, and do they know when to override it? An agent that is accurate but ignored is not performing.

What evidence should you bring?

A review without evidence becomes an argument between impressions. Prepare a small evidence pack before the meeting. It should come from the actual record of work, not from a demonstration.

• A sample of completed tasks, drawn without cherry-picking, with the reviewer reading a handful from each goal.
• Human corrections: where people edited, rejected or redid the agent's output, and why.
• Escalations and handoffs: how often the agent passed work to a person, and whether it did so at the right moments.
• Incidents and near misses: anything that touched a boundary, even if no harm followed.
• User feedback: short comments from the people who work with the agent every week.
• Usage and cost against the budget set for the role.
• The change log: what was modified since the last review, and what effect it had.

To gather most of this, the agent's work needs to be observable. The general idea in observability is that you can understand what a system did from the records it leaves behind. The [OpenTelemetry observability primer](https://opentelemetry.io/docs/concepts/observability-primer/) explains that idea clearly, and it applies directly: if you cannot reconstruct what the agent did with a given request, you cannot review it. For regulated settings, see also the notes on audit trail requirements.

It also helps to keep a fixed set of representative tasks, including some difficult ones, and run them again before each review. Comparing results across reviews shows whether a change helped or quietly hurt. Anthropic's article on [evals for AI agents](https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents) describes how teams build such task sets and why reading the actual transcripts matters as much as any score. The reviewer does not need to build this alone, but should insist that it exists.

Finally, include the quiet evidence: what work stopped arriving. If users have stopped sending a category of request to the agent, that is a finding, even if no one reported a problem. The method in collecting employee feedback on an AI agent is a good way to surface it.

How do you turn feedback into changes?

A person who receives feedback absorbs it and behaves differently next week. An agent does not work that way. Telling it in conversation that it was wrong may help for that session, but it changes nothing durable. Real feedback must land somewhere: in its instructions, its knowledge, its tools, its permissions, or in the process around it.

So the discipline in this part of the review is diagnosis. For each issue found in the evidence, ask which of these causes is most likely, then pick the matching response:

• The instruction was unclear or contradicted another one: rewrite the instruction and add an example of the correct behavior.
• The agent lacked current knowledge: update or add the source material it draws on, and name who keeps it current.
• A tool was missing, unreliable or too broad: fix or replace the tool, or narrow what it can do.
• The permission was wrong, too wide or too narrow: adjust access through the permission process, not informally.
• The task is not a good fit for an agent: move it back to a person, or split it so the agent handles only the predictable part.
• The surrounding human process is the problem: fix the handoff, the intake form or the approval step, not the agent.

The last two are the ones teams skip. Sometimes the best outcome of a review is to shrink the role, and that is a success, not a failure.

Every change should go through a visible request and approval path, including a way to roll back. Otherwise reviews produce a pile of untracked edits that make the next review harder. The process in requesting and approving changes to a managed AI agent is a workable model. For permission findings specifically, use the steps in the AI agent permission review.

How do you run the review meeting, step by step?

Keep the meeting short and the preparation thorough. The following plan works for a first review and can be trimmed later.

• Step one: send the evidence pack to the reviewer and the supporting roles a few days ahead, with the restated goals on the first page.
• Step two: open with the goals. Ask whether each still describes the job, and amend any that no longer do before judging performance.
• Step three: go through the evidence goal by goal. The reviewer states a rating in plain words: meets the bar, partly meets it, or does not, with one sentence of reasoning.
• Step four: review boundaries separately. List every near miss and confirm the agent stayed within its limits.
• Step five: collect user voices. One or two everyday users say what helps, what annoys them and what they now avoid.
• Step six: diagnose each gap using the causes listed above, and assign each a change with an owner.
• Step seven: decide the outcome for the role as a whole: continue as is, continue with changes, narrow the scope, expand the scope, or pause.
• Step eight: write a one-page record with the date, the reviewer, the decisions and the changes approved, and file it where the next reviewer will find it.

Expansion deserves caution. If the agent is doing well, the temptation is to give it more tools and wider access right away. Treat that as a separate change request with its own check, not as a reward granted in the meeting.

What does a review look like in practice?

Illustrative scenario: a mid-sized distribution company has an agent that drafts replies to routine questions about order status and delivery windows. The named reviewer is the customer operations manager. At the first review, the sampled drafts are accurate on order status but often cite an old delivery policy. Users report that they now rewrite any draft that mentions delays. The escalation list shows the agent correctly handed off every damage claim.

The reviewer's diagnosis is simple. The knowledge source for delivery policy was never updated after a change in the carrier arrangements, so this is a knowledge gap with a named owner on the operations side. One goal, replying to delay questions, is rated as partly meeting the bar. The decision is to continue with changes: update the policy source, add three corrected examples to the instructions, and rerun the fixed task set before the next review. Nothing about permissions changes. The one-page record notes all of this, and the next meeting starts by checking whether people stopped rewriting delay replies.

Nothing in that scenario required advanced technical knowledge. It required a goal, a sample, a diagnosis and an owner.

Be Human handles this as part of managed operations. A first engagement defines the role, the memory and tools it needs, the approval boundaries and an observable success criterion, which gives the review its starting point. Afterwards, the recurring work can include monitoring, recurring evaluations, permission review and controlled improvements, and the provider prepares the evidence pack so the named reviewer can spend the meeting on decisions. The reviewer stays on the client side and holds the authority to approve changes. For a view of the recurring work, see what managed AI services include and the first months with a managed AI agent.

For a broader frame on how risk and oversight fit around reviews like this, the [NIST AI Risk Management Framework](https://www.nist.gov/itl/ai-risk-management-framework) is a useful reference for governance conversations.

Frequently asked questions

How often should we review an AI agent? Hold a fuller review soon after launch, when the gap between plan and reality is largest, then move to a regular rhythm that matches how fast the work changes. Between those meetings, a short check of incidents, corrections and usage is enough. If a policy, tool or process the agent depends on changes, review earlier.

Can the agent write its own review? It can help by summarizing the record of its work and flagging patterns, which saves preparation time. It should not be the judge of its own performance, because the reviewer needs to read real samples and hear from the people affected. Treat any self-summary as a draft to be checked against the evidence.

Does the reviewer need technical skills? No. The reviewer needs to understand the work well enough to judge a result and the authority to approve changes. Technical operators supply the logs, the diagnosis of causes and the implementation, while the reviewer decides what the role should achieve and what is acceptable.

What if the review shows the agent is not worth keeping? That is a legitimate outcome, and it is better reached through a review than through slow neglect. Narrow the role, return the task to people, or pause the agent, and record why. Retirement should be handled deliberately, including access removal and saved records, as covered in AI agent lifecycle management.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.