Skip to content
be-human.ai
← Back to the journal

Human oversight

Human Approval as Superintelligence Grows: Raising Autonomy Safely

Approval for AI agents should loosen in stages, each change backed by logged evidence, a named signer and a rollback path. Here is how to set each step.

Be Human typographic cover with the title: Human Approval as Superintelligence Grows: Raising Autonomy Safely
Conceptual illustration created with AI · Be Human

Human approval for an AI agent should loosen in stages, and each stage should change only when recorded evidence shows that a defined class of action has gone right under observation. Start with a person approving every consequential action, move to sampled review once the record is clean, and reserve approval for exceptions only after behavior, tools and failure cases have been tested and logged. Each loosening needs a named owner, a written criterion and a tested way to tighten approval again.

Key takeaways

  • Approval is a control you adjust over time, not a setting you choose once.
  • Loosen one class of action at a time, never the whole agent at once.
  • Evidence should be specific: logged outcomes, reviewed samples and tested failure cases.
  • Every reduction in review needs a named signer, a written criterion and a rollback trigger.
  • Returning to full approval should be quick and rehearsed, not a debate held during an incident.

Why approval cannot be a one-time setting

A common pattern looks like this: a team sets approval rules during a pilot, the agent works well, and the rules are never revisited. Later on, the agent has new tools, changed instructions, access to another channel and a model version nobody evaluated. The approval rule was written for a different agent. The opposite error is also common. A demo goes well, someone removes review to save time, and nobody records why.

Human approval has a real cost in review time and delay, so it should be spent where a mistake would do the most damage. It should not be spent everywhere by default, and it should not be removed because the agent seems capable. As agents take on longer chains of work, the gap between what an agent can do in one session and what a manager can check keeps growing, which is why loosening needs a deliberate process rather than a feeling of trust. The NIST AI Risk Management Framework treats this kind of oversight as ongoing governance, with measurement and management continuing after deployment rather than ending at sign-off.

Start with approval on every consequential action

Begin by sorting the agent's work into action classes. Each class gets a note on what happens if the agent gets it wrong, and a current approval mode. A practical starting guide:

  • Reads and internal summaries that only the requester sees: can run without approval at first, with logs kept and sampled later.
  • Drafts that a person will send or publish: a person approves each one until draft quality is proven.
  • Reversible internal changes such as tags, notes or calendar holds: approve at first, then move to sampled review.
  • External messages, payments, deletions, permission changes or any action that touches personal data: keep approval, and consider two-person approval for the highest-impact cases.
  • Any action whose reasoning or tool call cannot be reconstructed from the log: keep approval until it can.

Keep this register short and visible. If an action class is not in the register, the agent should not act in it. The register also makes the next question easier to answer: which class has enough evidence to change?

What evidence should justify each change

Evidence has to match the risk. A long run of clean outputs on easy inputs proves very little about the difficult ones, so the record should cover routine cases, unusual inputs and deliberate attempts to push the agent past its scope. Before any approval level changes, the team should be able to show the following:

  • Logs record the action, the inputs it used, the tool it called and the outcome for every run in the class.
  • A reviewer has checked a sample across the whole review period, including hard cases and the cases the agent escalated.
  • Errors are sorted into types, such as wrong data, wrong recipient, wrong tone or action beyond scope, so patterns become visible.
  • Edge-case and adversarial tests for this class were run, and the agent refused, escalated or asked for approval where it should have.
  • The agent used only the permissions it needed and did not request access outside its role.
  • The operating context has not changed since the evidence was gathered: same model version, same tools, same instructions and same permissions.
  • Someone who did not build the agent has reviewed the evidence and agrees the criterion was met.

Evaluation tooling makes this repeatable. The OpenAI evals guide describes how to write tests that run against an agent's behavior, which turns a one-time review into checks that can be rerun after any change. Repeatability matters more than the specific tool.

Illustrative scenario: A support team lets an agent draft replies to routine order status questions. For the first weeks, a team member approves every draft. Review shows the agent misreads a shipping status field in one carrier's format, so the team corrects the tool and restarts the evidence period for that carrier only. Sampled review continues for the rest of the queue, and refunds stay under full approval the whole time.

A step-by-step plan for reducing approvals

The sequence below is deliberately slow. Each step produces a record that the next step depends on.

  • Step 1: Write the action class, the consequence if the agent gets it wrong and the approval mode it starts with.
  • Step 2: Define the success criterion in plain language, such as 'a reviewer accepts the draft without a material edit in the review sample'.
  • Step 3: Run the class under full approval long enough to see both routine and unusual inputs, not only the ones the team expected.
  • Step 4: Review the evidence with the business owner, and record the decision in writing with the date and an evidence reference.
  • Step 5: Move to sampled review, drawing the sample at random and reviewing it on a fixed schedule rather than when someone remembers.
  • Step 6: Write the trigger list that returns the class to full approval automatically, such as a new tool, a model change, a customer complaint or an error found in the sample.
  • Step 7: Only after sampled review holds for a sustained period, consider exception-only approval for that narrow class, then repeat the review before going further.

The triggers in Step 6 matter as much as the criteria. A team that has written them down can act within minutes when something changes. A team that has not will spend the incident arguing about whether the change is serious enough to review. For more on how staged gates work for company agents, see staged gates for company agents.

Who signs each change and how it is recorded

Approval changes need named people. The business owner is accountable for the risk tolerance of the class, the operations lead is responsible for the workflow evidence, and a reviewer who did not build the agent checks the record. In a small company one person may hold two of these roles, but the reviewer should still be someone other than the person who configured the agent. For the highest-impact classes, a second approver is a reasonable control, and the criteria for that are covered in two-person approval for AI agents.

Each change should be recorded with the date, the action class, the criterion, the evidence reference, the signer and the trigger list that would reverse the decision. Keep these records where the team can find them, because a later audit, a customer question or an incident will ask what the agent was allowed to do on a given day and why. Tightening approval should follow the same format. It is a change to a control, not a failure of the agent, and it should be handled as routine.

Reversal also needs a technical path. If the agent can act without approval in a class, the team should know how to pause that class quickly without losing logs or work already queued. Test that path before it is needed, not during the first unusual event.

How Be Human handles this in managed operations

Be Human designs approval boundaries at the start of an engagement. A first engagement defines one useful role, the memory and tools that role needs, the actions that require approval and an observable success criterion, so the evidence for any later change has something concrete to measure. In managed operations, monitoring, recurring evaluations, permission review and incident response produce the record that approval changes depend on. Proposed changes to approval levels are written up for the client's named owner to accept or decline, and approval is not loosened on the strength of activity volume alone.

The free Be Human AI diagnostic can help you find the first task worth handing to an agent and the approval mode it should start with.

Frequently asked questions

How many agent actions should a person review before approval is reduced? There is no universal count, because the right number depends on what a mistake would cost and how varied the inputs are. The review should cover routine cases and unusual ones, and the reviewer should agree on the criterion before the review starts. A low-consequence, reversible class can move sooner than a class that touches money, personal data or external recipients.

Can human approval be removed completely? For most company agents, complete removal is the wrong goal, because some actions should keep a person in the loop by policy, contract or regulation. A more realistic target is exception-only approval for a narrow, reversible class, with logging and sampled review continuing. Irreversible or high-impact actions usually keep a named approver.

What should send an agent back to full approval? A new tool, a model or version change, a changed instruction or permission, a complaint from a customer or staff member, or an error found in sampled review are common triggers. Write them down before any loosening happens, so the team does not debate them during an incident. Returning to full approval should take no longer than the original loosening took to justify.

Do vendor benchmarks or demos count as evidence? They can inform the design, but they are weak evidence for approval changes because they do not reflect your data, your tools or your edge cases. Approval should rest on observed runs in your own environment, measured against your own criteria. Vendor claims belong in the risk file as context, not as the reason to remove a control.

Your next step

See where an AI agent fits in your company first.

Take the free Be Human AI diagnostic. A few questions about your work show which task to hand to an agent first, what it needs to know and where a person approves.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.

Less busywork. More room for you.

Bring us the work you wish you had time for.

Book a call to try a personalized AI agent free for 7 days ↗Or take the free AI diagnostic →