Skip to content
be-human.ai
← Back to the journal

Personalized AI infrastructure

Approval Thresholds for Superintelligent Agents: Raise Them Slowly

Set AI agent approval limits by action risk and widen them only after logged evidence. A practical method that keeps authority earned, not assumed.

Be Human typographic cover with the title: Approval Thresholds for Superintelligent Agents: Raise Them Slowly
Conceptual illustration created with AI · Be Human

Set approval thresholds by the risk of each action, not by how capable the agent appears. Start every action class at approval required, route money, customer-facing messages and irreversible changes through a named human, and widen a threshold only when logged evidence shows the agent handled that class correctly over a defined review period.

Key takeaways

  • Classify each agent action by reversibility, money, data exposure and external visibility before choosing a limit.
  • Begin every action class at approval required, not at autonomous execution.
  • Widen a threshold only on logged evidence and recorded reviews, never on the agent's own confidence.
  • Treat each widening as a written, reversible change with an owner and a rollback rule.
  • Re-test thresholds on a schedule and whenever the model, tools or permissions change.

Why a more capable agent does not earn more authority

A better model writes stronger drafts and handles messier input, which makes it tempting to let it act more often. But authority belongs to the whole system: the model, the tools it can call, the credentials those tools use, the context it reads and the people who review its work. When any of those change, the evidence gathered under the old system no longer applies. A model upgrade is therefore a new trust question, not a reward for the previous version.

The NIST AI Risk Management Framework treats risk as something to map, measure and manage across the life of a system, not something settled once at launch. Apply the same logic to approvals. A threshold is a claim that a class of actions is safe under specific conditions, and the claim expires when those conditions change.

How do you sort agent actions into risk classes?

Most owners start with the question "what should the agent be allowed to do?" That question is too broad to answer well. Break the agent's reach into action classes, then judge each class on four points: can it be undone, does it move money or commitments, does it expose personal or confidential data, and does anyone outside the company see it.

  • Internal and reversible: summarizing, tagging, drafting internal notes. A candidate for early autonomy once the evidence is clean.
  • Internal but consequential: changing a status in a system of record, assigning work to a team. Keep a human check until the evidence is strong.
  • Customer-visible: replies, booking changes, account updates. Start with approval, because errors reach people outside the company.
  • Money-moving or contractual: refunds, invoices, discounts, purchase orders. Approval required, with any monetary ceiling set by the business owner, not by the agent.
  • Irreversible or sensitive: deletions, data exports, changes to access rights, anything touching regulated personal data. Approval required, and widening only after a separate review.

A step-by-step plan for setting the first thresholds

Use the same sequence for each agent, and keep it on one page so the operator and the approver read the same rules.

  • Step 1: List every action the agent can take today, including tool calls it can reach but does not normally use.
  • Step 2: Assign each action to a risk class and name one accountable human approver for each class.
  • Step 3: Set every class to approval required for the first review period, with no exceptions for cases that seem obvious.
  • Step 4: Write the success criterion for each class in plain terms: what a correct action looks like and who decides.
  • Step 5: Log each proposed action, the approver's decision, the final outcome and any correction made afterward.
  • Step 6: Review the log on a fixed cadence and record one decision per class: widen, hold or tighten.
  • Step 7: Widen one class at a time, with the rollback rule written before the change goes live.

What counts as evidence before a threshold moves?

Evidence is not the agent's own summary of how well it performed. It is a record that a named person can inspect. Useful evidence has three parts. Volume means enough cases in the class to show a pattern, not one good week. Variety means the awkward cases are included, such as unusual customers, incomplete records and conflicting instructions. Outcome means the action held up after a human or a downstream system examined it.

Build a fixed test set for each class and run it before and after every model or tool change. The OpenAI evals guide describes the general method of defining expected behavior and grading outputs against it, and the same discipline works for approval decisions. Keep the test cases separate from the examples used to shape the agent, so the score tells you something.

Illustrative scenario: a services company uses an agent to prepare refund recommendations for its support team. During the first review period, every refund needs approval. The approver records whether each recommendation was accepted, edited or rejected, and why. When the record is clean across the whole class, including the difficult cases, the team widens only the lowest-value refunds to execution, with a fixed ceiling and a daily report. Larger refunds stay under approval. When the model provider later releases an update, the team reruns the test set before the new version touches any refund at all.

Where should thresholds be enforced?

A threshold written only in the agent's instructions is a suggestion. The model can misread it, a new input can override it, and a hostile message can try to argue it away. Enforcement belongs where the action happens: in the tool, the credential and the system of record. Give each agent its own identity, grant only the scopes its approved classes need, and have the tool check authorization on the server side for every call. The OWASP authorization guidance makes the same point: the system that holds the resource should make the access decision, not the caller.

Put the approval step inside the tool. An action in an approval-required class should create a pending request that a human completes, not a direct write the agent might regret. The log should record who approved, when, and under which version of the rules. A quarterly permission review is the natural checkpoint for confirming that the scopes still match the approved classes.

Be Human designs this layer as part of a managed agent engagement. Each agent's action classes, approvers, scopes and logs are defined in writing before go-live, reviewed on a regular schedule, and changed only through controlled requests. The operating context stays under the client's control, so the company can inspect, export and change the rules without depending on a single model vendor.

How do you widen a threshold without losing control?

Widening is a change, so handle it as one. Write a short request that names the action class, the evidence period, the approver who signed off, the new level and the condition that would send the class back to approval required. Include the rollback rule: which signal triggers the return, and who can trigger it without waiting for a meeting. A consistent template keeps the record readable across agents, and the structure in how to request and approve changes to a managed AI agent fits this process well.

Widen one class at a time. If two classes move together, you cannot tell which change caused a later error. Keep one owner for each decision, even when several people contribute evidence.

When should you tighten, and how often should you re-check?

Thresholds should move in both directions. Tighten immediately after a serious error, a near miss involving sensitive data, an unexplained shift in approval rates, or a change to the model, tools or permissions that the test set did not cover. Tightening needs a clear trigger and a named person who can act, not a long review.

Re-check every class on a fixed schedule, even when nothing looks wrong. Quiet logs can mean the agent is performing well, or that nobody is reading the approvals. Review time is part of the evidence. If approvals are rubber-stamped in seconds, the threshold has not been tested, only ignored.

If you are not yet sure which action should be the first one an agent handles without asking, the free Be Human AI diagnostic is a practical place to start.

Frequently asked questions

How do I know a threshold is safe to raise? Check whether the class has a clean record across enough cases, including the difficult ones, and whether the approver's decisions were genuine reviews. Confirm that the rollback rule is written and that the tool enforces the new level. If any of these is missing, hold the threshold where it is.

Should a more capable model start with higher limits? No. A stronger model can change how failures happen, so start the new version under the same approval rules as the old one and widen on its own evidence. Run the test set against the new version before it touches live actions.

Who should own approval decisions? One named business owner per action class is the right answer, usually the person who carries the consequence of a wrong action. The operator maintains the logs and tests, but the business owner signs the threshold. Shared ownership often produces no owner at the moment something goes wrong.

What happens when we change models or tools? Treat the change as a new system. Rerun the test set for the affected classes, review permissions for any new tool, and keep approval required for classes the test set does not cover until evidence exists. Document the change so the history of thresholds stays readable, and consider how the permission review steps apply to the new tool.

Your next step

See where an AI agent fits in your company first.

Take the free Be Human AI diagnostic. A few questions about your work show which task to hand to an agent first, what it needs to know and where a person approves.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.

Less busywork. More room for you.

Bring us the work you wish you had time for.

Book a call to try a personalized AI agent free for 7 days ↗Or take the free AI diagnostic →