Skip to content
be-human.ai
← Back to the journal

Human oversight

AI Controls for Regulated Companies: Evidence Before Widening Scope

How regulated companies can set AI agent controls and collect evidence before widening an agent's scope: boundaries, logs, tests, owners and review.

Be Human typographic cover with the title: AI Controls for Regulated Companies: Evidence Before Widening Scope
Conceptual illustration created with AI · Be Human

AI controls for regulated companies are the scope limits, permissions, approval rules, logs, tests and named owners that keep an agent inside what it is allowed to do. Before a company widens an agent's access, it should hold evidence that each existing control works in practice, including the cases where it was overridden, failed or was never triggered. Expansion is a decision supported by records, not a feature upgrade or a vendor's assurance.

Key takeaways

  • Treat every permission, approval step and log as a claim that needs proof, and keep the proof on file.
  • Write the agent's scope, owner and stop conditions before changing its access, not after a problem appears.
  • Widen one dimension at a time (data, tools, channels or the value of actions) and keep the previous evidence intact.
  • Put a person on any action that is external, irreversible, financial or touches regulated personal data.
  • Review sampled agent work on a schedule that matches the risk, and record what each reviewer found.

What counts as a control for an AI agent?

For a business agent, a control is any mechanism that limits what the agent can see, decide or do, and that someone can inspect. Regulated companies already run controls for staff: role-based access, change approval, segregation of duties and audit trails. An agent needs equivalents, because it acts through tools and channels that were built for people and do not judge context the way a person does.

A useful control set covers seven areas. Identity defines which account the agent uses and how that account is named in logs. Memory defines what the agent may store, where, and for how long. Tools define which systems it can call and which operations each call may perform. Channels define where it may send messages. Approvals define which actions wait for a person. Logging defines what is recorded, who can read it and how long it is kept. The stop path defines how work halts without losing records or queued tasks.

The NIST AI Risk Management Framework organizes AI risk work around governing, mapping, measuring and managing. You do not need to adopt it as a whole to benefit from the distinction it draws between defining a risk and measuring whether your control reduces it. Each control in your set should name the risk it addresses and the observation that shows it works.

How do you write the scope before widening it?

Most expansion problems start with a scope that was never written down. The agent was introduced for one task, picked up a second because it was convenient, and then gained access to a third system because a colleague asked. A written scope stops that drift. Use this sequence before any change to access:

  • Name the role in one sentence, and state what the agent does not do.
  • List every system, data category and channel the agent currently touches, including the ones added informally.
  • Assign a named business owner who can approve changes and pause the agent.
  • Define the success criterion as something observable, such as a queue cleared with a recorded reviewer decision.
  • Write the stop conditions: the events that pause the agent automatically or send the work to a person.
  • Set a review date and the evidence that would justify reopening the scope.

The scope statement is short on purpose. If it runs to pages, nobody will read it before a change, and the owner will not be able to say whether a new request falls inside it.

Which evidence should you have before any expansion?

An evidence pack is the set of records that lets someone outside the project check whether the agent is still inside its boundaries. Regulators, auditors, insurers and internal risk teams will ask for different subsets, but the core pack is similar across sectors. Assemble it before the expansion request, not during the review.

  • A current scope statement, dated and approved by the business owner.
  • An inventory of tools, credentials and channels, with the permission each one grants and who issued it.
  • A memory inventory: what is stored, where it lives, the retention period, and how deletion is verified.
  • Sample log entries showing the input, the tool call, the output and the approver where an approval applied.
  • The results of the most recent test set, including failures and the change made in response.
  • A record of every override, pause and incident, with its cause and the control that was adjusted.
  • Vendor terms covering data processing and subprocessors, reviewed against the data the agent will handle. The subprocessor question for AI vendors is a useful starting point.
  • A revocation test: the measured time to remove the agent's access and confirm that it no longer works.

Records of decisions deserve particular attention. When an agent takes an action that later needs explaining, the question is rarely whether the output was correct. It is whether anyone can reconstruct what the agent knew, which rule applied and who was accountable. The guidance on decision records covers what to capture at the moment of action.

A log that cannot be connected to a scope, an owner and a rule is a record of activity, not evidence of control.

Where must a person approve an agent's action?

Approval boundaries are the most visible control and the easiest to set too loosely. A workable approach sorts actions by reversibility, audience and data sensitivity, then assigns each class a level of human involvement. The aim is to put people where a mistake is hard to undo, not to route every small task through a queue that nobody clears.

  • Reads of internal, non-sensitive information: the agent may act alone, with sampled review.
  • Drafts that a named person reviews and sends: the agent may prepare them, but the person sends them.
  • Messages to customers or other external parties: a person approves each message until sampled reviews show the pattern is stable.
  • Movements of money, changes to customer or vendor records, and deletion of data: two people approve, and the approval is logged.
  • Actions involving regulated personal data, health information or financial records: approval plus a written check of the legal basis for the processing.
  • Any action that cannot be reversed: no autonomy until a rollback path has been tested.

The human approval gates for agents guidance describes how to decide where the signature sits in each workflow. Whatever you choose, write the approver's role, not only a name, so the control survives staff changes.

How do you test controls and show they hold?

A control that has never been exercised is a hypothesis. Testing turns it into evidence, and the test should include cases that are likely to fail. For agents that read inbound messages, one of those cases is an instruction hidden in the content of the message, which the agent may treat as a request. The OWASP prompt injection guidance describes why untrusted content must be kept separate from the agent's instructions and tools, and it is a good reference for building those cases into your test set.

Sampling is the ongoing version of testing. A person reviews a selection of the agent's work on a schedule, records the result, and feeds failures back into the test set. The schedule should match the risk: higher for external messages and sensitive data, lower for internal summaries. The sampling audit approach shows how to decide how often a person checks.

For logs to serve as evidence, they need consistent structure. OpenTelemetry's observability concepts describe traces, metrics and logs as complementary signals, which maps well to agent work: a trace shows the chain of steps, metrics show the trend, and logs hold the detail needed for a review.

Illustrative scenario: a support team uses an agent to draft replies to billing questions. Its first evidence pack shows that drafts are accurate for refunds but sometimes cite an outdated policy page. Before the team lets the agent send replies without review, it adds the failed cases to the test set, points the agent to a policy source with a visible effective date, and samples drafts each week. Expansion waits until the sampled reviews stay clean across consecutive cycles. The team records each step, so an auditor can see what changed and why.

When is it safe to widen an agent's scope?

Expansion should be a deliberate event with its own record. Before approving it, the owner should be able to answer yes to each of the following:

  • The current scope has been reviewed, and the evidence pack is complete and current.
  • The most recent test and sampling results show no unresolved failures in the area being widened.
  • The new permissions are the minimum needed, and each is tied to a named tool or channel.
  • The approval boundary for the new action is written, and the approver knows the role.
  • The revocation test still passes after the change, and the rollback path has been exercised.
  • Someone outside the project has read the change request and can explain it in plain language.

If any answer is no, widen less or wait. A staged approach, where each stage has its own gate and evidence, is described in the guide to staged gates for agent autonomy.

Be Human designs, implements and operates personalized AI infrastructure for companies, including agents that keep their identity, memory, channels, tools, permissions and operating context under human control. A first engagement defines one useful role, the memory and tools that role needs, its approval boundaries and an observable success criterion. Managed operations can add monitoring, recurring evaluations, integration maintenance, permission review, incident response and controlled improvements, so the evidence pack is kept current instead of rebuilt before each review. The work is remote and available in English and Portuguese.

If you are unsure which of your workflows is ready for an agent and which needs more controls first, take the free Be Human AI diagnostic to find the first task worth handing to an agent.

Frequently asked questions

Do regulated companies need a full AI governance program before deploying an agent? Not necessarily at the start. A single agent with a written scope, a named owner, an approval boundary and a log that can be reviewed is a workable foundation. Governance grows as the number of agents, data categories and external channels grows, and the controls described here scale with that growth.

How often should a person review an agent's work? There is no universal interval, because the right frequency depends on risk. Review external messages, sensitive data and irreversible actions more often than internal summaries, and let the sampled results drive any change. Record each review so the frequency can be justified later.

What is the difference between a scope statement and an approval policy? A scope statement says what the agent is for and what it must not do. An approval policy says which of its actions need a person and who that person is. Both are needed, and an auditor will usually ask to see them side by side with the logs.

Can an agent's permissions be widened gradually without new paperwork each time? Small changes within a previously approved stage can be handled with a lightweight record, provided the scope and approval boundaries already cover them. Any change that adds a new system, data category, channel or class of action should go through the full expansion check again.

Your next step

See where an AI agent fits in your company first.

Take the free Be Human AI diagnostic. A few questions about your work show which task to hand to an agent first, what it needs to know and where a person approves.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.

Less busywork. More room for you.

Bring us the work you wish you had time for.

Book a call to try a personalized AI agent free for 7 days ↗Or take the free AI diagnostic →