Skip to content
be-human.ai
← Back to the journal

Human oversight

AI Governance Board: How a Small Company Approves Agent Upgrades

A small company can run a standing board that approves capability upgrades for its agents, using written criteria, named approvers and a decision record.

Be Human typographic cover with the title: AI Governance Board: How a Small Company Approves Agent Upgrades
Conceptual illustration created with AI · Be Human

An AI governance board for a small company is a standing group of named people who approve or refuse every change that gives an AI agent something new to do, such as a new tool, a wider data scope, a new channel or a higher limit. It works when it has written criteria, a fixed meeting rhythm with a fast path for urgent requests, and a record of each decision that someone can read later. This guide shows how to set one up with three to five people, what it should decide, and what it should leave to the agent's owner.

Key takeaways

  • Give the board one narrow job: approve changes to what an agent is allowed to do, not every output it produces.
  • Write the criteria before the first request arrives, and phrase each one so a reviewer can answer it with evidence.
  • Name one accountable owner per agent and give one reviewer the authority to hold a change.
  • Record every decision with the request, the evidence, the scope, the expiry and the rollback path.
  • Review approved capabilities on a fixed schedule, because permissions tend to outlive the reasons they were granted.

Why a small company needs a board at all

Capability changes rarely arrive as one large decision. An agent gets read access to a shared inbox, then a tool that can create invoices, then a longer memory window, then permission to offer a discount. Each step looks small on its own, and each is often approved in a chat thread by whoever happens to be online. A few months later, nobody can say which permissions are active, who approved them or why they were granted.

A board fixes this without becoming a bureaucracy. Three to five people are enough: the leader accountable for operations, someone responsible for customer data (a finance, legal or compliance delegate if you have one), and the person who runs the agent day to day. They meet briefly on a regular schedule and handle urgent requests through a short written path. The purpose is not to slow useful work. The purpose is that every capability the agent holds has a name, a reason and a person who said yes.

The board also protects the people who run the agent. Without a clear rule, an operator may feel pressure to grant access quickly to keep a project moving, and a manager may later discover that a permission was never discussed at all. A standing group with written criteria turns those private judgments into a shared process that the company can defend.

What the board decides, and what it leaves alone

Put these under board review: adding a tool or changing what an existing tool may do; allowing writes to a system of record; widening the categories of data the agent can read or store in memory; opening a new channel such as SMS, phone or outbound email; raising a spending, refund or messaging limit; changing the autonomy level, meaning how often a person must approve an action first; and switching to a new model version when the change can alter behavior on live work.

Leave these with the agent's owner: wording of routine replies inside an approved template, the ordering of tasks within a queue, and individual case decisions that fall inside the existing scope. Some decisions should never be delegated to an agent at all. Those belong on a separate list of red lines rather than in the approval queue. If your board is still deciding where the boundary sits, earning autonomy in stages describes how trust can grow one step at a time, with a check at each gate.

Write the criteria before the first request

Criteria written in advance protect the board from deciding under pressure. Each request should answer the following questions in plain language:

  • What business task does the change serve, stated in one sentence?
  • What is the smallest permission that achieves it, named by tool and action?
  • Does it touch personal data, and if so, which categories are new to the agent?
  • Can the action be reversed, and who is able to reverse it?
  • Which signal will show that the change is working, and which one will show that it is failing?
  • When will the permission expire unless someone renews it?
  • What did a test run in a non-production copy of the workflow show?

The NIST AI Risk Management Framework organizes this kind of work into four functions: govern, map, measure and manage. The checklist above is a small, practical version of those functions. The board sets the rules (govern), the request describes the context (map), the test run produces evidence (measure), and the operator keeps the change under watch (manage). You do not need the full framework to begin, but the shared vocabulary helps when an outside auditor, insurer or client asks how your decisions are made.

Keep the criteria short enough that a requester can complete them in one sitting. A form with forty fields will be filled in carelessly, and a careless form is worse than none because it creates the appearance of review without the substance.

Move each request through the board in five steps

  • The requester files a one-page change request with the criteria answered and the test evidence attached.
  • The agent's owner checks the request against the current permission map and confirms that nothing else in scope changes.
  • The reviewer reads the evidence and may return an incomplete request without waiting for the meeting.
  • The board decides at its next meeting: approve, approve with a narrower scope, defer, or refuse. Every decision is recorded the same day.
  • An operator deploys the change, not the requester, and sets the expiry and review date in the system before the change goes live.

Urgent situations need a separate rule: pause first, decide later. If an agent is behaving in a way that puts customers or data at risk, an authorized person stops it through a documented kill switch while the board convenes. Stopping should never delete logs or queued work, because the board needs those records to decide what went wrong. The kill switch guide explains how to halt actions without losing the trail.

Record each decision so it can be explained later

A decision that cannot be found later is as weak as no decision at all. Each record should hold the request and its date, the business reason, the exact permissions granted by tool and action with scope limits written out, the evidence reviewed, the names of the owner and reviewer, the board's decision, any conditions attached, the expiry and review dates, and the rollback steps. Keep the record outside the agent's own memory so that a change in the agent's behavior cannot quietly rewrite what was approved.

OWASP's authorization guidance favors denying access by default and enforcing rules in systems the agent cannot edit. The same discipline applies to governance records: the permission map should be the thing the agent is checked against, not a note that someone remembers. When a customer, auditor or insurer asks why an agent could send a message on a particular day, the record should answer in minutes. The decision records guide shows what a useful explanation contains.

Refusals belong in the record too. A refused request that is stored with its reason prevents the same request from returning in the same form without new evidence, and it shows that the board actually exercises its authority.

An illustrative scenario: SMS for a scheduling agent

The following scenario is illustrative and does not describe a specific company. A small home-repair business runs a scheduling agent that confirms visits by email. The operations lead asks the agent to also send rebooking offers by SMS. The request arrives with a business reason, a list of proposed message templates, and a test run over a fixed set of past conversations.

The member responsible for personal data notices that SMS needs consent records and quiet-hour limits that the email channel did not require. The board approves SMS only for customers with a recorded opt-in, only for rescheduling messages, and only for a short review period set in advance. It refuses open-ended promotional messages. The record lists the consent check the agent must perform before every send, the reviewer's name, the expiry date, and the rollback step: the operator disables the SMS tool in the permission map, which can be done the same day. The texting rules guide covers the consent and opt-out details that a board of this kind should verify.

Notice what the board did not do. It did not reject the idea, and it did not approve the request as written. It narrowed the capability until the business need and the risk were both visible.

Review approvals after go-live, and how Be Human fits

Permissions drift after launch. Reasons expire, staff change roles, vendors update models, and tools get used in ways the original request did not describe. Schedule a short review of every active permission on a fixed cadence, and sample the agent's work against the criteria the board used. Sampling is more reliable than trusting a summary, and a sampling routine helps decide how often a person should read the output. Evaluations add a second source of evidence. Anthropic's engineering guidance on evaluations for agents describes repeatable tasks with graded outcomes, which gives a reviewer something concrete to read besides an impression.

Be Human's managed approach handles this problem inside an operating relationship. The agent's identity, memory, channels, tools and permissions remain under the client's control. A first engagement defines one useful role, the memory and tools it needs, its approval boundaries and an observable success criterion. Managed operations then include monitoring, recurring evaluations, permission review and controlled improvements, so the board receives current evidence rather than recollections, and it decides from an up-to-date permission map.

If you are unsure which of your agent's tasks should be the first one reviewed by a board, the free Be Human AI diagnostic can help you find it.

Frequently asked questions

How many people should sit on an AI governance board? Three to five people is enough for most small companies. The group should include the leader accountable for operations, someone responsible for personal data, and the person who runs the agent. Fewer than three makes it hard to hold a change, and more than five turns each meeting into a scheduling problem.

Does every change to an agent need a board meeting? No. Routine changes that stay inside an approved scope can go through the agent's owner with a logged check. Board meetings handle changes on the review list, and urgent matters follow the pause-first path with a decision at the next meeting.

What if the board refuses a request the business actually needs? A refusal should come with a reason and, where possible, a smaller alternative such as a narrower tool, a shorter expiry or a test in a copy of the workflow. The requester can resubmit with new evidence. Keeping the refusal in the record makes the next discussion faster and more honest.

Can a small company start without special software? Yes. A shared folder with a permission map, a request template and a decision log is enough to begin. Dedicated tools become useful as the number of agents grows, but written criteria and a named owner matter far more than the software you choose.

Your next step

See where an AI agent fits in your company first.

Take the free Be Human AI diagnostic. A few questions about your work show which task to hand to an agent first, what it needs to know and where a person approves.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.

Less busywork. More room for you.

Bring us the work you wish you had time for.

Book a call to try a personalized AI agent free for 7 days ↗Or take the free AI diagnostic →