Skip to content
be-human.ai
← Back to the journal

Harness fundamentals

Tool Allowlists for AI Agents: Start Small and Widen With Evidence

Learn how to start each business AI agent with a short, reviewed tool allowlist, then widen it only when evaluations show a real need.

Be Human typographic cover with the title: Tool Allowlists for AI Agents: Start Small and Widen With Evidence
Conceptual illustration created with AI · Be Human

A tool allowlist is the explicit list of actions an AI agent may call, and the safest way to start is to give each business agent only the tools its single role requires. Add a tool only after an evaluation shows the agent failing a real task for lack of that capability, and remove any tool that sits unused. This keeps permissions small, easy to review, and quick to roll back as the agent proves itself.

Key takeaways

  • Start each agent with the fewest tools that let it complete one defined role.
  • Treat every tool as a delegated permission, with an owner, a scope and an approval rule.
  • Add a tool only after an evaluation shows a specific failure that this tool would fix.
  • Remove tools that go unused, because each one is extra surface for error and misuse.
  • Keep every change to the allowlist in a reviewed log so you can explain it and reverse it.

Why fewer tools is the safer starting point

Every tool an agent can call is a decision you have handed over. A tool that reads a customer record, sends a message or issues a refund carries consequences that the model does not weigh the way your team does. When an agent holds ten tools for a role that needs three, the extra seven are not neutral. They widen what a confused or manipulated run can do, add paths that are harder to test, and make a single incident harder to trace.

Security guidance for LLM applications already names this risk. The OWASP Top 10 for LLM Applications describes excessive agency, where an agent is given more functionality, permissions or autonomy than its task needs. The remedy is the principle that governs people in most organizations: least privilege, granted per role and reviewed over time. For a broader view of where agents fit in your operations, the AI Agent Guardrails for Business: Which Ones You Need First guide covers the controls to put in place before the first live run.

How do you build the first allowlist?

Start with the role, not the software. A tool list written before the role is clear tends to grow to match whatever the vendor offers. Work through the following steps in order, and keep the output short enough that a business owner can read it in one sitting.

  • Write the role in one sentence, and include what the agent must never do.
  • List the tasks that role performs in a normal week, using the same wording your team uses.
  • For each task, name the smallest tool that completes it and the data fields it reads or writes.
  • Mark each tool as read only, draft, or act, and name a human approver for every act tool.
  • Set a scope for each tool, such as one mailbox, one order status field or one shared calendar.
  • Run the agent against recorded test cases in a staging copy before it touches any live system.
  • Review the list with the business owner and with the person who will maintain it, then store it as a versioned file.

In Be Human's managed approach, the first engagement defines one useful role, the memory and tools it needs, its approval boundaries and an observable success criterion, so the allowlist is written before any agent is connected. Managed operations then review permissions on a recurring schedule, maintain the integrations behind each tool, and handle changes to the list as controlled improvements rather than informal additions.

When does a tool earn a place on the list?

A tool should earn its place by solving a documented gap, not by seeming useful. Use the questions below as a decision guide when someone asks to add a capability.

  • Add the tool if a recorded test case failed because the agent lacked that exact capability, and a read-only version would not have solved it.
  • Prefer a read-only tool over an action tool whenever a read answers the question.
  • Hold the tool back if a draft-and-approve step can replace direct action without slowing the business in a way that matters.
  • Hold the tool back if you cannot validate its output with a simple field check, because unvalidated output is hard to trust.
  • Decline the tool if the real gap is missing information, since information belongs in a governed retrieval source rather than in a new action.

The final question is often the most useful. Many requests that look like missing tools are actually missing context, and adding context is cheaper and easier to reverse than adding a new power.

What does a widened allowlist look like in practice?

Illustrative scenario: a small repair shop wants an agent to answer customer questions about order status. The first allowlist contains one read tool that looks up an order by its reference and returns the status and the promised pickup window. Recorded tests show the agent answering correctly for most orders but unable to say whether a part has shipped, because shipping information lives in a separate system. The team adds one read tool for that system, scoped to status only. Refunds, cancellations and price changes stay off the list, and the change is logged with the test cases that justified it.

The scenario shows the pattern. The widening was triggered by a specific failure, it added the narrowest capability that fixes that failure, and it left the high-consequence actions out. A team that wants to ask for more power than the case requires should be asked to show the failing test first.

Each tool on the list should be traceable to a test that needed it.

How should evaluations justify every widening?

An evaluation is a structured set of tasks with inputs and expected results, run the same way each time so changes can be compared. The OpenAI evals guide describes this approach for testing model behavior against defined criteria, and the same discipline applies to agent tool use. Your test set should contain two kinds of cases: tasks the current allowlist must complete, and requests the agent must refuse or escalate because the tool is not allowed.

Before widening, check whether the failure is really about tools. A weak instruction, a vague task contract or a missing example can cause the same symptom, and adding a tool to fix a prompt problem only moves the risk. When a grader scores results automatically, calibrate it against human judgment first, as explained in LLM Judges for Agent Evals: Calibrate Before You Trust Them. Otherwise the evaluation itself can approve a change that nobody actually reviewed.

When a widening is approved, rerun the full test set, not only the new cases. A new tool can change how the agent handles requests it already handled well, and the regression results are part of the change record.

How do approvals, scopes and permission checks keep the list honest?

An allowlist is only as strong as the check that enforces it. Permissions should be verified on every call, not only when the agent is configured. The OWASP Authorization Cheat Sheet recommends enforcing access control at the point where an action happens, with clear rules about who may do what. For an agent, that means the harness should refuse a tool call that falls outside the approved scope even if the model asks for it, and it should log the attempt.

Approval rules belong in the same record. For each act tool, write down who approves, what information the approver sees, and how long a request may wait before it expires. Tool outputs also deserve caution. Content returned from a web page, a document or a customer message can contain instructions that try to redirect the agent, so limit what a read tool's output is allowed to trigger.

How do you keep the allowlist small over time?

Allowlists tend to grow quietly. Someone adds a tool for a one-time project, the project ends, and the permission stays. Prevent this with a recurring review. At each review, check which tools were used in the logs, which were never called, and which scopes are broader than the current role requires. Remove unused tools, shrink scopes that proved wider than needed, and record each removal the same way you record additions.

The review also keeps the business honest about what the agent is for. If the role has changed, rewrite the role sentence first and derive the allowlist from it again. Mapping the first allowlist is also the quickest way to see which work is worth handing to an agent at all, so take the free Be Human AI diagnostic to find the first task worth handing to an agent.

Frequently asked questions

How many tools should a first agent start with? Start with the smallest set that completes one defined role, which is often only a handful of read tools and at most one draft or act tool. The right number is the one your test cases require, not a number chosen in advance. If the list feels too short, the next step is a failing test, not a larger list.

What is the difference between an allowlist and a blocklist for agents? An allowlist names the only tools an agent may call, so anything not listed is refused by default. A blocklist names tools that are forbidden and allows everything else, which means every new tool is permitted until someone remembers to block it. For business agents, the allowlist is the safer default because it makes each new capability an explicit decision.

How do I know when it is safe to add a tool? It is reasonable to add a tool when a recorded test shows a specific failure that only that capability can fix, when the tool's scope and approval rule are written down, and when the full test set passes after the change. Treat the addition as a versioned change with a named owner and a rollback plan. If any of those pieces is missing, the tool is not ready.

Who should own the allowlist after the agent goes live? A named business owner should own the role and the approval rules, and an operations person or managed service should own the technical enforcement and the review schedule. Shared ownership without a named person usually means no one reviews the list. Ownership should be visible in the same file that holds the allowlist, so anyone reading it knows whom to ask before a change is made.

Your next step

See where an AI agent fits in your company first.

Take the free Be Human AI diagnostic. A few questions about your work show which task to hand to an agent first, what it needs to know and where a person approves.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.

Less busywork. More room for you.

Bring us the work you wish you had time for.

Book a call to try a personalized AI agent free for 7 days ↗Or take the free AI diagnostic →