AI Agent Tool Permissions Matrix: A Least-Privilege Template

An AI agent tool permissions matrix is a table with one row for each tool an agent can reach. Each row states the actions allowed in that tool, the data those actions may touch, the conditions that apply, and the named person who approves anything outside routine work. Built well, it shows at a glance why an agent can read a customer thread but cannot issue a refund, and it gives the team one place to tighten access when a role changes.

Key takeaways

• A permissions matrix maps each tool to allowed actions, data scope and approver, with one row per tool and permission level.
• Start from the agent's role and its recurring tasks, not from the list of tools a vendor makes available.
• Separate read, draft and act permissions, and require a named human approver for anything that spends money, changes a contract or contacts a customer for the first time.
• Review the matrix on a schedule and after every incident, because access tends to grow quietly over time.
• Keep the matrix inside the agent's operating notes so the people who run it can check it without asking the builder.

Why a matrix beats a list of connected apps

A list of connected apps tells you what is installed. A matrix tells you what is permitted, and those two things diverge quickly. Teams usually grant an agent full access to a tool because one task needed one narrow action, and nobody went back to narrow it. Six months later the agent can delete records it only needed to read.

The risk is not only mistakes. An agent that reads inbound email, supplier documents or web pages is reading content written by people you do not control. OWASP's guidance on prompt injection treats that untrusted text as a reason to limit what the agent can do with it, not as a problem that a clever instruction will solve. The safest place to enforce that limit is at the tool connection, where the permission is real, rather than in a sentence inside the prompt that the model may or may not follow.

A matrix also makes access visible to the business owner. Instead of asking the technical team whether the agent can send emails, you can read one row and know.

Which columns does the template need?

Keep the template small enough that someone can review it in ten minutes. Each tool row should contain these fields:

• Tool: the system name and the specific account, connection or service identity the agent uses.
• Allowed actions: the verbs permitted, such as read, search, create draft, update a named field, send or delete.
• Data scope: the objects, folders, pipelines, customer groups or date ranges the actions may touch.
• Conditions: limits such as a maximum value, a daily count, or fields that must never change.
• Approver: the named role that must confirm actions outside routine scope, and the channel where that confirmation happens.
• Evidence: where each action is logged, so a person can audit what happened and when.
• Review date: when the row was last checked against the current role.

The approver field deserves the most attention. A role title such as "sales manager" is a starting point, but the matrix should name the person or backup who answers when the agent asks. An approval request with no one behind it is a stalled workflow, and teams often respond by removing the approval step altogether.

How do you build the matrix for one agent?

Build the matrix from the job outward. Begin with the agent's role, then work through its tasks, then the tools those tasks need. Do not start from the tool catalog, because that produces a row for everything the vendor offers.

• Write the role in one sentence, then list three to five recurring tasks the agent should complete.
• For each task, name the tools it needs and remove any tool that is only convenient.
• For each remaining tool, list the smallest set of actions that completes the task.
• Narrow the data scope until a person can explain, in one sentence, why each object is in scope.
• Assign an approver to every action that changes money, records, customer commitments or access rights.
• Set one observable success criterion per task, so you can tell whether the granted access is enough and not more than enough.
• Test with read-only access first, then widen one row at a time, and record each change with its reason.

The last step is where most teams skip discipline. Widening access in small steps keeps the cause of any problem easy to find. Widening everything at once makes the first incident hard to explain.

Read, draft or act: which permission level fits?

Most tool rows fall into one of three levels. Choosing the level before choosing the action names keeps the matrix honest.

• If the task only needs information, grant read access on the narrowest scope that answers the question.
• If the output is a proposal that a person will review, grant create-draft access and never send.
• If the action is reversible, low risk and covered by clear conditions, the agent may act within those conditions and log the result.
• If the action moves money, signs a document, deletes data, changes access or contacts a customer for the first time, require a named approver before execution.

This guide is deliberately conservative. Teams can move a row from draft to act later, once the logs show the agent behaves as the success criterion expects. Moving it earlier, before that evidence exists, is the usual origin of an avoidable incident. For a deeper look at this split, see Read, draft and send: three different AI permissions and Least privilege for AI agents: start with the smallest access.

The identity the agent uses also matters. A shared personal login gives the agent the same reach as the person who owns it, which breaks the matrix. A dedicated service identity with only the granted scopes keeps the matrix and the actual permissions aligned. The trade-offs are covered in Service accounts or OAuth tokens for AI agents: which to use.

Illustrative scenario: a quote agent at an equipment distributor

Illustrative scenario: a small equipment distributor wants an agent to prepare quotes for inbound requests from existing trade customers. The team writes the matrix in the following way. The agent reads the customer account and its assigned sales territory in the CRM, and nothing outside that territory. It reads the current price list but cannot edit it. It creates quote drafts in the quoting tool and cannot send them. Any discount beyond the standard list must be approved by the sales manager, and the request goes to the manager's review queue with the customer history attached. Every draft and approval is logged with the request identifier.

The matrix also records the success criterion: the sales team accepts most drafts with only minor edits over a trial period, and no quote leaves the building without a person's send action. When a new salesperson joins the territory, the team updates one data scope row, not the whole setup. The point of the example is the shape of the record, not any particular result. Anyone reading the matrix can see what the agent can touch, who decides, and where the evidence lives.

How Be Human keeps the matrix current, and what to review first

Be Human designs personalized AI agents with their identity, memory, channels, tools, permissions and operating context under human control. A first engagement defines one useful role, the memory and tools that role needs, the approval boundaries, and an observable success criterion, and the permissions matrix is a direct output of that definition. Managed operations then keep it accurate over time through permission review, integration maintenance and incident response, so rows are rechecked when a tool changes, a staff member moves, or an agent's task expands.

For most companies, the first review should cover three things: any row with send, delete or payment rights, any row where the approver is a role with no named person, and any connection that still uses a personal login. Those three checks usually reveal more than a full audit of every tool. If you are not yet sure which task deserves an agent first, the free Be Human AI diagnostic is a practical place to start.

Frequently asked questions

What is an AI agent tool permissions matrix? It is a table that lists every tool an agent can use, the actions allowed in each tool, the data those actions may touch, and the person who approves anything outside routine work. It turns a vague statement such as "the agent has access to our CRM" into a specific record that someone can review and change. Most teams keep it in a shared document next to the agent's operating notes.

How is least privilege different from simply limiting the agent's prompt? A prompt is an instruction the model is asked to follow, while a permission is a restriction the connected system enforces. Least privilege applies the restriction at the tool connection, so a misleading instruction cannot widen what the agent is allowed to do. Prompt rules still help with behavior, but they should not be the only control.

Who should approve the agent's actions? The approver should be a named person who understands the business consequence of the action, such as a sales manager for discounts or a finance lead for payments. Each approver needs a backup and a clear channel, or approval requests will stall. For actions that are low risk and reversible, the approval can be a logged rule rather than a live review.

How often should the matrix be reviewed? Review it on a fixed schedule, and also after any incident, any change in the agent's role, and any change in the people who approve its work. Quarterly is a reasonable default for stable roles, and sooner is better when the tasks change often. Each review should confirm that every row still matches a real task and that no row grants more than the success criterion requires.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.