How to Run a Security Review of an AI Agent Provider

An AI vendor security review checks four things before an agent connects to your systems: what access it gets, how your data is handled, what gets recorded, and what the provider does when something goes wrong. Run it as a short written exercise where every answer comes with evidence, and let the result set the agent's first permissions. A provider that cannot answer plainly is already giving you useful information.

Key takeaways

• Review the role, not just the company. The same provider can be low risk for a read-only research task and high risk for an agent that sends email.
• Ask for access in the smallest form first, and ask who can change it, how fast it can be revoked, and how you would know it was used.
• Trace your data through every stage: input, memory, logs, backups, model calls and deletion. Each stage needs a named owner and a rule.
• Logs should let you reconstruct what the agent did and why, without exposing more sensitive content than necessary.
• An incident process is a document plus a rehearsed habit. Ask to see both before you sign.

Why does an AI agent need a different security review than ordinary software?

A typical software review asks whether the product is safe to use. An agent review has to ask a second question: is it safe to let this system act on our behalf? An agent reads documents, remembers context, calls tools and sometimes writes to systems of record. That means the review covers behavior, not only infrastructure.

Three features make agents different. First, they take instructions from text, and text can come from anywhere: an email, a shared document, a web page. The OWASP Top 10 for LLM Applications lists prompt injection and excessive agency among the main risks for this reason. Second, they hold memory, so a mistake or a leak can persist beyond a single session. Third, they use credentials, so any weakness in how those credentials are stored and scoped becomes a weakness in your own systems.

This does not call for a heavy compliance program. It calls for a focused set of questions, asked in a sensible order, with answers you can check. If your company is still building its wider policies, the lightweight governance start is a good companion to this review.

What is the step-by-step plan for the review?

Do the work in this order. Each step produces something you can file, and each step can end the review early if the answer is unacceptable.

• Step one: write the role in one paragraph. Name the task, the systems the agent would touch, the data it would read, and the actions it could take. Everything else is judged against this paragraph.
• Step two: list the access the role truly needs. Separate reading, drafting and sending, and mark which actions would require a human approval.
• Step three: send the provider a written questionnaire covering access, data handling, logging and incidents. Prefer written answers to a sales call, because written answers can be compared and quoted later.
• Step four: ask for evidence behind the answers. That can be architecture descriptions, sample log entries, a data flow diagram, a draft incident procedure and the list of subprocessors.
• Step five: test one claim yourself. Ask for a demonstration of revoking a credential, exporting a log, or deleting a memory item, and watch it happen.
• Step six: record the decision, the conditions and the review date. Approve a narrow first scope, not the whole vision.

The questions to ask an AI agent vendor cover the commercial side of the same conversation. This plan stays on security.

How do you review what access the agent will have?

Access is where most avoidable harm comes from, so spend the most time here. The OWASP Authorization Cheat Sheet recommends denying by default and granting only what a task requires. Apply that to an agent literally: start with no access, then add one permission at a time with a reason.

Ask the provider these questions about access:

• How does the agent authenticate to our systems? Look for delegated authorization with scoped, revocable tokens rather than shared passwords or a human's personal login.
• Can we grant read access without write access, and draft access without send access? If these cannot be separated, the permission model is too coarse.
• Who at the provider can see or use our credentials, and how is that access logged?
• How quickly can we cut the agent off, and does revoking one credential stop every path to that system?
• What happens when a connection expires or a permission is removed? A good agent stops and reports rather than hunting for another route.

The distinction between reading, drafting and sending deserves emphasis. A tool that can only read can still leak information if it is tricked, but it cannot change your records. A tool that drafts leaves a human in the loop. A tool that sends commits your company. The article on least privilege for AI agents goes deeper on how to stage these permissions.

Also ask about the human side. Who inside your company owns each connection? Who approves a new permission? If the answer is nobody in particular, fix that before the provider's answers matter.

### An illustrative scenario

This is an illustrative example, not a real case. A services firm wants an agent to prepare follow-up emails after client calls. The review finds the provider's default setup asks for full mailbox access, including sending. The firm instead grants read access to a single shared inbox and draft-only access for outgoing messages. A person reviews and sends each draft. After a few weeks of reviewing logs, the firm decides whether any category of message is safe enough to send automatically. The agent is useful from day one, and the worst-case outcome of a mistake is a poor draft that a person declines to send.

How do you review data handling?

Trace the data, stage by stage. Many reviews stop at the question of whether the provider trains on your data. That matters, but it is only one stage. Ask for a plain data flow that covers every place your information could rest or pass through.

Work through these stages and get a written answer for each:

• Input: what content does the agent receive, and can we limit it by source, folder or field?
• Model calls: which model providers receive our content, under what terms, and with what retention?
• Memory: what does the agent store between sessions, where does it live, and can we inspect, correct and delete it?
• Logs and traces: do they contain full content or references, and who can read them?
• Backups and subprocessors: where do copies go, and who else touches them?
• Exit: when the contract ends, what is returned, what is deleted, and how is deletion confirmed?

Memory deserves special attention because it behaves differently from a chat history. Information the agent remembers can resurface in a later task for a different person. Ask whether memory can be scoped by team or role, and read the guidance on what an agent should remember before you decide what is allowed in.

Ownership belongs in this section too. Your prompts, instructions, memory and outputs should remain yours, in a form you can export. The guide on who owns AI prompts, memory and data lists the clauses worth checking.

If you handle personal data of people in Brazil or Europe, also confirm which party is controller and which is processor, and where processing takes place. Ask the provider how they support your obligations, and take the specifics to your legal adviser. This review does not replace legal advice.

What should logging and incident response look like?

Logging and incidents belong together, because you cannot respond to what you cannot see. The OpenTelemetry observability primer describes the idea well: a system is observable when you can understand its internal behavior from the signals it emits. For an agent, those signals should answer a simple question: what did it do, with what authority, and based on which inputs?

Ask to see a real sample log entry, with sensitive content masked, and check that it shows:

• The task or request that started the work, and who or what triggered it.
• The sources the agent read and the tools it called, with inputs and outputs or references to them.
• Any approval requested, who gave it, and when.
• Errors, retries and stops, including expired connections.
• The version of the instructions and configuration in force at the time.

Then ask who can read the logs, how long they are kept, and whether you can export them. Logs that capture everything can become a data risk of their own, so the best answer is a deliberate choice about what to store. The article on what to log for AI agents covers that trade-off.

For incidents, request the draft procedure and ask these questions:

• What counts as an incident, and who declares it?
• How and how fast will we be told, and through which channel?
• Can the agent be paused immediately, by us as well as by the provider?
• How are affected actions identified and, where possible, reversed?
• What does the review after an incident produce, and do we receive it?
• When was the procedure last rehearsed?

The last question separates a document from a habit. A provider that has practiced pausing an agent and tracing its actions will answer concretely. A provider that has not will answer in general terms.

The NIST AI Risk Management Framework is a useful reference for structuring this thinking: it organizes AI risk work around governing, mapping, measuring and managing. You do not need to adopt it formally, but its vocabulary helps when you write down who is responsible for what.

How do you read the answers and decide?

Treat the review as a decision guide, not a score. For each area, sort the provider's answers into three groups.

• Accept: the answer is specific, written, backed by evidence, and you saw it work. Proceed to a narrow first scope.
• Accept with conditions: the answer is reasonable but incomplete. Set a dated condition, such as draft-only access until log export is demonstrated.
• Stop: the answer is vague, deflects to marketing language, or reveals a missing control such as no way to revoke access quickly. Do not connect systems until it is resolved.

A few patterns usually justify stopping. The provider asks for broader access than the role needs and cannot explain why. Credentials are shared between customers. You cannot delete memory or export logs. Nobody can describe what happens when the agent is wrong. Each of these is easier to fix before a contract than after.

After approval, keep the review alive. Run it again when the role changes, when a new tool is connected, or when the provider changes subprocessors or models. A phased start helps here: the shadow mode pilot lets the agent work alongside people without authority while you check the logs against your expectations.

Be Human handles these questions as part of how it designs and operates agents, not as an afterthought. A first engagement defines one role, the memory and tools it needs, the approval boundaries and an observable success criterion, so the access list and data flow exist in writing from the start. Managed operations include monitoring, permission review, integration maintenance and incident response, with the agent's identity, memory and permissions kept under your control. You can request that documentation and ask for a demonstration of revoking a connection before anything goes live.

Frequently asked questions

How long should an AI vendor security review take? It depends on the role, not the company size. A narrow, read-only role can be reviewed in a few focused sessions, while an agent that writes to core systems deserves more time and a staged approval. Schedule the review before the pilot, not during it.

Do we need a security specialist to run this review? Not necessarily. A business owner or operations leader can run the process if someone technical helps verify the evidence, such as the access scopes and log samples. If the agent touches regulated or highly sensitive data, involve your security or legal adviser early.

What if the provider refuses to share their incident procedure? Treat that as a significant signal. Some detail may be confidential, but a provider should be able to share a summary under an agreement, covering notification, pausing and review. If they cannot describe the process at all, the safest choice is to wait.

Should the review change if we use a custom-built agent instead of a provider? The same four areas apply, but you own the answers. You still need to document access, data flow, logs and incident steps, and decide who maintains them. The build, buy or managed decision framework helps you weigh how much of that work you want to carry.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.