Superintelligence Vendor Security Questionnaire for Agent Providers
Questions to put to any AI agent provider before signing, on access, logging, data use, subprocessors and exit, with checks you can run.

An AI vendor security questionnaire is a written list of questions a company puts to an agent provider before signing, covering access, logging, data use and exit. Its purpose is to replace demo impressions with written answers you can check: what the agent can reach, what it records, who can read that record, and how your company gets its data and configuration back. Send it before the commercial discussion ends, and treat any provider that answers only in general terms as not yet ready to sign.
Key takeaways
- Ask every agent provider for written answers on access, logging, data use and exit before you compare features.
- Require the agent to run under its own identity with scoped permissions, never under a shared administrator login.
- Get the retention period for logs, memory and prompts in writing, along with the list of people who can read them.
- Confirm in the contract that your data is not used to train shared models unless you explicitly agree.
- Test the exit before you need it: ask for a documented export of memory, configuration and logs, and for written confirmation of deletion.
Why the questionnaire comes before the demo
A demo shows what an agent can do with prepared inputs on a good day. It does not show which systems the agent can reach, what it records when something goes wrong, or which people at the provider can read those records. A written questionnaire turns those gaps into answers that legal, security and operations colleagues can compare across vendors.
The NIST AI Risk Management Framework groups this work into four functions: govern, map, measure and manage. You do not need to adopt the framework to borrow its structure. Access and exit questions mostly belong to govern and manage, logging belongs to measure, and data use belongs to map, because it asks where your information actually flows.
Ask for evidence alongside every answer. A yes to 'do you log tool calls' means little without a sample log entry, a retention setting and an export format. If a provider cannot show you the artifact, record that gap in the questionnaire response.
Access: what the agent can reach and who can change it
Access is where most agent risk starts. An agent that can read a shared drive and send messages can cause harm faster than a person in the same position, and it can do so at any hour. The OWASP authorization guidance rests on a simple principle: grant only the access a function needs, and check that access at the point of use. Put these questions to every provider:
- Does the agent run under its own identity in our systems, or does it borrow a staff login or a shared administrator account?
- Which tools and data sources does it call by default, and can each one be limited to read only, or to specific folders, records or channels?
- Who on the provider side can change the agent's permissions, and is every change recorded with a reason?
- Which actions can require a second approver before they run, and can we set that rule ourselves rather than accept the vendor's default?
- How are credentials stored, rotated and revoked, and how quickly does a revocation stop the agent from acting?
Ask for answers written in terms of your setup, not the vendor's product language. 'We support role-based access' is a feature claim. 'The support agent can read ticket history and draft replies, but cannot issue refunds or change customer records' is a configuration you can test in a staging environment.
Logging: what you can see after something goes wrong
A log is only useful if it answers the questions you will ask after an incident: what the agent was asked, what it looked at, what it did, who approved it, and what failed. The OpenTelemetry observability primer describes logs, traces and metrics as separate signals that work together. That vocabulary helps you ask precisely what the provider records and what you can export.
- Which events are logged, including prompts, retrieved sources, tool calls, tool results, approvals, rejections and errors?
- Does each entry carry a timestamp and the identity that triggered it, so a sequence of actions can be rebuilt in order?
- Can we export logs to storage we control, in a documented and standard format, without waiting for support to prepare them?
- How long are logs kept by default, and can we set a shorter or longer period in writing?
- Are personal data masked in logs, and what happens to a log entry when a customer asks for deletion?
- Can a support engineer at the provider open our logs, and is that access itself logged and visible to us?
Audit detail and personal data pull in opposite directions. Keep enough detail to reconstruct a decision, and mask what you do not need to keep. The guide on PII redaction for AI agent logs sets out how to strike that balance before the first incident forces the question.
Data use: training, retention and subprocessors
Three different things often get blurred together: the data placed in the agent's working context, the data the provider stores, and the data the provider may use to improve its systems. Ask about each one separately. A vendor can keep conversation history for a limited period and still never train on it, or it can do the reverse. Do not accept a single sentence that covers all three.
- Is our data used to train or fine-tune any model, including models shared with other customers, and is that the default or an opt-in?
- Where is data stored and processed, and can we choose the region for memory, uploaded files and logs?
- Which subprocessors touch our data, and how and when are we told when that list changes?
- What is the retention period for conversation memory, uploaded files and tool outputs, and who can shorten it?
- Does the data processing agreement cover agent memory, logs and prompts, not only the core service?
Subprocessors deserve their own review, because an agent often calls several tools, each run by a different company. The article on AI vendor subprocessors explains how to map that chain before you sign.
Illustrative scenario, not a real engagement: a services company asks a vendor where its support agent's memory is stored. The answer names the main region but says nothing about backup copies. The team adds a clause requiring written notice before any new storage location is used, and asks for the backup location to be stated in the questionnaire response.
Exit: getting memory, configuration and logs back
Exit is the question buyers most often skip, and it becomes expensive later. An agent accumulates memory, tool configuration, prompt history and approval records. If all of that lives only inside the provider's system, leaving means losing context your team spent months shaping. The Model Context Protocol introduction describes an open way to connect models to tools and data, which is one reason to ask whether your tool definitions can move with you if you change providers.
Use this checklist as the exit section of the questionnaire:
- A written export of memory, configuration, prompts and logs in a documented format, tested on a sample before you need it
- A deletion timeline after termination, with written confirmation that memory and backup copies are removed
- Revocation of all credentials and tool connections on a stated schedule
- A rule for queued or in-progress actions at termination, such as pausing them rather than letting them complete
- A named contact for incidents during the transition period
Exit also connects to your own staff. When someone leaves, the same tools and approvals must be reassigned or removed, and the same-day offboarding checklist for agent tools shows the internal side of that process.
How Be Human handles these questions
Be Human designs, implements and operates personalized AI infrastructure in which each agent keeps its identity, memory, channels, tools, permissions and operating context under human control. A first engagement defines one useful role, the memory and tools that role needs, the approval boundaries, and an observable success criterion, so the access and data questions above have concrete answers before anything is connected. Managed operations then cover monitoring, recurring evaluations, integration maintenance, permission review and incident response, so the same access and logging questions are revisited as the agent's scope changes. The service is delivered remotely, in English and Portuguese.
Turn the answers into a go or no-go decision
Before signing, score the answers against a short set of conditions. Proceed only when the following are true:
- Every access answer maps to a named tool, role and approval rule in your own setup
- A redacted sample log entry is available for your review
- Training use and retention are stated in the contract, not only in the sales material
- The subprocessor list is complete and comes with a notice mechanism for changes
- The exit export has been tested on a sample dataset
If a condition is missing, the answer is not no for good, but the contract should wait until the gap is closed in writing. Compare the final scorecard with your pre-launch security review for agents before the agent touches live systems. If you are not yet sure which task should go to an agent first, the free Be Human AI diagnostic helps you find the first task worth handing to an agent.
Frequently asked questions
Can a provider refuse to answer these questions? Yes, and that refusal is itself information. A provider that will not describe its logging, data use or exit in writing before you sign is asking you to accept risk you cannot measure. Ask for the reason in writing and compare the response with other vendors before you continue.
Do we still need a legal review if we use this questionnaire? Yes, for the contract. The questionnaire gathers facts and evidence, while the data processing agreement and the terms turn those facts into obligations. A legal or privacy reviewer should confirm that each important answer appears in the signed document, not only in a sales conversation.
Is a questionnaire enough for a small company without a security team? It is a good starting point, but it works best with one named owner who keeps the answers and reviews them after each material change. Keep the list short, store the responses with the contract, and repeat the key questions whenever the agent gains a new tool or a new category of data.
How often should we repeat the questionnaire with the same vendor? Repeat it when the agent's scope changes, when the vendor announces a change to its subprocessors or data handling, and during your regular review cycle. Compare each new set of answers with the written record from signing, so that changes in practice are visible rather than assumed.
Your next step
See where an AI agent fits in your company first.
Take the free Be Human AI diagnostic. A few questions about your work show which task to hand to an agent first, what it needs to know and where a person approves.
Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.


