Personalized AI infrastructure
AI Implementation Partner: Questions to Ask Before You Sign
Questions to ask an AI implementation partner on memory ownership, permissions, evaluations and exit terms before you commit to a contract.

An AI implementation partner should be judged on what happens to your company's memory, permissions, evaluation records and exit path, not only on how fast the first agent reaches a demo. Before you sign, ask who owns each of those assets, how access is granted and reviewed, how quality is measured over time, and what you receive if the relationship ends. The questions below are written for business owners and operations leaders who must decide, not for engineers who will build the system.
Key takeaways
- Ownership comes first: confirm in writing that your memory, documents, instructions, configuration and logs belong to your company and can be exported.
- Treat permissions as a contract: every tool an agent can use needs a named owner, a narrow scope, an approval boundary and a scheduled review.
- Ask how quality is measured before launch, what counts as a failure, and who receives the evaluation results every month.
- Require a tested exit path, including export format, handover period and deletion confirmation, before you sign anything.
- Judge a first engagement by its boundaries: one role, the memory and tools it needs, and one observable success criterion.
Who owns the memory your agent builds?
An agent's memory is more than the underlying model. It includes the source documents it retrieves, the notes it stores, the operational context it picks up from conversations, and the corrections a person makes when the agent gets something wrong. After a year of use, that store becomes one of the most valuable records your team has, and it is easy to lose track of who controls it.
Ask the partner these questions and request the answers in writing:
- Which party owns the memory store, the indexes built from it and the summaries the agent writes?
- Where is the memory hosted, and in which region?
- Can we receive a complete export in a documented format, and how long would that take?
- Is our memory kept separate from the memory of any other client, and how is that separation tested?
- What happens to memory created from a departed employee's files or from private messages?
A common answer is 'we own the platform, you own the data.' That sentence does not settle whether your summaries, retrieval settings and tool configurations travel with you. Ask specifically about each of them.
Which permissions does the agent hold, and who approves changes?
Permissions decide what an agent can do without anyone noticing. Most early problems come from access granted for convenience: read access to an entire shared drive, write access to customer records, or an email account that can send messages under a senior manager's name. Those shortcuts are easy to approve during a demo and hard to unwind later.
Ask the partner to answer the following:
- List every tool and system the agent can call, with the exact scope granted for each.
- Which actions require a human approval, and who is the named approver?
- Do the agent's credentials sit apart from individual staff logins, and can they be revoked without disrupting your team?
- How often are permissions reviewed, and what evidence is kept from each review?
- How are access rules updated when the agent's role changes?
The OWASP authorization guidance is a useful reference here. It emphasizes checking access on every request rather than trusting the interface or the caller's assumptions. A partner that can explain how a permission is enforced, and not only that it was granted, is usually more careful with the rest of the system.
How will you measure whether the agent is working?
Ask for the evaluation plan before the build starts. An evaluation is a repeatable set of tasks with clear pass criteria, run against the agent before a change and again after it. Without one, the partner can only report that the agent seems better, and that statement cannot support a decision to expand its role.
The questions to put to the partner are these:
- What is the success criterion for the first role, written as something a person can observe in the business?
- Which test cases will be used, who wrote them, and how are they updated when the process changes?
- What is recorded for each task: the inputs, the tool calls, the outputs and any approvals?
- Who reviews failures, and how quickly do failed tasks reach that person?
- Will you show us evaluation results, including the cases the agent failed?
Anthropic's engineering guidance on evaluating agents describes combining automated checks with reading full transcripts. That combination is a practical standard to request. Automated scores show trends, while transcripts show why the agent chose a path, which is where most operational surprises appear.
The proof of concept, pilot or production distinction matters here, because each stage needs a different kind of evidence. Ask the partner which evidence it expects at each stage and who decides when a stage has passed.
What does your exit look like before you need it?
Exit terms are the clause most owners skip, and they are the one that matters most if the relationship fails. After launch, switching away from an agent costs more than the service itself, because the memory, the tool connections and the team's habits all depend on it. Settle the exit before the build, not during a dispute.
A usable exit package usually includes:
- A written list of what you receive on exit: memory export, instructions, prompts, evaluation sets, permission records and change history.
- An export format that someone on your team can read without the partner's software.
- A handover period during which both sides can run the agent in parallel and compare results.
- A written statement describing how credentials, copies and backups are removed from the partner's systems.
- Notice terms for ending or changing the service, and a clear statement of who bears the cost of the export work.
For a deeper look at dependency risk, see how to avoid AI vendor lock-in with business agents. Treat any partner that resists a test export as a warning, because a test export is the only way to confirm that the promised package actually works.
What happens to the agent after launch?
The first version of an agent is the cheapest moment to define operations. Once people depend on it, every incident and every change becomes a negotiation. Ask who watches the agent, who responds when it fails and how a change moves from request to live system.
The questions to settle are these:
- Who is alerted when the agent fails or a connected system changes, and which hours are covered?
- What is the path for a change request, and who gives final approval?
- How are changes from the underlying model or from a tool provider tested before they reach your team?
- Which reports reach you, how often, and in what form?
- Who decides when the agent should be paused, and how is the pause communicated?
A partner that only builds and then leaves may suit a short, well-defined project. An agent that touches customers, orders or money needs a named operating owner on both sides, and that owner should be visible in the contract.
Putting it into practice: a sequence for the first meeting
You do not need a technical background to run a strong first meeting. You need a single role, a clear list of questions and the discipline to request written answers.
- Ask for a draft statement of work before the second meeting, and check that it names memory, permissions, evaluations and exit terms as separate sections.
- Bring one role, one workflow and the person who will approve the agent's actions.
- Ask the partner to describe the memory and permissions for that single role on a whiteboard, then compare the description with the written draft.
- Score each answer as clear, partial or missing. Treat two or more missing answers in the ownership or exit areas as a stop signal.
- Request a short written evaluation plan and an exit summary before any build work begins.
Illustrative scenario: an operations director at a mid-sized distributor compares two partners. The first offers a faster demo but cannot say where the agent's supplier notes are stored or who approves a change to its email access. The second takes longer to scope, names the memory store, lists each tool with its scope and commits to a test export before go-live. The director chooses the second partner because its answers can be checked.
Be Human handles these questions as part of the engagement design. A first engagement defines one useful role, the memory and tools it needs, the approval boundaries and an observable success criterion, so the ownership and permission answers exist in writing before anything is built. Managed operations then cover monitoring, recurring evaluations, integration maintenance, permission review, incident response and controlled improvements, which keeps the evidence current after launch. The agent's identity, memory, channels, tools and operating context stay under your company's control.
If you want to find the first task worth handing to an agent, take the free Be Human AI diagnostic.
Frequently asked questions
Should we ask for references before signing? Reference conversations can help, but they should not replace direct questions about memory, permissions, evaluations and exit terms. Ask the partner to show how it handles each of those areas, and treat any unverified outcome as a claim rather than evidence. You can judge the answers in front of you without relying on anyone else's experience.
Do we need a lawyer to review the exit terms? For a contract that gives a partner access to company memory and tools, legal review is sensible, but it should be paired with an operational review. Legal review can check the wording of ownership, deletion and liability, while your operations lead checks whether the export is actually usable. Both reviews should be complete before signing.
Is a shared AI platform with other clients a problem? Not automatically. The real question is whether your memory, instructions and records are isolated from other clients, who can access them and how that isolation is tested. Ask for the answer in writing and check whether the partner can show separate stores or scopes for each client.
How early should the people who will use the agent be involved? Involve them before the scope is written, so the role matches the work they actually do. Users can point out which tasks are repetitive, which answers need a person's approval and where the agent would create extra checking. Starting with their input avoids a permission model built around the wrong workflow.
Your next step
See where an AI agent fits in your company first.
Take the free Be Human AI diagnostic. A few questions about your work show which task to hand to an agent first, what it needs to know and where a person approves.
Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.


