Red Flags When Hiring an AI Implementation Partner: What to Check

The clearest red flags in an AI consultant are vague scope, demos that cannot be repeated on your own material, no plan for running the system after launch, and contracts that leave ownership, access and exit unclear. A partner who can deliver will define one role, name the approval boundaries, show how success is measured and explain who operates the agent months after go-live. If a proposal avoids those points, treat its polish as a warning rather than a comfort.

Key takeaways

• A credible partner narrows the work to one role, with approval boundaries and an observable success measure, before talking about models or tools.
• A demo counts only if it runs on your material, shows a failure honestly and can be repeated by your own people.
• The contract should say who owns prompts, configuration, memory and logs, and how you leave without losing them.
• Ask who monitors, evaluates and fixes the agent after launch, and on what routine. A vague answer usually means nobody.
• Certainty without limits is the pattern behind most bad engagements: promises of results, no stated exclusions, no discussion of what the agent must never do.

What in a proposal suggests the partner cannot deliver?

A proposal is the first place a weak partner shows itself, because writing a precise one requires having done the work before. Weak proposals tend to describe an ambition. Strong ones describe a job.

Watch for these signs:

• The scope is a company-wide transformation instead of one defined role with a start and an end.
• Nobody on your side is named as the owner of the process the agent will touch.
• Success is described as adoption, efficiency or innovation, with no measure you could observe in a normal week.
• Models, platforms and tool names lead the document, while the process, the data and the people are barely mentioned.
• Permissions are not discussed. The proposal never says what the agent may read, what it may change and what needs a human to approve.
• The timeline has no review points, only a launch date.
• References are polished stories with no detail on what the partner built, what the client did, and whether the system is still running.

That last point deserves care. A case study is marketing unless you can see its method, its limits and its current state. The article on how to evaluate AI case studies without being misled covers what to ask when a partner leans on past work as proof.

Also notice what the proposal does not say. If it never mentions what the agent will get wrong, how errors are caught, or who is told when something fails, the author has probably not operated a system like this for long.

Which demo behaviors should make you cautious?

A demo is persuasive by design. Your job is to find out whether it is representative. The most common problem is not fakery, it is a controlled environment that hides how the system behaves with real inputs.

Be cautious when:

• Every input is typed by the presenter, and you are never allowed to supply your own.
• The data is generic sample material rather than something that resembles your documents, your naming habits and your mess.
• No answer cites where it came from, so you cannot tell retrieval from invention.
• The demo never fails. Real systems fail on ambiguous requests, missing records and contradictory documents, and a partner who has operated one will show you how that failure is handled.
• The agent acts with broad credentials, such as full access to a mailbox or a shared drive, because that was easier to set up.
• There is no log. If you ask what the agent did a moment ago and the answer is a recap from memory, there is no audit trail to rely on.
• The presenter cannot say what happens when a document contains instructions aimed at the agent. This is a known class of attack, described in guidance on prompt injection from OWASP, and a competent partner has a position on it.

A useful move is to ask for a short session on a small sample of your own material, with a few awkward cases you choose in advance. The guide on how to test an AI vendor's claims before you commit walks through how to set that up without a long project.

Pay attention to how the partner reacts. A good one is interested in the awkward cases, because they are the real work. A weak one steers back to the prepared flow.

What should worry you in the contract?

Contracts reveal who expects to be accountable. Several clauses, or their absence, predict how a relationship goes when something breaks.

• No defined deliverables. The agreement describes hours or a retainer but not what exists at the end, such as a documented role, a tested configuration or a handover pack.
• Unclear ownership. Prompts, workflows, memory, evaluation sets and logs are not assigned to you, or they are licensed in a way that stops working when the contract ends.
• No exit path. There is no description of export formats, transition support or how credentials and integrations are returned.
• Silence on data use. The agreement does not say whether your company material can be used to train anything, or where it is stored and processed.
• Credentials held only by the partner. If the accounts, API access or hosting are all in their name, you cannot leave quickly.
• Open-ended change rights. The partner can alter the agent in production without a request, a review or a record.
• Responsibility language that is one-sided. Everything that goes wrong is the client's responsibility, or nothing is anyone's.

A statement of work is where most of these get fixed. The article on what to put in an AI agent statement of work lists the sections to demand, and the one on avoiding AI vendor lock-in explains which dependencies are worth worrying about and which are normal.

One more flag sits outside the text. If a partner resists written answers and prefers to settle everything on calls, the lack of a record will matter later.

Who runs the agent after launch, and how do you find out?

This is the question most buyers ask too late. Building an agent is a project. Operating one is a service, and the skills are different. Models change, integrations break, documents go out of date, people leave and permissions drift. A system that was correct at launch will not stay correct without attention.

Red flags about operations:

• Handover is a document and a short call, then the partner is gone or billable only by request.
• The partner says the model is capable enough that little maintenance is needed.
• There is no monitoring plan. Nobody is described as watching errors, cost, latency or unusual actions.
• There is no recurring evaluation. Quality is judged by whether anyone complains.
• Changes are made by whoever has access, with no request, approval or rollback. A calmer process is described in how to request and approve changes to a managed AI agent.
• One person at the partner holds all knowledge of the setup, so their absence stops everything.
• Incident response is not defined. You do not know whom to contact, how fast the agent can be paused, or who decides.

Observability is a useful test. Ask what the partner would record to know whether the agent is healthy, and how they would trace a wrong answer back to its inputs. General observability concepts, such as those in the OpenTelemetry primer, give you the vocabulary to judge whether the answer is concrete.

How Be Human handles this: a first engagement defines one useful role, the memory and tools it needs, the approval boundaries and an observable success criterion. After launch, managed operations can include monitoring, recurring evaluations, integration maintenance, permission review, incident response and controlled improvements, with the agent's identity, memory, channels and permissions kept under human control on the client's side. The point is that the person who built the agent and the person who keeps it healthy are accountable to the same plan.

If your reason for hiring is to avoid running the system yourself, compare the options in AI automation agency vs managed AI provider before signing anything that ends at launch.

A checklist for your next vendor call

Use these questions in a call or in writing. They are built so that a prepared partner answers briefly and a weak one drifts.

• Which single role will the agent hold first, and what will it not do?
• Who on our side approves actions, and which actions need approval?
• What does the agent read, write and send, and with what level of access?
• How will we know it is working, using something we can observe without asking you?
• What happens when the agent is wrong, and how is that detected?
• Show us one failure from a past engagement and what changed afterwards, without naming the client.
• Who monitors the system after launch, and what do they check each cycle?
• How are changes requested, approved, tested and reversed?
• Who owns the prompts, configuration, memory, evaluation sets and logs?
• If we end the contract, what do we receive, in what format and with what help?
• Where is our data stored and processed, and is it used to train any model?
• Who is the second person who understands our setup?

Score the answers by pattern, not by polish. A missing answer is not a failure if the partner commits to a written reply. Repeated evasion, answers that restate the question, or answers that rely on reassurance are the signal to slow down. Several unanswered items in a row mean you should pause the process, not negotiate harder.

Illustrative scenario: two answers to the same request

This is an illustrative example, not a real case. A distribution company wants an agent to handle routine supplier order-status emails. It asks two partners for a proposal.

The first partner proposes an intelligent operations layer for procurement, finance and customer service. The demo is smooth, uses sample emails from the partner's library, and the agent has full mailbox access. The contract is a monthly retainer with no listed deliverables, and the prompts are described as proprietary tooling. When asked who watches the agent after launch, the answer is that the system learns over time.

The second partner proposes one role: read incoming status requests, look up the order in the company's system, draft a reply and send only the ones that match a defined pattern, leaving the rest for a person. The proposal names the buyer's operations lead as approver, lists what the agent can read and write, and defines a success measure the team can check weekly, such as drafts accepted without edits. The demo uses five real emails the buyer chose, including one with a wrong order number, and the agent flags it instead of guessing. The statement of work assigns ownership of configuration and logs to the company and describes the monitoring routine.

Neither partner is guaranteed to succeed. But only the second has given the buyer ways to find out, early and cheaply, whether the work is sound. That is the difference to look for.

What if the red flags are already there?

If you are already in an engagement and recognise several of these signs, you have options short of ending it. Ask for the missing pieces in writing: scope, permissions, success measure, ownership and operating routine. Request a reduced-scope checkpoint, such as one role in controlled use, so you can see evidence instead of promises. The article on what proof of concept, pilot and production each must prove helps you decide what evidence is fair to expect at each stage.

If the partner responds with clarity, the relationship may recover. If the response is defensiveness or more demos, you have learned what you needed to know while the cost of leaving is still low.

Frequently asked questions

Is a polished demo a bad sign? Not by itself. A polished demo is a problem only when you cannot change the inputs, see a failure or inspect what the agent did. Ask to bring your own sample and judge the partner by how they handle awkward cases.

Should I avoid partners who will not give client references? Not automatically, since many engagements are confidential. But a partner who cannot describe past work in concrete, anonymised terms, including what went wrong, has given you little to evaluate. Ask for the method and the lessons, and weigh those over a list of logos.

What is the single most important contract clause? Ownership and exit. If you own the configuration, memory, evaluation material and logs, and the contract describes how they are handed back, you keep the ability to change partners. Most other problems become easier to fix when that clause is sound.

How long should I spend vetting a partner? Match the effort to the access they will have. A narrow, low-permission first role needs a shorter review than an agent that can send messages or change records. Whatever the size, a short working test on your own material is worth more than another presentation.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.