AI Vendor Lock-In: Keep the Gains In-House as Models Change

AI vendor lock-in happens when what makes your agent useful (its memory, instructions, connections and test results) lives inside a supplier's product instead of in assets your company controls. You reduce it by keeping those assets in exportable formats, under your own accounts, so a model or vendor can change without the agent forgetting how your business works. The model is a part you rent. The accumulated knowledge should be something you own.

Key takeaways

• Lock-in rarely sits in the model itself. It sits in the memory, prompts, integrations and evaluation history built around it.
• Keep company memory in plain, documented formats you store yourself, and treat embeddings and indexes as disposable copies that can be rebuilt.
• Write instructions as versioned documents for a role, with a thin model-specific layer on top, and keep a test set of real tasks that any model must pass.
• Register integrations, credentials and permissions in your own accounts, and expose tools through open standards where possible.
• Rehearse a model switch in a safe environment before you need one, and measure what breaks.

What does AI vendor lock-in look like in practice?

Most leaders picture lock-in as a contract that is hard to leave. In AI work the contract is usually the smaller problem. The larger one is dependence that builds quietly as the agent improves.

The common forms:

• Memory held in a vendor's proprietary store, with no export in a readable format.
• Prompts that live only in a vendor dashboard, or that depend on the quirks of one model.
• Integrations built with the vendor's credentials, so access disappears when the relationship ends.
• Fine-tuned models that cannot be moved to another provider.
• Evaluation results and logs that the vendor keeps and you cannot inspect.

Each one looks reasonable alone. Together they mean that after a year of improvement the agent is worth a great deal to you and nearly impossible to move. For the contract side of the problem, see How to Avoid AI Vendor Lock-In With Business Agents. This article focuses on how to structure the assets themselves.

Why do stronger models raise the stakes?

Models keep changing. New ones appear, terms shift and providers retire older versions. Whatever pace you expect from more capable systems, including ones that may one day be called superintelligent, the practical question for a company stays the same: when a better model exists, how quickly can you put it to work on your own business?

A company that owns its memory, instructions, tools and tests can try a stronger model, run its test set and adopt it once the results hold. A company whose context is trapped must wait for its vendor to adopt the change, or start again elsewhere. The gains from better models then go to whoever holds the surrounding assets. This is the logic behind a company superintelligence layer: the durable part is what you build around the model.

Be honest about what portability does not give you. A new model will behave differently. Portability means you can measure the difference and adjust, not that nothing changes.

How should memory be structured so it stays yours?

Memory covers three different things, and each needs its own home.

• Company knowledge: policies, product information, procedures and templates.
• Learned facts: what the agent records over time, such as a customer's preferred channel, a decision and who approved it, or a recurring exception.
• Conversation history: the record of what was asked and answered.

Rules that keep all three portable:

• Store them in your own cloud account or infrastructure, in readable formats: markdown or text for documents, JSON or CSV for structured facts, a standard database for records.
• Give every learned fact a source, a date and an owner. A fact without a source cannot be checked after a migration.
• Keep raw sources separate from summaries the agent generated. Summaries can be regenerated. Sources cannot.
• Treat embeddings and search indexes as disposable. They are derived from your sources and tied to a specific embedding model, so plan to rebuild them rather than move them.
• Prefer retrieval over fine-tuning for company knowledge. Retrieved knowledge can be edited, deleted and moved, while knowledge baked into model weights usually cannot. The AWS guidance comparing retrieval and fine-tuning is a useful reference.

Memory the agent writes back needs governance too: who can correct a wrong fact, and how the correction is recorded. And you must be able to restore it. AI Agent Backup and Restore covers what to save and how to test it.

How do you write prompts and tests that survive a model change?

Prompts are where a company's judgment gets written down: what the role does, what it must never do, when it escalates, how it speaks to customers. That judgment is yours even if the wording is tuned to one model.

Structure it in layers:

• Role definition: purpose, scope, audience and success criterion, in plain language.
• Rules and boundaries: approval thresholds, topics to refuse, escalation contacts.
• Examples: anonymized real cases showing good and poor outputs.
• Model adapter: the small layer of phrasing, formatting and tool-call instructions that one model needs. Expect to rewrite it when the model changes.

Keep all of it in a version-controlled repository your company can read, with a short note on why each edit was made. Anthropic's writing on context engineering for agents is a good guide to what belongs in the working context and what should be fetched on demand, which also keeps the model-specific layer small.

The most portable asset is not a prompt. It is a test set. Collect real tasks with expected outcomes and a rubric: the quote request that needs a human, the refund that is within policy, the ambiguous email. OpenAI's evals guide and Anthropic's writing on evaluating agents both describe the practice. Run the set on every model and prompt change. When a vendor changes, the test set tells you in an afternoon whether the replacement is good enough, instead of a month of complaints telling you.

How should integrations and permissions be set up?

Integrations are where lock-in turns from inconvenient into operational. If the agent reads the inbox and writes to the CRM using credentials the vendor created and holds, the vendor leaves and takes the access with it.

• Register applications and service accounts in your own identity provider and your own tool accounts. Use standard authorization such as OAuth 2.0, with scopes limited to the role.
• Keep secrets in a vault your company administers. The vendor receives permission to use them, not ownership.
• Place a thin tool layer between the model and your systems, with names that describe business actions (create a draft quote, look up an order) rather than a vendor's internal functions. A model change then touches the adapter, not your systems.
• Expose tools through an open standard where you can. The Model Context Protocol exists so that tools and data sources can connect to different AI applications in a common way.
• Host webhook endpoints and automation on your own domain, and document their configuration.
• Keep action logs in your own store. They show what the agent did and serve as evidence in any audit.

Also keep an inventory of what the agent can read, write and approve. Every later permission review is easier when that list already exists.

What does a model switch drill look like?

A switch drill is a rehearsal of moving to a different model or vendor, run in a safe environment with no write access to production. Use it as a step-by-step plan:

• Step one: List where memory, instructions, tools, credentials, tests and logs live today, and who holds admin rights to each.
• Step two: Mark each asset as exportable, rebuildable from other assets, or trapped.
• Step three: Fix the trapped assets first. Export them, move them into your accounts, or record the gap and decide whether the risk is acceptable.
• Step four: Refresh the test set from real work, with a pass threshold agreed by the process owner.
• Step five: Point the agent at a different model, rebuild indexes from source, and rewrite only the model adapter.
• Step six: Run the test set and record where the new setup is worse, equal and better.
• Step seven: Review failures with the process owner and sort them into prompt fixes, tool issues and model limits.
• Step eight: Note the effort the drill took, and repeat it on a schedule or when a significant new model appears.

Illustrative scenario: a distribution company has an agent that drafts replies to supplier delay notices. Over time it learns which suppliers have standing exceptions and which contacts approve changes. In one setup those notes sit in the vendor's hosted memory feature. In another they are dated records in the company's own storage, with instructions in a repository and credentials in the company's own accounts. When a new model appears, the first company can only ask its vendor whether support is coming. The second runs its test set in an afternoon, finds the new model handles exceptions well but formats escalations poorly, fixes the adapter and adopts it with approval. The events are the same. The ownership is different.

How does Be Human handle this?

Be Human designs and operates agents so that identity, memory, channels, tools, permissions and operating context stay under human control. In practice, memory is kept in formats and accounts the client can access, instructions are versioned documents, integrations use approved scopes, and an evaluation set is defined together with the first role and its success criterion. Managed operations include recurring evaluations, integration maintenance, permission review and controlled improvements, so the switch drill can be repeated when a model changes. Export and exit terms belong in the written scope, as described in AI Agent Statement of Work.

Frequently asked questions

Is it lock-in if we choose one model provider on purpose? Not necessarily. Using one provider is a choice. Lock-in is when you cannot leave without losing the work. If your memory, instructions, tools and tests are yours and you have rehearsed a switch, a single provider is a defensible decision.

Should we fine-tune a model on our company data? Usually start with retrieval and well-written instructions, because they are editable and portable. Fine-tuning can make sense for narrow, stable tasks, but the resulting weights are often tied to one provider. If you do it, keep the training data and the test set in your own storage so the work can be repeated elsewhere.

Does using several models remove lock-in? It helps, but it is not a cure. Several models behind one tool layer and one test set give you real options. Several models, each wired separately into your systems, just multiply the dependencies.

What should we ask a vendor about portability? Ask where memory is stored and in what format, who owns the credentials and app registrations, whether prompts and logs can be exported, and how long export takes. Then test the answers with a small export before you rely on them. A vendor that welcomes the test is giving you useful information.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.