Skip to content
be-human.ai
← Back to the journal

Personalized AI infrastructure

Where to Host Company AI Agents: Cloud, Vendor or On-Site

Compare cloud, vendor-hosted and on-site options for an AI agent layer by data control, log access and how easily agent memory can move later.

Be Human typographic cover with the title: Where to Host Company AI Agents: Cloud, Vendor or On-Site
Conceptual illustration created with AI · Be Human

For most companies, the safest place to host an AI agent layer is a cloud account that the company owns and administers, with agent memory, logs and permission rules stored in formats it can export. Vendor-hosted agents start faster but usually give the company less visibility into logs and less control over where memory lives. On-site hosting offers the most direct control, but it adds operating work that most business teams should take on only with a clear reason.

Key takeaways

  • The model, the agent layer and the memory it uses are three separate decisions, and each can run in a different place.
  • A company-owned cloud account gives the best balance of data control, log access and portability for most mid-sized firms.
  • Vendor hosting can suit a short pilot with low-sensitivity data, but check log export, memory export and deletion terms in writing before you commit.
  • On-site hosting makes sense when a legal duty, a client contract or an internal policy requires it, and when someone will run it reliably.
  • Rehearse a memory export into a second environment before go-live, because a move you have never tested is a move you cannot count on.

What the hosting decision actually covers

An AI agent layer is more than a model call. It includes the agent's identity, its memory (source documents, records, summaries and the index used to find them), the tools it can call, the permission rules that govern those tools, the channels where people reach it, and the logs that show what it did. Most companies rent the model from an outside provider, so the hosting question is really about where the rest of the layer runs and where its data is stored.

Separate three questions. Where does the memory live? Where do the logs live? Where do the tools run, and which credentials do they hold? A retrieval-based design, the pattern described in Google Cloud's architecture guidance for retrieval-augmented generation applications, pulls relevant documents from a store at the moment of an answer. That makes the document store and its index the center of the decision, because whatever the agent knows at answer time comes from there.

How do cloud, vendor-hosted and on-site options compare on data control?

Cloud hosting in an account your company owns means the company, or the technology partner it trusts, controls the cloud project, the network boundaries, the encryption keys and the roles that can read the memory store. Control is high, and the work is setting these boundaries correctly and keeping them current. Your team can revoke access without waiting for a provider to act.

Vendor hosting means the agent platform runs in the provider's environment. Control then depends on the contract and on technical details: which regions store data, which of the vendor's staff can reach tenant data, which subprocessors are involved, and how deletion is confirmed. A vendor-hosted setup can be a sensible first step, but the questions shift from where the data sits to what exactly the company holds in writing.

On-site hosting runs the layer on servers the company controls, either on its premises or in a private facility. It gives the most direct administrative and physical control, which suits organizations with strict internal rules. The cost is operating work. Patching, backups, availability, access reviews and incident response all become the company's job. Unless the model also runs locally, model calls still leave the building, so on-site hosting does not by itself keep every byte inside.

For companies in Brazil or subject to the LGPD, questions about data location and transfer deserve a written review from the first day. Our guide on AI data residency for business explains where company data typically lives under each option and what to ask about it.

Who should be able to read the agent logs?

Logs are the audit trail of an agent. The OpenTelemetry observability primer describes how traces, metrics and logs help a team understand what a system did and why. The same logic applies to agents: a useful trace records the request, the memory retrieved, each tool call, each approval request and each error. Without that record, a permission review or an incident review turns into guesswork.

Ask four questions of any host. Can your operations leader open the logs without filing a ticket with the vendor? How long are logs kept, and can you change that period? Do the logs contain personal data or confidential content, and if so, who can see it? Can you export them to your own systems? Access to logs should follow the same least-privilege rule as access to tools. The OWASP authorization guidance recommends checking access rules on every request instead of trusting the interface, and that principle covers the log viewer as much as the agent.

Expect logs to hold sensitive material. Prompts and retrieved passages can include customer names, quoted prices or contract terms. Plan redaction and role-based viewing from the start, not after the first audit question arrives.

How easily can agent memory move later?

Memory is the part of the layer most likely to become stranded. Source documents are usually easy to copy, but the retrieval index built from them is often tied to a particular embedding method, and summaries or learned preferences may exist only in the platform's own format. Before signing, ask what leaves the system in a readable form and what would have to be rebuilt.

A practical test has three parts. Keep the source documents in a store the company controls. Keep a record of which permission tags apply to each document. Then rehearse rebuilding the index in a second environment. If the rehearsal works, switching hosts becomes a project with a known scope. If it does not, the memory effectively belongs to the vendor, whatever the contract says about ownership. The article on how to avoid AI vendor lock-in covers the contract terms that matter most here.

Illustrative scenario: a regional distributor starts with a vendor-hosted agent that answers order status questions for its sales team. A few months later, the operations director wants the same agent inside the company's own cloud account so it can connect to a new warehouse system. The team had kept its product catalog and policy documents in its own file store, with permission tags exported alongside them, so the move is a bounded migration. Had the memory lived only in the vendor's dashboard, the team would be starting over, and the warehouse connection would wait.

Which host fits your company?

Use these four rules as a starting point, then test them against your own data and approval needs.

  • Choose a company-owned cloud account when the team needs its own logs and memory under its administration and expects tools and channels to change over time.
  • Choose vendor hosting for a short pilot with low-sensitivity data, provided the contract states log access, memory export, regions and deletion in writing.
  • Choose on-site hosting when a legal duty, a client contract or an internal policy requires it, and the company already runs infrastructure reliably enough to carry the added work.
  • Choose a split design when memory, logs and permissions sit in a company-owned store while a reviewed external model service handles generation.

Be Human's managed approach handles this decision as part of the first engagement. The setup defines which role the agent holds, where its memory and logs are stored, which tools it can reach, what each tool may read or write, and which actions need human approval. Managed operations then keep the layer observable, review permissions on a schedule and respond to incidents when a connector or host changes. The company keeps control of the agent's identity, memory and operating context throughout.

Where to start this month

Start with the decision, not the vendor list. Work through these steps in order:

  • Write the agent's first role in one sentence and list the data it needs to answer questions in that role.
  • List every system the memory touches and name the person who administers each one.
  • Ask each candidate host for log access, retention period, regions and export formats in writing.
  • Export a sample of memory and rebuild its retrieval index in a second environment to see what really moves.
  • Set the first permission review date before go-live, and record who approves changes to tools or memory sources.

If you want help identifying the first task that justifies this setup, take the free Be Human AI diagnostic to find the first task worth handing to an agent.

Frequently asked questions

Is on-site hosting always the most secure option? Not necessarily. On-site hosting gives direct control over hardware and access, but it also moves patching, backups and access reviews onto your own staff. A well-run company-owned cloud account can have stronger controls than a poorly maintained server room, so the deciding factor is who will operate the layer and how reliably.

Can we use an outside model provider and still own our agent memory? Yes, and this split design is common. Keep the source documents, retrieval index, permission rules and logs in a store your company controls, and send only the context each request needs to the model. Review the provider's data terms before go-live, since those terms govern what the model side may retain.

Who should have access to the agent logs? Access should follow the roles that need it, such as an operations lead for incident reviews and a security or compliance owner for audits. Engineers who maintain connectors may need to read traces without seeing unrestricted personal data. Record who can view logs and review that list during each permission review.

What should we ask a vendor before choosing a hosted option? Ask where data is stored and processed, which staff can access tenant data, how long logs are kept, and whether logs and memory can be exported in readable formats. Ask how deletion works and how the vendor confirms it. Get these answers in writing, because a verbal assurance will not help you move memory later.

Your next step

See where an AI agent fits in your company first.

Take the free Be Human AI diagnostic. A few questions about your work show which task to hand to an agent first, what it needs to know and where a person approves.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.

Less busywork. More room for you.

Bring us the work you wish you had time for.

Book a call to try a personalized AI agent free for 7 days ↗Or take the free AI diagnostic →