Skip to content
be-human.ai
← Back to the journal

Context & memory

Vector Database or Plain Files for AI Agent Memory: A Decision Guide

Compare vector databases with reviewable plain files for AI agent memory. See the trade-offs in auditability, update speed and cost.

Be Human typographic cover with the title: Vector Database or Plain Files for AI Agent Memory: A Decision Guide
Conceptual illustration created with AI · Be Human

For most company agents, a vector database is worth adding only when the memory is large, varied and searched by meaning. For facts that must be checked, corrected and explained to a reviewer, reviewable plain files such as versioned Markdown or small structured records are usually the safer starting point, and a vector index can be added on top later. The deciding factors are who must read the memory, how often facts change, and what a wrong answer costs the business.

Key takeaways

  • Plain files give the strongest auditability: a person can open, compare, approve and roll back every fact the agent relies on.
  • Vector stores earn their place when the content is large, uneven in quality and searched by meaning rather than by exact names or codes.
  • A corrected plain file reaches the agent on the next read, while a vector index needs the changed document split and embedded again.
  • Running cost is mostly operational: indexing pipelines, re-embedding after a model change, and the review time needed to keep either form correct.
  • Start with a small set of reviewed files, then add a vector index only where retrieval clearly fails on real questions.

What each option actually stores

A vector database holds chunks of text as numerical vectors, stored together with metadata such as source and date. When the agent asks something, the system returns the chunks whose vectors sit closest to the question. This makes it good at finding passages phrased differently from the question, which is why it is popular for large document collections. Google Cloud's guidance on retrieval-augmented generation describes this pattern, in which retrieved content is supplied to a model alongside the question.

Plain files hold the same knowledge as readable text or records: a refund policy, a customer preference note, a product specification, each with an owner and a last reviewed date. The agent reads them directly or through a simple keyword or structured lookup. Nothing is hidden inside a representation that only an engineer can interpret, so the file serves as both the memory and the evidence for it.

The choice is less about which technology is more advanced and more about which representation your staff can inspect. For a finance rule or a refund exception, the key question is what exactly the agent knows, and the answer should be readable by a manager in a few minutes.

Auditability: can a person check what the agent remembers?

Auditing a vector store means looking at chunks, similarity scores and the way documents were split. Those artifacts help an engineer debug retrieval, but they rarely explain why a fact should be trusted. A chunk that cut a refund exception in half can look relevant and still produce a wrong answer, and the similarity score will not tell a reviewer that.

Plain files support a different kind of review. Each file has an owner, a version history and a diff that shows what changed and who approved it. If a fact was wrong last month, the team can find the file, see the previous text and roll it back. This is the same logic companies already apply to operating procedures, so it fits existing governance rather than adding a new layer.

Tracing matters in both cases. Retrieval calls, the files or chunks returned and the answer given should be logged so that a bad response can be followed back to its source. The OpenTelemetry primer explains the general practice of recording signals that let you trace a request through a system, and that discipline applies to agent memory as much as to any other service. For a deeper method, our guide to an AI agent memory audit shows how to review what an agent holds on a regular schedule.

Update speed: how fast does a correction reach the agent?

A plain file changes the moment it is saved and approved. The next time the agent reads it, the new fact is in effect. For small, frequently corrected sets such as delivery windows, product availability notes or escalation contacts, this speed is a practical advantage.

A vector store is slower to change. The corrected document must be split again, each new chunk embedded, the old chunks removed from every place they were copied, and the index updated without leaving stale vectors behind. Each step can be automated, but each one can also fail quietly. Teams that move to a new embedding model face a larger version of the same task, which is covered in our note on reindexing agent memory after an embedding model change.

The practical rule is simple. If a fact may change within the week and a stale answer would embarrass the company, keep it in a file the agent reads directly. Reserve the index for material that changes slowly and is too large to read whole.

A correction that takes a minute to make should not take a pipeline run to reach the agent.

Cost: what you pay for beyond the invoice

Cost comparisons often focus on the database bill, but the larger cost for a company is usually people's time. The categories to count are hosting the index, generating embeddings for new and changed content, reindexing when models or chunking rules change, monitoring retrieval quality, and the review hours needed to keep the content correct.

Plain files cost little to host and need almost no indexing. Their cost sits in editorial work: someone must write, own and retire each file. That work is real, but the business already does much of it for policies and procedures, and it improves the knowledge base even when no agent reads it.

Vector stores make sense when that editorial burden would be impossible to carry, for example across thousands of support tickets or long technical manuals. Evaluate them as a system with an owner, not as a storage line item. The FinOps Foundation's material on unit economics offers a useful frame: decide what one answer should cost to produce and what a wrong answer costs to repair, then compare both options on that basis.

A decision guide and a starting checklist

Use plain files first when most of these statements are true:

  • The memory is a few hundred documents or records at most, which a person could review in a working session.
  • Facts change weekly or faster, and a stale answer would cause a visible error.
  • Managers, auditors or customers may ask where a statement came from.
  • Staff already maintain the source material as policies or procedures.

Add a vector index when most of these statements are true:

  • The corpus is large, and finding relevant parts without search would be impractical.
  • Questions arrive in many phrasings, and keyword or structured lookup misses real matches.
  • You have a process to re-embed changed content and to test retrieval after each change.
  • A named owner reviews retrieval failures on a fixed routine.

Whichever you choose, the starting checklist is the same:

  • Name an owner for every memory source, including who approves changes.
  • Store the source, date and version next to each fact the agent may use.
  • Write ten to twenty real questions staff actually ask, with the answer each should produce.
  • Test the agent against those questions before and after every change.
  • Log which sources supported each answer.

Anthropic's guidance on context engineering makes a related point: give an agent the material a task needs, and keep the rest out of view. That principle applies to both storage forms. For a broader view of how to choose between retrieval approaches, our comparison of RAG and fine-tuning for business knowledge covers the same trade-offs from a different angle.

Illustrative scenario: a support team manages a refund policy, a warranty note and regional delivery rules. These documents change regularly, and an error would produce an incorrect promise to a customer. Plain files, each with an owner and a version, let the team correct a delivery window the same afternoon and show a reviewer the exact text the agent used. A vector index over the same files could be added later for the long archive of past cases, where searching by meaning is the real need.

How Be Human approaches this choice

Be Human designs agent memory around reviewable records first, then adds retrieval where the business needs it. In a first engagement, the work defines which facts belong in plain files, which material is large enough to justify an index, who approves each change, and how retrieval is tested against real questions. Managed operations then cover monitoring, recurring evaluations, permission review and controlled updates, so that a correction made by staff reaches the agent through a known path. The memory, tools and permissions stay under the company's control, and the storage choice is revisited when the content or the questions change.

Which memory task should you start with?

Choose one small, frequently used source where a stale or unexplained answer would cost the most, and move it into reviewed files before anything else. That first task usually shows whether your team will maintain the memory, and it makes the later decision about a vector index much easier to judge. The free Be Human AI diagnostic can help you find the first task worth handing to an agent.

Frequently asked questions

Can I start with plain files and add a vector database later? Yes, and this is the most common sequence for companies that want control first. The files become the source of truth, and the index is built from them, so approvals and version history stay in one place. When you add the index, keep the files as the origin of every fact so that a search result can always be traced back to an owned document.

Does a vector database make an agent more accurate? Not by itself. Accuracy depends on whether the right passage is retrieved, whether it is split in a way that keeps its meaning, and whether the model uses it correctly. A vector store can improve recall across large collections, but it also needs testing against real questions to show that it does.

How do I know whether retrieval is failing? Keep a set of real questions with the answers they should produce, run them after every change, and log the sources each answer used. When an answer is wrong, check whether the right source was retrieved at all before blaming the model. Repeating this routine each month turns a vague feeling about quality into a record a manager can read.

Is a vector database safe for sensitive company data? Its safety depends on the same controls as any other store: access rules by role, encryption, logging and a clear policy on what may be stored at all. Some material, such as credentials or personal data that the business does not need for the task, should stay out of memory entirely. Review who can read each source before the agent is connected to it.

Your next step

See where an AI agent fits in your company first.

Take the free Be Human AI diagnostic. A few questions about your work show which task to hand to an agent first, what it needs to know and where a person approves.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.

Less busywork. More room for you.

Bring us the work you wish you had time for.

Book a call to try a personalized AI agent free for 7 days ↗Or take the free AI diagnostic →