How to Build a Company AI Assistant That Knows Your Business

To build a company AI assistant, start with one defined role. Then give it only the knowledge, memory, tools and permissions that role needs, and put a human review step in front of anything it cannot undo. Most of the work is organizing how your business actually runs, not choosing a model. Build it in named layers so each one can be tested, corrected and replaced on its own.

Key takeaways

• An assistant that knows your business is a role with scoped knowledge, not a general chatbot with every file attached.
• Keep knowledge (documents that are maintained), memory (what the assistant learns over time) and tools (what it can do) as separate layers with separate owners.
• Permissions should follow the role, not the person who set it up. Give the assistant its own identity and the narrowest access that works.
• Review is a design decision made before launch: decide which actions are drafted for a human and which can run alone.
• Test against real questions from your own team, and treat the assistant as something you operate, not something you install once.

What does it mean for an assistant to know your business?

Knowing your business is not the same as having read your files. A new employee who has been handed a shared drive with thousands of documents does not know the business. They know where the documents are. Knowing the business means understanding which version of a policy is current, which customers have special terms, who approves what, and when to stop and ask.

An assistant gets there through layers. Each layer answers a different question, and mixing them is the most common reason assistants give confident but wrong answers.

• Role: what job is this assistant doing, for whom, and what is out of scope?
• Knowledge: which maintained sources can it rely on to answer?
• Memory: what should it remember between conversations, and who can correct it?
• Tools: which systems can it read from or act in?
• Permissions: under which identity, and with what limits?
• Review: which outputs need a human before they leave?
• Operations: who watches it, tests it and changes it?

This is a simplified version of the model described in The Seven Layers of Company AI Infrastructure, Explained. The names matter less than the habit of keeping the layers apart so a problem can be traced to one of them.

Step one: define one role before you touch any documents

The temptation is to begin by collecting everything: every policy, every shared folder, every old email thread. Resist it. Without a role, you cannot decide what belongs in the assistant's knowledge and what is noise.

Write the role the way you would write a job description. Name the person or team it serves, the recurring requests it handles, the systems it touches and the point at which it hands over to a human. The guide Write a Job Description for Your First AI Agent Role walks through this in detail. A useful role is narrow enough that you can name ten real questions it should answer and five it should refuse.

Illustrative scenario: a distribution company wants help for its customer service team. Rather than an assistant that knows the whole company, the role is order status and returns policy for the service desk. It answers staff questions, drafts replies, and routes anything involving credits or exceptions to a team lead. That one sentence already tells you which documents matter (returns policy, shipping terms, escalation rules) and which do not (marketing plans, finance reports, HR files).

Step two: how do you turn scattered documents into scoped knowledge?

Scoped knowledge is a short list of sources, each with an owner, a purpose and a way to stay current. Scattered documents become scoped knowledge through a sequence of small decisions.

• List candidate sources for the role only. Include documents, wiki pages, ticket histories and any spreadsheets people really use.
• Mark each source as authoritative, useful but informal, or outdated. Remove the last group before anything is indexed.
• Resolve conflicts. When two documents disagree, a person decides which wins and the other is retired or labeled.
• Assign an owner to each source. If nobody is responsible for keeping it current, it should not be in the assistant's knowledge.
• Split by sensitivity. Pricing rules, personal data and contractual terms may need to stay out of reach of some roles entirely.
• Decide how updates reach the assistant, and how quickly.

Technically, most assistants use retrieval: at question time, the system finds relevant passages from your sources and gives them to the model along with the question. This keeps answers tied to documents you control, and it lets you update knowledge without retraining anything. The [AWS comparison of retrieval and fine-tuning](https://docs.aws.amazon.com/prescriptive-guidance/latest/retrieval-augmented-generation-options/rag-vs-fine-tuning.html) explains why retrieval is usually the first choice when information changes often.

Two habits improve quality more than any tooling. First, make the assistant cite which source it used, so a reader can check. Second, let it say it does not know. An assistant that admits a gap is far more useful than one that fills it with a plausible guess.

Step three: what is the difference between knowledge and memory?

Knowledge is what your company maintains on purpose. Memory is what the assistant accumulates through use: a preference a user stated, a decision made last week, the context of an ongoing project. Both help the assistant feel informed, but they carry different risks.

Knowledge has an owner and a review cycle. Memory often has neither, unless you design it. Left alone, memory drifts: a temporary exception is remembered as a rule, a one-off instruction from one employee shapes answers for everyone, or an outdated fact survives long after the source was fixed.

Set a few rules early:

• Decide what the assistant may store, and what it must never store, such as personal data that has no reason to persist.
• Make memory visible. A person should be able to read what the assistant remembers and correct or delete it.
• Scope memory by person, team or company, so one user's context does not leak into another's conversation.
• Promote durable facts from memory into the maintained knowledge, with an owner, instead of letting them live only in memory.

Anthropic's article on [effective context engineering for AI agents](https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents) is a good technical companion here. Its core point applies to business too: more context is not better context. What the assistant sees on each request should be chosen, not dumped.

Step four: give it tools and permissions on purpose

Knowledge lets an assistant answer. Tools let it act: look up an order, create a ticket, draft a document, update a record. Each tool is a connection to a real system, so each one deserves a short, deliberate decision.

For every tool, answer four questions. What does it let the assistant read? What does it let the assistant change? Under whose authority does that happen? What happens if it is wrong? Start with read-only access and add write access only where the role clearly needs it. Standards such as the [Model Context Protocol](https://modelcontextprotocol.io/docs/getting-started/intro) exist to make connecting assistants to tools more consistent, which helps you avoid building one-off integrations that nobody can maintain.

Permissions are where many first attempts go wrong, usually because the assistant borrows a staff member's login. That makes its actions hard to trace and its access far wider than the role needs. The better pattern is a separate identity for the assistant with its own accounts and scoped rights, as argued in Why Your Company AI Agent Needs Its Own Identity and Accounts. The [OWASP authorization guidance](https://cheatsheetseries.owasp.org/cheatsheets/Authorization_Cheat_Sheet.html) describes the underlying principles: deny by default, grant the least privilege that works, and check authorization on every request, not only at login.

One more point: text inside documents and emails can contain instructions aimed at the assistant. Treat retrieved content as information, not as commands, and keep tool permissions narrow enough that a manipulated assistant cannot do serious damage.

Step five: where should a human review the work?

Review is not a sign of distrust. It is how you let an assistant take on real work before it has earned wide authority. The question is not whether to review, but where.

A simple decision guide for each action the assistant can take:

• Reversible and internal, such as summarizing a document or searching a wiki: let it run, and sample the results.
• Reversible but visible to others, such as drafting a customer reply: the assistant drafts, a person sends.
• Hard to reverse, such as changing a record, issuing a credit or sending a commitment: a person approves each time, with the assistant's reasoning and sources attached.
• Outside the role or touching sensitive data: the assistant stops and hands the case to a named person.

When a case is escalated, send the decision, not the mess: a short summary, the relevant sources and the options, so the person can act in a minute instead of rereading a whole thread. Running the assistant in observation mode first, where it drafts and humans compare, is a low-risk way to build evidence. The article Run an AI agent in shadow mode before giving it authority covers how.

Testing belongs here too. Collect real questions from the team, including awkward ones, and write down what a good answer looks like. Run them before launch and again after every change to knowledge, tools or model. The first set can be small, as long as it is real and kept.

How does Be Human handle this?

Be Human designs, implements and operates this stack as personalized AI infrastructure. A first engagement defines one useful role, the knowledge, memory and tools it needs, the approval boundaries and an observable success criterion. The assistant gets its own identity and scoped permissions, and humans stay in control of what it may do. Managed operations then cover monitoring, recurring evaluations, integration maintenance, permission review, incident response and controlled improvements, delivered remotely in English and Portuguese.

What should you do first this month?

You do not need a platform decision to begin. You need a role, a short source list and an owner. A practical first sequence:

• Pick one role with a clear audience and a handful of recurring requests.
• Write the role description, including what the assistant must refuse and when it hands over.
• Choose the sources, remove outdated ones, and name an owner for each.
• Decide what the assistant may remember and who can correct it.
• List the tools it needs, starting read-only, and set up its separate identity.
• Mark each action as run, draft or approve.
• Prepare a real question set and define what success looks like.

If you want to understand how this fits a wider plan, What Is Personalized AI Infrastructure for a Company? gives the broader frame. Whatever route you take, the principle is the same: an assistant knows your business to the degree that you have decided, deliberately, what it should know, remember, touch and ask about.

Frequently asked questions

Do we need to train a model on our own data to build a company AI assistant? Usually not. Most business assistants use retrieval, which supplies your maintained documents to the model at question time, so knowledge can be corrected without retraining. Fine-tuning can help with style or specialized formats, but it is a poor way to keep changing facts current.

How many documents should the assistant have access to? As few as the role requires. A smaller set of maintained, owned sources produces better answers than a large unfiltered archive, and it makes access control and updates far easier to manage. You can widen the scope later once the first role is performing well.

Who should own the assistant inside the company? Give it a named business owner who understands the role, plus someone responsible for the technical operation, even if that is an external partner. The business owner decides what good looks like and reviews results, while operations handle monitoring, permissions and changes. Without a named owner, knowledge goes stale and nobody notices.

How do we know the assistant is ready to take on more? Look at evidence from your own question set and from real use: accuracy against sources, how often it correctly declined or escalated, and how much reviewers had to change. When results stay steady across several review cycles, widen its scope one step at a time, such as a new source or a new tool, and test again after each change.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.