The Seven Layers of Company AI Infrastructure, Explained

AI infrastructure layers are the separate responsibilities that together let an AI agent do reliable work inside a company: a model that reasons, a harness that runs the work, memory, tools, permissions, channels and observability. Each layer answers a different question, and most failures happen when one layer is asked to do the job of another. Treat them as seven things you can inspect, own and replace, not as a single product.

Key takeaways

• The seven layers are models, harness, memory, tools, permissions, channels and observability, and each one answers a different question about how the agent works.
• The model reasons, but the harness decides how the work runs, so much of an agent's reliability comes from the layers around the model.
• Memory, tools and permissions should be owned and reviewed by people, because they decide what the agent knows, can touch and may do.
• Channels and observability connect the agent to your team: one is how work arrives and leaves, the other is how you show what happened.
• Diagnose problems by layer. A wrong output usually points to a layer that should have caught it, and the fix rarely requires a new model.

What are the seven layers of company AI infrastructure?

When a company says it uses AI, it often means a chat window connected to a model. That is one layer. A working agent that handles real tasks needs the rest of the stack too, even if some layers are small at the start. Here is each layer in one line:

• Models: the reasoning and language capability that reads input and proposes the next step.
• Harness: the runtime that assembles context, calls the model, runs tools, handles retries and decides when to stop.
• Memory: durable knowledge the agent keeps about the company, people, decisions and ongoing work.
• Tools: defined actions and lookups the agent can perform in other systems.
• Permissions: the rules for what the agent may read, draft or send, and when a person must approve.
• Channels: the places where people reach the agent and receive its work, such as email, team chat or messaging.
• Observability: the record that shows what the agent saw, decided and did.

Why separate them at all? Because separation makes each part testable and replaceable. If a model improves, you can adopt it without rewriting your approval rules. If a system changes its login method, only the tool layer needs attention. And when something goes wrong, you can ask which layer failed instead of arguing about whether the AI is good or bad.

A broader definition of this approach is in the Be Human journal under personalized AI infrastructure. The rest of this article stays with the layers themselves.

Models and harness: who thinks, and who runs the work?

The model is the part most people picture when they hear AI. It reads text, images or structured data and produces a response or a proposed next step. It is capable, but on its own it has no persistent knowledge of your company, no ability to act and no sense of what it is allowed to do. It also changes over time: providers release new versions, and a model that suits one task may be wasteful or weak for another. Treat the model as a replaceable component behind a stable interface, not as the foundation of the whole system.

The harness is the runtime around the model. It decides what goes into each request, which instructions apply, which tools are offered, what happens when a tool fails, when to save progress and when to stop and ask a person. It is where the agent's operating rules live. The article AI model, agent and harness: what is the difference? separates the three terms in more detail.

A useful test: if you swapped the model for another, would the work keep its shape? When the harness is doing its job, the answer is mostly yes. Instructions, stop conditions, checkpoints and approval boundaries live in the harness, so they survive the change, and you re-evaluate quality instead of rebuilding the system. Anthropic's engineering guidance on building effective agents makes a related point: begin with simple, composable patterns and add complexity only when it clearly helps.

What should the memory layer hold?

Memory is what lets an agent continue work instead of starting from zero each morning. It is also the layer most often confused with chat history. A transcript records what was said. Memory records what should still be true tomorrow: a client prefers formal replies, an exception was approved by a named person, a project is waiting on a signed document. The distinction is covered in AI memory vs chat history: what should your agent remember?.

Good memory has four properties:

• Source: each entry points back to where it came from, whether a document, a message or a decision.
• Owner: a person or role is responsible for correcting it.
• Scope: it is visible only to the work and the clients it belongs to.
• Freshness: it can expire or be marked as superseded.

Without these, memory becomes a quiet source of errors. An agent that confidently repeats a policy that has changed is not making things up. It is remembering faithfully something nobody updated.

Memory also includes the agent's own identity and operating context: its role, tone, boundaries and the work it is responsible for. Writing that down once, in a place people can read and correct, is cheaper than rediscovering it through mistakes.

How do tools and permissions work together?

Tools are how the agent touches anything outside the conversation: searching a CRM, reading a calendar, drafting an email, creating a task. Each tool should have a narrow purpose, clear inputs, predictable outputs and an error message the agent can act on. Standards such as the Model Context Protocol help systems expose tools in a consistent way, but a standard does not decide which tools your agent should have. That is a business decision.

Permissions are separate from tools, and the separation matters. A tool describes what is possible. A permission describes what is allowed, for this agent, on this data, at this moment. Permissions should be enforced by the systems the tool connects to, not by a sentence in the instructions asking the model to behave. The OWASP authorization guidance describes the same principle for software in general: deny by default and check on every request.

A practical way to set permissions is to split each capability into three levels:

• Read: look up information. Usually the safest place to start.
• Draft: prepare a message, record change or document for a person to review.
• Send or write: change something outside the agent's own workspace, often hard to undo.

Begin with read, move to draft, and grant send only for specific actions with a clear approval rule. Least privilege for AI agents: start with the smallest access walks through that sequence. Narrow permissions also limit the damage if a document the agent reads contains hidden instructions, a risk that grows with every file type and channel you connect.

Channels and observability: how does work arrive, and how do you know it was done?

Channels are where the agent meets people. They might be email, a team chat, a messaging app or an internal dashboard. This layer handles more than delivery. It answers who is allowed to instruct the agent, how a sender is identified, what format suits each audience, and how an informal request becomes a task with an owner. A channel that treats every message as a command is a security problem. A channel that loses context between messages is a usability problem.

Observability is the record of what happened. At minimum it should let someone answer, after the fact: what request arrived, what context the agent used, which tools it called, what it proposed, who approved it and what the outcome was. The OpenTelemetry primer describes observability as understanding a system's internal state from the signals it produces, such as logs, metrics and traces. The same idea applies to agent work, adapted to tasks and decisions. AI agent observability: what should you actually log? offers a practical list.

Observability is also what turns the other six layers into something you can improve. Without it, a disappointing result is an argument. With it, the result is a diagnosis.

Illustrative scenario: which layer failed?

This scenario is illustrative. A services company has an agent that drafts follow-up emails to clients after calls. One draft quotes a delivery term that was changed the previous month, and the client notices.

A team that blames the model might switch to another one and hope. A team that reads the stack by layer asks a different question at each step:

• Model: did it follow the context it was given? Yes, it quoted what it saw.
• Harness: did anything check whether the context was current? No step compared the term with its source.
• Memory: the old term sat in a client note with no owner and no expiry.
• Tools: the agent had a lookup for the current agreement, but nothing required it before quoting a term.
• Permissions: the draft could be sent without review.
• Channels: the request arrived in a message thread that did not mention the changed term.
• Observability: the log showed which note supplied the term, so the diagnosis took minutes instead of days.

The repair is spread across layers. Mark the note as superseded, require the agreement lookup before any quoted term, and hold client-facing drafts for approval until the check has proven reliable. None of those changes needed a new model.

How do you review your own stack, layer by layer?

Use these questions in a working session with the people who own the process. Any layer you cannot answer for is a layer nobody currently owns.

• Models: which model handles each task, and what evidence says it is good enough for that task?
• Harness: where are the instructions, stop conditions, retry rules and checkpoints written down?
• Memory: for any remembered fact, can you see the source, the owner and when it was last confirmed?
• Tools: does each tool have a narrow purpose, documented inputs and outputs, and a defined behavior when it fails?
• Permissions: is each capability set to read, draft or send, and is that enforced outside the instructions?
• Channels: who may instruct the agent, and how is each person identified?
• Observability: could you reconstruct yesterday's most important task from the record alone?

Once you have answers, pick the weakest layer and strengthen it before adding anything new. A modest agent with all seven layers understood is easier to trust than an ambitious one with three of them missing.

Be Human designs, implements and operates these layers as one system for a company. An engagement starts by defining one useful role, the memory and tools that role needs, the approval boundaries and an observable success criterion. Managed operations can then include monitoring, recurring evaluations, integration maintenance, permission review, incident response and controlled improvements, while the company's people keep control over what the agent may do.

Frequently asked questions

Do all seven layers need to exist from the start? Yes, in a small form. Even a simple agent has a model, instructions, some context, a tool or two, a limit on what it can do, a way to receive requests and some record of its actions. What you can defer is sophistication, such as broader memory or more channels, not the layer itself.

Which layer matters most? No single layer carries the system, but permissions and observability deserve early attention because they limit and reveal mistakes. A weaker model with clear boundaries and a good record is easier to manage than a stronger one nobody can inspect.

Can we assemble these layers from different vendors? You can, and many companies do. The risk sits in the joins: someone must own how memory feeds the harness, how tools respect permissions and how logs from each part connect into one record. If you are weighing that work, Build or buy an AI harness? Compare the work you will own is a useful next read.

How is this different from a chatbot? A chatbot is mostly a model plus a channel. It answers questions, but it usually does not keep governed memory, use tools under permissions or leave a record you can audit. The seven layers describe what is added when the goal is completed work instead of replies.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.