Skip to content
be-human.ai
← Back to the journal

Personalized AI infrastructure

Which Layer of a Company Superintelligence Stack to Fund First

Ranks the six layers of a company superintelligence stack by what compounds, from permissions and memory to models, so leaders fund the right layer first.

Be Human typographic cover with the title: Which Layer of a Company Superintelligence Stack to Fund First
Conceptual illustration created with AI · Be Human

A company superintelligence stack has six working layers: models, harness, memory, tools, permissions and observability. Fund first the layers that gain value as the company uses them, which means permissions, memory and observability, and treat the model as a replaceable input chosen for fit and kept reversible.

Key takeaways

  • Fund permissions and approval boundaries before you widen any tool access.
  • Treat governed memory as the asset that compounds, because it improves with every correct record and decision.
  • Build observability early so that every agent action leaves a record someone can review.
  • Keep the model choice reversible and the harness simple until one role meets its success criterion.
  • Start with one role, one memory scope and a short list of tools.

Why the order matters more than the components

Most leadership teams first meet the stack as a list of parts, and a list invites buying everything at once. A better question is which part still produces value after the next model release, the next vendor change and the next staff turnover. Models change quickly, so a model bought today is a rented capability. Memory, permissions and evidence belong to the company, and they make every later improvement cheaper to test and safer to ship.

The order also matters because each layer depends on the ones beneath it. A harness that orchestrates tools is only useful when the tools are narrow and the permissions are explicit. An agent with broad access and no logs cannot be improved, only replaced. Sequencing the spend means each layer has something solid to stand on, and it keeps the budget tied to evidence rather than to the newest announcement.

For a broader view of how these parts fit together, Draw Your AI Stack: A One-Page Architecture Map for Leaders is a useful companion before you assign budget to any single layer.

Ranking the six layers by what compounds

  • Permissions and approval boundaries: define what the agent may read, write, send and approve, and where a human must confirm. This layer limits damage from the first day and is reviewed over time rather than rebuilt.
  • Memory: the records, decisions and operating context the agent relies on. Its value grows as it is corrected and maintained, and it is the layer most worth keeping if you change vendors.
  • Observability: traces, logs, quality checks and review queues that show what the agent did and whether it succeeded. Without it, no improvement can be measured.
  • Tools: the connections to email, calendars, a CRM, spreadsheets or internal systems. Each tool adds capability and also adds a permission to govern.
  • Harness: the code and workflow that run the agent loop, call tools, handle failures and pass context between steps. It should start simple and grow only where a real task demands it.
  • Models: the language models that reason and write. Choose one that fits the role, keep the integration thin and plan for the day you replace it.

Ranking this way does not mean ignoring the model. It means the model is the cheapest layer to replace and the least useful one to fund far in advance of a working role.

Memory and records: the layer that gets more useful with use

Memory holds most of the lasting value, and it is where companies make the most avoidable mistakes. An agent that answers from scattered inboxes, outdated spreadsheets and private notes will sound confident and still be wrong. The fix is not a larger model. It is a defined memory scope: which documents are authoritative, who owns each record, how a correction is made and how old information is retired.

Retrieval is the usual mechanism. The Google Cloud architecture guidance on retrieval-augmented generation describes grounding model answers in documents retrieved at the moment of the request, which keeps company knowledge in company systems rather than inside a model. That separation is what makes memory portable. When the records are clean and owned by the company, changing the model later does not mean starting over.

The practical starting point is a short list of the records the first role truly needs, with one named owner for each and a note on what is out of bounds. Old drafts, personal files and unreviewed chat logs usually belong outside the scope until someone decides otherwise. For the data work that comes before this, Data Foundations for Superintelligence: Clean Records Before Agents covers what to clean first.

Permissions and tools: the layer that limits damage

Tools turn a model into a worker, so each tool needs a permission decision before it is connected. Start from the action, not the system. Reading a customer's order status and issuing a refund are different permissions even when they touch the same platform. Use a deny-by-default posture, grant the smallest scope that completes the task and keep high-impact actions behind an explicit human approval.

Authorization should be enforced by the systems the agent calls, not only by instructions in a prompt. The OWASP authorization guidance is a useful reference for checking that each request is evaluated against a policy instead of being trusted because it came from the agent. Prompt injection matters here as well. Content the agent reads, such as an email or a web page, can contain text that tries to change its behavior, so defenses belong in the permission design and not only in the wording of the prompt.

Write down each tool with its purpose, its read and write scope, its approver and its next review date. A regular permission review keeps that list accurate as the role changes, and How to Run a Permission Review for Your AI Agents, Step by Step gives a routine for doing it.

Observability: the evidence layer

Observability is the layer companies skip most often and regret soonest. Without it, you can see that the agent produced an answer but not which documents it used, which tool it called, where it stalled or whether a reviewer corrected it. The OpenTelemetry concepts primer describes three basic signals, traces, metrics and logs, which give a shared vocabulary for this work. A trace follows one task from start to finish, metrics show patterns across many tasks and logs hold the detail that explains a single event.

For an agent, the minimum useful record is simple: the request, the context used, each tool call and its result, the output, any human decision and the outcome against the success criterion. Recurring evaluations use these records to test whether the agent still meets its standard after a change to the model, the prompt, the memory or a tool. Observability is therefore what makes every later decision about spending a decision and not a guess.

Checklist for the observability layer before the role goes live:

  • Every task has an identifier that links the request, its tool calls and its output.
  • Each tool call records what was sent, what came back and whether it succeeded.
  • Human approvals and corrections are stored next to the output they affect.
  • A short set of test tasks with known correct outcomes runs after every change.
  • A named person receives failure alerts and reviews them within a set time.

A decision guide for the first funding round

Consider this illustrative scenario. A services company with a small team wants an agent to draft quotes from its price list and past proposals. Leadership is tempted to fund a stronger model first and connect the agent to the CRM, the accounting system and email at the same time. A better sequence is to write the approval rule for sending quotes, define the archive of past proposals as the memory scope with a named owner, connect read-only access to that archive and log every draft with the sources it used. The model stays a default choice that can be replaced later. Only after several reviewed drafts meet the success criterion does the team widen tool access to the CRM.

Use these questions to make the same decision for your own first role:

  • Which single role will the agent own, and what does success look like in one observable measure?
  • Which records does that role need, who owns each one and which ones are out of bounds?
  • Which actions require a human approval before they take effect?
  • Which tools are needed on day one, and which can wait until the role has proven itself?
  • Can someone review every output and every failure within a set time?
  • Could the model be replaced without losing the memory, permissions or logs?

If the answer to the last question is no, fund the layers beneath the model before expanding anything else.

How Be Human approaches the sequence

Be Human designs, implements and operates personalized AI infrastructure for companies, with agents that keep their identity, memory, channels, tools, permissions and operating context under human control. A first engagement defines one useful role, the memory and tools it needs, its approval boundaries and an observable success criterion, which puts permissions, memory and observability in place before the scope grows. Managed operations can then include monitoring, recurring evaluations, integration maintenance, permission review, incident response and controlled improvements. The model is handled as an input that can be changed under the same governance, so the company keeps the layers that compound. The early rhythm of that work is described in What the First Months With a Managed AI Agent Look Like.

Deciding what to fund this quarter

Fund the stack in this order and stop at each step until its evidence is in hand: permissions and the approval rule for the first role, a defined memory scope with owners, observability that records every task, the minimum tools the role needs, the simplest harness that completes the work and finally the model, chosen for fit and kept replaceable. A company that follows this order spends its first budget on assets it will still own after the next wave of model releases. If you are not sure which of these layers your company is ready to fund, take the free Be Human AI diagnostic to find the first task worth handing to an agent.

Frequently asked questions

Should we choose a model before we design memory and permissions? Usually not. The model is the layer most likely to change, and a choice made before the memory and permissions exist tends to lock in assumptions about data and access that you later have to undo. Pick a model that fits the first role, keep the integration thin and design the layers that hold company value first.

Do we need the full six-layer stack before the first agent goes live? No. A first role needs a permission rule, a small memory scope with an owner, basic logging and a few test tasks. The harness, tools and model can stay minimal until that role has met its success criterion, and the rest of the stack can grow from there.

How do we know whether a layer is worth funding now? Ask whether the layer will still be useful if you replace the model, the vendor or a team member. Memory, permissions and observability usually pass that test, while a larger model or a broader tool set often does not until a specific task proves the need.

What if we already bought a platform that bundles several layers? Map what the platform provides against the six layers and ask who owns the memory, the logs and the permission records if you leave. Bundled features can be useful, but the records and evidence should remain exportable and under your control, so that the platform is a tool you can change rather than the place where your company knowledge lives.

Your next step

See where an AI agent fits in your company first.

Take the free Be Human AI diagnostic. A few questions about your work show which task to hand to an agent first, what it needs to know and where a person approves.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.

Less busywork. More room for you.

Bring us the work you wish you had time for.

Book a call to try a personalized AI agent free for 7 days ↗Or take the free AI diagnostic →