Build a Company Superintelligence Layer That Improves With Models

A company superintelligence layer is the shared memory, tools, permissions, channels and measurement that surround an AI model so your whole business can use it under human control. You build it by keeping the model replaceable and keeping everything your company owns outside the model. When a better model arrives, you test it against your own work and swap it in, and the whole layer improves without a rebuild.

Key takeaways

• Treat the model as a replaceable part. Memory, tools, permissions, rules and test cases belong to the company and live outside it.
• Keep the harness (the loop that gives the model a task, context and tools) small and explicit, and remove scaffolding that only existed to compensate for an older model's weaknesses.
• Enforce permissions and approval boundaries in your systems, not in prompts, so a more capable model never becomes a more powerful actor by accident.
• Build a test set from real work before you need it. Without one, every upgrade is a debate of impressions.
• Roll out a new model by role and channel, with a clear way back, and record what changed.

What does company superintelligence mean in practice?

The phrase sounds grand, so define it narrowly. Here it does not claim that a machine exceeds people at everything. It means the combined knowledge, procedures and operating capacity of your company, made available to every role through agents that act within limits and get more capable as models improve.

The useful picture is an engine and a vehicle. The model is the engine: it reads, reasons, writes and decides what to try next. The vehicle is everything else: what the agent knows about your company, which systems it can touch, who approved what, where people reach it and how you know it worked. Engines get replaced often. A vehicle shaped around one engine's quirks must be rebuilt each time. A vehicle with a standard mount takes the new engine and simply goes further.

For the full inventory of parts, see the seven layers of company AI infrastructure. This article focuses on how to join them so an upgrade flows through.

Why do model upgrades force rebuilds in so many setups?

Rebuilds are rarely caused by the model itself. They come from early decisions that quietly tied company assets to one model. The usual suspects:

• Prompts grew into long instructions tuned to one model's habits, so a new model reads them differently and behaves differently.
• Memory sits in a provider-specific format or in chat history, so it cannot move.
• Tool definitions are buried inside prompts instead of described once in a shared, documented interface.
• Permissions ride on one person's login, so the agent holds whatever access that person happened to have.
• Nobody wrote down what good output looks like, so no one can say whether the new model is better or only different.

The last one does the most damage. Without a way to measure, companies either freeze on an old model or switch and hope. Both waste the gain a better model offers.

How should each layer be structured so it outlasts the model?

Think of seven layers with clean joints between them. Each joint is a place where one part can change without touching the others.

• Models: a configured choice, not architecture. Keep one routing point where you name which model serves which task, so a stronger model handles hard work and a lighter one handles routine work. When more than one model pays off covers that decision.
• Harness: the loop around the model. It receives a task, assembles context, calls tools, checks its own work and knows when to stop. Anthropic's engineering notes on building effective agents and on harnesses for long-running agents both favor simple, composable patterns. Review the harness at every upgrade and delete steps the new model no longer needs.
• Memory: company knowledge kept as plain, owned, readable records such as policies, glossaries, customer notes and past decisions, with retrieval you control. Separate stable knowledge, working notes and per-task context. Retrieved knowledge updates without retraining and moves between models, which fine-tuned behavior does not.
• Tools: narrow, well-described actions on your systems, defined once. The Model Context Protocol is one open way to describe tools so different models and clients can reuse the same definitions. Write clear names, clear descriptions and short, useful error messages.
• Permissions: enforced by the systems the agent touches, not by instructions to the model. Give each agent role its own identity, least-privilege access and approval gates for actions that spend money, contact outsiders or change records. OWASP's authorization guidance applies to agents as much as to people.
• Channels: where people reach the agent, such as a shared inbox, a chat space or a ticket queue. Keep the channel adapter separate from the reasoning so a new model does not change how people work with the agent, and so you can enable it in one channel first.
• Observability: a consistent record of each task, including the request, context used, tool calls, approvals, outcome and cost. Use a vendor-neutral format so the record survives a model change and lets you compare before and after.

The pattern is the same in each case. Anything the company must keep is stored in a form the company owns, and anything the model supplies is reached through a narrow, documented joint.

What is the step-by-step plan to build the layer?

Start small and complete rather than wide and thin. One role, built properly, teaches you the joints.

• Step one: choose one role with a visible outcome, such as an operations coordinator who reconciles orders or a support lead who drafts replies. Use cases by department can help you narrow it, but pick only one.
• Step two: write the role down in plain language, including what it may do alone, what needs approval and what it must never do.
• Step three: move the knowledge the role needs into owned, readable documents, and name an owner for each.
• Step four: describe each tool once, with its inputs, outputs and failure messages, and connect only what the role needs.
• Step five: give the agent its own identity and scoped access, with approvals on consequential actions.
• Step six: build a test set of real tasks with expected outcomes. Include hard cases, ambiguous cases and cases where the right answer is to refuse or escalate. Anthropic's guidance on evaluating agents is a good reference: grade outcomes and actions, not only wording.
• Step seven: turn on tracing and cost per task from the first day, so you have a baseline.
• Step eight: release to a small group, review the traces weekly, and widen only when the evidence supports it.

Once the first role works, the second is cheaper because memory, tools and observability are already shared.

What does an upgrade look like when the layer is built this way?

Here is an illustrative scenario, not a real case. A regional distributor runs an operations agent that reconciles purchase orders against supplier confirmations and flags mismatches for a person to approve.

A new model becomes available. The team does not announce it to staff. They run it against the stored test set in a staging copy of the layer. It handles messy supplier emails better, but it also tries to call the ordering tool more eagerly than the old model did. The permission layer blocks the extra calls because ordering still requires approval, and the traces show exactly where the behavior differed. One instruction in the harness is adjusted, and a workaround step added for the old model is removed.

The new model then serves one channel, the shared purchasing inbox, for a week. Traces and approval rates are compared with the baseline. It is promoted for that role, and the previous model stays configured as the fallback. Memory, tools and permissions never changed. For the mechanics of the release step, see how to update an AI agent in production without breaking it.

What should you check before promoting a new model?

Use this as a decision guide. Promote only when every line has an answer you can show.

• Does it meet or beat the baseline on the test set, including the cases that must be refused or escalated?
• Did any tool call, approval request or external message appear that the old model did not produce, and was each one expected?
• Do permissions still block everything they should, checked by attempting the blocked actions in staging?
• Is cost per completed task acceptable, and is response time acceptable for the people waiting?
• Can you return to the previous model quickly, and has that been tried once?
• Were harness instructions simplified where the new model allows it, rather than only added to?
• Is the change recorded, with who approved it and what evidence they saw?

Permissions deserve their own rhythm beyond upgrades. A scheduled review, such as the one in the quarterly AI agent permission review, catches access that grew quietly as roles and tools changed.

How does Be Human's managed approach handle model upgrades?

Be Human designs, implements and operates this structure for companies. A first engagement defines one useful role, the memory and tools it needs, its approval boundaries and an observable success criterion. In managed operations, when a new model appears, Be Human runs it against the role's evaluation set, reviews permissions and traces, proposes any harness changes for the owner's approval, and releases in stages with a fallback in place. The company keeps its memory, tool definitions and records in a form it can read and move.

Frequently asked questions

Do we need the newest model for this to work? No. The layer is valuable with whichever model passes your tests today. Its purpose is to make adopting a better model a routine, low-risk decision rather than a project, so you can move when the evidence supports it and wait when it does not.

Why not just fine-tune a model on our company data? Fine-tuning ties company knowledge to one model and has to be repeated when the model changes. Retrieval from owned records keeps knowledge current and portable. Fine-tuning can still help for narrow behavior or style, but treat it as an optional addition after memory and testing are in place.

How often should we review the harness? Review it at every model change and whenever traces show a repeated failure. Many instructions and checks exist to compensate for a specific weakness. When the weakness is gone, the instruction becomes noise, so removing it is part of the upgrade.

Who should own the layer inside the company? Assign a business owner for each role, who decides what the agent should do and approves boundaries, and a technical owner for tools, access and observability. A named sponsor above them keeps decisions moving. If you operate with a partner, make sure the contract states who owns the memory, definitions and records.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.