Skip to content
be-human.ai
← Back to the journal

Personalized AI infrastructure

Enterprise Superintelligence Platforms: Five Criteria That Hold Up

Five checkable criteria for judging an enterprise superintelligence platform: memory ownership, permissions, observability, exit terms and human approval.

Be Human typographic cover with the title: Enterprise Superintelligence Platforms: Five Criteria That Hold Up
Conceptual illustration created with AI · Be Human

An enterprise superintelligence platform is a managed layer that gives a company's AI agents a stable identity, memory, tools, permissions and operating context, all kept under human control. A buyer can judge any vendor making that claim on five checkable criteria: who owns the memory, how permissions are scoped and reviewed, whether every action is observable, what the exit terms return, and where a person must approve before an action takes effect.

Key takeaways

  • Memory and operating records should belong to the company, and the contract should say so in plain terms before any pilot starts.
  • Permissions should be scoped by role, tool and data set, with read and write access separated and reviewed on a fixed calendar.
  • Every agent action should leave a record that a person can search, read and audit without asking the vendor for help.
  • Exit terms should list what you receive on termination, such as memory, configuration, logs and integration maps, and the format each one arrives in.
  • Human approval should attach to named action types with clear thresholds, not to a general promise that someone is watching.

What a vendor's claim does and does not tell you

The phrase enterprise superintelligence platform is broad enough to describe a chat interface, a workflow tool, a consulting engagement or a bundle of model access. Its presence on a website says little about how the system will behave once it touches your customer records, your invoices or your staff calendar. Treat the claim as a hypothesis and ask for evidence against the five criteria below.

A live demonstration on the vendor's sample data is useful for seeing the interface. It does not show who holds the memory six months from now, who changed a permission last month, or how you would leave if the relationship ended.

1. Who owns the memory the agent builds?

Memory is what an agent keeps between tasks: customer history, approved procedures, decisions made and exceptions granted. In many systems it is split across a retrieval index, stored notes, conversation summaries and prompt configuration. Retrieval designs, which fetch relevant company documents at the moment of a question, are described in public references such as the AWS guidance on retrieval-augmented generation, and they make the location of company knowledge a concrete question rather than an abstraction.

Ask the vendor to answer these in writing:

  • Where is each memory store hosted, and in which region?
  • Can an administrator export every memory item in a documented, machine-readable format?
  • Can a specific record be deleted, and does the deletion reach retrieval indexes and backups?
  • Is company memory ever used to tune models shared with other customers?
  • Who at the vendor can read it, and is that access logged?

Illustrative scenario: a distribution company asks its vendor to remove a former supplier's contract history from the agent's memory after a dispute closes. The vendor deletes the primary store, but the retrieval index keeps the old passages for several weeks. A buyer who asked about index propagation during evaluation would have learned that before the dispute arrived.

In Be Human's managed approach, memory sources are defined during the first engagement and written into a record the company can read. Scope, retention and deletion steps are documented and tested before the agent handles live records, so ownership is a stated fact rather than a sentence in a sales deck.

2. How are permissions scoped, granted and reviewed?

An agent holding one broad credential can do everything that credential allows, and that is one of the most common ways a useful assistant becomes a liability. Permissions should be granted per tool and per action, with read access separated from write access. The OWASP authorization guidance states the general principle: access decisions should follow explicit rules that the system enforces, not the model's judgment about what seems appropriate. Delegated access to third-party systems usually runs through an authorization standard such as OAuth 2.0, where each grant is limited to named scopes that an administrator can see and revoke.

Checklist for the vendor conversation:

  • Each tool connection has its own named scope, and the scope list is visible to administrators.
  • Write actions can be disabled without disabling read actions.
  • Permission changes are logged with who requested them and who approved them.
  • Reviews happen on a schedule you set, and each review produces a written record.

A calendar-based process for this is laid out in the quarterly permission review steps guide.

In Be Human's managed approach, permissions are set as part of the first engagement, starting with one useful role, the tools that role needs and the actions that require approval. Later changes go through a permission review on a schedule the company agrees, and each review leaves a written record.

3. Can you see what the agent did, and why?

Observability means a person can reconstruct a sequence of events without relying on the agent's own summary of them. For an agent, that means recording inputs, tool calls, retrieved documents, outputs and the human decisions that followed. The OpenTelemetry concepts primer describes traces, metrics and logs as complementary signals. The same logic applies here: a final answer tells you little if you cannot see which documents were retrieved or which tool was called before it.

Questions to put to the vendor:

  • Can an operator open a single agent run and see every tool call, with its arguments and results?
  • How long are logs retained, and can you export them?
  • Can a fixed set of test cases run after every change, with results compared to the previous version?
  • Are sensitive fields masked in logs, and who can unmask them?

Recurring evaluations turn observability into a measurable signal. A fixed set of test cases, scored at each release, shows whether a change improved the agent or quietly broke a procedure it used to follow.

In Be Human's managed approach, monitoring and recurring evaluations are part of operations. The company sees run records and evaluation results, and changes are checked against the same test set before they reach live work.

Illustrative scenario: a small professional services firm notices that its agent's client reminders have become less specific. The run records show that a retrieval step started returning an outdated template. Without per-step logs, the team would have blamed the wording and rewritten the prompt instead.

4. What do you receive when the contract ends?

Exit terms are where lock-in becomes concrete. Before signing, a buyer should know what leaves the vendor's systems on termination and in what form. A list of usable artifacts is more useful than a general promise of portability: the memory export, the configuration of each agent (role, instructions, tool connections and approval rules), the logs, the evaluation sets and the integration map showing which systems the agent touched. Without the configuration and evaluation sets, a replacement starts from zero even if the raw memory arrives intact.

Checklist for the contract:

  • The termination notice period, and the date by which the export is delivered.
  • The export formats for memory, configuration, logs and evaluations.
  • Whether the vendor deletes its copies, and how it confirms that deletion.
  • Whether exit assistance is included or billed separately, stated in the contract.
  • Whether the underlying tool accounts remain in your name, so you can reconnect them elsewhere.

The steps to avoid AI vendor lock-in with business agents cover the negotiation side in more detail.

In Be Human's managed approach, each agent is described as a set of documented parts: memory sources, role, permissions, approval rules and evaluation cases. The aim is that those parts can be listed in the engagement record and handed over in a defined form. A company should ask for a sample export during evaluation, not at the end of the relationship.

5. Where must a person approve before an action happens?

The useful question is not whether humans are involved in general, but which actions need approval before they run and which can run and be reviewed afterward. Sending an external message, issuing a refund, changing a permission, deleting records and committing spend are usual candidates for prior approval. Drafts and routine lookups can be sampled in batches. The NIST AI Risk Management Framework is a useful reference for this mapping, since it asks organizations to set oversight according to the consequences of errors, not according to what the technology is capable of.

A practical sequence for setting the thresholds:

  • List the most common actions the agent would take in its first month.
  • For each action, write the worst realistic outcome of a wrong execution.
  • Assign one of three levels: runs freely with a sampled review, requires approval before execution, or is not permitted to the agent.
  • Name the person who approves each class, and the time window in which they must respond.
  • Write the thresholds into the permission set, so the system enforces them and the policy document is not the only record.

The guide to human approval thresholds for stronger agents goes deeper on setting these levels as capability grows.

Illustrative scenario: a small accounting office lets its agent draft client reminders freely but requires a partner to approve any reminder that mentions a payment amount. The rule is one sentence long, and it is enforced in the tool permissions rather than left to the agent's judgment.

In Be Human's managed approach, approval rules are part of the first engagement. The approval step is enforced in the agent's tool permissions, and the thresholds are reviewed whenever the agent's scope changes.

6. How do you run the five-criteria check in one working session?

A buyer can turn the five criteria into a short, decisive evaluation:

  • Send the criteria to the vendor as written questions, and ask for answers in writing before the meeting.
  • Ask for one live run from a sample task, opened in its run record, showing tool calls and approval steps.
  • Request a sample memory export and a sample permission review record, both with personal data removed.
  • Score each criterion as clear, partial or unclear, and require a written answer for every unclear item before anything is signed.
  • Agree on the first task the agent would handle and its approval rule, so the evaluation ends with a concrete scope rather than a general impression.

Keep the decision with the business owner. A platform that cannot answer these questions in writing is not ready for live records, however capable its demonstration looks. If you are not yet sure which task deserves an agent first, the free Be Human AI diagnostic can help you find the first task worth handing to an agent.

Frequently asked questions

Is an enterprise superintelligence platform the same thing as an AI agent? An AI agent is one component, a system that takes actions toward a goal with the tools it has been given. A platform in this sense supplies the surrounding layer: memory, permissions, records, approval rules and operating processes that let one or more agents work inside a company's control. A company can run a single agent without a platform, but the five criteria become harder to verify without that surrounding layer.

Which of the five criteria should a buyer check first? Memory ownership and exit terms are usually the criteria that carry the highest long-term cost if they are unclear, because they determine what you can take with you and what remains in the vendor's systems. Start with them in the first conversation, since they are the hardest to fix after data has already moved. Permissions and approval rules then define what the first live task is allowed to do.

How long should a vendor evaluation take? The evaluation should be scoped to one task, one set of written answers and one sample run. A vendor that needs a long, open-ended trial before it will answer the five questions in writing has already given you an answer about how it handles transparency. A short, structured evaluation is usually enough to decide whether to proceed to a limited pilot.

Do we need an internal AI team before we can judge a vendor? Not necessarily, but someone inside the company must be able to read the run records, understand the permission scopes and own the approval rules. That person does not need to build the system. They need enough operational knowledge to notice when something is wrong and to ask for the evidence described above before live records are involved.

Your next step

See where an AI agent fits in your company first.

Take the free Be Human AI diagnostic. A few questions about your work show which task to hand to an agent first, what it needs to know and where a person approves.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.

Less busywork. More room for you.

Bring us the work you wish you had time for.

Book a call to try a personalized AI agent free for 7 days ↗Or take the free AI diagnostic →