Skip to content
be-human.ai
← Back to the journal

Personalized AI infrastructure

Superintelligence Pilot Scope: What to Leave Out of the First Build

Your first AI agent build should leave out most features, integrations and roles. Here is what to exclude, why, and how to keep the pilot observable.

Be Human typographic cover with the title: Superintelligence Pilot Scope: What to Leave Out of the First Build
Conceptual illustration created with AI · Be Human

A first AI agent build should cover one role, one channel, one approved data source and one success criterion that a named person can check. Everything else, including extra features, integrations and permissions, belongs on a written exclusion list until the pilot produces logs you trust. A narrow pilot can be observed and corrected in a short cycle, and the memory, tools and permission patterns it creates can carry forward to later roles.

Key takeaways

  • Scope the first build to one role with one success criterion that someone can verify from logs or records.
  • Leave out autonomous outbound actions, extra channels and broad memory until the logs show stable behavior.
  • Postpone any integration that writes to systems of record, handles payments or touches personal data beyond what the role needs.
  • Keep administrative rights and cross-team permissions out of the pilot, and use scoped, expiring credentials.
  • Maintain an exclusion list where every deferred item has a reason and a condition for reconsidering it.

Why a smaller first build compounds better

Most leaders picture the agent they want in a year. The first build has a different job. Its purpose is to produce a clean record of what the agent did, what it was allowed to do, and where a person had to step in. When the scope is wide, that record becomes noisy, and no one can tell whether a poor result came from the instructions, the memory, a tool or a permission that was too loose.

Narrowness is what makes the gains reusable. Memory structures, tool definitions, approval steps and review routines are layers that can serve a second role later. If the first build covers sales, support and operations at once, those layers are shaped around too many unknowns and usually have to be rebuilt. A pilot that does one job well leaves behind components that another team can adopt without starting over.

Features to leave out on day one

The features requested most often early on are the ones that look impressive in a demonstration: open conversation on any topic, automatic summaries of everything, self-adjusting instructions and autonomous follow-ups to customers. Each one adds a surface that must be tested, monitored and explained to the people who depend on the output.

  • Open-ended conversation outside the role's task. Define the requests the agent handles and route everything else to a person.
  • Autonomous outbound messages. Drafts for human review come first, and sending comes later.
  • Memory writes or instruction changes without review. Proposed changes should be approved before they take effect.
  • Multi-step workflows that chain several tools in one run. Start with single actions that can be logged individually.
  • Dashboards and reporting views beyond the success criterion. Measure the one thing the pilot must prove.
  • Additional languages or voice interfaces unless the pilot's actual users need them on the first day.

Anthropic's engineering guidance on building tools for agents favors a small set of clearly described tools over a large, overlapping one. The same logic applies to features. Each feature you leave out is one less behavior to evaluate before you know what good looks like.

Integrations and data sources to postpone

Every integration is a credential, a data mapping, a permission and a failure mode. Retrieval, the approach that grounds an agent's answers in company documents, is well documented in the Google Cloud and AWS architecture guides, but indexing every shared drive, inbox and CRM at once makes it hard to know which source produced an answer. Choose one curated document set with a named owner and a known update routine.

  • Write access to the CRM, ERP or accounting system.
  • Email or chat channels beyond the one the pilot users already work in.
  • Connectors to third-party tools that have separate administrative consoles and permission models.
  • Historical data imports larger than the pilot's question requires.
  • Any source containing customer personal data until the data handling review is complete.
  • Reading external web pages or inbound messages from unknown senders, until you have tested how the agent treats instructions embedded in that content. The OWASP guidance on prompt injection prevention describes why untrusted content can attempt to redirect an agent.

Read-only access to one source of truth is usually enough to prove the first task. Write access can follow once the logs show the agent reads the right records and the approval step works in practice.

Roles and permissions to hold back

Leaders often want one agent to serve several departments from the start. Resist that. A first build should serve one role, one set of users, one business owner and one named approver. Each added role multiplies permissions and creates instructions that may conflict with each other.

The OWASP authorization guidance stresses that access should be checked at every request, not assumed from the interface a person happens to use. For a business agent, the practical version is that its tool permissions should be narrower than those of the human who owns the task, and every approval should be explicit and recorded.

  • Administrative or configuration rights for the agent itself. Changes go through a documented change process.
  • Approval authority over money, pricing, refunds or contract terms in the first build.
  • Actions that affect staff schedules, hiring decisions or performance records.
  • Memory shared with departments that are not part of the pilot.
  • Permanent credentials. Use scoped access that expires and can be revoked without redeploying the agent.

A scenario and a decision guide for scope requests

Illustrative scenario: a regional services company wants its support lead to have an agent draft replies to customer emails. Within the first weeks, the team asks to add refund approvals, a messaging channel and direct access to the billing system. The pilot owner applies a short decision guide. Drafting replies stays in scope. Refund approvals are deferred until the drafting logs show consistent accuracy. The messaging channel waits until the email pilot has a stable error record. Billing access is removed from the first build and replaced by a manual lookup step that the support lead performs.

Use these questions for every request that arrives after the scope is set:

  • Does it serve the single success criterion? If not, defer it.
  • Can every action it takes be logged and reviewed by one named person? If not, defer it.
  • Does it expand permissions, channels or data sources? If yes, it needs a separate approval and a change record.
  • Can a mistake be reversed without harm to a customer or to the company's finances? If not, it stays out until approval thresholds have been tested. The guide on raising approval thresholds slowly explains how to do that.

How to write the pilot scope, step by step

A scope is only useful if it is written down in a form that a person can check. The steps below produce a document short enough to read in one sitting and specific enough to settle disagreements later.

  • Name the role and the single person who owns its outcome.
  • Write one success criterion that can be verified from logs or records within a scheduled review cycle.
  • List the three to five tasks the agent will perform, and put everything else on the exclusion list.
  • Name the one memory source, the tools allowed, and the approval step for each action the agent can take.
  • Record each exclusion with a reason and the condition under which it will be reconsidered.
  • Set the review date, the evidence you will examine, and the rule for pausing the pilot if results degrade. The guide on stopping an AI pilot without wasting the work covers what to keep.
  • Attach the scope to the statement of work so that the boundaries are part of the agreement, not a side conversation.

Be Human's managed approach starts from this kind of written scope. A first engagement defines one useful role, the memory and tools it needs, its approval boundaries and an observable success criterion. Monitoring, recurring evaluations and permission reviews then show whether the pilot is ready for the next request, and each change goes through a documented process rather than being added informally. The evaluation approach follows the logic in Anthropic's guidance on evals for agents: define representative tasks, run them repeatedly, and read the results before changing anything.

If you are not sure which task should be handed to an agent first, the free Be Human AI diagnostic can help you find the first task worth handing to an agent.

Frequently asked questions

How many features should a first AI agent pilot include? Only the features needed to meet its single success criterion. For most companies that means a drafting, lookup or triage task with one channel and one approval step. Anything else goes on the exclusion list with a reason and a condition for review.

Should the pilot use company memory from the start? It should use a curated, named source of truth, read-only where possible. Broad memory across every drive, inbox and system makes errors hard to trace back to their cause. Memory writes should be proposed and approved, not saved automatically.

When should integrations be added to the agent? Add them after the logs show stable behavior and the first permission review is complete. Start with read access, and grant write access only for actions that still pass through a human approval step. Each integration needs its own credential, a failure plan and an owner.

What if stakeholders insist on the full scope from the beginning? Show them the exclusion list and the decision guide, and offer a scheduled review where deferred items are reconsidered against evidence. A narrow pilot that works is the strongest argument for expansion, because each new permission is then justified by results rather than by enthusiasm.

Your next step

See where an AI agent fits in your company first.

Take the free Be Human AI diagnostic. A few questions about your work show which task to hand to an agent first, what it needs to know and where a person approves.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.

Less busywork. More room for you.

Bring us the work you wish you had time for.

Book a call to try a personalized AI agent free for 7 days ↗Or take the free AI diagnostic →