Skip to content
be-human.ai
← Back to the journal

Business workflows

Designing a Superintelligent Agent as a Product: Users and Limits

Define an AI agent like a product: its users, one repeatable job, action boundaries and an observable success measure, so a stronger model has a clear job.

Be Human typographic cover with the title: Designing a Superintelligent Agent as a Product: Users and Limits
Conceptual illustration created with AI · Be Human

Designing an agent as a product means defining who it serves, the one job it performs, the limits it must respect and a measure that shows whether the job got done. A more capable model does not make that definition optional. It widens what the agent can attempt, so the written definition has to narrow what the agent should attempt and say when it must hand the work to a person.

Key takeaways

  • Name the users first: the person who relies on the output, the person who triggers the work, and the person who receives the result.
  • Write the job as one repeatable task with a defined input, output and finish line.
  • Sort every possible action into three tiers: act alone, propose for approval, or never do.
  • Pick a success measure you can read from logs and reviews, not from an impression of quality.
  • Treat the definition as a living document and review permissions, evaluations and the job statement on a schedule.

Why a stronger model needs a narrower job

Capability and clarity are different things. A model that reasons across more context can reach more systems, read more documents and suggest more actions than a simpler one. If the job is vague, it expands to fill the space. The agent answers questions nobody assigned, touches records nobody meant it to touch, and produces output that is hard to review because nobody said what good looks like.

Product teams already handle this discipline for software. A feature has a user, a use case, acceptance criteria and a release boundary. Agents need the same structure, with one difference: they read free text, call tools and choose between steps, so the boundaries must be written in terms a system can enforce and a person can audit. Anthropic's engineering guidance on building effective agents makes a related point. It recommends starting with simple, well-understood patterns and adding autonomy only where it clearly helps the task.

Define the users before you define the agent

Most weak agent definitions start with a list of tools. Start with people instead. Three roles usually matter:

  • The requester, who starts the work and expects a usable result within a normal working day.
  • The reviewer, who checks the output and has the authority to reject it.
  • The receiver, who acts on the result, such as a customer, a finance lead or a field technician.

For each role, write what they need to see, how often they will interact with the agent, and what would make them stop trusting it. A reviewer who cannot see the sources behind a statement will reject everything. A receiver who gets a long summary when they wanted a yes or no will skip it. These answers shape the output format more than any phrase in a prompt does.

Write the job as a contract

A job statement is short enough to fit on one page and specific enough that two people would judge the same output the same way. Fill in these fields in plain language:

  • Trigger: the event that starts the job, such as a new message in a shared mailbox or a new row in a request sheet.
  • Inputs: the sources the agent may read, and who maintains each one.
  • Output: one sentence describing what the agent produces and in what format.
  • Finish line: the condition under which the job is complete, and the condition under which it goes to a person.
  • Exception owner: the named person who handles cases the agent cannot resolve, and the time by which they should do it.

If you cannot fill these fields without vague words like handle, support or manage, the job is not ready for an agent. The work may need to be mapped first. Our guide on process mapping before an agent walks through that step, and the agent or deterministic workflow decision guide helps decide whether a fixed script would serve the job better.

Set boundaries in three tiers

Boundaries work best as a sorted list of actions rather than a paragraph of warnings. Place every action the agent could take into one of three tiers:

  • Act: actions it may complete alone, such as labeling a request, drafting a reply from approved material, or filling a checklist from sources it can cite.
  • Propose: actions it prepares and a person approves, such as sending a message to a customer, changing a price, issuing a credit or editing a contract clause.
  • Never: actions outside its role, such as deleting records, changing permissions, or contacting people who are not on an approved list.

Tools should follow the tiers. Give the agent only the access the act tier needs, and use separate credentials with narrow scopes for anything above it. OWASP's authorization guidance and the OAuth 2.0 framework both point toward granting specific permissions rather than broad accounts. Also treat anything the agent reads as data, not as instructions. An email or a document can contain text that tries to redirect the agent. The OWASP prompt injection cheat sheet explains why this control has to live in the system design, not only in a sentence of the prompt.

Illustrative scenario: a regional equipment rental company wants an agent to triage its shared service inbox. The agent may label each message by type and draft replies to booking questions using the published rate sheet, which is the act tier. It prepares, but does not send, any reply that mentions a damage charge, which is the propose tier. It never changes reservation dates or waives a fee. A dispatcher reviews the proposed damage replies each morning, and the list of approved sources is reviewed on a set schedule.

Choose a success measure you can observe

A success measure must be readable from the system. The statement that customers seem happier is not one. Better measures ask whether the output was accepted without changes, how often a reviewer changed it and why, and which exceptions reached a person. Record each run with its inputs, the sources used, the tool calls and the final decision, so a reviewer can trace why the agent acted as it did. OpenTelemetry's observability concepts describe traces and logs as the way to reconstruct behavior across multiple steps. Keep the record proportional to the job. You need enough detail to explain one bad outcome, not a warehouse of data nobody reads.

Build a small set of real cases from past work, including the awkward ones, and grade the agent against them before launch and after each change. Anthropic's guidance on evaluating agents recommends starting from real tasks and observed failures rather than relying only on synthetic benchmarks. If you use another model as a grader, calibrate it against human judgments first, because an unchecked grader can make a weak job look strong.

Keep the definition alive after launch

Users, tools and models all change after an agent goes live. Review the job statement whenever a new tool is connected, when the mix of requests shifts, after any incident, and at a regular interval set by the owner. Ask whether the three tiers still match the real risk, whether the exception queue is being cleared by the named owner, and whether the success measure still measures the thing you care about. When the job ends, retire the agent deliberately and keep its records, so the next person can see what it was allowed to do.

Be Human designs, implements and operates these agents with the definition kept on record. A first engagement sets out one useful role, the memory and tools it needs, its approval boundaries and an observable success criterion. Managed operations can then cover monitoring, recurring evaluations, integration maintenance, permission review, incident response and controlled changes to prompts or tools, so each change is logged and the team can see what the agent was allowed to do at any point.

If you are not yet sure which task in your business is worth handing to an agent first, the free Be Human AI diagnostic can help you find it.

Frequently asked questions

How many users should an agent serve at launch? Start with one primary reviewer and one clear requester. Adding more audiences usually means more output formats and more exceptions, which makes the job harder to evaluate. Widen the group only after the first reviewer accepts the work consistently.

Is a more capable model enough to skip the boundary work? No. A stronger model can follow more instructions, but a sentence in a prompt can be overridden by content the agent reads. Limits have to come from the tools and permissions the agent holds, so capability and access are controlled separately.

What if we cannot agree on a success measure? Begin with the test the reviewer already uses to accept or reject work, and record how often that reviewer changes the output and why. This gives you a baseline without inventing a score. You can refine the measure once you see which errors matter most to the business.

When should we revisit the job statement? Revisit it when a new tool is connected, when the request mix changes, after an incident, and at the review interval the owner has set. The statement is short, so the review takes little time compared with the cost of an agent drifting outside its role without anyone noticing.

Your next step

See where an AI agent fits in your company first.

Take the free Be Human AI diagnostic. A few questions about your work show which task to hand to an agent first, what it needs to know and where a person approves.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.

Less busywork. More room for you.

Bring us the work you wish you had time for.

Book a call to try a personalized AI agent free for 7 days ↗Or take the free AI diagnostic →