Personalized AI infrastructure
Superintelligence Feature Flags: Turn Agent Capabilities On by Role
Learn how to turn new AI agent capabilities on for one team at a time, test the change against clear success criteria, and widen access only after review.

Feature flags for AI agents are switches that control which abilities an agent can use, such as a new tool, a wider memory scope or a higher autonomy limit, for a defined group of people or roles. Instead of releasing a new capability to the whole company at once, you enable it for one team, measure what changes against a written success criterion, and widen access only when the evidence supports it. The same switch lets you turn the capability off quickly if it behaves badly.
Key takeaways
- Treat every new agent ability as a flag with a named owner, a defined scope and a review date.
- Enable a capability for one role first, chosen for clear work and a low cost of error.
- Write the success criterion and the stop condition before the flag goes on, not after the first result.
- Keep permissions and approval thresholds in their own records so that turning a capability on never widens access by accident.
- Widen by role only after reviewing traces, failures and reviewer corrections for the current team.
Why do new agent capabilities need a rollout switch?
Stronger agents rarely arrive as one change. A new capability usually bundles several things: a new tool, access to another data source, a longer chain of actions taken without a person in between, or a different way of planning the work. Each of these changes what the agent can do and what can go wrong. If they all reach the whole company at once, nobody can tell which change caused a good result or a bad one.
A flag separates the decision to build from the decision to use. The capability can be deployed to the agent platform in an off state, tested with the system's own logs, and then turned on for a named role. This is the practical side of the managing step in the NIST AI Risk Management Framework, which asks organizations to measure and manage risk across the life of a system rather than only at launch.
The usual temptation is to share a successful pilot with everyone. The result is that the team with the most friction, the least context or the highest cost of error meets the new behavior first, with no baseline to compare against. A role-based flag gives you a controlled comparison: one group using the capability, and a comparable group continuing with the current process.
What should a capability flag record contain?
Write each flag as a short record that a manager can read in two minutes. A usable record includes:
- The name and a plain purpose, written in the words the team already uses
- The role or team that receives the capability, and the roles that are explicitly excluded
- The tools, data sources and memory scopes the capability can touch
- The permissions it depends on, and the approval boundary for actions with cost, customer impact or legal weight
- The success criterion, stated as an observable result
- The stop condition, and who can trigger it
- The accountable owner and the review date
A flag without a success criterion is only a switch. The criterion should name what you expect to see in the work product, for example fewer handoffs back to a person, fewer corrections by reviewers, or a shorter time to a complete answer. Choose one or two measures so the test stays readable. If the team cannot agree on what good looks like, the capability is not ready for a flag yet.
If the flag changes what an existing agent can do, route it through the same change process you use for other agent edits. A short guide to that process is in How to Request and Approve Changes to a Managed AI Agent.
How do you choose the first team?
The first team should be useful enough that its feedback matters and forgiving enough that a mistake is recoverable. Good candidates usually share a few traits:
- A repetitive task with a clear finish line, so outcomes can be checked
- A manager who reads results and makes a decision on them
- A low cost of a wrong draft, with a person reviewing outputs before they leave the company
- Existing records the agent can use without inventing missing context
- Members willing to report friction honestly, including the cases where the agent saved no time
Avoid starting with the team under the heaviest deadline pressure, because it will have no patience for a test, and avoid the group that handles the most sensitive personal or financial records until the process has been proven elsewhere. The selection logic is covered in more depth in How to Choose the First Users for an AI Agent Pilot.
Illustrative scenario: a regional services company wants its scheduling coordinators to receive a new capability that drafts rebooking messages from the job history. The coordinators become the first team because every draft is reviewed before sending, the job history is complete, and the coordinator lead meets with the operations director every week. The flag stays off for field supervisors, who keep the current process so the comparison remains clean.
A staged plan for testing one team
A single-team test works best as a sequence with a decision at each stage. A practical version looks like this:
- Stage 1: Write the flag record and have the owner and the approver sign it, using the same discipline as any change request.
- Stage 2: Replay a set of real past cases from the team in a test environment, and record every place the agent is wrong, slow or unclear.
- Stage 3: Turn the capability on for the named team, with a human approval step for every action that leaves the company.
- Stage 4: Review traces and reviewer corrections at a fixed point, such as the end of the first full working cycle, and decide whether to keep, adjust or stop.
- Stage 5: Widen only to a second role, with its own record and its own baseline, rather than quietly extending the first flag.
Set the length of the test in the record before it starts. The test should cover the normal variety of the work, including one busy period and at least one exception case. Extending the test because the results are unclear is a decision that needs a written reason.
Why a flag is not a permission
A flag controls whether a capability is visible to a role. It does not decide what the agent may do with data and tools. OWASP's authorization guidance recommends enforcing access checks on the server side, denying by default, and checking permission for each action rather than assuming that an earlier step already approved it. Applied to agents, this means a tool call should succeed only when the role's permission allows it, whatever the state of the flag.
The OWASP guidance on prompt injection adds a second point. Text retrieved from documents, emails, web pages or tool results can contain instructions that try to change what the agent does. A capability that reads more sources can therefore create more exposure, even when the new feature looks harmless. Treat retrieved content as data, limit which tools can act on it, and keep approval steps in place for consequential actions. A broader review of this risk is in Quarterly AI Agent Permission Review: Steps and Evidence.
Keep permission grants in a separate record with its own review cycle. When someone asks why the agent can now do something, the answer should point to a permission entry and a change record, not to a flag that was switched on for convenience.
What should you measure during the test, and how does Be Human run it?
Every run of the agent should leave a trace: the request, the tools called, the data read, the approval decisions and the final output. The OpenTelemetry concepts for observability describe this pattern of traces, metrics and logs that lets a team reconstruct what happened in a system. For a flagged capability, the trace is the evidence that connects a result to the flag state at the time of the run.
Compare the enabled team with the baseline group on the criterion you wrote. Useful measures include the rate of reviewer corrections, the number of escalations to a person, tool errors, and the time needed to complete the task. Read the failures individually, not only the averages. An average can look fine while one class of case fails every time, and that class is usually the one that matters to customers.
In a managed engagement, Be Human designs and operates this layer for the company. The flag records, role scopes, approval boundaries and trace logs stay under the company's control, and changes pass through the same review as other agent changes. Monitoring, recurring evaluations and permission reviews run on a schedule, so a capability that looks sound in its first week is checked again before it reaches another team. The work is delivered remotely, in English or Portuguese.
When should you widen, pause or retire a flag?
The decision after a test is usually one of four. Use the record and the evidence to choose:
- Widen when the criterion is met, the failures are understood, and the owner signs the review.
- Pause when a failure touches customers, money or personal data, and investigate before the capability is switched back on.
- Adjust when the capability works but the approval boundary, the context or the success criterion needs a change, then test again.
- Retire when the capability is replaced, no longer used, or no longer matches the role.
Record each decision in the flag record with the evidence reviewed. Approval thresholds should be revisited as capability grows, and the reasoning is set out in Human Approval Thresholds for Stronger AI Agents: A Practical Guide. A rollout that moves in visible steps is easier to defend to a board, an auditor or a team asked to change its habits. If you are not yet sure which of your team's tasks is the right first one to hand to a new capability, the free Be Human AI diagnostic can help you find it.
Frequently asked questions
Do we need a dedicated feature flag platform to run this? Not necessarily. A small company can start with a written flag record, a configuration setting in the agent runtime that the platform enforces, and a log of who changed what and when. What matters most is that the flag is enforced by the system, so the agent cannot use a capability that the record says is off.
How long should a single-team test run? Long enough to cover the normal variety of the work, including a busy period and at least one exception case. Set the length in the flag record before the test begins. If you extend it, write down the reason and the new end date.
Can a flag replace a permission review? No. A flag controls whether a capability is visible to a role, while permissions decide what the agent may do with data and tools. Review both, but on their own schedules, so that a change in one does not silently alter the other.
What if the enabled team wants to keep the capability after the test? Make the decision with the evidence in the record. If the criterion was met and the failures are understood, convert the flag into a documented setting with an owner and a review date. If it was not met, retire or adjust the capability and keep the notes from the test so the next attempt starts from what was learned.
Your next step
See where an AI agent fits in your company first.
Take the free Be Human AI diagnostic. A few questions about your work show which task to hand to an agent first, what it needs to know and where a person approves.
Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.


