RACI for AI Agents: Who Is Responsible for What, Step by Step

A RACI for AI agents assigns every recurring decision about an agent (design, approval, operation, incident handling and change) to exactly one accountable person. It also names who does the work, who must be consulted and who must be told. Treat the agent as a role in your company with an owner, not as software that belongs to everyone and therefore to no one. The matrix below is a starting point: replace the titles with the real people in your company.

Key takeaways

• Every row of the matrix needs exactly one Accountable person. If two people share accountability, nobody has it.
• The agent owner is a business role, not a technical one. That person answers for what the agent does, even when someone else operates it.
• Approval boundaries (what the agent may do alone and what needs a human) are a decision for the sponsor, not a setting for the operator to pick.
• Incident handling needs a named person with the authority to pause the agent, decided before anything goes wrong.
• A RACI is only useful if it is short enough to be read. Review it whenever scope, permissions or the operator change.

Why does an AI agent need its own RACI?

An agent is unlike most business software. It reads data owned by several teams, acts through accounts and tools, remembers context between conversations, and changes whenever a model, an instruction or an integration changes. Each of those facts creates a decision, and each decision needs an owner.

When no one is named, the same three problems tend to appear. Permissions grow because it is easier to say yes than to ask who is accountable. Problems are noticed by the people who work next to the agent, but nobody is sure who to tell. Changes happen quietly, because the person who could stop them does not know they are happening.

A RACI fixes this with four letters:

• Responsible: the person or team that does the work.
• Accountable: the one person who answers for the outcome and has the final say. Only one per row.
• Consulted: people who give input before the decision or action.
• Informed: people who are told afterwards.

This sits inside a wider structure. If you are building the broader frame, AI governance for mid-sized companies covers the lightweight policies that the RACI plugs into. Frameworks such as the NIST AI Risk Management Framework make the same point in more formal language: governance starts with clear roles and lines of accountability.

Which roles do you need to name?

You do not need a committee. You need seven roles, and in a small company one person can hold several of them, as long as the person who is Accountable for a row is not simply approving their own work without a second view.

• Executive sponsor: funds the agent, accepts risk on behalf of the company and decides scope. Usually a director or owner.
• Agent owner: the business lead of the process the agent works in. Defines what good work looks like and answers for the agent day to day.
• Data owner: the person who answers for the information the agent may read, such as customer records, contracts or internal documents.
• Security and IT lead: owns accounts, permissions, network access and the company's security rules.
• Operator: the team that runs the agent technically, monitors it, maintains integrations and ships changes. This can be an internal team or an external partner.
• Approvers: the frontline people who review the agent's proposed actions when a human decision is required.
• Legal or compliance advisor: consulted on personal data, contracts, sector rules and notifications. Often an outside adviser in smaller firms.

The agent owner is the role companies skip most often, and it causes the most trouble. If the owner is missing, the operator ends up deciding business questions, such as how much the agent should be allowed to promise a customer. For the wider picture of who owns, reviews and runs AI, see the AI operating model for companies.

What does the matrix look like for design and approval?

Read each line as: the activity, then R (does it), A (answers for it), C (advises) and I (is told). Design and approval happen before the agent touches real work, so they are where the most important boundaries are set.

Design:

• Define the role, scope and success criterion. R: agent owner. A: executive sponsor. C: operator, security lead, data owner. I: affected teams.
• Decide which data and memory the agent may use. R: operator (implements the sources). A: data owner. C: agent owner, security lead, legal advisor. I: sponsor.
• Create the agent's own accounts and grant permissions. R: operator. A: security lead. C: data owner, agent owner. I: sponsor.

A separate identity matters here: when the agent has its own accounts, every action is traceable and permissions can be narrowed or revoked without touching a human's access. Why your company AI agent needs its own identity and accounts explains the reasoning. Authorization guidance such as the OWASP Authorization Cheat Sheet supports the same default: grant the least access that lets the role work, and deny the rest.

Approval:

• Set approval boundaries, meaning which actions the agent may take alone and which need a human. R: agent owner. A: executive sponsor. C: security lead, legal advisor, approvers, operator. I: affected teams.
• Approve go-live for the defined scope. R: agent owner. A: executive sponsor. C: security lead, data owner, operator. I: all teams that will interact with the agent.
• Approve individual actions inside the boundary, such as a refund draft or an outbound message. R: approvers. A: agent owner. C: none needed at the moment of action. I: operator (through logs).

Notice that approval boundaries have the sponsor as Accountable, not the operator. How much freedom the agent gets is a business risk decision. The operator can advise on what is technically safe, but should not be the one who decides what the company is willing to accept.

What does the matrix look like for operation, incident handling and change?

Once the agent is live, the work becomes recurring. These rows are the ones a good operating routine repeats every week or month.

Operation:

• Monitor behavior, errors, cost and logs. R: operator. A: agent owner. C: security lead. I: sponsor.
• Run recurring evaluations against agreed test cases. R: operator. A: agent owner. C: approvers (who see the real mistakes). I: sponsor.
• Review permissions on a schedule and remove what is unused. R: operator (prepares the list). A: security lead. C: data owner, agent owner. I: sponsor.
• Maintain integrations when connected systems change. R: operator. A: agent owner. C: security lead. I: approvers.

Observability is what makes the monitoring row real. The OpenTelemetry observability primer describes logs, metrics and traces as the signals that let you ask what a system did and why. If the operator cannot show the agent owner what the agent did last Tuesday, the row is not being performed.

Incident handling:

• Pause the agent when behavior is wrong or risky. R: operator, agent owner (either can pull the brake). A: agent owner. C: security lead. I: sponsor, affected teams.
• Investigate the cause and fix it. R: operator. A: agent owner. C: security lead, data owner. I: sponsor.
• Decide whether customers, partners or authorities must be notified, and do it. R: legal advisor. A: executive sponsor. C: security lead, agent owner. I: operator.
• Hold a post-incident review and record what changes. R: agent owner. A: executive sponsor. C: operator, security lead. I: affected teams.

Change:

• Request a change, such as a new task, tool, data source or model. R: agent owner. A: agent owner. C: operator. I: security lead.
• Test and release the change in stages. R: operator. A: agent owner. C: security lead, approvers. I: affected teams.
• Approve changes that widen scope or permissions. R: agent owner. A: executive sponsor. C: security lead, data owner, legal advisor. I: operator.
• Retire the agent or one of its tasks. R: operator. A: executive sponsor. C: data owner, security lead. I: affected teams.

The split on change is deliberate. Small adjustments that stay inside the agreed scope belong to the agent owner. Anything that gives the agent more reach goes back to the sponsor. For the technical side of releasing safely, see how to update an AI agent in production without breaking it.

What does this look like in an incident? An illustrative scenario

Illustrative scenario: a mid-sized distribution company runs an agent that drafts replies to order-status questions. An approver on the support team notices that the agent quoted a delivery window that does not match the carrier's current information. Here is how the matrix plays out.

• The approver does not send the draft and tells the agent owner, who is the head of customer operations. This is the Informed path built into the approval row.
• The agent owner asks the operator to pause the order-status task. The agent owner is Accountable for the pause, so no one needs to wait for a meeting.
• The operator investigates and finds that a carrier integration changed its response format. The agent was reading an old field. The operator fixes the integration and adds a test case to the recurring evaluation.
• The security lead is consulted and confirms that no extra permissions were involved. The data owner confirms that no personal data left the company.
• Because no customer received the wrong information, the legal advisor is informed rather than engaged. The sponsor receives a short summary.
• At the review, the agent owner records one change: the integration now has a check that fails loudly when the response shape changes.

Nothing in this story required heroics. Each person knew whether they were doing, deciding, advising or listening, and the pause happened within minutes of the first doubt.

How do you adapt this matrix in one week?

Do not paste the table into a document and call it done. Adapt it with the people who will carry the rows.

• Step 1: Write the agent's role in one paragraph, with its tasks, its data and its success criterion. If this is not clear, start with a job description for your first AI agent role.
• Step 2: Name a person, not a team, for each of the seven roles. Mark where one person holds two roles and ask whether a second view is needed.
• Step 3: Walk through every row and cross out any that do not apply yet, such as notification rules for an internal-only agent. Add rows for anything specific to your sector.
• Step 4: Check that every row has exactly one A. Where you find two, choose one. Where you find none, ask who would be called at night.
• Step 5: Run a thirty-minute tabletop exercise using a scenario like the one above. Note where people hesitate, and fix the row.
• Step 6: Store the matrix where the operator, approvers and sponsor can all find it, and attach it to the go-live decision.
• Step 7: Put a recurring date in the calendar to review the matrix, and trigger an extra review whenever the scope, a major permission or the operator changes.

Common mistakes to look for during the review:

• The operator is listed as Accountable for business outcomes.
• The sponsor is listed as Responsible for tasks they will never do.
• No one is named to pause the agent, or only one person who is often unavailable.
• Approvers are not consulted when evaluations are designed, even though they see the real errors.
• The matrix lists a team name where a person should be.

How does a managed model change who does what?

Moving operation to a partner changes the Responsible column, but not the Accountable column. At Be Human, a first engagement defines one useful role, the memory and tools it needs, the approval boundaries and an observable success criterion, and those become the first draft of this matrix. Managed operations can then cover monitoring, recurring evaluations, integration maintenance, permission review, incident response and controlled improvements, which means the operator rows are performed by Be Human while the company keeps the sponsor, agent owner and data owner roles. Decisions about scope, risk and what the agent is allowed to do remain with people in your company, and the agent runs under human control.

If you are weighing who should run the agent, in-house AI team versus external partner compares the options, and what managed AI services include lists the recurring work.

Frequently asked questions

Do we need a RACI if we only have one small agent? Yes, but a short one. Even for a single agent, someone must be able to decide what it may do, someone must be able to pause it, and someone must review its permissions. A one-page version with the same four groups of rows is enough.

Can the same person be Responsible and Accountable? Yes, and in small companies this is common. The risk is that no one else sees the work, so for the most sensitive rows, such as permissions and approval boundaries, make sure someone else is at least Consulted.

Who should be the agent owner if the agent serves several departments? Choose the leader of the process where mistakes cost the most, and make the other department heads Consulted for changes that affect them. If the agent's scope is too broad for one owner to answer for, that is a signal to split the roles, as discussed in one company agent or one per team.

How often should the RACI be reviewed? Review it at every change in scope, permissions or operator, and on a regular schedule even when nothing seems to have changed. People move roles, tools are replaced and agents quietly gain new tasks, so the matrix drifts if no one looks at it.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.