Accountability for Superintelligent Agent Errors: Who Answers

An AI agent's error is answered by the people and contracts assigned before it happened: the business owner of the workflow, the approver who allowed the action, and the operator or vendor who runs the system. The model itself cannot answer for anything, and asking who is to blame after an incident usually delays the fix. Responsibility works when each action an agent can take has a named owner, a named approver for high-impact cases, and a written boundary that tells everyone what to do when the agent steps outside it.

Key takeaways

• Assign accountability per action type before launch, not per incident.
• Keep three roles separate: business owner, approver and operator. A vendor supports these roles but cannot replace them.
• Give each high-impact action one approver and one backup who can act outside normal working hours.
• Check every vendor contract for incident notice, log access and what happens when the underlying model changes.
• Review the assignments whenever tools, permissions, channels or staff roles change.

Why responsibility stalls after an error

When an agent sends a wrong message, updates a record in the wrong system or takes an action nobody expected, the first questions are practical: who can stop it, who can see what it did, and who decides whether customers need a correction. If those answers were never written down, each person assumes someone else holds them. The IT lead thinks the business owner approved the rule. The business owner thinks the vendor configured the tool. The vendor assumes the customer defined the scope. Each explanation may be partly true, and that is exactly the problem.

The NIST AI Risk Management Framework treats governance, roles and accountability as part of managing AI risk across the system's lifecycle, not as a one-time paperwork step. The OECD AI Principles similarly call for accountability for the proper functioning of AI systems among the actors who develop and deploy them. Both point to the same practical conclusion: accountability has to be designed into the deployment, and the deployment has to name who holds each decision.

Name three roles before launch

Most agent deployments need three distinct roles. In a small company one person may hold two of them, but the duties should still be written separately so that nobody assumes a duty is covered by someone else.

• Business owner: answers for the outcome the agent serves and for the rules it must follow, such as which customers it may contact, what it may promise and what counts as success.
• Approver: a named person with authority to allow a specific class of action, such as issuing a refund, changing a contract date or deleting stored customer memory. Approvers answer for their decisions, not for the agent's general behavior.
• Operator: the person or team that runs the agent day to day, watches its activity, pauses it when needed and leads the response to an incident.

The vendor or platform provider answers for what it contracted to provide: hosting, security controls on its side, data handling terms and notice of incidents affecting your data. It does not decide your refund policy or approve your customer messages. If a vendor's model update changes behavior, the vendor should tell you, but your business owner still decides whether that change enters your workflow. For guidance on when an agent should halt and wait for a person, see where your AI agent should stop and ask.

Map every action to an accountable party

Responsibility is easier to assign at the level of individual actions than at the level of the agent as a whole. An agent that answers questions and reads a shared calendar carries different risk from one that books appointments, sends payment reminders or issues credits. Use this sequence before the agent goes live.

• List every action the agent can take, including reads that expose personal data, not only the actions that write or send.
• Mark each action low, medium or high impact, based on whether it can be reversed and how many people it reaches.
• Name the business owner of the workflow each action belongs to.
• For each medium and high impact action, name one approver and one backup approver.
• Record which vendor or internal team controls each system the action touches.
• Write the stop condition for the workflow and name who may trigger it.
• Store the map where the operator and every approver can find it without asking.

A quick test is useful here. If you cannot fill in the owner, approver and operator for one action, the agent should not perform that action yet. Limit it to drafting or reading until the gap is closed, and widen its permissions only after the map is complete and the operator has tested the stop condition.

Illustrative scenario: a wrong policy sentence reaches customers

Illustrative scenario, not a real case: a support agent answers customer questions about returns. After a policy change, its instructions still contain the old return window, and for two days it replies to customers with that window. A support lead notices a complaint and reports it to the operations team.

With the map in place, the responsibilities are already known.

• The business owner, the support lead, decides the correct policy wording and approves the correction message sent to affected customers.
• The refund approver, the finance manager, reviews any refund requested under the old window before it is paid.
• The operator pauses replies on that topic, exports the affected conversations from the logs and assigns a person to contact each customer.
• The vendor confirms that the log export is complete and that the instruction change was applied only where intended.

Without the map, each of these tasks tends to wait until someone claims it, and the first hour is spent deciding who is allowed to decide. With the map, the response is mostly a checklist. The incident still costs time and some customer goodwill, but it does not turn into a dispute about whose job it was.

What to ask vendors before you sign

A vendor agreement should describe responsibility in terms you can check. Ask these questions and keep the answers in writing, not in a sales conversation.

• Who can access our prompts, memory, logs and model calls, and in which locations are they stored?
• How will you notify us of an incident that affects our data or our agent's behavior, and through which channel?
• If your underlying model changes, how are we told, and can we test the change before it reaches our workflows?
• Can we pause the agent's actions and export its logs without waiting for your team?
• Which subprocessors handle our data, and how do we learn when that list changes?

The legal terms that answer these questions are covered in the data processing agreement checklist for AI vendors. Remember that a contract allocates liability between you and the vendor. It does not answer the question your customers ask, which is who speaks for your company. That answer stays with your business owner, whatever the contract says.

How Be Human runs this in managed operation

Be Human designs, implements and operates personalized AI infrastructure for companies, with agents that keep their identity, memory, channels, tools, permissions and operating context under human control. A first engagement defines one useful role, the memory and tools it needs, its approval boundaries and an observable success criterion. The owner, approver and operator for each boundary are written into that definition at the start, so the question of who answers is settled before the first live action. Managed operations then add monitoring, recurring evaluations, permission reviews and incident response, which gives the named operator a routine to follow rather than only a document to file.

The records matter as much as the roles. Logs should show which instruction version was active, which tool was called, which approver allowed a high-impact step and when it happened. Those records let a team reconstruct an error in minutes instead of days, and they show whether the boundary was respected or whether the agent acted outside it. The same questions apply if you run this in-house, and the answers should be just as specific.

Common mistakes and the first step

Several patterns cause accountability to fail after an error, and most can be avoided at the planning stage.

• Naming the model or the agent as the owner, which gives no person a decision to make.
• Assuming the vendor carries responsibility for your business rules and customer commitments.
• Listing one approver who is unavailable for long stretches of the month.
• Granting an approval once and treating it as permanent for every future case of the same kind.
• Forgetting the agent when the staff member who configured it leaves the company.

Start small. Choose one agent role, map its actions, assign the three roles and test the stop condition with the operator before you widen any permission. If you are not yet sure which task should go first, the free Be Human AI diagnostic can help you find the first task worth handing to an agent.

Frequently asked questions

Who is legally responsible when an AI agent makes a mistake? Legal responsibility depends on jurisdiction, contract terms and the specific facts, and this article does not give legal advice. In practice, the company that deploys the agent answers to its customers and regulators for what the agent does in its name, and then seeks recovery from vendors where the contract allows. Review your contracts and consult counsel about your specific exposure.

Can a vendor take responsibility for our agent's decisions? A vendor can be responsible for the service it contracted, including security measures, data handling, availability terms and incident notice. It usually cannot decide your refund policy, approve customer-facing wording or accept business risk on your behalf. Keep those decisions with a named owner inside your company, and write that ownership into your internal map.

Who should approve high-risk actions if we are a small company? Choose one approver and one backup, and make sure the backup can act when the first person is unavailable. The approver should hold authority over the outcome involved, such as the person who owns the money or the customer commitment at stake. A formal two-person rule is not always necessary, but high-impact actions should never be approved by the agent's own configuration.

How often should accountability assignments be reviewed? Review them whenever the agent gets a new tool, channel, permission or workflow, and whenever a named owner, approver or operator changes role or leaves. Many teams also check the assignments during their regular permission audits. Without a scheduled review, the map gradually drifts away from what the agent actually does.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.