Who Is Accountable When an AI Agent Makes a Mistake?

When an AI agent makes a mistake, the business that deployed it is accountable to the customer, employee or regulator who was affected, because the agent acts in that business's name. Responsibility for the cause is then shared: the owner for decisions and permissions, the operator for configuration and monitoring, and the provider for the model and platform behavior it commits to in writing. The practical work is agreeing that split before an error occurs, not arguing about it afterward.

Key takeaways

• Outward accountability sits with the company whose name, channel or account the agent used. You cannot hand that to a vendor.
• Inward accountability can be divided: owner, operator and provider each answer for different parts of the system.
• Most agent errors trace to scope, permissions, stale context or missing review, which are decisions, not model failures.
• A named person, a written boundary and a usable record of what the agent did matter more than any clause about blame.
• Settle the split before go-live, test it with a rehearsed incident, and review it whenever the agent's scope changes.

Why is there no single answer to who is responsible?

People ask this question as if an AI agent were a product with a defect, where the maker is liable, or a tool misused, where the user is liable. An agent is both and neither. It is a model built by one party, wrapped in tools and instructions by a second, and given authority over real work by a third. A mistake can enter at any of those layers, and often at the seams between them.

Two questions get mixed up in most conversations, and separating them helps.

• Who answers to the person harmed? Generally the business that put the agent in front of them. A customer who receives a wrong message from your company does not care which vendor built the model.
• Who answers for the cause? That depends on where the fault entered, and on what each party agreed to do. This is the question you can shape in advance.

This article is operational guidance, not legal advice. Liability rules differ by country and sector, and some regimes place specific duties on the organization deploying an AI system. If your agent touches regulated data or decisions, involve counsel early. Frameworks such as the [NIST AI Risk Management Framework](https://www.nist.gov/itl/ai-risk-management-framework) and the [OECD AI Principles](https://oecd.ai/en/ai-principles) both treat accountability as something an organization assigns deliberately, with named roles and traceable decisions, rather than something that emerges on its own.

What does each party actually own?

The cleanest way to avoid confusion is to define three roles in plain language and write down what each one is responsible for.

The business owner is the company that uses the agent. The owner decides what job the agent does, what data it may see, what it may do without asking, and who approves the exceptions. The owner also holds the relationship with the customer or employee on the receiving end.

The operator is whoever configures, connects, monitors and maintains the agent day to day. In a small company the operator might be an internal person. In others it is a managed service provider. The operator is responsible for the setup matching what the owner decided, and for noticing when behavior drifts.

The provider is the party supplying the underlying model or platform. The provider is responsible for what it states about availability, data handling and model behavior, and for the terms it publishes. It is not responsible for how you chose to use the output unless its terms say so.

A working split looks like this:

• Owner: the purpose, the scope, the approval boundaries, the quality of source information, and the final call on any consequential action.
• Operator: permissions as configured, integrations, monitoring, evaluation routines, change control and first response to incidents.
• Provider: service availability, security of its platform, disclosed model changes, data processing terms and support commitments.

If one person holds more than one role, say so in writing. A company that builds and runs its own agent on a third party's model is owner and operator at once, and carries both sets of duties.

Which kind of mistake belongs to whom?

Before an incident, map the kinds of error you expect to the role most able to prevent them. This works as a decision guide: find the symptom, then find the first place to look.

• The agent did something it should never have been allowed to do. Look at permissions and approval boundaries. This is usually owner and operator: the owner defined the boundary or failed to, and the operator configured access. The difference between read, draft and send permissions is often where this goes wrong.
• The agent used outdated or wrong information. Look at the sources it was given and how they are refreshed. This usually sits with the owner, who supplies the policies and records, and the operator, who maintains the connection.
• The agent was manipulated by text inside a document or message. Look at input handling and the limits on what it can do after reading untrusted content. The [OWASP Top 10 for LLM Applications](https://genai.owasp.org/llm-top-10/) catalogs these risks. Responsibility is shared between operator design and owner decisions about which sources to trust.
• The model behaved differently after an update, or the service was unavailable. Look at the provider's change notices and service terms. This is where provider commitments matter, and where the operator should have tests that catch regressions.
• The agent was right, but a person acted on it carelessly or skipped a required review. Look at the review step and who was assigned to it. This is an owner matter: the process existed and was not followed.
• Nobody can say what happened. This is a failure of records, and it belongs to the operator to fix and the owner to require.

Notice how few of these are pure model faults. Most are decisions about scope, access, information and review, which is good news: they can be controlled.

An illustrative scenario: the wrong refund promise

This is an illustrative scenario, not a real case.

A distribution company uses an agent to draft replies to customer emails about late deliveries. The owner, the head of customer service, set a rule that the agent may draft but a person sends. Months later, to save time, a team lead enables automatic sending for messages the agent rates as routine. The operator applies the change on request but does not ask for a written approval boundary.

A customer writes about a damaged pallet. The agent, drawing on an outdated policy document that nobody had retired, promises a full replacement at no charge, and the message goes out automatically. The customer relies on it.

Who is accountable? Toward the customer, the distribution company, without question. Internally, the cause has several parts. The owner allowed automatic sending without defining which topics were excluded. The operator changed a permission without a recorded approval and did not flag the stale document. The provider did nothing wrong. With a written split, the company can see this clearly, fix the three gaps and move on. Without one, the conversation becomes a search for someone to blame.

The lesson is not that the agent should never send. It is that the decision to let it send should have had a name, a boundary and a date attached. For guidance on where an agent should pause, see where your AI agent should stop and ask.

How do you assign accountability before go-live?

This plan can be done in a few working sessions and does not need special tooling. It produces a short document the owner and operator both sign off on.

• Name the role. Write one sentence describing the job the agent does and what it explicitly does not do.
• Name the accountable person. One individual in the company owns the agent's outcomes. A team or a title is not enough. Add a named backup.
• List the actions by consequence. Separate what the agent can read, what it can draft, and what it can send, change or spend. Mark which require human approval and who gives it.
• Assign each source of information an owner. Every policy, price list or customer record the agent relies on needs a person responsible for keeping it current, and a rule for retiring old versions.
• Decide what gets recorded. At minimum: the request, the sources used, the action taken, who approved it and when. Confirm that the operator can retrieve this on demand.
• Define what counts as an incident. Wrong external message, data exposure, action outside scope, repeated failure. Say who is told, how fast, and who can pause the agent.
• Agree on change control. Who may alter instructions, permissions or connected tools, and how those changes are approved and logged.
• Rehearse one failure. Walk through a plausible mistake and check that each person knows their part. Adjust the document where it was unclear.

For the response side of this, the AI agent incident response playbook covers containment, communication and review in more detail.

What should the contract and the records say?

Accountability that exists only in conversation disappears when people change roles. Put it in two places: the agreement with your operator or provider, and the operating records.

In the agreement, look for clear answers to these points:

• Which party configures and maintains permissions, and who approves changes to them.
• What the operator monitors, how often, and what it does when something looks wrong.
• How incidents are reported to you, and within what expectation. An AI agent service level works best when it states what is measured and avoids promises nobody can verify.
• What the provider's terms say about data handling, model changes and notice of them.
• Who owns the accounts, credentials and logs, so you can access them if the relationship ends.
• How liability is treated between the parties. Have counsel read this section. Do not assume that silence favors you.

In the records, keep a plain log of decisions: when the scope was set, when permissions changed, who approved each change, and the results of periodic reviews. Observability concepts such as traces and structured logs help here, but the purpose is simple: a person who was not present should be able to reconstruct what the agent did and why it was allowed to.

Also pay attention to access design. The [OWASP authorization guidance](https://cheatsheetseries.owasp.org/cheatsheets/Authorization_Cheat_Sheet.html) recommends least privilege and checking permissions on every request. For an agent, that means giving it only the access its role needs. Narrow access shrinks the set of mistakes anyone can be accountable for.

How does Be Human handle accountability?

Be Human designs, implements and operates personalized AI infrastructure for companies, and treats the accountability split as part of the first engagement. That engagement defines one useful role, the memory and tools it needs, the approval boundaries and an observable success criterion, so the owner's decisions are written down before the agent acts. Managed operations can then include monitoring, recurring evaluations, permission review, integration maintenance and incident response, with changes made in a controlled way. The company keeps control of identity, memory, channels, tools and permissions, and decides what the agent may do without a human.

Frequently asked questions

Can I blame the AI vendor if the agent makes a mistake? Sometimes, but rarely entirely. Your customers will hold your company responsible, and a vendor's liability depends on its terms and on whether the fault was in what it supplied or in how the agent was configured and used. Read the terms before you rely on any assumption.

Who is responsible if an employee follows a wrong agent suggestion? The business, in most cases, and internally the owner of the process that told the employee when to rely on the agent and when to check. If your policy required a review and the employee skipped it, that is a process and training matter. An acceptable use policy helps make those expectations clear.

Does human approval remove the company's accountability? No. It places a named person inside the decision, which improves quality and traceability, but the company remains accountable for the outcome. Approval works best when the reviewer has enough context to say no and is not pressured to approve quickly.

What is the first thing to do after an agent error? Contain it: pause the relevant action, preserve the records and tell the accountable person. Then correct the harm to the affected party, find which layer failed, and update the boundary or the information that allowed it. Resist assigning blame until the record shows what happened.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.