Insurance for AI Agent Mistakes: What to Ask Your Broker

Start by asking your broker whether your current policies respond to a mistake made by an AI agent, and ask for the answer in writing. The honest answer depends on the wording of each policy, not on the technology, and it often splits across professional liability, cyber, general liability and crime coverage. You will get a more useful answer if you arrive with a plain description of what the agent does and a prepared list of questions.

Key takeaways

• Insurance responds to wording, not labels. A policy that never mentions AI may still cover or exclude an AI-caused loss, so request written confirmation instead of assuming.
• One agent mistake can touch several policies. List the harms that could happen (wrong statement to a customer, data sent to the wrong person, payment made in error) and ask which policy answers each.
• Insurers care how the agent is controlled. Permissions, approval steps and activity records are evidence you can prepare before the meeting.
• Exclusions and conditions matter as much as the coverage grant. Look for wording on automated systems, required security practices, contractual promises and repeated incidents.
• Insurance sits behind controls, not in place of them. Human approval, limited access and a record of what the agent did reduce both the chance of a loss and the friction of a claim.

Why does an AI agent mistake not fit neatly into one policy?

Insurance policies were written around people, products and conventional software. An AI agent is a mix: it acts on instructions, uses tools, reads company data and sometimes talks to customers. When it makes a mistake, the resulting harm can land in more than one category, and each category belongs to a different policy with different conditions.

Think about the harm first, not the technology. These are the typical shapes a loss can take:

• A wrong statement or recommendation given to a customer or client, which usually points to professional liability or errors and omissions cover.
• Confidential or personal data exposed, sent to the wrong recipient or extracted by a malicious instruction, which usually points to cyber cover.
• A payment, refund or order issued in error, which may touch crime or financial loss cover, often with conditions about how the transaction was authorised.
• Injury or property damage connected to instructions the agent gave, which points to general liability.
• A missed deadline or failed service under a customer contract, which raises contractual liability questions.

A mistake can also start outside your company: in the model, a hosting service or an integration. Knowing who is responsible for which layer helps you explain the risk to a broker, and the shared responsibility model for AI agents is a good starting point for that map.

What should you bring to the broker meeting?

Brokers can only assess what you describe. A vague statement such as we use AI for customer service invites a vague answer, or a broad exclusion. A one-page description of each agent gives the conversation something concrete to work with.

Prepare these items for every agent that touches customers, money or sensitive data:

• The role in one sentence: what job the agent does and for whom.
• Whether it faces customers and partners or works only internally.
• The systems it can read from and the systems it can change.
• The actions it can take alone, and the actions that require a named person to approve first.
• The types of data it handles, such as personal data, payment details or contract terms.
• Who reviews its work, how often, and what happens to rejected output.
• What records exist: inputs, outputs, tool calls and approvals, and how long they are kept.
• Which providers are involved: the model, hosting, and any connected services.
• Any contract promises you have made to customers about accuracy, confidentiality or turnaround.

If an outside provider operates the agent for you, the scope document should already hold most of this. A clear AI agent statement of work is the easiest source, and it also shows the broker where responsibility sits.

Which questions should you ask, policy by policy?

Bring questions, not conclusions. Ask each one, and ask the broker to confirm the answer in writing, citing the policy section.

For professional liability or errors and omissions:

• Does the policy cover errors in services we deliver when an AI agent produced or helped produce the work?
• Is there any exclusion, endorsement or definition that mentions artificial intelligence, automated systems or software-generated content?
• Does coverage depend on a human reviewing the output before it reaches the client?
• Who counts as the professional when the work product was drafted by a system?

For cyber coverage:

• Does the policy cover data exposure caused by an agent we configured, for example a message sent to the wrong recipient?
• If hostile text hidden in an email or document manipulates the agent into leaking data, is that a covered security event? (This is the prompt injection problem described in the OWASP guidance listed under sources.)
• Are investigation, notification and recovery costs included?
• If our model or hosting provider has an outage or breach, does the policy respond to our resulting losses?

For crime and financial loss:

• If an agent sends a payment or changes bank details because of manipulated instructions, is that covered, or excluded because an authorised system initiated it?
• Does coverage require verification steps such as call-back checks or dual approval?
• Are errors treated differently from fraud?

For general liability and contracts:

• Does general liability respond if an agent's instructions contribute to injury or property damage, or is technology-related harm excluded?
• Do the indemnities and service promises in our customer contracts go beyond what our policies would pay for?

How do exclusions and conditions change the answer?

A coverage grant can look generous and then be narrowed by a condition further down. These are the places to push, because they decide whether a claim is paid.

• Security or control conditions: does the insurer require specific practices, and does an unmet practice reduce or void cover?
• Unapproved tools: if an employee connects an unsanctioned AI tool to company data, is the resulting loss still covered?
• Aggregation: if one flawed instruction causes the same error across many customers, is that one claim or many, and how does that affect deductibles and limits?
• Claims-made timing: if a mistake happens now and is discovered later, which policy period and retroactive date apply?
• Contractual liability: are promises you made in customer contracts excluded from cover?
• Change notification: must you tell the insurer when you deploy an agent or materially change what it does?

The last point is often missed. Applications and renewals ask about operations, and a new agent that sends customer communications is a change in operations. Ask your broker whether and when to disclose it, and record the answer.

What does an illustrative scenario show about coverage gaps?

Illustrative scenario: a distribution company uses an agent to draft replies to customer quote requests. The agent reads a price list. An outdated file remains in its knowledge source, so a draft quotes a lower price than intended. A busy employee sends it without checking, and the customer accepts.

Walk this through with a broker and several questions surface:

• Is a pricing error a claim by a third party, or simply a business loss the company absorbs? Many policies do not pay for the cost of honouring your own mistaken price.
• Would professional liability apply at all, or does a contractual promise exclusion remove it?
• If the customer had suffered a downstream loss and sued, would the AI-assisted origin change the response?

The useful lesson is that some losses are not insurable in practice and have to be prevented. Here, three controls would have mattered more than any policy: one owner for the price source, an approval step before any quote leaves the company, and a record showing which file the agent used. Insurance conversations get easier when you can show those controls exist.

Which controls make the conversation easier?

Brokers and underwriters are assessing how likely a loss is and how well you can show what happened. You do not need a certification to prepare, but you do need evidence that is current and specific.

• Scoped permissions: the agent can reach only what its role requires. A regular permission review for AI agents is the clearest proof of this.
• Approval boundaries: actions with financial, legal or customer impact require a named human decision.
• Activity records: inputs, outputs, tool calls and approvals are retained so an incident can be reconstructed. The audit trail requirements for regulated companies set a high but useful bar for what to log.
• Recurring evaluation: someone tests the agent against known cases and records the results.
• An incident procedure: who is told, who can switch the agent off, and how a customer is informed.

For a risk framework that insurers and boards often recognise, the NIST AI Risk Management Framework gives a shared vocabulary for mapping and managing these risks, and the OWASP Top 10 for LLM Applications lists the failure modes worth discussing with technical staff.

Be Human designs and operates agents with these controls built in: each agent has a defined role, scoped permissions, approval boundaries and activity records, and managed operations include permission review, recurring evaluations and incident response. This produces concrete documentation you can hand to a broker. Be Human does not sell insurance or give legal advice, and no record guarantees that a policy will respond.

What should you do after the meeting?

Treat the broker conversation as the start of a short project with an owner and a finish line.

• Step one: write the one-page description for each agent and have the person accountable for it confirm it is accurate.
• Step two: send your question list to the broker ahead of the meeting so the answers can be researched, not improvised.
• Step three: ask for written responses that cite policy sections, and for any endorsement or exclusion that mentions AI or automated systems.
• Step four: list the gaps, meaning harms that no policy clearly covers, and decide for each whether to buy cover, change a contract, add a control or accept the risk.
• Step five: check your customer contracts for promises that exceed your cover, and bring in your lawyer where they do.
• Step six: set a review trigger, such as a new agent, a new data type, a new provider or each renewal, so the conversation repeats when the facts change.

Keep the broker's answers with your other governance records. A lightweight AI governance approach for mid-sized companies can hold these documents in one place without becoming a large program.

Frequently asked questions

Do standard business insurance policies cover AI errors? Sometimes, but not reliably. Coverage depends on the policy wording, the type of harm and any conditions or exclusions, so a policy that does not mention AI can still respond or can still exclude the loss. Ask your broker to confirm in writing how each policy treats a mistake made by an automated system.

Is there special insurance for AI? Some insurers add AI-specific wording, either extending cover or restricting it, and the details vary by insurer and change over time. Ask your broker what is currently available, then compare it with the gaps you found, rather than buying a product before you know the gap.

Who is responsible if the model provider's error causes the loss? That depends on your contracts with the provider and with your customers. Providers commonly limit their own liability, which can leave the loss with you, so read the limits and ask your broker whether your policies respond to a provider failure.

Do we need to tell our insurer that we use an AI agent? Ask your broker, because applications and renewals typically ask about operations and material changes. Not disclosing something the insurer asked about can cause problems with a claim, so get the disclosure guidance in writing and involve your lawyer if the answer is unclear.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.