AI Agent SLA: What to Define in Writing, and What Not To

An AI agent SLA is a written set of service commitments covering what a provider can actually control: whether the agent and its connections are available, how quickly problems are acknowledged, what the agent may do, what gets logged, and how incidents are handled. It should not promise that every answer will be correct, because nobody can honestly guarantee that. Write down the service, the boundaries and the evidence, and treat output quality as something both sides measure on a schedule.

Key takeaways

• Commit to what the provider controls: availability of the service and its connections, incident response, change control and logging.
• Do not promise that every output is correct. Define how quality is measured, how often, and what happens when results drop.
• Treat approval boundaries and permissions as service terms with named owners, not as settings someone changed once.
• Define what an incident means for an agent, including quiet failures such as an expired connection or stale memory.
• Include data, memory and exit terms so the agent's configuration and context remain usable by your company.

Why does a normal uptime promise fall short for an agent?

A conventional software SLA answers one question: is the service responding? For an agent, that is the least interesting question. An agent can be fully online and still be useless, because its calendar connection expired overnight, its memory holds an instruction that was replaced last month, or the model behind it changed behavior after an update.

An agent is several parts working together: a model, a harness around it, memory, tools, channels and permissions. Each part can fail on its own, and each has a different owner. Some of those parts, such as the underlying model service, sit with a third party that neither you nor your provider controls. A useful agreement does not pretend otherwise. It splits the agent into the parts, says who is responsible for each, and states plainly what is outside anyone's control and how that will be reported.

There is a second reason the simple promise fails. Business owners often read an SLA as a guarantee of results. If a document says the agent will "handle support requests correctly," you have written a hope, not a commitment, and you have created a dispute waiting to happen. The goal is the opposite: commitments narrow enough that both sides can check whether they were kept.

Which service commitments are worth writing down?

Start with commitments that can be observed and that the provider can influence. These are the ones that belong in writing:

• Scope of availability: which components are covered (the channel the agent speaks on, the harness that runs it, each tool connection), and which are excluded, such as third-party outages, with a promise to report them.
• Acknowledgement and response times: how fast an incident is acknowledged and how fast work begins, by severity, and during which hours. Use the hours your team actually works.
• Task time expectations: an expected range for each task type, framed as a target you review together, not a guarantee. See the latency budget guide if you need a way to set these.
• Logging and retention: what each run records, how long records are kept, and who can read them.
• Change control: advance notice before changes to the model, instructions, tools or permissions, and the ability to roll a change back.
• Evaluation cadence: how often the agent is tested against an agreed set of cases, and how results are shared.
• Reporting: a regular summary of incidents, escalations, approvals and open corrective actions.
• Data and exit: where memory and configuration live, who owns them, how they are exported, and how they are deleted.

Notice what is missing: any promise about the content of a particular answer. Everything on this list is a promise about service, evidence or process. That is deliberate, and it is what makes the list enforceable.

The logging item deserves extra care, because without records no other clause can be verified. If you are unsure what to ask for, what an agent should log lays out a practical minimum, and the [OpenTelemetry observability primer](https://opentelemetry.io/docs/concepts/observability-primer/) explains the general idea of traces, metrics and logs that those records build on.

How do you handle quality without guaranteeing it?

Quality is where most agent agreements go wrong, in one of two directions. Either the provider refuses to say anything about it, which leaves you with no recourse, or the document promises accuracy it cannot deliver. There is a workable middle: guarantee the process, not the outcome.

In practice that means three written elements.

• A shared test set. You and the provider agree on a set of real examples from your own work, with the result you consider acceptable for each. Building one is a manageable job, and a golden test set explains how.
• A cadence and a threshold. The provider runs the test set on a stated schedule and after any significant change. You both agree in advance what level of results triggers a review. Choose that level together, from your own tolerance for error, not from a number borrowed from a brochure.
• A corrective action rule. When results fall below the threshold, the agreement says what happens: the provider investigates, reports the cause, proposes a fix, and re-runs the test before the change goes live.

This gives you something stronger than a vague accuracy promise. It gives you a visible signal, a trigger and an obligation to respond. For background on why evaluation of agents is harder than evaluation of a single model response, Anthropic's piece on [evaluating AI agents](https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents) is a useful read, and the [NIST AI Risk Management Framework](https://www.nist.gov/itl/ai-risk-management-framework) offers a vocabulary for measuring and managing risk that you can borrow in your own wording.

Pair this with a clear definition of done for each task. If the agent reports a task as finished, the agreement can require evidence: the record that was updated, the message that was drafted, the source it relied on. That keeps "done" from meaning "the agent said so."

Which approval and permission terms belong in the agreement?

Approval boundaries are the part of an agent engagement that protects you most, and they are usually the least documented. They should appear in the agreement as terms, with owners, not as a setting someone configured once and forgot.

Write three lists for each role the agent plays:

• Actions the agent may take alone, such as reading a document or drafting a reply.
• Actions that need a named human to approve first, such as sending a message to a customer or changing a record.
• Actions the agent must never take, however it is asked.

Then add the operational details people forget. Who are the approvers by name or by role? What happens when the approver is away: does the work wait, move to a deputy, or escalate? How long may a request sit before someone is reminded? Where an agent should stop and ask goes deeper into drawing these lines.

Permissions need the same treatment. The agreement should say that access starts at the smallest useful scope, that each connection is documented, and that permissions are reviewed on a schedule with a named reviewer on your side. The [OWASP authorization guidance](https://cheatsheetseries.owasp.org/cheatsheets/Authorization_Cheat_Sheet.html) is a solid reference for the principles behind this, such as deny by default and least privilege. A clause that says "access will be reviewed regularly" is weak; a clause that names the reviewer and the interval is one you can check.

What counts as an incident, and what happens next?

For a normal application, an incident is an outage. For an agent, the failures that hurt most are often quiet. Define incident categories broadly enough to catch them:

• Loud failures: the agent is unreachable, a channel is down, a run fails visibly.
• Quiet failures: a connection has expired and work silently stopped, a duplicate action was taken, the agent relied on outdated context, or an output went to the wrong recipient.
• Boundary failures: the agent attempted or completed an action outside its approved scope, or exposed information to someone who should not see it.

Once the categories exist, write a short response plan into the agreement. A step-by-step version looks like this:

• Detect: say how each category is noticed, by monitoring, by the agent's own reporting, or by a person on your team.
• Contain: state what the agent does while the problem is open. Usually it pauses the affected action, queues the work and tells a human. See what an agent should do when a connection expires for one concrete case.
• Notify: name who is told, through which channel, and within what time for each severity.
• Hand over: when a person must decide, the agent passes along the decision with context, not a pile of logs. Human escalation covers what a good handoff contains.
• Recover: describe how interrupted work is resumed or reversed, and how duplicates are avoided.
• Review: after serious incidents, the provider writes a short account of cause, impact and the change made, and you decide whether the change is acceptable.

This plan is worth more than any response-time number, because it tells everyone what to do before the pressure starts.

What does this look like in an illustrative scenario?

The following is illustrative, not a description of a real engagement. Imagine a small distribution company with an agent that drafts customer follow-ups and proposes updates to its customer records. The owner wants a short service agreement and agrees these clauses with the provider.

The agent drafts follow-ups on its own but never sends them; a named account manager approves each one, and a deputy covers when that person is away. Record changes are proposed, never written, until a human accepts them. The connection to the mailbox and the customer system is monitored, and an expired connection triggers a pause and a notice to the owner during working hours. Every run records the sources used, the draft produced and the approval decision. Each month the provider runs the agreed test set of past customer conversations, shares the results, and opens a corrective action if the results fall below the level the owner set. Permissions are reviewed on a schedule by the owner and the provider together.

No clause says the drafts will be perfect. Each one says what will be done, by whom, and what evidence will show it. If something goes wrong, the owner can see where, and can point to the clause that covers the next step.

How does a managed approach put this into practice?

Be Human designs, implements and operates agents with identity, memory, channels, tools and permissions kept under human control. A first engagement defines one useful role, the memory and tools it needs, the approval boundaries and an observable success criterion. Managed operations can then include monitoring, recurring evaluations, integration maintenance, permission review, incident response and controlled improvements, which line up with the commitments above. What gets written down is the scope of that work and the evidence it produces, not a guarantee of outcomes. The service is remote, in English and Portuguese. If you are comparing providers, questions to ask an AI agent vendor can help you test whether their terms say these things.

Frequently asked questions

Should an AI agent SLA include accuracy targets? Be careful with any target the provider cannot control. A better approach is to agree on a test set drawn from your own work, a schedule for running it, and a corrective action rule when results drop below a level you choose. That makes quality visible and actionable without promising a result that no one can guarantee.

Who is responsible when the underlying model provider has an outage? Usually neither you nor your agent provider controls that service, so the agreement should say so openly. Ask instead for prompt notice, a stated fallback behavior such as pausing and queuing work, and a report afterward. The commitment is about how the outage is handled, not whether it can occur.

How often should the agreement be reviewed? Review it whenever the agent's role, tools or permissions change, and on a regular schedule even when nothing seems to have changed. Agents drift as documents, connections and business rules change around them. A recurring review keeps the boundaries, owners and test set current.

Is an AI agent SLA legally binding? That depends on how it is incorporated into your contract and on the law that applies, so have qualified counsel review it. Whatever its legal weight, writing the commitments in plain language helps both sides agree on what is expected. The more specific and checkable each clause is, the more useful it will be in practice.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.