Model Outage Fallback for AI Agents: A Plan for Queues, Routing and Staff

When a model provider has an outage, a business agent should keep accepting requests, store them in a durable queue, pause any action it cannot safely repeat, and send urgent items to named staff with enough context to act. The plan only works if it was written before the outage, with clear owners, time limits and rules for what the agent may still do in a degraded mode.

Key takeaways

• Decide in advance which tasks can be held, which can be finished with a simpler method, and which must stop until the provider recovers.
• Give every queued request a stable identifier so a retry after recovery never creates a second quote, booking or message.
• Map each urgent category to a named person with a backup, reachable through a channel that does not depend on the failed provider.
• Keep the agent in draft or read-only mode after recovery until recent outputs pass your spot checks.
• Log every switch into and out of fallback, then update the plan after each outage, since the first real incident usually exposes a gap.

What counts as a model outage for a business agent?

A complete outage is the easy case, because every request fails and everyone notices. The harder failures are partial. Requests time out only during busy hours, responses arrive too slowly for a live chat, the provider starts rejecting calls because your account reached a limit, or the model keeps answering while its output quality drops. Your fallback plan has to cover these conditions, because a partly working agent can cause more damage than one that has cleanly stopped.

Give each condition a name, a detection signal and a response. A practical starting set looks like this:

• Hard outage: calls fail or return errors for every request.
• Latency degradation: calls succeed but take longer than the workflow can tolerate.
• Throttling: the provider or your gateway rejects calls because of volume.
• Quality degradation: outputs are well formed but fail your checks against known good answers.
• Tool failure: the model responds, but the booking system, inbox or document store the agent depends on does not.

Throttling deserves its own rules, because aggressive retries can make it worse. The API rate limits and AI agents guide explains how to avoid those surprises before they happen.

How do you write a fallback plan before you need it?

A plan written during an incident usually gets skipped, so write it while operations are calm and keep it short enough to follow under pressure. Work through these steps with the people who own the business process, not only the technical team:

• List every task the agent performs and mark each one as hold, finish in reduced mode, or stop.
• Write the trigger that switches the agent into fallback, such as consecutive failed calls or responses that exceed a time limit you set.
• Name the person who can declare the outage over, and the person who receives urgent items while it lasts.
• Set a maximum queue age for each task type. When an item passes that age, it goes to a human instead of waiting.
• Define what the customer or colleague sees: an honest holding message that states what happens next, never a guess presented as an answer.
• Specify the recovery sequence: replay order, duplicate checks, a sample review, and the point at which normal authority returns.
• Store the plan somewhere the operations lead can open without the agent, and rehearse it once before you need it.

A plan that fits on a few pages is easier to trust than a long document nobody reads during an incident.

How do you queue work without repeating it?

During an outage the agent should keep taking intake but avoid irreversible actions. Each incoming request goes into a durable queue with a stable identifier, the time received, the source channel and the task type. The queue is only safe if the agent also knows what it has already done. A customer who sends the same booking request three times while the agent is down should not produce three bookings when service returns.

The safe pattern is idempotency at the point of action. Every write to a booking system, invoice, ticket or outgoing message should carry a key derived from the original request, so a second attempt returns the first result instead of creating a new record. Payment platforms document this pattern in detail, and their guidance is worth reading even if your stack is different.

Replay design matters as much as the key. Releasing a backlog all at once can trigger throttling, duplicate actions and a flood of stale replies. Replay in age order, cap the rate, and re-check any item whose underlying facts have changed. The retry policy for AI agents covers how to keep the agent from repeating the same mistake in this phase.

How should urgent items reach named staff?

Some work cannot wait for a model. A client whose deposit is due today, a supplier waiting at the dock, or a customer with a complaint about a safety issue should never sit silently in a queue. Define a short list of urgent categories in plain language and map each one to a named person with a backup. The escalation channel should be email, phone or an internal chat tool that stays up independently of the failed provider.

When the agent escalates, it should send the decision, not the mess. That means the request in one or two sentences, what it already checked, what it could not check, the deadline, and the one choice it needs from the person. A raw transcript forces the person to reconstruct the case. The guide to human escalation for AI agents describes this format in more detail.

Use a simple decision guide when a queued item arrives during fallback:

• If the item involves money, a legal commitment or a safety decision, send it to a named person immediately.
• If the item is time sensitive but routine, hold it and notify the task owner as the queue age limit approaches.
• If the item needs only information the agent can gather from approved sources, hold it and answer after the model returns.
• If a reduced method can complete the item safely, use that method, record it, and flag it for review.

During fallback the agent should not improvise approvals. It cannot grant refunds, change agreed prices or confirm regulated information unless a named person has granted that authority for the period, in writing.

Illustrative scenario: a quote request during a provider outage

The following scenario is illustrative and does not describe a specific company. A small service business uses an agent to answer website quote requests, check availability and propose site visits. At midday the model provider begins timing out. The agent detects consecutive failed calls, switches to fallback, and tells each visitor that the request is saved and that a coordinator will reply within the window the business has published.

The agent stores each request with its form data and timestamp, and it does not propose visit times while the provider is down. Requests tagged as same day jobs go to the operations coordinator by text message through a channel that does not depend on the model. When the provider recovers, the agent replays the queue in age order, checks the scheduling system for existing bookings before creating any, and sends a draft summary to the coordinator for review before it resumes direct replies. The coordinator approves the first batch, and the agent returns to normal authority for that task type only.

How do recovery and review keep the plan honest?

Recovery is where many teams relax too early. Before the agent returns to full authority, confirm the state of the queue and the outputs it produced. Use a short checklist:

• The queue is empty, or every remaining item has a stated reason for waiting.
• No duplicate bookings, messages or tickets exist from replayed requests.
• A sample of outputs produced after recovery matches the standard used before the outage.
• Every item sent to a person has a recorded outcome.

Keep a brief incident note with the trigger, the time the agent entered fallback, the number of held items, the escalations sent, and any action a person had to correct by hand. Then update the plan. Most gaps are small, such as a task category nobody listed or an escalation contact who changed roles.

Be Human designs fallback behavior as part of the agent rather than as an afterthought. A first engagement defines one useful role, the memory and tools it needs, its approval boundaries, the fallback behavior for each task, and the named staff who receive escalations. Managed operations can then cover provider monitoring, recurring output checks, permission review and incident response when a fallback is triggered, with controlled changes to the plan as each event teaches something new.

If you are unsure which of your agent's tasks deserves a fallback plan first, the free Be Human AI diagnostic can help you find the first task worth handing to an agent.

Frequently asked questions

Should the agent keep answering customers during a model outage? It can acknowledge the request, explain that it has been saved, and say when a person will follow up, but it should not produce substantive answers it cannot verify. A short, honest holding message is better than a guess delivered with confidence. Anything that commits money, dates or a policy exception waits for a named person.

Can a second model or a cached answer replace the primary during an outage? Sometimes, for narrow tasks where you have tested the alternative on your own examples. A different model behaves differently, so run the same test cases before routing live work there and keep the same approval limits. Cached answers are only safe for facts that have not changed, so each one needs an owner and a review date.

How do you decide when the outage is really over? Do not rely only on the provider's status page. Require several successful calls, confirm that latency is back within your limit, and have the task owner review a sample of outputs. Only then should the agent return to full authority for the affected tasks.

Who should own the fallback plan? A business owner or operations lead should own the decision to enter and leave fallback, a technical owner should maintain the triggers, logs and replay tools, and each escalation category needs a named person. Write these roles down, because during an incident nobody has time to guess. The NIST AI Risk Management Framework is a useful reference for assigning responsibilities around AI risk.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.