AI Agent Rate Limit Handling: Retry, Backoff and Queue Rules

An AI agent handles rate limits well when it treats every throttling response or timeout as a signal to slow down, not a reason to try again immediately. The practical rule set has four parts: classify each error before retrying, retry with exponential backoff and jitter under a hard cap, place non-urgent calls in a queue with a concurrency limit, and stop at a fixed budget and hand the task to a person. Together these rules keep a shared business tool from being flooded and keep the agent's work accurate when the tool is slow or busy.

Key takeaways

• Classify each error before retrying: throttling and temporary failures may be retried, while permission and validation errors should stop the task.
• Use exponential backoff with jitter, and cap both the attempts for one call and the total time for the whole task.
• Queue non-urgent work and set one concurrency limit per tool, so no combination of agent jobs can overwhelm a system.
• Never retry a write action until you know whether the first attempt landed, using an idempotency key or a read of the target record.
• Log every retry, pause and stop with its error group, so a person can see whether the problem is capacity, permissions or bad inputs.

What a rate limit means for an agent

Every business tool limits how much it will accept in a given window. Some limits count requests per minute, others cap simultaneous connections, and some are set by the team that runs an internal system to protect its database. A person working through a screen rarely reaches these limits. An agent can, because it chooses its next call from model output, can fan out across several tools in one task, and can repeat a step without noticing that it already did so.

Limits are also shared. A spreadsheet API, a CRM, an accounting package and a legacy ERP may each serve the agent alongside staff, scheduled jobs and other integrations. A rule that looks safe for the agent alone can still break the day's invoicing run if the agent consumes the whole allowance.

Illustrative scenario: an agent is asked to update several hundred customer records in a CRM after a price change. Without a queue, it sends the whole batch within a minute, receives throttling responses after the first portion, and retries each failed record at once. The CRM stays throttled for the rest of the afternoon, and the sales team cannot log calls. The task itself was correct. The timing was the failure.

The rule set exists to prevent that pattern. The agent should slow down when a system asks it to, wait in line when work is not urgent, and stop when the rules say a person must decide.

Classify each error before you retry

Retrying every failure is the most common mistake. Errors fall into groups that need different handling, and the agent should identify the group before it does anything else.

• Throttling responses, such as HTTP 429: wait the period the service specifies, and if none is given, use the backoff rules described below.
• Temporary failures, such as HTTP 503, gateway timeouts and dropped connections: retry with backoff, because the request may never have reached the system.
• Authorization failures, such as HTTP 401 or 403: do not retry in a loop. The credential or permission has changed, and a person or a recovery process must resolve it.
• Validation failures, such as HTTP 400 or 422: do not resend the same payload. Correct the input or return it to the task owner.
• Conflicts, such as HTTP 409: read the current record before trying again, because another process may have changed it.

Record the group with each task. Over time, the counts show whether the problem is capacity, permissions or input quality, and each of those calls for a different fix. For credential failures specifically, see what an AI agent should do when a connection expires.

Queue non-urgent work instead of sending it all at once

A queue turns a burst of work into a steady flow. The agent does not decide how fast to call a tool. It places work in a line, and a worker releases calls at a rate the owner of that tool has approved.

Use this plan when you first connect an agent to a shared system:

• Ask the owner of each system for its concurrency and request limits, and write them into the tool contract. The agent should not discover them through errors.
• Set one concurrency cap per tool rather than per task, so two agent jobs cannot together exceed what the system can accept.
• Sort the queue by urgency. Customer-facing and time-bound work runs first, while bulk backfills and clean-up run in quiet periods.
• Batch reads and writes where the tool supports batching, since one batched call usually uses less capacity than many single calls.
• Set a maximum age for each queued item. If a task waits beyond its window, return it to a person instead of running an action that may now be stale.
• Define a pause condition in advance, such as a rising share of throttled responses, and let the queue stop itself when that condition is met.

The tool contract is the right place for these limits, because it states what the agent may send and how often.

Back off with jitter, then stop on a hard budget

Exponential backoff means the wait grows after each failed attempt: a short delay first, then a longer one, then longer again. Each pause gives the tool more room to recover instead of being hit again at the moment it is least able to answer. The starting values, the growth factor and the maximum single wait should be set per tool, not copied from a generic example.

Jitter adds a random spread to each wait. Without it, workers that failed together retry together, and the tool receives the same spike again. A small random offset breaks that synchronization.

Backoff alone is not enough. Every task also needs a budget that is defined before the agent starts: a maximum number of attempts for a single call, a maximum total time for the task, a maximum number of calls per tool, and, where a tool charges per request, a limit on the spend it may generate. The agent should not be able to raise these limits on its own.

When a budget is exhausted, the task moves to a stop state. The stop state is a record, not a silent failure. It should list what completed, what did not, the last error group, the idempotency keys in use, and the decision a person must make next. A short checklist helps at this moment:

• Confirm which actions completed and which did not.
• Record the error group and the time of the last attempt.
• Mark any write whose outcome is unknown, and do not repeat it until someone has checked.
• Name the person who decides whether to retry, reassign or cancel.
• Keep the agent's context attached, so the next run does not start from zero.

Make retries safe for write actions

Retrying a read is usually harmless. Retrying a write can create the same invoice, booking or message twice, especially when the first request succeeded and only the response was lost. Writes need a stricter rule: the agent must not retry until it knows whether the first attempt landed.

The usual mechanism is an idempotency key. The agent generates one key for each intended action, stores it with the task, and sends the same key on every retry. The service recognizes the repeat and returns the original result rather than performing the action again. Stripe documents this pattern for its payment API, and the same design can be built into internal tools with a record of completed keys. Where a tool offers no such mechanism, the safe option is to read the target record first and confirm whether the change already exists. The design details are covered in idempotency for AI agents.

How Be Human handles rate limits in managed operations

In Be Human's managed model, retry and queue rules are part of the first engagement, not defaults left to the agent. The team agrees concurrency caps, retry policies and stop conditions with the owner of each connected system, writes them into the tool contract, and sets approval boundaries for any write action. In ongoing operations, retries, queue depth, pauses and stop states are monitored, and recurring reviews check whether a limit needs adjusting. When a rule changes, it goes through a controlled improvement with a record of what changed and why, rather than an edit hidden inside the agent's instructions.

Test the rules before launch and watch them afterward

Test the policy before the agent touches a live system. In a test environment, simulate throttling, or ask the system owner to lower a limit for a short period, then confirm that the agent backs off, queues the remainder, stops at its budget and records the stop correctly. Include the case where a write succeeds but its response is lost. That single test shows whether the idempotency design works.

After launch, record each retry, wait, pause and stop as an event with the task identifier and the tool name. OpenTelemetry's observability concepts give a shared vocabulary for this kind of telemetry, and the detail matters: a retry count without its error group cannot tell you whether the cause is capacity or permissions. For a practical list of what to capture, see what to log for AI agent observability.

Review the rules on a regular schedule, and sooner after any change in the connected systems. A tool that was quiet last season may be busier now, and limits written for a smaller workload may no longer protect it. If you are unsure which of your business tools should be connected first, the free Be Human AI diagnostic can help you find the first task worth handing to an agent.

Frequently asked questions

Should an agent retry immediately after a rate limit error? No. An immediate retry usually receives the same throttling response and adds load to a system that has already asked for room. The agent should wait for the period the service specifies, or use backoff with jitter when none is given, and it should stop once its attempt or time budget is reached.

Is exponential backoff enough to protect a business tool? Backoff is necessary but not sufficient. It spreads retries over time, but it does not stop a burst of new tasks from arriving, so a concurrency cap and a queue are still needed. Pair backoff with per-tool limits set by the system owner and a hard stop for every task.

Can an agent safely retry a write action? Only when it can tell whether the first attempt landed. Use an idempotency key stored with the task, or read the target record before retrying. If neither is available, treat the write as unknown and send it to a person.

When should a rate-limited task go back to a person? Return the task when a budget is exhausted, when an authorization or validation error appears, when a write has an unknown outcome, or when a queued item passes its maximum age. The handoff should state what was completed, what was not, and the decision a person needs to make next.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.