AI Agent API Rate Limits: Retry and Backoff Rules That Hold
How to handle API rate limits with an AI agent: which errors to retry, how to back off with jitter, cap attempts, and avoid duplicate writes or lockouts.

An AI agent stays within API rate limits when it treats a rate limit response as a signal to slow down, not as a failure to push through. Retry only the errors that can succeed later, wait longer after each failed attempt with some randomness added, honor any retry guidance the server sends, and cap the total attempts for each task. Make every write safe to repeat before you allow any automatic retry, because a timeout does not tell you whether the first request went through.
Key takeaways
- Retry only errors that can succeed later, such as rate limiting, timeouts and temporary server failures.
- Honor the retry guidance a provider sends, and add randomized jitter to every backoff delay.
- Cap attempts per task and per time window, then stop and hand the task to a person.
- Make every write idempotent before any automatic retry is allowed to touch it.
- Count rate limit responses as an operating metric so limits are reviewed before access is cut off.
Why agent traffic looks different from a person's clicks
A person working in a software portal makes a few requests at a time, then reads the screen. An agent working through a queue can send dozens of requests in a short burst: list the open items, fetch the details for each one, check related records, then write results back. Many providers count usage per API key, per account or per minute, so several agents and scheduled jobs that share one credential can exhaust it together without any single one looking unusual.
Agents also tend to repeat themselves. A planning step asks for one more lookup, a failed call is attempted again, and a second worker starts the same job because nobody recorded that the first one was still running. Rate limits exist to protect the provider's service. The agent's job is to behave like a well-mannered client, and that behavior has to be designed into the integration, not left to the model's judgment in the moment. Limits differ by provider and by plan, so read the provider's documentation for the exact values rather than assuming them.
Read the signals before you retry
Not every error deserves a second attempt, and retrying the wrong ones is how a small problem becomes an access problem. A rate limit response, usually HTTP 429, means the caller asked for too much too fast. Many APIs also send a Retry-After header or a similar hint that says how long to wait. Server errors such as 500, 502, 503 and 504 often reflect temporary trouble on the provider side. Validation errors such as 400 or 422 mean the request itself is wrong, and sending it again with the same content will fail again. Authentication and permission errors, 401 and 403, need special care, because repeated attempts with a bad or expired credential can lock the account.
A timeout is the ambiguous case. The client gave up waiting, but the server may have finished the work. For reads, a retry is usually harmless. For writes, a retry can create a second invoice, booking or message. Treat a timed out write as an unknown outcome and check the target system before doing anything else.
- 429 or 503 with a retry hint: wait at least the stated time, then try again.
- 429 or 503 without a hint: back off exponentially with jitter, as described below.
- Timeout or dropped connection on a read: retry within the attempt cap.
- Timeout or dropped connection on a write: check whether the change exists before any retry.
- 400, 404 or 422: do not retry. Send the task and the payload to a person.
- 401 or 403: stop every call made with that credential and raise an alert.
Retry rules that hold
The rules below belong in the integration layer, the code that sits between the agent and the provider. The agent can decide what work to do. The harness decides when and how often a call is sent.
1. Set a concurrency limit for each provider and credential. Start low, and only raise it after observing responses over a normal working period. 2. Use exponential backoff. Each wait should be longer than the previous one, up to a ceiling so a task never stalls for hours without anyone knowing. 3. Add jitter. Randomize each wait within a range so that many workers that failed together do not all retry at the same moment. 4. Cap attempts per call. A small fixed number is usually enough. If a call fails after the cap, the task moves to a review state with the error recorded. 5. Set a total time budget per task. Even with few attempts, a task should not hold a worker open indefinitely. 6. Log every attempt with the task identifier, the response code, the wait time and the outcome, so a person can see the pattern later.
The model should not decide on its own to call a failing tool again. Left to improvise, an agent may retry the same request with slightly different wording, which looks to the provider like a new request and doubles the pressure. Fixed retry rules in code are easier to test and review than instructions written in a prompt.
A retry policy you cannot explain to the operations lead is not finished.
Keep retries from duplicating work
Retries and duplicates are the same problem seen from two sides. When a write succeeds but the response is lost, the next attempt creates a second record. The standard fix is an idempotency key: a unique identifier sent with the request so the provider can recognize a repeat and return the original result instead of acting twice. Stripe's documentation describes this pattern for its API, and the same principle applies to most business systems that support it.
Where the provider offers idempotency keys, generate the key from the task identifier, not from a random value created on each attempt. A random key defeats the purpose, because every retry looks new. Where the provider offers no key, read back the target record before retrying a write, and only write again if the record is missing. The full design of this check is covered in Idempotency for AI agents: prevent duplicate actions, which is worth reading before any agent is allowed to create invoices, bookings or messages automatically.
Queue, cap and alert before access is cut off
A queue is the simplest protection against bursts. Instead of sending everything as soon as it arrives, work items wait in line and a worker takes them at the rate the provider tolerates. Low priority work, such as bulk reporting or archive syncs, can be scheduled for quiet hours, while time sensitive requests keep a separate lane. Each credential should have its own queue so one busy job cannot starve the others.
Monitoring turns these rules into something you can review. OpenTelemetry describes telemetry in three forms: traces that follow one task across steps, metrics that count events over time, and logs that record the details. For a rate-limited agent, useful signals include the count of 429 responses per hour, the number of tasks sent to review after exhausting retries, the number of authentication failures, and the queue depth at the start of each working day. An alert should go to a named person when any of these moves outside its normal range. The mechanics of scheduled runs and the alerts that reach a person are described in Scheduled AI agent tasks: runs, retries and alerts that reach a person.
An illustrative scenario and how Be Human handles it
Illustrative scenario: a company wants an agent to sync its open support tickets from a helpdesk into a shared operations board every hour. On the first run, the agent requests the full ticket history at once. The helpdesk starts returning rate limit responses, the agent retries immediately, and the sync fails late in the afternoon with half the tickets copied and no record of which ones. A corrected design sends the first full import through the queue at a slow, steady pace, stores a cursor after each page, pauses on any 429 response, and resumes from the cursor after the stated wait. Hourly runs afterward only fetch changes since the last cursor, and any run that exhausts its retries posts an alert to the operations channel with the cursor position.
Be Human designs this kind of integration as part of a managed operation. The first engagement defines the role, the tools it needs, the rate and retry settings for each credential, and the point where a person must take over. Ongoing operations include monitoring the response patterns, reviewing permissions when a provider changes its limits, and responding to incidents, so a limit problem is caught as a trend before the credential is blocked. If you are not sure which of your recurring tasks would be the right first one to hand to an agent, the free Be Human AI diagnostic can help you find the first task worth handing to an agent.
Frequently asked questions
How many times should an AI agent retry a failed API call? There is no universal number, because the right cap depends on the provider, the task and how costly a delay is. A small fixed number of attempts, with growing waits between them, is a sound starting point for most business integrations. When the cap is reached, the task should stop and go to a review state with the error recorded, not keep trying in the background.
Should an agent retry when it receives a 429 response? Yes, but only after waiting. A 429 response means the request is valid and the provider wants the caller to slow down, so the retry is appropriate once the stated wait has passed or, if no wait is given, after an exponential delay with jitter. Retrying immediately usually produces more 429 responses and can make the limit tighter.
Is it safe for an agent to retry a write, such as creating an invoice? It is safe only when the write is idempotent, meaning a repeated request cannot create a second record. Use the provider's idempotency key if one exists, with a key tied to the task rather than generated fresh for each attempt. If there is no such key, read back the target record first and write again only when it does not exist.
What should happen when an agent keeps hitting rate limits every day? A daily pattern means the integration is asking for more than the credential allows, so the fix is structural. Reduce the volume with incremental syncs, spread low priority work into quiet hours, split workloads across credentials where the provider's terms allow it, or ask the provider about a higher tier. Review the pattern with the operations owner before the credential is restricted or suspended.
Your next step
See where an AI agent fits in your company first.
Take the free Be Human AI diagnostic. A few questions about your work show which task to hand to an agent first, what it needs to know and where a person approves.
Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.


