Skip to content
be-human.ai
← Back to the journal

Testing & reliability

Token Budgets for AI Agents: Caps That Halt Runaway Loops

Set per-run token and step caps for AI agents, define what happens at each limit, and stop looping agents before cost or repeated actions grow.

Be Human typographic cover with the title: Token Budgets for AI Agents: Caps That Halt Runaway Loops
Conceptual illustration created with AI · Be Human

An AI agent token budget is a hard limit on how many tokens, model calls and tool actions a single run may use before it must stop. Set it per run rather than only per day or per account, and attach a defined response to each limit: pause, checkpoint, or hand the case to a person. A budget without a stop action is only a report about a loop that has already finished.

Key takeaways

  • Cap every run on four axes: total tokens, model turns and tool calls, elapsed time, and repeated identical calls.
  • A cap must halt the run and preserve its state, not just send a notification.
  • Set first limits from the observed cost of completed tasks, then leave room for legitimate long cases.
  • Track cost per completed task alongside raw spend, so a loop shows up as a ratio problem and not only as a larger bill.
  • Treat every cap hit as a case to review, because it points to a task design gap, an unclear instruction, or a tool problem.

Why a daily spend limit does not protect you

A daily or monthly spend limit tells you the bill is too high after the money is spent. An agent that repeats a lookup, rereads the same document, or retries a failed step does not look dangerous on any single turn. Each turn usually resends a growing context, so cost rises faster than the step count suggests. The OWASP Top 10 for LLM Applications lists unbounded consumption as its own risk category, which is a useful reminder that cost control belongs in the agent's design, not only in the finance report.

Per-run caps move the decision earlier. They answer a narrower and more useful question: how much is this one piece of work allowed to consume before a person looks at it? That question stays answerable even when the agent is behaving well overall.

Choose the four caps and what each one protects

Each cap guards against a different failure, so none of them replaces the others.

  • Token cap: total input and output tokens across all model calls in one run. It limits cost and catches context that keeps growing.
  • Step cap: maximum model turns and tool calls. It stops an agent that plans, acts, and replans without converging.
  • Time cap: maximum elapsed wall-clock time. It catches waiting on a slow or stalled tool, which can hold a run open without producing tokens.
  • Repeat cap: the same tool called with the same arguments more than a small, predefined number of times. It catches the loop pattern directly.

Record these values in your run telemetry. The OpenTelemetry semantic conventions for generative AI describe attributes for model and token usage, so token counts can be read per run rather than inferred from an invoice at the end of the month.

How to set the first limits

Use this sequence for each task type.

  • Pull a sample of recent runs that finished correctly, and record tokens, steps, tool calls and elapsed time for each one.
  • Group the runs by task type. A refund question and a monthly reconciliation should never share one cap.
  • Start from the upper end of normal completed runs, then add a margin you can explain to a colleague in one sentence.
  • Set a warning level below the hard cap, so a run can be watched before it is stopped.
  • Replay past cases against the new caps in test mode to see how often legitimate work would be interrupted.
  • Review the first week of real cap hits with the person who owns the task, then adjust once, deliberately.

Financial operations teams usually measure cost against a business output rather than a raw total. The FinOps Foundation's unit economics guidance follows that logic, and it is the same logic behind measuring cost per completed task. A cap defined this way is easier to defend, because it is tied to what the work was supposed to produce.

Illustrative scenario: an agent answers questions about supplier invoices by searching a document store. A normal answer takes a few searches and one reply. One afternoon a search returns an empty result because of a permission error, and the agent keeps searching with slightly different phrasing. A repeat cap on the search tool stops the run once the same call fails to change anything, and the case goes to a staff member with the question and the searches already tried.

What should happen when a cap is hit

A cap without a defined response only moves the problem to a later review. Decide the response before the cap goes live.

  • Warning level reached: log the run and continue only if the remaining steps are read-only.
  • Token or step cap reached: stop before the next model call, write a checkpoint of the state so far, and mark the run as incomplete.
  • Repeat cap reached: stop and do not retry automatically, because the same input will probably produce the same loop.
  • Time cap reached: end the wait, release any locks, and queue the case for a person or a later attempt.
  • Any cap reached during a write action: check which actions completed before the stop. Idempotent tool design keeps a later retry from repeating work that already happened.

Anthropic's engineering guidance on building effective agents makes a related point: autonomous loops need explicit stopping conditions, and those conditions should be tested before anyone depends on them. The design of stop conditions deserves its own attention, and the guide on stop conditions that actually work covers that in more depth.

Keep budget limits separate from loop detection

Budget caps are blunt. They stop any run that costs too much, whether or not the agent is stuck, and they fire only after cost has already grown. Loop detection looks at the pattern instead: consecutive steps that add no new information, the same tool called with identical inputs, or the same plan repeated with no change in state. Use both. The budget is the backstop that always holds, and the pattern check gives an earlier and clearer reason to stop.

Store the reason for every stop in a form the operator can read without opening the logs. A halted run should explain itself in one line, such as which cap was reached, which tool repeated, and what the last useful result was. For aggregate alerts across all runs, spend guardrails for runaway agent usage describe how to catch a looping agent before the total becomes a problem.

How Be Human's managed approach handles caps

In a Be Human engagement, caps are written into the agent's operating definition beside its memory, tools, approval boundaries and success criterion, so the limits are visible to the business owner and not buried in configuration. Managed operations watch cap hits as part of monitoring, review whether the limits still fit the current mix of tasks during permission and quality reviews, and change caps through a controlled process when a model, a tool or a workflow changes. The goal is not the lowest possible spend. The goal is that every stop is explainable and every limit is a deliberate business decision.

Review every cap hit as evidence

A cap hit is information, and it is most useful when someone reads it on a schedule. Once a week, go through the stopped runs and sort them into three groups. Some tasks needed more room, and only those should move the limit. Others failed because the instruction was unclear, so the agent needs a clearer rule. The rest failed because a tool returned something the agent could not use, so the tool or its error message needs work. Raising a cap to hide the second or third group only delays the real fix. Change limits in small steps, record each change, and check at the next review whether it helped. A repeatable routine for this is described in the weekly maintenance checklist for managed AI agents.

If you are unsure which of your workflows should get a token budget first, the free Be Human AI diagnostic will help you find the first task worth handing to an agent.

Frequently asked questions

What is an AI agent token budget? A token budget is the maximum number of input and output tokens that one agent run may consume across all of its model calls. It works best alongside step, time and repeat limits, and with a defined action for each limit. Setting it per run keeps one stuck case from hiding inside a healthy monthly total.

How do I set the first token cap for an agent? Measure a sample of recently completed runs for the same task type, then start from the upper end of normal cases plus a margin you can justify. Run the cap in test mode against replayed cases before it can stop live work. Review the first week of real hits and adjust once, with the reason written down.

What should an agent do when it reaches its cap? It should stop before the next model call, save a checkpoint of what it has done and what remains, and mark the run as incomplete. If the cause is a repeated call with the same input, it should not retry automatically. The case should go to a person with enough context to choose the next step.

Is a token budget enough to stop a loop? No. A budget stops any expensive run, including legitimate ones, and it reacts only after cost has already grown. Pair it with a pattern check that looks for repeated calls and steps that add no new information, so loops are caught early and explained clearly.

Your next step

See where an AI agent fits in your company first.

Take the free Be Human AI diagnostic. A few questions about your work show which task to hand to an agent first, what it needs to know and where a person approves.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.

Less busywork. More room for you.

Bring us the work you wish you had time for.

Book a call to try a personalized AI agent free for 7 days ↗Or take the free AI diagnostic →