Skip to content
be-human.ai
← Back to the journal

Personalized AI infrastructure

Token Budgets and Spend Caps for Powerful AI Agents

Set token budgets and spend caps per agent and task so larger models cannot create unbounded cost. A step-by-step method with limits, alerts and approvals.

Be Human typographic cover with the title: Token Budgets and Spend Caps for Powerful AI Agents
Conceptual illustration created with AI · Be Human

A token budget limits how many tokens an agent may consume on one task, and a spend cap limits how much money an agent, or the team that runs it, may spend over a fixed period. Set both for each agent and each task type, enforce them in the infrastructure around the model rather than in the prompt, and decide ahead of time who may raise them and what the agent does when it reaches the ceiling. Stronger models make this more important because each call costs more and agents make many calls before a person sees any result.

Key takeaways

  • Enforce budgets in the infrastructure around the model, not in the prompt alone.
  • Give each agent a period spend cap and each task type its own token ceiling per run.
  • Set an alert below every hard limit so a named owner sees drift before work stops.
  • Measure cost per completed outcome, so a budget reflects the value of finished work.
  • Decide in advance what happens at the limit: finish partially, downgrade the model, or request approval.

Why unbounded cost is a design problem

An agent does not spend once per request. It reads context, calls a model, calls a tool, reads the result, and often calls the model again. A retry after a failed tool call can resend the full conversation history, and a long document added to the context makes every later call more expensive. When a model is more capable, teams tend to give it longer tasks and more steps, which multiplies the effect. The provider invoice arrives after the work is done, so it cannot stop a run that is already looping.

The OWASP Top 10 for LLM Applications lists unbounded consumption as a risk category, describing uncontrolled inference that can drain resources and inflate cost. The practical response matches what experienced teams already do with any metered service: quotas, rate limits and caps applied at the point where requests are made, with alerts that reach a person.

Budgets become useful when they are tied to outcomes. The FinOps Foundation's unit economics guidance frames cost as value per unit of business output. For an agent, that unit might be a reconciled invoice, a drafted proposal or a resolved support ticket. A cost per unit tells you whether an expensive run was worth it, and it gives you a fixed basis for comparing a new model against the one you already use.

A step-by-step plan for setting limits

Use this sequence for one agent before you extend it to others.

  • List every task the agent performs and mark which ones call a model, which call external tools, and which read long documents.
  • For each task type, record the outcome it produces and the normal cost of a good run, measured over a sample of real runs.
  • Set a per-run token ceiling above the normal range but low enough that a runaway loop stops early.
  • Set a period spend cap for the whole agent, with a separate reserve for urgent work so routine volume cannot block it.
  • Set an alert below each hard ceiling and route it to a named owner who can act on it the same day.
  • Define the action at the limit: finish with a clearly labeled partial result, switch to a smaller model, or stop and request approval.
  • Log every model call with the agent, task, model and token counts, so each limit can be audited later.

Write the limits in one place, such as the agent's operating record, so the people who maintain it can read them without opening the code. A limit that exists only in a configuration file is hard to review and easy to forget.

How do budgets change across models and tasks?

Not every step needs the strongest model. Classification, field extraction and formatting usually run well on smaller models, while judgment steps, such as choosing an exception path or weighing a contract clause, deserve the stronger one. Give each tier its own ceiling. A task that sends most of its calls to a small model can carry a generous total budget, while a step that reaches the large model should have a tighter per-call limit and a logged reason for each escalation.

Context is the second lever. Provider features such as prompt caching can lower the cost of repeated long prefixes, but only when that prefix stays stable, so the agent's standing instructions should not change on every call. Retrieval works for the same reason. Fetching the few passages that answer a question costs less than loading an entire archive into every request. Check the current caching and pricing rules in your provider's documentation before you build a budget that depends on them.

Illustrative scenario: a retry loop that never ends

Illustrative scenario: an agent drafts replies to supplier questions and checks order status through a tool. One afternoon the tool begins returning timeouts. The agent retries, and each retry adds the previous error messages and the full thread to the context, so every attempt is larger than the last. Without a per-run ceiling, the run keeps going until someone notices the bill or the stalled queue. With the controls described above, the run stops at its token ceiling, saves its state to the task record, alerts the named owner and waits. The owner sees a timeout reason and repairs the connection. The budget did its job, and the real defect was in the tool, which a larger budget would have hidden.

When is a higher limit justified?

Raise a limit only when the evidence supports the work it will buy. A sound request includes the observed cost per completed task, the results of recurring evaluations on a fixed sample of real tasks, and a clear description of the new work the increase enables. Run the same evaluation before and after the change, so the comparison rests on measured outcomes rather than impressions. OpenAI's evals guide describes this general approach of testing model behavior against defined criteria, and Anthropic's guidance on evaluating agents makes the same point about grading real results.

Treat every increase as a change request with a named approver, and keep the previous limit in the record. A budget that grows each time a run hits the ceiling is usually hiding a defect, and that is the moment to inspect the trace rather than add headroom. For the mechanics of approving changes, see How to Request and Approve Changes to a Managed AI Agent. For how approval thresholds should scale as agents become more capable, see Human Approval Thresholds for Stronger AI Agents.

How Be Human handles budgets in managed operations

In Be Human's managed operations, token and spend limits are written into the agent's operating context next to its permissions and memory scope, so the agent works within stated ceilings and the surrounding infrastructure enforces them. Monitoring tracks spend and token use per task and per agent, and it raises alerts before a limit is reached. Recurring evaluations check whether a stronger model is worth its cost on each task type. Permission review confirms who may change a limit, and incident response covers runs that hit a ceiling unexpectedly. Who receives those alerts and how quickly they must act is set out in On-Call Rotation for AI Agents.

Start with one budgeted task

Choose the task with the clearest outcome, a stable input and a cost you can already observe, then give it the full set of limits before you add a second task. Budgets are easier to set once you know how the work behaves on a normal day, and a first task that runs well makes the next budget easier to justify. If you are unsure which task should go first, use the free Be Human AI diagnostic to find the first task worth handing to an agent.

Take the free Be Human AI diagnostic to find the first task worth handing to an agent.

Frequently asked questions

Is a provider billing limit enough to control agent cost? No. A provider limit stops the account, usually after the spend has already happened, and it often applies to every workload at once. An agent needs ceilings at the level of the run, the task and the agent, so that one failing process cannot consume the budget meant for everything else. Provider limits are a useful last line of defense rather than the primary control.

Should the spend cap be set per agent or per task? Use both. The task ceiling stops a single run from looping, and the agent cap limits what a whole workflow can spend in a period. Each cap should have an alert below it and an owner who receives that alert and knows what to do.

What should happen when a budget runs out in the middle of a task? Decide this before deployment, not during an incident. The usual options are to finish with a partial result that is clearly labeled as incomplete, to switch to a smaller model for the remaining steps, or to stop and request approval. The right choice depends on whether an incomplete output could cause harm if someone acts on it without checking.

Does a stronger model always need a larger budget? No. A stronger model often completes a task in fewer steps, which can offset its higher cost per call. Compare cost per completed task across models on the same sample before you change any budget, and reserve the largest ceilings for the steps where judgment quality actually changes the outcome.

Your next step

See where an AI agent fits in your company first.

Take the free Be Human AI diagnostic. A few questions about your work show which task to hand to an agent first, what it needs to know and where a person approves.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.

Less busywork. More room for you.

Bring us the work you wish you had time for.

Book a call to try a personalized AI agent free for 7 days ↗Or take the free AI diagnostic →