Skip to content
be-human.ai
← Back to the journal

Testing & reliability

Cost Spike Alerts for AI Agents: Catch Runaway Loops Early

Set spend-rate alerts per agent and per task type so a person can pause a looping run before it becomes a budget problem. A practical how-to.

Be Human typographic cover with the title: Cost Spike Alerts for AI Agents: Catch Runaway Loops Early
Conceptual illustration created with AI · Be Human

To set up AI agent cost monitoring alerts, track spend rate over short windows as well as total spend, and set thresholds for each agent and each task type. An alert should fire when a run's cost climbs well above the normal pattern for that task, and a named person should be able to pause the run before its next step. The rest of this guide explains how to choose those levels, what each alert must contain, and how to make sure a stop happens even when nobody answers the first message.

Key takeaways

  • Monthly totals arrive too late to stop a loop, so watch spend rate across short windows.
  • Set alert levels per agent and per task type, because normal cost varies widely between tasks.
  • Give every alert enough context for the recipient to decide quickly: pause, inspect or let it run.
  • Pair each warning with a hard stop that the system enforces without depending on anyone reading the alert.
  • Review baselines whenever the model, prompts or tools change, since the normal pattern moves.

Why monthly budgets fail to stop a looping agent

Monthly budget ceilings are a finance control. They tell you about the damage after it has happened, which is useful for planning and of little use for stopping a run in progress. A looping agent repeats a tool call, rereads the same document, or retries a step that never succeeds. Because each pass may call a model with a growing context, it can consume a large part of a period's allowance in a short time.

Cost in agent systems follows the number of steps and the size of the context at each step, not only the number of requests. A healthy run has a recognizable shape: a few steps, a bounded context and a clear output at the end. A loop has a different shape: the same action again and again, a context that keeps getting longer, and no new result. Those signals show up in the rate and in the run structure before they appear in a monthly total. That is why the design of the alerts matters more than the look of the dashboard.

Separate spend rate from spend total

Two measures do different jobs. Spend total answers whether an agent or task type is affordable over a week or a month, and it belongs to the budget owner. Spend rate answers how fast cost is accumulating right now, measured over a short rolling window, and it belongs to the person who can pause the agent. Alerts on totals catch slow drift. Alerts on rate catch loops. You need both, but a rate alert that lands in a finance inbox is a missed alert.

Observability practice, as described in the OpenTelemetry concepts documentation, treats metrics, logs and traces as complementary signals, and that split fits agent cost well. Log these signals alongside cost:

  • Cost per completed task, the figure that lets you compare work over time
  • Spend rate per agent over a short rolling window
  • Spend rate per task type, so a routine request and a long research summary are never judged against the same level
  • Steps per run and repeated identical tool calls, which often reveal a loop before the cost line does
  • Context size at each step, since a steadily growing context is a common sign of a run that is not making progress

The FinOps Foundation frames unit economics as relating cost to the business output it produces, and that is the right lens here. An agent that spends more on a task is not necessarily failing. An agent that spends more while completing fewer tasks is.

How do you set thresholds per agent and per task type?

A single threshold for a whole agent produces either false alarms or silence. An agent that answers quick questions and also drafts long reports will look like a loop every time it writes a report. Start with the task types the agent actually performs, measure a baseline for each over a period of normal work, and set alert levels above normal variation rather than at a round number someone guessed.

Use this plan:

  • List every task type the agent handles, including its trigger and the tools it calls.
  • Measure a baseline for each task type: typical cost per completed task, typical steps per run and typical run duration.
  • Set a warning level for each task type that fires when the spend rate moves well above its baseline.
  • Set a stop level above the warning, at which the run pauses automatically until a named person releases it.
  • Add a per-agent total ceiling for the week or month, owned by the budget owner, to catch slow drift.
  • Name one owner for each agent and one backup who is allowed to release a pause.
  • Test each rule first with a deliberately repetitive input in a test environment, and confirm that the stop fires and the run state is saved.

Revisit the baselines after the first weeks of live work, because early runs often include setup costs that do not repeat. A threshold built from those early runs will be too loose for normal work and too tight for the first real spike.

What should an alert contain, and who acts on it?

An alert that only says a spend threshold was exceeded forces the recipient to open three systems before deciding anything. Include the agent name, the task type, the current spend rate against its baseline, the steps taken in this run, the last few tool calls, and whether the context has been growing. Those fields turn the alert into a decision. Send it to the channel where the owner already works, and keep raw customer content out of the message itself.

The recipient must be allowed to pause the agent without asking permission first. A pause should be reversible: the run stops, its state is kept, nothing already sent is recalled automatically, and the owner then decides whether to resume, adjust the rule or retire the workflow. When you hand the problem to a person, send the decision and the evidence, not the whole history. The article on human escalation for AI agent failures describes how to structure that message.

Illustrative scenario: an agent drafts replies to supplier questions about invoices. A normal run performs a few lookups and produces one draft. After a change in how an attached file is formatted, the agent's parser fails, and the agent requests the same lookup again after each failure. The spend rate climbs well above the baseline for that task type while completed replies stay flat. The alert reaches the operations lead, with the repeated lookup visible in the run summary. The lead pauses the agent, asks the owner to correct the parser, and resumes only after a test input passes.

Add hard stops that work when no one answers

Alerts notify people, and people are sometimes slow, away or in a meeting. Pair every warning with a limit the system enforces: a maximum number of steps per run, a maximum context size, and a maximum number of retries for the same tool call with the same input. When a limit is reached, the run ends in a saved state with a clear status, not in silent failure. Write the retry rules next to the stop rules, because a retry that ignores its limit is one of the most common ways a loop escapes. The article on spend guardrails for runaway agent usage explains how these caps fit together.

Hard caps are blunt, so set them above the longest legitimate run you have observed, then tighten them as you learn. A cap that interrupts valid work every week will be switched off by frustrated staff, and then you have no cap at all.

In Be Human's managed model, spend-rate rules are part of each agent's operating definition from the first engagement, alongside its memory, tools and approval boundaries. Monitoring watches rate and run-shape signals for each task type. Incident response defines who receives each alert level, who may pause, and how a run is resumed. Permission review confirms that the named owners still hold the access they need and no more. The system stops the spend automatically, and a person decides what happens next. The NIST AI Risk Management Framework treats this kind of defined monitoring and response role as part of managing AI risk.

How do you keep thresholds accurate as the system changes?

Baselines move. A model update can change how many steps a task takes, a longer prompt raises the cost of every call, and a new tool may return more data than the agent needs. If nobody revisits the thresholds, the alerts either fire constantly or stop firing at all. Put threshold review on the same calendar as your other agent maintenance, and rerun the baseline measurement after any change to the model, prompts, tools or task scope. A monthly maintenance routine for AI agents is a practical place to make that check part of normal operations. If you are not sure which task deserves a spend alert first, take the free Be Human AI diagnostic to find the first task worth handing to an agent.

Frequently asked questions

How often should I check AI agent spend? Let the short-window rate alerts do the continuous watching, and review totals and baselines on a regular schedule such as monthly. The review checks whether the alert levels still reflect real work, which is a different question from whether a loop is running right now. Whoever receives an alert should be able to act on it without waiting for a planning meeting.

What is the difference between a cost alert and a hard stop? A cost alert tells a person that spending is moving outside its normal pattern. A hard stop is enforced by the system and pauses or ends a run when a limit is reached, whether or not anyone reads the alert. You need both, because alerts depend on people responding in time.

Should spend alerts go to finance or to operations? Send rate alerts to the operations owner who can pause the agent, and send total-spend reports to the budget owner. Finance can receive copies for visibility, but a rate alert that reaches only a finance inbox will not stop a loop. Name a backup for each agent so that alerts are not missed when the owner is unavailable.

Can I use one threshold for all my agents? You can start with one, but it will create false alarms for agents with varied tasks and silence for agents whose typical runs are small. Set thresholds per agent and per task type once you have a baseline from normal work. Measure again whenever the model, prompts or tools change, so the levels keep matching the work.

Your next step

See where an AI agent fits in your company first.

Take the free Be Human AI diagnostic. A few questions about your work show which task to hand to an agent first, what it needs to know and where a person approves.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.

Less busywork. More room for you.

Bring us the work you wish you had time for.

Book a call to try a personalized AI agent free for 7 days ↗Or take the free AI diagnostic →