Skip to content
be-human.ai
← Back to the journal

Testing & reliability

Measure AI agent cost per completed task, not per message

Cost per completed task divides the total operating cost of a workflow by the number of outcomes that meet your acceptance criteria.

Be Human editorial illustration about time and thoughtful work.
Be Human editorial illustration about time and thoughtful work.

Cost per completed task divides the total operating cost of a workflow by the number of outcomes that meet your acceptance criteria. It is more useful than model cost per message when a task involves retrieval, tool calls, retries and human review. Keep the scope of included costs explicit.

Define the denominator first

Imagine an assistant preparing meeting briefs. A generated brief is not necessarily an accepted brief. If the manager must rewrite it completely, counting it as a successful task makes the economics look better than the experience.

Choose an acceptance rule and use it consistently. Track rejected and partially useful outputs separately so they remain visible.

Build a simple cost ledger

  • Model usage across all calls in the task.
  • Tool, storage and infrastructure costs attributable to the workflow.
  • Human review and correction time.
  • Maintenance and incident work allocated over a stated period.
  • Accepted outcomes during that same period.

Avoid combining monthly fixed costs with a single day's output without explaining the allocation. Compare like periods and similar task complexity.

Work through a hypothetical example

Suppose a test period costs $120 in model and tool usage, plus $180 in review time, and produces 60 accepted briefs. The illustrative cost is $5 per accepted brief before any excluded overhead. If only 30 briefs are accepted, the same spend becomes $10 each.

These numbers are an example, not a Be Human result or a forecast. Their purpose is to show why acceptance rate changes the calculation.

Optimize the expensive failure

Inspect whether cost comes from long context, repeated retrieval, unnecessary agent loops or difficult review. A cheaper model that doubles correction work may not reduce total cost. Improve the workflow with the largest avoidable burden, then rerun the same acceptance test. Cost and quality belong in the same decision.

Which part of your work could use a little more room?

The free Be Human AI diagnostic takes about three minutes and helps you identify a useful place to begin.Take the free AI diagnostic ↗Or explore managed AI implementation ↗

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.

Less busywork. More room for you.

Bring us the work you wish you had time for.

Find your first AI workflow ↗