Skip to content
be-human.ai
← Back to the journal

Harness fundamentals

Agent Harness Checklist: Loops, Tools and Stop Conditions

A practical agent harness checklist for loops, tool access and stop conditions, written for owners who must review an agent before it goes live.

Be Human typographic cover with the title: Agent Harness Checklist: Loops, Tools and Stop Conditions
Conceptual illustration created with AI · Be Human

An agent harness checklist is a short review list for the code and configuration that wrap a model: the loop that lets it take repeated steps, the tools it can call, the permissions attached to each tool, and the stop conditions that send work back to a person. A company can use it before launch to decide whether an agent is ready for real work. The model matters, but most operational risk sits in the wrapper, so review the wrapper directly.

Key takeaways

  • Review the harness around the model: loop, tools, permissions, stop rules and logs.
  • Every loop needs hard limits on steps and time, plus a clean exit path.
  • Give each tool the smallest permission that finishes its job, with a clear name and input schema.
  • Stop conditions should cover errors, low confidence, policy edges and spent budgets, not only success.
  • Accept an agent only when someone can replay a run and see why it acted.

What the harness does and why review it separately

The model reads text and proposes an action. The harness decides what happens next: whether the action runs, what data the model sees on the next step, how many attempts are allowed, and who is notified when something fails. Teams often review the prompt and the model choice, then skip the harness, even though the harness determines most of the behavior that touches customers and records. For a plain definition of the layer, see What is an AI harness? A practical business guide.

Separate review matters because each part fails in a different way. A capable model inside a loop with no ceiling can keep calling a search tool long after the answer is clear. A well-behaved loop with a tool that has write access to a live system can change records nobody meant to change. Review the parts one at a time, then check how they interact, because a safe loop and a safe tool can still produce an unsafe combination.

A useful test is to ask whether a person could stop the agent in the middle of a task, explain what it had already done, and resume or reverse that work without guessing. If the answer depends on reading the prompt, the harness is not yet reviewable.

Checklist part one: the loop

The loop is the cycle of reading context, choosing an action, running it, and reading the result. The checklist below asks whether each loop has clear edges.

  • The goal is written in one sentence, with a success test a non-engineer can read.
  • Each run has a maximum step count and a maximum elapsed time, both set before launch.
  • The model receives only the context needed for the current step, not the full history by default.
  • Every step records its input, the chosen action and the returned result.
  • Failed tool calls are retried a fixed number of times, then the run moves to a stop condition.
  • A running task can be cancelled, and any partial work is marked as partial rather than complete.

Budgets are the most common gap. Teams often set a spending limit at the account level and nothing at the run level, so one stuck task can consume the allowance meant for many tasks. Step and time limits belong inside the harness, close to the loop itself, so they apply even when nobody is watching. The details are covered in Runaway AI Agent Loops: Token and Step Budgets With Alerts.

Checklist part two: tools and permissions

Tools are the actions the agent can take outside the model, such as reading an order, drafting a reply or updating a ticket. The model can only request them; the harness decides whether the request runs. Anthropic's engineering guidance on building effective agents favors simple, well-defined patterns over elaborate ones, and the OWASP Top 10 for LLM Applications names excessive agency, meaning an agent with more capability than its task needs, as a risk category to design against.

  • Every tool has a plain name, a one-sentence description and a defined input schema.
  • Read tools and write tools are separate, and write tools stay disabled until a reviewer enables them.
  • Each tool has a named owner who approves changes to its scope.
  • Actions that move money, message a customer or delete data require a human approval gate.
  • Credentials are scoped to a single tool and can be revoked without touching the other tools.
  • Tool outputs are checked for expected fields before the model reads them.

Permission design is easier to judge when you can see which tools a run was allowed to use and which ones it actually called. Where to place approval gates in the flow is covered in Human Approval Gates for AI Agents: Where to Place Them in a Harness.

Checklist part three: stop conditions

A stop condition is a rule that ends the loop and hands the work to a person or to a safe state. A harness without stop conditions relies on the model deciding when it is finished, which works for simple tasks and fails when the model is unsure or when the input is unusual. The list below covers the cases that most often need an explicit rule.

  • The success test passes, and the run records which evidence satisfied it.
  • The same tool error repeats in a row beyond a set limit, which triggers escalation.
  • The model reports low confidence before an action that changes a record.
  • The request falls into a restricted category, such as a refund above an approved limit or a legal question.
  • The step or time budget is spent, and the owner receives an alert with the run summary.
  • Retrieved content contains text that asks the agent to ignore its rules, and the run stops for review.

The last item addresses prompt injection. Instructions hidden inside a document, an email or a web page can look like commands to a model, so the harness should treat retrieved content as data and never as a source of new permissions. Stop conditions act as the practical backstop when a model is manipulated, because they limit what a manipulated run can do before a person sees it. More on writing these rules is in Stop Conditions for AI Agent Loops: When to Halt and Escalate.

Checklist part four: logs and evidence

The harness should produce a record that a person can read after the fact. The OpenTelemetry observability primer describes traces, metrics and logs as complementary signals. That maps well to agent runs: a trace shows the sequence of steps, metrics show budget use and stop reasons across many runs, and logs hold the details of one failure.

  • Each run has an identifier, start and end times, and a stop reason taken from a fixed list.
  • Each tool call is recorded with its inputs, status and a trimmed result.
  • The prompt version, tool schema version and model name are attached to every run.
  • Personal data in logs is minimized or redacted according to your data rules.
  • Stop reasons are reviewed on a schedule, so a category that starts to rise gets attention early.

Without these fields, a bad run becomes an argument about what happened. With them, the review takes minutes and the fix can target the cause. Logging too little hides failures, while logging everything creates privacy and cost problems, so decide the fields deliberately.

Illustrative scenario: a returns request agent

The following scenario is illustrative and does not describe a real client. A returns agent reads a customer message, looks up the order, and drafts a reply for a staff member to send. Its loop has a step limit, and its lookup tool is read-only. During one run, the order lookup returns the same service error several times in a row. The repeated-error stop condition fires, the run ends without a reply, and the case moves to a staff queue with the order reference, the error text and the steps already taken.

The reviewer later sees that the lookup service was unavailable, not that the customer's request was wrong. The agent also never held permission to issue a refund, so its draft could not promise one. The harness behaved as designed, and the log explained the outcome without needing the model to explain itself.

How Be Human handles this in a managed setup

Be Human designs, implements and operates personalized AI infrastructure, including the harness around each agent. A first engagement defines one useful role, the memory and tools that role needs, the approval boundaries, and an observable success criterion, so the checklist above is applied to a specific agent rather than to a general idea. Managed operations then cover monitoring of stop reasons and budgets, recurring evaluations, permission review when tools or staff roles change, incident response, and controlled improvements that ship with a recorded version. The work is remote and available in English and Portuguese.

Take the free Be Human AI diagnostic to find the first task worth handing to an agent.

Frequently asked questions

What is an agent harness? An agent harness is the software layer around a model that runs its loop, passes it context, exposes tools, enforces permissions and records what happens. The model proposes actions, and the harness decides whether they run. For a business, the harness is where most operational risk is controlled, so it deserves its own review.

Which stop condition should a team add first? Start with the rule that protects the most expensive mistake, usually a rule that hands the work to a person when an action would change a record, send a message or spend money. Next, add a repeated-error rule and a budget ceiling. These three cover much of the runaway behavior teams see without requiring elaborate logic.

Should a company build its own harness or use a provider? The answer depends on who will maintain the wrapper when models and tools change. Building gives full control but creates ongoing work, while a managed service can carry part of that operations load. Compare the options by asking who handles monitoring, permission review and incident response, and who records each change.

How do we know the harness is working? Review run records over a fixed period: how often runs succeed, how often they stop and why, and whether escalations reach a person with enough detail to act. A harness is working when a reviewer can explain any run from its log alone. Regular evaluations against a fixed set of test tasks also show whether a change improved the agent or made it worse.

Your next step

See where an AI agent fits in your company first.

Take the free Be Human AI diagnostic. A few questions about your work show which task to hand to an agent first, what it needs to know and where a person approves.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.

Less busywork. More room for you.

Bring us the work you wish you had time for.

Book a call to try a personalized AI agent free for 7 days ↗Or take the free AI diagnostic →