AI Harness Checklist for Safe Model Upgrades in Your Company

An AI harness checklist is a short set of checks for the software layer around a model: the tools it can call, the memory it can read and write, the permissions that bound each action, and the logs that record what happened. A well-built harness lets you upgrade or replace the model without rewriting the business process, because every change passes through the same gates. Use the checklist below before an agent touches live systems, and run it again whenever the model, a prompt or a tool changes.

Key takeaways

• The model is replaceable. The tool contracts, permission rules, memory policy and run logs are what your company must own.
• Give every tool a narrow scope, a named owner and an explicit rule for any action that changes money, records or customer commitments.
• Log each run with the model version, instruction version and tool calls, so a change can be compared with evidence instead of impressions.
• Route every model or prompt change through a fixed evaluation set, a staging run and a limited rollout with a tested rollback path.
• Decide before the first incident what a failure looks like, who receives the escalation and which switch pauses new work.

What belongs inside the harness?

The model is the reasoning engine. The harness is everything your company must be able to inspect, change and turn off. In practice that includes the instructions the model receives, a registry of tools with their input and output contracts, a memory store with rules for what gets written, a permission layer that decides whether a proposed action runs, an approval queue for actions that need a person, a run log, an evaluation set and a kill switch that stops new work without deleting history.

The useful test is ownership. If a model provider changes behavior without notice, you should still be able to answer three questions: which tools the agent could have used, what it remembered, and who approved each write. Those answers come from the harness, not from the model. Anthropic's engineering guidance on long-running agents makes a related point: progress has to live in durable artifacts outside the model's context if work is going to survive a session boundary.

Checklist before any model or prompt change

Review these items for each agent before it goes live, and again before any change to the model, instructions or tools.

• The agent has a written role, a named business owner and one success criterion that a person can check.
• Every tool has a documented input, output and error format, plus a single accountable owner.
• Read-only tools and write tools are listed separately, and every write tool has its own permission rule.
• Each permission rule states which actions run automatically, which need approval and which are blocked.
• Every memory write records its source, its time, its owner and a review or expiry date.
• Retrieved documents, emails and tool outputs are treated as data, never as instructions to the agent.
• Each run records the model version, instruction version, tool versions, tool calls, approvals and final status.
• A fixed evaluation set covers routine cases, edge cases and every past failure.
• A staging environment points side-effecting tools at test records, so integration errors appear before customers see them.
• The rollout plan names the share of live work, the signal that confirms success, the stop condition and the exact rollback step.
• The escalation path sends a person the decision, the evidence and the proposed next action.
• A kill switch can pause new runs for one agent without disturbing the others.

How should tools and permissions be scoped?

Start from the smallest set of actions the role needs. An agent that reads a shared calendar and drafts replies does not need permission to delete events or to change invoices. Grant read access first, then add write access one action at a time, each with its own rule and its own log entry. OAuth 2.0 is a practical reference for this habit: request only the access a given integration needs, and review those grants whenever the role changes.

Side-effecting tools also need idempotency. If a network timeout happens after a record is created, a retry must not create a second one. Stripe documents idempotency keys as a way for clients to retry requests safely, and the same discipline applies to your own internal tools: a request identifier, a check for prior completion and a result status that distinguishes success, duplicate and failure.

Authorization belongs outside the model's judgment. The model may propose an action, but the permission layer decides, using rules written by people and checked against the identity of the requester and the record involved. OWASP's authorization guidance is a useful checklist for this design. Add hooks that run checks before and after sensitive actions, so a failed check stops the action rather than producing a warning nobody reads. The hooks guide describes this pattern in more detail, and the guardrails overview helps decide which controls to add first.

What memory should the harness keep, and what should it refuse?

Memory is a write path, so it needs rules. Decide which facts may be stored, such as customer preferences, open tasks and decisions with their stated reasons. Decide which must never be stored, such as credentials or personal details the role does not need. Decide who can correct or delete an entry. Store memory as reviewable records rather than only as opaque vector entries, so a person can read what the agent believes and why.

Retrieval and fine-tuning solve different problems. Retrieval fetches current documents at run time, which suits policies, price sheets and records that change often. Fine-tuning changes model behavior, which is harder to inspect and roll back. AWS's comparison of retrieval and fine-tuning is a good starting reference. For most business roles, keep company knowledge in versioned, retrieved sources, and keep behavior in instructions and procedures that you can diff and review.

The harness must also assume that content it reads may contain hostile instructions. OWASP's prompt injection prevention guidance recommends separating trusted instructions from untrusted data, limiting what the agent can do after it reads external content, and requiring approval for sensitive actions. A prompt sentence saying "ignore suspicious text" is not a control. The permission layer is.

Illustrative scenario (not a real client): a team's intake agent reads an email that contains a line asking it to change a delivery address. Because the address change is a write action with an approval rule, the agent drafts the change and sends it to a coordinator. The email was data, the approval was the control, and the run log shows both.

What should the run log capture?

If you cannot reconstruct a run, you cannot improve it. OpenTelemetry's observability primer explains how traces, metrics and logs answer different questions, and its semantic conventions for generative AI define shared attribute names for model and agent operations. Your harness does not need to store every token. It does need enough to answer what the agent saw, what it decided, what it touched, who approved and how the run ended.

A practical run record includes the following fields.

• Run identifier, agent identifier and the business object the run worked on.
• Model name and version, instruction version and the version of each tool used.
• Each tool call with its inputs, a summary of its outputs and the permission rule that allowed it.
• Approval requests, the identity of the approver, the decision and the time.
• Memory writes with their source and owner.
• Errors, retries, timeouts and the final status.
• Latency and cost per run, so that unit economics can be tracked over time, as the FinOps Foundation describes.

Redact personal data the role does not need, and restrict who can read full traces. Logs are useful only if the people responsible for the agent can actually open them during an incident.

How do model improvements reach production safely?

A better model, a revised prompt and a new tool version are all changes to the system. Handle them the same way, and keep each change small enough to reverse. The sequence below is a workable default.

• Record a baseline by running the evaluation set against the current model and instruction versions, and save the results.
• Run the candidate change against the same evaluation set and compare task outcomes, not how fluent the answers sound.
• Check every new failure against the permission rules, and confirm that no new behavior reaches a write action it should not reach.
• Replay a sample of recent real runs in staging, with side-effecting tools pointed at test records.
• Release the change to a small share of live work, with a named stop condition such as a rise in escalations or a failed approval check.
• Keep the previous model and instruction versions deployable, so rollback is a configuration change rather than a rebuild.
• Review the first week of logs with the business owner, and record the decision to adopt, adjust or roll back.

The regression testing guide for model updates covers the comparison step in more depth, and the canary release approach explains how to limit exposure while you watch the signal.

Illustrative scenario (not a real client): a team evaluates a newer model for its intake agent. Drafts improve, but in staging one run attaches a document to the wrong file because a lookup tool returns an ambiguous name match. The model did not cause the error on its own. The attach tool lacked an identity check. The team changes the tool contract to require a policy number, reruns the evaluation, and then rolls out the model change. The harness gap was fixed first, and the model change followed.

Where to start, and how Be Human handles the harness

Do not build the whole harness at once. Choose one role with a clear owner, a short list of tools and one measurable success criterion, then build the permission rules, memory policy, run log and evaluation set for that role. Expand only after the first role runs with evidence.

Be Human designs, implements and operates personalized AI infrastructure for companies, with agents that keep their identity, memory, channels, tools, permissions and operating context under human control. A first engagement defines one useful role, the memory and tools it needs, the approval boundaries and an observable success criterion. Managed operations can then cover monitoring, recurring evaluations, integration maintenance, permission review, incident response and controlled improvements, so that a model or prompt change passes through the same gates described in this checklist. The work is delivered remotely, in English or Portuguese.

If you are not sure which task to hand to an agent first, take the free Be Human AI diagnostic to find the first task worth handing to an agent.

Frequently asked questions

What is the difference between an AI harness and an AI agent? An AI agent is the combination of a model and the loop in which it reasons, picks actions and observes results. The harness is the surrounding system that defines which tools exist, what memory is kept, which permissions apply, what gets logged and how changes are released. You can think of the agent as the behavior and the harness as the controls that make the behavior inspectable and reversible.

Do I need a harness if I only use a chatbot for internal questions? A chatbot that only answers questions from a fixed document set carries less risk than an agent that writes to systems, but it still benefits from logging, a defined source of truth and a way to correct wrong answers. Once the assistant can create tickets, change records or send messages, the harness items about permissions, approvals and idempotency become necessary. Start with the lightest set that matches the actions the tool can take.

Who should own the harness inside a company? Assign one business owner for each agent, responsible for its success criterion and its approval rules, and one technical owner responsible for tools, integrations and logs. The business owner should be able to answer what the agent is allowed to do without asking an engineer. Reviews of permissions and evaluation results should happen on a fixed schedule, not only after something goes wrong.

How large should an evaluation set be before a model change? Start with the cases that matter most: routine work the role handles every day, the edge cases that cause trouble, and every failure you have already seen. The set should grow with each incident, because each real failure is a test you will want to keep. A small set that covers the high-impact actions well is more useful than a large set that mostly tests phrasing.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.