Skip to content
be-human.ai
← Back to the journal

Harness fundamentals

AI Harness Versioning: Release Notes for Every Agent Change

Version prompts, tools and permissions together in your AI harness so every agent change gets a dated, reviewable release note and a clear rollback path.

Be Human typographic cover with the title: AI Harness Versioning: Release Notes for Every Agent Change
Conceptual illustration created with AI · Be Human

Versioning an AI harness means treating the prompt, the tool definitions and the permission rules of an agent as one release unit. Each release gets an identifier and a dated note that states what changed, why, who approved it and how to return to the previous version. Done this way, any bad output can be traced to the exact configuration that produced it.

Key takeaways

  • Version the prompt, the tool surface and the permission set together, as one release.
  • Write a release note for every change that states what changed, why, who approved it and how to undo it.
  • Give each release an identifier that points to exactly one combination of parts.
  • Treat permission changes as the highest-review category, even when the prompt edit looks minor.
  • Test each release on a fixed set of tasks before it touches live work.

Why a prompt edit is never just a prompt edit

In most companies, an agent change begins as a text edit. Someone rewrites a paragraph in the system prompt, adds a tool, or widens a permission because a task failed the day before. The change looks small, but the agent's behavior depends on three things at once: the instructions it reads, the tools it can call, and the permissions that decide what those calls may touch. If only one of them is versioned, nobody can later say which combination produced a given outcome.

Software teams learned this lesson long ago. A deployed service is identified by a build, and its configuration is recorded next to that build. Agents need the same discipline with one difference: instructions are written in natural language, so a reviewer cannot judge the effect of a change by reading it the way they would read code. The release note has to carry the reasoning that the diff cannot show.

Anthropic's engineering guidance on long-running agents describes the harness as the structure around the model that carries state across sessions. For a business owner, the practical reading is that the harness is what your team changes, so the harness is what needs a history. If the term is new to your team, what is an AI harness explains it in plain language.

What belongs in one harness release

A release unit is the set of files and settings that together define how an agent behaves. For most business agents, that means five parts:

  • The instructions: the system prompt, the role description and any reusable procedures the agent loads.
  • The tool surface: which tools exist, their names, their descriptions and their input rules.
  • The permission set: what each tool may read, write, send or delete, and which actions need a named person to approve them.
  • The memory scope: which records the agent may retrieve and from which sources.
  • The model choice and the evaluation set used to test the release.

Tool descriptions shape how the model chooses actions as much as the prompt does, so they belong in the release and not in a separate tool catalog that changes silently. The same is true of memory scope. An agent that suddenly retrieves records from a folder it could not see before has changed behavior, even if its prompt is identical.

Keep these parts under one identifier. If the prompt is at one release and the permission set was changed in a separate ticket, you have two histories that do not match. The rule is simple: a release identifier points to exactly one combination, and a change to any part produces a new identifier.

A release note template your reviewers can read in two minutes

Use the same fields for every change. A note that takes twenty minutes to read will be skipped, and a skipped note gives false confidence, which is worse than no note. Keep the template short enough that the approver can review it before the next meeting:

  • Release identifier and date of publication
  • Agent and the single role it serves
  • Summary: one sentence stating the behavior that changes
  • Reason: the failure, request or business need that triggered the change
  • Prompt changes: what was added, removed or reworded
  • Tool changes: tools added, removed or redescribed
  • Permission changes: each permission shown as before and after
  • Evaluation: the fixed task set that was run, and the result in plain words
  • Approver, and the person who will watch the first live runs
  • Rollback: the previous identifier and the exact step to return to it

The permission lines deserve the most attention. Write them as pairs, before and after, so a change from read-only to write access cannot hide inside a longer paragraph. The same fields also serve as the starting point for the audit trail that regulated teams are often asked to produce.

Step by step: shipping an agent change

The following sequence works for a small team with one agent and scales to several agents without changing its logic:

  • Step 1: Write the reason before the edit. Describe the observed failure in one sentence and attach the example that shows it.
  • Step 2: Make the change in a copy of the current release, never in the live configuration.
  • Step 3: Mark which parts changed: instructions, tools, permissions, memory scope or model. Any permission change goes to a named approver.
  • Step 4: Run the fixed task set for this agent, including the cases that worked before, so regressions appear before launch.
  • Step 5: Publish the identifier and the note together, then send a small share of live work to the new release before full rollout. This is the step where a canary release keeps the blast radius small.
  • Step 6: Watch the first runs with a named owner, and record what they showed in the note.
  • Step 7: If the outcome is worse, return to the previous identifier and log the reason. A rollback is also a release, with its own note.

The logs for each run should carry the release identifier. The OpenTelemetry observability primer describes how traces and logs connect events to the context in which they happened, and the same idea applies here: a run without its release identifier cannot be matched to a configuration later. If your logs do not record it, add that field before you add anything else.

Illustrative scenario: a permission change that looked small

Illustrative scenario, not a client case: a support agent for a small parts seller answers order status questions. To reduce the number of handoffs to staff, the team adds a tool that can issue refunds below a set limit, and adds a sentence to the prompt asking the agent to reassure customers quickly. The prompt sentence is reviewed in a chat thread. The permission change is a single line in a configuration file that nobody reads closely. Soon after launch, the agent issues refunds for claims that should have gone to a person.

A proper release note would have exposed two things. The refund permission moved from read-only to write, and the approver field was empty. The rollback line would have named the previous identifier. Instead, the team reconstructed the sequence from chat history, which took longer than the fix itself.

The structural lesson is in line with the principle of least privilege that the OWASP authorization guidance describes: an agent should hold only the access its task needs, and granting more should be a deliberate, recorded decision. Publishing a release that adds write access without a named approver should be impossible in your process, not merely discouraged.

How managed operations keep release notes current

Be Human designs, implements and operates personalized AI infrastructure in which the agent's identity, memory, tools, permissions and operating context stay under human control. In a first engagement, the instructions, tool surface, permission set and memory scope for one defined role are placed under a single release identifier, and every change passes through the note template with a named approver. Managed operations then add recurring evaluations against the same fixed task set, scheduled permission reviews, and incident response that begins by reading the release a problematic output came from. Rollback to a previous identifier is a documented step, not a rebuild. A change to model fallback routing is also a release, so it receives the same note and the same review.

If you are unsure which agent task deserves this discipline first, take the free Be Human AI diagnostic to find the first task worth handing to an agent.

Frequently asked questions

How is a harness release different from a prompt version? A prompt version records only the instructions, while a harness release records the instructions, tool surface, permissions, memory scope and model choice under one identifier. The difference matters most when behavior changes because a tool description or a permission changed and the prompt did not. Keeping all the parts together makes each outcome traceable to a single configuration.

Who should approve a release note? The approver should be the person who owns the business outcome of the agent, not only the person who wrote the change. Permission changes need an approver who can judge whether the access is appropriate for the task, and that person should be named in the note. If nobody is named, the change is not ready to publish.

How often should we publish a new release? Publish when a change is ready to be tested and reviewed, not on a fixed calendar, and keep each release small enough to explain in the template. Frequent small releases are easier to roll back than rare large ones, because each note covers a narrow set of changes. Routine evaluations and permission reviews should run on their own schedule, and their results should also be recorded.

Do we need special software to version agents this way? A shared repository or a controlled folder can work if every release has an identifier, a note and a stored copy of the previous version. The essential part is the habit of recording all five parts together and refusing to publish without a note. Dedicated tooling can help later, and the decision between building and buying it is covered in build or buy an AI harness.

Your next step

See where an AI agent fits in your company first.

Take the free Be Human AI diagnostic. A few questions about your work show which task to hand to an agent first, what it needs to know and where a person approves.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.

Less busywork. More room for you.

Bring us the work you wish you had time for.

Book a call to try a personalized AI agent free for 7 days ↗Or take the free AI diagnostic →