How to Update an AI Agent in Production Without Breaking It

To update an AI agent in production safely, treat every change as a small release. Record how the agent behaves today, change one thing at a time (instructions, a tool or the model), test it against real past cases, release it to a limited slice of work, and keep a tested way back. An agent is not just a prompt. It is instructions, tools, memory, permissions and a model working together, so an edit in one place can shift behavior in another.

Key takeaways

• An agent is a system, so test its whole behavior, not only the part you edited.
• Change one layer at a time (instructions, tools or model) and write down what should change and what must not.
• Keep a small set of real past cases as a regression set, and run it before and after every change.
• Release to a limited slice of work first, with a named owner watching and a rollback you have already tried.
• Log every change with its reason, version and result, so the agent's history is readable by someone who was not there.

Why does a small edit break a working agent?

A traditional software change usually breaks in a visible place. An agent change often breaks somewhere you were not looking. You add one sentence to the instructions about tone, and the agent quietly stops asking for approval before a certain kind of action. You rename a field in a tool, and the agent keeps calling it with the old name and then explains away the error in a confident message. You move to a newer model, and the agent starts answering at twice the length, which pushes a downstream step past its time limit.

Three properties make this likely. First, the agent reads language, so every sentence is also a behavior, including the sentences in tool descriptions. Second, the model fills gaps with judgment, and a new edit changes which gaps it notices. Third, most failures are partial. The agent still works for most requests, so nobody raises an alarm until a customer or a colleague notices a pattern.

The answer is not to freeze the agent. A frozen agent drifts out of date as products, policies and people change. The answer is a routine that makes change ordinary, small and reversible. If you want the broader picture of where each change sits, what is an AI harness is not needed here, but it helps to remember the layers: instructions, tools, memory, permissions, model and the runtime that connects them. Each one can be edited, and each one needs its own kind of check.

What do you record before changing anything?

You cannot tell whether a change helped if you never captured the starting point. Before the first edit, write a baseline. It does not need to be elaborate. It needs to be findable and honest.

• The current instructions, exactly as deployed, with a version label.
• The tool definitions the agent can see, including names, descriptions and the permissions each tool runs with.
• The model identifier and any settings that affect output, such as length limits.
• The memory or knowledge sources the agent reads, and when they were last reviewed.
• A reference set of real cases: requests the agent handled well, requests it handled badly, and requests where it correctly stopped and asked a person.
• The success measure the role is judged by, written as something observable, for example the share of requests resolved without rework, as counted by the owner.

The reference set matters most. Pull it from real work, remove personal data you do not need, and include the awkward cases: the ambiguous request, the missing attachment, the customer who writes in a different language, the request that should be refused. Evaluation guides from [OpenAI](https://platform.openai.com/docs/guides/evals) and [Anthropic](https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents) both make the same underlying point: you need a repeatable set of tasks with clear grading, or you are comparing impressions.

Also write down what must not change. For most business agents that list is short and serious: it never sends external messages without approval, it never writes to the system of record outside its scope, it always escalates a defined class of requests. These become the first checks after every release.

How do you edit instructions without side effects?

Instructions are the most tempting thing to edit because it takes a minute and needs no deployment. That is also why they cause the most accidental damage. A few habits keep it under control.

Make one edit with one purpose. If you want to fix how the agent summarizes a thread and also adjust its escalation wording, that is two changes. Release them separately so that a regression has only one suspect.

Write the intent next to the change. A line in the change log such as "summaries should name the open decision first" lets a reviewer judge the wording against the goal. Without it, the next person cannot tell whether a rule is still needed.

Avoid patching for single incidents. When one odd case goes wrong, the quick fix is a new rule that names that case. After many of these the instructions become a pile of exceptions that contradict each other. Ask whether the incident reveals a missing principle, a missing piece of context, or a tool problem. Often the cleanest repair is deleting or rewording an existing line. The article on how to keep AI agent instructions maintainable covers structure that makes this easier.

Separate boundaries from style. Rules about approvals, permissions and refusals deserve extra scrutiny, a second reviewer, and their own tests. Rules about tone and format can be changed with lighter review. Put them in different sections so a style edit never touches a boundary by accident.

Illustrative scenario: a small services firm runs an agent that drafts replies to incoming quote requests and hands anything involving custom terms to a person. The owner adds a line asking for warmer, shorter replies. Nothing in the edit mentions escalation. But when the reference cases are run, two requests with custom terms now get a friendly short draft that answers the question directly instead of handing it over. The shorter style had crowded out the handoff behavior. Because the edit was small and tested before release, the fix is a one-line clarification that the handoff rule outranks the style rule. Without the test, the first sign would have been a customer receiving terms nobody approved.

How do you change a tool or integration safely?

Tools are where an agent touches real systems, so mistakes here cost more than a clumsy sentence. The safest default is to add rather than replace. Introduce the new version of a tool next to the old one, point the agent at the new one only in testing, and retire the old one after the new one has proven itself.

Treat the tool description as part of the instructions. The agent decides when and how to call a tool largely from its name, description and input fields. Anthropic's guidance on [writing tools for agents](https://www.anthropic.com/engineering/writing-tools-for-agents) stresses clear names, unambiguous parameters and useful error messages. If you change what a tool returns, such as field names, ordering or how much text it sends back, the agent's behavior can change even though the instructions are identical. For deeper design guidance, see AI tool contracts: design inputs and outputs that work.

Keep permissions fixed unless widening them is the point of the change. A common mistake is to grant broader access to make a new integration work during testing and then forget to narrow it. Write the permission level (read, draft or send) in the change record, and review it at release. If an action can cause duplicate effects, confirm that repeating a call is safe before the agent is allowed to retry.

Test against a safe copy first. Use sandbox accounts, test mailboxes or draft-only modes so that a faulty call produces a harmless artifact rather than a sent message or an edited record. A proper staging environment for AI agents makes this routine instead of improvised. Then check the unhappy paths: an expired connection, an empty result, a timeout, a response in an unexpected shape. The agent should stop and report, not invent a result.

How do you swap or upgrade the model?

A model change touches everything at once, which makes it the riskiest edit and the one most often treated casually. Providers release newer versions and retire older ones, and the pressure to move is real. Handle it as a project with its own evidence.

Run the reference set on the candidate model with the current instructions and tools, unchanged. Compare outcomes case by case against the baseline. Look past correctness to behavior that downstream steps depend on:

• Format: does it still produce the structure your tools and templates expect?
• Length: do answers get longer or shorter in ways that affect reading time or limits?
• Tool habits: does it call tools in the same order, with the same arguments, or does it skip steps?
• Stopping behavior: does it still ask when it should, and refuse when it should?
• Speed and cost per completed task: do they stay inside the range the workflow can tolerate?

Instructions tuned for one model sometimes carry workarounds that a newer model no longer needs, or even misreads. Plan a short pass to remove them, as a separate change after the swap is stable. Do not combine the two.

When the stakes allow, run the new model in shadow mode: it processes real requests, its outputs are recorded, and people compare them with what the live agent did, but nothing it produces reaches a customer. The article on running an AI agent in shadow mode describes how. If you are deciding whether to move at all, how to choose the right AI model for a business agent lists the criteria worth weighing. Whatever you decide, keep the ability to route back to the previous model for as long as the change is fresh.

What does a controlled release routine look like?

Here is a plan you can adopt as written and then adjust to your size. It works for an instruction edit, a tool change or a model swap.

• Name the change and the owner. One sentence on what is changing and why, and one person accountable for the result.
• Write the expectation. What should be different afterward, and what must stay exactly the same, including the approval boundaries.
• Snapshot the current state. Save the deployed instructions, tool definitions, model identifier and settings under a version label.
• Make the change in a test copy. Edit one layer only. Do not touch anything else while you are there.
• Run the reference cases. Compare with the baseline. Read the actual outputs, not only a pass or fail count, and look for new kinds of mistakes.
• Check the boundaries first. Confirm approvals, refusals, escalations and permissions behave as before.
• Release to a small slice. Choose a low-risk set of requests, a single channel or a single team, with a person watching the results daily.
• Watch the signals. Use the logs and traces you already collect (the [OpenTelemetry observability primer](https://opentelemetry.io/docs/concepts/observability-primer/) explains the vocabulary) to compare errors, handoffs, retries and completion against the baseline.
• Decide to expand, hold or roll back. Write the decision and the reason. Rolling back should mean restoring the snapshot, and you should have tried that restore at least once before you needed it.
• Close the record. Note what changed, what you learned and whether the reference set needs a new case because of what you saw.

Two habits keep this lightweight. Schedule changes at a time when someone is available to watch them, not at the end of a busy day. And batch nothing that you would not be willing to roll back together.

How does Be Human handle controlled improvements?

Be Human treats improvement as part of operating an agent, not as an occasional rewrite. Instructions, tool definitions and model choices are versioned, each change is tested against a reference set built from the client's own work, and releases go to a limited scope before they go everywhere. Approval boundaries and permissions are reviewed at every release, and the previous version stays available so a rollback is a restore rather than a rebuild. The agent's owner on the client side sees what changed and why. This sits alongside monitoring, recurring evaluations and integration maintenance in managed operations, and the client keeps control over what the agent is allowed to do.

Frequently asked questions

How often should we update a live AI agent? Update when there is a reason you can state: a policy changed, a recurring error appeared, a tool changed, or a model is being retired. Small, frequent, tested changes are safer than rare large ones, because each one has fewer suspects if something goes wrong. A regular review slot helps you decide what is worth changing and what to leave alone.

Do we need a separate test environment to update an agent in production? You need somewhere safe to run the change before real customers see it, and for agents that take actions that usually means sandbox accounts and draft-only modes at minimum. A full staging copy is better when the agent writes to important systems. Even a small company can start with a test mailbox, a test workspace and a saved set of reference cases.

How do we know a change made the agent better and not just different? Compare against the baseline using the same reference cases and the same success measure, and read the outputs yourself. A change is better when the measure improves, the boundaries hold and no new class of mistake appears. If you cannot say what you expected to improve before you made the change, you are not ready to judge it.

What should we do when a released change causes a problem? Roll back first, investigate second. Restore the saved version, tell the people who rely on the agent what happened, and review the cases affected during the window. Then add the failure to the reference set so the same mistake is caught before the next release.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.