Skip to content
be-human.ai
← Back to the journal

Personalized AI infrastructure

Superintelligence Tech Debt: Prompts and Tools Nobody Owns

Unowned prompts, stale tool schemas and undocumented permissions keep AI agents from improving as models get better. Here is how to find them.

Be Human typographic cover with the title: Superintelligence Tech Debt: Prompts and Tools Nobody Owns
Conceptual illustration created with AI · Be Human

AI agent tech debt is the accumulated set of prompts, tool definitions, permission grants and operating notes that a company relies on but nobody formally owns. When the underlying model improves, those unowned pieces hold the agent back: nobody knows which instructions are safe to change, which tool schemas still match live systems, or why a scope was granted in the first place. Finding and assigning these items is the practical first step toward a company AI layer that improves with each model generation instead of resisting it.

Key takeaways

  • Agent tech debt sits in four places: prompts, tool schemas, permission scopes and operating notes.
  • An item without a named owner cannot be changed safely, so a stronger model adds risk instead of value.
  • Find unowned items by tracing each agent action back to the instruction, tool and scope that produced it.
  • Give every prompt and tool an owner, a stated purpose and an evaluation case before changing the model underneath.
  • Treat changes as controlled releases with a record, not as silent edits made by whoever is online.

Why unowned prompts block improvement

A prompt usually starts as a quick fix. Someone adds a line after the agent gives a wrong answer to a refund question, another person adds a formatting rule for a weekly report, and a third pastes in a paragraph from an old procedure. Over time the prompt becomes a stack of patches, and nobody can say which lines are policy, which are workarounds and which were only meant for the original pilot.

That ambiguity matters when the model changes. A workaround written for an older model's weakness may be unnecessary now, but removing it is risky if the same line also encodes a business rule. A more capable model also follows instructions more faithfully, so a contradictory line that used to be ignored can now shape answers in ways nobody intended. Without an owner, the team has two poor options: leave everything alone and carry the cost of stale guidance, or change things blindly and hope the answers still look right.

The remedy is a record rather than a better prompt. Each instruction block needs a purpose, a named owner, the business rule it supports (if any) and the evaluation cases that show it still works. Anthropic's engineering writing on context engineering for agents treats the information an agent sees as a design decision, not a byproduct of past edits. The Be Human component inventory checklist, available as Superintelligence Layer Components: A Checklist Buyers Can Use, is a useful reference for deciding which parts of the layer belong on that record.

Stale tool schemas: the quiet mismatch

A tool schema tells the agent what a tool is called, what it does and which parameters it accepts. The agent makes its decisions from those descriptions, so a schema that no longer matches the live system is a silent source of error. A renamed field in a customer system, a status code that changed meaning or a search endpoint that now returns paginated results can each leave the agent with partial or misleading data.

Stronger models can make this harder to notice. A capable model reasons smoothly over whatever it receives, so it may produce a fluent answer from a mismatched result instead of stopping to flag the problem. The output looks right, which is exactly why the mismatch survives review. Anthropic's guidance on writing tools for agents recommends clear, specific descriptions and parameters that are hard to misuse, and that standard works well as an audit question: does this schema still say what the system actually does?

For each tool, ask two questions. Who maintains the live system it calls? And who was told the last time that system changed? If the answer to both is nobody, the tool is unowned even when the code still runs.

Undocumented scopes: permissions nobody can explain

Scopes define what an agent can read, write, send or approve. Many are granted during a demonstration, when broad access is the fastest way to show value, and then never narrowed. Months later the agent can edit records it never needs to touch, and no one can say whether that was a deliberate choice or a convenience that stuck.

Undocumented scope becomes more dangerous as agents improve. A more capable agent chains actions together more effectively, so combinations of permissions that nobody tested become realistic paths. OWASP's authorization guidance favors granting the minimum access needed and checking access on every request, which for an agent means each scope needs a written reason, an approver and a boundary that defines which actions still require a person. Prompt injection adds another reason. OWASP's prompt injection guidance warns that instructions hidden in content an agent reads can try to trigger actions, so broad scopes raise the cost of any successful injection.

A quarterly review is the simplest way to keep scopes honest. The method is described in Quarterly AI Agent Permission Review: Steps and Evidence.

Figure placeholder for the still-life image.

How to find them: a tracing method

You do not need a new platform to start. You need a complete list of what the agent carries and a way to connect each visible behavior back to its source. Observability concepts, such as following a single request through each step it touches, are a useful model here, and OpenTelemetry's observability primer describes that idea in general terms. The steps below work with whatever logs and documents you already have.

  • Export every live system prompt, instruction block and tool definition from each place the agent runs, including copies people keep in shared documents.
  • Pull a recent set of agent actions from logs and trace each one to the instruction, tool and scope that produced it.
  • Mark every item that has no named owner, no written purpose or no link to a current business rule.
  • Compare each tool schema with the live system it calls, field by field, and flag every name, type or meaning that has drifted.
  • List every permission the agent holds, write the reason beside each one and remove any scope that has no reason.
  • Assign an owner to each surviving item and set a review cadence that matches how often the underlying system changes.
  • Store the inventory where owners can edit it and where every change leaves a dated record.

Expect the first pass to be uncomfortable. Most teams find at least one prompt line that nobody can explain and at least one scope that predates the current process.

What to decide before the next model upgrade

Illustrative scenario: a regional distributor runs an assistant that answers order status questions for its sales staff. Its prompt still describes a product line that has since been discontinued, its order lookup tool expects a warehouse code that was retired, and it holds write access to order notes that was granted during the original pilot. After the team moved to a newer model, the assistant began answering questions about the discontinued line with more confidence than before. The trace showed the prompt, the outdated parameter and the unused write scope in one chain, and nobody had been assigned to any of the three.

The distributor's team could then decide each finding on its own terms:

  • If a prompt line encodes a current business rule, move the rule into a maintained policy document and reference it from the prompt.
  • If a line only patches an old model weakness, remove it in a test run against your evaluation cases before deleting it from production.
  • If a tool schema no longer matches its live system, correct the schema before any model change, because a better model cannot repair a wrong input.
  • If a scope has no written reason, remove it and watch the logs for failed tasks that needed it, using an observation window agreed in advance.
  • If two teams depend on the same prompt or tool, give it one owner and give the other team a documented request path.
  • If any of these changes is needed, run it through a written request with an approval step, as set out in How to Request and Approve Changes to a Managed AI Agent.

The point of the exercise is not a clean file. It is that every change now has a reason, an owner and a test that can show whether the upgrade made the agent better or only different.

Keep the inventory current

Tech debt returns when the inventory stops being maintained. Be Human's managed operations include monitoring, recurring evaluations, integration maintenance and permission review, so each prompt, tool and scope has a recorded owner, a stated reason and a scheduled review point. Changes move through a request, a test against the evaluation set and an approval by the business owner before they reach production. That keeps the agent's operating context under the company's control rather than in the memory of whoever edited it last. Take the free Be Human AI diagnostic to find the first task worth handing to an agent.

Frequently asked questions

What is AI agent tech debt? It is the buildup of prompts, tool definitions, permission scopes and operating notes that an agent depends on without a named owner or a written purpose. It becomes a problem when a better model arrives and the team cannot safely change those items, so the company gets less benefit from the upgrade. The debt is usually missing knowledge rather than broken code.

Does a small company need a formal inventory for this? Yes, but it can be a single document that lists each prompt block, tool and scope with an owner and a reason. The first pass is often manageable for a small team because the number of items is limited. What matters most is that the list stays current and that every change leaves a record.

Who should own a prompt or a tool? The owner should be the person accountable for the business outcome the item supports, not simply the person who wrote it. For a tool, the owner also needs to hear about changes to the live system it calls. Ownership can be shared during review, but it should never be ambiguous.

Should we fix unowned prompts before upgrading the model? Usually yes, at least for items that encode business rules or call live systems. An upgrade tested against an unclear configuration cannot show whether the new behavior is better or worse. Once the inventory is in place, run the upgrade against your evaluation cases and then decide what to change.

Your next step

See where an AI agent fits in your company first.

Take the free Be Human AI diagnostic. A few questions about your work show which task to hand to an agent first, what it needs to know and where a person approves.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.

Less busywork. More room for you.

Bring us the work you wish you had time for.

Book a call to try a personalized AI agent free for 7 days ↗Or take the free AI diagnostic →