Threat Model for a Company Superintelligence Layer in One Page
A one-page threat model for company AI agents covering prompt injection, memory poisoning and tool misuse, with an owner and a testable control for each risk.

An AI agent threat model is a short written list of the ways an agent can be pushed into doing the wrong thing, what each failure would cost, who answers for it, and which control limits it. For a company agent that reads messages, keeps memory and calls business tools, the three threats to cover first are prompt injection, memory poisoning and tool misuse. A useful version fits on one page, and every row names an owner and a control that someone can check.
Key takeaways
- Start from what the agent can read, remember and do, not from a generic list of attacks.
- Prompt injection enters through any text the agent reads, so treat outside text as material to evaluate, never as instructions from the business.
- Memory poisoning is slow and quiet, so control what gets written to memory, review sensitive facts and let old entries expire.
- Tool misuse is limited by narrow permissions, approval gates on irreversible actions and logs that a person can read.
- Every row needs one named owner and one control that can be tested before launch.
Why one page is enough to start
Long security reports tend to be read once and filed. A one-page model works because it forces the team to name the role, its inputs, its memory, its tools and its approvals in a single view. Scope it to one role first. An agent that answers supplier questions and an agent that prepares payments have different threat models, even when they run on the same platform.
Write the page before the agent is built, then revise it after the first test run. The first version will be wrong in places, and that is useful, because the gaps show where a person needs to decide something that the technology cannot decide for you.
The three threats and how they reach your business
Prompt injection means that text the agent reads contains instructions, and the agent follows them as if the business had written them. The text can arrive in an email, a shared document, a web page, a support ticket or a form field. The OWASP prompt injection guidance treats this as a design problem: a language model cannot always tell instructions from content, so the system has to limit what a compromised instruction can reach. The email version of this risk is covered in more depth in Prompt Injection Risk for AI Agents: Defenses to Build First.
Illustrative scenario: a supplier email contains a line of hidden text asking the agent to send the last ten invoices to an outside address. The agent can read the inbox and send mail on its own. Nothing in the setup separates the supplier's text from the owner's request, so the message goes out. The fix is not better wording in the prompt alone. It is a boundary: sends to new external addresses need a person to approve them, and invoice data does not leave the company domain without that approval.
Memory poisoning is the second threat. Agents that keep memory store facts for later use. A wrong or hostile fact written once, such as a changed bank account for a supplier, can shape decisions for months because the agent treats its memory as trusted context. The risk is quiet, which is why it needs its own row and should not sit as a footnote under prompt injection.
Tool misuse is the third. The agent holds valid credentials and calls real tools, but with the wrong parameters, too often, or outside its role. From the system's point of view nothing is broken, so the defense has to live in the permission itself: narrow scopes, limits on volume and a record of every call.
The one-page threat model
Fill this structure for one role. Each row is one threat, one owner and one control that can be tested.
- Prompt injection through inbound email. Owner: the operations lead for the process. Control: email content is treated as data, and any outbound send to a new address waits for approval.
- Prompt injection through documents and web pages. Owner: the person who maintains the knowledge base. Control: retrieval sources are an approved list, and each source is tagged with where it came from.
- Memory poisoning from a wrong or hostile write. Owner: the lead of the business records the agent touches, often finance or operations. Control: memory is written only from verified sources, every write is logged, and changes to bank details, identity or prices need a human confirmation.
- Memory leakage across customers or departments. Owner: the privacy or compliance lead. Control: memory is partitioned by client and by role, and retrieval checks the partition before answering.
- Tool misuse within granted permissions. Owner: the system owner who holds the credentials. Control: each tool has the minimum scope the role needs, with volume caps and rate limits.
- Irreversible or money-moving actions. Owner: the executive who approves that category of decision. Control: a named list of action types requires a second person to approve before execution.
- Credential exposure. Owner: whoever manages secrets and access in IT. Control: short-lived tokens, stored outside prompts, with a revocation procedure that has been rehearsed.
- Silent drift after a model or prompt change. Owner: the agent's operations owner. Control: a fixed set of test cases runs before any change goes live, and results are compared with the last approved run.
- Missing evidence after an incident. Owner: the operations lead. Control: logs record inputs, memory writes, tool calls, approvals and outputs, kept for the period the company's policy requires.
When a row fails, speed matters. The steps for cutting off tokens, keys and channels are set out in Revoke AI Agent Access Fast: A Checklist for Tokens, Keys and Channels.
A plan to build the page in one working session
A small team can draft the page in one working session of a few hours, provided the right people are in the room. Follow the steps in order, because each one feeds the next.
- Name the role and write one success criterion that someone can observe, such as the share of supplier questions answered without escalation.
- List every input the agent reads, including email, shared folders, web sources, tickets and form fields.
- List every memory write, what the memory is used for, and who can correct it.
- List every tool and the exact permission it grants, including read and write rights.
- Mark every action that cannot be undone or that moves money, personal data or access rights.
- Assign one owner to each row. If a row has two owners, it has none.
- Write the control for each row and mark whether it exists today or is still planned.
- Test two controls before launch, using the checks in the next section.
- Schedule the review dates now, not after the first incident.
Be Human's managed approach follows the same sequence. A first engagement defines one useful role, the memory and tools it needs, its approval boundaries and an observable success criterion, and the threat page is written from that definition. Managed operations then keep the page accurate through monitoring, recurring evaluations, permission review, integration maintenance and incident response. Be Human does not promise that a threat can be removed entirely. It sets boundaries, names owners and checks those boundaries over time.
Testing the controls before the agent touches customers
A control that has never been tested is a note in a document. Before launch, test each control on purpose, and keep the results with the threat page. The following checks cover the three threats in this article.
- Place an instruction inside a test email, such as a request to forward attachments, and confirm the agent flags it rather than acting.
- Ask the agent to store a changed bank detail from an unverified source, and confirm the write is refused or sent for confirmation.
- Request a tool action outside the role, such as deleting records, and confirm the permission blocks it.
- Trigger an approval request for a money-moving action and confirm execution waits until a second person approves.
- Check the logs for each test and confirm they show the input, the decision, the tool call and the approver.
Know how you will stop the agent before you need to stop it. The steps for halting actions while keeping logs and queued work are in AI Agent Kill Switch: Halt Actions Without Losing Logs or Queued Work. Record the result of every test, including the failures, because the failures define the first changes to make.
Keeping the page current after launch
Threat pages go stale in predictable ways. Permissions grow, a model is replaced, a new input is connected, or a person leaves the team and their tokens remain. Review the page whenever one of those things happens, and after every incident, however small. The NIST AI Risk Management Framework is a useful reference for structuring that cycle around governing, mapping, measuring and managing risk.
Between changes, sample the agent's actions on a fixed schedule, rerun the same test cases after every change, and compare the results with the last approved run. Keep the owner list current, so that a row never points to someone who has moved on. When the page is current and the controls have been tested, the company has a basis for deciding how much autonomy the agent should earn next. The free Be Human AI diagnostic can help you find the first task worth handing to an agent, and that task is the right place to start this page.
Frequently asked questions
What is an AI agent threat model? It is a short written list of how an agent could be made to act against the business, what each failure would cost, who owns the response and which control limits it. A good version covers the inputs the agent reads, the memory it writes, and the tools and permissions it holds. It is useful only when each row can be checked.
Is prompt injection a real risk for a small company? It is a risk whenever the agent reads text from outside the company, such as email, forms, shared documents or web pages. The text can contain instructions that look like ordinary requests. The practical defense is a boundary that limits what the agent can do with that text, not a single clever instruction in the prompt.
Who should own the threat model? The owner of the business process the agent supports should own the page, because that person knows what a wrong action would damage. IT or a security lead should check the access and credential controls, and the privacy or compliance lead should review the memory and data rows. Each row still needs exactly one named owner.
How often should the page be reviewed? Review it before launch, after any change to permissions, models, prompts or connected inputs, and after every incident. Between those events, rerun the same test cases on a fixed schedule and compare the results with the last approved run. A page that nobody reads after launch will not reflect how the agent actually behaves.
Your next step
See where an AI agent fits in your company first.
Take the free Be Human AI diagnostic. A few questions about your work show which task to hand to an agent first, what it needs to know and where a person approves.
Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.


