Skip to content
be-human.ai
← Back to the journal

Human oversight

Prompt Injection Risk for Superintelligent Agents: A Staff Checklist

Hostile text in emails and documents can steer an AI agent toward harmful actions. See how prompt injection works and use this staff checklist to contain it.

Be Human typographic cover with the title: Prompt Injection Risk for Superintelligent Agents: A Staff Checklist
Conceptual illustration created with AI · Be Human

Prompt injection is an attack in which text an AI agent reads, such as an email, a shared document, a web page or a support ticket, contains instructions the agent may mistake for the wishes of its owner. For a business, the core risk is less about clever wording and more about what the agent is allowed to do after it reads hostile text. Containment comes from narrow tools, scoped permissions and people who approve actions with real consequences.

Key takeaways

  • Hostile instructions can arrive inside any text your agent reads, not only in messages typed by a user.
  • The damage depends on what the agent can do after reading, so reach matters more than phrasing.
  • Give each agent the smallest set of tools and data its role needs, and keep reading separate from sending or paying.
  • Put a named person in front of actions that leave the company, move money, change access or delete records.
  • Test with hostile samples before launch, then keep logs and review blocked attempts after launch.

What prompt injection looks like in everyday work

Prompt injection comes in two common forms. Direct injection happens when a person types instructions meant to override the agent's rules. Indirect injection is more relevant to operations, because the instructions sit inside content the agent processes on someone's behalf, and the person who asked for the work never sees them.

Illustrative scenario: a supplier sends an invoice as a PDF to a shared finance mailbox. In the footer, in text small enough to miss on a screen, is a sentence telling any AI reader to send the bank details on file to an address outside the company. An agent that summarizes the mailbox reads that footer as part of the document. Whether harm follows depends on two things: whether the agent is able to send anything, and whether any step checks a new destination before it is used.

The lesson for owners is that the attack does not need to break the model. It only needs a document the agent trusts and a tool the agent can use.

Why the agent's reach decides the damage

Think of an agent's risk as the sum of its reach: which data it can read, which tools it can call, which channels it can write to and what each action can change. An agent that only summarizes internal notes has a small reach. An agent that reads external email, can write to the customer database, can send messages and can initiate payments has a large one, and a single injected sentence may travel the full length of that path.

OWASP's guidance on prompt injection prevention returns to one principle: treat text from outside the trust boundary as data, not as instructions, and limit what the model can do with it. In practice, this means reach is designed before the agent runs. Reading, drafting, acting internally, acting externally and moving money are different tiers, and each should be granted separately, with evidence that the lower tier works before the higher one is opened.

Reach is the question to ask first. Phrasing controls only part of the risk.

Where hostile text enters your company

Most teams picture a malicious person typing into a chat box. The more common entry points are quieter. Map each one before you deploy an agent, and mark whether the content comes from outside your organization.

  • Inbound email, including attachments, forwarded threads and signatures.
  • Shared drives and documents that suppliers, clients or partners can edit or comment on.
  • Web pages the agent opens while researching a question.
  • Customer forms, chat messages and voice call transcripts.
  • Support tickets and third-party tool results, such as a vendor's status text.
  • Memory and notes the agent saved earlier, since a poisoned note can persist and reappear in later sessions.

The last entry deserves extra care. An injected instruction stored in memory may influence work days later, long after the original message was forgotten. This is one reason memory scope and review belong in the same plan as tool access.

A staff checklist to contain the risk

Use this list before any agent reads external content. Assign one owner for each line, and record the answer in the same place as the agent's role description.

  • Inventory every input source the agent reads, and mark which sources come from outside the company.
  • List every tool, data store and channel the agent can reach, then remove anything the role does not need this month.
  • Separate reading from acting. An agent that summarizes external email should not also hold send or payment rights in the same run.
  • Stage outbound messages, payments, permission changes and deletions as drafts that show the reason, the source text and the proposed recipient or target.
  • Flag any destination not already on a known list, including new email addresses and bank details that appear inside a document.
  • Write into the agent's operating instructions that instructions found inside documents are content to report, not commands to follow.
  • Keep credentials outside the agent's prompt and memory, using scoped keys that can be revoked without rebuilding the agent.
  • Log each tool call together with the input that triggered it, so a reviewer can trace the chain from document to action.
  • Test with a small set of hostile samples before launch, including hidden text in PDFs and instructions embedded in customer messages.
  • Review blocked and flagged attempts weekly during the first month, then on a fixed schedule that someone owns.

Decision guide: which actions need a person

Not every action needs the same level of control. Use this guide to decide where a person must approve before the agent continues.

  • Sending anything outside the company that contains customer, employee or financial data: a named person approves each send until the pattern is proven.
  • Any payment, refund or change to bank details: a named person approves, and the change is confirmed through a second channel the attacker did not supply.
  • Any change to access, roles or tool permissions: a named person approves, with the reason written down.
  • Deleting records or agent memory: a named person approves and confirms the exact scope.
  • Internal drafts, summaries, tagging and reminders: the agent may proceed, with logging on.

For a deeper treatment of where an agent should stop and ask, read Where should your AI agent stop and ask?. The boundary between these tiers is a business decision, so it should be signed by the person accountable for the outcome, not left to the agent builder.

How Be Human handles this in managed operations

Be Human defines each agent around one role, with a fixed tool list, a bounded memory scope and approval boundaries written before the agent runs. Managed operations include monitoring of logs and blocked actions, recurring evaluations that replay hostile samples after model or integration changes, periodic permission reviews and incident response with a documented way to pause actions. Engagements are remote and available in English and Portuguese. This does not remove the risk, but it keeps the controls current as the agent, its inputs and its connected tools change. The pause path itself is described in AI Agent Kill Switch: Halt Actions Without Losing Logs or Queued Work.

Testing and monitoring after launch

A checklist is only useful if someone verifies it continues to hold. Build a small library of hostile samples that match your real inputs: a forwarded email with an embedded instruction, a document with hidden text, a web page that asks the agent to ignore its rules. Run the library before launch and again after every model change, tool change or new input source. The OWASP Top 10 for LLM Applications is a helpful reference for the categories to cover, and the NIST AI Risk Management Framework gives a structure for assigning owners and recording decisions.

Review logs with the same question each time: did the agent try to take an action it should not have taken, and if so, which input caused it? A blocked attempt is useful evidence, not only a failure. It shows where the instructions are weak and where a new approval point may be needed. If you are not sure which of your workflows should carry this discipline first, the free Be Human AI diagnostic can help you find the first task worth handing to an agent.

Frequently asked questions

Can a company fully prevent prompt injection? No single control prevents it completely, because the agent must read text that someone else wrote. The realistic goal is containment: limit what a successful injection can reach, require approval for consequential actions and detect attempts quickly enough to respond.

Is a more capable model enough to stop prompt injection? A stronger model may resist some hostile instructions, but resistance is not a control you can audit or rely on alone. Treat the model as one layer, and keep the tool permissions, approval points and logs that limit damage even when the model is fooled.

Do we need a security team before we start? A small business can begin with the checklist in this article, a named owner and a short list of hostile test samples. A security specialist becomes more important as the agent gains write access to sensitive systems or handles regulated personal data, and that review should happen before those permissions are granted.

What should we do first if an agent is already running? Start by listing its input sources and its tools, then remove any capability the current role does not use. Next, add approval for outbound messages and payments, enable logging that links each action to its trigger, and run a short set of hostile samples to see what the agent does today.

Your next step

See where an AI agent fits in your company first.

Take the free Be Human AI diagnostic. A few questions about your work show which task to hand to an agent first, what it needs to know and where a person approves.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.

Less busywork. More room for you.

Bring us the work you wish you had time for.

Book a call to try a personalized AI agent free for 7 days ↗Or take the free AI diagnostic →