Sensitive Data in AI Agent Memory: What Should Never Be Stored

Sensitive data in AI memory should be limited to what the agent's role truly needs, and a short list of categories should not be stored at all: credentials, full payment and identity numbers, health details, private personal circumstances, employee matters and other parties' confidential material. You enforce that with three layers that work together: a written memory rule, a filter that runs before anything is saved, and a recurring review of what the memory actually holds. A rule the system cannot check is only a hope, so every category needs a mechanism behind it.

Key takeaways

• Memory persists, so a sensitive detail saved once can resurface in later sessions, for other users and in other channels.
• Decide the never-store categories before the agent goes live, and write down a narrow alternative for each, such as a reference instead of a value.
• Put the main control in the system that writes memory, not only in the agent's instructions.
• Review stored memory on a schedule, because filters miss things and people paste unexpected material.
• Using sensitive data inside one task is different from keeping it. Design for use without retention.

Why does persistent memory change the risk?

A chat window forgets when the session ends. Agent memory is built to do the opposite: it keeps facts so the next conversation starts with context. That is the value, and it is also the risk. A detail typed in a hurry, such as a password pasted into a support thread, can be summarized, stored, retrieved next month and shown to someone who was never meant to see it.

Three properties make stored data harder to manage than data in a single conversation. It lasts longer than the task that created it. It travels, because memory can be retrieved in other sessions and, depending on design, by other people (the question explored in shared vs personal memory). And it gets rewritten, because summaries turn raw text into short notes that are easy to overlook in a review.

Security and risk guidance treats this as a known problem. The OWASP Top 10 for LLM Applications lists sensitive information disclosure among its risks, and the NIST AI Risk Management Framework treats privacy as part of building trustworthy AI. Neither tells you which facts your agent should hold. That decision belongs to you, and it is easier to make before launch than after.

Which categories of data should an agent never store?

The list below is a starting point. Adjust it to your sector and the role of the agent, but keep the default as no. Anything a role needs must be added deliberately, with an owner and a reason.

• Credentials and secrets: passwords, access tokens, API keys, recovery codes and private keys. Tools should reach systems through proper authorization flows, so the agent never needs to remember a secret.
• Full payment and identity numbers: card numbers, bank account details, national identity numbers, passport numbers and tax identifiers. If the role needs to refer to them, store a pointer such as payment method on file in the billing system.
• Health, biometric and similar special categories: medical conditions, disability, genetic or biometric data. These stay out unless a role is designed, approved and governed specifically for them.
• Private personal circumstances: family situations, financial hardship, religion, politics, relationship status. People mention these in passing, and passing remarks should not become permanent records.
• Employee matters: performance notes, disciplinary history, pay and complaint details, unless the agent is an HR role with matching access rules.
• Other parties' confidential material: another client's contracts, pricing, drafts or internal documents. The practical checklist in client data isolation for AI agents covers how to keep these walls intact.
• Privileged and regulated records: legal advice, investigation files and anything under a duty of confidentiality.
• Raw attachments and pasted threads: screenshots, full email chains and scanned forms, which carry sensitive details nobody chose to include.
• Inferences about individuals: guesses about mood, honesty, creditworthiness or personality. An agent's opinion of a person is not a fact, and it should not be saved as one.

Notice that several items are not secrets in the narrow sense. A family detail told in a friendly message is not dangerous in the moment. It becomes a problem when it sits in memory with no purpose, no expiry and no owner.

How do you decide whether a fact is safe to remember?

Borderline cases will come up daily, so give the system and the people reviewing it the same short decision guide. A fact is saved only if every answer is acceptable.

• Does the role need this to do its work next week? If not, do not store it.
• Can a pointer replace the value? Store where the data lives, not the data itself.
• Who could retrieve it later? If the audience is wider than the person it concerns and the owner of the role, do not store it.
• Can the person it concerns have it corrected or removed? If there is no route for that, do not store it.
• Does it have an expiry or a review date? If not, set one before saving.
• Would the owner be comfortable seeing it in a memory review? If the answer is no, the answer to storing it is also no.

This guide also separates use from retention. An agent may need a card statement for one task, such as explaining a failed charge in the moment. Using it inside the task is a different decision from writing it into memory, and the second one should need its own justification. A source of truth outside the agent, such as the billing system, lets the agent point to the data without copying it.

How do rules, filters and reviews enforce the policy?

Each layer catches what the others miss, so use all three.

Rules are the written policy. They name the never-store categories, give the allowed alternative for each, and name the person who owns the policy. Put the rule in the agent's instructions too, but do not treat that as the control. Instructions can be ignored, misread or overridden by hostile text inside a document or message, which is why prompt injection is treated as its own risk in the OWASP guidance.

Filters sit in the harness, the layer around the model that decides what gets written, retrieved and logged. A memory write should be a controlled action, not free text appended to a file. Useful filter behaviors include:

• Detecting recognizable formats before saving, such as card-like numbers, identity number patterns and token or key shapes, and redacting or rejecting them.
• Using a second check that classifies a proposed note against the never-store categories before it is written.
• Quarantining a flagged note for human review instead of silently dropping it, so the team learns what slips through.
• Applying the same filter to document ingestion, tool outputs and summaries, because sensitive data enters through all of them.
• Limiting who and what can read memory, following least privilege and deny by default, as in the OWASP authorization guidance.

Reviews are the recurring human check. Filters will miss unusual formats and context-dependent details, so someone must read samples of real memory on a schedule, search for the categories above, work through the quarantine queue and update the filter patterns. The method in AI agent memory audit describes how to run that review. When a review finds something, the fix is deletion with a record, and how to delete AI agent memory covers the difference between correcting, expiring and removing.

One more place deserves a look: memory is not the only store. Transcripts, logs, evaluation datasets and backups can hold the same details. Apply the same categories and retention limits to them, or the policy only moves the problem.

A step-by-step plan for the first month

You can set this up without a large project. Work in this order.

• Define the role in one sentence and list the facts it genuinely needs to remember. Everything else is out of scope by default.
• Write the never-store list for that role, with the pointer or summary allowed in each case.
• Name an owner for the policy and a backup person who can approve exceptions.
• Turn memory writes into a controlled action with a fixed note format, and add format detection for the clearest cases such as credentials and payment numbers.
• Add a quarantine step for flagged notes and decide who looks at it and how often.
• Test before launch with fake data: paste invented card numbers, fake tokens and invented personal stories, and confirm nothing sensitive is stored, logged or summarized.
• Set a review rhythm, with sample reads of memory and a check of logs, and record every change to rules and filters.
• Give users a plain way to say remove this, and make sure that request reaches a real deletion process.

The test step matters most. Teams often confirm that the agent behaves well in a demo but never check what was written behind the scenes.

What does this look like in practice? An illustrative scenario

Illustrative scenario: a support agent for a subscription business handles messages from customers. A customer writes that a payment failed and pastes the full card number to help. The write filter recognizes the card format and blocks the note. The agent replies that it cannot use card details in this channel and points to the secure billing page. The stored note reads: payment failed, customer directed to billing, resolved by the customer. No number reaches memory, the summary or the log.

Later the same customer explains that a medical situation caused the delay and asks for more time. The agent saves only that an extension was requested and who approved it. The reason is not stored, because the role needs the decision, not the diagnosis. If the request needs a person, the agent hands it over with the facts needed, in line with human escalation for AI agent failures, and the human decides what, if anything, to record in the proper system.

In the weekly review, the owner reads a sample of notes, checks the quarantine queue and finds one note with a pasted email chain that the filter did not catch. She removes it, adds a pattern for that case and records the change. That small loop is what makes the policy real.

How does Be Human handle sensitive data in agent memory?

Be Human defines memory as part of the role in the first engagement: what the role needs to remember, which categories it must never hold, and where approval is required. The memory rules, write filters and permissions are implemented in the harness around the agent, and under managed operations we run recurring reviews of stored memory, permission reviews and controlled improvements to the filters. The client keeps control of the policy and the exceptions, and every change to either is recorded.

Frequently asked questions

Is it enough to tell the agent not to remember sensitive data? No. Instructions are one layer, and they can fail through misreading, unusual input or hostile text inside a document. Put a filter in the system that writes memory and review what was stored, so the policy does not depend on the model behaving perfectly.

Can an agent use sensitive data during a task without storing it? Yes, if the design allows it. The agent can read data from the system of record for one task and discard it, keeping only a note such as the action taken. Check that logs and summaries do not capture the same data, because that is the most common way retention happens by accident.

What should we do if sensitive data is already in memory? Remove it, record what was removed and why, and look for copies in summaries, logs and backups. Then find how it got in, and add a filter or rule to close that path. A review routine like the one in the memory audit guide helps you find similar cases before they accumulate.

Does this replace legal or data protection advice? No. This article describes engineering and operating controls, not legal requirements. Your obligations depend on your sector, your clients and your jurisdiction, so involve your legal or data protection contact, and in Brazil follow the guidance of the ANPD, when deciding which categories your policy must cover.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.