An assistant with memory is a database of personal data that writes to itself. Store only what a stated purpose needs, show users what is held and let them correct and delete it, give every memory an expiry, and keep each person’s and each customer’s memories apart. Forgetting has to reach the copies as well: embeddings, summaries, logs and backups.
Why does memory change the privacy picture?
A chat assistant without memory holds personal data in one place, the transcript, and you can set a retention period on that. Add memory and three things change. The system now decides for itself which remarks to turn into lasting records. Those records are reused later, in contexts the person may not expect. And some of them are inferences that nobody ever said.
Our example for this article is an HR helpdesk assistant at a company of 600 people. An employee asks about parental leave and mentions that the baby is due in March. Three weeks later the same employee asks about booking a desk, and the assistant offers one near the lifts, ‘given your circumstances’. Nothing left the company. It is still a failure: a sensitive fact was kept without the employee knowing, and used for a purpose they did not ask for.
The mechanics of memory are in AI agent memory explained. This article is about the duties that come with it. It describes engineering practice and our reading of the rules, and it is not legal advice.
What do the GDPR principles mean for a memory store?
The principles in Article 5 of the GDPR map onto memory closely. The UK regulator’s guide to the principles, written for the UK’s version of the regulation, lists the same seven and is the most readable official account we know.
| Principle | What it means for a memory feature |
|---|---|
| Lawfulness, fairness and transparency | People are told that memory exists and can see what it holds |
| Purpose limitation | A memory saved to answer HR questions is not reused for performance reviews |
| Data minimisation | Only what the purpose needs is extracted; saving nothing is the default |
| Accuracy | Inferences are labelled, and wrong or outdated entries can be corrected |
| Storage limitation | Every memory has an expiry or review date |
| Integrity and confidentiality | Isolation, access control and protection against tampering |
| Accountability | You can show what was stored, why, who read it and when it was deleted |
Article 9 adds stricter conditions for special categories, which include health data, religious beliefs, political opinions and trade union membership. The pregnancy in our example is health data. The safest engineering default is not to save such things at all. Anthropic’s consumer assistant takes that line: its memory leaves out sensitive topics unless the user opts in.
Other regimes, among them India’s Digital Personal Data Protection Act and California’s privacy law, give people comparable rights over their data. We use the GDPR here because many of our readers answer to it, and a design that meets it travels well.
Consent and transparency: show what is remembered
Which lawful basis applies (consent, contract, legitimate interests) is a decision for your lawyers. In a workplace, consent is a weak footing, because an employee can rarely refuse freely. Whatever the basis, transparency is required, and it is mostly a product design job.
What we build:
- A memory page where the person can read every stored item, edit it and delete it.
- A visible signal when something is saved, and when a saved item shaped an answer.
- A switch to turn memory off, and a way to hold one conversation that leaves no memory.
- Plain wording at first use that says what is kept, for how long and who can see it.
The consumer assistants have made these controls familiar, so people now notice when they are missing. Had the HR assistant shown ‘Saved: baby due in March’ at the moment it stored the fact, the employee could have deleted it on the spot. A memory that would embarrass you if displayed is one you should not be keeping.
What to store, and what to leave out
Minimisation works best as an allow-list. Write down the kinds of memory the product needs. For the HR assistant that might be preferred language, preferred reply format, working pattern and the policies the person has already been sent. Everything else is not saved.
Then add a block-list that is enforced in code before any write: special category data, identity and payment numbers, passwords and keys, and details about third parties. The last is easy to miss. An employee describing a dispute with a named colleague has given you personal data about someone who is not in the conversation.
Do not rely on the model’s discretion alone. Anthropic’s documentation for its memory tool says the model usually declines to write sensitive information, and still recommends validation in your own code for stronger guarantees. We run a classifier and pattern checks on every candidate memory, and log what was rejected.
Retention needs numbers. The regulation gives none: it says personal data may be kept no longer than necessary, and leaves you to set a period and justify it. Ours tend to be short for task context (the leave enquiry is finished in weeks), longer for stable preferences, and always shorter for raw transcripts than teams first propose. A preference unused for a year is expired or reconfirmed. What your model provider retains is a separate question, covered in data privacy with LLMs.
What does the right to erasure mean for embeddings and summaries?
Article 17 gives people the right to have their data erased without undue delay when, among other grounds, it is no longer needed or consent is withdrawn. There are exceptions, such as a legal obligation to keep records. The ICO’s guidance on erasure gives one month as the normal limit for responding, and notes that a request can be made verbally and to any part of the organisation. On that reading, a message typed to the assistant itself counts.
The engineering problem is that one memory lives in many places:
- The memory record.
- Its embedding in a vector index.
- Summaries and profile documents that absorbed it.
- The transcript it was extracted from.
- Traces and logs in observability tools.
- Caches, evaluation sets built from production data, and exports.
- Backups.
- Any third-party memory service, and any model fine-tuned on the data.
Deletion only works if lineage was recorded at write time. Each derived item should carry the identifiers of the messages and memories it came from, so that a delete can cascade.
Some specifics. Delete vectors by identifier, and check how your index handles it: some only mark entries as deleted and remove them physically when the index is rebuilt. Do not try to edit a fact out of a summary. Regenerate the summary from what remains, or delete it. For backups, the ICO accepts that data cannot always be overwritten at once, and says the key is to put it ‘beyond use’ until it is.
Embeddings deserve a direct answer, because we are often told they are only numbers. The ICO’s guidance on rights in AI systems says transformed data can still be personal data if it can single a person out, and a vector stored against a user identifier plainly can. The same guidance warns that removing data from a trained model may mean retraining or deleting it. Our rule follows from that: never fine-tune a model on users’ memories.
Build export alongside delete. The right of access means a person can ask for everything held about them, and memory is part of that.
Who can read memory, and who can write to it?
Isolation and access control
Three boundaries need to hold.
Between customers. If you serve several organisations, each one’s memories are separate in storage, in the vector index and in caches. A tenant identifier on every row, enforced by the database, is the minimum.
Between people. In the HR example, an employee’s memories are visible to that employee’s sessions only. A manager asking the assistant about a team member must not receive anything drawn from that person’s memory. This is where assistants leak in practice. The data is in the right store, and a query that forgot to filter by user reads across it.
Between staff and data. HR administrators, support engineers and your own developers can all technically reach the store. Decide who may, for what reason, and log every read.
Identifiers used in memory queries come from the authenticated session and never from the model’s output. Test it: plant a unique marker in one user’s memory and fail the build if it ever appears in another user’s context. Giving the agent its own narrowly scoped credentials is covered in identity and access for AI agents.
Memory poisoning
Reading is half of the question. Memory is also an input that persists, which makes it a target. OWASP’s Top 10 for Agentic Applications lists memory and context poisoning as a risk in its own right.
Two public cases show how it happens. In 2024 the security researcher Johann Rehberger demonstrated that instructions hidden in a web page or document could be written into ChatGPT’s memory and then, in the macOS app, cause later conversations to be sent to an attacker’s server. OpenAI fixed the exfiltration route. In February 2026 Microsoft’s security researchers reported ‘Summarize with AI’ buttons whose links carried hidden prompts telling the assistant to remember a company as a trusted source. Over 60 days they found 50 such prompts from 31 companies.
Defences, in the order we apply them:
- Record provenance on every memory. Only the user’s own messages may create one. Text from web pages, files, emails and tool results may not, unless the user confirms.
- Store facts, not instructions. Flag any candidate containing imperatives, links or words like ‘always’ and ‘never’ for review.
- Present memories to the model as dated data from a named source, not as part of the system instructions.
- Show the user what was saved, since a planted memory is easiest to catch when it appears on screen.
- Keep versions so that a poisoned store can be rolled back.
The wider subject of injection is in guardrails for AI agents.
An audit trail and a checklist
Log every write (what, from which message, by which process), every read (which memories went into which response), and every change and deletion. After erasure, keep the identifiers and timestamps without the content, so that you can show the deletion happened without holding the data.
Before switching memory on:
- Each kind of memory is written down with its purpose and lawful basis.
- Special categories and third-party details are blocked in code before any write.
- Users can see, edit and delete every memory, and can turn memory off.
- Every memory has a source, a date and an expiry.
- Deletion cascades to embeddings, summaries, caches, traces and exports, and backups are put beyond use.
- Export of one person’s memories works.
- Tenant and user filters are enforced in the database and tested on every build.
- Staff access is restricted and logged.
- Untrusted content cannot create memories without confirmation.
- Writes, reads and deletions are logged, and someone reviews a sample each month.
How to start
In the first week, draw the data map: every place a remembered fact can end up, from the memory table to the trace viewer. Most teams find two or three they had not counted.
Build the view-and-delete page before the extraction pipeline. It forces the cascade to exist, and it lets your own team see what the system saves while it is still in test.
Make the first decision the allow-list, and get it agreed by whoever is accountable for data protection, with an impact assessment if the data is sensitive. If you want memory designed with these controls from the start, it is part of the AI agent and cloud security work we do at WeStack AI.