BlogAI agents

How to build long-term memory for an AI assistant

Long-term memory for an assistant is a small data system. Decide what is worth keeping, write each memory as a short dated statement tied to one user, fetch the relevant few at the start of a turn, and update or delete them as things change. Rows in the database you already run are enough to start. The hard parts are choosing what to save and handling contradictions, and a vector store solves neither.

Decide what is worth remembering

Most memory projects go wrong at the first step, by saving too much. A store full of passing remarks makes retrieval worse and gives users the uneasy sense of being watched.

We use one running example: an assistant for a travel agency that books trips for a few thousand regular customers. A customer says many things in a booking conversation. Few deserve to outlive it.

A candidate memory should pass four tests.

  1. It would change a future answer. ‘Prefers an aisle seat’ does. ‘Was in a hurry today’ does not.
  2. It is likely to hold for weeks or longer.
  3. The user would expect it to be remembered, and would not be alarmed to see it written down.
  4. No system of record already holds it. Loyalty numbers and past bookings live in the booking system, so look them up.

That leaves a short list of kinds: standing preferences, standing constraints (never takes overnight flights), stable context (home city, usual travelling companions) and decisions that explain later behaviour. Anything about health, beliefs or other people needs a deliberate decision, which we cover in AI memory and privacy. For the vocabulary of memory types, see types of memory in AI systems.

How memories get written

There are three places to put the write.

Write path When it runs Strength Weakness
The model calls a save tool During the turn Immediate, and the user can see it happen Adds latency; the model forgets to save, or saves too much
Extraction after the turn or session Just after the reply No delay for the user; sees the whole exchange Needs a reliable end-of-session trigger; one more model call
Background consolidation On a schedule Can merge duplicates and generalise across sessions Memories arrive late; harder to debug

LangChain’s memory documentation draws the same line, between writing in the hot path and writing in the background, and sets out the same trade-off.

Our default is a mix. Give the model a save tool for things the user explicitly asks to be remembered, because the user expects an immediate ‘noted’. Run extraction when the session ends for everything else. Add a consolidation job later, if duplicates become a problem.

The extraction prompt decides the quality of the whole system. Four rules:

  • Each memory is one self-contained sentence that names its subject. ‘Prefers aisle seats on flights’, not ‘prefers that’.
  • Each memory cites the message it came from, so that a person can check it.
  • Stated and inferred are separate labels. ‘I’m vegetarian’ is stated. Ordering the vegetarian meal once is, at most, an inference.
  • ‘Nothing to save’ is a valid output, and should be the most common one.

Where to store them

Store Suits Watch for
Rows in a relational database Profiles and typed facts, filtered by user and kind You design the schema yourself
Vector index Many free-text memories where wording varies Similar is not the same as relevant; hard to audit
Knowledge graph Relationships between people and things; facts that change over time Cost to build and run; extraction errors become edges
Plain files Notes an agent keeps for itself; readable and easy to diff Weak querying; concurrent writes

We start with a table in Postgres and add a vector column when search needs it, for the reasons given in vector database or pgvector. One user’s memories number in the tens or low hundreds. Finding the right ones is a filter and a sort, with no need for an approximate index. A knowledge graph earns its cost only when relationships are the product: who travels with whom, which policy covers which traveller.

create table memory (
  id           uuid primary key,
  tenant_id    uuid not null,
  user_id      uuid not null,
  kind         text not null,     -- preference | constraint | context | decision
  subject      text not null,     -- e.g. home_airport, seat, diet
  content      text not null,     -- one self-contained sentence
  source       text not null,     -- user_stated | inferred | imported
  source_ref   text,              -- conversation and message id
  valid_from   timestamptz not null default now(),
  valid_to     timestamptz,       -- set when superseded; null means current
  last_used_at timestamptz,
  expires_at   timestamptz,
  embedding    vector(1024)
);

Three columns do most of the work. subject lets you find the memory that a new one replaces. source and source_ref let anyone trace a memory to the words that produced it. valid_to lets you retire a fact without losing the history.

Per-user isolation

Every row carries a tenant and a user, and every query filters on both. The values come from the authenticated session on the server. They never come from the model’s output or from anything said in the conversation. Enforce the filter in the database with row-level security as well as in application code, and test the boundary on every build. Memory shared across a team is a separate feature with its own rules, and should look different to users.

Retrieval at the start of a turn

Reading has a budget. We give memory a fixed share of the context, often a few hundred tokens, and fill it in two passes.

First, the core: a handful of current memories that apply to almost everything, such as home city and language. These are loaded on every turn.

Second, the relevant: search the rest using the user’s message, and take the top few. Combining keyword and vector search works better than either alone, as we explain in hybrid search and reranking. Filter out anything superseded or expired before ranking.

Then present the memories to the model as dated data, and say what they are:

Saved facts about this customer (may be out of date; the current message takes priority):
- [2026-03-02, stated] Flies from London Heathrow.
- [2026-05-19, stated] Prefers aisle seats on flights.

The dates and the caveat matter. Without them the model treats a two-year-old remark as today’s instruction. For the long tail, give the model a search tool so that it can look for more when a question calls for it.

Updates, contradictions and decay

Here is the case every memory system has to survive. In March the customer says: ‘I always fly from Heathrow.’ In September: ‘We’ve moved to Leeds, so Manchester is easier now.’

Extraction produces a new memory with the subject home_airport. Before saving, the system fetches current memories with the same subject, or the nearest few by similarity, and asks a model to classify the new one against them.

  • Duplicate: discard it, and refresh the date on the existing row.
  • Update: set valid_to on the old row and insert the new one as current.
  • Contradiction with no clear winner: keep both, mark them, and have the assistant ask at the next relevant moment.

In our example the Heathrow row gets an end date and the Manchester row becomes current. The old row is kept. When the customer asks why April’s trip went from Heathrow, the history explains it. It is never retrieved for a new booking.

The tools on the market disagree about this step, which tells you it is not settled. Google’s Memory Bank consolidates: it merges new information into existing memories and keeps revisions. Graphiti gives each fact a validity window and invalidates old facts instead of deleting them. Mem0’s current algorithm, introduced in April 2026, only adds: memories accumulate, and retrieval takes time into account. We prefer explicit validity dates because they are easy to inspect.

Decay needs the same care. Give each kind a default lifetime: a note about one trip expires when the trip ends, and a preference that has not been used for a year is dropped or confirmed with the user. Prune on last_used_at. When a user asks for something to be forgotten, delete the row and its embedding outright.

Should you build it or use a memory tool?

The options at the time of writing, from their own documentation:

  • Mem0 is an open-source library and self-hosted server, with a hosted platform. On each add, a model extracts facts from the messages, and memories are scoped by user, agent, app and run identifiers.
  • Graphiti, from Zep, is an open-source Python framework that builds a temporal graph and needs a graph database such as Neo4j or FalkorDB. Zep sells a managed service built on it.
  • Letta is an agent framework in which memory blocks are labelled sections pinned in the context window, which the agent rewrites with tools.
  • LangGraph provides a store of JSON documents under namespaces. The extraction logic is yours to write, or to add from a library.
  • Memory Bank on Google’s Agent Platform is a managed service with extraction, consolidation, isolation by scope and expiry.
  • AgentCore Memory on AWS stores session events and extracts long-term records using strategies you choose. If you configure none, nothing long-term is extracted.

Anthropic’s file-based memory tool, described in AI agent memory explained, is a further route when the agent should manage its own notes.

Use a tool when several agents need memory quickly, when facts that change over time are central, or when you are already on that cloud. Build it yourself when you have one assistant, need to audit every write, or have strict rules on where data lives. Either way, you still own the decisions in the first section. Treat vendors’ published benchmark scores with caution, and measure on your own conversations.

How do you know memory is helping?

Offline, write multi-session scripts. Session one plants a fact. Session three needs it. Session five changes it. Session six needs the new value. Score four things: the fact was used when it mattered, it was left out when irrelevant, the old value was not used after the change, and the assistant asked when it lacked a fact instead of guessing. Thirty scripts drawn from real conversations will find most faults. The wider method is in how to evaluate AI agents.

Online, run memory on and off for comparable users and watch how often users repeat themselves, how many turns a task takes, and how often they correct the assistant with something like ‘no, I told you’. Count the errors that memory causes as a separate number. A memory feature that removes ten repeated questions and introduces one confident wrong booking is not ahead.

And read the store. Every week, sample fifty memories and label each one as worth keeping or junk. If a large share is junk, fix extraction before touching retrieval.

How to start

Week one: create the table, define five to ten explicit profile fields for your product, and build the page where a user can see and delete what is stored. Ship that with no extraction at all.

Week two: add end-of-session extraction for one kind of memory, with the source message attached, and review every write by hand for a fortnight.

Only then add search, the update classifier and expiry. The first decision to make is the one in the opening section: write down the kinds of fact you will keep and the kinds you will refuse. If you want this built into an assistant, it is part of the conversational AI work we do at WeStack AI.

Questions

Asked often.

What is the best way to store long-term memory for an AI assistant?

Start with rows in the relational database you already run: one row per memory, with the user, a subject, the text, its source and dates. Add a vector column when free-text search is needed. A knowledge graph pays off only when relationships between people and things, or facts that change over time, are central to the product.

Do I need a vector database for AI memory?

Usually not at first. A single user rarely has more than a few hundred memories, which can be filtered by user and searched with a simple query, or with a vector column in Postgres. A dedicated vector database becomes useful when memories number in the millions or are shared across large teams.

How should an assistant handle contradictory memories?

Before saving a new memory, look up existing ones on the same subject and decide whether the new one repeats, updates or contradicts them. For an update, mark the old memory as superseded with an end date and save the new one as current. Keep the old row for audit, and exclude superseded rows from retrieval.

How do you test whether memory improves an assistant?

Write multi-session test scripts in which one session plants a fact, a later one needs it and another changes it. Score whether the assistant used the fact when it mattered, ignored it when it did not, and stopped using the old value after the change. Then compare live sessions with memory on and off.

Not on the list? Ask us directly

Keep reading

More notes from the build.

All blogs

Let’s build your stack.

Send a brief and we’ll reply with next steps.