The four types, how the major frameworks differ, and how to add real memory to an agent you're building.
By Anthony Conti · Astra AI, LLC ·
Agent memory is the mechanism that lets an AI agent retain information — facts, preferences, past actions, and outcomes — across turns, sessions, and tools, instead of starting from zero every time its context window resets. An agent without memory can reason brilliantly inside one conversation and remembers nothing about the person or task the moment that conversation ends. Memory is the layer that fixes that, and it is a genuinely separate concern from the model, the framework, and the context window.
Why this matters more in 2026 than it did in 2023.
Early chatbots got away with no memory because a session was the whole product. Agents now run multi-day tasks, hand off work between tools, and are expected to remember a decision made three weeks and two model switches ago. Context windows grew — many models now offer well over 100,000 tokens, and some exceed a million — but a bigger window is still working memory: it resets the instant the call ends. None of that solves persistence.
The four kinds of memory an agent can have
Borrowed loosely from cognitive science, and useful because most "add memory to my agent" questions are really "which of these four do I actually need?"
Type
Lives in
Survives
Example
Working memory
The context window — the tokens sent to the model on this call
Nothing past this conversation, and often not even the whole conversation once it's long
The last ten messages of the chat you're in right now
Long-term / semantic memory
External storage — a vector database, a knowledge graph, a document store
Sessions, restarts, and (if the storage is shared) the specific agent that wrote it
"This user prefers TypeScript over JavaScript" — retrieved months later
Episodic memory
A log of specific past events, usually timestamped
As long as the log is kept — often pruned or summarized over time
"On March 3rd we tried Redis and rolled it back because of latency"
Procedural memory
Learned patterns about how to do something, not what happened
Indefinitely, and usually generalizes across many episodes
"This codebase always wants tests in the same PR as the feature"
Most real systems only need working memory plus long-term/semantic memory to cover the common cases. Episodic and procedural memory matter more for agents that operate over long, multi-day tasks where the specific sequence of past events, not just the facts extracted from them, changes what the agent should do next.
Is agent memory the same as RAG?
No, though the two are frequently built together and get conflated because of it. Retrieval-augmented generation is a generation technique: fetch relevant chunks, put them in the prompt. Agent memory is the broader system responsible for deciding what an agent should retain, consolidate, or forget over time — RAG is typically the retrieval mechanism that memory system uses at answer time, but memory also covers the write path (what gets stored and when) that RAG alone says nothing about.
Why can't a bigger context window just replace memory?
Because a context window is working memory, and working memory has never been what persistence problems are about. Even a model with a million-token window forgets everything the moment a new conversation starts unless something outside that window wrote the relevant facts down first. Stuffing an entire history into every prompt also gets slow and expensive long before you hit a hard token limit — most production agents summarize or retrieve selectively well under the max, which is a memory-system decision, not a context-window one.
How the major frameworks handle it differently
There is no single "agent memory" implementation — five well-known projects take meaningfully different approaches, verified against each project's own documentation this session:
An extraction-and-consolidation layer — an LLM decides what from a conversation is worth remembering, then stores it with vector + optional graph indexing
A temporal knowledge graph — facts are nodes and edges with validity windows, so the system can reason about what was true when, not just what's true now
Protocol-level — memory exposed as MCP tools any compatible agent can call, independent of which framework built the agent
How do you add memory to an agent via MCP?
Instead of wiring your agent to one framework's specific memory API, you point it at an MCP memory server — an external process exposing memory as standard MCP tools (store_memory, search_memory). The advantage over a framework-native memory module: the same memory becomes reachable from any MCP-compatible client, not just the one framework you built the agent in. The practical setup takes about 10 minutes — see the setup guide for exact config.
Scope is the other decision worth making explicitly before wiring anything up. Memory can be scoped per-user (every fact tied to one person, the default most products assume), per-agent (a fact one specific agent learned, not shared with others acting on the same account), or global (shared across every agent and every user on an account, which is rare and usually only appropriate for organization-wide facts). Getting this wrong in either direction — leaking one user's memory into another's context, or siloing memory so tightly that no two agents can ever share it — is a more common failure than picking the wrong storage backend.
A short checklist for choosing an approach
One agent, one framework, never leaves it? A framework-native memory module (LangGraph checkpointing, Letta's built-in archival memory) is the least friction.
Multiple agents or tools need the same memory? An MCP memory server is the right layer — the memory becomes reachable independent of which framework wrote or reads it.
Need to reason about what was true when, not just what's true now? A temporal knowledge graph (Zep/Graphiti) is purpose-built for that; most vector-only stores are not.
A person, not only an agent, needs to see and edit the memory? That rules out pure developer-infrastructure options — you need a product with a UI on top, which is the gap MIND was built for.
Common mistakes when adding memory to an agent
Most agent-memory bugs are not exotic — they are one of the same handful of mistakes, repeated across nearly every framework and storage backend, and nearly all of them show up as the same symptom: the agent confidently says something that used to be true, or something nobody actually told it, with no obvious way to tell where the wrong context came from.
Storing everything, forgetting nothing. An agent that writes every message to long-term memory builds a haystack, not a memory — retrieval quality degrades as duplicate and low-value facts pile up. Most production systems run an extraction step (an LLM decides what is actually worth keeping) before anything gets written.
No decay or consolidation. A fact from six months ago and a fact from six minutes ago usually should not carry equal weight at retrieval time. Systems that never expire, merge, or re-rank older memories tend to surface stale context alongside current context with no way to tell them apart.
Treating memory as a cache instead of a source of truth. If the memory store can silently be wrong (a preference the user later changed, a fact that was corrected), and nothing in the system ever revisits or overwrites it, the agent will confidently repeat outdated information — often more confidently than it would have guessed without any memory at all.
No write-side review for anything sensitive. An agent with unrestricted write access to memory can also be an agent that stores something it should not have, permanently, with no human ever reviewing what went in. Any memory system that will hold real personal or business information needs a visible way to audit and delete what it stored.
Coupling memory format to one framework. Building memory storage directly into a LangGraph checkpoint or a Letta archival store works fine until you need a second agent, built in a different framework, to read the same facts. This is the specific problem an MCP memory server is built to avoid — the storage format stops being tied to any one framework's internals.
No, though the two are often built together. RAG is a generation technique — retrieve relevant text, put it in the prompt. Agent memory is the broader system responsible for deciding what an agent should retain, forget, and update over time; RAG is frequently the retrieval mechanism that memory system uses at answer time, but memory also covers what to write, when to consolidate duplicate facts, and how to expire stale ones.
Why can't a bigger context window just replace memory?
A larger context window (many models now offer well over 100,000 tokens, and some exceed a million) buys you more working memory per call — it does not persist anything after the call ends, and stuffing an entire history into every prompt gets slow and expensive well before you hit the limit. Memory is the system that decides what's worth carrying forward at all, independent of window size.
Do I need a framework to add memory to my agent?
No. The framework (LangGraph, Letta, CrewAI, a bare loop you wrote yourself) decides how the agent plans and acts. Memory is a separate concern you can bolt on via an MCP memory server regardless of framework — that's the point of a protocol-level integration instead of a framework-specific one.
What's the difference between agent memory and a second brain?
Mostly audience. "Agent memory" is the developer-facing term for the same underlying idea a "second brain" describes for a person: information that persists and gets recalled at the right moment instead of re-explained every time. Our /second-brain page covers the consumer-facing version of this same question.
MIND is an MCP memory server backed by a real knowledge graph — the memory layer Claude, GPT, Gemini, Cursor and Windsurf all read from. Free tier, no credit card.