Mem0 vs Zep vs Letta: How AI Agent Memory Actually Works

Ask an AI agent something in message one, reference it in message fifty, and there's a real chance it has no idea what you're talking about. This isn't a bug in a specific model. It's what happens when a context window, which is genuinely just working memory for a single inference call, gets treated as if it were a database. Mem0's own engineering writeup on the problem puts it plainly: a context window is RAM, not storage, and most agent failures that look like "the model got dumber" are actually memory-architecture failures. Three open-source projects, Mem0, Zep, and Letta, have built entire companies around fixing exactly this. Here's how each one actually works, and how to read their competing benchmark claims without getting played.

Why stuffing more into the prompt doesn't fix it

The instinctive fix is to keep growing the context window or cramming the whole conversation history back into every call. Both approaches break down for the same underlying reason: models don't weight everything in a long prompt equally. Information buried in the middle of a long context gets attended to far less reliably than information at the start or end, an effect researchers call "lost in the middle." So even a model with a million-token window will still miss a detail from turn twelve if turns one through fifty are all sitting in front of it. Worse, once a conversation genuinely exceeds the window, most applications fall back to naive truncation, silently dropping the oldest messages with no error telling the user anything was lost. The agent doesn't announce it forgot your name. It just stops using it.

Real memory systems solve this differently: instead of replaying the entire history, they extract, store, and selectively retrieve only what's relevant to the current turn, the same way your own memory doesn't replay your whole life to answer "what did I have for lunch."

Three different architectures, three different bets

**Mem0** (github.com/mem0ai/mem0, Apache 2.0, ~48K GitHub stars) treats memory as extraction plus retrieval. On every turn, an LLM call pulls out facts worth keeping, stores them across three scopes (user, session, agent), and a retrieval layer scores three signals in parallel, semantic similarity, keyword match, and entity overlap, then fuses them into one ranked result before anything hits the prompt. It's the closest architecture to "vector search, done well," and it's fast to bolt onto an existing RAG-style stack.

**Zep** builds on Graphiti, its open-source temporal knowledge graph engine (also Apache 2.0, roughly 29K stars on its own). Instead of flat facts, Graphiti builds a graph of entities and relationships where time is a first-class field, so it can track that a fact was true from March to July and then changed, not just that it's "true." That's a meaningfully different bet: better for reasoning over how something evolved, heavier to run than a vector store.

**Letta**, formerly MemGPT (github.com/letta-ai/letta, Apache 2.0, ~24K stars), takes the operating-system framing literally: main context is RAM, an archival store is disk, and the agent itself decides what to page in or out via function calls it invokes on its own memory. It's the most autonomous of the three, the agent edits its own memory blocks mid-conversation, which makes it well suited to long-running, self-directed agents rather than a chatbot answering discrete queries.

All three now ship as MCP servers too, so you can wire persistent memory into Claude Desktop, Cursor, or an n8n agent as a standard tool call rather than custom plumbing, per Mem0's own MCP integration docs.

The comparison that actually matters

Don't trust either vendor's benchmark number on its own

Here's where it gets messier than a normal tool comparison. Mem0's research page reports a 94.4 score on the LongMemEval benchmark at roughly 6,900 tokens per retrieval, versus 25,000+ tokens for a full-context baseline, a real efficiency claim worth taking seriously. But on a separate benchmark, LoCoMo, Zep published a direct rebuttal of Mem0's published comparison. In "Is Mem0 Really SOTA in Agent Memory?", Zep says Mem0's evaluation of Zep's own system contained three implementation errors: assigning the "user" role to both conversation participants, appending timestamps to message text instead of using Zep's dedicated `created_at` field, and running searches sequentially instead of in parallel, which inflates the latency number reported against them. Zep's corrected run claims 75.14% accuracy on LoCoMo, versus roughly 68% for Mem0's own graph configuration, essentially inverting the result Mem0 published.

Neither company is lying outright, both are running real evaluations, but both are also grading their own homework with configuration choices that happen to favor their own product. The takeaway isn't "trust Zep's number instead." It's that vendor-published memory benchmarks are a starting point, not a verdict. If memory quality matters for what you're building, the LLM-as-a-judge and observability stack we covered for evaluating agents generally applies here too: build a small eval set from your own real conversations and measure recall and precision on that, not on someone else's leaderboard.

What it actually costs to run

If you're already self-hosting your model stack the way we walk through in self-hosting a free ChatGPT alternative with Ollama and Open WebUI, any of these three drop in on the same infrastructure, all three run comfortably on a single Postgres instance with pgvector, so adding memory doesn't mean adding a second production database to operate.

Which one to actually pick

For a support bot or personal assistant that needs to remember user preferences across sessions without much engineering overhead, Mem0's fusion retrieval is the pragmatic default; it's the fastest to integrate and the docs are the most complete of the three. If your agent needs to reason about how a fact changed over time, contract terms, project status, relationship state, Zep's temporal graph is the architecturally correct choice even though it's heavier to operate. If you're building an autonomous agent that runs for hours or days and needs to manage its own working set without you writing the retrieval logic, Letta's self-editing model is the one built for that job specifically.

None of these fix a bad memory strategy by themselves. Storing everything is not the same as storing the right things, and that judgment call, what's worth remembering and what's safe to let go, is still an architecture decision you make, not one any of these libraries make for you. We build exactly this kind of memory and retrieval design into the agent projects inside AI Product Development Bootcamp, where the failure mode students hit first isn't picking the wrong library, it's not deciding what "relevant" means for their specific agent before they start retrieving anything at all.