Ask an agent built in 2023 what you told it last week and it will cheerfully make something up, because it has no idea. The model's context window — however large — is working memory, not long-term memory: it holds what fits in the current prompt and forgets everything the moment the conversation ends or the window overflows. For a chatbot that answers one-off questions, that is fine. For an agent meant to assist you over weeks, remember your preferences, track a project, or reason about facts that change over time, it is a fatal limitation. Bigger context windows do not fix this; they just delay the forgetting and make each call more expensive. What agents need is a memory layer — a system that decides what to persist, structures it so it can be retrieved, and injects the relevant pieces back into the context when they matter.
By 2026, agent memory has become its own discipline with its own tools, benchmarks, and architectural debates. This guide surveys the landscape: why context windows are not memory, the three dominant architectural approaches (vector, graph, and temporal), and the leading open-source frameworks that implement them — Mem0, Cognee, Graphiti and Zep, and Letta/MemGPT. The goal is to leave you able to reason about what kind of memory your agent actually needs and which tool fits, rather than reaching for whichever framework trended last.
Why context windows are not memory
The seductive argument goes: context windows keep growing, so just put everything in the prompt. This fails for three concrete reasons. First, cost and latency scale with context. Every token in the prompt is paid for on every call, so an agent that stuffs a month of history into each request burns money and slows down linearly with how much it "remembers." Second, relevance degrades in a sea of tokens. Models attend imperfectly over very long contexts, and burying the one relevant fact among tens of thousands of irrelevant tokens measurably hurts retrieval and reasoning — the "lost in the middle" problem. Third, and most fundamentally, the window is ephemeral. When the session ends, the context is gone. Nothing persists to the next conversation unless something outside the model deliberately stores it.
A memory layer solves all three by inverting the approach. Instead of carrying everything, it stores information durably outside the context, and at each turn retrieves only the small, relevant slice to inject. The agent's prompt stays lean, the cost stays bounded, relevance stays high, and — crucially — memory survives across sessions. The interesting question is not whether to have a memory layer but how it should be structured, and that is where the approaches diverge.
Approach one: vector memory
The simplest memory layer stores facts as embeddings in a vector database and retrieves them by semantic similarity — essentially RAG applied to the agent's own history. When the agent learns something ("the user prefers dark mode"), it embeds and stores it; when it needs context, it embeds the current situation and retrieves the nearest stored memories. This is the foundation, and it works well for a specific job: personalization and recall of discrete facts.
Mem0 is the leading framework in this mold, and it is more sophisticated than a raw vector store. It offers a multi-tier system — user, session, and agent scopes — backed by a hybrid store that combines vectors with graph relationships and key-value lookups, and it does active memory management: extracting salient facts from conversations, consolidating them, and updating rather than blindly appending. For conversational personalization — an assistant that remembers your name, your preferences, your recurring tasks — this is often exactly right, and it is the strongest choice when the memory you need is essentially a well-managed set of facts about a user.
The limitation of pure vector memory is that it treats each fact as an isolated point. It can retrieve "the user works at Acme" and "the user is a CTO," but it does not inherently represent that these facts are connected, or reason across a web of relationships. When memory needs structure — when the relationships between facts matter as much as the facts — a graph enters the picture.
Approach two: graph memory
Graph-based memory stores information as a knowledge graph: entities as nodes, relationships as edges. Instead of a bag of independent facts, the agent's memory becomes a connected structure it can traverse, which unlocks reasoning that vector similarity cannot reach — multi-hop questions, "how are X and Y related," and synthesis across many linked facts.
Cognee exemplifies the graph-native approach with its ECL pipeline — Extract, Cognify, Load. It ingests data from many source types, "cognifies" it by building a knowledge graph of entities and relationships, and loads it into graph plus vector stores for hybrid retrieval. The result is memory as an active, queryable structure rather than a passive store, well suited to local-first, privacy-critical deployments where you want graph reasoning without cloud dependencies. When your agent needs to connect the dots across a body of knowledge — not just recall isolated facts — a graph memory like Cognee's is the architecture that supports it.
The strength of graph memory is exactly its structure, and its cost is that building and maintaining a graph is more work than dropping vectors in a store. Extraction has to identify entities and relationships correctly, and the graph has to be updated as new information arrives. For agents whose value depends on reasoning over connected knowledge, that cost is worth paying; for simple personalization, it is overkill.
Approach three: temporal memory
Graphs capture relationships, but a plain graph has a subtle blind spot: it represents what is true, not when it was true or how it changed. Real-world facts have histories — someone changes jobs, a project moves phases, a preference updates — and an agent that overwrites the old fact loses the ability to reason about change, while an agent that keeps both without temporal structure gets confused by contradictions. Temporal knowledge graphs solve this by attaching validity time to every fact.
Graphiti, the engine behind Zep, is the leading open-source implementation. Its edges are bi-temporal, tracking both when a fact was true in the world and when it was ingested, and — critically — when a fact changes, Graphiti does not delete the old one. It marks the previous edge invalid with a timestamp and records the new one, so history is preserved and point-in-time queries ("what was true as of last month?") are possible. It ingests data incrementally, adding episodes without recomputing the whole graph, which suits memory that must stay current cheaply. When your agent depends on facts that change over time and it matters that the agent reasons with the current truth while retaining history, temporal memory is the approach, and Graphiti/Zep is its clearest expression.
This temporal capability is the frontier of agent memory in 2026 precisely because so many real agent tasks involve evolving state. An agent tracking a customer relationship, a codebase, or a long project is drowning without it — every update either overwrites history or accumulates as contradiction. Temporal graphs give a principled answer.
Approach four: OS-style memory management
A fourth approach reframes the problem entirely. Rather than a separate store the application queries, MemGPT — now the Letta framework — models memory after an operating system. The context window is RAM: fast, small, holding what is active right now. Archival storage is disk: large, searchable, holding everything else. And the agent itself is the OS, deciding via tool calls what to page into main context and what to write out to archival memory. The agent edits its own always-in-context "core memory" blocks as it learns, and searches archival memory when it needs something it has paged out.
The elegance of this model is that memory management becomes the agent's own responsibility, exercised through tools, rather than logic bolted on by the application. This makes Letta especially suited to long-running autonomous agents that must maintain coherent state over extended operation with minimal external orchestration — the agent manages its own memory the way a program manages its own address space. The tradeoff is that you are trusting the agent's judgment about what to remember and retrieve, which works well when the agent is capable and the task rewards autonomy, and less well when you want tight external control over exactly what is stored.
Memory operations: extraction, consolidation, forgetting
Beyond the storage architecture, a memory layer has to manage what it stores, and this operational side separates a real memory system from a glorified log. Three operations matter. The first is extraction: turning raw conversation into storable memories. Not every sentence is worth remembering, and storing everything reproduces the context-window problem in a different place. Good memory systems extract the salient facts — preferences, decisions, entities, relationships — and discard the chatter, which is why frameworks like Mem0 do active fact extraction rather than dumping whole transcripts into a store.
The second is consolidation: reconciling new information with what is already stored. When an agent learns something that updates or contradicts an existing memory, naive systems either create a duplicate (so the store fills with near-identical facts) or blindly overwrite (losing history). Sophisticated memory layers detect that a new fact relates to an old one and consolidate — merging duplicates, updating values, or, in temporal systems, invalidating the old fact while recording the new one with a timestamp. This is the difference between memory that gets sharper over time and memory that degrades into a pile of contradictions.
The third, underrated, operation is forgetting. Human memory forgets adaptively, keeping what matters and letting irrelevant detail fade, and agent memory needs an analog. Without any pruning, a long-lived agent's memory grows without bound, retrieval slows, and stale facts pollute results. Deliberate forgetting — decaying low-value memories, archiving what has not been accessed, or capping memory size — keeps the system healthy. The frameworks differ in how much of this they automate versus leave to the application, and it is worth checking, because a memory layer that only ever accumulates is a memory layer that eventually degrades. When evaluating a framework, ask not just how it stores memories but how it extracts, consolidates, and forgets them, because that operational behavior determines whether memory quality improves or rots as the agent runs.
Choosing a memory layer
The decision follows from what your agent actually needs to remember and how. If the job is personalization and recall of user facts — an assistant that remembers preferences and history — start with Mem0; its managed, multi-tier vector-centric memory is purpose-built for that and the least heavyweight to adopt. If your agent must reason over connected knowledge, synthesizing across a web of related facts, choose a graph-native layer like Cognee, especially when local-first privacy matters. If your agent depends on facts that change over time and must reason with current truth while preserving history, choose the temporal graph of Graphiti/Zep. And if you are building a long-running autonomous agent that should manage its own memory with minimal orchestration, choose Letta/MemGPT.
These categories are not rigid — Mem0 incorporates graph relationships, Cognee blends graph and vector, and real systems often combine approaches. But the center-of-gravity framing is the useful one: match the memory architecture to the shape of what your agent must remember. A common mistake is reaching for a temporal knowledge graph when simple personalization would do, paying the complexity cost for capability you do not use; the opposite mistake is bolting a flat vector store onto an agent whose whole value depends on reasoning about change. Diagnose the memory need first, then pick the architecture that fits it.
The bottom line
Context windows are working memory, not long-term memory: they are ephemeral, they get expensive and unfocused as they grow, and they forget everything between sessions. Real agent memory lives in a dedicated layer that persists information outside the context and retrieves the relevant slice on demand, and in 2026 that layer comes in four flavors — vector for personalization (Mem0), graph for connected-knowledge reasoning (Cognee), temporal graph for facts that change over time (Graphiti/Zep), and OS-style paging for autonomous long-running agents (Letta/MemGPT). Diagnose what your agent actually needs to remember, match it to the architecture that fits, and your agent stops making things up about last week — because it actually remembers.
References and Resources
Frameworks
- Mem0 — GitHub and Cognee — GitHub
- Graphiti — GitHub and Zep
- Letta (MemGPT) — GitHub and the MemGPT paper
Background and analysis
- Best Open Source Agent Memory Frameworks 2026 — EverMind
- AI Agent Memory Frameworks in 2026: Memory vs. Context — Graphlit
- Best AI Agent Memory Frameworks in 2026 — Atlan
Related 1337skills cheatsheets