Memory Architecture for Long-Lived Agents

Ronni Holmvig Strøm · 2026-05-17

Most AI demos still have the shape of a séance. A user arrives, a model appears to know them for a while, the conversation becomes useful, and then the room is cleared. The next session begins with the same polite amnesia. For a one-off assistant, this is irritating. For an agent expected to live

We have been running a persistent memory architecture in production for months now, across two sides of the same agentic platform. One side is entity-facing: the AI that talks to humans, remembers who they are, and forms continuity through repeated interaction. The other is code-facing: the engineering agents that build and maintain the platform itself. The content of those memories differs. The architectural discipline is the same.

The central lesson is simple enough to sound banal until it is priced. Persistent memory is not primarily a storage problem. Storage is cheap. The hard problem is deciding what an agent must carry in active context on every call, what it should retrieve only when the work reaches for it, and what should remain stored until a slow consolidation process decides that a passing event has become a durable fact.

Memory Is A Context Budget, Not A Database

The easiest way to add memory to an LLM system is to put more material into the system prompt. The second easiest way is to run every user message through a vector search and append the nearest results. Both approaches work for demos, and both collapse in different ways when the agent becomes long-lived.

Every token loaded into context is paid for on every call where it appears. It costs money, it costs latency, and it consumes attention. Anthropic’s context engineering work frames this in the right register: context is a finite resource with diminishing marginal returns, and agent builders should seek the smallest high-signal token set that supports the desired behavior. That becomes more important as agents run longer, because the possible universe of relevant past information grows faster than the context window can responsibly absorb.

This is why the memory problem changes when a system moves from conversation to companionship, or from coding autocomplete to an engineering agent that edits files over weeks. The question is no longer “how do we store everything the agent has learned?” That part is trivial. The question is which facts are expensive enough, important enough, and general enough to deserve residence in the always-loaded tier.

Our current production rule is severe by design. The always-loaded memory tier stays under roughly 10 KB of curated markdown. That tier is not a scrapbook. It is a compact operating manual for continuity: identity, standing instructions, durable preferences, architectural invariants, and a small number of lessons that every future session must know before it begins.

Everything else belongs elsewhere.

Three Bands Of Persistent Memory

The architecture has three bands: working memory, episodic memory, and semantic memory. The terms are familiar from both cognitive science and agent-memory research, but the useful part is not the naming. It is the cost boundary between them.

Working memory is the live token window. It contains the active conversation, tool results, current files, task instructions, and enough recent history for the model to act coherently. It is pruned with token-budget awareness. When a session grows long, older material is compacted through LLM summarization so the thread can continue without carrying every raw turn forward.