Episodic Memory - The Third Stage of the LLM Lifecycle

Ronni Holmvig Strøm · 2026-05-14

Deployment-time learning begins from a deceptively simple observation: most deployed language-model systems still behave as if learning ended before the first user arrived. The base model may be large, instruction-tuned, preference-shaped, and surrounded by tools, but the live system usually treats each new episode as an inference problem rather than as part of an accumulating history.

Two recent papers make that boundary harder to maintain. In CASCADE: Case-Based Continual Adaptation for Large Language Models During Deployment, Guo et al. formalise deployment-time learning (DTL) as a third stage of the LLM lifecycle, after pretraining and finetuning, and report a 20.9% macro-averaged success-rate improvement over zero-shot prompting across 16 tasks by allowing agents to reuse episodic cases during deployment (Guo et al., 2026). In From Storage to Experience, Luo et al. offer a complementary taxonomy of agent memory, tracing its evolution from Storage, through Reflection, into Experience (Luo et al., 2026). Read together, the papers describe a shift in where learning is happening. The weights remain fixed. The system does not.

That distinction matters for any architecture concerned with long-lived agents, human-AI collaboration, or the slow accumulation of continuity across interaction. If an agent can retain prior episodes, select among them under uncertainty, receive feedback, and improve future behaviour without updating model parameters, then memory is no longer merely auxiliary context. It has become part of the learning apparatus.

Deployment Becomes A Third Learning Stage

The conventional lifecycle of a large language model has been organised around two privileged phases: pretraining, where broad statistical structure is absorbed from large corpora, and finetuning, where behaviour is shaped toward instruction-following, alignment, and task competence. Deployment then arrives as a distributional exposure problem. The model is released into the world and asked to perform, while the world quietly changes underneath it.

CASCADE names the missing phase. Guo et al. define deployment-time learning as a process in which LLM agents improve from experience during deployment while the underlying model parameters remain fixed. This is not reinforcement learning over the base model. It is not online gradient descent hidden in production. The adaptation moves into the agentic layer: prompts, memory, retrieval, tools, decision policies, and feedback loops.

There is an obvious objection here, and it should be faced cleanly. One could say that deployment-time learning is in-context learning with a case bank and a bandit policy, given a more ambitious name. At the mechanism level, that criticism has some force. The naming move is partly rhetorical. But the architectural shift it points at is real: once case selection, retention, and feedback-sensitive retrieval persist across episodes, the live system has acquired a learning surface that is not reducible to a longer prompt.

This is an important relocation. In many real systems, especially those built on black-box API models, parameter updates are unavailable, expensive, or operationally undesirable. Yet the system still needs to adapt to changing users, changing task distributions, changing environments, and the peculiar local facts that never belonged in pretraining data in the first place. DTL treats this constraint as architectural rather than accidental. The learning surface becomes the system surrounding the model.

CASCADE implements this by giving the agent an explicit episodic memory of successful cases. When a new task arrives, the agent retrieves a relevant prior case, reuses and revises it to generate a solution, receives scalar feedback, and updates the retrieval policy. Successful cases are retained in a case bank. The base LLM remains fixed, but the agent’s future behaviour changes because its memory and selection policy have changed.

There is a quiet inversion here. We often speak as if the model is the intelligent object and the application layer is scaffolding. DTL makes the scaffolding adaptive enough that it begins to carry part of the intelligence.

Episodic Cases Turn Memory Into Policy

The technical move in CASCADE is to frame case reuse as a contextual bandit problem. Given a task context, the agent must decide which stored case to retrieve. Some cases are known to be useful, some are uncertain, and some may only become valuable in regions of the task distribution the agent has barely seen. The agent therefore faces the classic exploration and exploitation trade-off: use what has worked, or test what may work better.