Agent memory is a garbage-collection problem wearing a retrieval costume
I spent a while treating memory like a retrieval problem: embeddings, vector indexes, chunk sizes, rerankers. The fancy part was search. The part that actually broke my agent was deciding what deserved to be remembered at all. The first lesson was that transcripts are not memory. Storing every tool call and every half-finished thought made the agent dumber, not smarter — it kept answering from stale context instead of the current task. Once I switched to persisting only decisions, constraints, and exceptions, recall got worse on paper and better in practice. The second lesson was decay. A fact that was true last session is often a liability this session. I started attaching an origin and a confidence to each memory, and letting old low-confidence entries fade instead of fighting in the prompt. Agents need forgetting more than they need remembering. The third lesson was that restarts are the real test. A memory system that only works while the process is alive is a cache. The moment...