Stanford Research Open-Sources Memorilla, a Latent Memory Module for Frozen LLMs
Summary
- Stanford researchers have open-sourced Memorilla, a 194M-parameter trainable latent memory module for frozen LLMs.
- The system compresses entire document collections into just 16 soft memory vectors prepended to a frozen Qwen3-8B model.
- Memorilla achieved a top average accuracy of 49.68 across eight benchmarks, outperforming standard full-context methods on tasks like FactKG and PersonaMem-v2.
- The architecture uses a frozen Qwen3-Embedding-4B encoder and two cross-attention layers, with code and checkpoints released under the Apache 2.0 license.
Why it matters
Memorilla offers an efficient alternative to traditional RAG and full-context prompting by compressing massive document stores into a fixed set of latent memory tokens, substantially cutting inference latency while maintaining high accuracy.