原文 · 保持原样，未翻译原文 · 未翻译
Computer Science > Artificial Intelligence
Title:$δ$-mem: Efficient Online Memory for Large Language Models
Jingdi Lei
Di Zhang
Junxian Li
Weida Wang
Kaixuan Fan
Xiang Liu
Qihan Liu
Xiaoteng Ma
Baian Chen
Soujanya Poria
Abstract:Large language models increasingly need to accumulate and reuse historical information in long-term assistants and agent systems. Simply expanding the context window is costly and often fails to ensure effective context utilization. We propose $\delta$-mem, a lightweight memory mechanism that augments a frozen full-attention backbone with a compact online state of associative memory. $\delta$-mem compresses past information into a fixed-size state matrix updated by delta-rule learning, and uses its readout to generate low-rank corrections to the backbone's attention computation during generation. With only an $8\times8$ online memory state, $\delta$-mem improves the average score to $1.10\times$ that of the frozen backbone and $1.15\times$ that of the strongest non-$\delta$-mem memory baseline. It achieves larger gains on memory-heavy benchmarks, reaching $1.31\times$ on MemoryAgentBench and $1.20\times$ on LoCoMo, while largely preserving general capabilities. These results show that effective memory can be realized through a compact online state directly coupled with attention computation, without full fine-tuning, backbone replacement, or explicit context extension.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.12357 [cs.AI] |
| (or arXiv:2605.12357v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2605.12357 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jingdi Lei [
Tue, 12 May 2026 16:31:44 UTC (609 KB)
Access Paper:
Current browse context:
References & Citations
Bookmark
