跳到正文
The Decoder:AI News· Jonathan Kemper·· 1 小时前AI 评分56

Anthropic 谈智能体记忆:关键在于知道该忘什么,而不是存多少

Anthropic says the key to agent memory is knowing what to forget, not how much to store

AI 导读

Anthropic 发布研究文章,探讨 Claude 如何跨会话保留知识,结论是选择性笔记优于全量存档。在 Pokémon Red 测试中,Claude Sonnet 3.7 存了约 413 个文件只拿到 1 枚徽章,而 Claude Fable 5 只记策略和已验证规则,获得 4 枚徽章。

正文 · 原文

Anthropic studied how Claude retains knowledge across sessions. Its tests show that selective notes outperform exhaustive archives and that having Claude review its own memories after the fact can correct errors.

Anyone who works with an AI agent on a project over several days hits a wall fast, because every new session starts from scratch. In a research post, Anthropic describes how Claude is supposed to retain knowledge across sessions while the model stays unchanged. Claude writes notes to a folder that it can read back later.

The core problem, according to Anthropic, is selection. Before it knows what comes next, the agent has to decide in the moment what might matter later, so every note is a bet on the future.

Two Pokémon Red map segments showing the player character at a ledge near Mt. Moon. On the left, it walks toward the next city with a wrong note; on the right, it heads back to the cave entrance with the corrected note.
A wrong note about the Pokémon Center's location caused Claude to repeatedly fall back over a ledge. | Image: Anthropic

Agents that save everything lose track

In an Anthropic test with Pokémon Red, the older Claude Sonnet 3.7 treated everything as potentially important and, across roughly 10,000 game steps, created 413 files in 225 folders, often a separate one for each battle and item it found. Because the game state keeps changing, the archive goes stale fast. At times, the model did nothing but reorganize its notes instead of playing, and it finished with just a single badge.

The newer Claude Fable 5 limited its memory to a few files and earned four badges. Fable 5 didn't record information already visible on screen and instead wrote down plans, strategies, and rules it had tested. Anthropic's comparison, however, covers only a single playthrough per model.

Two treemaps at the same scale compare the memory folders of Claude Sonnet 3.7 with 413 files and about 978,000 characters against Claude Fable 5 with 10 files and about 43,000 characters.
Sonnet 3.7 saved nearly everything while playing Pokémon, whereas Fable 5 saved only strategies and verified rules and got much further. | Image: Anthropic

Anthropic recommends explaining the purpose of memory to Claude and letting the model decide how to use it. In one benchmark, according to Anthropic, rigid instructions such as "only record mistakes" wiped out most of the advantage for stronger models. Older models, by contrast, benefited from clear instructions, such as being told to actually read their notes before every response.

Left, a bar chart for Sonnet 4.6 showing 0.330 without memory, 0.364 with memory, and 0.641 with memory plus refined instructions. Right, the drop in memory gains when models are told to record only failures.
Specific memory rules help older models, while stronger models perform worse with rigid instructions. | Image: Anthropic

Dreaming aims to fix false memories

The bigger risk comes from faulty notes that the agent blindly trusts later. Dreaming, an idea that builds on research from Letta and UC Berkeley, is designed to address this. Much like the human brain processes experiences during sleep, Claude goes through transcripts of earlier sessions during a break and checks its notes against them.

Diagram of the memory hierarchy between two sessions, showing transcripts, memory notes, memory representations like wikis, and injected context, all framed by a region labeled "Dreaming."
During "Dreaming," Claude processes old transcripts in idle periods and builds the upper layers of its memory from them. | Image: Anthropic

In the Pokémon test, Claude had incorrectly noted which side of a mountain a landmark was on. Relying on that faulty note, Claude fell into the same trap three times, eventually wanted to give up, and missed its goal. After a Dreaming pass, Claude caught the error by checking the note against the transcript and found the right path.

Line chart tracking progress toward Cerulean City over 3,000 actions. The run with memory and Dreaming reaches the city after 1,935 actions; the runs without memory and without correction never do.
Claude made it past Mt. Moon to Cerulean City only with the note Dreaming had corrected. | Image: Anthropic

According to Anthropic, Dreaming can also organize larger knowledge bases. Claude built a linked wiki from about 350,000 internal documents from Stainless, a software company Anthropic acquired. Dreaming is supposed to generate new pages while fixing gaps, duplicates, and broken links. The payoff was greatest when Claude could combine the wiki with search, so the wiki worked as a complement rather than a replacement.

Line chart showing the share of graded criteria met across 25 questions about a document corpus. Wiki plus search rises from 54 to 61 percent, the wiki alone from 13 to 48 percent, and search alone stays at 53 percent. Two network graphs show the wiki after 2 and after 128 sessions.
The longer Claude worked on the wiki of Stainless documents, the better it answered questions, with the best results coming from combining the wiki with search. | Image: Anthropic

In a separate legal test by Anthropic, a wiki helped most with questions whose answers were spread across multiple documents. Claude scored 87 percent with the wiki, compared to 73 percent with search alone. For simple lookup tasks, the wiki performed roughly on par with search, and in one case it even grew larger than the source documents.

Memory is still a work in progress

Anthropic acknowledges that key questions remain open. Taking notes mid-task distracts the agent from its work, Dreaming costs extra compute, and its results arrive only after a delay. Wikis and notes also need regular upkeep, or errors and hallucinations pile up. Anthropic also warns against automatically loading too many memories into every session, because they eat up context and pass any bad note along to every future session.

All of these tests come from Anthropic. The overall takeaway is that a useful agent memory depends less on storing as much as possible than on keeping the right information and correcting mistakes later.

Anthropic introduced Dreaming in May as a research preview for Claude Managed Agents, the company's cloud platform for AI agents. The service reads an existing memory store and up to 100 past sessions, removes duplicate and outdated entries, and creates a reorganized store, leaving the original untouched. At launch, Dreaming supported only Claude Opus 4.7 and Sonnet 4.6.

OpenAI also calls ChatGPT's memory, which it overhauled in June, "Dreaming." That system maintains a user profile in the background and automatically corrects outdated information. According to OpenAI, this raised temporal accuracy from 52.2 to 75.1 percent.

Researchers at Google and Google DeepMind also let AI agents dream, but they use dreaming to improve an agent's search strategy. Their agent works through alternative solutions using results it has already stored, without calling the model again. Google has also built WikiSkill, a wiki for agents. As in Anthropic's system, WikiSkill keeps raw transcripts, distilled knowledge, and specific instructions in separate layers.

AI News Without the Hype – Curated by Humans

Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.

来源:The Decoder:AI News · the-decoder.com

事件后续 · 2

查看事件全部