跳到正文
DAIR.AI· @dair_ai · X·· 3 小时前AI 评分58
AI 导读

Meta 等机构提出 Context Language Models(CLM),把上下文作为文件让模型用 Bash 自由编辑,自主决定保留、重写或删除内容,并区分于 RLM 不允许模型直接编辑实时交互上下文的做法。zero-shot 下在 BrowseComp-Plus 上比现有上下文管理策略高 11.4% 准确率、省 21.5% FLOPs;在线 RL 使 Qwen3.5-9B 提升 47.6% 且少用 12% FLOPs;配套 Suffix Cache Reuse 相比标准 SGLang 降低 35% 服务端计算。论文链接:https://academy.dair.ai/papers/context-language-models-2609.37725

正文

Great paper for the weekend ahead. Meta presents Context Language Models, which can natively manage their own context.

引用elvis@omarsar0
Banger paper from Meta and colleagues. (bookmark it) They discuss benefits of giving your agent write access to its own context. In other words, they investigate how effective it is to allow a language model to natively manage its context. Context Language Models keep the context as a file the model edits with Bash, so the model decides what to keep, rewrite or remove. It's a bit different from RLM, for those who are wondering but it pulls an interesting theme. RLMs place a large input in an external variable that the model can read and process recursively, but they do not let the model directly edit its own live interaction context. CLMs instead expose the live context as a read-write file, including information accumulated during execution. Applied zero-shot, this gets 11.4% higher accuracy with 21.5% fewer FLOPs on BrowseComp-Plus than existing context-management strategies. On a 24-hour task where a swarm of agents works across six repositories, it gets 65% more improvement for the same compute. The strategy can also be trained. Online RL improves Qwen3.5-9B on BrowseComp-Plus by 47.6% while using 12% fewer FLOPs. Edits in the middle of the context break prefix caching, so the authors add Suffix Cache Reuse, which cuts server compute by 35% against standard SGLang. Paper: https://academy.dair.ai/papers/context-language-models-2609.37725
在 X 查看被引用的帖子

来源:DAIR.AI · x.com