Stanford 等机构论文:27B 开源模型可无训练自行改写权重
Stanford、UIUC 等机构的论文提出自改写方法,Qwen3.5-27B 在梯度被封锁的条件下,比较答完题与耗尽 token 答案的隐藏状态并削弱分离两者的方向,自行改写权重。该改动使 490 道未见题目的准确率从 57.2% 升至 63.8%,测试提前固定;2B 模型未做改动,4B 和 9B 模型则改坏了自身。
New Stanford+other top labs paper finds that a 27B open model can improve itself by editing its own weights through reasoning, with no training or gradients.
Improving a deployed model today means another training run on large clusters. Prompts and extra inference compute help, but the gain disappears once the task ends.
They gave Qwen3.5-27B its own weights, its graded practice answers, and a Python session, with gradients blocked. It compared its hidden states on answers that finished against answers that ran out of tokens, then weakened the direction separating them.
Its own edit raised accuracy from 57.2% to 63.8% on 490 unseen problems, in a test fixed in advance. The 2B model made no edit, and the 4B and 9B models broke themselves.
If you run a large open model, let it propose weight edits for its own failures, but keep only edits that pass held-out tests.
来源:Rohan Paul · x.com