# Apple 提出 probe guidance：用冻结内部状态引导扩散语言模型

- 来源：Apple Machine Learning Research（RSS）
- 发布时间：2026-09-23 08:00
- AIHOT 分数：42
- AIHOT 链接：https://aihot.news/items/cmueppk4c0eniroynxp4v73f6
- 原文链接：https://machinelearning.apple.com/research/guide-language-flow

## AI 摘要

Apple 提出 probe guidance，利用现有扩散模型冻结的内部状态构建引导信号，无需在推理时增加额外前向计算，即可让弱模型与强模型保持相近动态。该方法在连续扩散语言模型的无条件生成上取得新的 SOTA，应用于 1.7B 扩散语言模型时在多项选择题基准上持续提升。研究还发现，传统 autoguidance 中的弱模型必须来自训练的低熵区域。

## 正文

We introduce a new method to guide flow matching models. Our approach, which we call probe guidance, uses the frozen internal states of an existing diffusion model to construct a guidance signal. This works using a similar principle as autoguidance, but eliminates the need for an additional forward pass at inference time and provides a reliable path to ensure that the weak and strong model share similar dynamics. We apply and benchmark this method on continuous diffusion language models, where probe guidance sets a new state-of-the-art performance on unconditional generation. When applied to a 1.7B diffusion language model, probe guidance consistently improves on multiple choice question answering benchmarks. Using our probes, we study the traditional autoguidance setting where the strong model is a weak checkpoint, and find that the weak model must come from a low-entropy region of training. These findings both provide a practical way to improve diffusion language models and shed light on the actual mechanism behind autoguidance, which is currently poorly understood.

† Caltech

** Work done while at Apple