如今的推理模型利用思考 token 在基准测试上取得了比其指令微调版本更强的性能。人们通常也认为,这种更具“审慎性”的模式应该能提升对齐与安全性,因为它为模型提供了一个安全空间,使其能够考虑对用户请求的预设回答是否违反了自身的安全原则。
我们提供的证据表明,这一直觉并非总是正确的。在涵盖 GPT-OSS、Qwen、Olmo 和 Phi 系列的前沿开放权重推理模型中,我们发现,在模型进行任何可见的思考之前,通过一个在首个 token 的隐藏表示上训练的分类头,就已经能够强有力地预测其最终的拒绝/遵从结果(预测拒绝/遵从的 AUROC 为 0.84-0.95,平衡准确率约为 88%)。
事实证明,思考过程更像是一种前缀补全,而非审慎的修正;尽管在文本层面呈现出审慎的表象(约 74% 的文本层面审慎行为发生在响应分布已经锁定在拒绝或遵从某一侧之后),但最终结果在思考过程的前约 20% 之后便极少改变。我们还发现,现有的基于推理时和基于训练的安全干预措施,尽管其动机是诱导审慎思考,但在很大程度上却将模型行为转向了过度拒绝,同时抑制了本已稀缺的审慎信号。
我们的结果表明,当前推理模型中的安全行为远没有人们普遍认为的那么审慎,并凸显了开发能够引发真正安全审慎的方法的必要性。
Today's reasoning models use thinking tokens to attain stronger performance on benchmarks than their instruction-tuned counterparts. It is also generally believed that this more "deliberative" mode should improve alignment and safety, by providing the model a safe space to consider whether its planned answer to a request violates its safety principles. We present evidence that this intuition is not always correct. Across frontier open-weight reasoning models spanning GPT-OSS, Qwen, Olmo, and Phi families, we find that the eventual refusal/compliance outcome is already strongly predictable via a trained head on the first token's hidden representation ($0.84$-$0.95$ AUROC and $\sim88%$ balanced accuracy for predicting refusal/compliance) before any visible thinking.
The thinking process turns out to be more akin to prefix completion than to deliberative revision, with the final outcome rarely changing after the first $\sim20%$ of thinking, despite giving the appearance of deliberation at the text level ($\sim74%$ of text-level deliberations occur when the response distribution is already locked to one refusal/compliance side). We also find that existing inference-time and training-based safety interventions, despite being motivated by the goal of inducing deliberation, largely shift model behavior toward over-refusal while suppressing already-scarce deliberation signals.
Our results suggest that safety behavior in current reasoning models is much less deliberative than commonly assumed, and highlight the need for methods that induce real safety deliberation.