# Negative Self-Distillation：通过规避缺陷来学习推理

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-09-10 08:00
- AIHOT 分数：37
- AIHOT 链接：https://aihot.news/items/cmtwwm8j005rproua4adkkvb2
- 原文链接：https://arxiv.org/abs/2609.11699

## AI 摘要

研究者提出 Negative Self-Distillation（NSD）框架，让模型自行生成问题特定的负面条件并将学生分布推离这一自生成负面教师，而非模仿特权解答。NSD 设计动态门控机制自动识别并隔离推理关键 token，避免语言能力崩塌。实验显示 NSD 持续优于 OPSD 及其他无标签自举强化学习基线。

## 正文

Abstract:On-Policy Self-Distillation (OPSD) has emerged as a popular paradigm for large language model (LLM) self-improvement, allowing models to act as their own teachers by leveraging privileged information such as ground-truth solutions. However, recent findings indicate that OPSD can severely degrade the performance of LLMs on complex reasoning tasks: By forcing the student to imitate an artificially confident reasoning trace conditioned on privileged information, OPSD inadvertently suppresses expressions of uncertainty and penalizes the exploratory, self-corrective behaviors required to solve challenging problems. To address this, we introduce Negative Self-Distillation (NSD), a new framework that optimizes LLMs by diverging from flawed reasoning rather than imitating privileged solutions. Instead of relying on ground-truth answers or external supervision, NSD uses the model itself to generate a question-specific negative condition (eg, acting as a ``careless reasoner'') and pushes the student's distribution away from this self-generated negative teacher. Naively applying unlearning objectives to achieve this divergence is problematic, as flawed reasoning tokens are confounded with basic linguistic tokens; indiscriminately penalizing both risks catastrophically degrading the model's foundational language capabilities. We resolve this by designing a dynamic gating mechanism that automatically identifies and isolates reasoning-critical tokens, ensuring gradient updates target only behavioral flaws while preserving the model's linguistic priors. Empirically, NSD consistently outperforms OPSD and other label-free, self-bootstrapping reinforcement learning (RL) baselines.

Comments:

Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG)

ACM classes: I.2.6; I.2.7

Cite as: arXiv:2609.11699 [cs.CL]

(or arXiv:2609.11699v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2609.11699

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Shuyao Xu [

Thu, 10 Sep 2026 15:24:17 UTC (3,269 KB)

Access Paper:

Current browse context:

cs cs.LG

References & Citations

Bookmark

Bibliographic and Citation Tools

Code, Data and Media Associated with this Article