# Transformer 能同时容纳两种思路：LLM 中线性叠加的证据

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-09-24 08:00
- AIHOT 分数：44
- AIHOT 链接：https://aihot.news/items/cmugzodaj03throlziaa8vk70
- 原文链接：https://arxiv.org/abs/2609.29845

## AI 摘要

研究提出「叠加线性假设」，证明 Transformer 对来自不同文本流的输入做线性组合时，输出是各自下一 token 分布的叠加，且该性质是架构固有而非训练涌现，并随预训练推进而减弱。轻量微调可显著恢复这种线性，缩小预测分布与各分布均值的偏差。团队还提出引导式解码，从单次前向传播中解耦叠加输出，同时生成两条连贯续写。

## 正文

While Large Language Models (LLMs) rely on highly non-linear components, in this work we demonstrate that they exhibit fundamental linearity: when inputs from distinct text streams are linearly combined, the model outputs a superposition of the individual next-token distributions. We term this the Superposition Linearity Hypothesis. We provide evidence that superposition is an intrinsic property of the Transformer architecture rather than an emergent consequence of training; in fact, we observe that it tends to diminish as pretraining progresses. However, we demonstrate that linearity can be substantially restored through lightweight fine-tuning, significantly reducing the divergence between the predicted next-token distribution and the average of the individual next-token distributions. Finally, we introduce a guided decoding procedure that disentangles superposed outputs, enabling the simultaneous generation of two coherent continuations from a single forward pass.