# Complex KDA：理解并增强 Kimi Delta Attention 的表达能力

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-09-21 08:00
- AIHOT 分数：45
- AIHOT 链接：https://aihot.news/items/cmucp4gxr0igaroedql23hvzm
- 原文链接：https://arxiv.org/abs/2609.24797

## AI 摘要

研究者提出 Complex KDA（CKDA），通过将单个 delta-rule 变换与通道门控提供的第二次反射结合，使 Kimi Delta Attention（KDA）无需提高更新秩即可实现 2D 旋转。

## 正文

Linear RNNs based on the delta-rule enable efficient sequence modeling, but their linear updates with a low-rank correction constrain their expressivity. Prior work has shown that composing two delta-rule transitions in a single recurrent update can model a 2D rotation, but this increases the rank and the cost of the updates compared to a single transition. We show that Kimi Delta Attention (KDA) can realize 2D rotations by combining a single delta-rule transformation with a second reflection supplied by its channel-wise gate. This requires extending the parameter ranges of KDA by combining two existing range extensions: allowing gates in [-1,1] and the delta-rule coefficient β in [0,2]. We call the resulting model Complex KDA (CKDA). It preserves KDA's stability and efficiency, with transitions that remain diagonal-plus-rank-one and non-expansive, while reaching the state-tracking expressivity of DeltaProduct_2. We characterize the expressivity of CKDA and prove that every orthogonal diagonal-plus-rank-one matrix is exactly a CKDA transition matrix. A single CKDA layer can track every finite group isomorphic to a subgroup of SO(3), and many state-tracking results use one fewer layer for CKDA compared to other diagonal-plus-rank-one Linear RNNs. Empirically, combining both extensions yields the strongest length extrapolation among tested KDA range settings on S_3, S_4, and periodic audio continuation. In language modeling, CKDA outperforms Transformers and other linear RNNs, obtains similar results to a KDA baseline, and shows promising scaling behavior. Our code is open source at https://github.com/OpenEuroLLM/ComplexKDA and our models are available at https://huggingface.co/collections/openeurollm/complexkda.