视频扩散模型为何违反物理规律?揭示注意力机制中的缺陷

HuggingFace Daily Papers(社区热门论文)·2026-09-20 08:00·2天前
AI 导读

首个针对文生视频扩散模型"运动规划"过程的可解释性研究发现,RoPE 导致空间注意力过度衰减,使早期候选区域过早锁定在物理上不合理的位置,抑制相邻帧的合理轨迹并引发生成失败。作者据此提出一种轻量级架构修改,按去噪步数缩放 RoPE 频率,在免训练和基于训练的实验中均提升了生成视频的物理常识。

HuggingFace Daily Papers(社区热门论文)
42AI 编辑部评分,满分 100

视频扩散模型为何违反物理规律?揭示注意力机制中的缺陷

2026-09-20 08:00· 2天前
AI 导读

首个针对文生视频扩散模型"运动规划"过程的可解释性研究发现,RoPE 导致空间注意力过度衰减,使早期候选区域过早锁定在物理上不合理的位置,抑制相邻帧的合理轨迹并引发生成失败。作者据此提出一种轻量级架构修改,按去噪步数缩放 RoPE 频率,在免训练和基于训练的实验中均提升了生成视频的物理常识。

Despite impressive visual quality, state-of-the-art video diffusion models often generate content that violates real-world physical laws. While existing solutions rely on external priors or specialized data, we investigate the root cause by exploring the internal mechanisms of these models. Specifically, we present the first interpretability study on the ''motion planning'' process of text-to-video diffusion models, revealing how motion trajectories form during early denoising stages. Building upon the ''first shape, then details'' finding, we combine cross-attention trajectory patterns with causal head contributions to identify a specific subset of attention heads driving motion planning.

Further, our self-attention analysis shows that Rotary Position Embedding (RoPE) induces excessive spatial attention decay. This causes early candidate regions to prematurely lock into physically implausible positions, suppressing reasonable trajectories in adjacent frames and triggering generation failure modes. To address this fundamental flaw, we propose a lightweight architectural modification that scales the frequency of RoPE across different denoising steps. This strategy reduces excessive attention decay, helping the model explore better candidate regions to establish coherent physical motion.

Finally, training-free and training-based experiments confirm the effectiveness of our approach in enhancing the physical commonsense of generated videos.

来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org