Representation Autoencoders (RAEs) enable diffusion models to operate in the feature spaces of pretrained visual encoders. However, many off-the-shelf encoders are not optimized for faithful reconstruction, discarding fine-grained visual details. As expected, finetuning these encoders for image reconstruction recovers such details. However, perhaps counterintuitively, this procedure reduces the effective dimensionality of the resulting representation, and the altered geometry has downstream effects on generation. Specifically, we show that using the standard velocity prediction in flow matching in this high-dimensional space requires the model to fit orthogonal noise directions outside the low-dimensional signal manifold, making optimization inefficient. This motivates using the clean data parameterization (x_{0}-prediction) instead, which focuses learning on the underlying signal manifold. Across experiments with multiple strong-reconstruction encoders, we show that x_{0}-prediction consistently improves text-to-image generation performance.
高维潜空间的可扩散性:RAE 中 x0-prediction 如何提升文生图生成
AI 导读
针对 Representation Autoencoders(RAEs)中高保真重建编码器会降低表征有效维度、导致 flow matching 标准速度预测需拟合信号流形外的正交噪声方向、优化低效的问题,论文提出改用干净数据参数化(x₀-prediction),将学习集中在底层信号流形上。在多个强重建编码器实验中,x₀-prediction 一致提升了文生图生成性能。
HuggingFace Daily Papers(社区热门论文)
39
AI 编辑部评分,满分 100高维潜空间的可扩散性:RAE 中 x0-prediction 如何提升文生图生成
针对 Representation Autoencoders(RAEs)中高保真重建编码器会降低表征有效维度、导致 flow matching 标准速度预测需拟合信号流形外的正交噪声方向、优化低效的问题,论文提出改用干净数据参数化(x₀-prediction),将学习集中在底层信号流形上。在多个强重建编码器实验中,x₀-prediction 一致提升了文生图生成性能。
原文 · 保持原样,未翻译原文 · 未翻译
来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org