Image relighting is traditionally tackled via complex inverse rendering pipelines, which suffer from ill-posed optimization, or single-image generative models that ignore crucial multi-view cues necessary for understanding 3D geometry and material interactions. To address these limitations, we introduce a feed-forward generative Transformer for direct single- and multi-view image relighting that entirely bypasses explicit intrinsic property estimation. Adapted from a video foundation model, our architecture features a latent illumination module that dynamically injects target environment maps into spatial features via cross-attention. Furthermore, we employ permutation-invariant positional encodings to symmetrically process unordered multi-view inputs without sequential bias. To train this robust data-driven model, we construct the massive Laval Objaverse Dataset (LOD), comprising 90K objects and 39K unique illuminations. Extensive experiments demonstrate state-of-the-art visual quality, photorealistic relighting quality, and strong zero-shot generalization across single-view, multi-view, and novel-view relighting tasks.
RelightFormer:用于多视角物体重打光的自回归生成式Transformer
AI 导读
RelightFormer是一种前馈生成式Transformer,可直接对单视角和多视角图像进行重打光,绕过了显式的固有属性估计。该架构从视频基础模型适配而来,通过潜在光照模块和置换不变位置编码处理无序多视角输入。研究团队构建了包含90K物体和39K种独特光照的Laval Objaverse数据集(LOD),实验显示其在单视角、多视角及新视角重打光任务上达到SOTA视觉质量并具备强零样本泛化能力。
HuggingFace Daily Papers(社区热门论文)
33
AI 编辑部评分,满分 100RelightFormer:用于多视角物体重打光的自回归生成式Transformer
RelightFormer是一种前馈生成式Transformer,可直接对单视角和多视角图像进行重打光,绕过了显式的固有属性估计。该架构从视频基础模型适配而来,通过潜在光照模块和置换不变位置编码处理无序多视角输入。研究团队构建了包含90K物体和39K种独特光照的Laval Objaverse数据集(LOD),实验显示其在单视角、多视角及新视角重打光任务上达到SOTA视觉质量并具备强零样本泛化能力。
原文 · 保持原样,未翻译原文 · 未翻译
来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org