Instruction-guided 3D editing is essential for interactive content creation, yet it faces a significant bottleneck: the severe scarcity of high-quality paired training data. Existing approaches attempt to bypass this by either relying on slow test-time optimization or training on pseudo-pairs constructed via complex pipelines, which often introduce structural drift and geometric artifacts. In this paper, we propose a novel framework that learns feed-forward 3D editing without paired 3D supervision via Generative Prior Distillation. Instead of relying on ground-truth 3D pairs, our core idea is to distill visual, semantic, and geometric knowledge from powerful foundation models directly into a 3D editing model. Specifically, through a differentiable rendering pipeline, we supervise the 3D representation using two complementary signals: a 2D visual prior from an image editing model at the main editing view, and a semantic prior from a Vision-Language Model at novel views to ensure strict instruction following and source identity preservation. Crucially, to address the geometric collapse and multi-view inconsistencies inherent in 2D projection supervision, we introduce a 3D-aware Distribution Matching regularization. Acting as a geometric prior, this term operates in the 3D latent space, constraining the edited output to remain within the manifold of realistic 3D assets defined by a pretrained image to 3D teacher model. Extensive experiments demonstrate that our method achieves superior instruction fidelity and cross-view consistency, significantly outperforming state-of-the-art baselines. Our project is available at: https://github.com/thiamine128/PriorEdit3D.
Learning 3D Editing without Paired Supervision via Generative Prior Distillation
AI 导读
提出一种无需成对3D监督即可学习前馈3D编辑的新框架,通过生成式先验蒸馏,将基础模型中的视觉、语义和几何知识直接注入3D编辑模型。该方法利用可微渲染,以图像编辑模型的2D视觉先验和视觉语言模型的语义先验进行监督,并引入3D感知分布匹配正则化以防止几何塌陷和多视图不一致。实验表明,该方法在指令遵循度和跨视图一致性上显著优于现有基线。
HuggingFace Daily Papers(社区热门论文)
35
AI 编辑部评分,满分 100Learning 3D Editing without Paired Supervision via Generative Prior Distillation
提出一种无需成对3D监督即可学习前馈3D编辑的新框架,通过生成式先验蒸馏,将基础模型中的视觉、语义和几何知识直接注入3D编辑模型。该方法利用可微渲染,以图像编辑模型的2D视觉先验和视觉语言模型的语义先验进行监督,并引入3D感知分布匹配正则化以防止几何塌陷和多视图不一致。实验表明,该方法在指令遵循度和跨视图一致性上显著优于现有基线。
原文 · 保持原样,未翻译原文 · 未翻译
来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org