Degradations vary widely across images, so a practical restoration system has to handle many degradation types with one model. A recent and effective recipe adapts a large pretrained image-editing model to restoration using a small low-rank adapter with a text prompt. We replace that prompt with an instruction derived from the degraded image itself. The image reaches the editor through two paths: its structure comes from the model's VAE, and its semantic instruction comes from a lightweight token mapper that shifts the degraded image's vision-language embedding toward the embedding a clean image would produce. Because the instruction is a continuous vector, scaling it yields a family of valid restorations for tasks whose target is not unique, such as low-light enhancement. We adapt one Qwen-Image-Edit model to six tasks with a single adapter trained in about three hours on one GPU. The image instruction outperforms text conditioning under a matched comparison, and it supports task agnostic restoration without a degradation label, which the text variant does not.
ImIR:用图像指令微调实现全能图像修复
AI 导读
ImIR 提出用退化图像自身生成的图像指令替代文本提示,通过 VAE 保留结构、轻量 token mapper 将退化图像的视觉-语言嵌入向干净图像的嵌入偏移,从而驱动图像编辑模型完成修复。
HuggingFace Daily Papers(社区热门论文)
37
AI 编辑部评分,满分 100ImIR:用图像指令微调实现全能图像修复
ImIR 提出用退化图像自身生成的图像指令替代文本提示,通过 VAE 保留结构、轻量 token mapper 将退化图像的视觉-语言嵌入向干净图像的嵌入偏移,从而驱动图像编辑模型完成修复。
原文 · 保持原样,未翻译原文 · 未翻译
来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org