智能体视觉生成:从生成模型到智能体控制

HuggingFace Daily Papers(社区热门论文)·2026-09-06 08:00·3天前
AI 导读

视觉生成正从单次调用的生成模型演变为可规划、选工具、检查中间输出并修正失败的智能体控制流程。本文按控制器能直接控制的内容划分层级:L1条件控制、L2执行控制、L3结果自适应控制、L4经验自适应控制,L0固定支持指无控制器的固定流程。该框架适用于图像、视频、编辑、3D、世界模型、幻灯片及UI生成。

HuggingFace Daily Papers(社区热门论文)
40AI 编辑部评分,满分 100

智能体视觉生成:从生成模型到智能体控制

2026-09-06 08:00· 3天前
AI 导读

视觉生成正从单次调用的生成模型演变为可规划、选工具、检查中间输出并修正失败的智能体控制流程。本文按控制器能直接控制的内容划分层级:L1条件控制、L2执行控制、L3结果自适应控制、L4经验自适应控制,L0固定支持指无控制器的固定流程。该框架适用于图像、视频、编辑、3D、世界模型、幻灯片及UI生成。

Visual generation is evolving from generative models used through a single invocation into agentic control processes that can plan, select tools, inspect intermediate synthesized outputs, revise failures, and reuse prior experience. In most existing systems, the controller is an LLM or VLM, while visual generation models serve as tools or executors. However, existing work lacks a consistent criterion for determining when a generation system becomes agentic. Planning depth, tool use, multi-role collaboration, and reinforcement learning are often treated as evidence of agenticity, even though none of them necessarily determines which generation decisions the controller can make. We organize the field according to what the controller can directly control in the generation process. At L1 Conditioning Control, the controller prepares the input to a predetermined generator but does not control which visual operation is executed. At L2 Execution Control, it selects and invokes actual generation, editing, rendering, or other content-modifying operations. At L3 Outcome-Adaptive Control, it observes an intermediate outcome and uses that observation to change a subsequent operation within the current task. At L4 Experience-Adaptive Control, it retains experience from completed tasks and uses that experience to change decisions on future tasks. L0 Fixed Support separately denotes generators, editors, evaluators, reward models, benchmarks, and fixed pipelines without a deployed controller that makes generation-level decisions. These levels describe a progressively broader decision-making scope rather than model size, system complexity, output quality, tool or role count, or training method. Applying this framework across image, video, editing, 3D, world, slide, and user-interface generation reveals how controller capabilities have evolved and how their mechanisms are distributed across levels.

来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org