WBench:面向交互式世界模型评估的多轮基准

HuggingFace Daily Papers(社区热门论文)·2026-05-25 08:00·118天前
AI 导读

WBench 是一个用于系统评估交互式世界模型的多轮基准。它提出了一个五维评估框架,涵盖视频质量、场景设定遵循度、交互指令遵循度、一致性与物理符合性。该基准包含 289 个测试案例与 1,058 轮交互,覆盖了多样化的场景、风格、主体及第一/第三人称视角。评估使用 22 个结合专业视觉模型与大型多模态模型的自动子指标,所有指标均经过人工校验。对 20 个 SOTA 模型的评测发现,目前尚无模型在所有维度上表现均优。

HuggingFace Daily Papers(社区热门论文)
精选
70AI 编辑部评分,满分 100

WBench:面向交互式世界模型评估的多轮基准

2026-05-25 08:00· 118天前
AI 导读

WBench 是一个用于系统评估交互式世界模型的多轮基准。它提出了一个五维评估框架,涵盖视频质量、场景设定遵循度、交互指令遵循度、一致性与物理符合性。该基准包含 289 个测试案例与 1,058 轮交互,覆盖了多样化的场景、风格、主体及第一/第三人称视角。评估使用 22 个结合专业视觉模型与大型多模态模型的自动子指标,所有指标均经过人工校验。对 20 个 SOTA 模型的评测发现,目前尚无模型在所有维度上表现均优。

推荐理由

视频世界模型的评估终于有了统一尺度,WBench 从画面质量到物理一致性覆盖五个维度,289 个测试用例把 20 个模型拉平一看,没有谁全面领先,做这方向的值得拿来跑一遍。

Interactive world models are advancing rapidly, yet existing benchmarks cover only part of the required competencies, leaving no unified standard for systematic evaluation. To fill this gap, we introduce WBench, a comprehensive multi-turn benchmark for interactive world model evaluation along five dimensions, namely video quality, setting adherence, interaction adherence, consistency, and physics compliance. WBench contains 289 test cases and 1,058 interaction turns, where each case specifies a world setting and a multi-turn interaction sequence, covering diverse scenes, styles, subjects, and both first- and third-person perspectives, together with four interaction types, including navigation, subject action, event editing, and perspective switching. For navigation, WBench unifies text, 6-DoF pose, and discrete-action control, enabling evaluation of models with different native input interfaces. Evaluation uses 22 automatic sub-metrics that combine specialist vision models with large multimodal models, and all metrics are validated against human judgments. Across 20 state-of-the-art models, we find that no single model performs strongly across all dimensions. We provide detailed diagnostic insights into the characteristic strengths, weaknesses, and open challenges of each model. Code and data are available at https://github.com/meituan-longcat/WBench.

来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org