WorldReasonBench:面向未来世界状态预测的视频生成器人类对齐压力测试

HuggingFace Daily Papers(社区热门论文)·2026-05-11 08:00·133天前
AI 导读

研究团队发布WorldReasonBench基准,旨在直接评估视频生成模型作为“世界模拟器”的推理能力。该基准包含436个测试案例,涵盖物理、社会、逻辑和信息四大维度及22个子类,要求模型根据初始状态与动作生成状态演化一致的未来视频。评估采用人类对齐的双部分方法:过程感知推理验证通过结构化问答检测时序与因果错误;多维质量评估则对推理质量、时序一致性和视觉美学进行评分。测试发现,当前先进模型在视觉合理性与世界推理能力间存在显著差距,生成的视频可能看似逼真却违反动态、因果或信息守恒规律。相关资源已开源。

HuggingFace Daily Papers(社区热门论文)
精选
70AI 编辑部评分,满分 100

WorldReasonBench:面向未来世界状态预测的视频生成器人类对齐压力测试

2026-05-11 08:00· 133天前
AI 导读

研究团队发布WorldReasonBench基准,旨在直接评估视频生成模型作为“世界模拟器”的推理能力。该基准包含436个测试案例,涵盖物理、社会、逻辑和信息四大维度及22个子类,要求模型根据初始状态与动作生成状态演化一致的未来视频。评估采用人类对齐的双部分方法:过程感知推理验证通过结构化问答检测时序与因果错误;多维质量评估则对推理质量、时序一致性和视觉美学进行评分。测试发现,当前先进模型在视觉合理性与世界推理能力间存在显著差距,生成的视频可能看似逼真却违反动态、因果或信息守恒规律。相关资源已开源。

推荐理由

视频生成越来越像真的,但逻辑和因果一塌糊涂,这个基准把问题量化了,想做世界模拟器的团队可以拿来测测自己的模型到底懂不懂世界。

Commercial video generation systems such as Seedance2.0 and Veo3.1 have rapidly improved, strengthening the view that video generators may be evolving into "world simulators." Yet the community still lacks a benchmark that directly tests whether a model can reason about how an observed world should evolve over time. We introduce WorldReasonBench, which reframes video generation evaluation as world-state prediction: given an initial state and an action, can a model generate a future video whose state evolution remains physically, socially, logically, and informationally consistent? WorldReasonBench contains 436 curated test cases with structured ground-truth QA annotations spanning four reasoning dimensions and 22 subcategories. We evaluate generated videos with a human-aligned two-part methodology: Process-aware Reasoning Verification uses structured QA and reasoning-phase diagnostics to detect temporal and causal failures, while Multi-dimensional Quality Assessment scores reasoning quality, temporal consistency, and visual aesthetics for ranking and reward modeling. We further introduce WorldRewardBench, a preference benchmark with approximately 6K expert-annotated pairs over 1.4K videos, supporting pair-wise and point-wise reward-model evaluation. Across modern video generators, our results expose a persistent gap between visual plausibility and world reasoning: videos can look convincing while failing dynamics, causality, or information preservation. We will release our benchmarks and evaluation toolkit to support community research on genuinely world-aware video generation at https://github.com/UniX-AI-Lab/WorldReasonBench/.

来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org