Programmable World Model:将世界状态演化与视觉生成解耦的可编程世界模型

HuggingFace Daily Papers(社区热门论文)·2026-09-09 08:00·1天前
AI 导读

Programmable World Model 将世界状态演化与视觉观察生成解耦,由智能体把自然语言指令翻译为可执行程序,指定实体状态与状态转移规则,再由轻量引擎维护包含屏幕外实体和非视觉属性的持久全局世界状态。

HuggingFace Daily Papers(社区热门论文)
47AI 编辑部评分,满分 100

Programmable World Model:将世界状态演化与视觉生成解耦的可编程世界模型

2026-09-09 08:00· 1天前
AI 导读

Programmable World Model 将世界状态演化与视觉观察生成解耦,由智能体把自然语言指令翻译为可执行程序,指定实体状态与状态转移规则,再由轻量引擎维护包含屏幕外实体和非视觉属性的持久全局世界状态。

Recent video world models generate increasingly realistic and interactive visual experiences, yet lack reliable mechanisms for maintaining persistent world state and enforcing programmable rules over extended interactions. We introduce Programmable World Model, a framework that decouples world-state evolution from visual observation generation. An agent translates natural-language instructions into executable programs that specify entity states and state-transition rules, enabling direct control over individual entities and their interactions. A lightweight engine executes these programs to update and maintain an explicit, persistent global world state, including off-screen entities and non-visual attributes. To connect world state with visual generation, we introduce state-augmented 3D oriented bounding boxes (OBBs) as an intermediate representation. This representation, together with the target camera trajectory, is deterministically compiled into pixel-aligned spatiotemporal conditioning signals for a pretrained video model serving as the generative renderer. This design allows users to create playable games with predefined mechanics, direct control over individual entities, and persistent world state throughout gameplay. We further introduce CombatStateBench, a benchmark for evaluating programmable world models. On CombatStateBench, our method achieves 94% Count Accuracy and 98% State Accuracy, substantially outperforming existing interactive video world models while supporting coherent long-horizon generation. These results demonstrate the effectiveness of separating explicit state evolution from generative rendering for building persistent, programmable worlds.

来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org