Pelican-Sim 1.0:面向具身智能的通用世界模型模拟器

HuggingFace Daily Papers(社区热门论文)·2026-09-10 08:00·5天前
AI 导读

Pelican-Sim 1.0 是面向具身智能的通用世界模型模拟器,可从视觉上下文和机器人动作预测未来观测。它采用 28 维统一动作空间、动作-视觉注入、稀疏 MoE 和四步自回归蒸馏,实现 5.67 倍加速,在 AgiBotWorld Beta、RoboMIND、RoboTwin 上分别将 PSNR 提升 4.636、2.080 和 10.343。

HuggingFace Daily Papers(社区热门论文)
50AI 编辑部评分,满分 100

Pelican-Sim 1.0:面向具身智能的通用世界模型模拟器

2026-09-10 08:00· 5天前
AI 导读

Pelican-Sim 1.0 是面向具身智能的通用世界模型模拟器,可从视觉上下文和机器人动作预测未来观测。它采用 28 维统一动作空间、动作-视觉注入、稀疏 MoE 和四步自回归蒸馏,实现 5.67 倍加速,在 AgiBotWorld Beta、RoboMIND、RoboTwin 上分别将 PSNR 提升 4.636、2.080 和 10.343。

In this technical report, we propose Pelican-Sim 1.0, a general world model simulator for embodied intelligence that predicts future observations from visual context and robot actions to support downstream learning and decision making. The model incorporates four key design features: (1) Unified action representation: a 28-dimensional action value space covering most mainstream embodiments, keeping one model valid across heterogeneous devices. (2) Action-visual injection: URDF- and camera-rendered action videos bridge actions and pixels, giving markedly better controllability across embodiments, scenes, and tasks (PSNR +0.904 over alternative fusion baselines).

(3) Sparse mixture-of-experts (MoE): sparse MoE layers add capacity for heterogeneous dynamics and absorb the action modality while reducing inter-modality conflict (FVD -6.530 vs. the dense backbone). (4) Efficient rollout generation: causal adaptation and few-step distillation yield a four-step autoregressive simulator, achieving a 5.67-fold speedup over the 35-step model. Benefiting from these designs, we train on approximately one million real-world and simulated trajectories and obtain large gains in action controllability and video quality: PSNR improves over the strongest evaluated baselines by 4.636 on AgiBotWorld Beta, 2.080 on RoboMIND, and 10.343 on RoboTwin, with the adapted EWMBench DYN score up 0.426 on RoboTwin.

Relying on this, four downstream applications on RoboTwin succeed: 500 generated trajectories added to 50 demonstrations per task raise policy success from 70% to 93%; policy evaluation reaches a Pearson correlation of 0.994 across five checkpoints; and relative success gains reach 47.7% for action selection and 20.3% for policy improvement. Qualitative generalization across trajectory, scene, object, embodiment, and viewpoint shifts highlights its potential as a general-purpose world model simulator.

来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org