基于大语言模型/视觉语言模型的数字智能体,得益于用于编码、网页导航和计算机操作的可扩展沙盒环境(这些环境提供了丰富的交互式训练场),已取得快速进展。相比之下,具身智能体仍然缺乏丰富、多样且可自动生成的3D环境来进行交互式学习。
现有的具身模拟器依赖于人工构建的场景或程序化模板,而近期基于大语言模型的3D生成系统主要生成静态场景,而非带有可验证任务和标准学习接口的可部署环境。我们推出了SimWorld Studio,这是一个基于虚幻引擎5构建的开源平台,用于生成不断演化的具身学习环境。
其核心是SimCoder,一个经过工具/技能增强的编码智能体,它能够编写并执行引擎级代码,从而根据语言/图像指令构建物理上合理的3D世界。SimCoder通过利用验证器反馈(例如编译错误、物理检查、视觉语言模型批评)来修改环境,并自主地将可复用的工具和技能添加到其库中,从而实现自我进化。
生成的世界被导出为Gym风格的环境,用于具身智能体的学习。SimWorld Studio进一步实现了环境生成与具身学习之间的协同进化:智能体性能反馈引导SimCoder在学习者能力边界附近生成自适应课程,使得随着具身智能体的提升,环境难度也逐步增加。
三项关于具身导航的案例研究表明,自我进化提升了生成的可靠性;生成的环境显著提升了具身智能体的性能,且该性能可泛化至未见过的基准测试;协同进化相较于固定环境学习带来了18个百分点的成功率提升,相较于未经训练的智能体则带来了40个百分点的提升。
LLM/VLM-based digital agents have advanced rapidly thanks to scalable sandboxes for coding, web navigation, and computer use, which provide rich interactive training grounds. In contrast, embodied agents still lack abundant, diverse, and automatically generated 3D environments for interactive learning. Existing embodied simulators rely on manually crafted scenes or procedural templates, while recent LLM-based 3D generation systems mainly produce static scenes rather than deployable environments with verifiable tasks and standard learning interfaces.
We introduce SimWorld Studio, an open-source platform built on Unreal Engine 5 for generating evolving embodied learning environments. At its core is SimCoder, a tool/skill-augmented coding agent that writes and executes engine-level code to construct physically grounded 3D worlds from language/image instructions. SimCoder self-evolves by using verifier feedback (e.g., compilation errors, physics checks, VLM critiques) to revise environments and autonomously add reusable tools and skills to its library. Generated worlds are exported as Gym-style environments for embodied agent learning.
SimWorld Studio further enables co-evolution between environment generation and embodied learning: agent performance feedback guides SimCoder to generate adaptive curricula near the learner's capability frontier, so that environments become increasingly challenging as the embodied agent improves. Three case studies on embodied navigation show that self-evolution improves generation reliability, generated environments substantially improve embodied agent performance that generalizes to unseen benchmarks, and co-evolution yields an 18-point success-rate gain over fixed-environment learning and a 40-point gain over an untrained agent.