Embodied navigation requires agents to interpret visual observations, accumulate spatial knowledge, and execute actions to follow instructions or locate objects. Training-based methods face generalization challenges, while training-free methods exploit multimodal large language models (MLLMs) but often lack mechanisms to reconcile proposed actions with spatial evidence, task progress, and execution failures. We present HarnessVLN, a zero-shot, training-free framework whose Agent Harness coordinates perception, retrieval, grounding, navigation, recovery, and termination through a unified tool interface. The Harness validates planner proposals against spatial evidence, geometric feasibility, and subgoal consistency, incorporating structured tool feedback into subsequent decisions. Hierarchical event memory tracks task progress and execution history, while a persistent Spatiotemporal Graph maintains reusable spatial evidence and failure annotations for verification and recovery. A replaceable Navigation Executor converts validated targets into executable motions, allowing the same Harness protocol to support instruction-following and object-goal navigation. HarnessVLN achieves success rates of 60.8%, 53.9%, 76.0%, and 59.3% on R2R, RxR, HM3D-v2, and HM3D-OVON, respectively, surpassing prior training-free SOTA results. Humanoid deployment further demonstrates its applicability to both tasks in real-world environments. The project page is: https://harnessvln.netlify.app/.
HarnessVLN:用 Agent Harness 统一免训练具身导航
AI 导读
HarnessVLN 是一个零样本、免训练的具身导航框架,通过统一工具接口的 Agent Harness 协调感知、检索、定位、导航、恢复与终止,并用分层事件记忆和持久化时空图校验规划器提案。它在 R2R、RxR、HM3D-v2、HM3D-OVON 上分别取得 60.8%、53.9%、76.0%、59.3% 的成功率,超过此前免训练 SOTA,并已在人形机器人上完成真实环境部署。
HuggingFace Daily Papers(社区热门论文)
36
AI 编辑部评分,满分 100HarnessVLN:用 Agent Harness 统一免训练具身导航
HarnessVLN 是一个零样本、免训练的具身导航框架,通过统一工具接口的 Agent Harness 协调感知、检索、定位、导航、恢复与终止,并用分层事件记忆和持久化时空图校验规划器提案。它在 R2R、RxR、HM3D-v2、HM3D-OVON 上分别取得 60.8%、53.9%、76.0%、59.3% 的成功率,超过此前免训练 SOTA,并已在人形机器人上完成真实环境部署。
原文 · 保持原样,未翻译原文 · 未翻译
来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org