恢复策略引发的错误:鲁棒GUI智能体的基准测试与轨迹合成

HuggingFace Daily Papers(社区热门论文)·2026-05-28 08:00·116天前
AI 导读

针对GUI智能体缺乏从自身错误中恢复能力的问题,本研究提出了GUI-RobustEval基准测试和RoTS轨迹合成框架。GUI-RobustEval包含1216个可执行测试用例,系统评估智能体在多种错误模式下的恢复能力。RoTS框架通过基于树的流程合成了80万条高质量数据。在此基础上训练的RoTS-7B和RoTS-32B模型,在GUI-RobustEval及传统基准上均获得显著提升。其中RoTS-32B在OSWorld上取得了47.4%的成功率和33.8%的All-Pass@4分数,表明长时程错误恢复能力的增强对鲁棒性和整体性能均有贡献。

HuggingFace Daily Papers(社区热门论文)
精选
70AI 编辑部评分,满分 100

恢复策略引发的错误:鲁棒GUI智能体的基准测试与轨迹合成

2026-05-28 08:00· 116天前
AI 导读

针对GUI智能体缺乏从自身错误中恢复能力的问题,本研究提出了GUI-RobustEval基准测试和RoTS轨迹合成框架。GUI-RobustEval包含1216个可执行测试用例,系统评估智能体在多种错误模式下的恢复能力。RoTS框架通过基于树的流程合成了80万条高质量数据。在此基础上训练的RoTS-7B和RoTS-32B模型,在GUI-RobustEval及传统基准上均获得显著提升。其中RoTS-32B在OSWorld上取得了47.4%的成功率和33.8%的All-Pass@4分数,表明长时程错误恢复能力的增强对鲁棒性和整体性能均有贡献。

推荐理由

GUI Agent 能不能从自己犯的错里爬起来,才是落地的关键,阿里这篇论文给出了一个不错的基准和训练方案,做 Agent 的可以看看。

While GUI agents have advanced rapidly, they often lack the robustness to recover from their own errors, hindering real-world deployment. To bridge this gap at both the evaluation and data levels, we introduce GUI-RobustEval and propose Robustness-driven Trajectory Synthesis. GUI-RobustEval contains $1,216$ executable test cases that systematically measure error recovery capabilities across a broad and realistic spectrum of error modes. At the data level, RoTS is a scalable synthesis framework that creates $800k$ high-quality data via a tree-based pipeline that proactively discovers diverse error modes and synthesizes corresponding recovery steps. Our two models, RoTS-7B and RoTS-32B, fine-tuned on our dataset, both demonstrate significant gains on GUI-RobustEval and traditional GUI benchmarks. Notably, RoTS-32B achieves state-of-the-art performance on OSWorld, with a $47.4%$ success rate and a $33.8%$ All-Pass@4 score, suggesting that improved long-horizon error recovery ability contributes to both robustness and overall performance. Our code is available at https://github.com/AlibabaResearch/RoTS.

来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org