软件工程（SWE）智能体的评估目前主要依赖一个二元信号：最终补丁是否通过测试。这种仅关注结果的视角，将基于原则的解决方案与混乱的试错过程视为等同。我们证明这种等同性在实证层面并不成立。我们在 60 个 SWE-bench Verified 任务上，评估了来自八个模型后端的 2,614 条 OpenHands 轨迹。
其中，47 个任务拥有足够多的通过轨迹，可以构建任务级过程参考，由此得到一个包含 1,815 条轨迹的评估子集。在该子集的通过轨迹中，有 10.7% 表现出我们称之为"幸运通过"的行为：回归重试、盲目重试、缺少验证，或探索、实现与验证在时间顺序上混乱。
我们引入了 AgentLens，一个用于对 SWE 智能体轨迹进行过程级评估的框架，并定义了 AgentLens-Bench，这是一个包含 1,815 条轨迹的数据集，每条轨迹都标注了质量分数、浪费信号、分歧点以及 47 个任务级前缀树接收器（PTA）参考。
AgentLens 通过合并同一任务的多个通过解决方案来构建 PTA 参考，并使用上下文敏感的意图标注器，基于轨迹历史（而非仅凭工具身份）将动作归类为探索、实现、验证或编排。在 AgentLens-Bench 上，质量分数将通过的轨迹划分为幸运、扎实和理想三个等级，并进一步将"幸运通过"分解为五种常见机制。
在八个模型后端中，幸运率从 0.5% 到 23.2% 不等，当按质量分数而非通过率排序时，某些模型的排名变动多达五位。我们计划很快发布项目仓库，包括 AgentLens-Bench 工件、AgentLens SDK 以及分析工具。
Evaluation of software engineering (SWE) agents is dominated by a binary signal: whether the final patch passes the tests. This outcome-only view treats a principled solution and a chaotic trial-and-error process as equivalent. We show that this equivalence is empirically false. We evaluate 2,614 OpenHands trajectories from eight model backends on 60 SWE-bench Verified tasks. Of these, 47 have enough passing trajectories to construct task-level process references, yielding a 1,815-trajectory evaluation subset. Among passing trajectories in this subset, 10.7% exhibit behavior we call a Lucky Pass: regression cycles, blind retries, missing verification, or temporally disordered exploration, implementation, and verification.
We introduce AgentLens, a framework for process-level assessment of SWE-agent trajectories, and define AgentLens-Bench, a dataset of 1,815 trajectories annotated with quality scores, waste signals, divergence points, and 47 task-level Prefix Tree Acceptor (PTA) references. AgentLens builds PTA references by merging multiple passing solutions for the same task, and uses a context-sensitive intent labeler to assign actions to Exploration, Implementation, Verification, or Orchestration based on trajectory history rather than tool identity alone. On AgentLens-Bench, the quality score separates passing trajectories into Lucky, Solid, and Ideal tiers and further decomposes Lucky Passes into five recurring mechanisms.
Across the eight model backends, Lucky rates range from 0.5% to 23.2%, and some models move by as many as five rank positions when ranked by quality score instead of pass rate. We plan to release the project repository soon, including AgentLens-Bench artifacts, the AgentLens SDK, and the analysis tooling.