# WhatWorkedBench：衡量 AI 智能体实验理解能力的基准

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-09-23 08:00
- AIHOT 分数：37
- AIHOT 链接：https://aihot.news/items/cmuewn5g2033tro3kmekh2mgp
- 原文链接：https://arxiv.org/abs/2609.27490

## AI 摘要

WhatWorkedBench 被提出，用于衡量 AI 研究智能体的实验理解能力，即预算受限实验后对组件改动效果的预测准确度。基准覆盖 30 个数据源、8 类工作流的 36 个任务，含 1248 条配置记录，核心评测整合 4206 条数值控制记录与 108 个智能体 episode。对同一批智能体观测拟合高斯过程后，Flash 队列的效果恢复率从 0.632 升至 0.698。

## 正文

AI research agents need reliable knowledge of how their experiments change outcomes. We introduce WhatWorkedBench to measure experimental understanding, the accuracy of predictions about component changes after budgeted experimentation. Agents inspect code, select measurements, and submit a response surface, a table predicting scores for every configuration of component settings. Exhaustive CPU execution supplies reference effects for changing each component while holding the others fixed. These effects capture combinations of changes across 36 tasks from 30 data sources and 8 workflow types, with 1248 configuration records.

Core evaluation combines 4,206 numerical-control records across all eight families and 108 agent episodes across the original six. At eight new measurements, pair-effect ridge selects an optimum on 15 of 22 sources and limits every effect error to 10% of score range on three. Fitting a Gaussian process (GP) to the same agent observations raises effect recovery, accuracy relative to true effect magnitude, from 0.632 to 0.698 in the original Flash cohort and from 0.621 to 0.720 in an additional cohort. On six completed beat-detection and graph submissions, the same-observation GP raises family-macro recovery from 0.303 to 0.455.

On six workflows with six binary options at 20 new measurements, encoding code equivalences, configurations with identical behavior, raises GP recovery from 0.248 to 0.462. WhatWorkedBench supports research on experimental agents, adaptive experimental design, numerical inference, and use of program structure.