# IdeaAMBIG：评测研究想法规格中阻碍实现的关键缺口

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-09-09 08:00
- AIHOT 分数：40
- AIHOT 链接：https://aihot.news/items/cmtx5710r07cjroedbuvlqkaz
- 原文链接：https://arxiv.org/abs/2609.10539

## AI 摘要

研究者提出 IdeaAMBIG 基准，包含 660 个有证据支撑的实例，其中 163 个来自复现报告和 GitHub issue 的真实缺口、497 个注入到可编码参考中的受控合成缺口，用于评测编码就绪度评估、缺陷定位和澄清动作生成三项能力。

## 正文

A research idea may be novel, coherent, and scientifically plausible, yet its proposed method may remain insufficiently specified for faithful implementation. We study the codification readiness of implementation-facing research-method specifications, defined by whether they provide sufficient methodological information for a competent implementer or coding agent to construct the intended method without unsupported assumptions. We construct evidence-grounded specifications and their supported resolutions from papers, codebases, issue threads, and reproduction artifacts. We introduce IdeaAMBIG, a benchmark of 660 evidence-grounded instances: 163 real-world gaps from reproducibility reports and GitHub issues, and 497 controlled synthetic gaps injected into codification-ready references. IdeaAMBIG evaluates three capabilities: codification-readiness assessment, defect localization, and clarification action generation. Defect localization receives only the specification, whereas clarification additionally receives the annotated defect. Across 13 LLMs, the best model achieves 9.6% Macro Defect Recovery Rate on real-world instances but 80.6% Macro Clarification Action Success Rate when given the defect. In an oracle study, supplying the gold resolution raises the downstream codification-ready rate from 14% to 98%. Across all evaluated models, defect localization is the main bottleneck, with stronger clarification given the defect.