新实验:AI 编码智能体在模拟软件公司中完成不到十分之一的任务
AI coding agents ace isolated tests but complete fewer than one in ten tasks in a simulated workplace
一项新实验首次让 AI 编码智能体在模拟软件公司中作为数字员工工作,环境包含同事、项目看板和聊天记录。结果显示即使 Claude Opus 5.5 和 GPT-6 Astra 也只完整完成不到十分之一的任务,而 SWE-bench 等标准基准已接近饱和。

ChatGPT-5.6 prompted by THE DECODER
AI coding agents are nearing top scores on standard benchmarks. A new experiment puts them to work for the first time as digital employees inside a simulated software company with colleagues, project boards, and chat histories. Even Claude Opus 5.5 and GPT-6 Astra fully complete fewer than one in ten tasks.
Success stories about AI coding agents keep piling up. Anthropic says Claude writes more than 80 percent of the company's production code, and standard benchmarks like SWE-bench are nearly saturated.
来源:The Decoder:AI News · the-decoder.com