跳到正文
The Decoder:AI News· Jonathan Kemper·· 1 小时前AI 评分63

新实验:AI 编码智能体在模拟软件公司中完成不到十分之一的任务

AI coding agents ace isolated tests but complete fewer than one in ten tasks in a simulated workplace

AI 导读

一项新实验首次让 AI 编码智能体在模拟软件公司中作为数字员工工作,环境包含同事、项目看板和聊天记录。结果显示即使 Claude Opus 5.5 和 GPT-6 Astra 也只完整完成不到十分之一的任务,而 SWE-bench 等标准基准已接近饱和。

正文 · 原文

Image description

ChatGPT-5.6 prompted by THE DECODER

AI coding agents are nearing top scores on standard benchmarks. A new experiment puts them to work for the first time as digital employees inside a simulated software company with colleagues, project boards, and chat histories. Even Claude Opus 5.5 and GPT-6 Astra fully complete fewer than one in ten tasks.

Success stories about AI coding agents keep piling up. Anthropic says Claude writes more than 80 percent of the company's production code, and standard benchmarks like SWE-bench are nearly saturated.

来源:The Decoder:AI News · the-decoder.com