# τ^τ-bench 评测：最强编码智能体设置在真实客户模拟中仅通过 23.9%

- 来源：Rohan Paul (@rohanpaul_ai)
- 发布时间：2026-09-21 03:47
- AIHOT 分数：54
- AIHOT 链接：https://aihot.news/items/cmua96ce907maro5tsjn3ebd4
- 原文链接：https://x.com/rohanpaul_ai/status/2101760224713687437

## AI 摘要

Sierra 与普林斯顿大学推出 τ^τ-bench 基准，把智能体构建模拟成真实客户交付任务，智能体需基于公司记录、客户需求、生产 API、现有代码和预算交付可部署的客服智能体。

## 正文

The best coding-agent setup passed only 23.9% of held-out customer simulations on this benchmark.

ττ-bench finds that today's coding agents fail most realistic agent-building work because they do not understand requirements deeply enough, so improving code generation alone will not solve the problem.

ττ-bench treats agent building like a real client job.

The model gets scattered company records, a client, an API, existing code, and a budget, then must ship an agent for unseen customer requests.

The best setup, Claude Opus 5 with Claude Code, passed just 23.9% of held-out customer simulations across 53 tasks, versus 82.2% for the expert-authored reference.

Most failures were outside pure coding.

Agents searched records instead of understanding them, barely questioned the client, reused familiar designs, and wrote tests that often missed their own errors.

On client-enabled tasks, builds asking 0 questions averaged 0.16; those asking 4+ averaged 0.50.

So, better coding models are not enough, we also need coding agents that gather requirements, compare designs, use budgets intelligently, and run tests that expose their own blind spots before deployment.