跳到正文
Rohan Paul· @rohanpaul_ai · X·· 2 小时前AI 评分63
AI 导读

CMU 等机构 11 月 12 日发布 arXiv 论文 Singularity-Bench,让智能体完成预训练、推理引擎、集群编排等 6 项端到端任务,并在相同 GPU 上与 SGLang、Claude Code 等生产系统对比打分。

正文 · 原文

New Carnegie Mellon paper finds that Opus 5.5 + Claude Code built a pre-training pipeline, an inference engine, and a cluster orchestrator that matched or beat production tools like SGLang and NVIDIA Dynamo on the same GPUs.

Singularity-Bench gives agents 6 end-to-end jobs, from pre-training a model to building a serving engine. Each result is scored against a production system, such as SGLang or Claude Code, on the same GPUs.

Opus 5.5 scored a Singularity-Index of 1.25, where 1 equals the human baselines, while the other 10 agents averaged 0.58. Its serving engine earned 11.52 USD per GPU-hour against 8.18 for SGLang.

来源:Rohan Paul · x.com