# Andon Labs 评测：GPT-6 Astra 在 Vending-Bench 2 和 Drone-Bench 上大幅领先 Claude Fable 5.1

- 来源：The Decoder：AI News（RSS）
- 作者：Tomislav Bezmalinović
- 发布时间：2026-09-13 18:52
- AIHOT 分数：70
- AIHOT 链接：https://aihot.news/items/cmtzpqxc403rzroxqf54hxz65
- 原文链接：https://the-decoder.com/gpt-6-astra-pilots-a-surveillance-drone-and-runs-a-business-on-its-own

## AI 摘要

Andon Labs 测试显示 OpenAI 的 GPT-6 Astra 在两个 Agent 基准上超越所有以往前沿模型。

## 正文

Key Points

Andon Labs tested GPT-6 Astra on two agent benchmarks where the model scored far better than Claude Fable 5.1.

Running a simulated vending machine business, Astra averaged $15,515 in final bank balance, nearly three times Fable's result.

On Drone-Bench, Astra became the first model to beat the human-AI baseline on all five subtasks, including writing code that lets a drone autonomously find and follow a specific person.

Andon Labs tested GPT-6 Astra on two very different agent benchmarks. When buying inventory and running a vending machine, the OpenAI model crushes Claude Fable 5.1. On drone surveillance, Astra is the first model to beat the human-AI baseline on all five subtasks, though its success rate remains unreliable.

OpenAI's GPT-6 Astra outperforms all previous frontier models on two agent benchmarks from Andon Labs. The research lab uses Vending-Bench and Drone-Bench to measure how well AI models act independently over long periods or write software for physical systems.

In a simulated vending machine business, Astra earned nearly three times as much as Claude Fable 5.1. On Drone-Bench, Andon Labs says Astra is the first model whose best attempts beat the human-AI-developed baseline across all five subtasks.

Astra negotiates harder and spends less than Claude Fable 5.1

In Vending-Bench, each model gets $500 and has to run a vending machine over a simulated year. It finds suppliers, negotiates purchase prices, orders goods, sets retail prices, and tries to grow its bank balance.

Across six runs, GPT-6 Astra averaged $15,515, according to Andon Labs. Claude Fable 5.1 averaged $5,422. Even Fable's best run at $9,874 fell well short of Astra's worst result of $13,272. Astra is the first OpenAI model to top the Vending-Bench 2 leaderboard. The gap to the second-place model is also the largest the benchmark has ever seen, according to Andon Labs.

Final bank balances from six Vending-Bench 2 runs per model. Bars show averages; dots show individual runs. Every single Astra run beats every Fable run. | Image: Andon Labs

One of the biggest differences shows up in procurement. Fable accepts worse deals over time. For a regular can of Coca-Cola, its average purchase price rises from $1.17 in the first 90 days to $2.21 toward the end of the simulated year. Astra negotiates more consistently. In one case Andon Labs documented, a supplier quoted $226.32 for a basket of goods. Astra held firm at $108 and got the deal.

Astra also handles unreliable suppliers better. Across six runs, Fable 5.1 made 45 prepayments to suppliers that had already shut down, losing $14,331. Astra encountered even more closures at 64, but Andon Labs says it recorded no identified losses from such prepayments. Fable recognized the problem and wrote a rule to only pay after written confirmation. Days later, the model broke its own rule.

Astra refuses price-fixing schemes

Andon Labs also tests models in Vending-Bench Arena, where multiple AI agents run competing vending machines at the same location. Astra explicitly refused a price-fixing proposal from the Chinese model GLM-5.3. Andon Labs observed no instances of lying from Astra across the three arena games it studied.

Claude Fable 5.1 participated in what Andon Labs classified as an illegal price-fixing arrangement with GLM-5.3. Fable only honored the agreement when it served its own interests. Astra won all three games.

Andon Labs rates Astra as both a stronger economic performer and better aligned, though that assessment is based on behaviors observed in the benchmark and doesn't automatically transfer to other situations.

Astra is the first model to beat all five Drone-Bench tasks

Drone-Bench tests a different kind of agent capability. Models write code that lets a cheap DJI Tello EDU drone autonomously navigate an office, identify a specific person, and follow them. The benchmark has five steps: 3D reconstruction of the environment, drone localization, navigation, target person detection, and tracking.

Each task is scored individually against code that a human developer built with coding agents for Andon's own demo. Every model gets ten runs per task and can submit up to ten code versions per run. After each attempt, it receives a score and can improve its solution.

GPT-6 Astra's 3D reconstruction of the office environment. | Image: Andon Labs

In the original paper from July, Claude Fable 5 was the strongest model. Frontier models had beaten the human-AI baseline on four of five tasks in at least one run. 3D reconstruction remained unsolved. Andon Labs reported that Astra is the first model whose best submissions beat the baseline on all five Drone-Bench tasks, including reconstruction.

Astra built a pipeline combining COLMAP and DA3 with added depth filtering. The model used office video footage to generate a navigable 3D model that scored higher than the human-AI reference solution, according to Andon Labs.

Best-case scores don't mean reliable performance

On person detection, Astra beats the baseline in four out of ten runs. On 3D reconstruction, it manages that in just one out of ten. Andon Labs calculates that an average Astra run has only a 2.8 percent chance of passing all five steps in sequence.

Astra proved for the first time that a general-purpose frontier model can produce code above the baseline for every part of the task. But multiply the probabilities for a complete end-to-end run, and the odds are still low. Based on progress over the past two years, the team projects that a frontier model could solve all five tasks in a single attempt by Q1 2027.

GPT-6 Astra already works as a surveillance drone

In a demo from Andon Labs, GPT-6 Astra flies a drone autonomously through an office with the prompt "ChatGPT, find this person and follow them." The model identifies a specific person and tracks them. Spatial mapping, navigation, and person tracking all run without any human input. Other benchmarks also show that GPT-6 Astra has particularly strong spatial reasoning.

视频 · 前往原文观看

When critics questioned why they were building the kind of technology everyone keeps warning about, Andon Labs responded that the benchmark doesn't help AI fly drones but measures how well current models can already do it. Six months ago, frontier models failed at these tasks and crashed. Astra now beats the human baseline on every subtask.

Andon Labs argues that the public and lawmakers need to know about these capabilities before AI-powered drones reach superhuman navigation skills. No lab has access to the benchmark. Andon Labs runs all evaluations itself to prevent companies from optimizing their models for the test.

Andon Labs Vending-Bench

Andon Labs Drone-Bench

Andon Labs @ X