#评测/基准

SpaceXAI launched Grok Voice 3.0 tops Artificial Analysis’s Speech-to-Speech Index at 86.9, beating OpenAI and Google. The index blends spoken reasoning, simulated support calls, blind preference votes and task success, where GPT-Live-1 (Astra, Medium) reaches 83.2. The largest gain comes on τ-Voice, where a simulated caller asks the model to rebook flights or dispute charges using domain tools and a policy document. There, Grok’s success rate jumped from 56.5% under Grok Voice 2.0 to 71.6%. Grok’s edge is speed: it answers 40% sooner than GPT-Live-1, whose 1.34 seconds implies roughly 0.8 seconds, slightly slower than Grok Voice 2.0’s 0.70. Introducing Grok Voice 3.0 +2 It also costs $0.025 per task on Artificial Analysis’s Big Bench Audio run, against $0.030 for GPT-Live-1 and $0.055 for GPT-Realtime-2.1 (High).

Real-world results are in for Claude Haiku 5.5 (High) by @AnthropicAI. Debuting at #30 with 1587 pts in Code Arena: WebDev, this release sits just outside the Pareto frontier while still offering strong cost efficiency. It matches GPT‑6 Luna’s price at $0.10/$0.50 per 1M input/output tokens while scoring six pts higher. Compared with other Claude models, Haiku 5.5 offers: - 90% lower cost than Haiku 4.5 at +257 pts - 95% lower cost than Sonnet 5.5 (High) at -128 pts - 98% lower cost than Opus 5 (High) at -70 pts - 99% lower cost than Fable 5.1 (Max) at -157 pts Congrats to the team at @AnthropicAI on this release!

Real-world results are in for Claude Haiku 5.5 (High) by @AnthropicAI. Debuting at #30 with 1587 pts in Code Arena: WebDev, this release sits just outside the Pareto frontier while still offering strong cost efficiency. It matches GPT‑6 Luna’s price at $0.10/$0.50 per 1M input/output tokens while scoring six pts higher. Compared with other Claude models, Haiku 5.5 offers: - 90% lower cost than Haiku 4.5 at +257 pts - 95% lower cost than Sonnet 5.5 (High) at -128 pts - 98% lower cost than Opus 5 (High) at -70 pts - 99% lower cost than Fable 5.1 (Max) at -157 pts Congrats to the team at @AnthropicAI on this release!

685 output tokens/s per user on @Alibaba_Qwen Qwen3.8-Flash-Next. Running TokenSpeed on 4× NVIDIA GB300 with NVFP4. A look at the kernel and serving optimizations behind the results: 👉 https://lightseek.org/blog/tokenspeed-qwen3-8-flash-next.html ⭐ https://github.com/lightseekorg/tokenspeed

我们很高兴推出 Muse Wonder 系列，包括 Muse Wonder 和 Muse Wonder Full，这是我们迄今为止最强大的模型。两者都是前沿模型，旨在为个人和专业代理任务提供最先进的性能。 Muse Wonder 今天可通过 Meta Model API 和 Muse Code 使用。Muse Wonder Full 今天面向 Muse Code 订阅者提供。Muse Wonder 很快还将为我们的个人代理产品 Muse 中的体验提供支持，首先向订阅者推出。

Gemini 4 Argon 来了。 今天起面向全球开发者与企业开放，可通过 Gemini API 和 Gemini 平台的付费层级使用；Ultra 订阅者可在 Google Antigravity 中使用；Ultra 和 Pro 订阅者可在 Gemini 应用和搜索中使用。 希望大家喜欢！

Gemini 4 Argon (High) ranks in the top 5 for Agent Arena. Congrats to the @GoogleDeepMind team! Read more about the latest with Agent Arena in thread.

DeepSeek V4.1 Flash from @deepseek_ai on ARC-AGI (Verified): - ARC-AGI-2: 72.9%, $0.13/task - ARC-AGI-1: 94.5%, $0.07/task DeepSeek V4.1 Flash beats V4 Flash's best scores by 5.5 points on ARC-AGI-1 and 11.5 on ARC-AGI-2, but costs about 250% more per task.

Jev as a judge for LLM evals! JevEval uses Jev as the judge for custom LLM evals instead of relying on another generative LLM for every evaluation. The idea is pretty simple. You define the exact decisions you want the evaluator to make, Jev evaluates each one and returns probabilities, and DeepEval uses those results to calculate the final score. For example, with an AI agent you can evaluate the trace itself, checking whether the agent selected the right tools, followed the expected sequence of actions, and actually completed the task. Jev handles those semantic judgments, while the scoring logic stays outside the model. That is the main difference from a normal LLM-as-a-judge setup. With a generative judge, the model often has to interpret the rubric, reason about the response, and generate the evaluation itself. With JevEval, you define the decision structure upfront. You can also assign different weights to different checks or use `strict_mode` when every requirement needs to pass. So if one part of the agent workflow matters more than another, you can reflect that directly in the scoring logic instead of leaving the entire evaluation to the judge model. Key capabilities: - Evaluate agent outputs and traces with Jev - Define custom checks as bounded decisions - Use yes/no, score, and closed-choice questions - Weight different checks or require all of them to pass - Evaluate both single-turn outputs and multi-turn conversations This is where Jev makes a lot of sense for evals. If you already know what needs to be checked, you can define those decisions upfront, let Jev handle the semantic judgment, and keep the final scoring deterministic. I’ve shared the DeepEval post in the comments.

We're launching our leaderboard for FrontierCode 1.1, a private evaluation from Cognition that tests models on completing coding tasks in a way that would actually get merged by maintainers of real-world repositories @cognition announced FrontierCode in June 2026 and updated it to version 1.1 in July. Cognition built the dataset with open source maintainers who spent more than 40 hours per task. Each task requires the agent to make a change to a real open source repository, then checks both unit tests and qualitative tests for what maintainers would require to merge the change. We run FrontierCode 1.1 Main, the primary 100-task set of the benchmark's 150 tasks, using our Stirrup open source reference harness for apples-to-apples comparisons across models. All tasks are private to avoid contamination. Each attempt is graded with both blocking and non-blocking criteria. An attempt that fails any blocking check scores zero, and one that passes them all scores between 70% and 100%. Unlike other coding evaluations, FrontierCode has a more holistic view of success based on real usage. Models need to pass unit tests, but are also assessed on behaviors like making excessive changes and large diffs, modifying files outside the relevant scope, or adding unhelpful tests. We think this approach is valuable in the current context: models are now very capable at coding tasks, but their changes can still require substantial cleanup before a maintainer would merge them. Key takeaways: ➤ Leading models still fail more than 40% of the time: Claude Opus 5.5 (max with fallback) leads at a score of 52% after passing blockers in 58% of its attempts. Nine of the 100 tasks were not passed in any attempt by any of the models we evaluated for launch ➤ The top of the leaderboard is tightly packed: nine models from six labs score between 43% and 52%. MiMo-V2.6-Pro is the leading open weights model at 45%, followed by DeepSeek V4.1 Flash (max) and Kimi K3 (max) at 44% ➤ GPT-6.1 Sol (max) offers the strongest performance for its cost: At $0.81 per task it scores 49%, matching Claude Fable 5.1 (max with fallback). This means it achieves similar performance to leading Anthropic models, at less than a tenth of the cost per task Thank you to @silasalberti, @ybenpan, @cl571128, and the rest of the team at Cognition for building FrontierCode and collaborating with us to bring it to Artificial Analysis - we’re working to add FrontierCode to our coding agent leaderboard very soon.

MiniCPM-V 4.7 beat Qwen 3.6 35B-3A at Pagoda 🏯 @OpenBMB's new 35B-A3 model made a better 3D Voxel Japanese Pagoda while consuming 10% less tokens than Qwen 3.6 35B-3A Run AI models locally -> https://atomic.chat

Meta Muse's retention is out, and it's almost as good as ChatGPT! It's higher than ChatGPT at D7 (>30%) and just lower at D14 (22%). Not quite in the elite D14 tier of WA (67%) YT (59%) Insta (54%) or TikTok/Snap (37%), though. Muse is one of the highest retaining AI products!

I appreciated @OpenBMB giving a few of us an early look at these new models. They are the real deal. When I put MiniCPM-V 4.7 35B A3B through my own personal agent benchmarking (see comments for more info on what I test and why), it outperforms 35B MoE incumbents that one would normally reach for. Lite Agent testing (A1) was a perfect result, and PA4 (bringing a complex agent loop workflow together with lots of context and tool calls) was surprisingly strong. This is even before looking at the new vision and other capabilities, but as a life admin 'brain' - it's quite stellar.