跳到正文
5月9日周六
  1. Ethan Mollick72

    嗯。 [引用 @METR_Evals]:我们于2026年3月的有限窗口内评估了Claude Mythos Preview的早期版本进行风险评估。在我们的任务套件上,我们估计其50%时间范围至少为16小时(95%置信区间8.5小时至55小时),这处于我们无需新任务即可测量的上限。

    引用METR@METR_Evals

    We evaluated an early version of Claude Mythos Preview for risk assessment during a limited window in March 2026. We estimated a 50%-time-horizon of at least 16hrs (95% CI 8.5hrs to 55hrs) on our task suite, at the upper end of what we can measure without new tasks.

5月8日周五
5月7日周四
5月6日周三
  1. Xiaomi MiMo59

    MiMo V2.5 🥰🥰 [引用 @Designarena]:突发:小米MiMo-V2.5在设计竞技场开放权重模型中综合排名第六! 其Elo评分为1297,与@Kimi_Moonshot的Kimi K2.5(思考版)处于同一性能区间。 祝贺@XiaomiMiMo团队发布成功!

    引用Design Arena@Designarena

    BREAKING: MiMo-V2.5 by @Xiaomi is #6 overall out of open weight models on Design Arena! This has an Elo of 1297 and is in the same performance band as Kimi K2.5 (Thinking) by @Kimi_Moonshot Congrats to the @XiaomiMiMo team on the launch!

  2. Hugging Face 社区博客47

    Open ASR 排行榜引入私有数据集以对抗基准过度优化

    Hugging Face 的 Open ASR Leaderboard 加入了来自 Appen Inc. 和 DataoceanAI 的高质量私有英语 ASR 数据集,涵盖多种口音及脚本式/会话式语音,总时长约 28.7 小时。数据集保持私有以降低 benchmaxxing(针对基准的过度优化)或测试集污染风险。默认平均 WER 仍基于公开数据集计算,用户可通过切换查看私有数据集影响。标准化采用基于 Whisper 的标准化器,UI 代码和评估脚本已开源。自 2023 年 9 月上线以来,该榜单访问量已超过 71 万次。

  3. Jeff Dean50

    Databricks和Perplexity AI联合创始人、Laude Institute创始人Andy Konwinski将于本月晚些时候在首届CAISconf上发表主题演讲。Laude Institute秉持“交付你的研究”使命,通过Moonshots、Slingshots和Open Frontier等项目资助开源AI研究。其支持的智能体基准Terminal-Bench在推出126天后即被列入Anthropic的Claude 4模型卡,并已成为衡量命令行性能的行业标准。CAISconf会议定于5月26日至29日在圣何塞举行。

    引用ACM Conference on AI and Agentic Systems@CAISconf

    📢Keynote announcement: @andykonwinski (Andy Konwinski), co-founder of Databricks and Perplexity AI, founder of @LaudeInstitute, is keynoting #CAIS2026. Laude's "Ship Your Research" mission funds open-source AI research through its Moonshots, Slingshots, and Open Frontier programs. Case in point: Terminal-Bench, a Laude-backed agent benchmark, reached Anthropic's Claude 4 model card 126 days after inception and has since become the industry-wide standard for measuring command line performance. We're thrilled to have him. San Jose · May 26–29 https://caisconf.org

5月5日周二
  1. Luma71

    Luma Labs 推出的 UNI-1.1-Max 和 UNI-1.1 多模态模型在 Image Arena 的文本生成图像与图像编辑综合排名中位列第三,且未采用智能体搜索技术。具体来看,在文本生成图像竞技场中,两款模型分别排名第六和第七;在多图像编辑和单图像编辑竞技场中,它们均进入前十一名,其中 UNI-1.1-Max 在单图像编辑中排名第七。这一成绩标志着 Luma Labs 在多模态前沿领域取得了扎实进展。

    引用Arena.ai@arena

    Exciting news: UNI-1.1-Max and UNI-1.1 debuts making @LumaLabsAI the #3 lab in the Image Arena across both Text-to-Image and Image Edit! These are versions released without agentic search. Text-to-Image Arena - UNI-1.1-Max #6 overall (1193), +12 points over MAI-Image-2 - UNI-1.1 #7 overall (1190), +13 points over Reve-v1.5 Multi-Image Edit Arena - UNI-1.1-Max #7 overall (1315), +21 points over Seedream 4.5 - UNI-1.1 #8 overall, (1298) Single-Image Edit Arena - UNI-1.1-Max #7 overall (1337) - UNI-1.1 #11 overall, (1310) on par with Grok-Imagine-Image (20260207) Congratulations to @LumaLabsAI on this solid performance!

  2. Simon Willison 博客44

    Granite 4.1 3B模型SVG鹈鹕图集

    IBM近日发布了采用Apache 2.0许可证的Granite 4.1大语言模型系列。随后,Unsloth提供了该系列3B模型的21个GGUF量化变体。作者利用这些大小从1.2GB到6.34GB不等的量化模型,尝试生成“鹈鹕骑自行车”的SVG图像以测试其能力。实验结果显示,所有模型生成的图像质量均较差,且未观察到模型大小与输出质量之间存在明显关联。基于此次不理想的尝试,作者表示未来将选用更擅长图像生成的模型重新进行此类实验。

  3. Elon Musk41

    在“Vals AI”的私人基准测试中,Grok 4.3在法律和金融领域展现出领先的智能水平。它在CaseLaw (v2)测试中以79.31%的准确率排名第一,该测试基于真实加拿大法庭案例,评估深度法律推理和先例理解能力,表现优于GPT-5.1。同时,它在针对复杂长期信贷协议的CorpFin (v2)测试中以68.53%的准确率夺冠,评估了对多页金融合同条款、风险的理解。这些模拟高风险现实挑战的测试表明,Grok 4.3在最困难的任务中具备卓越的推理能力。xAI正致力于构建世界所需的推理引擎。

    引用X Freeze@XFreeze

    Grok 4.3 just became the smartest AI in the world at law and money It took #1 on TWO brutal private tests no other model could win on “Vals AI” benchmarks #1 CaseLaw (v2) - 79.31% accuracy Private Q&A benchmark over real Canadian court cases. Tests deep legal reasoning, precedent understanding, and precise answers from complex judgments. (outranking GPT-5.1 at 73.42%) #1 CorpFin (v2) - 68.53% accuracy Private benchmark on long-context credit agreements. Evaluates how well models truly understand dense, multi-page financial contracts, terms, risks, and clauses These are not just basic tests - they’re real-world, high-stakes legal + financial reasoning challenges Grok 4.3 leads in accuracy on both, proving it’s not just fast or cheap… it’s the smartest at the hardest real world tasks xAI is building the reasoning engine the world needs

5月3日周日
  1. The Decoder:AI News54

    美国政府基准测试称中国在AI竞赛中落后,但独立数据并不支持

    美国政府机构评估称中国在人工智能竞赛中落后八个月,但独立数据并未证实这一差距。当前美国实验室持续追求更智能的模型,而中国玩家如深度求索(Deepseek)等提供的价格优势可能成为更关键的竞争筹码。这场竞赛的衡量标准正从单纯的技术指标扩展到包括成本效益在内的综合维度。

  2. The Decoder:AI News41

    Same prompt, different morals: 前沿AI模型在伦理困境上的分歧

    一项新基准测试让领先的语言模型处理100个日常伦理场景,涵盖从销售数据滥用到肿瘤学违规操作等领域。测试结果显示,不同前沿模型对相同伦理提示给出了差异显著的回应。这引出了一个核心问题:究竟由谁来决定AI被允许做什么,以及它应遵循谁的伦理准则?该基准旨在揭示和量化主流AI系统在道德判断上的不一致性。

5月2日周六
  1. The Decoder:AI News53

    ARC-AGI-3 分析显示,即便是最新 AI 模型仍存在三种系统性推理错误

    ARC Prize Foundation 对 OpenAI 的 GPT-5.5 和 Anthropic 的 Opus 4.7 在 ARC-AGI-3 基准测试中的 160 次任务运行进行了分析。研究发现,三种系统性错误模式导致这两个模型在人类能轻松解决的任务上得分均低于 1%。这些错误揭示了当前顶尖大语言模型在抽象推理能力上仍存在根本性缺陷。

  2. Elon Musk39

    Grok Voice 目前正被 Starlink 使用 [引用 @XFreeze]:Grok Voice 在 τ-voice 基准测试中占据绝对优势 Grok 得分为 67.3%,而 Gemini 为 43.8%,GPT Realtime 为 35.3% 这遥遥领先于竞争对手,优势巨大 目前最优秀的实时推理语音助手

    引用X Freeze@XFreeze

    Grok Voice brutally dominates the top of the τ-voice Bench Grok scores 67.3%, while Gemini sits at 43.8% and GPT Realtime at 35.3% This is a massive lead over the competitors and it's not even close The best real-time reasoning voice agent out there

  3. TestingCatalog News 🗞66

    谷歌新的Gemini Flash模型已在LM Arena上出现。同时,Vertex AI客户收到邮件,Gemini 3.1 Flash Lite即将正式发布。引用推文指出,虽然模型在竞技场中仍显示为“Gemini 3 Flash”,但其输出质量已跃升两个层级,性能更接近当前的Gemini 3.1 Pro,是一次重大升级,实际版本可能是3.1、3.2或3.5 Flash。

    引用can@marmaduke091

    🚨 Google updated Gemini 3 Flash in arena It still has the same name "Gemini 3 Flash". However, output quality is two tiers above it. It could be 3.1, 3.2 or 3.5 Flash. Not sure what they'll call it. It's performance is closer to current 3.1 Pro than the current 3 Flash. Huge upgrade.

  4. François Chollet70

    强化学习在已知领域能提升模型性能,但在未知领域可能导致模型产生幻觉,误以为在执行其他训练过的任务。这一现象在GPT-5.5等大模型的ARC AGI 3基准测试中有所体现,其得分仅为0.43%,与Claude 4.6、Gemini 3.1等模型表现相近。分析指出GPT-5.5的主要失败原因包括:局部效应正确但世界模型错误、从训练数据中提取的抽象层级不当,以及虽解决问题却未强化奖励机制。深入分析此类失败案例,有助于全面理解大模型在特定模态上的能力局限与改进方向。

    引用Chris@chatgpt21

    GPT-5.5 在 ARC AGI 3 上得分 0.43%! - GPT-5.5:0.43% - Opus 4.7:0.18% - GPT-5.4:0.20% - Claude 4.6:0.45% - Gemini 3.1:0.4% GPT 5.5 报告的失败原因包括: - 真实的局部效果,错误的世界模型 - 来自训练数据的抽象层级错误 - 解决了关卡,但没有强化奖励 我认为完整的分析将帮助 OpenAI 全面理解模型在某些模态中失败的地方

    推荐理由:Chollet 用 ARC AGI 3 冷冰冰的数字撕开了 RL 的局限,GPT-5.5 0.43% 的得分说明在未知领域模型会做完全不相干的事,比任何安全论文都来得更直击要害。

5月1日周五
  1. PixVerse49

    主推文感谢了用户@TomLikesRobots分享的文本生成视频模型对比。对比在SeeDance 2.0和HappyHorse 1.0之间进行,使用了统一的提示词来生成具有低保真、温馨、赛璐珞风格动漫美学的视频。其中,HappyHorse由@PixVerse_提供,目前对会员免费。由于两个模型自带的音频效果不佳,创作者最终使用@Suno来生成背景音轨。

    引用TomLikesRobots🤖@TomLikesRobots

    SeeDance 2.0 vs HappyHorse 1.0 Very quick text-to_video comparison. Which do you prefer? Universal Prompt: "Aesthetic: lofi, cozy, warm tones, soft film grain, gentle bloom, analog textures. Camera default: 35 to 50mm equivalent, shallow depth of field, slow subtle moves, cut length 0.6 to 1.2 seconds. Style: Cozy cel-shaded anime." Thanks to @PixVerse_ for HappyHorse generations which are currently free to members. lofi audio tracks were pretty poor for both so I used @Suno for audio.

  2. TestingCatalog News 🗞55

    Grok 4.3 在 Artificial Analysis Index 中升至第 7 位,超越了 Meta 的 Muse Spark。

    引用Artificial Analysis@ArtificialAnlys

    This release shows increased cost efficiency to run the Artificial Analysis Intelligence Index, with Grok 4.3 sitting comfortably on the Pareto frontier for intelligence versus cost Driven by 37.5% lower input token prices and 58.3% lower output token prices, it costs $395 to run the Intelligence Index evaluations, an overall ~20% decrease from Grok 4.20 0309 v2

  3. Mark Chen39

    这只是一项评估,但很重要——英国 AISI 的网络靶场测试了长周期、智能体的能力。5.5 的表现与 Mythos 相似。 前沿模型的风险是真实存在的。但我们尽最大努力通过缓解措施来部署人们真正能用的 AI。

    引用AI Security Institute (AISI)@AISecurityInst

    OpenAI’s GPT-5.5 is the second model to complete one of our multi-step cyber-attack simulations end-to-end 🧵

  4. Ethan Mollick61

    xAI发布Grok 4.3,其在Artificial Analysis智能指数得分53,性能优于Grok 4.20、Muse Spark等模型。核心改进在于“性价比”:输入与输出价格较前代分别降低约40%和60%,且基准测试套件运行成本下降。该版本在GDPval-AA等现实智能体任务上表现显著提升,指令遵循与客服任务强劲。但推文指出,其表现仍落后于最新的中国开源模型,并批评GDPval-AA测试本身价值有限。

    引用Artificial Analysis@ArtificialAnlys

    xAI has launched Grok 4.3, achieving 53 on the Artificial Analysis Intelligence Index with improved agentic performance, ~40% lower input price, and ~60% lower output price than Grok 4.20 The release of Grok 4.3 places @xAI just above Muse Spark and Claude Sonnet 4.6 on the Intelligence Index, and a 4 points ahead of the latest version of Grok 4.20. Grok 4.3 improves its Artificial Analysis Intelligence Index score while reducing cost to run the benchmark suite. Key Takeaways: ➤ Grok 4.3 improves on cost-per-intelligence relative to Grok 4.20 0309 v2: it scores higher on the Intelligence Index while costing less to run the full benchmark suite. Grok 4.3 costs $395 to run the Artificial Analysis Intelligence Index, around 20% lower than Grok 4.20 0309 v2, despite using more output tokens. This makes it one of the lower-cost models at its intelligence level ➤ Large increase in real world agentic task performance: The largest single benchmark improvement is on GDPval-AA, where Grok 4.3 scores an ELO of 1500, up 321 points from Grok 4.20 0309 v2’s score of 1179 Grok 4.3, surpassing Gemini 3.1 Pro Preview, Muse Spark, Gpt-5.4 mini (xhigh), and Kimi K2.5. Grok 4.3 narrows the gap to the leading model on GDPval-AA, but still trails GPT-5.5 (xhigh) by 276 Elo points, with an expected win rate of ~17% against GPT-5.5 (xhigh) under the standard Elo formula ➤ Grok 4.3’s performs strongly on instruction following and agentic customer support tasks. It gains 5 points on 𝜏²-Bench Telecom to reach 98%, in line with GLM-5.1. Grok 4.3 maintains an 81% IFBench score from Grok 4.20 0309 v2 ➤ Gains 8 points on AA-Omniscience Accuracy, but at the cost of lower AA-Omniscience Non-Hallucination Rate of 8 points, so Grok 4.20 0309 v2 still leads AA-Omniscience Non-Hallucination Rate, followed by MiMo-V2.5-Pro, in line with Grok 4.3 Congratulations to @xAI and @elonmusk on the impressive release!

  5. Sam Altman43

    lisan 多说点我们的坏话 你太客气了 [引用 @scaling01]:GPT-5.5 is on par with Claude Mythos - GPT-5.5 平均通过率 71.4% (±8.0%) - Mythos Preview 68.6% (±8.7%) - GPT-5.5 在 11 分钟内以 1.73 美元成本完成了一项人类专家需约 12 小时的任务

    引用Lisan al Gaib@scaling01

    GPT-5.5 is on par with Claude Mythos - GPT-5.5 average pass rate of 71.4% (±8.0%) - Mythos Preview 68.6% (±8.7%) - GPT-5.5 solved a task that takes a human expert ~12 hours in under 11 minutes at a cost of $1.73