#论文/研究

Introducing ✨Token-Gated Language Models✨ for up to 11x faster prefill by learning to skim context, conditionally forwarding only tokens that matter. Akin to the shift from dense models to MoEs, the next frontier of efficient LLMs is conditional computation over a new scaling axis: context tokens🧵👇

Introducing http://AgentReady.org, the first open standard for agent experience. Agents are a new class of user, but there's still no shared definition of what it takes to be ready for them. So we partnered with the leading companies in the agentic web such as Vercel, Stripe, Auth0, Mintlify, Tavily and Elementor. Our research lab ran agents on real tasks more than 250,000 times to measure what actually decides whether an agent succeeds, and every requirement traces back to that data. The result is a short set of essentials in the order agents meet a product: find, read, act. We've been overwhelmed by requests to join. If you build or study the surfaces agents touch, come help shape it. The spec: http://agentready.org The story behind it: http://ora.ai/blog/agentready-launch

LLMs deployed in specialized domains, from enterprise codebases to law firms, dig through a mountain of unstructured context! Our new paper, Librarian Agents, asks: can an agent adapt by curating its own long-context workspace, anticipating queries before they arrive? 🧵⬇️

New research: We developed JOLT, a post-training method where one model acts as both teacher and student, with the teacher trained to give feedback the student can use. On MATH, it matched GRPO's accuracy with 6.5x fewer training tokens. https://www.perplexity.ai/hub/blog/a-good-teacher-meets-the-student-where-they-are

GPT-Rosalind surfaced a new lead for drug discovery: could restoring cholesterol transport protect neurons from degeneration? DCTN1 is a gene linked to Perry syndrome, a rare neurodegenerative disorder. In mouse-brain data, disrupting it increased expression of genes involved in making cholesterol. GPT-Rosalind suggested a possible explanation: cholesterol may not be reaching where neurons need it. That gives scientists a specific mechanism to test and, if confirmed, a direction for developing new treatments for neurodegenerative disorders. With @perturbai_tx and @scrippsresearch, a team of thousands of GPT-Rosalind agents worked in parallel, connecting gene-expression results from millions of mouse brain cells with scientific literature in just 32 minutes. We’re now expanding this collaboration to identify new drug targets. https://openai.com/index/scaling-ai-research-for-complex-biological-data/

DeepSeek 居然把支撑他们大规模生产服务的高性能推理框架开源了：DeepInfer 这是 DeepSeek 线上真实跑 DeepSeek 服务的同源技术栈首次对外公开，本次发布聚焦离线推理，采用 Apache 2.0 许可证，主要基于 @vllm_project 构建，并借鉴了 @sgl_project 和 TensorRT-LLM。 开源地址：https://github.com/deepseek-ai/DeepInfer DeepSeek 此前已陆续开源了推理栈的“零件”：FlashMLA（MLA 注意力内核）、DeepGEMM（矩阵乘法内核）、DeepEP（MoE 通信库）、EPLB（专家负载均衡）等，但一直缺少把它们组装成完整引擎的“骨架”。DeepInfer 补上了这一环：它是这些内核的集成者，加上调度、并行、投机解码等系统层的完整实现。配合本次同时出现的 DeepSeek-V4.1-Flash 模型，构成了“模型 + 引擎”的完整开源推理方案。硬件层面同时支持 NVIDIA SM100 GPU 和华为昇腾 950 NPU，双平台支持是它的一个鲜明特征。 # 核心技术拆解 1. 内核与运行时。 集成 FlashMLA、DeepGEMM、DeepEP、TileKernels、DeepSelect，以及用 TileLang 编写的自定义内核（位于 deepinfer/model_executor/layers/tilelang_ops）。这些是吞吐和延迟的底层保障。 2. 并行策略：Attention 与 MoE 分治。 注意力部分采用 DP（数据并行）或均衡的 CP（上下文并行，处理超长序列）；MoE 部分采用 EP（专家并行）加 EPLB（专家并行负载均衡），并支持冗余专家（默认 32 个，向上取整到能被 EP 数整除）来缓解专家负载不均。这种“注意力 DP/CP + MoE EP”的组合是 DeepSeek 生产验证过的架构。 3. DSpark 投机解码。 基于 MTP（multi-token prediction）draft token 加速解码，亮点是“置信度调度验证”；按置信度决定接受多少 draft token，而非固定规则。默认最大 draft 长度 5。这是推理加速比的主要来源之一。 4. YoCo SWA 有界重放。 针对 YoCo 式解码器 + 滑动窗口注意力（SWA）的优化：prefill 时只保留并重放有界的 token 后缀（默认 140 个，需覆盖 SWA 窗口加最大回溯长度），大幅削减 decoder prefill 的计算量。仓库还专门提供了 YoCo mask/compact 等价性验证。 5. 异步调度。 CPU 上的批调度与 GPU/NPU 上的执行重叠，做到“零开销调度器”，默认开启。 6. 批不变与确定性推理。 开启 DEEPINFER_BATCH_INVARIANT=1 后，不同批次组合下结果完全一致；再配合固定 seq_seed、关闭 MTP varlen，可实现逐 token 确定性的采样输出。这对可复现评测和回归测试（仓库用逐样本精确对比的 PPL 基线来把关）很关键，目前仅限 NVIDIA 平台。 7. 结构化输出。 原生支持 regex、JSON schema、EBNF 三种约束生成，通过 SamplingParams 的 so_args 传入。

New on our Science Blog: designing protein binders against scorpion neurotoxins. Scorpions sting about 1.2m people every year, and antivenoms are often costly or in short supply. Claude has made an early step toward finding new treatments. https://www.anthropic.com/research/claude-scorpion-toxin-binder-design