Ant Ling· @AntLingAGI · X·· 3 小时前AI 评分60
AI 导读
SGLang 与 Google TPU 团队、Meta PyTorch 团队合作发布 SGLang-TorchTPU，这是 SGLang 的 TPU 插件，已在 Meta 生产推理中使用，代码开源于 github.com/sgl-project/sglang-torchtpu。它支持 Muse Spark Open、GLM-5.3 Flash、Kimi K3、Ling 3.1 Flash、MiniMax H3 等开源模型及大规模智能体工作负载，自定义 MegaKernel 让 Muse Spark Open 在 InferenceX 8K/1K、bs=1 下达到 770 tok/s 解码。蚂蚁百灵转发表示，Ling 3.1 Flash 和 Ling 3.0 家族入选发布阵容，PyTorch 原生 TPU 服务扩展了 MoE 推理的硬件选择。
正文 · 原文
Excited to see Ling 3.1 Flash and the Ling 3.0 family in the SGLang-TorchTPU launch lineup! Thanks @sgl_project, Google and Meta for bringing PyTorch-native serving to TPUs—expanding hardware choice for efficient MoE inference and agentic workloads.
Today we launch SGLang-TorchTPU, built with @Google TPU team and Meta @PyTorch team. We’re excited to see it powering production inference at Meta! SGLang-TorchTPU is a TPU plugin for SGLang that brings: - Top open models on TPU: Muse Spark Open, GLM-5.3 Flash, Kimi K3, Ling 3.1 Flash, MiniMax H3 - SGLang’s production features for large agentic workloads - Custom MegaKernel: Muse Spark Open hits 770 tok/s decode at bs=1 on InferenceX 8K/1K SGLang-TorchTPU is available now: http://github.com/sgl-project/sglang-torchtpu在 X 查看被引用的帖子
来源：Ant Ling · x.com