X.PIN· @thexpin · X·· 49 分钟前AI 评分72
AI 导读
DeepSeek 宣布将内部推理框架 DeepInfer 完全开源，代码位于 github.com/deepseek-ai/DeepInfer。
正文 · 原文
DeepSeek is open-sourcing DeepInfer, its internal framework for serving models at scale.
Inference frameworks coordinate compute clusters to shape AI throughput, latency, cost, and energy efficiency. DeepSeek says DeepInfer powers high-throughput, low-latency inference for V4.1 Flash.
Supporting both Nvidia and Huawei Ascend hardware, DeepInfer enables distributed inference, speculative decoding, SWA Bounded Replay, and structured outputs. Alongside the framework, DeepSeek also released deepseek-recipe, a production-grade tokenization component fully aligned with its official API.
🚀 Introducing DeepInfer, our in-house inference framework, now fully open source! Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable. 🔗 http://github.com/deepseek-ai/DeepInfer 1/4在 X 查看被引用的帖子
来源：X.PIN · x.com