长时程智能体已经把 LLM 服务变成了一种输入密集型负载。反复的预填充和百万级 token 上下文留下的 KV cache 给 HBM、SSD 容量和带宽带来了巨大压力。DeepSeek AI围绕这一瓶颈打造了其最新发布。DeepSeek-V4.1-Flash是一款多模态混合专家模型，拥有 552B 骨干参数、196B 额外 Engram 参数，以及 1M token 上下文窗口。它在预填充阶段每个 token 激活 8B 参数，在解码阶段激活 16B。核心数字是每 token 全局 KV cache 占用仅 890 字节，约为 DeepSeek-V4-Flash 的 1/4，比 DeepSeek-V1 小约 437 倍。
它能部署吗？能。开放权重以MIT 许可证发布，在 Hugging Face 上提供 vLLM、SGLang 和 Transformers 路径，研究团队还描述了一个公开 API，设有低、高、最高三档推理层级。
因果编码器-解码器：预填充减半
40 层骨干被拆分为 20 层因果编码器和 20 层解码器。受YOCO启发，解码器不计算自身的全局 KV。相反，逐层投影权重从编码器最终隐藏状态中推导出它。因此提示词 token 止步于编码器，这几乎将预填充计算量减半。128 token 窗口的滑动窗口注意力（SWA）仍在每一层运行，因此解码器 SWA 状态只需重放最后 128 个提示词 token 即可重建。研究团队称之为解码器 SWA 有界重放。
压缩稀疏注意力 2（CSA2）
DeepSeek-V4 将 CSA 与高度压缩注意力（Heavily Compressed Attention）混合使用。V4.1-Flash 采用纯 CSA2，并沿层轴压缩缓存大小。每个 CSA2 层被静态分配为以下 3 种模式之一：
- Full：计算自身的主 KV，从中投影出 indexer K，并选出全新的 Top-512 索引。
- Reindex：复用上一个 Full 层的主 KV 和 indexer K，但用自身的 indexer Q 对它们重新打分。
- Reuse：同时复用主 KV 和最新的 Top-K 索引，完全跳过 indexer。
每一层都保留自己的主 Q 和 SWA KV。18 个 CSA2 编码器层采用压缩比 2，分为 3 组、每组 6 层（1 个 Full，5 个 Reuse）。20 个解码器层采用压缩比 1，分为 5 组、每组 4 层：第一组为 Full 加 3 个 Reuse，其余为 Reindex 加 3 个 Reuse。解码器中的分层稀疏索引器（Hierarchical Sparse Indexer）让 Full 层构建一个最多 16,384 个位置（2,048 个块，每块 8 个）的候选池，从而使后续的 Reindex 层只需对有限集合打分，而非整个上下文。
FP4 KV、有界重放及其他扩展
主 KV 缓存被量化为 E2M1，每 16 个通道配一个 E4M3 缩放因子，遵循 NVFP4 但不含其全局缩放因子。这通过后训练阶段的量化感知训练引入，相比 V4 的 FP8 缓存几乎将存储减半。
在部署层面，SWA KV 不再持久化到 SSD。它驻留在一个分布式池中，该池从主机 DRAM 的 10% 中划分而来，TTL 为分钟级，而全局 KV 则保证 72 小时的生命周期。未命中时，Encoder SWA Bounded Replay 只重新计算 128 个 token，而非层数乘以窗口大小。
其他改动包括 Single-Pass mHC，它将输入混合系数偏移一个 block，使融合的 Mega-mHC kernel 能将激活内存流量减半；位于第 1 层和第 14 层的 Engram 条件记忆模块；在预训练后以冻结主干训练的 DSpark 投机解码；以及 head-wise Muon。当上下文从 4K 增长到 1M 时，单 token 解码 FLOPs 仅增加 1/4。
训练与结果
预训练覆盖 45T 多模态 token，文本与多模态比例为 7:1。稀疏注意力从零开始以 64K 序列长度训练，无稠密预热，并在 34T token 时将上下文扩展至 1M。基础模型在世界知识和编程方面与 DeepSeek-V4-Pro-Base 相当，而总参数量仅为其 1/3，激活参数量仅为其 1/4。
后训练未引入新算法。收益来自大规模合成可验证的智能体任务、跨异构脚手架（Claude Code、Codex、OpenCode、Pi、mini-SWE、DeepSeek Harness）的 RL，以及来自 40 多个教师模型的 on-policy 蒸馏。精选的最大努力结果如下：
| 基准测试 | DS-V4.1-Flash | DS-V4-Flash | Opus-5 | GPT-5.6 Sol |
|---|---|---|---|---|
| Terminal-Bench 2.1 | 90.6 | 82.7 | 89.1 | 88.8 |
| DeepSWE v1.1 | 74.2 | 54.4 | 74.0 | 73.0 |
| Terminal-Bench 4.0 | 31.2 | 7.0 | 51.8 | 39.9 |
| Automation-Bench | 54.8 | 37.7 | 50.3 | 45.8 |
| GPQA Diamond | 90.9 | 89.9 | 93.4 | 94.1 |
| Codeforces（评分） | 3471 | 3289 | n/a | n/a |
交互式讲解
核心要点
- 全局 KV cache 降至每 token 890 字节，约为 V4-Flash 的 1/4，比 V1 低 437 倍。
- CED 在 prefill 阶段仅运行 20 个编码器层，激活 8B 参数，而 decode 阶段为 16B。
- CSA2 在 Full、Reindex 和 Reuse 模式下跨层共享主 KV、indexer K 和 Top-K 索引。
- FP4 主 KV 加上 SWA 有界重放，将持久缓存削减至 V4-Flash 的约 1/8。
- 在 Terminal-Bench 2.1 和 DeepSWE v1.1 上以 MIT 权重击败 Opus-5 和 GPT-5.6 Sol。
Long-horizon agents have turned LLM serving into an input-heavy workload. Repeated prefills and million-token contexts leave KV caches that strain HBM, SSD capacity, and bandwidth. DeepSeek AI built its newest release around that exact bottleneck. DeepSeek-V4.1-Flash is a multimodal Mixture-of-Experts model with 552B backbone parameters, 196B additional Engram parameters, and a 1M-token context window. It activates 8B parameters per token during prefill and 16B during decode. The main number is a global KV cache footprint of 890 bytes per token, about 1/4 of DeepSeek-V4-Flash and roughly 437x smaller than DeepSeek-V1.
Is it deployable? Yes. Open weights ship under an MIT license with vLLM, SGLang, and Transformers paths on Hugging Face, and the research team describes a public API with low, high, and max reasoning tiers.
Causal Encoder-Decoder: Half the Prefill
The 40-layer backbone is split into a 20-layer causal encoder and a 20-layer decoder. Inspired by YOCO, the decoder does not compute its own global KV. Instead, per-layer projection weights derive it from the final encoder hidden state. Prompt tokens therefore stop at the encoder, which nearly halves prefill compute. Sliding-window attention (SWA) with a 128-token window still runs in every layer, so decoder SWA states are rebuilt by replaying only the last 128 prompt tokens. The research team calls this Decoder SWA Bounded Replay.
Compressed Sparse Attention 2 (CSA2)
DeepSeek-V4 mixed CSA with Heavily Compressed Attention. V4.1-Flash uses pure CSA2 and attacks cache size along the layer axis. Each CSA2 layer is statically assigned one of 3 modes:
- Full: computes its own main KV, projects indexer K from it, and selects fresh Top-512 indices.
- Reindex: reuses main KV and indexer K from the last Full layer but rescores them with its own indexer Q.
- Reuse: reuses both the main KV and the latest Top-K indices, skipping the indexer entirely.
Every layer keeps its own main Q and SWA KV. The 18 CSA2 encoder layers use a compression ratio of 2 in 3 groups of 6 (1 Full, 5 Reuse). The 20 decoder layers use ratio 1 in 5 groups of 4: the first is Full plus 3 Reuse, the rest Reindex plus 3 Reuse. A Hierarchical Sparse Indexer in the decoder lets the Full layer build a candidate pool of up to 16,384 positions (2,048 blocks of 8), so later Reindex layers score a bounded set instead of the entire context.
FP4 KV, Bounded Replay, and Other Extensions
The main KV cache is quantized to E2M1 with one E4M3 scale per 16 channels, following NVFP4 without its global scale. This is introduced through quantization-aware training in post-training and nearly halves storage against V4’s FP8 cache.
At the deployment level, SWA KV is no longer persisted to SSD. It lives in a distributed pool carved from 10% of host DRAM with a TTL of minutes, while global KV keeps a guaranteed 72-hour lifetime. On a miss, Encoder SWA Bounded Replay recomputes only 128 tokens instead of layers times window.
Other changes include Single-Pass mHC, which shifts input-mixing coefficients by one block so a fused Mega-mHC kernel can halve activation memory traffic, the Engram conditional memory module at layers 1 and 14, DSpark speculative decoding trained after pre-training with the backbone frozen, and head-wise Muon. Single-token decode FLOPs rise by only 1/4 when context grows from 4K to 1M.
Training and Results
Pre-training covers 45T multimodal tokens at a 7:1 text-to-multimodal ratio. Sparse attention is trained from scratch at 64K sequence length with no dense warmup, and context is extended to 1M at 34T tokens. The base model matches DeepSeek-V4-Pro-Base on world knowledge and coding while using 1/3 of the total and 1/4 of the activated parameters.
Post-training introduces no new algorithms. Gains come from large-scale synthesis of verifiable agent tasks, RL across heterogeneous scaffolds (Claude Code, Codex, OpenCode, Pi, mini-SWE, DeepSeek Harness), and on-policy distillation from over 40 teachers. Selected max-effort results:
| Benchmark | DS-V4.1-Flash | DS-V4-Flash | Opus-5 | GPT-5.6 Sol |
|---|---|---|---|---|
| Terminal-Bench 2.1 | 90.6 | 82.7 | 89.1 | 88.8 |
| DeepSWE v1.1 | 74.2 | 54.4 | 74.0 | 73.0 |
| Terminal-Bench 4.0 | 31.2 | 7.0 | 51.8 | 39.9 |
| Automation-Bench | 54.8 | 37.7 | 50.3 | 45.8 |
| GPQA Diamond | 90.9 | 89.9 | 93.4 | 94.1 |
| Codeforces (rating) | 3471 | 3289 | n/a | n/a |
Interactive Explainer
Key Takeaways
- Global KV cache falls to 890 bytes per token, about 1/4 of V4-Flash and 437x below V1.
- CED runs only 20 encoder layers in prefill, activating 8B parameters against 16B in decode.
- CSA2 shares main KV, indexer K, and Top-K indices across layers in Full, Reindex, and Reuse modes.
- FP4 main KV plus SWA Bounded Replay cut persistent cache to about 1/8 of V4-Flash.
- Beats Opus-5 and GPT-5.6 Sol on Terminal-Bench 2.1 and DeepSWE v1.1 with MIT weights.