# MiMo-V3 新架构 HySparse2 发布

- 来源：Fuli Luo (@_LuoFuli)
- 发布时间：2026-09-23 22:25
- AIHOT 分数：43
- AIHOT 链接：https://aihot.news/items/cmue7co7k0pvhroghnfrhcg7o
- 原文链接：https://x.com/_LuoFuli/status/2102766365190901957

## AI 摘要

罗福莉发布 MiMo-V3 新架构核心 HySparse2，相比 MiMo-V2.6 的 Hybrid SWA 架构，在 1M tokens 下 prefill FLOPs 降低 5.02×、KV cache 缩小 4.5×，MRCRv2 与 RULER-v2 分数更高，AgentPPL 和 LongPPL 更低。

## 正文

MiMo-V3 is getting a new architecture. The core of it, HySparse2, is out today.

Less prefill, a smaller KV cache, better long-context retrieval—and we got all three at once.

Compared with MiMo-V2.6's Hybrid SWA architecture:
• 5.02× lower prefill FLOPs at 1M tokens
• 4.5× smaller KV cache at 1M tokens
• Better MRCRv2 and RULER-v2 scores, plus lower AgentPPL and LongPPL

Why build a new architecture?
Agentic inference is a very different workload. Each round, a short action can return a long observation that needs to be prefilled, while the context keeps growing. That puts prefill cost, KV-cache size, and retrieval accuracy on the critical path at the same time.

HySparse2 tackles all three with two levels of KV sharing:
• KV Bridging: Following YOCO, full-attention layers in the cross-decoder build their K/V from self-decoder hidden states.
• KV Reuse: Within each hybrid block, sparse layers reuse the preceding full-attention layer's KV cache and selection indices.

Two more changes: token-level selection replaces block-level selection, and a forced window of recent tokens replaces the separate SWA branch, so local and global tokens share one KV cache. Since all cross-decoder KV caches now come from the self-decoder, prefill can stop once the self-decoder finishes.

Paper: https://arxiv.org/pdf/2609.26368