# KVMem 虚拟化百万 token 智能体工作区

- 来源：elvis (@omarsar0)
- 发布时间：2026-09-09 13:33
- AIHOT 分数：47
- AIHOT 链接：https://aihot.news/items/cmtto6dy80g2hrofpqmefsq3y
- 原文链接：https://x.com/omarsar0/status/2097558879194558838

## AI 摘要

KVMem 将智能体超长工作区溢出内容以分页 KV 状态保存，分布于 GPU 内存、主机内存与 NVMe，并用模型原生轻量注意力索引选取相关历史块，在原生上下文窗口内物化查询相关视图。

## 正文

Nice paper to improve inference efficiency.

It's been a while we haven't seen good work on efficiency.

Here is why it matters:

A long-running agent's workspace outgrows its context window long before the task finishes.

The first approach commonly used, compaction, loses the fine-grained execution evidence. And text retrieval re-prefills content the model already processed.

KVMem keeps the overflow as paged KV state instead, spread across GPU memory, host memory and NVMe.

Lightweight attention-space indexes, native to the model, pick the relevant historical blocks and materialize a query-dependent view that fits inside the native context window.

On the DeepSWE long-context test with Qwen3.8-27B, task success goes from 43.8% under compaction to 48.4%.

The local deployment result stands out. It runs Qwen3.6/3.8-27B NVFP4 with MTP on a laptop with a 24GB RTX 5090, virtualizing an agent workspace up to 1M tokens, four times the model's native 256K window, at around 50 tokens per second.

Paper: https://academy.dair.ai/papers/kvmem-virtualizing-million-token-agent-workspaces-on-a-consumer-gpu-2609.04852