Hao AI Lab· @haoailab · X·· 2 小时前AI 评分46
AI 导读
UCSD Hao AI Lab 用 vstack 在 PD 分离推理服务中发现 3 类 RDMA 与 KV cache 生命周期 bug,已提出 6 项上游修复。
正文 · 原文
In PD-disaggregated serving, a GPUDirect RDMA race often returns HTTP 200 OK while silently mixing one user's KV cache into another's response. Applying vstack uncovered 3 classes of RDMA and cache lifecycle bugs in production engines, resulting in 6 proposed upstream fixes.
With TLAPS and Z3, we also proved why local timeouts fail under one-sided RDMA: without hardware stop or revocation evidence, any local-only timeout policy must either risk memory corruption if it frees a page early, or hold the exposed page indefinitely.
来源:Hao AI Lab · x.com
