跳到正文
Hao AI Lab· @haoailab · X·· 2 小时前AI 评分46
AI 导读

UCSD Hao AI Lab 用 vstack 在 PD 分离推理服务中发现 3 类 RDMA 与 KV cache 生命周期 bug,已提出 6 项上游修复。

正文 · 原文

In PD-disaggregated serving, a GPUDirect RDMA race often returns HTTP 200 OK while silently mixing one user's KV cache into another's response. Applying vstack uncovered 3 classes of RDMA and cache lifecycle bugs in production engines, resulting in 6 proposed upstream fixes.

With TLAPS and Z3, we also proved why local timeouts fail under one-sided RDMA: without hardware stop or revocation evidence, any local-only timeout policy must either risk memory corruption if it frees a page early, or hold the exposed page indefinitely.

来源:Hao AI Lab · x.com