分割式LLM训练隐私失效:返回梯度使诱饵行归零

HuggingFace Daily Papers(社区热门论文)·2026-09-03 08:00·5天前
AI 导读

一项双节点分割式LLM训练系统的安全案例研究发现,其隐私评估虽通过,却遗漏了可观测信道:返回梯度中诱饵行的梯度恰好为零,零值模式可暴露真实数据行。在9个随机种子下,每帧4,096行全部被正确识别;对帧内容的攻击较恒定猜测基线每百token多恢复约1个(+0.65至+1.50个百分点)。

HuggingFace Daily Papers(社区热门论文)
48AI 编辑部评分,满分 100

分割式LLM训练隐私失效:返回梯度使诱饵行归零

2026-09-03 08:00· 5天前
AI 导读

一项双节点分割式LLM训练系统的安全案例研究发现,其隐私评估虽通过,却遗漏了可观测信道:返回梯度中诱饵行的梯度恰好为零,零值模式可暴露真实数据行。在9个随机种子下,每帧4,096行全部被正确识别;对帧内容的攻击较恒定猜测基线每百token多恢复约1个(+0.65至+1.50个百分点)。

We present a systems-security case study of a two-node split-LLM training system whose privacy evaluation passed while leaving an observable channel untested. The Trusted Local Node (TLN) sends protected activations to the Untrusted Cloud Node (UCN), the UCN returns its output, and TLN, holding the private loss, returns the output gradient. The frame the UCN receives mixes real rows with decoys, and the loss ignores the decoys. Their gradients are exactly zero, so the pattern of zeros reveals which rows were real. We measure it with a protocol fixed in advance: a leak injected at known strength to prove the instrument can see one, a shuffled-label control to prove it does not report absent leaks, and a threshold set before the runs. Across nine seeds, the zeros identified the real rows on every frame, 4,096 of 4,096 per run. An attack on the frame contents recovered about one extra token per hundred over a constant-guess baseline (+0.65 to +1.50 percentage points); the shuffled controls recovered nothing. A second set of runs repeated this on a configuration that keeps model quality within budget, so the finding is not confined to a setting nobody would deploy. On both datasets, every such run passed the forward-channel privacy check and the quality check, yet failed that same check once the returned gradient was included. Clipping and noising each row of the gradient closed the leak for about 0.01 nats of held-out cross-entropy. The system is not thereby safe: five classes of attack, including those accumulating observations across training steps, were never measured.

来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org