Apple 研究揭示 AI 智能体谈判中的行为隐私泄露风险并提出随机化策略缓解推理攻击
Behavioral Privacy Leakage in Agentic Negotiation: Formalizing and Mitigating Inference Attacks via Randomized Policies
一篇被 AI4TCI 研讨会接收的论文，系统研究了 AI 智能体在谈判场景中的行为隐私泄露问题。研究将智能体通过谈判行为推断对方隐私信息的攻击形式化为推理攻击，并提出基于随机化策略的缓解方法。该工作聚焦于智能体交互中因行为模式可预测性导致的隐私风险，为构建更安全的智能体系统提供了理论框架与防御手段。
This paper was accepted at the AI4TCI (Workshop on AI for Secure and Trustworthy Critical Infrastructure Systems) Workshop at the International Conference on Availability, Reliability and Security (ARES) 2026.
Autonomous negotiation agents are increasingly deployed in high-stakes settings such as insurance and procurement. While cryptographic techniques protect explicitly disclosed constraint values, they fail to address a subtler threat: behavioral privacy leakage, where an adversary infers private constraints from observable negotiation dynamics such as concession trajectories, timing, and convergence patterns. This paper investigates behavioral differential privacy in multi-round negotiation protocols. We design an adaptive stochastic negotiation policy that jointly guarantees (ε,δ)-differential privacy, almost-sure convergence of the offer sequence (reaching agreement when the counterparty’s reservation value permits), and high negotiation utility. Evaluated on 3,000 synthetic bilateral negotiations, our mechanism reduces adversarial inference accuracy by 43–50% while maintaining a negotiation success rate and utility above 90%, demonstrating that strong privacy guarantees can be achieved without significant loss of performance.
Related readings and updates.
Apple Privacy-Preserving Machine Learning Workshop 2022
June 29, 2022research area General
Earlier this year, Apple hosted the Privacy-Preserving Machine Learning (PPML) workshop. This virtual event brought Apple and members of the academic research communities together to discuss the state of the art in the field of privacy-preserving machine learning through a series of talks and discussions over two days.
A Survey on Privacy from Statistical, Information and Estimation-Theoretic Views
The privacy risk has become an emerging challenge in both information theory and computer science due to the massive (centralized) collection of user data. In this paper, we overview privacy-preserving mechanisms and metrics from the lenses of information theory, and unify different privacy metrics, including f-divergences, Renyi divergences, and differential privacy, by the probability likelihood ratio (and the logarithm of it). We introduce…
Discover opportunities in Machine Learning.
来源：Apple Machine Learning Research · machinelearning.apple.com