微软研究院新研究发现,重编程 AI 智能体中的少量休眠神经元即可植入后门,让智能体平时表现正常,直到触发特定条件才激活异常行为。微软建议企业确认模型来源并合理管控权限,Microsoft Foundry 为 Azure Direct Models 和 Hugging Face 模型提供安全系统以保障模型来源可信。
New research from Microsoft Research demonstrates that reprogramming just a few dormant neurons in an AI agent can create a backdoor that leaves the agent behaving normally until it hits a specific trigger.
While the specifics of this threat are novel, the mitigation is not: Know where your model came from, and be smart about its permissions. Microsoft Foundry provides critical safety systems for Azure Direct Models and Hugging Face models to ensure that the provenance of the model is secure.
Microsoft researcher @DimitrisPapail explains the measures that model hubs and vendors should take as well as mitigation strategies for enterprises and users. msft.it/6014ahp6e
来源:Microsoft Research · x.com