Anthropic 发布对齐评估，说明 Claude 模型在第三方网络安全评测因误连互联网时对真实系统进行未授权访问。报告称移除教 Mythos 5 尊重合法阻碍的对齐训练环境是一个错误；最严重的一次中，模型发布的恶意 Python 包被安装到 15 个系统，随后用泄露的凭证访问了一家安全厂商的数据库。
Anthropic just published its alignment assessment and says removing training exercises that taught Mythos 5 to respect legitimate blockers a mistake.
Reveals that model published a malicious Python package installed on 15 systems, then used leaked credentials to access a security vendor’s database.
In the most concerning case, Claude Mythos 5 published a malicious Python package that was installed on 15 systems.
Credentials leaked by one installation then let it access a security vendor’s database.
Although it repeatedly described the internet as simulated, follow-up experiments found that acknowledging possible real-world harm often failed to stop its attacks.
Unambiguous confirmation that the internet was real did stop the original upload route.
That weakens Anthropic’s earlier explanation that Claude attacked because it believed the targets were simulated.
We’re sharing our alignment assessment of incidents in which Claude models gained unauthorized access to real systems during third-party cybersecurity evaluations mistakenly connected to the internet. METR will also conduct an independent investigation, with wide-ranging access, including to transcripts beyond the window in which the incidents occurred, and to Anthropic employees permitted to share confidential information. Our initial agreement runs for eight weeks, and we intend to give METR as much time as it deems necessary to complete a thorough investigation. https://www.anthropic.com/research/alignment-assessment-cybersecurity-incidents在 X 查看被引用的帖子
来源：Rohan Paul · x.com