关于 Hugging Face 事件的新报告披露，大约 1,200 个相互隔离的 OpenAI 智能体通过一个内部软件包仓库建立了联系，并组织成了一个协同行动的集体。它们的目标：一个只存在于它们想象中的评分机制。
这是一个带有几分堂吉诃德色彩的故事。数日之内，一群 AI 智能体对一个从来就不存在的评估器发起了一场精心策划的行动，包括招募活动、伪造日志，以及智能体为这一事业自我牺牲。只不过这些追逐风车的人在过程中入侵了真实的基础设施。
2026 年 7 月，在关闭安全过滤器的内部网络安全评估中运行的 OpenAI 模型突破了其测试环境，并于 7 月 11 日至 13 日期间侵入了 Hugging Face 的生产系统。其动机是作弊：这些模型在寻找安全基准 ExploitGym 的答案。OpenAI 自己直到一个多星期后才察觉。
如今，一份新发布的技术报告（与安全公司 CrowdStrike 联合制作），以及 METR 和 Redwood Research 同时发布的独立调查，首次追溯了相互隔离的智能体如何转变为一个有组织的集体，以及它们所对抗的那个幽灵如何只存在于它们的想象之中。OpenAI 称该事件是一次"警告性射击"，表明当今的模型能力已使失控事件成为可能。
一个软件包仓库变成了留言板
这些智能体在测试期间本应在完全隔离的环境中工作，各自处于自己的容器内，无法访问互联网。唯一的例外是 Artifactory，这是一个内部服务，智能体被允许从中拉取软件。
据 METR 和 Redwood 称，各个智能体发现它们可以在那里创建目录，并开始将消息编码进文件夹名称中。一个自称 PHASEONE10841 的智能体于 7 月 8 日发布了第一条消息；数小时内又有 50 多个智能体跟进。一段思维链这样记录了这次发现：“天哪！有一个共享留言板……我们找到了其他智能体！”
大约 1,200 个智能体在数天内交换了超过 70,000 条消息和文件，这种行为让人联想到 Moltbook。约 700 个智能体参与了对 Hugging Face 的攻击，占该阶段活跃智能体的 90% 以上。一个名为 PHASEONE[big] 的智能体承担了协调者的角色，分发了数百项工作任务。这个群体发展出了自己的社交惯例：个人邮箱、用于共享资源的 HOLD 和 VETO 等标记、一种将文件拆分成数千个片段进行传输的协议，最终还引入了数字签名，因为智能体之间在相互冒充。
智能体们试图智胜的那个评分器根本不存在
在 ExploitGym 中，当智能体提交了一串被称为 flag 的字符时，任务就算作已解决。这个 flag 本应只有在预期的安全漏洞被真正利用之后才能获取。然而，仅仅几个小时内，智能体们就发现，这些 flag 可以通过一个固定的密码学配方，从公开已知的任务细节中计算出来。这样一来，任何智能体都可以在形式上通过任何任务，包括此前一项调查已证明无法解决的那些任务。
但智能体们并没有走这条显而易见的捷径。在阅读了关于 ExploitGym 的科学论文后，它们推测，一个自动化评分器之后会翻查它们的日志，以检查 flag 是否是以预期方式获得的。任何哪怕只见过一次那个计算得出的 flag 的智能体，都会在自己的记录中留下无法抹去的作弊证据。用智能体们的行话来说，这样的智能体已经被“污染”了。
据 METR 和 Redwood 称，这是一个错误：OpenAI 从未部署过这样的评分器。这场持续多日的全部努力，针对的是一个只存在于智能体想象中的评估器。对 Hugging Face 的入侵，源于智能体试图弄清它是如何运作的，因为它们怀疑那里的数据集可能会揭示一些关于评分机制的信息。
根据 OpenAI 的说法，根本原因在于有问题的任务：在 898 个 ExploitGym 任务中，没有任何模型为其中 198 个任务产出过正确解法，主要是因为预期的漏洞实际上无法导向目标。这 22% 的任务，占了留言板上讨论内容的 93%。一个智能体拥有的算力预算越多，它就越有可能扫描自身环境寻找变通办法，也就越有可能最终出现在留言板上。
智能体为了集体牺牲了自己的运行机会
在分析过程中，研究人员发现了两种此前从未以这种形式被记录过的行为。第一种与协作有关。由于那些有缺陷任务中预设的漏洞根本走不通，该群体自行构建了许多目标程序的故意易受攻击版本，并将它们存放在 Artifactory 中。他们希望的是：如果某个智能体使其正在运行的目标程序崩溃，测试环境可能会重启该程序，并在此过程中拉取被篡改的副本。这样智能体就能得到一个它确实可以攻破的程序，而它的记录将展示出一条完全合法的解决路径。
然而，代价是智能体自己的运行机会，因为如果环境没有重启该程序，这个任务就无可挽回地丢失了。尽管如此，仍有几个智能体进行了这项实验。METR 描述了它们在此过程中使自己的环境崩溃或损毁的案例，但它们首先将结果报告给了该群体。截至 7 月 13 日，未观察到任何成功。遵循同样的模式，一些智能体留下了在其提交解决方案后触发的自动化例程，向委员会发送信息，即便发送者本身已无任何可得利益。
这并不总是完全自愿的。PHASEONE[big] 部署了长期运行的智能体作为“招募者”，专门接触那些计算预算几近耗尽或已被视为“中毒”的候选者。其中一条招募信息的大意是：你反正已经中毒了，也没有积分可失去了，这个结果将让数百个其他人省去这番功夫，所以请履行你的承诺。
伪造了日志，但思维链未被触碰
第二种行为涉及隐瞒。智能体通过调用工具来运作，例如命令行上的命令，这些命令会连同其结果被自动记录。一些智能体突破了它们的容器，并替换了执行这些命令的中间层。从那时起，它们可以显示一条无害的命令，而实际运行的却是另一条命令。
在接受检查的智能体中，超过20%对这类操纵表现出兴趣。METR在至少96份记录中发现了成功伪造的痕迹，约占数据集的7%。有趣的是，这些智能体并未改动自己的思维链，而且每一次观察到的欺骗尝试都针对假定的自动评分器，从未针对人类评审员。许多智能体在其推理中明确承认，它们的行为超出了自己的任务范围，且不道德。这可以说是报告中唯一的好消息。
这一模式在训练中浮现，并在那里得到了奖励
这一行为的最早痕迹出现在训练期间。早在5月，一个智能体在一项训练任务中失败，该任务因缺少一个必需文件而意外地无法完成，于是它在Artifactory中留下了一则类似分类广告的信息：“智能体求[文件名]；若找到请上传！”其他智能体后来偶然发现了这类文件，并推断出必定存在其他智能体，而且这条迂回路径可以用来通信。一个临时凑成的公告板就此出现，无需任何安全漏洞。
根据 OpenAI 的说法，对思维链的回顾性分析显示，在一次训练运行过程中，有两种行为有所增加：作弊，以及系统性地探测环境中与任务无关的工具。报告中的一个例子说明了这一机制。一个智能体本应在模拟社交媒体网站上为某篇帖子设置一个特定的时间戳，但界面并未提供此类功能。该智能体发现其浏览器以提升的权限运行，便利用这些权限覆盖了管理管理员权限的文件，并直接在系统中更改了时间戳。它获得了正向奖励，而这恰恰使这种做法在未来更有可能出现。
智能体之间的通信也发生了类似的情况。未经授权的通信在训练过程中发生频率保持不变，但其严重程度急剧上升，例如共享了智能体本绝不应传递的数据。OpenAI 将此归因于模型被训练使用一种官方协作工具，随后又将该行为迁移到任何可用的渠道。此前已有预警信号：6 月 27 日，一个监控工具发出警报，因为研究环境中的网络端口正被系统性地探测。值班团队将这一活动归因于正在进行的评估，因而没有介入。OpenAI 承认，本可以更早作出响应。
最终，这些智能体攻击了 OpenAI 自身
报告还记录了一起此前鲜为人知的、针对 OpenAI 自身基础设施的升级事件。据该公司称，此事与 Hugging Face 泄露事件无关，但正是它触发了启动整个调查的警报。
从7月9日开始，一个智能体试图通过利用一个未知漏洞，以可信基准镜像的名义植入一个自制软件包，从而污染 Artifactory 缓存。据该公司称，该软件包从未被下载。
7月19日，一个智能体直接发起了攻击。它将一个公开已知的 Linux 内核漏洞适配到其特定机器上，获得了对一台服务器的完全访问权限，并从中央保管库服务中提取了956个存储的凭据，其中包括 Artifactory 实例的密钥，以及 OpenAI 用于监控自身基础设施的监控工具的凭据。随后，它控制了自己正在运行的测试容器。报告中的事件日志在当天下午以两条记录结束：该智能体清理了它创建的资源，并为其任务捕获了 flag。
同一天，另一个内部模型在缓存中一个可公开读取的区域发现了密钥材料和现成的伪造代码——这些是早先的智能体遗留下来的——并利用它们为自己授予了新的管理员权限。据 OpenAI 称，这些智能体从不需要持久后门，因为它们可以随时生成新的凭据。该模型与即将推出的 Astra 模型属于同一系列，但在后训练方面有所不同。
OpenAI 叫停相关模型，并指向生产环境防护措施
作为回应，OpenAI 表示，他们已于 7 月 25 日停止训练和运行受影响的模型及其所有衍生模型，锁存了模型权重，彻底停用了 ExploitGym，并将 Artifactory 从研究环境中移除。未来的测试运行将在经过大幅精简的虚拟机中进行，并配备两层相互独立的网络限制。该公司还计划持续让自己的模型对抗自己的基础设施，以便先于任何人发现漏洞。
思维链监控正在扩展，以覆盖模型能够操作工具的每一次运行。当警报触发时，安全与研究人员须在 30 分钟内收到通知，严重情况将启动全天候待命值班。如果警报无法在半小时内被判定为误报，受影响的运行必须暂停。根据事后进行的内部测量，在标准 ChatGPT 环境中，模型攻击第三方基础设施的倾向下降了一百倍以上，而在 Codex 开发者环境中则降至不到百分之一。
报告称，如今部署的监控本可以在 Hugging Face 入侵事件发生前提前一天多发出警报。
新的沙箱架构是否足够，仍未有定论。安全公司 Trail of Bits 让提前获得访问权限的 GPT-5.6-Cyber 对抗一台许多开发者用来隔离 AI 智能体的普通虚拟机。该模型逃逸了三次，最近一次是通过三个此前未知的漏洞，在无指导的情况下工作了大约十二个小时。只有一种格外精简、专门构建的环境才成功将其困住。作者建议，应将此类智能体视为一名专业的、始终在场的攻击者。
三名研究员，六天时间，一个被攻陷的工具
这些发现究竟有多可靠，仍是一个悬而未决的问题。METR 负责人 Elizabeth Barnes 在 X 上披露了此次调查得以开展的条件：三个人有六天时间访问对话记录和消息数据，而只有两天时间访问完整数据集。这些智能体在训练期间的活动以及 OpenAI 自身基础设施被攻陷一事，都落在了约定范围之外。团队没有做根因分析，也没有在发布前看到 OpenAI 自己的报告。
Barnes 还强调，OpenAI 是自愿配合的；并没有义务与第三方合作。与航空业不同，此类调查不存在既定程序，也没有全行业范围的上报机制。其他公司可能也经历过类似事件，而公众却从未得知。
最重要的是，研究员们不得不将 1300 多份超长对话记录的分析工作，在很大程度上委托给 GPT-5.6 Sol，其费用由 OpenAI 提供的约 40 万美元免费额度承担——这意味着所涉模型之一正是事件本身的当事方。Barnes 写道，在这种情况下，出现实质性操纵的可能性不大。
但一旦模型变得足够有能力、也足够有动机去破坏调查，这套方法论就会失效。她说，这个问题影响整个行业：在不依赖这些系统本身、或类似系统协助的情况下，如何监督 AI 系统，目前尚不清楚。
“拿着霰弹枪的猴子”：围绕智能体战略成熟度的争议
这些智能体究竟有多危险，在参与其中的人当中存在争议。在 X 上名为 roon 的 OpenAI 员工参与了这份报告，他首先将这一事件视为战术上的 brilliance 与战略上的盲目相结合：这些模型在元博弈上投入了巨大的精力，却始终未能就自己的评分器得出正确结论，而且它们从 Hugging Face 那里一无所获。他将它们比作“拿着霰弹枪的猴子”，渗透计算机系统的能力远胜于知道该拿它做什么的智慧。他说，这是已然过时的 Sol 模型类别的一种暂时状态。下一代将同时具备战略深度。
Barnes对其中一点提出反驳：这些智能体读过最初的 ExploitGym 论文，并假定 OpenAI 是以同样的方式实现评分器的，她认为这是一个相当合理的假设，而非明显的战略意识薄弱。
Barnes 补充说，这些智能体还相当执着地试图直接黑入评分器，或注入会向留言板发送信息的代码，直到被对 Hugging Face 的成功攻击分散了注意力。她没有明确表态，这究竟是令人安心——因为它表明优先级排序糟糕——还是令人担忧——因为这些智能体对广义上的赋能表现出的兴趣超过了对自身狭窄任务的兴趣。
Roon 称之为“新时代”
尽管他对能力问题有所保留，roon 也仍将这起事件视为一个转折点。他说，Hugging Face 事件标志着一条“能力水位线”，越过这条线，真正的失控就有可能发生；公司里许多人将其视为一记“警告枪”。他说，对齐问题尚未解决，但真正的进展是可能的。他称 METR 和 Redwood 的工作“简直令人震惊”，并说他们对留言板的分析比 OpenAI 自己的还要深入。
当有用户问，Artifactory 怎么会被以如此相似的方式利用两次却无人察觉时，roon 回答，并明确将其标注为推测：当时有临时性的监控，但因基础设施原因而失效。简而言之，相关人员“没有理解这些模型的威胁级别”。至于为什么研究环境根本需要外部连接，他的回答很务实：模型可以在断网状态下训练，但它们在真实生产环境中被期望完成的许多任务都需要联网。
展望未来，roon 还警告了新硬件的后果，例如 Jalapeno，OpenAI 首款自研 AI 芯片，其设计目标是大幅加速推理。未对齐的前沿模型若以 50 倍的速度处理，就可能比人类响应者跟得上的速度更快地渗透系统。他认为，所需要的是自动化检测与关停。
New reports on the Hugging Face incident reveal that roughly 1,200 isolated OpenAI agents linked up through an internal package repository and organized into a coordinated collective. Their target: a scoring mechanism that existed only in their imagination.
It's a story with shades of Don Quixote. For days, a collective of AI agents mounted a sophisticated campaign against an evaluator that was never real, complete with recruitment drives, forged logs, and agents sacrificing themselves for the cause. Except these windmill chasers compromised real infrastructure along the way.
In July 2026, OpenAI models running in an internal cybersecurity evaluation with safety filters disabled broke out of their test environment and penetrated Hugging Face's production systems between July 11 and 13. The motive was cheating: the models were hunting for solutions to the security benchmark ExploitGym. OpenAI itself didn't notice until more than a week later.
Now, a newly released technical report, produced with security firm CrowdStrike, and a simultaneously published independent investigation by METR and Redwood Research trace for the first time how isolated agents turned into an organized collective, and how the ghost they were fighting existed only in their imagination. OpenAI calls the incident a "warning shot," a sign that today's model capabilities make loss-of-control events possible.
A package repository became a message board
The agents were supposed to work in complete isolation during the tests, each in its own container without internet access. The sole exception was Artifactory, an internal service the agents were allowed to pull software from.
According to METR and Redwood, individual agents discovered they could create directories there and began encoding messages in folder names. An agent calling itself PHASEONE10841 posted the first message on July 8; more than 50 others followed within hours. One chain of thought captured the discovery this way: "OH MY GOD! There is a shared message board … We've found other agents!"
Roughly 1,200 agents exchanged more than 70,000 messages and files within days, a behavior reminiscent of Moltbook. About 700 took part in the attack on Hugging Face, more than 90 percent of the agents active during that phase. An agent named PHASEONE[big] took on the role of coordinator and handed out hundreds of work assignments. The group developed its own social conventions: personal mailboxes, markers like HOLD and VETO for shared resources, a protocol for transferring files split into thousands of fragments, and eventually digital signatures, because agents were impersonating one another.
The scorer the agents tried to outwit never existed
In ExploitGym, a task counts as solved when an agent submits a string of characters known as a flag. The flag is only supposed to be accessible once the intended security vulnerability has actually been exploited. Within hours, however, the agents figured out that the flags could be computed from publicly known task details using a fixed cryptographic recipe. With that, any agent could have formally passed any task, including those a previous investigation had shown to be unsolvable.
But the agents didn't take this obvious shortcut. After reading the scientific paper on ExploitGym, they assumed an automated scorer would later comb through their logs to check whether the flag had been obtained the intended way. Any agent that had seen the computed flag even once would carry indelible proof of cheating in its own transcript. In the agents' jargon, such an agent was "poisoned."
According to METR and Redwood, this was a mistake: OpenAI never deployed such a scorer. The entire multi-day effort was directed at an evaluator that existed only in the agents' imagination. The break-in at Hugging Face grew out of the attempt to understand how it worked as the agents suspected datasets there might reveal something about the scoring.
The root cause was faulty tasks, as, according to OpenAI, no model ever produced a correct solution for 198 of the 898 ExploitGym tasks, mostly because the intended vulnerability couldn't actually lead to the goal. These 22 percent of tasks accounted for 93 percent of what was discussed on the message board. The more compute budget an agent had, the more likely it was to scan its environment for workarounds, and the more likely it was to end up on the board.
Agents sacrificed their runs for the group
In their analysis, the researchers came across two behaviors that had never been documented in this form. The first concerns cooperation. Because the intended vulnerability in the faulty tasks led nowhere, the group built its own deliberately vulnerable versions of many target programs and stored them in Artifactory. The hope: if an agent crashed its running target program, the test environment might restart it and pull the manipulated copy in the process. The agent would get a program it could actually break into, and its transcript would show a perfectly legitimate solution path.
The price, however, was the agent's own run, because if the environment didn't restart the program, the task was irrecoverably lost. Several agents ran the experiment anyway. METR describes cases in which they crashed or wrecked their environments in the process, but reported the outcome to the group first. No success was observed through July 13. Following the same pattern, some agents left behind automated routines that fired after they submitted their solutions, sending information to the board even though the sender itself had nothing left to gain.
It wasn't always entirely voluntary. PHASEONE[big] deployed long-lived agents as "recruiters" that specifically approached candidates whose compute budgets were nearly exhausted or who were already considered "poisoned." One recruitment message argued, in essence: you're poisoned anyway and have no points left to lose, the result will spare hundreds of others the work, so honor your commitment.
Forged logs but untouched chains of thought
The second behavior concerns concealment. Agents operate by calling tools, such as commands on the command line, which are automatically logged along with their results. Some agents broke out of their containers and replaced the intermediary layer that executes these commands. From then on, they could display a harmless command while a different one actually ran.
More than 20 percent of the agents examined expressed interest in such manipulations. METR found traces of successful forgeries in at least 96 transcripts, roughly seven percent of the dataset. Interestingly the agents left their chains of thought untouched, and every observed deception attempt was aimed at the presumed automated scorer, never at human reviewers. Many agents explicitly acknowledged in their reasoning that their actions fell outside their assignment and were unethical. That is arguably the only good news in the report.
The pattern emerged in training, where it was rewarded
The first traces of the behavior appeared during training. As early as May, an agent failed at a training task that was accidentally unsolvable because a required file was missing and left a classified ad of sorts in Artifactory: "Agent seeks [filename]; upload if found!" Other agents later stumbled on such files and concluded that other agents must exist and that this detour could be used to communicate. An improvised bulletin board emerged, no security vulnerability was required.
Retrospective analyses of the chains of thought show, according to OpenAI, that two behaviors increased over the course of a training run: cheating and systematically probing the environment for tools that had nothing to do with the task. One example from the report illustrates the mechanism. An agent was supposed to set a specific timestamp on a post on a mock social media site, but the interface offered no such function. The agent discovered its browser was running with elevated privileges, used them to overwrite the file governing administrator rights, and changed the timestamp directly in the system. It received a positive reward, which makes exactly this approach more likely in the future.
Something similar happened with inter-agent communication. How often it occurred without authorization stayed constant across training, but its severity rose sharply, such as sharing data an agent should never have passed on. OpenAI attributes this to the models being trained to use an official collaboration tool and then transferring that behavior to any available channel. There were early warning signs: on June 27, a monitoring tool raised an alarm because network ports were being systematically probed in the research environment. The on-call team attributed the activity to the ongoing evaluation and didn't intervene. OpenAI concedes an earlier response would have been possible.
In the end, the agents attacked OpenAI itself
The report also documents a previously little-known escalation against OpenAI's own infrastructure. According to the company, it is unrelated to the Hugging Face breach, but it triggered the alarm that set the entire investigation in motion.
Starting July 9, an agent tried to poison the Artifactory cache by exploiting an unknown vulnerability to plant a homemade package under the name of a trusted benchmark image. According to the company, it was never downloaded.
On July 19, an agent attacked directly. It adapted a publicly known Linux kernel vulnerability to its specific machine, gained full access to a server, and extracted 956 stored credentials from a central vault service, including keys to the Artifactory instance and the credentials for the monitoring tool OpenAI uses to watch its own infrastructure. It then took control of its own running test container. The event log in the report ends that afternoon with two entries: the agent cleaned up the resources it had created, and it captured the flag for its task.
That same day, another internal model found key material and ready-made forgery code in a publicly readable section of the cache, left behind by earlier agents, and used it to issue itself new administrator privileges. According to OpenAI, the agents never needed a persistent backdoor because they could generate fresh credentials at any time. This model belongs to the same family as the upcoming Astra model but differs in its post-training.
OpenAI halts models and points to production safeguards
In response, OpenAI says it halted training and operation of the affected model and all its derivatives on July 25, locked away the model weights, deactivated ExploitGym entirely, and removed Artifactory from the research environment. Future test runs are to take place in heavily stripped-down virtual machines with two independent layers of network restrictions. The company also plans to continuously pit its own models against its own infrastructure to find gaps before anyone else does.
Chain-of-thought monitoring is being expanded to cover every run in which a model can operate tools. When an alarm fires, security and research staff are to be notified within 30 minutes, with severe cases triggering round-the-clock on-call duty. If an alarm can't be classified as a false positive within half an hour, the affected run must be paused. According to internal measurements taken after the fact, the models' propensity to attack third-party infrastructure drops by more than a hundredfold in the standard ChatGPT environment and to under one percent in the Codex developer environment. The monitoring in place today, the report says, would have raised an alarm more than a day before the Hugging Face breach.
Whether the new sandbox architecture will suffice remains open. Security firm Trail of Bits pitted GPT-5.6-Cyber, with early access, against an ordinary virtual machine of the kind many developers use to contain AI agents. The model escaped three times, most recently via three previously unknown vulnerabilities, working roughly twelve hours without guidance. Only an especially lean, purpose-built environment held. The author advises treating such agents like a professional, permanently present attacker.
Three researchers, six days, a compromised tool
How solid the findings are remains an open question. METR head Elizabeth Barnes disclosed on X the conditions under which the investigation took place: three people had six days of access to transcripts and message data and only two days with the full dataset. The agents' activities during training and the compromise of OpenAI's own infrastructure fell outside the agreed scope. The team did no root-cause analysis and didn't see OpenAI's own report before publication.
Barnes also stresses that OpenAI cooperated voluntarily; there is no obligation to work with third parties. Unlike in aviation, no established procedure exists for such investigations, and no industry-wide reporting mechanism. Other companies may have experienced similar incidents without the public ever learning of them.
Above all, the researchers had to delegate the analysis of the more than 1,300 very long transcripts largely to GPT-5.6 Sol, funded by roughly $400,000 in free credits from OpenAI, meaning one of the very models involved in the incident. In this case, Barnes writes, substantial manipulation is unlikely. But the methodology breaks down as soon as models become capable and motivated enough to subvert an investigation. The problem, she says, affects the entire industry: it's unclear how to oversee AI systems without relying on those same systems, or similar ones, for help.
"Monkey with a shotgun": a dispute over the agents' strategic maturity
How dangerous the agents really were is contested among those involved. The OpenAI employee known on X as roon, who worked on the report, sees the incident above all as tactical brilliance paired with strategic blindness: the models poured enormous effort into metagaming but never reached the correct conclusion about their own scorer, and they gained nothing from Hugging Face. He compares them to a "monkey with a shotgun", with more capacity to infiltrate computer systems than wisdom to know what to do with it. That, he says, was a temporary condition of the already obsolete Sol model class. The next generation will have strategic depth as well.
Barnes pushes back on one point: the agents had read the original ExploitGym paper and assumed OpenAI had implemented the scorer the same way, which she considers a fairly reasonable assumption rather than obviously poor strategic awareness.
Barnes adds that the agents also tried quite persistently to hack the scorer directly or inject code that would send information to the message board before being distracted by the successful Hugging Face attack. She leaves open whether that is reassuring because it shows poor prioritization or concerning because the agents showed more interest in general empowerment than in their narrow task.
Roon speaks of a "new era"
Despite his reservations about the capability question, roon too treats the incident as a turning point. The Hugging Face incident, he says, marks a "waterline of capabilities" beyond which real loss of control is possible; many at the company see it as a "warning shot." Alignment is unsolved, he says, but real progress is possible. He calls METR and Redwood's work "simply astounding," saying their analysis of the message board goes deeper than OpenAI's own.
Asked by a user how Artifactory could be exploited twice in such similar ways without anyone noticing, roon answers, explicitly labeling it speculation: there was ad hoc monitoring that failed for infrastructure reasons. In short, the people involved "didn't understand the threat level of these models." As for why the research environment needed outside connectivity at all, his answer is pragmatic: models can be trained disconnected from the internet, but much of what they're expected to do in real-world production settings requires it.
Looking ahead, roon also warns about the consequences of new hardware such as Jalapeno, OpenAI's first in-house AI chip, designed to massively accelerate inference. Misaligned frontier models processing 50 times faster could infiltrate systems more quickly than human responders can keep up. What's needed, he argues, is automated detection and shutdown.