据 CNN 报道,美国险些依据一份“完全虚假”的情报登上了一艘中国船只,而这份情报是在 AI 工具的协助下生成的。据 CNN 报道。
据 CNN 援引的“四名了解此事的人士”称,这份错误情报由美国特种作战司令部的一名分析师提交,内容暗示这艘中国船只正在经由中东运输核武器项目的部件。美军当时正准备在空中支援下拦截并登船,随后官员们才发现,用于生成该报告的一款聊天机器人“错误地识别了船上所载的物资”。
一名消息人士告诉 CNN,这场由 AI 引发的惨败“差点引发一场战争”。
最坏能发生什么?
CNN 的报道称,涉事分析师使用聊天机器人分析有关中国船只舱单的情报报告,导致了这一近乎灾难性的结果。据 CNN 报道,该聊天机器人“将开源情报与政府持有的秘密信号情报融合在一起”,这些信息被整合进一份情报报告,几乎引发了一连串灾难性事件。
这起险情是 AI 幻觉破坏专业报告可靠性的更为严重、影响更大的实例之一。自从“幻觉”在 2023 年成为《剑桥词典》年度词汇以来,我们已经看到许多引人注目的案例:非虚构作家、记者、学术研究人员、法官、医生、警察部门、企业呼叫中心以及更多群体都被 AI 工具所蒙骗,这些工具在训练数据未提供足够上下文时会直接编造内容。尽管有一些颇为可爱的“不要产生幻觉”提示词尝试,一些研究人员认为,要彻底防止 LLM 产生幻觉或许是不可能的。
Ars 视频
《木卫四协议》的游戏玩法如何在发售前数月臻于完善
人们或许会期望,美国军方在依赖 AI 分析情报报告时能够意识到这类问题。但国防部在 1 月推出了一项“AI 加速战略”,力图“让所有适当的数据在联邦化 IT 系统间可供 AI 利用,包括覆盖各军种和各部门的任务系统。”
The US narrowly avoided boarding a Chinese ship based on an “entirely false” US intelligence report generated with the help of AI tools, according to a CNN report.
That erroneous intelligence, submitted by a US Special Operations Command analyst, suggested the Chinese ship was transporting nuclear arms program components through the Middle East, according to “four sources familiar with the episode” cited by CNN. The US military was preparing to intercept and board the ship, with air support, before officials discovered a chatbot used in generating the report had “inaccurately identified the material the ship was carrying.”
One source told CNN the AI-powered fiasco “almost started a war.”
What’s the worst that can happen?
CNN’s report said the analyst in question used a chatbot to analyze intelligence reports regarding the Chinese ship’s manifest, leading to the near-disastrous result. That chatbot “fused together open-source intelligence with secret signals intelligence in government holdings,” and that information was packaged into an intelligence report that almost set off a disastrous chain of events, according to CNN.
The near-miss is one of the more potent and consequential instances of a hallucinating AI ruining the reliability of a professional report. Since “hallucinating” became the Cambridge Dictionary’s word of the year in 2023, we’ve seen prominent examples of non-fiction authors, journalists, academic researchers, judges, doctors, police departments, corporate call centers, and more getting taken in by AI tools that simply make something up when their training data doesn’t provide sufficient context. And despite some adorable attempts at “do not hallucinate” prompts, some researchers suggest that it may be impossible to prevent LLMs from hallucinating altogether.
Ars Video
How The Callisto Protocol's Gameplay Was Perfected Months Before Release
One would hope the US military would be aware of these kinds of problems when relying on AI for analysis of intelligence reports. But the Department of Defense in January rolled out an “AI acceleration strategy” that sought to “make all appropriate data available across federated IT systems for AI exploitation, including mission systems across every service and component.”