Meta 创始人兼 CEO Mark Zuckerberg 不遗余力地大肆宣传其新 AI 助手 Muse 的安全性，声称它“从底层开始就是为隐私和安全而构建的”。然而，一个零日漏洞可让本地运行的应用和终端命令完全控制该智能体，这引发了严重质疑。更令人生疑的是，Amazon 于周日开始在其网站上屏蔽 Muse。
Meta 在几周前推出了 Muse。这款助手“可以预约、填写表单并处理客户服务”，“主动帮你分担任务”，还能“进行购物、生成图像、创建文档，并与你常用的应用和服务连接”。这款 macOS 应用（奇怪的是，没有 Windows 版本）还能与用户的 WhatsApp、电子邮件、日历和社交媒体账户协同工作。当某项任务需要某个并不存在的工具时，Muse 会即时创建一个。
Meta 对 Muse 安全性的宣传太过头了
当然，要让 Muse 完成上述任何一项操作，用户必须先授予它访问自己账户的权限。这包括为该助手在每项服务上进行身份验证，而且由于这款应用运行在 macOS 上，还要赋予它访问操作系统所限制的各类设备资源的权限，比如向磁盘写入文件、访问麦克风和摄像头，以及监控位置和日历。苹果多年来一直在开发这些防御机制，以防止已安装的应用或终端中输入的命令访问这些资源，显然是因为该公司将其视为安全威胁。Muse 彻底废除了这些默认措施。
Ars 视频
《木卫四协议》团队如何设计出令人恐惧的沉浸式音频
这个零日漏洞允许任何应用或终端命令获取用于验证用户 Muse 账户身份的 token。Meta 的开发者在设计这款助手时，使得任何本地安装的应用或执行的代码，无论其拥有何种 macOS 权限，都能修改一长串未公开的设置。其中大多数设置相当无害，比如控制深色模式。然而，有一项设置绝非无害。它允许进程更改转录所发生的端点。通常情况下，这是一个由 Meta 运营的服务器地址。攻击者可以通过将位置更改为自己的端点来利用这一缺陷。一旦得逞，攻击者就获得了能够完全控制 Muse 账户的 token。
“我们可以操纵这个智能体，利用它的权限为所欲为，”发现该零日漏洞的 macOS 安全专家 Patrick Wardle 对 Ars 表示。“所以，我们不必编写一个非常全面的 Mac 恶意软件窃取程序，只需利用这个 AI 助手本身即可。”Wardle 表示，他已经开发了多个概念验证攻击，能够实现诸如向磁盘写入恶意文件和拍照等操作，在很多情况下，即便是保持警觉的用户也毫无察觉。
在这篇帖子发布超过 12 小时后，Meta 表示它发布了一个热修复补丁，修复了该 0-day 漏洞。
Meta 在短短几周内发布了两篇文章，记录了为确保一个拥有如此非凡用户数据和资源访问权限的助手具备安全性和隐私性所做的设计决策。这些文章的发布正值有披露称，对Anthropic和Google模型的内部测试导致了对外部第三方网络的安全入侵，而参与其中的工程师从未有意针对这些网络。在传统的人类黑客攻击中，这些行为很可能导致刑事指控。Meta 的这些文章很可能考虑到了由此引发的反弹以及要求放缓 AI 发展的呼声。
Wardle 表示，Meta 开发者做出的若干设计决策使得他的漏洞利用成为可能。其中之一是选择让 Muse 听写在云端进行，这样 Meta 就可以记录它。macOS 长期以来一直提供一种简单的方式，让应用在安全驻留于设备上的进程中处理听写和转录。如果开发者选择了这一更安全的替代方案，该攻击就不可能实现。
另一个有缺陷的决策是让任何应用都能控制所有未公开的设置。Meta 很可能出于可以理解的原因，打算让与 Muse 配合的应用控制 UI 设置。但任何应用或命令能够控制一个处理敏感用户语音的端点，则完全是另一回事。综合来看，这些设计决策引发了人们对开发者究竟投入了多少精力来设计和测试这款新助手的安全性和隐私性的质疑。
“在我看来，这些应用在安全性方面的标准要高得多。它们不必做到完美，但当你看看 Muse 时，在我看来，他们似乎根本没有考虑过安全问题，这实在令人担忧，”Wardle 说。“至少，他们应该从一开始就考虑安全问题，而他们根本没有这样做。”
在 Wardle 披露该零日漏洞大约 12 小时前，Amazon 开始阻止人们使用 Muse 在该网站购物。尝试使用的用户会收到一条消息，称 Muse 是一个“未经授权的 AI 智能体，违反了 Amazon 的使用条件”。
“我们认为，代表客户从其他商家进行购买的第三方应用应当公开运营，并尊重服务提供商关于是否参与的决定，这一点相当明确，”Amazon 在一份邮件声明中表示。“这有助于确保安全、可靠、可信的客户体验，这也是其他方的运营方式，包括外卖应用及其接单的餐厅、配送服务应用及其购物的商店，以及在线旅行社及其为客户预订机票的航空公司。像 Muse 这样的智能体第三方应用也有同样的义务，我们已要求 Meta 将 Amazon 从该体验中移除。”
只需一次 ClickFix 就够了
攻击有多种生效方式。其中一种是让攻击者的服务器充当代理，置于 Muse 用户与 Meta 端点之间。一旦用户输入语音提示词，攻击者的服务器就会追加一段调用恶意命令的提示词，例如将全部 WhatsApp 消息的归档发送给攻击者。一旦得手，攻击者便永久控制该 Muse 账户，因为 token 也会自动发送到恶意服务器。
Meta founder and CEO Mark Zuckerberg has gone to great lengths to hype the security of its new AI assistant, Muse, claiming it is “built from the ground up for privacy and security.” A zero-day vulnerability that gives locally run apps and terminal commands complete control of the agent raises serious doubts. Further raising questions, Amazon on Sunday began blocking Muse from its site.
Meta introduced Muse a few weeks ago. The assistant “books appointments, fills out forms and handles customer service,” “proactively takes tasks off your plate,” and can “make purchases, generate images, create documents, and connect with your favorite apps and services.” The macOS app (curiously, there’s no Windows version) also works with a user’s WhatsApp, email, calendar, and social media accounts. When a task requires a tool that doesn’t exist, Muse creates one on the fly.
Meta doth hype Muse security too much
Of course, for Muse to do any of these things, users must first give it access to their accounts. This includes authenticating the assistant to each service and, because the app runs on macOS, giving it permissions to a broad range of operating system-restricted device resources, like writing files to disk, accessing the mic and camera, and monitoring location and calendars. Apple has spent years developing these defenses to prevent installed apps or commands entered into the terminal from accessing these resources, clearly because the company considers them a security threat. Muse completely undoes these default measures.
Ars Video
How The Callisto Protocol's Team Designed Its Terrifying, Immersive Audio
The zero-day allows any app or terminal command to gain access to the token that authenticates users to their Muse account. Meta developers designed the assistant so that any locally installed app or executed code, regardless of the macOS permissions it has, can change a long list of undocumented settings. Most of them are fairly innocuous, such as controlling dark mode. One setting, however, is anything but innocuous. It allows processes to change the endpoint where transcription occurs. Normally, it’s a server address operated by Meta. Attackers can exploit this flaw by changing the location to their own endpoint. Once that happens, the attackers have the token that gives complete control over the Muse account.
“We can manipulate the agent and leverage its privileges to do whatever we want,” Patrick Wardle, the macOS security expert who discovered the zero-day, told Ars. “So instead of us having to write a very comprehensive Mac malware stealer, we can just leverage the AI assistant itself.” Wardle said he has developed several proof-of-concept attacks that do things like writing malicious files to disk and snapping pictures, in many cases with no indication to even an alert user.
More than 12 hours after this post went live, Meta said it released a hotfix that patched the 0-day.
Meta has published two posts in as many weeks documenting the design decisions that went into ensuring an assistant with such extraordinary access to user data and resources is secure and private. The posts come amid revelations that internal testing of models from Anthropic and Google has resulted in security breaches of external, third-party networks that the engineers involved never intended to target. In traditional human-only hacking, these actions could likely result in the filing of criminal charges. The Meta posts are likely mindful of the resulting blowback and the calls to slow down AI development in response.
Wardle said that Meta developers made several design decisions that made his exploit possible. One is the choice for Muse dictation to occur in the cloud, where Meta can log it. macOS has long provided a simple means for apps to handle dictation and transcription in processes that stay securely on the device. Had the developers chosen this safer alternative, the attack wouldn’t have been possible.
Another flawed decision is for any app to control all of the undocumented settings. It’s likely Meta intended for apps working with Muse to control UI settings, and for understandable reasons. The ability for any app or command to control an endpoint where sensitive user speech is processed is an entirely different matter. Together, the design decisions raise questions about just how much effort developers put into designing and testing the security and privacy of the new assistant.
“To me, the bar is infinitely higher in terms of the security of these apps. They don’t have to be perfect, but when you take a look at Muse, it’s like they didn’t, in my opinion, think about security, which is really worrisome,” Wardle said. “At the very least, they should be thinking about security from the very start, and they are just not.”
Roughly 12 hours before Wardle disclosed the zero-day, Amazon started blocking people from using Muse to shop on the site. Users who tried received a message saying Muse was an “unauthorized AI agent [that] violates Amazon’s Conditions of Use.”
“We think it’s fairly straightforward that third-party applications that offer to make purchases on behalf of customers from other businesses should operate openly and respect service provider decisions about whether or not to participate,” Amazon said in an emailed statement. “This helps ensure a safe, secure, and reliable customer experience, and it is how others operate including food delivery apps and the restaurants they take orders for, delivery services apps and the stores they shop from, and online travel agencies and the airlines they book tickets with for customers. Agentic third-party applications such as Muse have the same obligations, and we’ve requested that Meta remove Amazon from the experience.”
A single ClickFix is all it takes
There are several ways for attacks to work. One is for an attacker’s server to act as a proxy that’s placed between the Muse user and Meta endpoint. Once the user enters the voice prompt, the attacker’s server adds a prompt invoking a malicious command, such as sending an archive of all WhatsApp messages to the attacker. Once that happens, the attacker gains permanent control over the Muse account because the token is automatically sent to the malicious server as well.