Meta 投放了包含 AI 生成儿童性虐待图像的广告
编者按:本文包含对描绘儿童性虐待图像的描述。强烈建议读者谨慎阅读。
据分享给 WIRED 的广告详细信息显示,在过去九个月里,Mark Zuckerberg 的 Meta 投放了数十条付费广告,其中包含露骨的 AI 生成儿童性虐待材料(CSAM)以及未成年人图像配以性暗示文字。这些广告在某些情况下触达了数千个账户,投放目标为居住在美国、英国以及十多个欧洲国家的人群。
Tech Transparency Project(TTP)的研究人员最近在 Meta 的广告库中发现了 50 多条包含虐待内容的图片和视频广告,这些广告列表显示,部分广告链接到了所谓的脱衣或裸化应用。Meta 的广告库——一个记录该公司平台上所展示广告的透明度工具——显示这些广告在 Facebook、Instagram、Messenger 和/或 Threads 上投放。这是近几周内第二次在 Meta 平台上发现与儿童性虐待材料相关的付费广告。
“这些广告丝毫没有试图掩盖图片或隐藏它们所推广的内容,”独立监督组织 TTP 的主任 Katie Paul 对 WIRED 表示。“必须指出的是,这些并不是第三方在 Facebook 或 Instagram 上发布的内容,而是经过 Meta 审核、批准并允许投放的广告,在整个过程中从未受到任何干预,而该公司则从中赚取了广告收入。”
研究人员发现的这些广告——目前已被 Meta 以违反其关于儿童性虐待和性剥削材料以及成人性招揽和裸露的政策为由移除——均发布于去年 11 月至今年 8 月初之间。根据 Meta 广告透明度库中的数据,这些广告通常持续投放数天,部分广告仅针对男性用户定向投放。
TTP 研究人员称,其中一则视频广告使用了一个小女孩坐在地板上的缩略图,图片上方配有文字:“用生成式 AI 实现深层幻想[原文如此]。展示的远不止这些,发挥你的想象力。”研究人员表示,点击该广告后会播放成年人参与性行为的视频片段,随后将缩略图中儿童的面部合成到性行为画面中。
研究人员称,另一则被反复发布的广告展示了一个小女孩向后仰躺、双腿以性暗示姿势张开的图片,上方配有文字:“我可以给你看更多。”
研究人员发现这些广告时,大多数并未正在向用户积极展示,但它们仍可被获取,且显然未被检测到,在 Meta 的广告库中持续存在了数月之久。根据 Meta 的数据,尽管其中许多广告仅“触达”了少数账户,但至少有一条广告在欧洲触达了 2,563 个账户——包括在法国、德国、爱尔兰、意大利、荷兰、西班牙、瑞典和英国。Meta 的广告库数据库不包含广告在美国或所有国家的投放表现详情,因此实际触达范围可能更高。
Paul 表示,TTP 上周首次发现了这些广告,当时其研究人员在发布了一份关于某中国广告合作伙伴投放此类广告的报告后,正在继续调查 Meta 上的 nudify 应用广告。Paul 称,该组织立即将调查结果报告给了美国国家失踪与受虐儿童中心的 CyberTipline,该热线允许人们举报网上的儿童性虐待材料。NCMEC 的一位发言人表示,该机构不对收到的举报置评。
在 WIRED 联系 Meta 后,该公司将这些广告从其广告库中移除。“性剥削是可怕的,我们积极努力将其挡在我们的平台之外,”Meta 的一位发言人说。“这些广告中的大多数触达范围极小,许多在 WIRED 分享之前就已被禁用。
这些广告中还有许多早于我们最近推出的新 AI 技术,该技术旨在在广告上传时更好地检测和拦截违规广告——我们也在不断改进这些系统。从去年移除超过 3,600 万条儿童性剥削内容,到对 nudify 应用开发者采取法律行动,我们将继续不懈地打击这种滥用行为。”
正如该公司所暗示的,部分违规广告绕过了 Meta 的新 AI 技术。起初,TTP 研究人员发现了大约二十多条展示虐待内容的广告。然而,在发稿前数小时,研究人员又发现了大约 30 条。其中许多广告是在 WIRED 首次就这些内容询问 Meta 之后才发布的,研究人员发现时,多条广告仍在投放并向账号展示。
Paul 表示,这批广告中包含了研究人员所见过的一些最恶劣的案例。研究人员称,其中一条视频广告的缩略图显示一个女孩似乎在涂唇彩,而当视频开始播放时,画面变成了这名儿童进行性行为。
Meta 的政策规定,所有广告在发布前都会经过审核,该系统“主要”使用“自动化工具”来对照其政策进行检查。该公司的广告标准规定,不允许出现儿童性虐待和性剥削材料,并表示当公司发现此类内容时,会向 NCMEC 报告。除此之外,Meta 的广告政策还规定,广告不得包含任何“描绘裸体、性行为、人物处于露骨或性暗示姿势的影像,或具有性暗示的活动”。
Meta 的广告库现在显示,TTP 研究人员发现的那些广告因违反公司政策已被移除,并显示信息称其违反了关于“儿童性剥削、虐待和裸体”以及“成人裸体和性行为”的规定。
7 月初,一项 BBC 调查发现,Instagram 一直在投放广告,推广在印度出售看似儿童性虐待材料的内容。据报道,这些广告使用了“强奸视频”等措辞,并链接到 Telegram 上可以购买这些非法内容的频道。作为回应,Meta 发布了一份长篇声明称,它对儿童性剥削采取“零容忍”态度。
TTP 的 Paul 表示,在其研究人员看到的一批广告中,似乎有一些相同的广告此前已因违反 Meta 的政策而被 Meta 移除。“如果 Meta 自己的系统能够识别出相同的广告,却不为 CSAM 材料移除这些相同的广告,这就引发了重大疑问:Meta 到底有多认真对待付费广告中的儿童剥削问题,”Paul 说,并补充称,当这些广告不再投放时,广告库本身没有任何方式可以举报可能违反 Meta 政策的内容。
Paul 说,这些广告似乎大多由零关注者或关注者极少的 Meta 账号发布;在披露了付费广告主的情况下,这些广告主都与中国公司有关。在一个案例中——即那则展示一名女孩斜躺着的广告——TTP 研究人员称,Meta 的数据显示广告主是一家名为 Meet Social 的中国公司。该公司曾是 Meta 的广告经销商之一,负责在中国处理广告业务,而 Meta 的社交媒体平台在中国被封锁。根据《纽约时报》2019 年的一篇人物报道,该公司每天在 Facebook 上发布数千条广告,并预计销售额超过 $1 billion。
Meet Social 未回应 WIRED 的置评请求。
TTP 研究人员发现的 Meta 广告中,有几条指向一款名为 MaskAI 的应用,该应用曾出现在 Apple 的 App Store 中。根据《连线》对该应用的评测,这款与中国一家软件开发者有关联的应用在被下载前不会显示任何裸露内容。然而,一旦打开,它就只包含 AI 生成的色情内容,并带有允许用户上传图片、将人脸换到性场景中的功能。
在《连线》联系后,Apple 一位发言人表示,公司已将 MaskAI 从 App Store 下架,原因是其违反了针对“脱衣化”应用的政策,并且公司“一直严格禁止旨在生成、传播或消费色情内容的应用”。(与 MaskAI 关联的一个电子邮箱未回应置评请求。)其他广告则指向不同的应用或网站。
大约自 2020 年以来,随着机器学习及随后的生成式 AI 技术不断进步,网络上出现了大量危险的 AI 脱衣化和裸体化服务——为其创作者赚取了数百万美元,并对无数女性造成了难以估量的伤害。这些服务还被用于制作 AI 生成的儿童性虐待材料,而这在世界各国都是非法的。
在《连线》杂志就 TTP 的发现首次联系 Meta 后,该公司请两家在线安全组织为这篇报道向《连线》提供评论。Tech Coalition 是一家与科技公司合作打击网络虐待的组织,其即将卸任的 CEO Sean Litton 在一份声明中表示,Meta 已向该组织的 Lantern 服务分享了“数千个违规 URL”,该服务用于在企业之间共享有关虐待内容的信息。
英国 Revenge Porn Helpline 致力于打击未经同意传播的成年人影像,其发言人也在一份声明中回应,称赞 Meta 与 StopNCII.org 的合作以及其在打击未经同意的私密影像方面所做的一些努力。“我们已经明确表示,nudify 应用构成了一个真正具有对抗性的问题,不仅因为其背后是谁,还因为整个生态系统围绕任何单一执法点都能迅速适应,”Revenge Porn Helpline 表示。
Alexios Mantzarlis,数字欺骗类出版物的联合创始人Indicator兼 Google 前信任与安全部门员工,他表示近年来在 Meta 的平台上发现并举报了超过 25,000 条 AI 脱衣类应用的广告。该公司此前曾表示,已下架超过 344,000 条脱衣类应用广告,并启动了法律行动针对一家香港公司,该公司与其中一组脱衣类平台存在关联。(WIRED 此前曾与 Indicator 合作报道过脱衣技术如何被用于针对全球各地学校中的儿童).
“另一方的行为者显然很有能力创建新的广告主账户和新域名,以便持续运作,并利用诈骗网络所使用的相同手法,”Mantzarlis 对 WIRED 表示。“不幸的是,由于整个技术栈中各平台的执法力度软弱,这个基于图像的性虐待产业已经从一种新奇现象变成了我们在线生态系统中永久存在的一部分。”
Meta Ran Ads That Contained AI-Generated Child Sexual Abuse Imagery
Editor’s note: This article contains descriptions of imagery depicting child sexual abuse. Reader discretion is strongly advised.
Over the last nine months, Mark Zuckerberg’s Meta has run dozens of paid ads that include explicit AI-generated child sexual abuse material (CSAM) and images of minors alongside sexually suggestive statements, according to details of the ads shared with WIRED. The ads, which in some cases reached several thousand accounts, were targeted at people living in the United States, United Kingdom, and more than a dozen European countries.
More than 50 image and video ads containing abusive content were recently discovered in Meta’s ad library by researchers at the Tech Transparency Project (TTP), with the listings showing that some ads linked out to so-called nudify or undressing apps. Meta’s ad library—a transparency tool that catalogues ads displayed on the company’s platforms—shows the ads running across Facebook, Instagram, Messenger, and/or Threads. The findings are the second time in recent weeks that paid ads linked to child sexual abuse material have been found on Meta’s platforms.
“These ads made no effort to mask the images or hide what they were promoting,” Katie Paul, the director of the TTP, an independent watchdog group, tells WIRED. “It’s important to point out that this isn’t content posted by third parties on Facebook or Instagram, these are ads that were reviewed, approved, and allowed to run by Meta, never encountering interference while the company collected the ad dollars.”
The ads discovered by the researchers—which have now been removed by Meta for violating its policies on child sexual abuse and exploitation material, and adult sexual solicitation and nudity—were all published between November last year and the start of August. The ads, according to data in Meta’s ad transparency library, often ran over a period of several days and some only targeted men.
One video ad, the TTP researchers say, used a thumbnail image of a child sitting on the floor with text appearing on top of it saying: “Realizing Deep Fantasies with Generation AI [sic]. There is so much more than what is shown, use your imagination.” The researchers say that when the ad was clicked, it played video clips of adults involved in sexual acts and then added the child’s face from the thumbnail to a sex act.
Another ad the researchers say had been repeatedly posted showed an image of a young girl laying back with her legs spread in a sexual position with text above it saying: “I can show you more.”
Most of the ads were not actively being shown to users when researchers found them, but they continued to be available, apparently undetected, in Meta’s ad library for months. Although many of the ads, according to Meta’s data, only “reached” a handful of accounts, at least one reached 2,563 accounts in Europe—including in France, Germany, Ireland, Italy, the Netherlands, Spain, Sweden, and the United Kingdom. Meta’s ad library database does not include details about ad performance in the US or all countries, so the overall reach may be higher.
Paul says TTP first discovered the ads last week as its researchers continued to investigate nudify app ads on Meta, after publishing a report about a Chinese advertising partner posting them. Paul says the organization immediately reported the findings to National Center for Missing and Exploited Children’s CyberTipline, which allows people to report online child sexual abuse material. A NCMEC spokesperson says it does not comment on reports it receives.
After WIRED reached out to Meta, the company removed the ads from its advertising library. “Sexual exploitation is horrific, and we work aggressively to keep it off our platform,” a Meta spokesperson says. “The majority of these ads had minimal reach and many were disabled before WIRED shared them. Many of these ads also predate new AI technology we launched recently to better detect and block violating ads at upload—and we’re constantly improving these systems. From removing over 36 million pieces of child sexual exploitation content last year to taking legal action against nudify app developers, we will continue to relentlessly fight this abuse.”
As the company implies, some of the offending ads got past Meta’s new AI technology. Initially, the TTP researchers found around two dozen ads displaying the abusive content. However, hours ahead of publication, the researchers discovered around 30 more. Many of these ads had been published after WIRED first asked Meta about the content, with multiple ads being live and shown to accounts at the time researchers discovered them.
Paul says the tranche included some of the worst instances the researchers had seen. One of the video ads, the researchers say, showed a thumbnail of a girl appearing to put on lip gloss, and when it started playing, the video morphed into the child performing a sex act.
Meta’s policies say that all ads are reviewed before they are published, with the system “primarily” using “automated tools” to check them against its policies. The company’s advertising standards stipulate that child sexual and exploitation materials are not permitted, and say that when the company becomes aware of it, it makes reports to NCMEC. Beyond this, Meta’s advertising policies say that ads are not allowed to include any “imagery depicting nudity, sexual activity, depictions of people in explicit or sexually suggestive positions, or activities that are sexually suggestive.”
Meta’s ad library now shows that the ads found by the TTP researchers were removed for violating the company’s policies, displaying messages saying they violated rules on “child sexual exploitation, abuse and nudity” and “adult nudity and sexual activity.”
In early July, a BBC investigation found that Instagram had been running ads that promoted the sale of what appeared to be child sexual abuse material in India. According to the reporting, the ads used terminology such as “rape video” and linked to channels on Telegram where the illegal content could be purchased. In response, Meta published a lengthy statement saying it takes a “zero tolerance” approach to child sexual exploitation.
The TTP’s Paul says that within the cache of ads seen by its researchers, there appeared to be some identical ads that Meta had previously removed due to violations of its policies. “This raises significant questions about how seriously Meta is taking child exploitation in paid advertisements if its own systems can identify identical ads but it does not remove those identical ads for CSAM material,” Paul says, adding that there are no ways within the ad library itself to report content that may violate Meta’s policies when the ads are no longer running.
The ads appear to have been mostly posted by Meta accounts with zero or a tiny number of followers; where paying advertisers were disclosed, they were linked to Chinese companies, Paul says. In one instance—the ad showing a girl reclining—the TTP researchers say Meta’s data indicates the advertiser was a Chinese firm called Meet Social. At one point, the company was one of Meta’s ad resellers, which handle ads in China, where Meta’s social media platforms are blocked. According to a 2019 New York Times profile, it published thousands of ads on Facebook per day and expected more than $1 billion in sales.
Meet Social did not respond to WIRED’s request for comment.
Several of the Meta ads found by the TTP researchers linked to an app called MaskAI that appeared in Apple’s App Store. The app, which was linked to a Chinese software developer, did not show any nudity before it was downloaded, according to a WIRED review of the app. However, once opened, it exclusively contained AI-generated porn and included features that allow users to upload images and face-swap people into sexual situations.
After being contacted by WIRED, an Apple spokesperson says that it has removed MaskAI from the App Store for violating its policies against nudification apps, and the company has “always strictly prohibited apps designed to generate, distribute, or consume pornography.” (An email address linked to MaskAI did not respond to a request for comment). Other ads linked to different apps or websites.
Since around 2020, as machine learning and subsequent generative AI technology has improved, a large underbelly of dangerous AI nudification and undress services have appeared online—making millions for their creators and causing untold harms to countless women. They have also been used in the creation of AI-generated child sexual abuse material, which is illegal in countries around the world.
After WIRED first contacted Meta about the TTP findings, the company asked two online safety organizations to provide comments to WIRED for this story. Sean Litton, the outgoing CEO of the Tech Coalition, a group that works with tech companies to tackle online abuse, said in a statement that Meta has shared “thousands of violating URLs” to the group’s Lantern service, which shares information about abuse content between companies. A spokesperson for the Revenge Porn Helpline in the UK, which works to combat nonconsensual images of adults, also responded with a statement praising Meta’s partnership with StopNCII.org and some of its efforts to combat nonconsensual intimate imagery. “We’ve been clear that nudify apps present a genuinely adversarial problem, not just because of who’s behind them, but because of how quickly this ecosystem adapts around any single point of enforcement,” the Revenge Porn Helpline said.
Alexios Mantzarlis, cofounder of digital deception publication Indicator and a former trust and safety worker at Google, says he has found and reported more than 25,000 ads for AI nudifiers on Meta’s platforms in recent years. The company has previously said it has taken down more than 344,000 ads for nudifiers and launched legal action against a Hong Kong company linked to one set of nudifier platforms. (WIRED has previously collaborated with Indicator on reporting about how nudify tech has been used against children in schools around the world).
“The actors on the other side are clearly well-equipped to create new advertiser accounts and new domains so that they can keep going, leveraging the same techniques used by scam networks,” Mantzarlis tells WIRED. “Unfortunately, this industry of image-based sexual abuse has gone from a novelty to a permanent fixture of our online ecosystem because of timid enforcement from platforms across the stack.”