
如果你错过了,昨天 Anthropic 的 Dario Amodei 写了一篇文章,这可能会被证明是今年——甚至可能是这十年——最具影响力的文章之一:3500 字论述当前 AI 局势,至少在一定程度上主张对 AI 发展进行“节奏控制”,Sam Altman 和 Elon Musk 迅速对此表示支持。
我们都应该向他们初步达成的放缓这项技术加速的共识致敬,毕竟没有一家 AI 公司似乎能很好地控制它。我希望他们能用鲜血签署,让协议真正长出牙齿,也希望政府能约束他们履行承诺,迫使它变成实质性的东西。如果 Dario 能推动这一切,愿上帝保佑他。
我尤其欣赏 Dario 对透明度的承诺;Sam 似乎也认同,我希望 Elon 也愿意支持这一点。
但没错,也有一些理由让人持怀疑态度。
首先,这篇文章以惯常的炒作式废话开头,用来给生成式 AI 做宣传。正如我昨天在这里已经指出的,这既包括夸张的末日论,也包括夸张的欢呼论。AI 并不会真的搞垮互联网,而且(正如另一篇更早的文章中所讨论的)它也不会治愈癌症,至少不会在短期内或靠它自己做到。
然后是整篇文章自我放纵式的结构,David Sacks 昨天在一条长推文中对此做了很好的解构。我最喜欢的一段是这个:

我也很喜欢François Chollet 所写的内容:

两人都心存疑虑,并担心这里真正的目的可能只是过河拆桥。
质疑声四起,而且理由充分。在给我的一条短信中,有人说:“我认同[他这篇文章]的某些方面,程度超过他写过的任何东西,但我也认为这不过是更多的炒作,目的是抢先压制真正的监管。”
当我分享 Amodei 所说的那段话时——“最有效的节奏控制方法是通过针对所有美国前沿 AI 公司的监管,因为这样能覆盖到那些不愿自愿合作的公司。Anthropic 长期以来一直支持合理且有针对性的人工智能监管,特别是那些聚焦于透明度和第三方审计的法案。我认为所有前沿实验室都应与政府合作,将常设嵌入式评估员的构想正式化,以更好地预防和记录过去几个月里发生的那种内部对齐事件,并实施专注于让能力与安全保持平衡的监管。”Rasmussen 回击道:“我同意[这一点],但我也觉得这是胡说八道。这并非真正意义上的支持政府监管。这是‘与政府合作’,我认为这是个好模式,但公平的伙伴关系并不是它的本意。至少我不这么认为。这是一种表面上支持监管、实际上却继续影响监管进程的做法。”
以下是我看到的其他一些合理的担忧:
以及 Hugging Face 的联合创始人:
不少质疑来自 Dario 提出的使用独立评估者的建议。这有一定道理,但他指向了一个名为 METR 的团体,而该团体与 AI 公司本身处于极为相似的轨道之中:

理所当然地,这招致了一些反对。点名 METR 是一种监管俘获的形式:
因为现实是,METR 尽管很优秀,却是硅谷的产物:
而正如 Gillian Hadfield 所指出的,任何嵌入太久的企业都可能被裹挟其中:

当然,Anthropic 自己不应该来选择由谁来做评估。
与此同时,一如往常,Amodei 未能平衡他的商业欲望与他的伦理诉求。这一点在他拿中国——一个对他经济利益的威胁——当作沙袋/刺激物,以便让自己无论嘴上说什么都能继续加速时,表现得最为明显。但正如 Robert Wright 昨天所说,“任何严肃的 AI 减速与治理努力,都应当包含与中国立即、广泛且真诚的对话”。
Amodei 的这类段落无助于与中国的对话:

另外,虚伪警报:Anthropic 本身就建立在把全世界的思想蒸馏进自家模型之上;抱怨别人蒸馏他们的蒸馏成果,是教科书式的“抽掉梯子”之举。
无论如何,妖魔化中国只会让他们更不愿达成协议。而这又会让 Dario 说:“算了,我们没法同步节奏,因为他们不肯。”
关于“同步节奏”的另一点是,它回避了我们至少还应该考虑的其他两个重要政策选项。
首先,我们应当坚持对构建出以疏忽方式危害社会的 AI 智能体的公司追究责任,甚至提起刑事指控。推出无法被良好控制的 AI 智能体,就是一种疏忽行为。我们知道这是危险的,而且这一点长期以来显而易见,甚至在 Hugging Face 事件之前就是如此:
但正是智能体的使用,让 Anthropic 这样的公司赚得盆满钵满,因为智能体是当今最耗费 token 的工具之一。如果它们要为后果承担责任,就会三思而后行。
我们可以考虑的第二种做法,就是干脆把这些东西召回,这会大有帮助:

正如 Codestrap CTO 针对上述内容所评论的那样:

矛盾的是,Dario 对夸张言辞的沉迷,实际上可能给 AI 行业带来更糟糕的结果,而他大概也心知肚明。
我猜想,Dario 之所以现在提出自己的方案,部分原因是他知道,另一种可能是一条像 Sanders-Casar 法案那样严苛的路线——该法案有着试图回应 Dario 最深层恐惧的善意初衷,但实施方式却糟糕、专横,简直是连孩子带洗澡水一起倒掉,甚至仅仅因为研究超级智能就把人关进监狱。这很糟糕,因为没有人应该因为想要造出《星际迷航》里的计算机而被扔进监狱。
Dario 所呼吁的内容中,大概最好的部分是透明度:

这正是我长期以来一直大力倡导的东西;它是我上一本书中政策提案的核心。像 Deb Raji、Meg Mitchell 和 Timnit Gebru 这样的研究者,为透明度奔走呼吁的时间甚至更久。事实上,外部评估(其起点正是透明度)这一整套理念,正是他们多年来一直在倡导的。
如果 AI 行业最终愿意自己来做这件事,哪怕只是作为摆脱他们自己陷入的这团乱局的一种方式,那太好了!
而且别忘了还有责任追究和产品召回这类工具。
附:今早的突发新闻中,特朗普总统完全反对放缓的想法,他辩称:“我们在 AI 领域领先中国。我们是世界上最先进的国家,坦率地说,我希望保持这种状态,因为谁赢得 AI 谁就赢得一切”——这多少忽略了这样一个事实:中国已经几乎追赶上来了。我仍然反对这种零和思维,并将在未来的一篇文章中论述为什么特朗普应该重新考虑这一点。

In case you missed it, yesterday Anthropic’s Dario Amodei wrote what may prove to be one of the most consequential essays of the year, if not the decade: 3,500 words on the current AI situation that (at least to some degree) advocates for a “pacing” of AI development, which Sam Altman and Elon Musk quickly endorsed.
We should all salute their tentative agreement to slow down the acceleration of this technology, given that not one of the AI companies seems to have good control over it. I hope they will sign in blood, and put actual teeth in their agreement, and that the government will hold them to it and force it to become something substantive. If Dario catalyzes all that, god bless him.
And I especially love Dario’s commitment to transparency; Sam seems on board and I hope Elon is down for that, too.
But yeah, there are some reasons to be cynical, too.
To begin with, the essay begins with the usual hypey bullshit, which is used to advertise generative AI. As I already noted here yesterday, that includes both hyperbolic doom and hyperbolic cheer. AI is not really going to take down the internet, and (as discussed in another earlier essay) it’s not going to cure cancer either, at least not anytime soon or on its own.
Then there is the self-indulgent construction of the whole thing, which David Sacks well-deconstructed in a long tweet yesterday. My favorite bit was this:

I also liked what François Chollet wrote:

Both are suspicious, and worried that maybe the real goal here is just to pull up the ladder.
Skepticism abounds, with good reason. In a text to me said “I agree with aspects [of his esssay] more than anything he’s ever written but I also think it’s just more hype and is meant to preempt actual regulation”
When I shared the part where Amodei says “The most effective method of pacing is via regulation that targets all US frontier AI companies, as that covers even those who are unwilling to cooperate voluntarily. Anthropic has long supported sensible and targeted AI regulation, specifically bills that focus on transparency and on third-party auditing. I believe all frontier labs should partner with government to formalize the idea of permanent embedded evaluators to better prevent and document internal alignment incidents like those that have occurred in the last few months, and to implement regulation focused on keeping capabilities in balance with safety. Rasmussen shot back with “I agree with [this] but I also think it’s bullshit. It’s not really pro-government regulation. It’s “partner with government”, which I think is a good model, but an equitable partnership is not what is meant. At least I don’t think so. It’s a way of appearing pro-regulation but actually continuing to influence the regulatory process.”
Here are some other legitimate concerns I have seen:
and Hugging Face’s co-founder:
A bunch of the skepticism comes from Dario’s proposal to use independent evaluators. That makes some sense but he points to a group called METR that is very much in the same orbit as the AI companies themselves:

Rightly, this got some pushback. Dropping METR’s name is a form of regulatory capture:
Because the reality is that METR, as good as they are, are creatures of the Valley:
And as Gillian Hadfield notes, any company that is embedded too long might get swept up:

Certainly Anthropic themselves shouldn’t be choosing who does the evaluation.
Meanwhile, as is often the case, Amodei fails to balance his commercial desires with his ethical desires. This comes out clearest when he uses China, a threat to his economics, as a punching bag/goad to allow him to keep accelerating regardless of the rest of what he says. But as Robert Wright put it yesterday “any serious AI slowdown-and-governance effort should involve immediate, broad-gauged, and good-faith dialogue with China”.
Passages like this from Amodei aren’t helping the dialogue with China:

Also, hypocrisy alert, Anthropic is built on distilling the world’s ideas into their models; moaning about how others are distilling their distillation is a textbook pull-up-the-ladders move.
In any case, demonizing China is a sure way to deter them from making a deal. Which would lead to Dario saying, “never mind, we can’t pace because they won’t.”
The other thing about “pacing” is that it sidesteps at least two other important policy options that we should consider.
First, we should insist on liability and even criminal charges against companies that build agents that harm society in negligent ways. It is negligent to put out AI agents that can’t be well-controlled. We KNOW this is dangerous, and it’s been obvious for a long time, even before the Hugging Face incident:
But the use of agents is what allows companies like Anthropic to rake in big bucks, because agents are some of the most token-hungry tools out there. If they were held liable for the consequences, they would think twice.
The second approach that we could consider is simply recalling the damn things, and that would help a lot:

As Codestrap CTO put it, commenting on the above:

Paradoxically, Dario’s addiction to hyperbole might actually lead to worse outcomes for the AI industry, and he probably knows it.
Part of why I imagine Dario is making a proposal of his own now is that he knows that the alternative could be something draconian like the Sanders-Casar bill, which has a positive spirit of trying to address Dario’s worst fears, but a terrible, overbearing throw-the-baby-out-with-the-bathwater implementation, imprisoning people for even doing research on superintelligence. Which is bad because nobody should be thrown in jail for wanting to build the Star Trek computer.
Probably the best part of what Dario called for is transparency:

That’s something I have been strongly advocating for, for a long time; it was central to the policy proposals in my last book. Researchers like Deb Raji, Meg Mitchell, and Timnit Gebru have been lobbying for transparency for even longer. Indeed, the whole idea of outside evaluation (which starts with transparency) is something they have been advocating for, for years.
If the AI industry finally is finally willing to do that on its own, even if only as a way of steering out of this mess they have gotten themselves into, great!
And let’s not forget tools like liability and product recalls, too.
P.S. In breaking news this morning, President Trump has opposed the idea of a slowdown altogether, arguing that “"We're leading China in AI. We're the most sophisticated country in the world, and frankly I want to keep it that way because whoever wins AI wins” — somewhat ignoring the fact that China has already nearly caught up. I continue to oppose this zero-sum thinking, and will write about why Trump should reconsider it, in a future essay.