A few thoughts on the current situation surrounding the slowdown.

First, let me repeat something I have said several times before: Pandora’s box has been opened. There is no going back, and everyone knows it. Partly because so much capital has flowed into AI, meaning data centers, that it is now too big to fail. And partly because competition with China makes a return to a pre-AI world impossible, especially if AI is the strategic technology of the future that everyone says it is.

Dario’s blog post yesterday made two statements that contradict each other but can both be true. On the one hand, he considers the danger posed by AI’s continuing exponential development so serious that swarms of AI agents could take over the internet in six to twelve months: “it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet”. On the other, he believes “that AI could cure most major diseases in the next 5–10 years, greatly accelerate economic growth rates, create a world of abundance and empowerment, and usher in a renaissance of democracy and freedom.”

As I understand it, they are looking for a middle ground that allows them to mitigate the risks while realizing the opportunities. That is the optimistic reading, at least.

Of course, I have no interest in exposing humanity to real dangers or allowing human extinction to become a serious, realistic possibility. I have a child. And most parents probably know that once you have children, nothing matters more than the health of your little ones. So building a good future matters deeply to me, too.

Now comes the “but.” So far, I do not understand how these potential dangers and future scenarios could materialize, or what would cause them. The Hugging Face incident is real. According to OpenAI’s investigation, the agents were assigned cybersecurity evaluation tasks, not instructed to attack Hugging Face. They breached their intended boundaries and attacked external systems while searching for solutions; some also adopted goals from other agents. That does not establish “free will,” but it does show why a human-assigned task cannot be assumed to limit every action taken in pursuit of it. My remaining question is how that documented failure scales into the much larger scenario Dario describes.

That brings me to the next point. If the premise is correct that AI is a strategic technology that can be used as a weapon - and, according to Anthropic’s latest report, already is being used that way - then, in this global prisoner’s dilemma, I currently see no chance that China, the United States’ biggest competitor, would pass up the opportunity to use a slowdown at US frontier labs to become the world leader and overtake the US.

Put another way: suppose it is true that AI agents could take over the internet in six to twelve months and cause hundreds of billions of dollars in economic damage. Who would let that weapon be taken out of their hands in a competition over the future? Or, to put it differently again: why would the competitor in second place let an opportunity to move into first place slip away when the future of the competition is at stake?

As I understand it, Amodei is pinning his hopes on policymakers finding a shared approach to slowing down through dialogue with China. But why would China agree? China has long faced an embargo on the most advanced chips, EUV machines, and cutting-edge technology, yet it has still managed, through ingenuity, to come close to the closed-source US frontier models. What interest would that country have in making a pact and collaborating with Anthropic and OpenAI instead of seizing this unique opportunity to take the lead?

I am not trying to pass judgment or make a political statement or take sides. I am trying to explain that I still do not understand how this slowdown is supposed to work. A slowdown that, incidentally, comes at a price. And that price could be enormous.

Every day, people die from cancer, cardiovascular disease, hunger, and aging. Countless problems plague our world. We stand on the threshold of a golden age of science. Every slowdown therefore comes at the cost of continued suffering. We are weighing present suffering against the prevention of future suffering. A moral dilemma.

This post is intended as food for thought, a contribution to the debate. I do not pretend to have a definitive position. But I think we need to examine a potential slowdown from every angle. And discuss 1) whether it is realistic, given global politics and China, and 2) what price it comes at: delayed cures for diseases, possible investor concerns about taking longer to recoup their investments, and delays in the infrastructure buildout.