Is this "Don't Look Up" for real? Leading mathematicians are warning of existential risks from advanced AI. In an open letter to the president of the Royal Society, 42 mathematical fellows urge the British Academy of Sciences to alert the government and the media. Signatories include Fields Medal winners Martin Hairer, Peter Scholze, and Wendelin Werner. None are affiliated with AI companies.

Within three months, leading models from OpenAI and Anthropic have solved open research problems, including one of the seven Millennium Problems (note: a second is likely to follow soon). These models now perform "at the level of the top human mathematicians in many parts of the subject," the letter states, with a "significant chance of them developing superhuman abilities within a similarly short timeframe."

The impact on math isn't what prompted the letter, though. The group warns that "we must assume that the rate of change of their capabilities will be similar" in areas like cybersecurity, autonomous weapon control, development of biological and chemical agents, and targeted misinformation. Estimates from AI labs putting existential risk above ten percent "must not be dismissed as 'hype,'" they write. "By the time the situation becomes obvious to the wider public it may be too late to act."

Letter