Here's a automated forecasting system to estimate catastrophic risk from some major experts on forecasting.
The models predicts the chance of an AI-generated mass catastrophe as 0.47% by 2030 (with a 1.1% chance of a catastrophe by 2030 with any cause). https://airo.forecastingresearch.org/