If there really is a high chance of AI leading to the extinction of humanity within years/decades, then the only rational stance towards safety monitoring and research pacing should be stringent, top-down government involvement and universally ratified international treaties. Similar to what we have been doing for a long time with nuclear energy and nuclear weapons non-proliferation.
If that's not what we're doing, then we must infer that the risk isn't being seriously considered by anyone involved.
There are only two options that make sense here:
- The near-term species extinction risk is real and we are taking it seriously, with maximally heavy-handed regulation.
- The extinction risk isn't real, actual risks are mild, so we're only pursuing mild safety measures and self-certification.
But you cannot say "yes this is likely to end humanity in your lifetime, and we're going to completely wing it." That is simply not rational.