Can you even imagine the nation state level attacks METR & other eval orgs will come under?
Eval orgs also need world class cybersecurity as well as AI talent to do this right, they are not overlapping fields from my experience.
🚨 Attackers stole a METR API key and used it for three weeks, consuming credits worth about $600,000. A fail-open bug disabled Google authentication on a publi...