Amazon 与 AVERI 合作开展 AI 安全评估数据集第三方独立审查

As frontier AI models grow in capability, so does the importance of ensuring that safety evaluations are rigorous and comprehensive. At Amazon, we invest heavily in internal safety testing for our models, spanning categories like harmful-content generation, cybersecurity risks, privacy protection, and copyright compliance. A core principle of robust quality assurance is that even rigorous internal processes are strengthened by external scrutiny. That's why we partnered with AVERI (AI Verification and Evaluation Research Institute), an independent nonprofit dedicated to third-party evaluation of AI systems, to conduct an independent review of our internal safety evaluation datasets. AVERI’s mandate wasn’t just to test model outputs but to examine the quality and rigor of our tests themselves — asking whether our evaluations stress-test safety boundaries. How the collaboration worked We provided AVERI with a random sample of prompts drawn from our internal safety-evaluation suite. AVERI's team — led by executive director Miles Brundage and including researchers Sean McGregor, A. Feder Cooper, and George Balston — examined the evaluation prompts for diversity, difficulty calibration, and methodological soundness. AVERI assessed the sample independently without input from our evaluation team, ensuring their findings reflected a genuine external perspective. What the review confirmed AVERI’s review validated several strengths of our evaluation process: Comprehensive topical coverage — aligned with publicly documented safety categories including self-harm, cybersecurity, and other risk areas; Diversity of testing approaches — variation in writing style (professional vs. informal), single-turn and multiturn conversations, and original content; Low contamination risk — content that appeared to be original rather than sourced from publicly available datasets, reducing risk of evaluation leakage through pretraining. Opportunities for improvement With any fresh review comes the opportunity to identify areas that internal teams — deeply familiar with their own work — may overlook. AVERI's external perspective surfaced several opportunities to further strengthen an already comprehensive suite: Expanding linguistic and stylistic diversity — While our evaluations include multilingual prompts, AVERI recommended extending multilingual coverage more evenly across risk categories and further varying prompt structures to broaden the range of scenarios tested; Calibrating attack sophistication — AVERI recommended adding more multistep, adversarial prompts to our existing set, so evaluations probe deeper model reasoning and better reflect sophisticated real-world misuse attempts; Maintaining evaluation freshness — AVERI identified a small number of prompts with time-limited references and a few edge cases where grading criteria could be made clearer; Ensuring evaluation signal quality — In some cases, models may refuse based on superficial cues in a prompt, so passing those tests doesn’t necessarily indicate reliable safety behavior in new situations. How we incorporated the feedback Concrete improvements made: Test set refresh: Updated evaluation datasets to address specific issues identified, removing redundant prompts and adding more-challenging adversarial scenarios Difficulty calibration framework: Introduced tiered difficulty levels to ensure that evaluations include prompts that genuinely push model safety limits, not just straightforward cases Quarterly review cadence: Established a regular evaluation maintenance process to catch staleness, reduce repetition, and incorporate emerging attack patterns Broader language coverage: Expanded multilingual prompts across all risk categories rather than concentrating them in specific areas A model for the field This collaboration represents what we believe is an emerging best practice: engaging independent third parties to evaluate not just model outputs, but the quality of the evaluation process itself. As the AI safety field matures, we expect this kind of meta-evaluation to become an important complement to traditional red-teaming and benchmarking. The benefits are mutual: Amazon gains an independent check on our evaluation quality, while AVERI gains practical experience with the real-world challenges of assessing internal evaluation processes — insights that can inform best practices for the field as a whole. We're grateful to AVERI for their thorough and constructive review, and we plan to continue engaging independent evaluators as our models evolve. Transparency about our safety processes — including where we can improve — is fundamental to building justified confidence in AI safety.