跳到正文
Greg Brockman· @gdb · X·· 2 小时前AI 评分52

Greg Brockman 解读 OpenAI 基于困惑度的预训练共同安全上限提议

AI 导读

Greg Brockman 介绍 OpenAI《Perplexity and Intelligence》一文,主张基于困惑度等指标设立开发者承诺不逾越的共同安全上限,并配以明确的退出标准(如 CoT monitors 的后继工具问世)。他认为对齐与监控进展可能不够快,无法支撑预训练的充分加速,目标是形成逐级设限、控制发展节奏的共同框架。

正文 · 原文

I helped start OpenAI because I want AGI to be safe and beneficial for people. This post provides transparency that can be the foundation of industry-wide shared safety bars.

Perhaps the most important concept from the post is the following: "AI will also significantly contribute to progress on alignment and monitoring, I think that progress is unlikely to be fast enough to allow the full potential acceleration in pretraining."

Towards ensuring that safety keeps pace with general intelligence of pretrained models (as measured by "perplexity"), the post proposes "common limits, based on perplexity and other measurements, that developers are committed to not crossing until our ability to understand and monitor AI behavior achieves a certain concrete milestone (such as the development of a successor to CoT monitors)."

The general idea is to create a first shared limit, on some shared key metric, along with concrete exit criteria that would allow going beyond that. This can lead to a "common framework for setting successive limits and pacing further development".

Excited to move forward the conversation on pacing towards shared standards that are rooted in the science!

引用Jakub Pachocki@merettm

We decided to provide more transparency into pretraining progress at OpenAI for the sake of moving toward shared safety bars. In “Perplexity and Intelligence” I argue for common limits based on perplexity and other measurements, with progress beyond them gated on our ability to understand AI behavior.

openai.com/index/perplexity-a…

在 X 查看被引用的帖子

来源:Greg Brockman · x.com

事件后续 · 1

查看事件全部