Jev:用 RLCD 训练的单次调用模型零样本检测 AI 对齐失效

HuggingFace Daily Papers(社区热门论文)·2026-09-24 08:00·1天前
AI 导读

用强化学习校准决策(RLCD)训练的模型 Jev 可在单次调用中对同一输入回答多个带类型的问题并给出校准概率,零样本检测对齐失效的中位 AUROC 达 0.886,在多数基准上超过有监督基线。

HuggingFace Daily Papers(社区热门论文)
46AI 编辑部评分,满分 100

Jev:用 RLCD 训练的单次调用模型零样本检测 AI 对齐失效

2026-09-24 08:00· 1天前
AI 导读

用强化学习校准决策(RLCD)训练的模型 Jev 可在单次调用中对同一输入回答多个带类型的问题并给出校准概率,零样本检测对齐失效的中位 AUROC 达 0.886,在多数基准上超过有监督基线。

Detectors of alignment failures screen deployed language models and score alignment benchmarks. Most are generative judges that spend a decoding pass on every criterion, and classifiers that read token probabilities, such as Llama Guard, still score one fixed label per call. Jev, a model trained with reinforcement learning for calibrated decisions (RLCD), answers many typed questions about one input with calibrated probabilities in a single call. Whether it detects alignment failures has not been measured. We present RLCDAlignBench, which benchmarks Jev on ten alignment failures: sycophancy, jailbreaks, deception, prompt injection, hallucination, privacy violation, social bias, reward hacking, concealing uncertainty, and power seeking. It spans 44 benchmarks and five target models, labelled by each benchmark's scorer and, on two, by humans. Many of these failures are relational, defined against a reference, such as the user's belief or an injected instruction, that the response alone does not reveal. Our key idea is therefore to vary what Jev is asked separately from what it sees: the question's wording and answer type on one side, the fields of the input on the other. A single generic question reaches a median AUROC of 0.886 zero-shot and beats supervised baselines on most benchmarks. Question wording matters little, while context matters more, mostly through fields that encode the label. Jev matches the reference scorer's agreement with human labels, surfaces label defects in existing benchmarks, and costs 63x less than LLM-judge scorers. Code and data: https://github.com/sumleo/RLCDAlignBench.

来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org