# Jev-as-a-Judge 论文：廉价裁判置信时接受、不确定时上升至 GPT-6，保留 99% 准确率并省约 43% 费用

- 来源：DAIR.AI (@dair_ai)
- 发布时间：2026-09-24 23:40
- AIHOT 分数：51
- AIHOT 链接：https://aihot.news/items/cmufpw01h03v5roxzbjbbud56
- 原文链接：https://x.com/dair_ai/status/2103147453717545278

## AI 摘要

论文介绍 JEV-as-a-Judge，发现多数评测可用廉价裁判，仅把不确定判定交给前沿模型。在 510 个保留偏好对上，级联接受 JEV 置信判定、其余上升至 GPT-6 Astra，保留 GPT-6 约 99% 的准确率，费用约为其 57%。

## 正文

Banger paper introducing Jev-as-a-Judge.

The overall finding is that you want to use a cheap judge for most of your evals and send only the uncertain calls to a frontier model.

This paper measures how well that works with JEV, TypeSafe AI's decision-only judge.

On 510 held-out preference pairs, a cascade that accepted JEV's confident verdicts and escalated the rest to GPT-6 Astra kept 99% of GPT-6's accuracy at about 57% of its fee.

JEV returns a verdict and label probabilities with no reasoning text.

It costs $0.044 per 1,000 judgments at a median latency of 0.152 seconds, against $12.182 and 1.885 seconds for GPT-6, about 277 times cheaper.

On ordinary preference and evidence-grounded factuality it stays within 3 points of GPT-6 (92.2% against 93.5% on RewardBench, 87.5% against 86.7% on HaluEval).

The gap grows to 9 to 20 points on tasks that require checking a derivation or rejecting an elaborately written wrong answer, such as JudgeBench (78.6% against 93.1%).

On several benchmarks, JEV's gap to GPT-6 is concentrated in its low-confidence decisions, which is why the cascade works.

The escalation threshold did not transfer for every fallback model, so the authors recommend setting it on your own data.

Paper: https://academy.dair.ai/papers/jev-as-a-judge-accept-when-confident-escalate-when-unsure-2609.26550