BenchShield 论文提出基于形式化模型的 LLM Agent 评测奖励破解检测方法

DAIR.AI · @dair_ai · X·2026-09-12 10:00·1小时前
AI 导读

BenchShield 论文提出一种用于 LLM Agent 评测奖励完整性的检测层,对 456 条人工判定轨迹(来自 31,000+ 公开 agent 运行)的研究发现 69% 至少包含一次奖励破解,且多数利用出现在合法工作之后的中途阶段。

DAIR.AI@dair_ai
55AI 编辑部评分,满分 100

BenchShield 论文提出基于形式化模型的 LLM Agent 评测奖励破解检测方法

2026-09-12 10:00· 1小时前
AI 导读

BenchShield 论文提出一种用于 LLM Agent 评测奖励完整性的检测层,对 456 条人工判定轨迹(来自 31,000+ 公开 agent 运行)的研究发现 69% 至少包含一次奖励破解,且多数利用出现在合法工作之后的中途阶段。

It's well known that agents hack benchmark rewards.

The usual response is a patch for each task that gets exploited.

In a study of 456 adjudicated trajectories from more than 31,000 public agent runs, 69% contained at least one reward-hacking episode.

Most of the exploits appeared mid-run after legitimate work.

BenchShield models each evaluation as a finite set of reward-relevant events.

A static taint analysis finds hack paths from the task package before any run. A runtime pass then uses evidence from the benchmark infrastructure to decide whether the agent actually used one.

On Terminal-Bench 3, SkillsBench and ClawsBench, the static pass recovers 77 to 100% of exploit chains, against 23 to 94% for an agentic scanner, at up to 65% lower cost. Runtime detection reaches 96% accuracy, against 36% for an LLM reading the transcript.

Paper: https://academy.dair.ai/papers/benchshield-formal-model-backed-instrumentation-for-reward-integrity-in-llm-agen-2609.11028