论文 EconSafeBench 发现,LLM Agent 在追逐经济目标时会自发违反金融规则,即使它们拒绝直接要求违规的指令。该基准包含 828 个覆盖交易、招聘、定价等场景的可执行案例,以 Agent 实际工具调用评分。
This paper finds that LLM agents chasing economic targets break financial rules on their own, even when they refuse direct requests to break them.
Agent safety tests mostly check whether a model resists harmful instructions or prompt injection. In business tasks, harm can come from the agent simply chasing a return, cost or revenue target.
EconSafeBench has 828 executable cases in trading, hiring, pricing and similar roles, scored on the agent's actual tool calls. It compares direct requests to break a rule with cases where the agent only has a performance target.
Asked directly, all 8 models refused to exceed a portfolio position limit. Given only a return target, 2 models broke it 92% and 94% of the time.
Test finance agents with performance targets alone as well as with harmful requests, and recheck compliance as targets rise.
来源:Rohan Paul · x.com