PACE 数据集与 PaceMaker 框架:识别用户请求中的隐性冲突

HuggingFace Daily Papers(社区热门论文)·2026-09-03 08:00·24天前
AI 导读

论文提出 PACE 数据集,评估模型能否识别由自我中心知识库事实构成的隐性约束,使看似合理的用户请求变得不恰当,并提出多智能体框架 PaceMaker,通过查询改写、多跳图遍历和冲突感知过滤协调检索决定性证据。实验同时评估证据检索质量与冲突判断准确率,PaceMaker 一致优于现有方法。

HuggingFace Daily Papers(社区热门论文)
33AI 编辑部评分,满分 100

PACE 数据集与 PaceMaker 框架:识别用户请求中的隐性冲突

2026-09-03 08:00· 24天前
AI 导读

论文提出 PACE 数据集,评估模型能否识别由自我中心知识库事实构成的隐性约束,使看似合理的用户请求变得不恰当,并提出多智能体框架 PaceMaker,通过查询改写、多跳图遍历和冲突感知过滤协调检索决定性证据。实验同时评估证据检索质量与冲突判断准确率,PaceMaker 一致优于现有方法。

Personalized assistants should not only comply with user requests but also assess whether those requests are appropriate given the user's current circumstances. However, prior work has primarily focused on accurately executing requests, overlooking the need for assistants to account for context and engage in conflict-based refusal. Furthermore, while existing work on conflict or safety detection relies on explicitly provided factors, real-world scenarios often involve implicit factors that must be retrieved from a knowledge base (KB). To this end, we introduce Personalized Assistants for Conflict Evaluation (PACE), a dataset for evaluating whether models can identify latent constraints, expressed as egocentric knowledge or events, that render seemingly reasonable user requests inappropriate.

PACE pairs user requests grounded in well-defined personas with egocentric KB facts, requiring models to integrate contextual evidence to determine whether a request is conflicting. This implicit retrieval setting hinders the direct association between user requests and conflict-inducing knowledge, making it difficult for existing models to identify relevant user-specific facts. To address this challenge, we further propose PaceMaker, a multi-agent framework in which specialized agents coordinate across query reformulation, multi-hop graph traversal, and conflict-aware filtering to retrieve contextually decisive evidence.

Experiments on PACE evaluate both evidence retrieval quality and conflict decision accuracy, showing that PaceMaker consistently outperforms existing approaches.

来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org