# Fuse：面向 LLM 助手的可验证社会推理评测框架

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-09-15 08:00
- AIHOT 分数：39
- AIHOT 链接：https://aihot.news/items/cmu7c2smb0528rogrncbasdlr
- 原文链接：https://arxiv.org/abs/2609.17496

## AI 摘要

研究者提出多智能体模拟框架 Fuse，用于评测 LLM 助手在用户转述场景下的社会推理能力，通过隐藏动机的目标智能体与用户智能体交互构造可验证的 ground truth，并用 24k 条标注的人类研究验证模拟保真度。该框架测试了 12 个 LLM，发现用户转述会加剧社会推理难度、模型对用户偏见框架存在系统性敏感、且更长对话并不总能提升表现。Fuse 与含 21k 样本的数据集已开源。

## 正文

LLM assistants are widely used for daily social advice, yet evaluating their social reasoning in such consultation settings remains challenging since (i) it requires setups where the assistant learns about social situations from subjective user narratives, and (ii) social properties, such as others' intentions, typically lack verifiable ground truth. To address these challenges, we introduce Fuse, a multi-agent simulation framework for studying user-mediated social reasoning. In Fuse, a target agent with a hidden motive interacts with other agents including one representing the user, who then consults the evaluated assistant to infer the target's motive, providing verifiable ground truth by construction.

Simulation faithfulness is validated through a human study with 24k annotations. We apply Fuse to 12 LLMs and demonstrate its analytical utility by systematically isolating key factors, showing that (i) user mediation compounds the inherent difficulty of social reasoning; (ii) LLMs exhibit systematic sensitivity to biased user framing; (iii) models can require more details than humans need to reach a correct prediction; and (iv) longer conversations do not always improve performance despite providing opportunities for clarifying questions. We open-source Fuse and a dataset with 21k examples.