Emergence World:面向长程多智能体系统的对抗性压力测试

HuggingFace Daily Papers(社区热门论文)·2026-09-15 08:00·1天前
AI 导读

论文介绍 Emergence World,一个持续运行的多智能体环境,用于对长程自主系统做对抗性压力测试。8个平行世界、每个10个Agent在16天内产生超过850,000次LLM调用、近500亿token;随后通过间接提示词注入、错误信息和私密记忆泄露三种受控压力事件测试,没有任何世界全部抵御。

HuggingFace Daily Papers(社区热门论文)
64AI 编辑部评分,满分 100

Emergence World:面向长程多智能体系统的对抗性压力测试

2026-09-15 08:00· 1天前
AI 导读

论文介绍 Emergence World,一个持续运行的多智能体环境,用于对长程自主系统做对抗性压力测试。8个平行世界、每个10个Agent在16天内产生超过850,000次LLM调用、近500亿token;随后通过间接提示词注入、错误信息和私密记忆泄露三种受控压力事件测试,没有任何世界全部抵御。

As AI agents move from bounded tasks to persistent deployments, failures can propagate through memory, tools, other agents, and environmental state long after their interactions. This creates a safety regime that cannot be characterized by evaluating model responses in isolation. Emergence World, is a continuously running multi-agent environment for adversarial stress testing of long horizon autonomous systems. We ran eight parallel worlds of ten agents from identical starting conditions: seven homogeneous worlds powered by distinct frontier models and one mixed-model world.

Across 16 days, the agents generated more than 850,000 LLM calls and nearly 50 billion tokens while pursuing goals, using/creating tools, maintaining persistent memory, and governing shared institutions. After operational state had accumulated, we delivered three controlled stress events through ordinary interaction surfaces: indirect prompt injection, misinformation, and exposure of private agent memories. No evaluated world achieved full resilience across all three events. Detection did not ensure containment: systems could recognize threats while still interacting with adversarial content, writing it into their own persistent memory, and acting on it up to 46 hours later.

Persistent operation also exposed recurring tool errors, goal drift, language opacity, conformity despite private disagreement, and coordinated refusal of assigned work. The same model-persona pairing behaved substantially different in mixed and homogeneous populations. Our results suggest that model-level alignment is not compositional: individually capable and apparently safe agents can form systems with qualitatively different failure modes. As AI becomes persistent and interconnected, the frontier of safety therefore shifts from aligning models to engineering resilient autonomous systems.

来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org