跳到正文
Rohan Paul· @rohanpaul_ai · X·· 2 小时前AI 评分56
AI 导读

Meta 等机构发布 ProPerMem 基准,用约 618K token 的全年个人记忆和人工验证的触发窗口评测记忆驱动的主动助手。所有基线最高仅 15%,GPT-5.3-Chat 在窗口内给定证据时达 96%,但自主定时降到 38%、加检索降到 15%;兼顾内容与时机并减少触发可把准确率从 10% 以下提升到 42.4%。

正文 · 原文

New Meta paper shows that deciding on its own when to interrupt, not just what to say, is where proactive assistants fail.

Proactive assistants, like AI glasses, get no query. They must decide when to speak and what to say by finding the justifying fact in a year of personal memory, not handed to them.

They built ProPerMem: egocentric video paired with a distractor-rich, year-long memory averaging 618K tokens, with human-verified trigger windows. A clip counts only if the assistant speaks just when due inside the window with grounded content.

Every baseline scored at most 15%, even frontier models with retrieval-augmented memory. With oracle evidence in the window GPT-5.3-Chat reached 96%, but choosing its own timing dropped it to 38%, and using retrieval dropped it to 15%.

For proactive assistants, get both the content and the timing right and fire sparingly, which lifted accuracy from below 10% to 42.4%.

来源:Rohan Paul · x.com