Rohan Paul· @rohanpaul_ai · X·· 3 小时前AI 评分62
AI 导读
东京都立大学等机构的论文发现，基于厂商旧模型训练的 AI 文本检测器在模型代际更迭前可捕获 99% 以上的改写，更迭后仅捕获 3.8%。论文用 4,000 篇前 ChatGPT 时代摘要由 25 个 LLM 版本改写进行测试；引用案例中 Pangram 漏掉 79.8% 被 Meta Muse-Glimmer 改写的科学摘要，但对 5,000 篇人写摘要仅误报 1 篇，作者建议每次模型发布都重新验证检测器。
正文
This paper finds that AI-text detectors trained on a vendor's older models caught over 99% of rewrites before a generation change and only 3.8% after it.
Detectors that screen scientific papers for AI writing can stop working when a new LLM generation arrives, so anyone relying on them should re-test them with every model release.
Pangram, the AI-text detector, missed 79.8% of scientific abstracts rewritten by Meta's Muse-Glimmer, while flagging just 1 of 5,000 human abstracts. In a new paper from Tokyo Metropolitan University, reseaerchers find the share of AI-rewritten abstracts that Pangram misses depends strongly on the LLM version Shows that it caught 93.5% of GPT-5 rewrites but missed 79.8% from another new model. Its miss rate depended mostly on which model did the rewriting.在 X 查看被引用的帖子
来源：Rohan Paul · x.com