来自 @skyfallai 的新基准 Morpheus 测试 LLM 的持续学习能力。与 Atari、OpenAI Gym 等每几分钟重置的游戏式环境不同，Morpheus 模拟永不重置的真实企业工作流，涉及资源分配与调度。测试发现，前沿模型在传统基准上的稳定分数掩盖了真相：当规则、奖励和约束发生漂移时，模型并未真正适应。Task 1（资源分配）表现稳定，但 Task 2（条件变化下的调度）变得波动甚至崩溃。结论是 LLM 并非持续学习者。
New Continual Learning benchmark, "Morpheus" from
@skyfallai
Shows an important area LLMs were never properly tested on.
Most AI benchmarks reward strong average scores inside environments whose rules barely change. But Continual learning demands more because deployed agents must detect shifts and revise behavior continuously.
Morpheus creates those shifts inside enterprise workflows involving resource allocation and scheduling under drift.
Each configuration change tests whether performance reflects fresh learning or recycled pretraining habits.
Morpheus asks the harder question: does the model update when rules, rewards, and constraints begin to drift?
Morpheus therefore separates continual learning from broad pretraining coverage, which conventional averages often confuse.
Overall, the benchmark finds that stable benchmark scores of frontier models concealed a harder truth: top frontier models were not truly adapting.
Check out their below graph.
Task 1 tests how well the system allocates resources, while Task 2 tests whether it can keep scheduling correctly as conditions change.
Task 1 stays fairly stable; Task 2 becomes volatile and sometimes collapses.
Today we present Morpheus, a persistent enterprise simulation platform designed to make Continual Learning a reality. Morpheus is the world’s first real world Reinforcement Learning environment. Every Reinforcement Learning environment operates in the game world. Benchmarks like Atari, OpenAI Gym, MuJoCo, and Procgen are all small, game-like worlds that reset every few minutes. But the real world never resets. A business keeps running and evolving everyday. We tested how frontier LLMs would perform in realistic and dynamic business environments 🧬on Morpheus. The main conclusion was that LLMs are not continual learners. 🧵Here’s how we did it and what we learned:在 X 查看被引用的帖子
来源：Rohan Paul · x.com