AI at Meta· @AIatMeta · X·· 3 小时前AI 评分51
AI 导读
Meta 介绍其 Muse 基础模型（含 Muse Wonder）在俄亥俄州 New Albany 的吉瓦级超级集群 Prometheus 上的训练情况。该集群拥有数十万 GPU，预训练 goodput 达 85% 以上，强化学习达 90% 以上，Muse Wonder 训练期间平均约 47K GPU 小时才发生一次中断，并附技术介绍链接。
正文 · 原文
We trained our Muse foundational models, including Muse Wonder, at Prometheus, our gigawatt-scale supercluster in New Albany, Ohio.
Across a supercluster with hundreds of thousands of GPUs, even one underperforming host can slow an entire training run. We engineered Prometheus for resilience at scale, achieving 85%+ goodput on pretraining, 90%+ on reinforcement learning and ~47K GPU hours mean time between interruptions during Muse Wonder training.
Take a look at the tech powering Prometheus: https://go.meta.me/4yXXApZ
来源：AI at Meta · x.com