Mira Murati· @miramurati · X·· 2 小时前AI 评分54
AI 导读
Mira Murati 发布 Connectionism 博客，详解 Inkling 2 Small 在 NVIDIA GB300 上以原生 4-bit 前向传播从预训练第一步到最终 RL rollout 的完整训练过程。这是全栈多项改进之一，将预训练时间从数周缩短到 68 小时；量化训练更快，但需解决量化误差干扰学习的问题。
正文 · 原文
Today on Connectionism: how we trained Inkling 2 Small with a native 4-bit forward pass on @nvidia GB300s, from the first pretraining step through the final RL rollout, one of the many improvements across the stack that took pretraining from weeks to 68 hours.
在 X 查看被引用的帖子Training with fewer bits is faster, but quantization error can interfere with learning.
Today on Connectionism, we detail how we trained Inkling 2 Small with a native 4-bit forward pass from the first step of pretraining through the final RL rollout.
thinkingmachines.ai/blog/end-…
来源：Mira Murati · x.com