Ant Ling· @AntLingAGI · X·· 2026-06-24AI 评分41
AI 导读
蚂蚁百灵发表UFP4论文，提出均匀网格FP4训练配方。在Dense 1.5B、MoE 7.9B和MoE 124B长程预训练中，该配方相比强E2M1基线实现了更低的BF16相对损失退化。论文指出，配合细粒度缩放和RHT后，FP4训练的瓶颈从动态范围转向局部分辨率，E1M2/INT4格式能更好利用RHT改进的桶分配，而E2M1可能使RHT有害。论文地址：https://arxiv.org/abs/2606.20381
正文 · 原文
Great breakdown from Qian.
In our recent UFP4 paper, we show that a uniform-grid FP4 recipe achieves lower BF16-relative loss degradation than strong E2M1 baselines across Dense 1.5B, MoE 7.9B, and MoE 124B long-run pretraining.
Full paper: https://arxiv.org/abs/2606.20381
Should FP4 training still default to E2M1?🤔 With fine-grained scaling + RHT, the bottleneck may shift from dynamic range to local resolution. E1M2/INT4 better turns RHT’s improved bucket use into quantization quality, while E2M1 can make RHT harmful. 🚀 https://arxiv.org/abs/2606.20381在 X 查看被引用的帖子
来源：Ant Ling · x.com