百度发布 ERNIE 5.5 Flash Base 稀疏 MoE 基座语言模型并开放权重。模型总参数 280B、每 token 激活 14B，预训练 33.5T tokens，微调和推理上下文最长 1M-token，提供 FP8 与 BF16 权重。官方称其在同规模基座模型中于多个基准取得 SOTA，权重直接来自预训练、未做中后期训练，方便研究者在同一 checkpoint 上比较不同方法的效果。
ERNIE 5.5 Flash Base just dropped! Time to open that project folder. 🚀
Our efficient sparse MoE language model draws on data spanning multiple domains and languages, achieving SOTA on multiple benchmarks among similarly sized base models. Its architecture and training system were designed together for efficient training and inference, with advances in attention mechanisms, expert routing, and training objectives:
- 280B total parameters; 14B activated per token
- Pre-trained on 33.5T tokens
- Up to 1M-token context for fine-tuning and inference
- FP8 and BF16 weights
The weights come straight from pre-training, with no mid- or post-training. Compare methods from the same checkpoint, measure what each adds, and shape the model around your own tasks.
来源：Baidu Inc. · x.com