面壁智能发布UltraX框架，用于大规模预训练数据精炼。不同于端到端重写数据，UltraX通过结构化编辑函数（插入、删除、修改）精确提升训练数据质量。其LAM+DCR管道生成高质量监督信号，实现可扩展的精炼。在1B模型预训练实验中，UltraX在五个语料库上取得最佳平均性能。相关论文、代码及数据集（UltraX-Preview）已在arXiv、GitHub、Hugging Face和ModelScope开源。
🚀 A new way to refine LLM training data is here!
Scaling LLMs is no longer just about collecting more data — it’s about making every token count.
Introducing UltraX, our framework for large-scale pre-training data refinement. Instead of rewriting data end-to-end, UltraX uses structured editing functions to precisely improve training data quality.
What’s inside? 👇
✅ Function-calling refinement
Fine-grained editing with insertion, deletion, and modification.
✅ Reliable data transformation
LAM + DCR pipelines generate high-quality supervision for scalable refinement.
✅ Better data efficiency
Achieves the best average performance across five corpora in 1B-model pre-training experiments.
No gatekeeping — explore how better data can help build better models.
📄Paper: http://arxiv.org/abs/2607.08646
💻 GitHub: http://github.com/openbmb/UltraX
🤗 Hugging Face: http://huggingface.co/datasets/openbmb/UltraX-Preview
Modelscope: http://modelscope.cn/datasets/OpenBMB/UltraX-Preview
http://modelscope.cn/models/OpenBMB/UltraX-0.6B-Preview
来源：OpenBMB · x.com