Dongxi 东锡 NLP· @dongxi_nlp · X·· 2026-08-11AI 评分33
AI 导读
新鲜数据是有限的。 下一个缩放定律将找出真正限制学习的因素。 论文： 《弥合计算最优与数据最优的预训练》
正文 · 原文
Fresh data is finite.
The next scaling law finds what truly limits learning.
Paper:
Bridging Compute- and Data-Optimal Pretraining
(1/N) 🧵 Chinchilla assumes you'll never run out of fresh data. That era is ending! Compute keeps growing exponentially, but high-quality tokens don't. So what's the exchange rate between extra compute and fresh, high-quality data? We propose Compute-Data (CD) scaling laws, which measure the exchange rate between extra compute and fresh data.在 X 查看被引用的帖子
来源：Dongxi 东锡 NLP · x.com