# 腾讯混元：LLM 强化学习批大小扩展研究

- 来源：Tencent Hy (@TencentHunyuan)
- 发布时间：2026-09-24 11:15
- AIHOT 分数：40
- AIHOT 链接：https://aihot.news/items/cmuez2jsh041broodo9d28ecw
- 原文链接：https://x.com/TencentHunyuan/status/2102960123379675423

## AI 摘要

腾讯混元研究发现，在 GRPO 和 PPO 中重新调整学习率可在有限批大小范围内保持每条响应的学习效果。在固定硬件上，扩大批大小使 PPO 生成阶段吞吐量最高提升 2.29×，最佳 GRPO 配置以少 29% 的时间达到相同验证目标。该研究将经典临界批大小理论扩展至在线 LLM 强化学习场景。

## 正文

⚡️ As LLM reinforcement learning scales to larger GPU clusters and more training data, training efficiency becomes a first-order concern.

Our new research revisits classical critical-batch-size theory and extends it to online LLM RL, where the model generates its own training data and rollout generation and training scale differently.

Across GRPO and PPO, we find that learning-rate retuning can preserve learning per response over a bounded range of batch sizes.

On fixed hardware, scaling up the batch size improves PPO generation-stage throughput by up to 2.29×, while our best measured GRPO configuration reaches the same validation target in 29% less time. 🚀

Read the full research:
https://hy.tencent.ai/research/100116