# StepAudio 3 Music 技术报告：支持显式音乐规划的长音频生成模型

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-09-11 08:00
- AIHOT 分数：47
- AIHOT 链接：https://aihot.news/items/cmu3rgl5r0cairoc15bqi4rhf
- 原文链接：https://arxiv.org/abs/2609.16034

## AI 摘要

StepAudio 3 Music 是一款支持显式音乐规划与开放域文本控制的大规模长音频生成模型，可生成最长 5 分 30 秒的歌曲、器乐、干声伴奏及翻唱。其 Tokenizer 以 50-Hz 单码本 65536 条目表示音频，配合 flow-matching diffusion Transformer 预测连续 VAE latent，解码为 48-kHz 音频。

## 正文

We introduce StepAudio 3 Music, a large-scale, long-form music generation model that supports explicit musical planning and open-domain text-controlled generation. The StepAudio Music Tokenizer represents audio as a 50-Hz stream from a 65536-entry single codebook, using semantically informed self-supervised and multi-task training to preserve musical structure and reconstruction-relevant information. A flow-matching diffusion Transformer (DiT) predicts continuous StepAudio VAE latents, which our VAE decoder converts into 48-kHz audio. This discrete-continuous design is guided by comparisons of single-codebook VQ, Semantic and Acoustic RVQ, and different DiT configurations. For explicit planning, a Mixture-of-Experts autoregressive model uses ABC notation to produce an intermediate arrangement plan (ABC-CoT) before predicting music tokens, making harmony, rhythm, and melodic structure part of the generation context. A progressive training curriculum and supervised fine-tuning support song and instrumental generation, accompaniment generation from dry vocals, and cover-song synthesis for up to 5 minutes and 30 seconds. With reinforcement learning via direct preference optimization (DPO), the final model achieves the highest AudioBox Content Enjoyment, Content Usefulness, and Production Quality scores and the highest MuQ-MuLan similarity among the evaluated systems, with competitive SongBench results. On the preliminary Artificial Analysis Music Arena Vocals leaderboard, it obtains a Quality Elo of 1105, behind only Suno V5.5 and Mureka and ahead of Suno V5, MiniMax models, and other systems. Audio demonstrations are available at https://stepaudiollm.github.io/step-audio-3-music.