字节 Seed 提出 Recycler：通过隐藏状态回收实现无额外成本的潜在推理

Recycler: Cost-Free Latent Reasoning via Hidden State Recycling

ByteDance Seed Abstract

Standard autoregressive decoding links successive steps only through sampled discrete tokens, discarding the continuous semantics encoded in previous hidden states. In contrast, recurrent multi-token prediction (MTP) heads propagate both token embeddings and hidden states across steps. Remarkably, an MTP head with only ∼ 2% of the backbone parameters achieves a loss comparable to that of a standalone model ∼ 25% its size, underscoring the value of continuous hidden state feedback. Motivated by this observation, we introduce Recycler, a simple and effective architecture that feeds both embedding and the previous token’s hidden state through a lightweight adapter into the next step, preserving richer cross-step semantics with negligible decoding overhead. We develop a complete training and inference pipeline covering pretraining, SFT, continued pretraining and RL. Experiments demonstrate strong parameter efficiency, particularly in reasoning and mathematics. Although earlier integration yields larger gains, Recycler remains effective when introduced at later stages, outperforming models with 2–3× more parameters.

Date: October 25, 2026

1 Introduction

Modern language models [3–6, 18–20, 42] generate text autoregressively. Each decoding step produces a top-layer hidden state ht processed by the full Transformer [51] stack, yet only a sampled discrete token

xt+1 is directly passed to the next step. This lossy discretization discards rich information encoded in ht.Preserving this state suggests a complementary computational scaling axis that reuses computation across decoding steps, rather than relying only on more parameters or training tokens. An empirical clue supporting this direction comes from recurrent multi-token prediction (MTP) [10, 22]. A recurrent MTP module resembles a standard autoregressive language model, but additionally receives the backbone’s previous-position hidden state alongside the token embedding. With only about 2% additional parameters, it achieves a loss comparable to that of a standalone model one quarter the size of the backbone. This observation motivates our hypothesis that the backbone itself can benefit from recycling its previous output hidden state. Building on this observation, we introduce Recycler , a simple architecture that augments the current token embedding with the previous token’s top-layer hidden state through a lightweight adapter. Whereas standard autoregressive decoding compresses the latest top-layer output into a sampled discrete token before the next step, Recycler retains this token path while adding a parallel continuous path that directly carries the full-stack hidden state forward, as illustrated in Figure 2. This recycle branch bypasses the discrete bottleneck and creates a continuous recurrent computation whose effective depth grows with the generated sequence. Notably, since the previous hidden state is already available and the adapter is lightweight, Recycler has approximately the same autoregressive decoding cost as a standard Transformer. 12% 10% 100% Parameter Scale (% of Full Model)

Training Loss

• Extra Hidden Full-Model Scaling Law MTP Head Observed MTP Head Expected 1× 2.4× Model size Evaluation performance 58% smaller model size Recycler (1×) Baseline Baseline (2.4×) 81% abilities gains Small Models, Outsized Capability (a) Hidden-state leverage (b) Model-size efficiency

Figure 1 Comparison of observed and expected MTP head loss.

Although Recycler is straightforward during au-toregressive inference, its sequential dependency prevents exact one-pass parallel training. We ad-dress this challenge with a multi-pass approxima-tion, where each pass uses the recycled hidden states from the preceding pass while retaining par-allelism across sequence positions. We empirically show that the resulting model generalizes to autore-gressive recurrence while maintaining a decoding cost approximately equal to that of a standard Transformer. Beyond the architecture itself, we develop an end-to-end framework that supports Recycler through-out model training and deployment. The frame-work covers pretraining, continued pretraining, supervised fine-tuning, reinforcement learning. Experiments validate Recycler throughout the entire pipeline. It consistently improves pretraining and downstream per-formance, with particularly pronounced gains on mathematics and reasoning tasks. These benefits persist when Recycler is introduced only during continued pretraining or supervised fine-tuning, and are further strengthened through reinforcement learning. Finally, we resolve Recycler’s parallel-verification challenge in MTP-based speculative decoding, enabling further inference acceleration.

2 Approach

2.1 Architecture

As shown in Figure 2 (a), during standard autoregressive decoding, token xt is mapped to its embedding and fed into the Transformer backbone T to produce the top-layer hidden state ht. The hidden state is then projected by the unembedding layer, from which the next token xt+1 is sampled and re-embedded as the input to the next decoding step. Let Φ denote this stochastic discrete feedback operation, consisting of unembedding, softmax normalization, token sampling, and re-embedding. The relationship between two consecutive hidden states can then be written compactly as

ht+1 = T (Φ( ht)) . (1) The discrete feedback operator Φ forms an information bottleneck: it maps a continuous hidden state carrying rich contextual information to a discrete token ID before reconstructing an input embedding. Our key observation is that this bottleneck is not necessary for propagating information across autoregressive decoding steps, because the previous hidden state has already been computed and remains available. Recycling that state provides a richer input to the next step without requiring an additional backbone forward pass, incurring only the negligible cost of a lightweight adapter. As shown in Figure 2(b), Recycler introduces a continuous feedback pathway alongside the lossy discrete operator Φ. This pathway transforms the previous hidden state with a lightweight adapter A composed of RMS normalization and a learnable linear projection. It therefore preserves substantially more of the information in the hidden state than the discrete feedback pathway. We define the ⊕ operator in Figure 2(b) as

A(h) := Linear(RMSNorm( h)) ,x ⊕ h := Emb( x) + A(h). (2) The relationship between consecutive hidden states in Recycler then becomes

ht+1 = T (xt+1 ⊕ ht) = T (Φ( ht) + A(ht)) . (3) 2x1 x2 x3 x4 x5 x6

Model

h1h2h3h4h5h6 x1x2x3x4x5x6

Model

h1h2h3h4h5h6 h1h2h3h4h5x1x2x3x4x5x6

Model

h1h2h3h4h5h6 x1x2x3x4x5x6

Model

h1h2h3h4h5h6 x1x2x3x4x5x6

Model

h1h2h3h4h5h6 Loss Loss Loss

(a) Standard Decoding

(b) Recycler Decoding (c) Recycler Training and Prefill

xiToken hiHidden State Recycle Step Fuse Operator Unembed Hidden Recycle

Figure 2 (a) Decoding process of the standard model, (b) decoding process of the Recycler model, and (c) training and prefill procedure of the Recycler model.

Unlike the standard model, whose cross-step communication is restricted to the discrete token pathway, Recycler therefore propagates continuous hidden-state information recurrently across decoding steps.

2.2 Recycler Decoding, Training and Prefill

We next describe how the recurrent Recycler architecture is executed during decoding and approximated efficiently during training and prompt prefilling.

2.2.1 Decoding

At generation time, Eq. 3 is applied recursively after each sampled token: the hidden state from the current step is adapted and combined with the next token embedding to produce the next hidden state. We write

h(kdec )

t

, where the superscript kdec denotes the effective decoding recycle depth . At depth kdec , the current hidden state can incorporate continuous information propagated through the most recent kdec positions. This recurrence is not truncated during autoregressive decoding, so kdec grows with the generated sequence length. This creates an effectively unbounded-depth information pathway without requiring additional backbone passes per generated token.

2.2.2 Training

Exact training under Eq. 3 would introduce a sequential hidden-state dependency across token positions, preventing the standard token-parallel training of Transformers. We instead truncate the recycle depth to a fixed training depth ktrain and approximate the recurrence using ktrain sequence-level forward passes: 3h(1)

t

= T (Emb( xt)) ,

h(i)

t

= T(︂ Emb( xt) + A(︂ h(i−1)

t−1

)︂)︂ , i = 2 , . . . , k train ,

L(i) =∑︂

t

NTP (︂ h(i)

t

, x t+1 )︂ , L =

ktrain

∑︂

i=1

L(i).

(4) Here, h(i)

t

is the hidden state at position t and recycle depth i, and NTP denotes the next-token cross-entropy loss. Given all states from pass i − 1, all token positions in pass i can be computed in parallel. Thus, the method preserves token-level parallelism while allowing h(ktrain )

t

to incorporate continuous information propagated through the most recent ktrain positions. When ktrain ≥ t (assuming one-indexed positions), it recovers the exact recurrent state in Eq. 3. We use three implementation choices to stabilize this approximation: • We zero-initialize the linear projection in A, making A(·) = 0 at initialization and recovering the standard Transformer computation. • We apply the NTP loss to all ktrain passes, which aligns the representation spaces of h(1)

t

, . . . , h(ktrain )

t

and reduces the train–decode distribution gap. • We tie the input embedding and output projection weights to better align their representation spaces, reducing the mismatch between successive recycle depths.

2.2.3 Prefill

During prompt prefilling, reproducing Eq. 3 token by token would match autoregressive decoding exactly, but would eliminate the sequence parallelism of standard Transformer prefilling. We therefore use parallel forward passes over the full prompt and consider two operating points: • Prefill kprefill = 2 . We apply two parallel passes over the prompt and initialize decoding with the final prompt-token state from the second pass. This doubles prefill computation relative to a standard model. • Prefill kprefill = 1 . We perform one standard parallel pass over the prompt and initialize decoding with its final token state. This matches the prefill computation of a standard model. Both strategies subsequently use the same recurrent decoding procedure in which the effective recycle depth grows with the generated sequence length.

2.3 Recycler Across the LLM Lifecycle

Building on the decoding recurrence, parallel training approximation, and prefill strategies introduced above, we now describe how Recycler is integrated into each stage of the LLM lifecycle. Specifically, we detail its stage-specific implementation for pretraining, supervised fine-tuning, reinforcement learning.

2.3.1 Pretraining

We apply the parallel approximation from Section 2.2.2 with ktrain = 2 to the full pretraining sequence and supervise both recycle depths with the NTP objective. Because pretraining examples are treated as continuous token sequences, there is no separate prompt-prefill region during optimization.

Extrapolation Analysis.

Considering that Recycler is trained using an approximate algorithm, its inference procedure is not fully aligned with the training process. This discrepancy raises a natural concern that the approximation used during training might not generalize well to the decoding stage. We first conduct a validation experiment to examine this: we train a tiny Recycler model (MoE architecture, 151M activated parameters, 1B total parameters) with ktrain = 2 for 200B tokens. During inference, we vary kdec from 1 to 32 to assess whether performance remains stable beyond the training depth. The results are shown in Figure 3: 40 10 20 30

k 2.36 2.37 2.38 2.39 Eval Loss Recycler Baseline 010 20 30 k 0.34 0.35 0.36 0.37 MMLU Recycler Baseline 010 20 30 k 0.40 0.42 0.44 0.46 C-Eval Recycler Baseline

Figure 3 Extrapolation analysis under increasing decoding recycle depth kdec . The model is trained with ktrain = 2

and evaluated with kdec ∈ [1 , 32] on an i.i.d. evaluation set and the downstream benchmarks MMLU and C-Eval.

• When kdec = 1 , the model already achieves substantial gains over the baseline, indicating that even without using the hidden state of the previous token, this training practice of incorporating future token loss into consideration still yields performance benefits. • When kdec > 1, the Recycler model achieves further gains and remains stable across subsequent extrapolation steps. This indicates that the training-inference inconsistency in Recycler can be naturally mitigated through extrapolative generalization. The training cost of Recycler is 2 × that of the standard model. Through the extrapolation experiments above, we have preliminarily demonstrated the effectiveness of our approximate training method. However, as the model size and data volume grow, extrapolation issues may arise and lead to a higher evaluation loss, as illustrated in Figure 4. To address this, we perform a third forward pass on a subset of steps during training. Compared to Recycler training with two forward passes, this approach incurs only an additional 3.3% in training cost while fully resolving the extrapolation problem of Recycler. 0 20 40 60

k 1.525 1.530 1.535 1.540 1.545 Evaluation loss Evaluation Loss under Increasing k Two forward passes Partial three forward passes

Figure 4 Evaluation loss under increasing kdec for the original ktrain = 2 recipe and a partial ktrain = 3 recipe with additional 3.3% training cost.

2.3.2 Supervised Fine-Tuning

SFT mainly requires aligning response training with the prompt-prefill strategy used at inference. Response tokens follow the parallel training approximation in Section 2.2.2 with ktrain = 2 , and the NTP loss is masked to the response region. For kprefill = 2 , both prompt passes retain the recycled-hidden pathway. For kprefill = 1 ,the second pass is still required to train response tokens, but the recycled contribution A(h(1)

t−1

) is set to zero at prompt positions to match single-pass prefilling. 5Generally speaking, models trained and inferred with prefill kprefill = 2 exhibit markedly better performance on downstream metrics than those using prefill kprefill = 1 .

2.3.3 Reinforcement Learning

For RL, the main adaptation lies in the rollout process: the prompt follows the selected prefill strategy, whereas the response is generated through recurrent Recycler decoding. To further stabilize training, we adopt GSPO [65] as our reinforcement learning algorithm:

JGSPO (θ) = Ex∼D ,{yi}Gi=1 ∼πθold (·| x)[︄ 1

G

G

∑︂

i=1

min (︂ si(θ)ˆ︁ Ai, clip ( si(θ), 1 − εlow , 1 + εhigh )ˆ︁ Ai)︂]︄ , (5) where G is the number of generated responses to each query x, and the group-based advantage estimation is: ˆ︁

Ai =

r (x, y i) − mean (︂ {r (x, y i)}Gi=1 )︂

std (︂ {r (x, y i)}Gi=1 )︂

, (6) and the importance ratio si(θ) is defined on sequence level, which addresses the high-variance issue of the importance ratio in GRPO:

si(θ) = (︃ πθ (yi | x)

πθold (yi | x))︃

1 |yi|

= exp

⎛⎝ 1

|yi|

|yi|

∑︂

t=1

log πθ (yi,t | x, y i,<t )

πθold (yi,t | x, y i,<t )

⎞⎠ . (7) As we demonstrate later, under an equal RL training budget, the Recycler architecture yields greater score gains than the standard model.

3 Experiments

We evaluate Recycler throughout the model-development pipeline, covering pre-training (PT), continued pre-training (CT), supervised fine-tuning (SFT), reinforcement learning (RL). All Recycler models in this section are trained with a recycle depth of k = 2 . At inference time, the recurrent hidden-state pathway is rolled out autoregressively for as many decoding steps as required by each evaluation, thereby extrapolating beyond k = 2 . Prompt prefill is treated separately. For selected evaluations, we report both prefill ×1 which recycle step k = 1 and prefill ×2 which recycle step k = 2 : the former performs one parallel forward pass over the prompt, whereas the latter performs two and initializes decoding from the final hidden state of the second pass. Both modes subsequently use the same recurrent Recycler decoding process. For comparisons designed to be controlled, the baseline and Recycler share the same backbone, data mixture, optimization recipe, and training budget, with the Recycler pathway as the intended architectural difference. Unless otherwise specified, models evaluated after CT use the CT-introduced configuration, in which Recycler is activated during continued pre-training and retained through SFT and RL. Section 3.2 examines why this configuration offers a favorable balance between cumulative training cost and final performance.

3.1 Pre-Training

Setup. We validate Recycler on two independently trained model versions, denoted model α and model β .

model α provides a controlled comparison between Recycler and its baseline, allowing us to examine both optimization behavior and the evolution of downstream capabilities. model β provides a complementary scaling comparison. 6Token Efficiency. As shown in Figure 5, Recycler exhibits markedly lower training and validation loss curve than the baseline. At the end of the training, Recycler achieves 0.02 gains in training loss by approximately, On the held-out validation set, the first forward ( L(1) ) already achieves a reduction of approximately 0.013 ,while the recycled second forward ( L(2) ) enlarges the reduction to approximately 0.033 . This indicates that Recycler converges faster and achieves greater token efficiency. On the other hand, this also indicates that, when the training cost of the two forward passes is not taken into account, Recycler can achieve better performance with the same number of training tokens. 0% 25% 50% 75% 100%

Training Progress 1.8 1.9 2.0 2.1 2.2 Training Loss Baseline Recycler (1) Recycler (2) 0% 25% 50% 75% 100% Training Progress 1.9 2.0 2.1 Validation Loss Baseline Recycler (1) Recycler (2)

Figure 5 model α training and validation loss for Baseline and Recycler in the pre-training stage. The L(1) and L(2)

indicate the loss curves with one and two forward passes, respectively.

Gains across Benchmark Sources. We evaluate intermediate model α checkpoints on 14 datasets cover-ing knowledge and comprehensive understanding (MMLU [28], MMLU-Pro [54], C-Eval [30], GPQA [45], TriviaQA [34], SimpleQA [56], and ChineseSimpleQA [27]), logical reasoning (AGI-Eval [66], DROP [13], ARC-AGI [9], and BBH [49]), code generation (HumanEval [7] and MBPP+ [1, 38]), and mathematics (MATH [29]). At the end of the training, Recycler model α outperforms the baseline on all 14 datasets. The downstream average rises from 42 .97 to 46 .21 , an absolute gain of 3.24 . The largest gains occur on DROP ( +6 .31

points), BBH ( +5 .53 ), AGI-Eval ( +5 .35 ), MBPP+ ( +4 .76 ), and MMLU-Pro ( +4 .40 ), while knowledge-heavy benchmarks show smaller but still positive improvements. This source-level consistency indicates that the loss reduction in Figure 5 transfers broadly to downstream capabilities, with particularly strong effects on reasoning and structured generation. To further validate the effectiveness of Recycler, we additionally evaluate recycle step k = 1 . As shown in Figure 6, even when the prompt is processed with only a single forward pass—the same prefill procedure as standard autoregressive inference—Recycler yields clear improvements over the baseline on most evaluation benchmarks.

Comparison with a 2.4× Larger Baseline. We next evaluate model β to test whether Recycler’s additional latent computation can substitute for model scaling. Figure 7 compares three systems: the model β baseline, a baseline with 2.4× the model size (Baseline ×2.4), and the same-size Recycler. Recycler retains the same token-by-token decoding cost and KV-cache footprint as its same-size baseline, whereas Baseline ×2.4 increases the cost of every decoding step with model size. The model β results reproduce the broad source-level gains observed with model α. Over the available overlap, Recycler improves substantially over the same-size baseline across the benchmark suite. More importantly, at the end of the displayed training window, Recycler reaches a downstream average of 64 .76 , compared with

66 .43 for Baseline ×2.4—a gap of only 1.68 despite the 2.4× difference in model size. Recycler is close to the larger baseline on many individual sources and remains competitive across knowledge, reasoning, mathematics, and code benchmarks. The same-size baseline has a shorter available pre-training trajectory, ending near 50%

of the shared display range; we therefore show it without extrapolation and do not treat the three endpoints 70% 25% 50% 75% 100%

0.55 0.60 0.65 0.70 0.75

MMLU

0% 25% 50% 75% 100% 0.25 0.30 0.35 0.40 0.45

MMLU-Pro

0% 25% 50% 75% 100% 0.60 0.65 0.70 0.75 0.80

C-Eval

0% 25% 50% 75% 100% 0.40 0.45 0.50 0.55 0.60 0.65

AGI-Eval

0% 25% 50% 75% 100% 0.30 0.35 0.40 0.45 0.50 0.55 0.60

DROP

0% 25% 50% 75% 100% 0.02 0.03 0.04 0.05 0.06

ARC-AGI

0% 25% 50% 75% 100% 0.21 0.22 0.23 0.24 0.25 0.26 0.27 0.28 0.29

GPQA

0% 25% 50% 75% 100% 0.40 0.45 0.50 0.55 0.60 0.65 0.70

BBH

0% 25% 50% 75% 100% 0.15 0.20 0.25 0.30 0.35 0.40

HumanEval

0% 25% 50% 75% 100% 0.20 0.25 0.30 0.35 0.40 0.45

MATH

0% 25% 50% 75% 100% Training Progress 0.30 0.35 0.40 0.45 0.50 0.55

MBPP+

0% 25% 50% 75% 100% Training Progress 0.35 0.40 0.45 0.50 0.55 0.60

TriviaQA

0% 25% 50% 75% 100% Training Progress 0.015 0.020 0.025 0.030 0.035 0.040 0.045 0.050 0.055

SimpleQA

0% 25% 50% 75% 100% Training Progress 0.15 0.20 0.25 0.30 0.35

ChineseSimpleQA

0% 25% 50% 75% 100% Training Progress 0.275 0.300 0.325 0.350 0.375 0.400 0.425 0.450 0.475

Downstream Average

Benchmark Score

Baseline Recycler ( k = 2) Recycler ( k = 1) Figure 6 model α benchmark trajectories over normalized pre-training progress. Markers denote observed checkpoints; no score smoothing, resampling, or imputation is applied. The bottom-right panel is the equal-weight average of the 14 datasets, computed at checkpoints shared by all datasets within each evaluation path. Recycler ( k = 1 ) is shown only over its available early-training range.

as an equal-training-budget comparison. The central comparison is instead computational: Recycler nearly matches the substantially larger model while preserving the decoding cost of the smaller model.

Summary. The results consistently validate the effectiveness of the Recycler mechanism. Recycling hidden states provides an additional latent-computation step that lowers both training and validation loss, and the resulting gains transfer across all 14 downstream benchmark sources, with particularly strong improvements on reasoning and structured-generation tasks. Moreover, Recycler approaches the performance of a 2.4× larger baseline while retaining the smaller model’s autoregressive decoding cost and KV-cache footprint. These findings show that Recycler uses additional latent computation to improve optimization and downstream capability without requiring a proportional increase in model size or token-by-token decoding cost. 80.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00

0.65 0.70 0.75 0.80 0.85 0.90 MMLU 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00 0.35 0.40 0.45 0.50 0.55 0.60 0.65 MMLU-Pro 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00 0.70 0.75 0.80 0.85 0.90 0.95 C-Eval 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00 0.50 0.55 0.60 0.65 0.70 0.75 AGI-Eval 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00 0.40 0.50 0.60 0.70 0.80 DROP 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00 0.025 0.050 0.075 0.100 0.125 0.150 0.175 ARC-AGI 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00 0.20 0.25 0.30 0.35 0.40 0.45 GPQA 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 BBH 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 HumanEval 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00 0.30 0.40 0.50 0.60 0.70 MATH 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00 Token Consume Ratio 0.45 0.50 0.55 0.60 0.65 0.70 0.75 MBPP+ 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00 Token Consume Ratio 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 TriviaQA 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00 Token Consume Ratio 0.025 0.050 0.075 0.100 0.125 0.150 0.175 0.200 SimpleQA 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00 Token Consume Ratio 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 ChineseSimpleQA 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00 Token Consume Ratio 0.40 0.45 0.50 0.55 0.60 0.65 0.70 Downstream Average Benchmark Score Baseline (1×) Baseline (2.4×) Recycler (1×)

Figure 7 model β benchmark trajectories for the same-size Baseline, Baseline X2.4, and Recycler over a common normalized training-progress reference. Markers are the recorded checkpoints; no smoothing, interpolation, or score-based filtering is applied. The bottom-right panel is the equal-weight average over the 14 benchmarks. The shorter same-size baseline trajectory is shown only over its available pre-training coverage and is not extrapolated.

3.2 Continued Pre-Training and Supervised Fine-Tuning

3.2.1 Training Efficiency

A central design question is when to introduce Recycler in a multi-stage training pipeline. We therefore compare three routes that activate Recycler at pre-training (PT), continued pre-training (CT), or supervised fine-tuning (SFT), respectively. Figure 8 summarizes the resulting cost–performance trade-off. Because the PT/CT and SFT results are reported on different benchmark suites, we restrict the analysis to the paired Baseline–Recycler comparison within each suite rather than comparing absolute scores across suites. The paired comparisons exhibit a consistent pattern: enabling Recycler improves benchmark performance regardless of whether it is introduced at PT, CT, or SFT. This indicates that the mechanism does not depend on being present from the beginning of training; it can also be integrated into an existing model at a later stage and still yield a stable improvement over the corresponding baseline. The introduction stage nevertheless has a substantial effect on training efficiency. Activating Recycler from PT changes the training paradigm for the entire pre-training trajectory. Although this route provides the longest exposure to recycled hidden states and achieves a strong final result, the additional computation is incurred throughout the most expensive stage of model development, making its cumulative training overhead prohibitively high. Introducing Recycler only at SFT lies at the opposite extreme: it requires little additional compute and still improves the paired baseline, but leaves a comparatively short adaptation window for the new recurrent information path. Introducing Recycler at CT provides the most favorable compromise. It avoids retrofitting the additional training computation across the full PT budget, while retaining a sufficiently long continuation phase for the model to adapt to the modified training objective, fully optimize the recycled-hidden pathway, and converge before downstream post-training. The CT-introduced route therefore preserves a clear performance benefit without incurring the dominant cost of PT-time adoption. Based on this balance between effectiveness, 91.0× 1.2× 1.4× 1.6× 1.8× 2.0×

Estimated cumulative training FLOPs (PT-B = 1×) 26 28 30 32 34 36 38 Benchmark score Δscore +1.87 Δcost +100.50% Δscore +0.60 Δcost +13.54% Δscore +1.14 Δcost +0.19% PT-B PT-R CT-B PT-B + CT-R SFT-B PT-B + CT-B + SFT-R Baseline Recycler

Figure 8 Benchmark score versus estimated cumulative training compute for introducing Recycler at different stages. Each arrow connects a stage-matched baseline to the corresponding Recycler route within the same benchmark suite. Across PT-, CT-, and SFT-introduced settings, Recycler consistently moves the paired result toward higher benchmark performance. The PT/CT and SFT suites are shown separately because their absolute scores are not directly comparable.

cumulative training compute, and optimization horizon, we use the CT-introduced configuration in the subsequent SFT and RL experiments.

3.2.2 Supervised Fine-Tuning Evaluation

We further perform supervised fine-tuning (SFT) using the CT-stage checkpoints described above and summarize the results on 22 benchmarks in Table 1. All Recycler SFT variants are initialized from the CT-trained Recycler checkpoint. We evaluate two matched SFT–inference configurations in which the recycle depth is held consistent across the two stages: prefill recycle step k = 1 uses k = 1 during both SFT and inference, whereas prefill recycle step k = 2 uses k = 2 during both stages. The auxiliary k = 2 -trained/ k = 1 -inference result is included only as a reference point, evaluating the same k = 2 -trained checkpoint with a single prefill recycle step at inference. The 22 benchmarks cover mathematics (AIME24 [41], AIME25 [41], OlympiadBench [26], OmniMath [16], HARP [61]), reasoning (ARC-AGI [9], KORBench [40], ProcBench [15], ZebraLogicBench [37]), instruction following (IFEval [67], COLLIE [60]), coding (LiveCodeBench [32], SWE Verified [33]), and comprehensive evaluations (GPQA-Diamond [45], SuperGPQA [39], HLE [44], MMLU [28], MMMLU [43], MMLU-Pro [54], ArenaHard [36], SimpleQA [56], ChineseSimpleQA [27]).

Results. Recycler improves the aggregate SFT performance under both prefill settings, with prefill recycle step k = 2 providing the strongest endpoint. Relative to the baseline, its Avg. score increases from 62.9 to 66.7 ( +3 .8 points). It outperforms the baseline on 19 of the 22 benchmarks, and the complete collection shows gains in all seven ability groups. The largest ability-level improvements are in instruction following ( +9 .3)and reasoning ( +7 .3). The per-benchmark results reinforce this pattern. With prefill recycle step k = 2 , Recycler improves ProcBench by 8.2 points, ARC-AGI by 9.3, ZebraLogicBench by 6.0, and KORBench by 5.6. It also yields substantial gains on SuperGPQA ( +9 .6), AIME25 ( +5 .4), HARP ( +5 .0), SWE Verified ( +4 .2), and MMLU ( +4 .0). 10 Table 1 SFT results (%) on 22 Open Benchmark datasets. Best displayed values are bold. Prefill recycle steps k = 1

and k = 2 use one and two prompt prefill passes, respectively. The k = 2 -trained/ k = 1 -inference route evaluates the

k = 2 checkpoint with one prefill recycle step.

Benchmark Baseline Prefill recycle step k= 1 Prefill recycle step k= 2 k= 2 -trained / k= 1 -inference Mathematics AIME24 81.3 85.3 84.0

85.7

AIME25 75.3 79.0

80.7 80.3 OlympiadBench 88.7 88.0 89.3 86.7 OmniMath 64.7 65.3 65.3 66.7

HARP 79.0

86.3 84.0 85.7

Reasoning ARC-AGI 32.6 39.4

41.9 37.0 KORBench 59.3 62.9 64.9 62.5 ProcBench 59.8 62.6 68.0 63.4 ZebraLogicBench 80.5 83.5 86.5 81.5

Instruction Following IFEval

86.5 83.9 84.8 84.1 COLLIE 60.4 50.2 70.9 44.4

Coding LiveCodeBench 57.3

63.4 59.5 59.5 SWE Verified 41.8 45.8 46.0 40.4

Comprehensive Evalsets GPQA-Diamond 72.1 73.2

74.8 73.3 SuperGPQA 50.4 56.0 60.0 56.8 HLE 9.3 12.0 10.7 12.0

MMLU 82.7 84.0

86.7 82.7 MMMLU 78.7 80.7 82.0 77.3 MMLU-Pro 82.4 82.5 82.4 82.0 ArenaHard 71.0 77.4 68.6 77.6

SimpleQA 13.7

16.7 15.3 13.3 ChineseSimpleQA 57.3 57.7 60.3 53.3 Avg. 62.9 65.3 66.7 63.9

These improvements span mathematical reasoning, logical reasoning, coding, and comprehensive evaluation, which is consistent with intended role of recycled hiddens as an additional latent reasoning during generation. Prefill recycle step k = 1 also retains a clear aggregate advantage while matching the baseline’s single-pass prefill computation. It improves Avg. score from 62.9 to 65.3 ( +2 .4), with notable gains on HARP ( +7 .3), ARC-AGI ( +6 .8), LiveCodeBench ( +6 .1), SuperGPQA ( +5 .6), and AIME24 ( +4 .0). The k = 2 -trained/ k = 1 -inference reference remains above the baseline in aggregate, but its smaller gain indicates that the strongest SFT performance is obtained when prefill recycle depth used at inference is aligned with k = 2 training path. Overall, the SFT results show that Recycler improves broad aggregate performance after SFT, delivers its largest gains on reasoning-intensive tasks, and preserves a meaningful advantage even with prefill recycle step k = 1 . Prefill recycle step k = 2 achieves the best overall quality, whereas k = 1 offers the more compute-efficient operating point for single-pass prefill.

3.3 Reinforcement Learning

3.3.1 Algorithm and Setup

We train the baseline and two Recycler variants with GSPO [65] on an internal math RL corpus in long-CoT format, using a rule-based verifier as the sole reward signal. Each run starts from its corresponding SFT checkpoint, with the Recycler variants retaining their SFT recycle steps of k = 1 and k = 2 throughout RL. All runs use the same per-step prompt and sampling budgets and are evaluated at mid-training checkpoints, as training was stopped manually before convergence. 11 3.3.2 Normal RL Results

We compare the three training routes introduced above: the non-Recycler baseline and the Recycler routes trained with recycle step k = 1 and k = 2 . Since the recycle step is also an inference-time choice, we additionally evaluate the k = 2 route at k = 1 , yielding four evaluation configurations in total. Table 2 reports these four alongside the SFT starting points of the three routes, which serve as the reference for the stage-wise comparison below. The recycle step is specific to Recycler and is therefore undefined for the baseline, whose inference cost matches the k = 1 configurations.

Table 2 SFT and RL results on Open Benchmark. The first three rows are the SFT starting points of the three training routes; the remaining rows are their RL endpoints. Training k and inference k denote the recycle step used during training and at inference, respectively; both are undefined for the baseline, whose inference cost matches the

k = 1 configurations. Bold marks the best value among the RL rows.

System Training kInference kTotal Compr. Reason. Math Code Instr.-Follow Knowl. Multil. Baseline (SFT) ––58.67 61.00 58.04 77.80 39.97 63.70 35.50 78.67 Recycler (SFT) 1160.23 62.81 62.09 80.80 39.87 61.85 37.17 80.67 Recycler (SFT) 2262.46 64.18 65.31 80.67 41.24 72.97 37.83 82.00 Baseline ––60.06 62.06 62.26 83.80 38.28 63.71 34.00 80.67 Recycler 1161.66 64.89 65.00 82.80 40.22 63.77 35.50 81.33 Recycler 2161.42 63.84 64.28 84.27 42.54 62.46 34.67 78.33 22

64.38 66.12 66.98 84.80 43.33 74.18 37.17 83.67

As shown in Table 2, every Recycler configuration ends above the baseline after RL, whether the recycle step is varied at training time or at inference time. Trained and evaluated at k = 2 , Recycler reaches 64 .38 versus

60 .06 for the baseline, a gain of 4.32 points, and leads on all seven ability groups. The advantage does not depend on retaining the second prompt forward: training and evaluating at k = 1 reaches 61 .66 (+1 .60 ), and the k = 2 -trained route evaluated at k = 1 reaches 61 .42 (+1 .36 ). Both k = 1 configurations match the baseline’s prefill and decode cost, so the improvement at these two operating points comes at no additional inference cost. The two k = 1 results are close in aggregate but reach it differently—the k = 1 -trained model is stronger on comprehensive evaluation ( +2 .83 over baseline) and instruction following, whereas the

k = 2 -trained model evaluated at k = 1 is stronger on code ( +4 .26 ) and weaker on instruction following (−1.26 ) and multilingual evaluation ( −2.33 ). The aggregate gains should therefore not be read as uniform improvement across every ability group.

RL widens Recycler’s advantage rather than merely preserving it. Comparing each route with its own SFT starting point separates what RL adds from what Recycler already brought into RL. The baseline improves by 1.39 points from SFT to RL, and the k = 1 route by 1.43 ; the k = 2 route improves by 1.92 points when evaluated at k = 2 and by 1.76 when evaluated at k = 1 . Consequently, the k = 2 route’s margin over the baseline widens across the stage, from 3.79 points at SFT to 4.31 after RL, whereas the k = 1 route’s margin is essentially unchanged ( 1.56 → 1.59 ). The larger stage-wise increment of the k = 2 route is consistent with recycled hidden states supplying additional latent computation that RL can exploit, suggesting a higher performance ceiling than baseline under the same post-training recipe. Because the SFT/RL points are paired by route and three routes differ in code revision and rollout schedule, these increments support a descriptive comparison of complete training routes rather than an equal-compute claim about RL stage in isolation.

4 Related Work

We discuss related work along two lines. From a decoding perspective, Recycler feeds output hidden states forward to the next token, superficially resembling latent chain-of-thought (CoT) methods—though this is not its motivation. From a training perspective, Recycler also relates to models with recurrent structures.

4.1 Latent Chain-of-Thought

Chain-of-thought techniques let LLMs reason step by step through prompting [35, 55], fine-tuning [62], or reinforcement learning [52, 63]. Subsequent work pushes models to think longer before answering. Long-CoT 12 models such as DeepSeek-R1 [24] realize test-time scaling and deliver substantial gains on mathematics and reasoning, adopting a reflective pattern that verifies candidate answers before responding. Beyond natural-language CoT, several methods generate CoT in latent space. Goyal et al. [23] insert learnable

tokens, while Su et al. [48] use an auxiliary codebook to interleave latent tokens with text. Most closely related is the autoregressive generation of continuous CoT tokens, where training is far less straightforward than inference. Xu et al. [57, 58] inject continuous tokens from an assistant model as latent CoTs. Hao et al. [25] propose COCONUT, which skips unembedding and feeds the output hidden state directly as the next token’s input, adopting iCoT’s progressive learning [12] to make training tractable. Tan et al. [50] build ground-truth continuous tokens by merging discrete CoT tokens and train with next-token prediction (NTP), whereas Shen et al. [47] derive them through self-distillation. Departing from COCONUT’s direct reuse, Zhang et al. [64] reconstruct the hidden state from the previous token’s NTP probabilities, replacing one-hot lookup with a weighted summation and bridging the input–output embedding gap under untied embeddings without training. Recycler does not replace discrete CoT tokens with continuous counterparts; it lets the two coexist. Hidden states encoding rich context are preserved, while decoding still operates on discrete semantic tokens. Recycler therefore needs no explicit objective on output hidden states and trains with standard NTP alone.

4.2 Recursive Transformers

Recursive Transformers reuse the same module multiple times per prediction, adding computation but not parameters. Some loop-based methods call this latent thinking, but the computation is uniform across tokens, unlike the extensive pre-response reasoning of latent CoT. Universal Transformer [11] first shows that repeating a single block can match a deeper model with unshared layers. Loop Transformers are formulated as programmable computers [21] and prove effective for iterative algorithms [59] and length generalization [14], and are now viewed as another dimension of test-time scaling [17, 46]. A second line routes over looped layers: Mixture-of-Recursions [2] assigns per-token recursion depths through a trainable router, and related schemes follow [8, 68]. From the recursive-Transformer perspective, Recycler reuses no intra-token hidden states, recycling only those of the previous token and thus incurring 1×

decoding overhead. More importantly, whereas recursive models typically supervise only the final forward pass, Recycler retains NTP at every step and benefits from future-token prediction signals. Concurrent work likewise recycles hidden states. For example, Latent Recurrent Transformer [31] injects an intermediate source-layer state into the current token’s layers via KV projection and residual injection, while Full-Bandwidth Transformer [53] fuses the previous top-layer state with the next-token embedding through a gated linear unit. Recycler is closer to the latter in injection site, but uses a simpler additive adapter with a shallow MTP-style multi-pass approximation that relies on decode-time extrapolation.

5 Conclusion

We introduced Recycler , a simple architecture that augments each token embedding with a lightweight projection of the preceding token’s top-layer hidden state. This continuous cross-token pathway preserves information that would otherwise be discarded through discrete token sampling, while adding negligible overhead to autoregressive decoding. To train the resulting recurrent dependency efficiently, we develop a parallel multi-pass approximation that generalizes from a shallow recycle depth during training to long autoregressive decoding at inference. Across pre-training, continued pre-training, SFT, and RL, Recycler consistently improves model quality, with especially strong gains in reasoning and mathematics, and approaches the performance of a 2.4× larger baseline at the decoding cost of the smaller model. Introducing Recycler during continued pre-training offers the best balance between training cost and downstream quality. Together, these results establish hidden-state recycling as a practical axis for scaling latent computation across decoding steps, complementary to scaling model size or training data. 13 References

[1] Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc V. Le, et al. Program synthesis with large language models, 2021. URL

https://arxiv.org/abs/2108.07732 .[2] Sangmin Bae, Yujin Kim, Reza Bayat, Sungnyun Kim, Jiyoun Ha, Tal Schuster, Adam Fisch, Hrayr Harutyunyan, Ziwei Ji, Aaron C. Courville, and Se-Young Yun. Mixture-of-recursions: Learning dynamic recursive depths for adaptive token-level computation. In Advances in Neural Information Processing Systems, volume 38, pages 96572–96617, 2025. doi: 10.52202/085713-3229. URL https://proceedings.neurips.cc/paper_files/paper/ 2025/hash/8b08bbf8b420faa6eeb4020720582ec7-Abstract-Conference.html .[3] Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877–1901. Curran Associates, Inc., 2020. URL

https://proceedings.neurips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html .[4] ByteDance Seed. Seed1.5-Thinking: Advancing superb reasoning models with reinforcement learning, 2025. URL

https://arxiv.org/abs/2504.13914 .[5] ByteDance Seed. Seed1.8 model card: Towards generalized real-world agency, 2026. URL https://arxiv.org/ abs/2603.20633 .[6] ByteDance Seed. Seed2.0 model card: Towards intelligence frontier for real-world complexity, 2026. URL

https://arxiv.org/abs/2607.00248 .[7] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021. URL https://arxiv.org/abs/2107.03374 .[8] Yilong Chen, Junyuan Shang, Zhenyu Zhang, Yanxi Xie, Jiawei Sheng, Tingwen Liu, Shuohuan Wang, Yu Sun, Hua Wu, and Haifeng Wang. Inner thinking transformer: Leveraging dynamic depth scaling to foster adaptive internal thinking. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 28241–28259. Association for Computational Linguistics, 2025. doi: 10.18653/ v1/2025.acl-long.1369. URL https://aclanthology.org/2025.acl-long.1369/ .[9] François Chollet. On the measure of intelligence. arXiv preprint arXiv:1911.01547, 2019. URL https://arxiv. org/abs/1911.01547 .[10] DeepSeek-AI. DeepSeek-V3 technical report, 2024. URL https://arxiv.org/abs/2412.19437 .[11] Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Łukasz Kaiser. Universal transformers. In International Conference on Learning Representations, 2019. URL https://iclr.cc/virtual/2019/poster/ 1068 .[12] Yuntian Deng, Yejin Choi, and Stuart Shieber. From explicit cot to implicit cot: Learning to internalize cot step by step, 2024. URL https://arxiv.org/abs/2405.14838 .[13] Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2368–2378. Association for Computational Linguistics, 2019. doi: 10.18653/v1/N19-1246. URL https://aclanthology.org/N19-1246/ .[14] Ying Fan, Yilun Du, Kannan Ramchandran, and Kangwook Lee. Looped transformers for length generaliza-tion. In International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id= 2edigk8yoU .[15] Ippei Fujisawa, Sensho Nobe, Hiroki Seto, Rina Onda, Yoshiaki Uchida, Hiroki Ikoma, Pei-Chun Chien, and Ryota Kanai. ProcBench: Benchmark for multi-step reasoning and following procedure, 2024. URL https: //arxiv.org/abs/2410.03117 .

14 [16] Bofei Gao, Feifan Song, Zhe Yang, Zefan Cai, Yibo Miao, Qingxiu Dong, Lei Li, Chenghao Ma, Liang Chen, Runxin Xu, Zhengyang Tang, Benyou Wang, Daoguang Zan, Shanghaoran Quan, Ge Zhang, Lei Sha, Yichang Zhang, Xuancheng Ren, Tianyu Liu, and Baobao Chang. Omni-MATH: A universal olympiad level mathematic benchmark for large language models. In International Conference on Learning Representations, 2025. URL

https://arxiv.org/abs/2410.07985 .[17] Jonas Geiping, Sean McLeish, Neel Jain, John Kirchenbauer, Siddharth Singh, Brian R. Bartoldson, Bhavya Kailkhura, Abhinav Bhatele, and Tom Goldstein. Scaling up test-time compute with latent reasoning: Arecurrent depth approach. In Advances in Neural Information Processing Systems, volume 38, pages 41340–41391, 2025. doi: 10.52202/085713-1380. URL https://proceedings.neurips.cc/paper_files/paper/2025/hash/ 3b01972cf31e6fa0fe29e4b8b5c2a0a1-Abstract-Conference.html .[18] Gemini Team, Google. Gemini: A family of highly capable multimodal models, 2023. URL https://arxiv.org/ abs/2312.11805 .[19] Gemini Team, Google. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context, 2024. URL https://arxiv.org/abs/2403.05530 .[20] Gemini Team, Google. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities, 2025. URL https://arxiv.org/abs/2507.06261 .[21] Angeliki Giannou, Shashank Rajput, Jy-Yong Sohn, Kangwook Lee, Jason D. Lee, and Dimitris Papailiopoulos. Looped transformers as programmable computers. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 11398–11442. PMLR, 2023. URL

https://proceedings.mlr.press/v202/giannou23a.html .[22] Fabian Gloeckle, Badr Youbi Idrissi, Baptiste Rozière, David Lopez-Paz, and Gabriel Synnaeve. Better & faster large language models via multi-token prediction. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 15706–15734. PMLR, 2024. URL

https://proceedings.mlr.press/v235/gloeckle24a.html .[23] Sachin Goyal, Ziwei Ji, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv Kumar, and Vaishnavh Nagarajan. Think before you speak: Training language models with pause tokens. In International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=ph04CRkPdC .[24] Daya Guo, Dejian Yang, Haowei Zhang, et al. DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning. Nature, 645(8081):633–638, 2025. doi: 10.1038/s41586-025-09422-z. URL https://www.nature.com/ articles/s41586-025-09422-z .[25] Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason E. Weston, and Yuandong Tian. Training large language models to reason in a continuous latent space. In Conference on Language Modeling (COLM), 2025. URL https://openreview.net/forum?id=Itxz7S4Ip3 .[26] Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun. OlympiadBench: A challenging benchmark for promoting AGI with olympiad-level bilingual multimodal scientific problems. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3828–3850. Association for Computational Linguistics, 2024. doi: 10.18653/v1/2024.acl-long.211. URL https: //aclanthology.org/2024.acl-long.211/ .[27] Yancheng He, Shilong Li, Jiaheng Liu, Yingshui Tan, Weixun Wang, Hui Huang, Xingyuan Bu, Hangyu Guo, Chengwei Hu, Boren Zheng, et al. Chinese simpleqa: A chinese factuality evaluation for large language models, 2024. URL https://arxiv.org/abs/2411.07140 .[28] Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR), 2021. [29] Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, NeurIPS Datasets and Benchmarks 2021, December 2021, virtual, 2021. URL https://datasets-benchmarks-proceedings.neurips.cc/paper/2021/ hash/be83ab3ecd0db773eb2dc1b0a17836a1-Abstract-round2.html .

15 [30] Yuzhen Huang, Yuzhuo Bai, Zhihao Zhu, Junlei Zhang, Jinghan Zhang, Tangjun Su, Junteng Liu, Chuancheng Lv, Yikai Zhang, Jiayi Lei, Yao Fu, Maosong Sun, and Junxian He. C-Eval: A multi-level multi-discipline chinese evaluation suite for foundation models. In Advances in Neural Information Processing Systems, 2023. URL

https://arxiv.org/abs/2305.08322 .[31] Zeyi Huang, Xuehai He, LiLiang Ren, Yiping Wang, Baolin Peng, Hao Cheng, Shuohang Wang, Pengcheng He, Jianfeng Gao, Yong Jae Lee, and Yelong Shen. Latent recurrent transformer: Architecture exploration, training strategies, and scaling behavior, 2026. URL https://arxiv.org/abs/2605.26797 .[32] Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. LiveCodeBench: Holistic and contamination free evaluation of large language models for code. In International Conference on Learning Representations, 2025. URL https://arxiv.org/abs/2403. 07974 .[33] Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. Swe-bench: Can language models resolve real-world github issues?, 2024. URL https://arxiv.org/abs/2310.06770 .[34] Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1601–1611. Association for Computational Linguistics, 2017. doi: 10.18653/v1/ P17-1147. URL https://aclanthology.org/P17-1147/ .[35] Tushar Khot, Harsh Trivedi, Matthew Finlayson, Yao Fu, Kyle Richardson, Peter Clark, and Ashish Sabharwal. Decomposed prompting: A modular approach for solving complex tasks. In International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=_nGgzQjzaRy .[36] Tianle Li, Wei-Lin Chiang, Evan Frick, Lisa Dunlap, Tianhao Wu, Banghua Zhu, Joseph E. Gonzalez, and Ion Stoica. From crowdsourced data to high-quality benchmarks: Arena-hard and benchbuilder pipeline, 2024. URL

https://arxiv.org/abs/2406.11939 .[37] Bill Yuchen Lin, Ronan Le Bras, Kyle Richardson, Ashish Sabharwal, Radha Poovendran, Peter Clark, and Yejin Choi. ZebraLogic: On the scaling limits of LLMs for logical reasoning. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research. PMLR, 2025. URL

https://proceedings.mlr.press/v267/lin25i.html .[38] Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. Is your code generated by ChatGPT really correct? rigorous evaluation of large language models for code generation. In Advances in Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=1qvx610Cu7 .[39] M-A-P Team, Xinrun Du, Yifan Yao, Kaijing Ma, Bingli Wang, et al. SuperGPQA: Scaling LLM evaluation across 285 graduate disciplines, 2025. URL https://arxiv.org/abs/2502.14739 .[40] Kaijing Ma, Xinrun Du, Yunran Wang, Haoran Zhang, Zhoufutu Wen, Xingwei Qu, Jian Yang, Jiaheng Liu, Minghao Liu, Xiang Yue, Wenhao Huang, and Ge Zhang. Kor-bench: Benchmarking language models on knowledge-orthogonal reasoning tasks, 2025. URL https://arxiv.org/abs/2410.06526 .[41] Mathematical Association of America. American Invitational Mathematics Examination (AIME). https: //artofproblemsolving.com/wiki/index.php/AIME , 2024. Competition problems from 2024 and 2025 editions. [42] OpenAI. GPT-4 technical report, 2023. URL https://arxiv.org/abs/2303.08774 .[43] OpenAI. OpenAI o1 system card, 2024. URL https://arxiv.org/abs/2412.16720 .[44] Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Mohamed Shaaban, John Ling, Sean Shi, Michael Choi, et al. Humanity’s last exam, 2025. URL https://arxiv.org/abs/2501.14249 .[45] David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling, 2024. URL https://openreview.net/forum?id=Ti67584b98 .[46] Nikunj Saunshi, Nishanth Dikkala, Zhiyuan Li, Sanjiv Kumar, and Sashank J. Reddi. Reason-ing with latent thoughts: On the power of looped transformers. In International Conference on Learning Representations, 2025. URL https://proceedings.iclr.cc/paper_files/paper/2025/hash/ 2676109d49d1eb26d6bc584a8f556305-Abstract-Conference.html .

16 [47] Zhenyi Shen, Hanqi Yan, Linhai Zhang, Zhanghao Hu, Yali Du, and Yulan He. CODI: Compressing chain-of-thought into continuous space via self-distillation. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 677–693. Association for Computational Linguistics, 2025. doi: 10.18653/v1/2025.emnlp-main.36. URL https://aclanthology.org/2025.emnlp-main.36/ .[48] Dijia Su, Hanlin Zhu, Yingchen Xu, Jiantao Jiao, Yuandong Tian, and Qinqing Zheng. Token assorted: Mixing latent and text tokens for improved language model reasoning. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 57144–57163. PMLR, 2025. URL https://proceedings.mlr.press/v267/su25g.html .[49] Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V. Le, Ed H. Chi, Denny Zhou, and Jason Wei. Challenging BIG-Bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261, 2022. URL https://arxiv.org/abs/2210.09261 .[50] Wenhui Tan, Jiaze Li, Jianzhong Ju, Zhenbo Luo, Ruihua Song, and Jian Luan. Think silently, think fast: Dynamic latent compression of LLM reasoning chains. In Advances in Neural Information Processing Systems, volume 38, pages 4646–4668, 2025. doi: 10.52202/085713-0164. URL https://proceedings.neurips.cc/paper_ files/paper/2025/hash/0706261aedab63814a2b73c32564b4c4-Abstract-Conference.html .[51] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30, pages 5998–6008, 2017. URL https://proceedings.neurips.cc/paper/2017/hash/ 3f5ee243547dee91fbd053c1c4a845aa-Abstract.html .[52] Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. Math-shepherd: Verify and reinforce LLMs step-by-step without human annotations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9426–9439. Association for Computational Linguistics, 2024. doi: 10.18653/v1/2024.acl-long.510. URL https://aclanthology.org/ 2024.acl-long.510/ .[53] Xi Wang, Ziyang Cai, Zheng Zhan, Harry Dong, Ying Fan, Gustavo de Rosa, Tim Pearce, and John Langford. Full-bandwidth transformer, 2026. URL https://arxiv.org/abs/2608.08888 .[54] Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arul-raj, Xuan He, Ziyan Jiang, et al. MMLU-Pro: A more robust and challenging multi-task language understanding benchmark, 2024. URL https://arxiv.org/abs/2406.01574 .[55] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems, volume 35, pages 24824–24837, 2022. doi: 10.52202/068431-1800. URL https://proceedings.neurips.cc/paper_files/paper/2022/hash/ 9d5609613524ecf4f15af0f7b31abca4-Abstract-Conference.html .[56] Jason Wei, Nguyen Karina, Hyung Won Chung, Yunxin Joy Jiao, Spencer Papay, Amelia Glaese, John Schulman, and William Fedus. Measuring short-form factuality in large language models, 2024. URL https://arxiv.org/ abs/2411.04368 .[57] Yige Xu, Xu Guo, Zhiwei Zeng, and Chunyan Miao. SoftCoT: Soft chain-of-thought for efficient reasoning with LLMs. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 23336–23351. Association for Computational Linguistics, 2025. doi: 10.18653/v1/2025. acl-long.1137. URL https://aclanthology.org/2025.acl-long.1137/ .[58] Yige Xu, Xu Guo, Zhiwei Zeng, and Chunyan Miao. SoftCoT++: Test-time scaling with soft chain-of-thought reasoning, 2025. URL https://arxiv.org/abs/2505.11484 .[59] Liu Yang, Kangwook Lee, Robert Nowak, and Dimitris Papailiopoulos. Looped transformers are better at learning learning algorithms. In International Conference on Learning Representations, 2024. URL https: //openreview.net/forum?id=HHbRxoDTxE .[60] Shunyu Yao, Howard Chen, Austin W. Hanjie, Runzhe Yang, and Karthik Narasimhan. COLLIE: Systematic construction of constrained text generation tasks. In International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=kxgSlyirUZ .[61] Albert S. Yue, Lovish Madaan, Ted Moskovitz, DJ Strouse, and Aaditya K. Singh. HARP: A challenging human-annotated math reasoning benchmark, 2024. URL https://arxiv.org/abs/2412.08819 .

17 [62] Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. MAmmoTH: Building math generalist models through hybrid instruction tuning. In International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=yLClGs770I .[63] Eric Zelikman, Georges Harik, Yijia Shao, Varuna Jayasiri, Nick Haber, and Noah D. Goodman. Quiet-STaR: Language models can teach themselves to think before speaking, 2024. URL https://arxiv.org/abs/2403.09629 .[64] Zhen Zhang, Xuehai He, Weixiang Yan, Ao Shen, Chenyang Zhao, and Xin Eric Wang. Soft thinking: Unlocking the reasoning potential of LLMs in continuous concept space. In Advances in Neural Information Processing Systems, volume 38, pages 168990–169012, 2025. doi: 10.52202/085713-5629. URL https://proceedings.neurips.cc/ paper_files/paper/2025/hash/f7396d1c54d51416958d63e285377103-Abstract-Conference.html .[65] Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, Jingren Zhou, and Junyang Lin. Group sequence policy optimization, 2025. URL https: //arxiv.org/abs/2507.18071 .[66] Wanjun Zhong, Ruixiang Cui, Yiduo Guo, Yaobo Liang, Shuai Lu, Yanlin Wang, Amin Saied, Weizhu Chen, and Nan Duan. AGIEval: A human-centric benchmark for evaluating foundation models, 2023. URL https: //arxiv.org/abs/2304.06364 .[67] Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction-following evaluation for large language models, 2023. URL https://arxiv.org/abs/2311.07911 .[68] Rui-Jie Zhu, Zixuan Wang, Kai Hua, Tianyu Zhang, Ziniu Li, Haoran Que, Boyi Wei, Zixin Wen, Fan Yin, He Xing, Lu Li, Jiajun Shi, Kaijing Ma, Shanda Li, Taylor Kergan, Andrew Smith, Xingwei Qu, Mude Hui, Bohong Wu, Qiyang Min, Hongzhi Huang, Xun Zhou, Wei Ye, Jiaheng Liu, Jian Yang, Yunfeng Shi, Chenghua Lin, Enduo Zhao, Tianle Cai, Ge Zhang, Wenhao Huang, Yoshua Bengio, and Jason Eshraghian. Scaling latent reasoning via looped language models, 2025. URL https://arxiv.org/abs/2510.25741 .

18