# RLT：跨 token 循环的 Transformer 架构设计提案

- 来源：elvis (@omarsar0)
- 发布时间：2026-09-14 01:12
- AIHOT 分数：33
- AIHOT 链接：https://aihot.news/items/cmu02nq8s02oyrory1c4kgmc2
- 原文链接：https://x.com/omarsar0/status/2099184337195565335

## AI 摘要

一份新技术报告提出 Recurrent Looped Transformer（RLT），让解码器对每个 token（含 prompt 与回复）循环复用，因果编码器构建全局 KV 记忆。48 层解码器下，t 个 token 的计算路径经过 48t 个解码块，深度随序列增长而单 token 成本不变。该报告仅为设计提案，作者称推理增益、硬件加速与 RL 扩展尚未实测。

## 正文

Looped transformers are a popular architecture topic right now.

This new technical report extends the loop across tokens.

Recurrent Looped Transformer (RLT) makes the decoder recurrent over every token, prompt and response included.

A causal encoder builds global KV memory. For each new token, the decoder combines the token's encoder representation with its own final hidden state from the previous token and a sliding-window cache of recent activations.

With a 48-layer decoder, the computation path after t tokens runs through 48t decoder blocks, while each token still executes a fixed number of blocks. Depth grows with the sequence and per-token cost stays the same.

The same state transition is used for pretraining, SFT, sampling and RL replay, and nothing resets at the prompt-response boundary. RL replay rebuilds states under the current weights instead of reusing stale rollout states.

The report is a design proposal. The author states that reasoning gains, hardware speedups and RL scaling are goals that have not been measured yet.

Paper: https://github.com/yifanzhang-pro/recurrent-looped-tranformer

Chat with Paper: https://academy.dair.ai/papers/recurrent-looped-transformer