# 研究：AI 模型文本中的推理步骤对应内部不同模式

- 来源：The Decoder：AI News（RSS）
- 作者：Jonathan Kemper
- 发布时间：2026-09-12 21:39
- AIHOT 分数：46
- AIHOT 链接：https://aihot.news/items/cmtygbeh70d78rojhophp9jf6
- 原文链接：https://the-decoder.com/ai-models-written-reasoning-steps-correspond-to-distinct-internal-patterns-a-new-study-finds

## AI 摘要

韩国 KAIST 与 Naver AI Lab 研究发现，推理模型文本输出中的推理步骤可在其内部数值表示中被区分开，信号在中间层最强。团队定义提取、分解、公式回忆、演绎、计算等八种推理操作，用 GPT-5 为 Qwen2.5-7B、Qwen3-8B 和 Gemma4-31B 的解题路径分段打标，结果在三个模型上均成立，并复现于 Llama-3-8B。

## 正文

Can the distinct reasoning steps a language model shows in its text output also be found in its internal states? A new study put it to the test.

When a reasoning model solves a task step by step, it does different things along the way: reading data, breaking down the problem, retrieving a formula, running a calculation. Researchers at South Korea's KAIST and Naver AI Lab wanted to know whether those reasoning steps can also be separated from one another inside the model's numerical representations. They can, and the signal is strongest in the middle layers.

The team defined eight recurring reasoning operations, including extraction, decomposition, formula recall, deduction, and computation. They had three models (Qwen2.5-7B, Qwen3-8B, and Gemma4-31B) solve math problems, split the solution paths into segments, and then used GPT-5 to label each segment with one of those operations.

The same response produces a different activation pattern depending on which reasoning operation is being probed. Segments cluster along the corresponding direction in the scatter plot. | Image: Jeong et al.

Reasoning steps are clearly separable inside the model

The different reasoning operations can be reliably told apart in the models' internal representations, and this holds across all three models tested. The separation peaks in the middle layers.

The researchers checked whether simple word choice could account for the effect. A classifier that only looked at the tokens used performed worse than one analyzing internal representations. Position within the solution path didn't explain it either. That means the internal states carry information about the type of reasoning step that goes beyond surface-level wording.

Across all three models, reasoning operations can be reliably separated, with the clearest signal in the middle layers. | Image: Jeong et al.

Same words, different representations depending on the reasoning step

Common function words like "a," "is," or "the" show up across very different reasoning steps. In the early layers, their representations are still jumbled together, but by the middle and later layers they separate according to the surrounding operation. The same word gets a different internal representation depending on which reasoning step it belongs to.

The researchers also tested whether a reasoning step forms in isolation. When they blocked attention to the preceding 30 tokens through a targeted intervention, the signal for that operation weakened. Reasoning steps don't emerge on their own but build on the preceding context.

The same words overlap in early layers and separate by surrounding operation in the middle and late layers. | Image: Jeong et al.

Even on incorrectly solved problems, the type of step the model was performing stayed identifiable, whether it was computing, retrieving a formula, or deducing. A flawed computation step still looked like a computation step internally, even when the result was wrong.

The separability held up in additional tests too. It replicated with Llama-3-8B, and for Qwen3-8B the trained classifiers transferred successfully to GPQA-Diamond and MATH-500. That said, the experiments are limited to math tasks and a handful of models. Whether these findings can be used to catch errors or steer a model mid-generation remains an open question for future work.

The relationship between text output and internal computation matters for AI safety. Reading the chain of thought is one of the few oversight tools available, according to OpenAI, but Anthropic showed that models only disclose the hints they used in 25 to 39 percent of cases. A method that translates a model's internal vectors into readable text revealed that Claude Opus 4.6 processes more than what shows up in its output reasoning. And with OpenAI's Astra model, the Recurrent Depth technique shifts part of the reasoning into internal numerical representations, which is the space the KAIST study investigates.