Can the distinct reasoning steps a language model shows in its text output also be found in its internal states? A new study put it to the test.

When a reasoning model solves a task step by step, it does different things along the way: reading data, breaking down the problem, retrieving a formula, running a calculation. Researchers at South Korea's KAIST and Naver AI Lab wanted to know whether those reasoning steps can also be separated from one another inside the model's numerical representations. They can, and the signal is strongest in the middle layers.

The team defined eight recurring reasoning operations, including extraction, decomposition, formula recall, deduction, and computation. They had three models (Qwen2.5-7B, Qwen3-8B, and Gemma4-31B) solve math problems, split the solution paths into segments, and then used GPT-5 to label each segment with one of those operations.

Reasoning steps are clearly separable inside the model

The different reasoning operations can be reliably told apart in the models' internal representations, and this holds across all three models tested. The separation peaks in the middle layers.

The researchers checked whether simple word choice could account for the effect. A classifier that only looked at the tokens used performed worse than one analyzing internal representations. Position within the solution path didn't explain it either. That means the internal states carry information about the type of reasoning step that goes beyond surface-level wording.

Same words, different representations depending on the reasoning step

Common function words like "a," "is," or "the" show up across very different reasoning steps. In the early layers, their representations are still jumbled together, but by the middle and later layers they separate according to the surrounding operation. The same word gets a different internal representation depending on which reasoning step it belongs to.

The researchers also tested whether a reasoning step forms in isolation. When they blocked attention to the preceding 30 tokens through a targeted intervention, the signal for that operation weakened. Reasoning steps don't emerge on their own but build on the preceding context.

Even on incorrectly solved problems, the type of step the model was performing stayed identifiable, whether it was computing, retrieving a formula, or deducing. A flawed computation step still looked like a computation step internally, even when the result was wrong.

The separability held up in additional tests too. It replicated with Llama-3-8B, and for Qwen3-8B the trained classifiers transferred successfully to GPQA-Diamond and MATH-500. That said, the experiments are limited to math tasks and a handful of models. Whether these findings can be used to catch errors or steer a model mid-generation remains an open question for future work.

The relationship between text output and internal computation matters for AI safety. Reading the chain of thought is one of the few oversight tools available, according to OpenAI, but Anthropic showed that models only disclose the hints they used in 25 to 39 percent of cases. A method that translates a model's internal vectors into readable text revealed that Claude Opus 4.6 processes more than what shows up in its output reasoning. And with OpenAI's Astra model, the Recurrent Depth technique shifts part of the reasoning into internal numerical representations, which is the space the KAIST study investigates.