Topology of Reasoning: Understanding Large Reasoning Models through Reasoning Graph Properties

We sincerely thank the reviewer for their thorough and constructive feedback, which significantly helped us strengthen both the empirical scope and clarity of our paper.

>W1&Q1

The conclusions of this paper are only validated by three math reasoning datasets

To address your concern, we have extended our evaluation to include two additional tasks beyond math, inspired by prior work such as [1]: (i) StrategyQA, a multi-hop question answering dataset, and (ii) LogicalDeduction, a logical reasoning dataset from BIG-Bench.

Below, we report the comparison of reasoning graph properties between DeepSeek-R1-Distill-Qwen-32B (reasoning model) and Qwen2.5-32B (base model) on these datasets. As with the results reported on math tasks in the main paper, the reasoning model exhibits markedly different graph characteristics, such as significantly higher cycle rates, larger diameters, and stronger small-world properties.

These consistent patterns across several domains provide further evidence for the generality of our findings. We appreciate the reviewer’s suggestion, which helped us strengthen the empirical validation of our claims.

---

(i) StrategyQA

---

Cycles

---

Diameter (mean±var)

---

Small world index

---

(ii) LogicalDeduction

---

Cycle

---

Diameter (mean±var)

---

Small world index

[1] Understanding the Reasoning Ability of Language Models From the Perspective of Reasoning Paths Aggregation, ICML'24

>W2&W3&Q2

it fails to explicitly specify how to construct such data in practice.

when a large amount of supervised data has been collected, how can the authors' method be applied to select samples for constructing an optimal dataset?

To directly address this concern, and following Reviewer 8fWp’s suggestion, we extended our approach to analyze the reasoning graph during the forward pass of a frozen LLM (Qwen2.5-32B-Instruct, the same model used in S1). Specifically, we fed various SFT datasets into the model and measured the resulting reasoning graph diameters without requiring any training.

In this experiment, we used two versions of the S1 dataset: S1-v1.0 and S1-v1.1, where it is known that models trained on S1-v1.1 achieve better downstream reasoning performance. The table below compares the diameters of reasoning graphs induced by these datasets. We find that S1-v1.1 consistently yields larger graph diameters, even before any training has occurred, suggesting that the diameter of the reasoning graph can serve as a useful signal for evaluation of SFT data quality before any model training.

This provides a concrete answer to your question: in scenarios like S1, where a large amount of supervised data is available, our method allows one to feed candidate subsets into the target model, construct the corresponding reasoning graphs, and use their properties (e.g., diameter) to guide data selection before training. This analysis does not require prohibitive computational costs due to finetuning as needed in the original paper.

We will include this result in the revised version of the paper to clarify the practical applicability of our method to SFT data selection.

Diameter (mean±var)

>W4

the reverse does not necessarily hold: SFT data with more cycles and larger diameters do not inherently lead to better performance.

We agree with you that datasets with more cycles or larger graph diameters may not inherently guarantee better model performance. Our intention was not to suggest that solely optimizing for graph properties such as diameter will lead to better results.

Rather, we position graph metrics such as diameter as auxiliary signals for data selection, not as standalone objectives. In practice, high-quality data pools should first be constructed using strong reasoning models (e.g., Gemini or DeepSeek-R1), as in the S1 pipeline. Once such data is available, reasoning graph metrics can serve as informative heuristics to filter or prioritize samples that are more likely to support effective reasoning. These metrics are meant to complement, not replace, other quality criteria.

We explicitly do not advocate for selecting data solely to maximize graph diameter, as this could promote undesirable behaviors like overthinking. Instead, we propose reasoning graph analysis as a diagnostic tool, complementary to other strategies, to inform data selection.

To make this intended usage clearer, we will revise the relevant sections in the paper.

>W5

The attribution of redundant cycles to overthinking (Lines 296-298) conflicts with the empirical analysis in Section 4.4, which conclusively traces their origin to language-mixing effects.

In Section 4.4, we presented language mixing as one concrete example that may lead to redundant cycles. Our mention of overthinking was not meant to contradict this, but rather to discuss another pattern of redundant cycles observed in prior work.

During a rebuttal, we newly observed a failed output from Distill-Qwen-32B on GSM8K, which involved a simple arithmetic problem. Despite the simplicity, the model showed repetitive Wait outputs and trial-and-error loops, similar to patterns noted in [2]. This type of redundant cycle differs from the language-mixing discussed in Section 4.4 of the main paper.

…
They have 78 left, so S + 59 = 78 → S = 19.  
Same result. So, perhaps the problem is that the initial stock is 19, but they used 38 on Day 1, ...  
Wait, maybe the problem is that the orders are received before ...  
Wait, that's what I did initially. So, perhaps the problem is ....  
Wait, maybe I made a mistake in the equations. Let me try to model it again, ...  
Wait, perhaps the problem is that the orders are received before the usage on the...  
Wait, let me try that.  
…

To avoid confusion, we will revise the relevant sentence to clarify that language mixing is one identified cause of redundant cycles, while overthinking represents another distinct pattern.

[2] Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs.

>W6

The paper has many minor flaws that may affect the reader's impression:

Thank you for pointing out the minor issues related to notation, citations, and descriptions. We appreciate your careful reading.

We will thoroughly review and revise these details in the final version to improve clarity and presentation.

>Q3

The authors should explain why they chose the hidden states from the layer of 90% depth as the default rather than the last layer.

As there is no established consensus on which layer of an LLM best reflects reasoning-related representations, we conducted a comprehensive analysis using hidden states from five different depths: 10%, 30%, 50%, 70%, and 90%. The figures in the paper report results from all these layers. Among them, the 90% layer consistently showed the clearest trends, which is why we used it as the default in our main analysis.

Following your suggestion, we also analyzed cycles using the hidden states from the final (100%) layer. As shown in the table below, the reasoning model (Distill-Qwen-32B) consistently exhibits more cycles than the base model (Qwen2.5-32B), and the final layer also follows the same trends. This result suggests that our analysis holds even at the final layer, supporting the robustness of our findings across different depths.

Cycles

>Q4

The authors could demonstrate the statistics (e.g., average number of nodes and edges)

To address your question, we report below the average number of nodes and edges in the reasoning graphs generated for each dataset: GSM8K, MATH500, and AIME2024. The model used was Distill-Qwen-32B, with a target layer ratio of 0.9 and K = 200 for K-means clustering. As shown below, the number of nodes increases as task difficulty increases, reflecting the expansion of the reasoning search space. For a more detailed interpretation of what each node represents in the reasoning process, please refer to our response to Reviewer mySX, W2. We will include this table in the Appendix of the revised paper to improve the clarity.

We sincerely thank the reviewers for their thoughtful and constructive feedback. We conducted additional analyses and clarifications to address your concerns, as detailed below.

>W1

The paper is a bit hard to read with some grammar issues. For example in the first paragraph "These recent reasoning models are characterized with to think and reason for longer before responding". Also the choice of math notations are also not helping the readers to better understand the paper. For example, the author used "T" to define the number of reasoning steps, then the readers will naturally expect to use "t" to index the step, but then the author used "t" to index the token within each step. Same for "l" and "L". In general I find much can be improved in the writing.

Thank you for pointing this out. Regarding the sentence
These recent reasoning models are characterized with to think and reason for longer before responding
we have corrected it to:
These recent reasoning models are characterized by their ability to think and reason for longer before responding

As for the notations, we agree that the original usage was potentially confusing. To improve clarity:

• We now use $t$ to index the reasoning step,
• and use $m$ to index the token within each step.
• The layer index previously denoted by $\ell$ is now replaced by $\mu$.

If there are any other specific points that you found difficult to follow or unclear, we would greatly appreciate further feedback so we can improve the paper’s readability.

>W2

For the clustering of segments, there are many ad hoc choices that are not explained well. For example, the choice of K=200. Also the authors only showed 3 representative centroids among the 200. Can the authors add the others in the Appendix? I am curious if the other centroid make sense.

As noted in lines 181–183 of the paper, we already included an analysis of different values of $k$ in Appendix D. Specifically, we tested $k = 50, 100$, and $200$, and found that the trends discussed in the main paper were consistent across these $k$-values. We invite you to refer to Appendix D for full details.

Furthermore, to address your concern about the other centroids when $k = 200$, we conducted an automatic labeling experiment using an LLM to explore what kinds of reasoning patterns these centroids represent. Specifically, we used the GPT-4o-mini API to assign a theme to each centroid, based on the reasoning steps associated with it in the reasoning graph of DeepSeek-R1-Distill-Qwen-32B on the GSM8K dataset. We provided the following system prompt to GPT, and then input multiple reasoning steps corresponding to each centroid as the user prompt:

You are a data analyst. The following is an output from a LLM.
Your task is to carefully read the text and summarize its main theme in 1–3 English words.

The following table presents the assigned theme for each centroid, along with example reasoning steps that were mapped to that centroid.

This analysis revealed that centroids often align with meaningful reasoning patterns. In addition to previously reported examples such as Add, Multiply, and Wait, we discovered centroids associated with higher-level computations (e.g., Calculations Totals, Averages), semantics-bound reasoning (e.g., Age Calculations, Cost Calculations), and structural elements (e.g., Answer Formatting, Placeholder Tags).

Interestingly, we also identified centroids linked to reasoning behaviors, such as Planning, which reflects the model’s initial steps when approaching a math problem. Moreover, nodes like Wait appear to encompass diverse subtypes. For instance, we found centroids such as Calculation Correctness and Reevaluation, which reflects the model's tendency to reassess or double-check its own output. We also identified a distinct centroid labeled Inconsistencies, which captures moments where the model recognizes contradictions in its own reasoning and attempts to recalculate accordingly.

We believe that our additional analysis effectively addresses your concern regarding whether each centroid makes sense.

To improve clarity and better support the validity of our analysis, we will include the complete list of centroid themes with illustrative examples in the revised Appendix.

>W3

It seems the experimental results are restricted to the Qwen family of models. The results will be stronger if more model families can be included.

Recent studies on reasoning [1,2] have focused on the Qwen family. In line with this trend, we chose Qwen models for our main experiments due to their widespread use in recent reasoning research.

To further test the generality of our findings, and following your suggestion, we conducted the same reasoning graph analysis using models from the LLaMA family. Specifically, we compared LLaMA-3.1-8B (base model) and DeepSeek-R1-Distill-LLaMA-8B (reasoning model) on the GSM8K dataset.

The tables below present results for cycle ratio, graph diameter, and small world index. As with the Qwen models, we observe that the reasoning model exhibits more cycles, larger diameters, and larger small-worldness than the base model across all layer ratios. This consistency supports the generality of our findings in another series of models.

We will include these results in the revised version of the paper. We will be happy to address if the reviewer has further suggestions on open-source reasoning and base model pairs, which need to be included.

---

Cycles

---

Diameter (mean±var)

---

Small world index

[1] Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?
[2] The Invisible Leash: Why RLVR May Not Escape Its Origin

Thank you for your thoughtful and constructive feedback. We carefully addressed each of your questions below and made several clarifications and extensions to strengthen the paper.

>W1&Q1

Lack of Implementation Details

Can you provide more details on the construction of reasoning graphs, especially regarding how the nodes are constructed and whether they are biased towards specific large reasoning models?

To clarify, the nodes in the reasoning graph are constructed independently for each model. Therefore, there is no risk of bias arising from using pre-constructed nodes generated by a single model such as Qwen2.5-32B-Instruct. Each model produces its own reasoning steps, which are then segmented and embedded to form the graph.

Similarly, when analyzing different layers, we construct segment representations separately for each layer of each model, as described in Lines 114–116 of the paper. Thus, the reasoning graph at each layer is based on the outputs from that specific layer.

To improve clarity, we will revise the relevant sections in the paper to explicitly explain these details. We appreciate your feedback for helping us identify this point of confusion.

>W2&Q2

Would this really reflect the internal reasoning process of large reasoning models or just a statistical analysis of the output tokens?

How do you justify the validity of the reasoning graphs in reflecting the true reasoning process of large reasoning models?

To address this concern about the validity of reasoning graphs, we conducted an automatic labeling experiment using an LLM to examine whether the nodes (centroids) in the reasoning graph correspond to meaningful reasoning behaviors, rather than simply grouping surface-level token patterns. Specifically, we used the GPT-4o-mini API to assign semantic themes to each centroid in the reasoning graph of DeepSeek-R1-Distill-Qwen-32B, constructed from its reasoning traces on the GSM8K dataset. For each centroid, we provided the associated reasoning steps and prompted the model with the following instruction:

You are a data analyst. The following is an output from a LLM.
Your task is to carefully read the text and summarize its main theme in 1–3 English words.

The following table presents the assigned theme for each centroid, along with example reasoning steps that were mapped to that centroid.

This analysis revealed that centroids often align with meaningful reasoning patterns. In addition to previously reported examples such as Add, Multiply, and Wait, we discovered centroids associated with higher-level computations (e.g., Calculations Totals, Averages, Divisions), semantics-bound reasoning (e.g., Age Calculations, Cost Calculations), and structural elements (e.g., Answer Formatting, Placeholder Tags).

Interestingly, we also identified centroids linked to reasoning behaviors, such as Planning, which reflects the model’s initial steps when approaching a math problem. Moreover, nodes like Wait appear to encompass diverse subtypes. For instance, we found centroids such as Calculation Correctness and Reevaluation, which reflect the model's tendency to reassess or double-check its own output. We also identified a distinct centroid labeled Inconsistencies, which captures moments where the model recognizes contradictions in its own reasoning and attempts to recalculate accordingly.

This analysis supports the claim that the reasoning graph provides a meaningful abstraction of the model’s reasoning process, rather than merely capturing surface-level token patterns. The presence of consistent and semantically interpretable themes across centroids—such as arithmetic operations, planning behaviors, and self-correction—indicates that the graph structure reflects underlying reasoning behaviors.

To improve clarity, we will include the full list of labeled centroid themes and examples in the Appendix of the revised version.

>W3

Can we apply the proposed method to construct the reasoning graph offline and then analyze the graph characteristics to directly judge the quality of the SFT-Data? A naive solution I can think of is to feed the SFT-Data into the forward process of the LLMs in a teaching-forcing manner and extract the hidden states to construct the reasoning graph.

You are right in pointing out that the original version of our paper focused on analyzing reasoning graphs constructed post hoc—i.e., after training LLMs on different SFT datasets. While this analysis revealed interesting trends, it required training models before assessing the quality of the datasets.

To improve its practicality for dataset construction with your suggestion, we extended our approach to evaluate reasoning graphs during the forward pass of a frozen model, without any retraining. Specifically, we input various SFT datasets into a frozen LLM (Qwen2.5-32B-Instruct, the same model used in S1), and constructed reasoning graphs by extracting hidden states using teacher forcing. This allowed us to evaluate the structural properties of the datasets prior to training.

We applied this method to two datasets: S1-v1.0, and S1-v1.1, where it is known that models trained on S1-v1.1 achieve better downstream reasoning performance (e.g., higher accuracy on MATH500 and AIME2024). The table below compares the diameters of reasoning graphs induced by these datasets.

We find that S1-v1.1 consistently yields larger graph diameters, even before any training has occurred, suggesting that reasoning graph properties can serve as a useful signal for evaluation of SFT data quality before training.

Thank you again for the valuable suggestion. We will include this result in the revised version as a more direct method for evaluating SFT data quality.

Diameter (mean±var)

We thank the reviewer for their thoughtful comments and valuable suggestions, which helped us strengthen our analysis and clarify key aspects of our work.

>W1&W2

One possible confound is that these graph metrics may increase with longer reasoning chains in general.

The reasoning steps are segmented at newlines. Some models may have a propensity to generate longer sequences. Is comparing the graphs generated from these models fair?

First, we note that some of the key graph metrics we use—such as the small-world coefficient—are normalized by definition (line 154). Specifically, small-worldness is computed by dividing the graph’s clustering coefficient and average path length by those of a random graph with the same number of nodes and edges. This normalization is designed to enable fair comparisons even when graphs vary in size, such as when models differ in the number of reasoning segments.

Second, our comparisons are not limited to base vs. reasoning models. We also analyze multiple reasoning models of different parameter sizes, all of which tend to produce long outputs. In those comparisons as well, we observe consistent trends: models with stronger reasoning performance tend to exhibit more structured graph properties, such as more cycles and higher small-worldness. This supports the claim that the graph metrics reflect reasoning capability rather than just output length or compute.

Finally, and most directly addressing your suggestion, we did attempt experiments to control for output length by engineering prompts that elicit structured, similar-length responses across models. However, we found that constraining the output in this way significantly degraded the performance of the reasoning models. Therefore, in the main paper, we report results using unconstrained prompts, which better reflect the models’ natural and effective operating regimes.

For these reasons, we believe the graph properties we observe reflect real differences in how the models reason, not just differences in output length or how much they compute.

>W3

Figures 7b and 7c show a massive increase in AIME24 accuracy between Distill Qwen-7B and Qwen-14B. However, the cycle counts are similar between the two, and graph diameter is noticeably larger for Qwen-7B.

Figures 7 only present the cycle ratio, cycle count, and diameter metrics, but Appendix G includes results for the small-world index, which shows that the 14B model exhibits higher small-worldness than the 7B model. The table below summarizes the average values of all four graph properties—cycle ratio, cycle count, diameter, and small-world index—based on the figures in the paper. As shown, the 14B model demonstrates better graph properties than the 7B model on three out of four metrics．

This suggests that, when considering the full set of graph metrics, there is a meaningful relationship between stronger graph properties and better model performance.

>Q1

Are the nodes in the reasoning graphs studied in the paper always directly tied to concrete reasoning techniques?

To answer your question, we conducted an automatic labeling experiment using an LLM to investigate whether the nodes in the reasoning graph correspond to meaningful reasoning techniques.

Specifically, we used the GPT-4o-mini API to assign a theme to each node, based on the reasoning steps associated with nodes in the reasoning graph of Distill-Qwen-32B on the GSM8K dataset. We provided the following system prompt to GPT, and then input multiple reasoning steps corresponding to each node as the user prompt.

You are a data analyst. The following is an output from a LLM.
Your task is to carefully read the text and summarize its main theme in 1–3 English words.

The table below presents some of the identified themes along with reasoning steps corresponding to each node. In addition to previously reported categories like Add, Multiply, and Wait, we discovered centroids associated with:

• Higher-level computations (Calculations Totals, Averages, Divisions)
• Semantics-bound reasoning (Age Calculations, Cost Calculations)
• Structural elements (Answer Formatting, Placeholder Tags)
• Reasoning behaviors such as Planning
• Variants of "Wait" behavior, including Reevaluation, Inconsistencies, and Calculation Correctness

These findings suggest that many nodes align with concrete reasoning techniques, supporting the validity of the reasoning graph.

While not all 200 nodes are directly tied to concrete reasoning operations—some, like </think>, reflect structural or formatting artifacts commonly produced by the model—the majority of nodes correspond to interpretable reasoning patterns. This demonstrates that the reasoning graph captures valid reasoning techniques. We believe this supports the soundness of our approach.

To improve clarity, we will include the full list of labeled node themes and examples in the Appendix of the revised version.

>W5

Additional insight into what the cycles accomplish would be helpful.

As you point out, not all cycles may contribute equally to reasoning quality, and differentiating between helpful and redundant cycles is a key challenge. To provide further insight, we conducted an additional analysis comparing the reasoning graphs of samples that were answered correctly (n = 936) and incorrectly (n = 64) by the reasoning model (Distill-Qwen-32B). The results are summarized below:

Interestingly, incorrect samples exhibit a higher proportion of cycles, yet have smaller graph diameters. This suggests that not all cycles are beneficial—specifically, cycles that do not significantly increase the diameter may represent shallow or localized loops that fail to expand the reasoning space meaningfully.

This observation supports our interpretation in the main paper (e.g., discussion around language mixing and overthinking) that redundant cyclic structures may harm performance, and only cycles that meaningfully contribute to graph expansion (i.e., diameter) are potentially beneficial.

This highlights the difference between trivial and non-trivial cycles and suggests new directions for reasoning graph analysis.

>W4

Were the examples presented in this figure solved by both models? If not, are all the nodes explored by the reasoning model essential to solving the answer.

Do you think that overlaying graphs from multiple attempts from a base model would results in similar graph coverage to that of the reasoning model?

As discussed in our response to W5, we found that cyclic patterns that do not contribute to increased diameter tend to be less helpful for reaching the correct answer.

You also raise an interesting hypothesis: whether aggregating reasoning graphs from multiple base model attempts (e.g., pass@k) could resemble the graph produced by a single pass of a reasoning model. While this is a valuable line of inquiry, we emphasize a key distinction: In our setting, the reasoning graph is built from a single, coherent forward pass, where all reasoning steps are connected. We quantified the coverage of the graph by measuring diameter among the connected graph nodes. By contrast, pass@k consists of multiple disjoint generations, which may not form a connected graph under our definition of diameter. As a result, even if the node coverage is similar between the aggregated pass@k graph and the reasoning model's graph, their diameters need not align, since disconnected components are excluded from the diameter computation.

However, we think this is a promising research direction, and exploring pass@k phenomena through the lens of graph theory could yield deeper insights into model exploration.