Q1: Reliability of using LLMs to assess the correctness of intermediate reasoning steps.

A1: We fully understand your concerns about the reliability of using LLMs to assess the correctness of intermediate reasoning steps, particularly the dependency on model capabilities and prompt sensitivity. In fact, we have explicitly addressed this issue in lines 246-258 of our submitted manuscript. As reported: "Our assessment approach achieves accuracies of 82.9% for correct examples and 81.8% for incorrect examples, demonstrating its effectiveness." These results offer quantitative validation of our evaluation method, although we agree further validation and model-independent analysis are valuable future work directions.

---

Q2: Reliability of using Llama-3.3-70B to automatically segment a response into reasoning thoughts.

A2: Thank you very much for your insightful suggestion. We selected Llama-3.3-70B because, at the time of our ICML submission deadline, it was one of the strongest and most widely adopted open-source models available. To further validate our segmentation approach, we followed your recommendation and performed a manual evaluation on 100 randomly selected thought segments generated by DeepSeek-R1 on the AIME 2024 dataset. This manual check yielded an accuracy of 86%, further supporting the validity and effectiveness of our automated thought segmentation approach. We will explicitly report this additional validation result in the revised manuscript. Once again, we appreciate your valuable feedback.

---

Q3: Effectiveness of the proposed TIP method.

A3: We have carefully addressed your points by conducting additional experiments to demonstrate the effectivenss and universality of the proposed TIP method. Below, we summarize our improvements with respect to your main comments:

To further clarify the effectiveness of TIP, we conducted analyses on key reasoning metrics, specifically examining:

• the average number of thought-switching tokens;
• the length of intervals between thought switches.

To examine TIP's generality explicitly, we reused hyperparameters tuned initially on the QwQ-32B-Preview model directly for the subsequent models — R1-Distill-Qwen-32B and DeepSeek-R1— without additional tuning.

Our analysis across multiple benchmarks consistently showed that TIP significantly reduces the overall number of thought-switching tokens while increasing the average intervals between them. This suggests the models become more committed to exploring individual reasoning threads thoroughly before pivoting to alternatives, aligning with our design intent of addressing underthinking explicitly.

In addition, TIP consistently and effectively enhanced reasoning performance across different models (R1-Distill-Qwen-32B, DeepSeek-R1) even without model-specific hyperparameter tuning. These results demonstrate both the universality and the empirical robustness of TIP.

To thoroughly assess the robustness and generality of our TIP approach, we combined it with best-of-N sampling methods, and the results confirm that our method remains consistently effective under these conditions.

Q1: More experimental results & analyses.

A1: Thank you for your insightful feedback and constructive suggestions. We have carefully addressed your points in the revised manuscript and conducted additional experiments to substantiate our claims. Below, we summarize our improvements with respect to your main comments:

To further clarify the effectiveness of TIP, we conducted analyses on key reasoning metrics, specifically examining:

• the average number of thought-switching tokens;
• the length of intervals between thought switches.

To examine TIP's generality explicitly, we reused hyperparameters tuned initially on the QwQ-32B-Preview model directly for the subsequent models — R1-Distill-Qwen-32B and DeepSeek-R1— without additional tuning.

Our analysis across multiple benchmarks consistently showed that TIP significantly reduces the overall number of thought-switching tokens while increasing the average intervals between them. This suggests the models become more committed to exploring individual reasoning threads thoroughly before pivoting to alternatives, aligning with our design intent of addressing underthinking explicitly.

In addition, TIP consistently and effectively enhanced reasoning performance across different models (R1-Distill-Qwen-32B, DeepSeek-R1) even without model-specific hyperparameter tuning. These results demonstrate both the universality and the empirical robustness of TIP.

To thoroughly assess the robustness and generality of our TIP approach, we combined it with best-of-N sampling methods, and the results confirm that our method remains consistently effective under these conditions.

---

Q2: How does the proposed underthinking metric perform on tasks that require creative or divergent thinking, where the notion of a single "correct" answer may not be well-defined?

A2: We fully acknowledge the importance and potential value of broadening our evaluations beyond purely math and science tasks. Your thoughtful comment highlights critical considerations regarding the broader relevance of the underthinking concept and the TIP method.

In this work, closely following the technical approach in the DeepSeek-R1 study [1], we concentrated primarily on reasoning-intensive quantitative/formal knowledge tasks, as these tasks are directly aligned with clearly defined correctness criteria and enable systematic measurement and analysis of underthinking behaviors in a reproducible manner.

We recognize that exploring more diverse reasoning tasks, such as logical reasoning and common-sense scenarios, could amplify our method's scope and impact. While tasks requiring creative or open-ended thinking represent valuable but challenging new directions (due to ambiguous evaluation criteria and limitations in large-scale reinforcement learning methods), we aim to pursue these avenues in follow-up studies. Carefully investigating these scenarios will indeed help establish clearer boundaries regarding when TIP is beneficial, when frequent thought switching might be advantageous, and under which conditions it might even be necessary.

We will explicitly discuss this direction as an important avenue for future work in the revised manuscript. For now, the current scope is clearly declared and motivated as an initial, controlled investigation into the robustness and effectiveness of addressing underthinking in reproducibly measurable tasks.

[1] DeepSeek. "Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning." arXiv preprint arXiv:2501.12948 (2025).

Q1: More insights about how TIP works and the generality of TIP.

A1: Thank you for your insightful questions and constructive feedback. We fully agree that more in-depth insights into why TIP enhances models' reasoning depth and accuracy would significantly reinforce our method.

To further clarify the effectiveness of TIP, we conducted analyses on key reasoning metrics, specifically examining:

• the average number of thought-switching tokens;
• the length of intervals between thought switches.

To examine TIP's generality explicitly, we reused hyperparameters tuned initially on the QwQ-32B-Preview model directly for the subsequent models — R1-Distill-Qwen-32B and DeepSeek-R1— without additional tuning.

Our analysis across multiple benchmarks consistently showed that TIP significantly reduces the overall number of thought-switching tokens while increasing the average intervals between them. This suggests the models become more committed to exploring individual reasoning threads thoroughly before pivoting to alternatives, aligning with our design intent of addressing underthinking explicitly.

In addition, TIP consistently and effectively enhanced reasoning performance across different models (R1-Distill-Qwen-32B, DeepSeek-R1) even without model-specific hyperparameter tuning. These results demonstrate both the universality and the empirical robustness of TIP.

---

Q2: I suggest the authors include a case study or a clearer explanation and analysis of Figure 2 to better illustrate how TIP affects reasoning depth and thought switching in practice.

A2: Thank you for this valuable suggestion. Due to length constraints, we did not include the complete response within the paper nor clearly specified whether each thought was promising. In the revised manuscript, we will provide the full response in an appendix and enhance the clarity and interpretability of Figure 2 accordingly.

Thank you for the valuable suggestions, which helped clarify and strengthen the paper.

Q1: Provide the code for reproduction.

A1: We will make our code publicly available soon.

---

Q2: The term "o1-like models" isn't precisely defined.

A2: Thank you for pointing this out. We acknowledge that our original description of “o1-like” models lacks precision. A clearer term would be “Long Reasoning Models”, which refers to models that generate detailed CoT reasoning by iteratively producing intermediate reasoning steps and sequentially refining solutions until reaching a final conclusion.

---

Q3: More details about the temperatures.

A3: In our experiments, we set the temperature to 0.7 and the top-p value to 0.95.

We followed your suggestion and conducted a more detailed analysis regarding the impact of temperature settings. The table below summarizes the results of QwQ-32B-Preview evaluated on the Math Level 1 and 5. Clearly, the temperature hyperparameter has only a marginal effect on the number of generated thoughts, further confirming our assertion that "underthinking" is a fundamental behavior inherent to these models.

---

Q4: There is no formal definition of the "Underthinking" issue.

A4: We recognize the terminology requires clarity. We clarify definitions explicitly:

• Underthinking (formal definition): Defined in Section 2.3 as: "Underthinking occurs when models generate potentially correct intermediate reasoning thoughts initially, but prematurely shift away without thoroughly exploring and continuing these promising reasoning paths, eventually yielding incorrect answers."
• Different Reasoning Thoughts: Defined explicitly in Section 2.1 as: "Individual cognitive steps within a model’s reasoning strategy. These transitions often explicitly indicated through terms like "alternatively"."
• Promising Paths: Defined in Section 2.2 as: Reasoning trajectories identified as promising based on intermediate correctness evaluations. We describe our detailed implementation and assessment strategies for thought correctness clearly in that section.

---

Q5: The percentage of underthinking issue in all the wrong responses.

A5: We presented in Table 1 the quantitative results of the underthinking score, which focuses the token efficiency specifically on the wrong responses. The results show that all Long Reasoning Models suffer from significant underthinking issues.

---

Q6: Figure 2 needs more explanations.

A6: The final correct answer for the case in Figure 2 is 23. Thoughts #1,3,4,5 are promising and can lead to the correct answer if explored sufficiently.

---

Q7: The insight is limited and the overall observations are superficial.

A7: In this pioneering work, our main contributions include:

• Clearly defining and evaluating underthinking in long reasoning models.
• Introducing quantitative evaluation measures.
• Proposing a simple yet effective decoding approach that significantly mitigates underthinking on multiple reasoning benchmarks such as Math500, GPQA Diamond, and AIME2024.

At this stage, we acknowledge that the existing community lacks complete open-source reproductions of large-scale long reasoning models, making it prohibitively expensive to empirically investigate training-related foundational causes. Therefore, we do not claim contributions toward understanding the training mechanism behind underthinking. However, clearly defining the issue and presenting empirical mitigation techniques is a valuable foundational step that future studies can further build upon toward fundamental analyses.

---

Q8: More convinced analyses.

A8: We conducted extensive experiments across multiple models on the AIME24 benchmark. The results demonstrate that our proposed approach significantly reduces both the average number of thoughts generated and the average length of each thought, underscoring the broad effectiveness of our method.

To thoroughly assess the robustness and generality of our TIP approach, we combined it with best-of-N sampling methods, and the results confirm that our method remains consistently effective under these conditions.