Thoughts Are All Over the Place: On the Underthinking of Long Reasoning Models

Thank you for your strong endorsement (rating 5) and for noting our clean motivation, experiments, and TIP's effectiveness in improving performance and efficiency. We appreciate your detailed questions on metrics and implementation — they've helped clarify key aspects.

---

Q1: My main concern with this paper is the proposed UT score. If I understand correctly, the score measures the percentage of tokens required to achieve the first correct thought, but not how frequently the model switches between thoughts. It is still possible for a model to quickly reach the first correct thought (therefore having a low UT score), but then switch between different thoughts afterward. Therefore, I’m not sure if the UT score correctly captures what the authors intended to measure.

A1: Thank you for the opportunity to clarify. The UT score is intentionally designed to measure the inefficiency caused by switching after a correct path has been found, not just the raw frequency of switching. A model that quickly finds a correct thought (low initial token use) but then switches away from it frequently will receive a high UT score, as many tokens are "wasted" after the first point of success. Conversely, a model that finds a correct thought and develops it to the final answer will have a UT score near zero. This design directly operationalizes our definition of underthinking: prematurely abandoning a promising path. We apologize if this was not perfectly clear and will refine the description in Section 2.3.

---

Q2: Additionally, based on Table 1, there is not a clear correlation between accuracy and the UT score, so it is unclear if the proposed metric can provide useful insights on the relation between underthinking issues and model performance. Instead, I think the "switch tokens" number and the interval measure in Table 2 would be more informative.

A2: This is a very insightful observation. You are correct that there isn't a simple, universal correlation across different model families in Table 1, and for good reason.

1. UT Score vs. "Switch Tokens": As you note, other metrics like "switch tokens" are informative, and we include them for this reason. However, switching itself is not always bad; abandoning an incorrect path is a feature of good reasoning. The UT score is more nuanced because it specifically penalizes abandoning correct paths.
2. Cross-Family vs. Intra-Family Correlation: Different model families may use fundamentally different strategies (e.g., some are trained for iterative refinement, others for more direct solutions). This can lead to high accuracy with either high or low UT scores, masking a direct correlation. However, as shown in our new results for Reviewer-Ffc3, within a single model family (Qwen3), there is a clear negative correlation between model size/accuracy and the UT score. This strongly suggests that for a given architecture and training style, reducing underthinking (lower UT score) is aligned with better performance. We will add this crucial discussion to the paper.

---

Q3: The TIP decoding strategy requires a predefined set of thought-switching tokens. How are these tokens selected? What if a word is tokenized into multiple tokens?

A3: That's a great practical question.

1. Selection: The tokens (e.g., alternatively, messy) were selected empirically by observing common phrases that preceded a strategy switch in the models' outputs. While they function as tunable hyperparameters, we found these general terms worked well across tasks without extensive tuning.
2. Tokenization: When a switching keyword is split into multiple subwords, our implementation penalizes only the first subword token of that keyword. This ensures the penalty is applied once per intended switch.

We will clarify these implementation details in the paper.

---

Q4: The legend for Figure 5 is confusing. There are dots in the figure that look different from the rest. What is the difference between these dots and the others?

A4: Thank you for pointing this out, and we apologize for the confusion. The dots in Figure 5 represent individual thoughts, plotting their position in the reasoning chain against the probability of being abandoned. The visually distinct dots are simply statistical outliers and do not represent a different category of data. They do not change the overall trend shown by the regression line. We will revise the figure caption in the final version to explicitly state this and improve clarity for the reader.

Thank you for your encouraging review (rating 4) and for emphasizing strengths like our insightful observation of underthinking, the novel UT metric, TIP's practical efficiency gains, and our qualitative examples. We value your suggestions for broader coverage — they've pushed us to expand our analysis.

---

Q1: Limited Model-Size Coverage

A1: This is an excellent point. To investigate whether underthinking is specific to very large models, we ran new experiments on a range of smaller models from the Qwen3 family (4B, 8B, 14B, 32B) on the AIME24 benchmark.

Our findings are twofold and very insightful:

1. Underthinking is a general phenomenon: It occurs consistently across all tested model sizes, not just the largest ones.
2. Underthinking severity correlates with model scale: Within the Qwen3 family, smaller models exhibit more severe underthinking than their larger counterparts. For instance, the 4B model has a much higher UT Score than the 32B model.

This new analysis significantly strengthens our paper by demonstrating the broad relevance of our work across model scales. We will add a new section with these results.

---

Q2: Restricted Evaluation Scope

A2: Thank you for the valuable suggestion. Inspired by your feedback, we conducted new experiments on a multimodal benchmark.

Our analysis toolkit generalizes well to multimodal reasoning. We evaluated two multimodal LRMs (QVQ-72B-Preview, GLM-4.1V-Thinking) on the OE_MM_maths_en_COMP subset of OlympiadBench, which requires reasoning over both text and images. The results clearly show that the underthinking phenomenon persists in this more complex, multimodal setting.

This confirms that underthinking is a broad challenge not limited to text-only math problems. We will add these new results and a discussion to the final manuscript to strengthen the paper's general applicability.

Thank you for your supportive review and for praising the paper's clarity, the interesting concept of underthinking as a lens on LRMs, and our thorough analysis with convincing evidence. We're grateful for your detailed questions, which helped us refine our insights.

---

Q1: Table 2 is missing results for R1-Distill-Qwen-32B and Deepseek-R1 on MATH500-hard and GPQA Diamond. Also, from the table, the results suggest that TIP yields more substantial accuracy improvements for models with lower baseline performance (such as QwQ on GPQA and AIME-24). Do you have any insight on this?

A1: Thank you for pointing this out. We have updated Table 2 with new results.

1. New Results: We have added results for R1-Distill-Qwen-32B on MATH500-hard and GPQA Diamond. These results reinforce our claims: the model exhibits underthinking, and our TIP method successfully mitigates it to improve accuracy. Due to the limited rebuttal period and high computational cost, we were unable to complete the runs for DeepSeek-R1 but will ensure they are included in the final camera-ready version.
2. Insight on Performance Gains: That’s a sharp observation. We hypothesize that models with lower baseline accuracy (like QwQ on GPQA) may have been trained on data that inadvertently encourages underthinking, or were simply less optimized against this specific failure mode. Consequently, TIP, which directly targets this issue, provides a larger relative boost. However, we emphasize that TIP consistently improves performance across all tested models, demonstrating its broad utility.

---

Q2: The paper does not clearly explain why underthinking behavior appears in LRMs. A further discussion of its possible causes would help improve understanding.

A2: Thank you for asking for clarification. We hypothesize that underthinking is an unintended byproduct of training LRMs to mimic complex, multi-path human reasoning. The causal chain is likely:

1. To enable deep reasoning, models like OpenAI's o1 demonstrated the value of exploring multiple reasoning paths.
2. This influenced the creation of supervised fine-tuning (SFT) datasets, which likely over-represented examples with frequent thought-switching to encourage this exploratory behavior.
3. As a result, models learn to switch paths too readily, even when a current path is promising. This leads to the premature abandonment of good ideas that we term "underthinking".

We will add this explicit explanation to the paper to provide better context.

---

Q3: The TIP method works by penalizing the logits of certain keywords (like “alternatively”) that indicate thought switching. But some of these keywords might also reflect other useful cognitive behaviors, as discussed in a related work. Do you think TIP might unintentionally suppress those behaviors too? Any insight or analysis would be helpful.

A3: That's a great question. No, we designed TIP specifically to avoid this. TIP does not penalize all thought-switching, only behavior indicative of underthinking: frequent, rapid, and unproductive switching. Specifically, TIP applies a penalty only when a switching keyword (e.g., "alternatively") appears with high frequency in a short span (in our work, ≥2 times within 600 tokens). An isolated, deliberate switch to a new strategy (like backtracking or reflection, as in [1]) would not be penalized. Therefore, TIP complements, rather than suppresses, productive reasoning strategies. We will clarify this important nuance in the revised manuscript.

[1] Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs.

Thank you for your positive assessment (rating 5) and for highlighting the novelty of our systematic characterization of underthinking in LRMs, our proposed SUT score (which you called a "valid metric"), and how it expands understanding beyond simple accuracy metrics. We appreciate your curiosity about extensions and are glad our work resonated with you.

---

Q1: The correct thought score is judged by the distilled LLMs, which may introduce bias.

A1: We agree this is a critical aspect and have already quantified the classifier's reliability in the paper. As detailed in Lines 131-137, we validated our LLM-based classifier on a manually annotated set. It achieves 82.9% accuracy on correct thoughts and 81.8% on incorrect thoughts. While not perfect, this level of accuracy is sufficiently high to reliably identify the trends central to our paper's claims, such as the correlation between frequent thought-switching and incorrect final answers.

We believe this establishes a strong baseline for this novel analysis task. We fully agree that developing more robust, model-independent evaluation methods is a valuable direction for future work, and we will emphasize this in the limitations section of the revised paper.

---

Q2: Rather than saying as a weakness point, I am a little bit curious that if this kind of analyse toolkit could be easily adapt to coding and multi-modal reasoning problems?

A2: Thanks for suggesting we explore the applicability of our findings beyond math reasoning. Inspired by your feedback, we conducted new experiments on a multimodal benchmark.

Our analysis toolkit generalizes well to multimodal reasoning. We evaluated two multimodal LRMs (QVQ-72B-Preview, GLM-4.1V-Thinking) on the OE_MM_maths_en_COMP subset of OlympiadBench, which requires reasoning over both text and images. The results clearly show that the underthinking phenomenon persists in this more complex, multimodal setting.

This confirms that underthinking is a broad challenge not limited to text-only math problems. We will add these new results and a discussion to the final manuscript to strengthen the paper's general applicability.