Think Only When You Need with Large Hybrid-Reasoning Models

We are grateful to the Reviewer hw6N for the extensive review. We address your questions point by point below.

---

Q1: How evaluations were done? Were all models evaluated with the same hyperparameters?

A1: Thank you for raising this question. We performed rigorous evaluations on our models across various benchmarks, ensuring that both NHTMs and baseline models were evaluated under the same experimental setup to guarantee a fair comparison. As detailed in Appendix D ("Evaluation Settings"), we consistently used a temperature of 0.6, a top-p value of 0.95, and a maximum generation length of 35,000 tokens for all models across all tasks. To ensure the stability and reliability of our results, the reported scores for all reasoning tasks are the averages of five independent runs.

For specific benchmark suites, we adhered to widely-accepted evaluation frameworks. MATH500, AIME24, and LiveCodeBench were evaluated using the OpenR1 evaluation framework, while MBPP and MBPP+ were evaluated using the EvalPlus framework. Further details are provided in Appendix D.

---

Q2: Can authors provide evaluations on GPQA-Diamond, HumanEval, HumanEval++ for fair comparison with other LRMs?

A2: Thank you for the valuable suggestion. We have conducted these additional evaluations to ensure a fair and comprehensive comparison at Table A and Table B. Our experimental results demonstrate that our NHTMs-1.5B and NHTMs-7B models outperform the DeepSeek-R1-Distill-Qwen counterparts on both HumanEval and HumanEval++ benchmarks, and achieves the highest average performance among all compared models.

Although our NHTMs-7B performs slightly lower on GPQA-Diamond compared to DeepSeek-R1-Distill-Qwen-7B, it still shows significant improvements over the HFT-1.5B baseline. This is particularly noteworthy because our training dataset explicitly excluded Science-related data, which is primarily the domain covered by the GPQA-Diamond benchmark (as illustrated in Appendix C, Table 2 and Table 3). Therefore, the observed performance indicates that our Hybrid Group Policy Optimization (HGPO) method effectively enhances generalization, allowing the model to achieve robust performance even on tasks outside its direct training distribution.

In summary, the additional evaluations further validate the effectiveness and generalization capability of our proposed model.

Table A: Performance Comparison on Comprehensive Benchmarks (1.5B size)

Table B: Performance Comparison on Comprehensive Benchmarks (7B size)

---

W1: No comparison with DeepScaleR-1.5B-Preview[1] although the DeepScaleR data was used.

A3: Thank you for your valuable feedback. We have now included the results of DeepScaleR-1.5B-Preview in the comparison table below, alongside NHTMs-1.5B for a fair evaluation across key benchmarks:

As shown in the table, our NHTMs-1.5B model achieves the highest average performance among all compared models, outperforming both DeepScaleR-1.5B-Preview and DeepSeek-R1-Distill-Qwen-1.5B.

In particular, NHTMs-1.5B achieves comparable results to DeepScaleR-1.5B-Preview on math-focused tasks such as MATH500, and demonstrates clear advantages on tasks requiring adaptive reasoning, including MBPP and AlpacaEval2.0. Although it underperforms on AIME24, we attribute this to the difference in training budgets—DeepScaleR was trained for 1.7k steps, whereas our model was only trained for fewer than 500 steps in this comparison.

These results highlight the efficiency and generalization capability of our hybrid-thinking framework, even under limited training. We appreciate your suggestion and will put DeepScaleR-1.5B-Preview as a comparison point in next version of our paper to provide a more comprehensive evaluation.

---

W2: No information on training data decontamination. E.g. AIME dataset[3] might contain samples from AIME24 benchmark.

A4: Thank you for pointing this out. Although we briefly mentioned the removal of overlapping samples in Section 2.2 ("Data Construction", line 96), we agree that a more detailed explanation of our decontamination process is necessary.

To ensure the fairness of our evaluation, we uniformly applied the Simhash algorithm [1] to calculate the similarity between each training sample and all samples across our evaluation benchmarks. Any instances with a similarity score greater than 0.8 were removed to strictly prevent data contamination.

Specifically regarding the AIME dataset, we took extra precautionary measures to avoid any temporal or content overlap with the AIME24 benchmark:

• Source Filtering: Our training data only included AIME problems from 2023 and earlier, thereby ensuring there was no temporal overlap with the AIME24 benchmark at the source level.

• Precise Deduplication: In addition to source filtering, we applied the aforementioned Simhash deduplication process against the AIME24 test set as an additional safeguard. As reflected in Table 2 of our paper, the initial AIME dataset contained 933 problems. After applying this rigorous decontamination process, 741 problems were retained for training, which further confirms the execution of our decontamination procedure.

Through these two measures—source control and precise deduplication—we have ensured there is no data leakage from the AIME24 benchmark. We will add a more detailed description of our decontamination process to the appendix in the final version to improve the transparency and reproducibility of our work.

Reference

[1] Detecting near-duplicates for web crawling. Manku, Gurmeet Singh and Jain, Arvind and Das Sarma, Anish.

---

W3: Grammar errors ("to lean a effective NHTMs").

A5: Thank you for pointing out the grammatical error, and we will correct the grammar in the latest version of the paper.

---

We want to express our sincere gratitude for your review. If you have any further questions or concerns, please feel free to contact us at any time. We are always available and look forward to further discussions with you. :)

Best regards,

All Authors

Dear Reviewer hw6N,

Thank you for taking the time to engage with our rebuttal and provide valuable feedback. We sincerely appreciate your efforts in considering our responses and the additional experiments.

We hope that the supplementary experiments and clarifications we have provided address all your concerns. If any issues remain unresolved or if you have further questions, please feel free to contact us at any time. We would be happy to engage in further discussion and provide any additional clarifications as needed. :)

Thank you once again for your time and effort in reviewing our work.

Best regards,

All Authors

Q1.: In Equation (10), I am particularly interested in the term “GRPO for inter-group advantage .” However, I do not fully understand how this term effectively contributes to the decision of whether to think or not. I would appreciate it if the authors could provide a more detailed explanation. It may also be helpful for other readers if the authors include additional clarification or discussion of this term in the paper.

A1. Thank you for your valuable suggestion. We agree that the “GRPO for inter-group advantage” term in Equation (10) plays a crucial role and deserves a more detailed explanation. To clarify, the process unfolds as follows:

1. Scoring responses (Eq. 7): For each query, we sample both Thinking and No-Thinking responses, and score them using the reward model.
2. Computing average rewards (Eq. 8): We then compute the average reward for both reasoning modes across these samples.
3. Selecting the better mode: By comparing the two average rewards, we identify which reasoning mode—Thinking or No-Thinking—is more appropriate for the given query.
4. Assigning learning signals (Eq. 9): We then get inter-group reward by assigning a high reward (1) to responses sampled from the better mode and a low reward (0) to those from the less effective mode.
5. Learning via GRPO (Eq. 10): The “GRPO for inter-group advantage” then learns from this inter-group reward contrast, encouraging the model to favor the reasoning mode that is more beneficial for a specific query.

We will revise the manuscript to enhance the explanation of this term and its role in reasoning-mode selection, as we believe this will indeed help readers better understand our method. Thank you again for your insightful comment.

W1: In Figure 5, the y-axis label for "Thinking Ratio" appears to be missing.

A2：Thank you for your careful review and for pointing out the issue with Figure 5. We agree with your observation — the "Thinking Ratio" and "Accuracy" currently share the same y-axis, but the label for "Thinking Ratio" was inadvertently omitted. This was an oversight on our part. In the revised version of the paper, we will explicitly add the missing label to ensure clarity and avoid any potential confusion for readers. We appreciate your attention to detail in helping us improve the quality of our paper.

W2: A key contribution of the proposed NHTMs is their ability to autonomously decide whether to think, thereby saving computational budget. It would be more intuitive to include the average number of output tokens for both the baselines and NHTMs across all benchmarks.

A3: As suggested, we have provided a detailed comparison of the output lengths across all benchmarks in Tables 1 and 2, alongside the corresponding accuracy results. In each cell, the number in parentheses indicates the average number of output tokens. These tables cover the proposed NHTMs (Hybrid-Thinking Models), the full-thinking baselines (HFT-1.5B-Think and HFT-7B-Think), and the DeepSeek-R1-Distill-Qwen models.

• Across most benchmarks, the hybrid-thinking models consistently produce shorter outputs while maintaining equal or better performance, demonstrating their ability to reduce unnecessary reasoning for simpler queries and thus improve inference efficiency.
• On the more challenging AIME24 benchmark, the hybrid-thinking models generate longer outputs compared to both the thinking-only and DeepSeek models, as they engage in more elaborate reasoning to solve complex problems. This additional reasoning leads to significantly improved performance—NHTMs achieve the best accuracy at both the 1.5B and 7B model sizes, with the 7B variant outperforming DeepSeek-R1-Distill-Qwen by 8 percentage points.

These results clearly demonstrate that our hybrid-thinking model achieves improved inference efficiency by adaptively reducing unnecessary reasoning steps for simpler queries.

Table 1. Accuracy and Output Length Comparison on Various Benchmarks (1.5B size)

Table 2. Accuracy and Output Length Comparison on Various Benchmarks (7B size)

W3: In my understanding, Hybrid Accuracy is a model-specific metric, as $m_{gt}$ is determined by the evaluated LLM. If multiple NHTMs are evaluated, the proposed Hybrid Accuracy metric may not fairly compare them.

A4: Thank you for the insightful comment. We agree with your observation that Hybrid Accuracy is indeed a model-specific metric, as it directly depends on the selection strategy employed by the evaluated hybrid model.

We would like to clarify that Hybrid Accuracy is not intended as an absolute metric to evaluate overall model performance across different models. Instead, it specifically reflects the effectiveness of a given model in making correct choices between the "thinking" and "no-thinking" modes within its own decision-making mechanism.

For example, consider a hypothetical hybrid model that always performs better in the "thinking" mode compared to the "no-thinking" mode (perhaps due to insufficient training in the "no-thinking" mode). If this model always selects "thinking," it will achieve a high Hybrid Accuracy score, but this does not necessarily indicate that the model itself has superior overall capabilities—it merely signifies that it correctly identifies its optimal internal mode.

Conversely, if this model mistakenly prefers the "no-thinking" mode under similar conditions, the Hybrid Accuracy score will be low, clearly indicating poor internal decision-making. Thus, Hybrid Accuracy primarily assesses the model’s internal decision logic rather than its absolute performance on tasks.

In the revised version of our paper, we will explicitly clarify this aspect and provide illustrative examples to better convey the scope and intention of the Hybrid Accuracy metric.

---

We sincerely thank the Reviewer ofyf for the thoughtful comments and constructive suggestions :). Your insights have been invaluable in improving the clarity and completeness of our work.

If you have any further questions or concerns, please feel free to contact us at any time. We are always available and look forward to further discussions with you. :)

Best regards,

All Authors

Thank you very much for your positive feedback! We are thrilled to hear that all of your concerns have been well addressed. We will take your advice into account in the latest version to further improve the clarity and quality of our work.

We truly appreciate your time and thoughtful review.

Best regards,

All Authors

We are grateful to the Reviewer for the extensive review. We address your questions point by point below.

Q1: It is surprising that adaptive thinking shows significantly greater improvement on AIME 2024 than on Math500. Could you compare the thinking ratio and thinking token length changes between MATH500/AIME 2024 to explain this phenomenon?

A1: We compared the thinking ratio and average reasoning length across MATH500 and AIME24 (see Table 1). The results are consistent with the intuition that simple problems benefit more from reasoning compression, while difficult problems require full reasoning.

On MATH500, which contains many relatively simple problems, NHTMs significantly reduce both the thinking ratio and the average reasoning length, indicating that the model effectively skips unnecessary reasoning by selecting the No-Thinking mode. This improves efficiency without sacrificing accuracy. In contrast, on the more challenging AIME24, the model applies 100% Thinking with slightly longer reasoning chains, reflecting its consistent use of detailed step-by-step reasoning where it is truly needed.

Together, these trends explain the greater performance gains on AIME24: the model allocates full reasoning to difficult tasks while leveraging No-Thinking to bypass reasoning for simpler ones.

Table 1: Comparison of thinking ratio (%) and average reasoning length (tokens) between HFT and NHTMs on MATH500 and AIME24.

---

W1: Direct comparison with DeepSeek-R1-Distill-Qwen-7B/1.5B & NHTMs-7B/1.5B is insufficient to demonstrate the method's effectiveness....as you have expended additional computational resources that might enable the model to reach a significantly higher ceiling, making adaptive CoT no longer cost-effective. This point requires experimental validation.

A2: We thank the reviewer for pointing out the need to isolate the effect of adaptive thinking from computational budget. To address this, we conducted additional experiments where baseline models (HFT-1.5B and HFT-7B) were trained with standard GRPO using the exact same RL dataset (76K queries from DeepScaleR and Tülu3) and training setup as our HGPO method.

As summarized in Table B, both training strategies yield similar performance on reasoning-intensive tasks. However, HGPO-trained models consistently outperform GRPO-trained counterparts in general tasks, leading to higher overall performance. We attribute this to HGPO’s adaptive selection constraint, which encourages more effective use of the "no-think" mode on simpler queries, and preserves reasoning capacity for complex problems. In contrast, GRPO lacks this adaptive mechanism and underutilizes the "no-think" mode during rollouts, limiting gains on general tasks.

Moreover, HGPO models achieve significantly higher hybrid accuracy ($H_{\text{acc}}$), demonstrating stronger ability to select the appropriate reasoning mode per query. This confirms that adaptive thinking not only avoids additional computational cost, but also enhances the model's effectiveness across diverse tasks.

These results validate the value of HGPO in building a more efficient and capable hybrid reasoning system under equivalent training resources.

Table B: Performance Comparison of Standard GRPO vs. HGPO

---

W2: Reporting the thinking token length statistics for each dataset and comparing with CoT compression works would strengthen the work.

A2: We thank the reviewer for the insightful suggestion on analyzing thinking token lengths and comparing with existing CoT compression methods [1–3]. In response, we present detailed results in Table C, including average reasoning lengths and corresponding accuracies:

• Compared to the full-length thinking baselines (HFT-1.5B-Think and HFT-7B-Think), our NHTMs significantly reduce token lengths across most tasks while achieving comparable or better accuracy. This confirms the efficiency and effectiveness of our adaptive reasoning strategy.

• When compared with prior CoT compression methods—O1-Pruner, CoT-Valve, and DAST—our approach achieves a better balance between compression and accuracy. Although some methods (e.g., O1-Pruner, DAST) yield shorter outputs, they often suffer from performance drops. In contrast, NHTMs maintain or improve accuracy with meaningful reductions in reasoning length.

These results validate the practical advantage of our adaptive CoT framework for efficient and reliable inference. We wil add these results into our newest version.

Table C. Comparison of Accuracy and Average Thinking Token Length (in Parentheses) Across CoT Compression Methods and Model Variants

---

W3: Tulu-3-8B-RM is used both for RL training and for computing Hybrid Accuracy. Could this lead to unfair Hybrid Accuracy evaluation?

We appreciate the reviewer’s thoughtful question. To ensure the fairness and reliability of our Hybrid Accuracy (HAcc) evaluation, we have additionally computed HAcc using both GPT-4o annotations and human annotations. For the human annotations, we sampled 100 queries per model and engaged three annotators, each labeling 800 samples independently.

As presented in Table D, the results indicate that the relative ordering (partial ordering) of models based on Hybrid Accuracy remains consistent across annotations derived from GPT-4o, human evaluators, and Tulu-3-8B-RM. All annotation methods confirm that NHTMs-7B and NHTMs-1.5B achieve substantial improvements in selecting the correct reasoning mode.

Furthermore, we conducted an agreement analysis across these annotation approaches (see Table E). The average agreement between human annotators was 60%, while the agreement between human annotators and Tulu-3-8B-RM was slightly higher at 62.5%. Notably, GPT-4o demonstrated the highest agreement rates, comparable to previously reported agreement levels in MT-Bench [4] and UltraFeedback [5]. These results collectively validate the reliability and fairness of employing Tulu-3-8B-RM for Hybrid Accuracy evaluation.

Table D: Hybrid Accuracy (%) of Different Models

Table E: Agreement Rates (%) Between Annotators

[4] Cui, Ganqu, et al. "Ultrafeedback: Boosting language models with scaled ai feedback."

[5] Xu, Jiazheng, et al. "Imagereward: Learning and evaluating human preferences for text-to-image generation."

---

We sincerely thank the Reviewer jtLz for the thoughtful comments and constructive suggestions :). If you have any further questions or concerns, please feel free to contact us at any time.

Best regards,

All Authors

Thank you for the detailed experimental supplements provided by the authors. Most of my concerns have been resolved. Based on this, I will increase my confidence score and support the acceptance of this paper.

However, I have an additional question. According to your response to Q1, the Think Ratio for AIME2024 is 100%, which means the model does not switch reasoning mode per query when solving AIME2024 problems. Given this, why does NHTMs-7B still show improvements over HFT-GRPO-7B on AIME2024 in Table B?

We are grateful to the Reviewer for the extensive review. We address your questions point by point below.

Q1: In Table 2 in Appendix C, is WildChat-1M used as the Thinking dataset, and the rest used for Non-Thinking?

A1: In Appendix C, Table 2, WildChat-1M is actually used as the Non-Thinking dataset, while the remaining datasets are employed as the Thinking dataset. We apologize for any confusion caused and will clarify this point explicitly in the latest version of our manuscript.

---

Q2: Is the hybrid-thinking model more efficient compared to the thinking model, i.e., by using less output tokens?

A2: We have compared the output lengths between the proposed NHTMs (hybrid-thinking models), the thinking-only models (NHTMs-Thinking), and the DeepSeek-R1-Distill-Qwen model in Table 1 & 2. We find that:

• Across most benchmarks, the hybrid-thinking models consistently produce shorter outputs while maintaining equal or better performance, demonstrating their ability to reduce unnecessary reasoning for simpler queries and thus improve inference efficiency.
• On the more challenging AIME24 benchmark, the hybrid-thinking models generate longer outputs compared to both the thinking-only and DeepSeek models, as they engage in more elaborate reasoning to solve complex problems. This additional reasoning leads to significantly improved performance—NHTMs achieve the best accuracy at both the 1.5B and 7B model sizes, with the 7B variant outperforming DeepSeek-R1-Distill-Qwen by 8 percentage points.

These results clearly demonstrate that our hybrid-thinking model achieves improved inference efficiency by adaptively reducing unnecessary reasoning steps for simpler queries.

Table 1. Output Length Comparison Across Hybrid, Thinking-Only, and DeepSeek-R1-Distill-Qwen Models on Various Benchmarks (1.5B size)

Table 2. Output Length Comparison Across Hybrid, Thinking-Only, and DeepSeek-R1-Distill-Qwen Models on Various Benchmarks (7B size)

---

Weaknesses1: The authors do not compare the proposed hybrid thinking model with an equivalent non-hybrid thinking model. The baseline model in Table 1, DeepSeek-R1-Distill-Qwen, may use a different training protocol from that of the proposed hybrid thinking models.

A3: We appreciate the reviewer’s concern and have conducted an additional experiment for a direct comparison. Specifically, we fine-tuned both the 1.5B and 7B Qwen-2.5-Math models on ~1.1M math and code reasoning data in Thinking-style format, and compared them against our hybrid-trained models under controlled settings. The two model types are defined as follows:

• PureThink-SFT models:
  These models are trained exclusively on ~1.1M math and code Thinking-style data using standard supervised fine-tuning. No hybrid formatting or No-Thinking data is involved.

• HFT (Think Only) models:
  These models are trained via our Hybrid Fine-Tuning (HFT) pipeline, which includes both Thinking and No-Thinking data. During evaluation, they are constrained to always generate responses in the Thinking mode.

As shown in Table 3, both variants achieve highly similar performance across multiple reasoning benchmarks at both 1.5B and 7B scales. This demonstrates that hybrid thinking does not degrade performance on reasoning-intensive tasks, validating the soundness of our hybrid training design.

Table 3. Comparison of Hybrid and Non-Hybrid Models on Reasoning-Intensive Tasks

---

Suggestions1: Some parts of the method explanation could be simplified to help readers better understand

A4: We sincerely thank the reviewer for this valuable suggestion. We fully agree that simplifying parts of the method explanation, such as equation (5), using plain language can significantly improve reader understanding. As suggested, we will revise equation (5) and other parts of the method explanation in the latest version to improve clarity and help readers better understand our method.

---

Suggestions2: (Future work) Consider allowing the model to reason before deciding on or <no_think> instead of making the decision at the first decoding step

A5: We appreciate the insightful idea! Indeed, enabling the model to first reason about the context and complexity of the query could potentially enhance the accuracy of selecting the appropriate thinking mode, thereby improving both the overall performance and efficiency. We plan to explore this approach in future work to further improve our hybrid thinking model. Meanwhile, we will include a discussion of this potential enhancement in the latest version of the manuscript.

---

We sincerely thank the Reviewer tCEC for the thoughtful comments and constructive suggestions :). Your insights have been invaluable in improving the clarity and completeness of our work. In response, we will incorporate the additional comparisons, clarify the dataset usage, simplify the method description, and include a discussion of this direction in the future work section of the latest version of our manuscript.

If you have any further questions or concerns, please feel free to contact us at any time. We are always available and look forward to further discussions with you. :)

Best regards,

All Authors

We sincerely appreciate your thoughtful response and kind recognition of our additional experiments. We are very glad to hear that all your concerns have been satisfactorily addressed.

As suggested, we will incorporate the supplementary experiments and analyses you recommended into the latest version of our manuscript.

Thank you again for your valuable insights and positive feedback.

Best regards,

All Authors