We sincerely thank you for the encouraging and constructive feedback. We are glad that you found the motivation, prompt insight, and stage-wise design of AutoThink to be elegant and effective. We address your concerns below:

1. On the Generality of the Ellipsis Prompt and Applicability to Other Models

Our method is specifically designed for Large Reasoning Models (R1-style models),

Thank you for this valuable comment. We would like to clarify that our method specifically targets at Large Reasoning Models, where Thinking refers to generating a step-by-step reasoning trace within the <think> {thinking_content} </think> tag, typically involving continuous reflection and backtracking. This structured reasoning format has been adopted in recent large reasoning models with explicit <think>-style reasoning blocks to modularize thought and improve interpretability.

Among open-source models, DeepSeek-R1-Distill-Qwen is a representative R1-style distilled model trained from Qwen2.5 by using DeepSeek-R1-671B reasoning data, making it a natural choice for our main experiments.

To further validate generality, we have already included DeepScaleR in our experiments (Table 1), which is built upon DeepSeek-R1-Distill-Qwen-1.5B and strengthened via multi-stage context extension and RL. Even though DeepScaleR has undergone extensive fine-tuning to favor long reasoning chains, we observe that under the ellipsis prompt, the average token usage still drops from 5817 to 5511, suggesting that our prompt design still elicits a small amount of no-thinking behavior. Moreover, AutoThink stages applied on top of DeepScaleR demonstrate clear ability to further reduce token usage with minimal or no accuracy loss (see Table 1).

In addition, we conducted further validation on Skywork-OR1-Math-7B, a stronger RL-trained R1-style model that has undergone deeper optimization for mathematical and code reasoning. We report below the results on different prompting/training stages:

Although ellipsis prompting alone induces minimal no-thinking behavior (only ~0.5% occurrence), our method successfully encourages such behavior to emerge through structured reward shaping, reducing the average token usage from 8933 to 3966 in Stage 1 while maintaining comparable accuracy. During Stage-1 training, we observed a sharp drop in thinking ratio and response length around step 120, indicating a successful emergence of dual-mode balance.

Finally, due to time and compute constraints during the rebuttal phase, we were unable to run the full training pipeline on Qwen3. However, we conducted a preliminary evaluation to assess the effectiveness of ellipsis prompting, recording both accuracy and token length:

The use of the ellipsis prompt can encourage a certain degree of no-thinking behavior, and this behavior does not correlate with task difficulty (AIME problems are significantly harder than those in MATH500, yet elicit a lower thinking rate). Based on the experimental results from the above Skywork-OR1-Math-7B (about 0.5% occurrence of no-thinking behavior with Ellipsis Prompt), we believe that the AutoThink strategy could also induce autonomous reasoning in Qwen3 (~13% occurrence of no-thinking behavior with Ellipsis Prompt).

2. Hyperparameter Sensitivity

We appreciate your concern regarding the complexity of our method. We would like to clarify that the three-stage structure is intentionally modular and interpretable, with each component driven by distinct and simple design principles:

• Stage 1 introduces a batch-wise reward balance to prevent mode collapse between thinking and no-thinking behaviors.
• Stage 2 focuses purely on reinforcing accuracy within each mode (no additional reward shaping).
• Stage 3 adds length-aware shaping to encourage brevity for correct responses and elaboration for incorrect ones.

Among the three stages, only Stage 1 and Stage 3 involve reward design beyond naive correctness. Even then, the reward formulations are simple and intuitive:

• In Stage 1, we balance the modal ratio using a linear penalty with hyperparameters γ and λ. These parameters were not carefully tuned, but simply set to commonly used values. As shown in Figure 3, the reward curve exhibits a natural symmetric form.
• In Stage 3, we reuse default shaping terms (α, β) inspired by GRPO-LEAD, additional any revision.

We have already provided ablations in Section 4.3 and Figure 5, which show:

• Without Stage 1, the model collapses into a single mode (thinking or no-thinking).
• Skipping Stage 2 results in a clear drop in final accuracy (51.7% → 47.6%).

We acknowledge the lack of hyperparameter sweeps in the main paper. In additional experiments during the rebuttal phase, we find:

• Increasing γ encourages more thinking behavior in Stage 1.
• Higher λ enforces stricter adherence to the target balance.
• Larger α/β in Stage 3 accelerate reward decay/growth in response_length.

To illustrate these trends, we record the thinking ratio measured as the running average over the most recent 20 steps at two key checkpoints (step 100 and step 200) during Stage 1 training:

Additionally, we record the response lengths in Stage 3 under different values of α, with β fixed at 0.05:

These training trends are consistent with the intended behavior of the shaping functions. We will include these findings in the appendix and release them alongside the camera-ready version.

3. On the Reward Impact and Early Stopping in Figure 5a

We appreciate your question regarding the relationship between RL steps and accuracy in Stage 1. First, we would like to clarify that the primary objective of Stage 1 is to stabilize the coexistence of thinking and no-thinking modes, rather than to directly improve accuracy. The effects of different reward coefficients have already been discussed in detail in the previous section.

In Figure 5a, we observe that all three reward variants generally show upward trends. Among them, the naive reward increases slightly faster than the batch balance reward, as it implicitly encourages fully "thinking" responses to achieve higher accuracy.

In contrast, the length-aware reward exhibits a non-monotonic pattern: it decreases initially and then gradually increases. This trend aligns closely with the change in response length. Our interpretation is that the length-aware reward encourages shorter, more concise answers. In the early stages of training, when the model’s overall accuracy is still low, responses with short and no-thinking mode tend to receive higher rewards, leading the model to collapse into a no-thinking mode. As training progresses, however, longer no-thinking responses tend to be more accurate, guiding the policy to optimize toward generating longer responses, resulting in a moderate increase in response length.

We observed that after step 110, training under the length-aware reward leads to a stable response length range. However, since the model remains largely no-thinking single-modal, its accuracy is generally lower than that of models trained with the batch-balance or naive reward.

To improve clarity, we will include RL step–accuracy curves and more detailed discussions in the camera-ready version, offering a clearer picture of how each reward function influences learning dynamics.

We sincerely hope that these clarifications and additional results help address your concerns. If you have any additional feedback or suggestions, we would greatly welcome them and be happy to address them in detail.

Thanks for your feedback. You have clarified my concerns. I will raise my score.

We sincerely thank you for the positive and thoughtful evaluation of our work. We're especially grateful for your recognition of AutoThink’s contribution to improving reasoning efficiency and your appreciation of the progressive reward shaping design. We address your concerns below:

1. Generality Beyond Mathematical Reasoning

We appreciate the your concern regarding generalization beyond math tasks. In response, we have conducted additional evaluations of our AutoThink models on three non-mathematical benchmarks:

• GPQA for scientific multi-hop reasoning,
• MMLU for general multi-task language understanding,
• Live-Code-Bench for code generation, we use the newest 20250727 version.

The results show that AutoThink-trained models retain competitive accuracy while reducing average token usage, suggesting that the learned adaptive reasoning behaviors generalize beyond math.

Although these findings are preliminary, they suggest that math-domain training can endow LLMs with transferable control over reasoning depth. In future work, we will explore domain-specific training to further strengthen AutoThink’s adaptability in broader tasks.

2. Complexity of the multi-stage RL

Regarding your comment on the complexity of the multi-stage RL pipeline and tailored reward functions, we would like to emphasize that each component was introduced to serve a clear, intuitive purpose—as you summarized precisely:

• Stage 1 ensures balanced coexistence between thinking and no-thinking modes via a simple linear penalty mechanism. The associated hyperparameters (γ, λ) were not carefully tuned, but set to commonly used values. As shown in Figure 3, the reward curve is naturally symmetric and stable.
• Stage 3 applies a length-aware shaping strategy using default (α, β) values adopted directly from GRPO-LEAD.

Thus, while the overall pipeline appears multi-stage, each stage was deliberately designed with minimal tuning and clear interpretability.

Looking forward, we believe it is possible to unify these stages through a more holistic reward formulation, enabling the model to learn adaptive reasoning behavior in a single-stage process. This is an exciting direction we intend to pursue in future work.

If you have any additional feedback or suggestions, we would greatly welcome them and be happy to address them in detail.

We thanks for your thoughtful feedback and valuable suggestions. We have carefully addressed each concern and added clarifications, analyses, and new results to strengthen our work.

1. Definition of No-Thinking

Thank you for pointing this out. In R1-style models (e.g., DeepSeek-R1), Thinking refers to generating a step-by-step reasoning trace within the <think> {thinking_content} </think> tag, typically involving continuous reflection and backtracking. This structured reasoning format has been adopted in recent large reasoning models with explicit <think>-style reasoning blocks to modularize thought and improve interpretability.

For the No-Thinking mode [1], the model immediately closing the <think> tag without producing any meaningful intermediate reasoning, e.g., generating a shallow structure like <think> </think> and proceeding directly to the final answer. It is also called as Non-Thinking mode in Qwen3 Technical Report. This phenomenon often emerges under our ellipsis prompt, which stochastically toggles the model between reasoning and shortcut modes. Our work leverages this property as a control signal to investigate adaptive reasoning behavior.

We apologize for not formalizing this definition clearly in the original submission and will include more detailed definitions, examples, and connections to related work in the camera-ready version.

[1] Reasoning models can be effective without thinking. arXiv:2504.09858.

2. Validation beyond DeepSeek-R1-Distill-Qwen

Thank you for this valuable comment. We would like to clarify that our method specifically targets at Large Reasoning Models. Among open-source models, DeepSeek-R1-Distill-Qwen is a representative R1-style distilled model trained from Qwen2.5 by using DeepSeek-R1-671B reasoning data, making it a natural choice for our main experiments.

To further validate generality, we have already included DeepScaleR in our experiments (Table 1), which is built upon DeepSeek-R1-Distill-Qwen-1.5B and strengthened via multi-stage context extension and RL. Even though DeepScaleR has undergone extensive fine-tuning to favor long reasoning chains, we observe that under the ellipsis prompt, the average token usage still drops from 5817 to 5511, suggesting that our prompt design still elicits a small amount of no-thinking behavior. Moreover, AutoThink stages applied on top of DeepScaleR demonstrate clear ability to further reduce token usage with minimal or no accuracy loss (see Table 1).

In addition, we conducted further validation on Skywork-OR1-Math-7B, a stronger RL-trained R1-style model that has undergone deeper optimization for mathematical and code reasoning. We report below the results on different prompting/training stages:

Although ellipsis prompting alone induces minimal no-thinking behavior (only ~0.5% occurrence), our method successfully encourages such behavior to emerge through structured reward shaping, reducing the average token usage from 8933 to 3966 in Stage 1 while maintaining comparable accuracy. During Stage-1 training, we observed a sharp drop in thinking ratio and response length around step 120, indicating a successful emergence of dual-mode balance.

Finally, due to time and compute constraints during the rebuttal phase, we were unable to run the full training pipeline on Qwen3. However, we conducted a preliminary evaluation to assess the effectiveness of ellipsis prompting, recording both accuracy and token length:

The use of the ellipsis prompt can encourage a certain degree of no-thinking behavior, and this behavior does not correlate with task difficulty (AIME problems are significantly harder than those in MATH500, yet elicit a lower thinking rate). Based on the experimental results from the above Skywork-OR1-Math-7B (about 0.5% occurrence of no-thinking behavior with Ellipsis Prompt), we believe that the AutoThink strategy could also induce autonomous reasoning in Qwen3 (~13% occurrence of no-thinking behavior with Ellipsis Prompt).

3. Why AutoThink underperforms DeepScaleR despite similar training recipe?

This is a reasonable concern. We emphasize that DeepScaleR and AutoThink are designed with different optimization objectives:

• DeepScaleR focuses solely on maximizing accuracy, leveraging a deep 3-stage RL training schedule with progressively extended context and a large computational budget (~2200 H100 GPU hours).
• In contrast, AutoThink is designed to trade off accuracy and efficiency, aiming to reduce unnecessary reasoning by learning when to think, just costs ~700 H100 GPU hours.

More importantly, we also apply AutoThink on top of DeepScaleR. As shown in Table 1, this leads to:

• A reduction in average token usage from 5817 to 5277,
• While maintaining comparable accuracy (from 56.7% → 57.3%).

This shows that our method can serve as a post-training compression module, applicable even to models like DeepScaleR that are already highly optimized. In principle, for excellent performance and efficiency, we can first train a model with the DeepScaleR recipe to maximize performance, and then apply AutoThink to further improve inference-time efficiency without significant performance drop.

4. Concern about too many design choices and simple ablation

We appreciate your concern regarding the complexity of our method. We would like to clarify that the three-stage structure is intentionally modular and interpretable, with each component driven by distinct and simple design principles:

• Stage 1 introduces a batch-wise reward balance to prevent mode collapse between thinking and no-thinking behaviors.
• Stage 2 focuses purely on reinforcing accuracy within each mode (no additional reward shaping).
• Stage 3 adds length-aware shaping to encourage brevity for correct responses and elaboration for incorrect ones.

Among the three stages, only Stage 1 and Stage 3 involve reward design beyond naive correctness. Even then, the reward formulations are simple and intuitive:

• In Stage 1, we balance the modal ratio using a linear penalty with hyperparameters γ and λ. These parameters were not carefully tuned, but simply set to commonly used values. As shown in Figure 3, the reward curve exhibits a natural symmetric form.
• In Stage 3, we reuse default shaping terms (α, β) inspired by GRPO-LEAD, additional any revision.

We have already provided ablations in Section 4.3 and Figure 5, which show:

• Without Stage 1, the model collapses into a single mode (thinking or no-thinking).
• Skipping Stage 2 results in a clear drop in final accuracy (51.7% → 47.6%).

We acknowledge the lack of hyperparameter sweeps in the main paper. In additional experiments during the rebuttal phase, we find:

• Increasing γ encourages more thinking behavior in Stage 1.
• Higher λ enforces stricter adherence to the target balance.
• Larger α/β in Stage 3 accelerate reward decay/growth in response_length.

To illustrate these trends, we record the thinking ratio measured as the running average over the most recent 20 steps at two key checkpoints (step 100 and step 200) during Stage 1 training:

Additionally, we record the response lengths in Stage 3 under different values of α, with β fixed at 0.05:

These training trends are consistent with the intended behavior of the shaping functions. We will include these findings in the appendix and release them alongside the camera-ready version.

We sincerely hope that these clarifications and additional results help address your concerns. We would be grateful if you would kindly consider reassessing our work in light of the new evidence presented.

Thank you for your time and thoughtful review of our response.

Regarding the concern about the Ellipsis prompt, in addition to DeepSeek-R1-Distill-Qwen, we validated the effectiveness of the Ellipsis prompt and three-stage RL across broader model families, including Skywork-OR1-Math and Qwen3-8B.

For boarder scenarios, we conducted additional evaluations of our AutoThink-trained models on three non-mathematical benchmarks (GPQA, MMLU, LiveCodebench). The results show that AutoThink-trained models retain competitive accuracy while reducing average token usage, suggesting that the learned adaptive reasoning behaviors generalize beyond math.

According to your suggestions, we analyzed the prompt selection (Standard prompt, Ellipsis Prompt and No-Thinking Prompt) , 3-stage training, hyperparameters in reward design (γ, λ and α/β). Our key findings are as follows:

Ellipsis Prompt Necessity: The Ellipsis Prompt is critical for triggering stochastic thinking/no-thinking modes, enabling adaptive reasoning.

Training Stage Ablation: Without Stage 1, the model collapses into a single mode (thinking or no-thinking); Skipping Stage 2 results in a clear drop in final accuracy (51.7% → 47.6%).

Hyperparameter Sensitivity:

• Increasing γ encourages more thinking behavior in Stage 1.
• Higher λ enforces stricter adherence to the target balance.
• Larger α/β in Stage 3 accelerate reward decay/growth in response_length.

Please let us know if we have addressed your concerns. We are more than delighted to have further discussions to improve our manuscript. If our response has addressed your concerns, we would be grateful if you could re-evaluate our work.

We sincerely thank you for the encouraging and constructive feedback. We are glad that you found the motivation, prompt insight, and stage-wise design of AutoThink to be elegant and effective. We address your concerns below:

1. Generality Beyond Mathematical Reasoning

We appreciate the your concern regarding generalization beyond math tasks. In response, we have conducted additional evaluations of our AutoThink models on three non-mathematical benchmarks:

• GPQA for scientific multi-hop reasoning,
• MMLU for general multi-task language understanding,
• Live-Code-Bench for code generation, we use the newest 20250727 version.

The results show that AutoThink-trained models retain competitive accuracy while reducing average token usage, suggesting that the learned adaptive reasoning behaviors generalize beyond math.

Although these findings are preliminary, they suggest that math-domain training can endow LLMs with transferable control over reasoning depth. In future work, we will explore domain-specific training to further strengthen AutoThink’s adaptability in broader tasks.

2. Reward Hacking, Token Budget, and Dataset Noise

We appreciate the your encouragement to further discuss the limitations we acknowledged. We agree that all three are non-trivial and active challenges in the area of reasoning control:

• Token-budget control has been partially addressed in prior works [1,2] by designing budget-aware reward functions that penalize long completions. Such formulations could be integrated with AutoThink to enforce a global compute constraint in a plug-and-play manner.

• For dataset noise, existing works ([3], etc.) shows that curriculum learning or filtering by answer correctness or difficulty can help improve learning efficiency. These strategies are orthogonal to our reward design and could further boost training quality when combined.

• As for reward hacking, where reasoning continues after the </think> tag. we believe this can be mitigated by explicitly penalizing reasoning-related patterns outside the <think> span, or by rewarding clean separation between thought and answer. These ideas can be incorporated into future iterations of our reward function.

Overall, we view these solutions as complementary and composable with our framework. We are excited to explore tighter integration of these mechanisms, and incorporate a more detailed discussion of these directions in the camera-ready version to enhance the completeness and forward-looking perspective of the paper.

[1] L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning. COLM 2025.

[2] ShorterBetter: Guiding Reasoning Models to Find Optimal Inference Length for Efficient Reasoning. arXiv:2504.21370.

[3] FastCuRL: Curriculum Reinforcement Learning with Stage-wise Context Scaling for Efficient Training R1-like Reasoning Models[J]. arXiv:2503.17287.

3. Training Cost Comparison

We thank the reviewer for the suggestion to clarify training efficiency. While we compare training steps in Appendix B.2, we now provide estimated GPU-hour costs for all methods using Distill-R1-1.5B as the base model:

As shown, AutoThink operates within the same order of compute as concise baselines like ThinkPrune and ShorterBetter, yet it achieves notably stronger performance. In contrast, DeepScaleR, a performance-maximizing method, requires over 3× more compute, with significantly larger context length and RL iterations.

We also note that AutoThink training can be completed within 24 hours on a 4-node H100 cluster, making it practical and reproducible under typical academic compute environments of R1-Style model training. Our code is fully open-sourced, and all results in the paper are reproducible, further supporting the accessibility and reliability of our approach.

4. On Single-Stage Design

We appreciate your thoughtful question regarding the complexity of our multi-stage RL framework. While AutoThink adopts a stage-wise structure for clarity and training stability, we emphasize that each stage serves a distinct and interpretable purpose:

• Stage 1 balances dual-mode behavior via a simple linear penalty using (γ, λ), which were not carefully tuned but chosen as common defaults. The resulting reward curve (Figure 3) is symmetric and robust.
• Stage 3 prunes redundant reasoning with length-aware shaping using default parameters (α, β) from GRPO-LEAD, also without fine-tuning.

Thus, while multi-stage, the design is lightweight and requires minimal hyperparameter tuning.

We agree that consolidating into a single-stage RL framework is a promising direction. One potential solution is to dynamically switch the reward formulation during training based on simple training signals:

• When the thinking rate stabilizes, automatically transition from Stage 1 to Stage 2.
• When accuracy gain plateaus, switch to Stage 3 to prioritize efficiency.

We plan to explore such self-adaptive reward scheduling mechanisms in future work to reduce engineering complexity while preserving adaptive reasoning capabilities.

We sincerely hope that these clarifications and additional results help address your concerns. If you have any additional feedback or suggestions, we would greatly welcome them and be happy to address them in detail.