On Reasoning Strength Planning in Large Reasoning Models

Thanks for your positive feedback, and we sincerely appreciate your valuable suggestions. We respond to your questions as follows.

Results on another type of reasoning benchmark.

Thanks for your concerns. We extend our experiments in Table 1 to another complex logic reasoning task GPQA diamond [1] for better demonstrating the generalization capabilities. We show the results on GPQA diamond in the following table:

Table 1: The effect of steering and performance on GPQA diamond

The choice of steering strength

Thanks for your concern about the choice of steering strength. Actually, in most cases, we found λ=0.05 generally works fine on all the datasets. In few cases, λ =0.1 works a little bit better than λ = 0.05. Therefore, we would suggest using a default λ = 0.05, since it generally benefits. Additionally, for better predicting how much steering is beneficial versus harmful, we can observe the performance on the validation set for choosing the proper λ. Moreover, since the main contribution of this paper is revealing the findings in reasoning strength planning, we just choose the best λ for testing the effectiveness of steering with such pre-allocation vectors. I believe it would also be important to discuss how to make our findings into practical methods, such as more fine-grained controls [2], although this is not our main contribution.

Questions about reasoning strength.

Thanks for your questions. In the first sentence of the abstract, we define the reasoning strength as the number of reasoning tokens. Therefore, it is generally the same as the reasoning length you have mentioned. I also agree with the insight that overthink can lead to decreased performance, as shown in the previous work [3, 4]. Additionally, our findings also support such insights. As shown in Figure 6, the performance rises and then drops as the reasoning token number increases. This indicates that adding reasoning strengths moderately can improve the performance, while overthink can lead to decreased performance.

Questions

• Question 1. We have added experiments on GPQA diamond as shown above.

• Question 2. We test whether we can yield more reasoning tokens by simply adding one sentence: "Please reason with more tokens." As shown in this table, naively prompting with more reasoning tokens does not consistently bring more reasoning tokens, as we even observe some reasoning token number drop in 1.5B and 14B models. Such results are consistent with previous findings that LRMs suffer from prompting with token numbers [5].

  Table 2: The effect of prompting LRMs to reason with more tokens

  	Original	+ Prompting
  R1-Distill-Qwen-1.5B	4899.34	4421.98
  R1-Distill-Qwen-7B	3828.50	4004.16
  R1-Distill-Qwen-14B	3495.39	3091.04
  R1-Distill-Qwen-32B	3430.73	3453.11
  QwQ-32B	4222.67	4718.66

• Question 3. Generally, as we have responded to the weakness 2, for increasing the performance, we can generally use a default λ=0.05. For reducing the token numbers, we can steer with negative strength λ on easy questions (in practice, we can train another detector for detecting the difficulty of questions). However, if the steering strength is too big, it will cause harmful results, such as decoding errors or overlong responses.

[1] GPQA: A Graduate-Level Google-Proof Q&A Benchmark

[2] SEAL: Steerable Reasoning Calibration of Large Language Models for Free. 2025.4.

[3] When More is Less: Understanding Chain-of-Thought Length in LLMs

[4] Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs

[5] L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning. Arxiv 2025.3.

We sincerely appreciate your detailed review and positive feedback for our work! We address your questions as follows.

Domain limitation

Thanks for your suggestions.

• Generalization capabilities in other domains. We discussed the generalization capabilities of our findings on other general domains beyond MATH, such as AlpacaEval and MMLU in Section 5, where our findings of reasoning strength prediction and strength control also hold on these two datasets. Due to the evaluation difficulty of code tasks on LRMs [1], following previous works [2, 3], we add one more experiment on the complex logic reasoning task GPQA diamond for better demonstrating the generalization capabilities.

Table 1: The effect of steering and performance on GPQA diamond

• Exploration on VLMs. Thanks for your valuable suggestions. We believe this is a promising research direction and would be another good research work. Extending our findings requires extending our methodologies to new modalities, datasets, and benchmarks. This requires a large amount of computational resources, since this research is very computation-consuming (i.e., we spent more than 100 A100 GPU days for the whole experiment) and experiments on VLMs would be even more computation-consuming. Adding experiments would be our future work, given the time limit of the rebuttal and the lower maturity of current reasoning VLMs.

Counter cases.

• Thanks for your suggestion, but I probably may not fully understand your question. Although our findings are consistent in 90% of the cases across models and datasets, we do find some cases where the model doesn’t follow the found relationships between reasoning token lengths and activation magnitude. As shown in Figure 18.a in Appendix C.2, on R1-Distill-Qwen-14B, the reasoning strength slightly decreases under a positive steering strength. We hypothesize this is probably due to some inner differences between R1-Distill-Qwen-14B and other models, which is worth exploring in our future work. We may discuss this later in our discussion period if my understanding is inaccurate.

Limited baseline comparisons and evaluation

Thanks for your suggestions.

• Comparison with other baselines. Thanks for your suggestions. Existing length controlling methods typically require training on additional corpora [1, 2]. Since our paper does not contain any training parts, it is unfair to compare it with these methods. We also add one more training-free method, which multiplies the logits of the end-of-reasoning token with a factor γ (i.e., logit_new = γ*logit_old). We report the results on MATH500 in Table 2. We find that naively changing the logits will lead to unstable manners, either leading to overly increased answer token number (i.e., γ = 2 on R1-Distill-Qwen-7B) or easily leading to decoding errors (i.e., γ = 0.8). This further indicates the effectiveness of steering.

**Table 2: Effect of changing the logits with γ **

• Expansion to harder tasks. As we have detailed in Section 5.2, our main goal is to test whether the LRMs indeed largely overthink on easy questions and how to make them think more efficiently on these questions. As shown in Figure 6, we can still reduce a certain number of reasoning tokens while maintaining the performance on harder questions, but the degree is much smaller than that on easy questions. Such findings are consistent with existing works that LRMs tend to overthink more on easy questions [4].

Questions

• High volatility across different layers. Thanks for your question. We hypothesize the volatility may come from the fact that we find the main direction for reasoning strength planning, while there may exist several sub-directions for more fine-grained reasoning strength planning, as previous works have observed in the field of LLM safety [5, 6].
• Results for QwQ-32B. Thanks for your question. We observe similar patterns on QwQ-32B, and we will put the results in the appendix later.
• Less reduction in token length for QwQ-32B. We hypothesize that this difference may come from the training difference between the Distilled-R1 series and QwQ-32B. Distilled-R1 series are trained with supervised finetuning (SFT) while QwQ-32B is trained with reinforcement learning (RL). We hypothesize that SFT may introduce more overthink patterns since they are trained with pre-collected data, while RL may make the reasoning strength planning more accurate since the training data is generated by the model itself. Therefore, QwQ-32B may be more sensitive to the changes in their reasoning token numbers.
• Proper negative steering strength. Sorry for making this confusing. We choose the steering strength that can largely maintain the performance while reducing the reasoning token numbers as much as possible. This is what we call proper steering strength. We will make this clearer in our latest paper.

[1] s1: Simple test-time scaling. Arxiv 2025.1.

[2] L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning. Arxiv 2025.3.

[3] Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model. Arxiv.3.

[4] A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond. Arxiv 2025.5.

[5] The Hidden Dimensions of LLM Alignment: A Multi-Dimensional Safety Analysis. ICML 2025.

[6] The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence. ICML 2025.

We sincerely appreciate your suggestions and interest in our work!

The control of token logits

Thanks for your question. We dive deeper into the logit control mechanism.

• Issues about superfacial control. We study the logits of the end-of-reasoning token since it is the most intuitive. Following your suggestions, we further test the changes in the logits of reasoning-related tokens. We find that with a positive steering strength, the logits of complex reasoning-related tokens also increase, beyond merely decreased logits of the end-of-reasoning token </think>.

  We report the changes in the logits for these reasoning-related tokens as follows in Table 1. These tokens are usually regarded as reflection patterns in the R1-style response. Increasing the logits of these tokens increases the probabilities of reflection and increases of reasoning token numbers, and decreasing them has the opposite effect. As shown in Table 1, positive steering strengths increase the logits of these tokens while negative steering strengths decrease them. This indicates the reasoning strength control is not superficial. We will add these experiments to our latest paper.

Table 1: Impact of activation steering on logits of reasoning-related tokens.

​ This indicates that such pre-allocated vectors do not superfacially change the logits of the end-of-reasoning token </think>.

• Comparison of superfacially changing the logits of the end-of-reasoning token </think>. To further test whether simply changing the logits of the end-of-reasoning token can achieve the same performance, we conduct experiments by multiplying the logits of the end-of-reasoning token with a factor γ (i.e., logit_new = γ*logit_old). We report the results on MATH500 in Table 2. We find that naively changing the logits will lead to unstable manners, either leading to overly increased answer token number (i.e., γ = 2 on R1-Distill-Qwen-7B) or easily leading to decoding errors (i.e., γ = 0.8).

  These results further suggest that the changes in the logits of the end-of-reasoning token are the result of planning, rather than the cause of planning.

Table 2: Effect of changing the logits with γ

Prediction accuracy & Random CoTs from the same question

• Prediction accuracy. We report the mean absolute error (MAE) as the accuracy metric. The predictor in Figure 1 yields MAE = 591.37 for y under 4000 tokens. Meanwhile, we add one more experiment to demonstrate that this prediction is non-trivial. We randomly permute the label y (i.e., the actual reasoning token number) and make a prediction again. The MAE under this random prediction setting is 1428.72, which is significantly higher than our prediction. We would like to highlight again that this work reveals whether and how LRMs plan the reasoning strength rather than making this prediction as accurate as possible. It is worth exploring in the future for improving the accuracy by adding more training data, adopting a more complicated prediction model structure, and carefully tuning the hyperparameters. For example, if we merely add 500 more training datapoints from MATH500, the MAE further reduces from 591.37 to 563.42. Currently, we only have fewer than 7k training datapoints, and generating these datapoints is very time-consuming, which limits current accuracy.

• Random CoTs from the same question. Thanks for your question. In our work, we use the setting of rollout = 8 and take the average as the final reasoning token number. We also calculate the MAE between each rollout and the average of for one rollout. The final averaged mean absolute error MAE of QwQ-32B on the MATH dataset is 602.10, indicating moderate consistency among their responses, given the average reasoning token number is over 4000.

Small performance gain

Thanks for your questions. We would like to answer this question from the following aspects.

• First, we would like to highlight that the main contribution of this paper lies in revealing whether and how LRMs plan their reasoning strength in advance, rather than designing a sophisticated method for improving the performance.
• Second, given the naive design and the high saturation of math datasets, the improvement of steering is non-trivial [1]. Additionally, some concurrent works also suggest that steering with a more carefully designed vector can further enhance the performance, indicating that such findings can be largely improved with careful design [2].
• Third, we add one more experiment on the general reasoning dataset GPQA diamond in Table 3. Results show that steering with the pre-allocation vector can generalize on this task and can even provide more performance gain than MATH. We attribute this to the lower saturation of such general reasoning tasks.

Table 3: The effect of steering and performance on GPQA diamond

Questions

• Chosen layers. We apply the activation steering to every layer l following previous works [3], as explained in Appendix C.2. We will add this information in the main body of this paper.

• Choice of strength λ. In most cases, we find λ = 0.05 or λ = 0.1 works fine on all the datasets. Usually, a default choice of λ = 0.05 can yield a good performance among all the benchmarks in most cases, and 0.1 sometimes works a little bit better. We will add the chosen strengths later in the table.

• Why fig2c has x=0 point but not the others? Thanks for your detailed review. Sorry for making this confusing. We will unify this by adding x=0 for all the figures. The low performance when x=0 indicates that the same representation of end-of-reasoning token </think> at the input embedding layer can not yield any useful prediction (i.e., the place where we extract the representations), which is intuitive and reasonable.

• The average reasoning token number of overthink questions. Yes. These overthink questions consistently yield more reasoning token numbers. We report the average reasoning token numbers in Table 4. The results indicate that these overthink attacks indeed make these LRMs spend more tokens for reasoning.

Table 4: Effect of overthink attack on the reasoning token number.

[1] DAPO: An Open-Source LLM Reinforcement Learning System at Scale. Arxiv 2025.5.

[2] SEAL: Steerable Reasoning Calibration of Large Language Models for Free. Arxiv 2025.4.

[3] Refusal in Language Models Is Mediated by a Single Direction. NeurIPS 2024.

We sincerely appreciate your detailed evaluation and valuable suggestions! We respond to your questions as follows:

---

Scope & Generalizability

• Other reasoning domains. We discussed the generalization capabilities of our findings on other general domains beyond MATH, such as AlpacaEval and MMLU in Section 5, where our findings of reasoning strength prediction and strength control also hold on these two datasets. We also add one more experiment on the complex logic reasoning task GPQA diamond to better demonstrate the generalization capabilities. As shown in Table 1, the number of reasoning tokens generally exhibits a similar pattern with the λ, indicating the generalization of this pre-allocation vector to other domains. Additionally, proper positive steering can also bring performance improvements.

Table 1: The effect of steering and performance on GPQA diamond

• Other model families. We add experiments on another kind of LRM DeepSeek-R1-Distill-Llama-8B. All of our findings also hold on this model, and we will add more results in our paper. We illustrate some datapoints of our main observations on DeepSeek-R1-Distill-Llama-8B. Interestingly, Llama is more sensitive to the steering strength λ.

Table 2: Prediction on middle layers

Table 3: Cosine similarity between pre-allocated vectors on layer 31

Table 4: Mean cosine similarities on middle layers

Table 5: Activation Steering

Methodological / Experimental Design Justification:

• Choice of residual stream activations. Sorry for this confusion. Actually, we use the input embeddings at each layer as you have mentioned. These input embeddings are called the (input) residual stream in previous literature [1, 2], where representation at this place is a common usage. We will make this clearer by adding references and add explanation for this in our paper.

• Selection of α and rationale for Lasso regression. Actually, we do not carefully tune this hyperparameter α since tuning this hyperparameter at each layer is laborious. Therefore, we just choose a moderate α=10 that works fine on DeepSeek-R1-Distill-Qwen-7B and adopt this α for all the models. It is possible that we can further improve the prediction accuracy by carefully tuning α. For the choice of L1 regression, we found that both L1 and L2 work similarly. Therefore, we choose L1 following the choice of previous work [3]. We show the comparison experiments on the 50th layer of DeepSeek-R1-Distill-Qwen-32B in Table 6 and Table 7. When α is small, the regression starts to overfit, showing a high value of R on the training set and a low value on the test set.

Table 6: Comparison of α

Table 7: Comparison with L1 and L2 regularization.

• Layers of activation steering. We apply the activation steering to every layer l following previous works [2], as explained in Appendix C.2. We will add this information in the main body of this paper.

• Choice of strength λ. In most cases, we find λ = 0.05 or λ = 0.1 works fine on all the datasets. Usually, a default choice of λ = 0.05 can yield a good performance among all the benchmarks in most cases, and 0.1 sometimes works a little bit better. We will add more details in our paper.

Analysis

• Training of linear probing. We train a single predictor handling all the difficulties. We also provide experiments predicting other mathematical reasoning tasks in Table 8. Results show a still high R value, indicating the generalization capabilities.

Table 8: Prediction on other benchmarks

• Accuracy metrics. We report the mean absolute error (MAE) as the accuracy metric. The predictor in Figure 1 yields MAE = 591.37 for y under 4000 tokens. Meanwhile, we add one more experiment to demonstrate that this prediction is non-trivial. We randomly permute the label y (i.e., the actual reasoning token number) and make a prediction again. The MAE under this random prediction setting is 1428.72, which is significantly higher than our prediction.

  We would like to highlight again that this work reveals whether and how LRMs plan the reasoning strength rather than making this prediction as accurate as possible. It is worth exploring in the future for improving the accuracy by adding more training data, adopting a more complicated prediction model structure, and carefully tuning the hyperparameters. For example, if we merely add 500 more training datapoints from MATH500, the MAE further reduces from 591.37 to 563.42.

• Reasoning capabilities in later layers. Previous mechanism explanation works suggest that the early layers of language models may be responsible for low-level tasks such as syntax and grammar, and more complicated understanding capabilities are developed in the later layers [4]. Recent findings also suggest that reasoning capabilities may be developed in the later layers [5], which provides more support for our findings.

• Figure 7 appears inconsistent with Figures 2/4 & Figures 6 and 7 should be presented together. Thanks for the detailed review, and also sorry for the confusion. Actually, Figure 7 depicts the effect of the steering vector with strength λ=0.2, rather than the prediction for the whole response. We apologize for using a misleading annotation y_hat in Equation 4, and we will revise this in our latest paper. For Figure 7, we suggest mostly focusing on the average predictions ≈ 1700, 1500, and 500, which are similar to the changes in the reasoning token numbers from a zero strength to a strength λ=0. Moreover, we attribute the disturbance across layers to the fact that we may find the main direction vector for strength planning, while there may exist sub-directions for more fine-grained control, as has been evidenced in similar works [2, 6].

Baseline and Comparison Issues

• Cosine similarity. We add experiments comparing with difference-in-mean vectors from randomly sampled question pairs. We randomly sample three sub-dataset, each with 2000 datapoints, and calculate the mean activation of them, denoted as diff_1, diff_2, and diff_3. Then we calculate the cosine similarity between diff_3 - diff_1 and diff_2 - diff_1 as the comparison. This random cosine similarity is generally around 0.45, with 0.4634 at the same layer of Figure 3.a. This result further supports the existence of the pre-allocation vector for reasoning strength planning.

Application Practicality

Thanks for your interest. As we have highlighted in the first paragraph in Section 5, to make this approach in practice, more efforts are required. One possible solution for making this overthink detection in practice is probably that we can additionally train a difficulty classifier. Once this problem is identified as simple, but the reasoning strength prediction yields an overlong prediction, we can provide a warning for the possible occurrence of overthink. This is worth exploration in the future, and we think it deserves the effort for another research paper.

Related Works

Thanks for this suggestion. We believe the idea in this paper that LLMs know their correctness can further provide support for our claims that LRMs know their reasoning strengths in advance. We will cite this paper and discuss it in our latest paper.

Minor

Thanks for your detailed suggestions. We will revise them in the latest paper.

[1] A Mathematical Framework for Transformer Circuits. Anthropic 2021.

[2] Refusal in Language Models Is Mediated by a Single Direction. NeurIPS 2024.

[3] Language models represent space and time. ICLR 2024.

[4] Safety Layers in Aligned Large Language Models: The Key to LLM Security. ICLR 2025.

[5] Detection and Mitigation of Hallucination in Large Reasoning Models: A Mechanistic Perspective. Arxiv 2025.5.

[6] The Hidden Dimensions of LLM Alignment: A Multi-Dimensional Safety Analysis. ICML 2025.