ThoughtTerminator: Benchmarking, Calibrating, and Mitigating Overthinking in Reasoning Models

Thank you for your insightful and detailed review.

Weaknesses

1. Token length isn’t explained by overthinking alone, relative model weakness also matters

Great point. This is why we analyze both local and global overthinking. Local overthinking only compares reasoning chains within model; comparing the distribution of reasoning chain lengths for each question to the observed minimum successful length for that model. For a weaker model which requires brute forcing longer reasoning chains than necessary still contain overthinking. In other words local overthinking will capture “real overthinking” in this case while global overthinking would capture both “model weakness” as well as real overthinking.

In Table 1 we report both local and global overthinking for each model. The two methods have a PCC of 0.97, suggesting that the impact of relative model strength on global overthinking scores is low.

We will include your point, as well as this response, in the discussion section of our CR. Overthinking in RMs has yet to gain a consensus definition, so it is important that we precisely convey what implications and potential problems lie in our operationalization, and this point about the correlation between the two scores strengthens our overall argument.

2. Simpler baseline suggestions

These are interesting baseline proposals. Re: your first proposal, we add two baselines:

• Baseline 1: Adds the instruction “Answer concisely and avoid unnecessary elaboration.”
• Baseline 2: Adds the instruction “Do not overthink.”

We evaluated these along with the base setting and our method (ThoughtTerminator) on two models (R1-1.5B and QwQ-32B) across two datasets (MATH500 and GPQA). The results are shown below:

These baselines, while sometimes outperforming the base condition, are beaten clearly by Terminator.

We will leave the other proposed baselines for the CR.

3. Do older baselines contain questions as easy as DUMB500?

We found that for the math questions, no extant dataset was as simple as we wanted. We wanted to expand out a set of questions of similar difficulty to the canonical “what does 2+2 equal?” example, which were too easy to be worth including in prior datasets, even for the most rudimentary ones 5 years ago. By the time researchers were bothering to evaluate LMs on math, they were already good enough to get such questions right.

4. Narrative structure feels incoherent, figures seem unfinished

We recognize that our paper has an unconventional structure. We felt that the traditional “problem statement->methods->results” format was inappropriate for this work, as the initial overthinking quantification results inform the need for the DUMB500 dataset, and then overthinking/accuracy results for the four datasets inform the design of Terminator. We believe this is the ideal structure, but will revise the introduction to more clearly broadcast the logic and layout to readers to reduce confusion. Re: figures, see answers to questions.

Questions

1. Different tokenizers, different overthinking scores?

This is a good question. We believe that the correlation between global and local overthinking discussed above suggests that this is a minor issue. However, in the CR we will provide an analysis where we compare the token lengths of a set of reference strings between each model, and recompute overthinking scores after normalizing based on these as an ablation.

2. Why are models bad at DUMB500 chats and tasks?

Task and chat examples are actually graded using LM judges and numeric rubrics rather than accuracy, so average scores <<1 do not necessarily represent severely bad performance as they would on an accuracy scale. We have revised the axis labels to reflect this. Furthermore, we want to highlight that the math subset of DUMB500 is most central to our story, the other three partitions are mainly introduced to (a) enable future work and (b) show overthinking on diverse domains.

3. What do real-min and pred-diff-ref mean?

Sorry for this oversight! real-min is an ablation where we simply fix the minimum observed token spend for each example to be the deadline, isolating the effect of applying Terminator by removing the influence of poor difficulty predictions for a given sample. pred-diff-ref in turn is an ablation where the ground truth bucketed difficulty level prediction for each question is fixed, but the difficulty->token spend predictor is still used. We have clarified this explanation in the manuscript.

1. On more baselines for comparison

We thank the reviewer for this suggestion.

Following the feedback, we added two prompt-based baselines:

• Baseline 1: Adds the instruction “Answer concisely and avoid unnecessary elaboration.”
• Baseline 2: Adds the instruction “Do not overthink.”

We evaluated these along with the base setting and our method (ThoughtTerminator) on two models (R1-1.5B and QwQ-32B) across two datasets (MATH500 and GPQA). The results are shown below:

These results show that:

• The Terminator consistently achieves much lower token usage, often by more than 50%, compared to all baselines;
• In most cases, it also improves or maintains performance, especially on GPQA;
• These baselines offer modest improvements in a few cases, but they do not reliably reduce overthinking or token inefficiency.

We believe these experiments confirm that ThoughtTerminator is more effective and consistent than simple prompt-based heuristics.

2. On the assumptions behind difficulty and overthinking scores

2.1 Why we chose an empirical definition

Our difficulty and overthinking scores are empirical, i.e., they reflect how today’s models behave in practice. Our choice aligns with prior work like Dynabench [1], which also embraces an empirical, model-relative view of task difficulty.

We use observed model behavior as the reference:

• Difficulty – average error rate across a diverse pool of 12 models (non-reasoning + reasoning). This captures how hard today’s models find the question, even if the theoretical difficulty is low.
• Overthinking score – excess tokens relative to the shortest successful trace seen anywhere. This tells us how far a given model is from the best practice we have observed in the current ecosystem.

This design intentionally reflects practical rather than intrinsic efficiency.

2.2 Robustness to the choice of model pool

If there is one specific model never thinks, the score will drastically change.

Our paper introduces two overthinking metrics:

• Global Overthinking Score, which compares each model’s token usage to the minimum across all models (i.e., global baseline), and
• Local Envelope Score, which only uses a single model's own behavior and thus is independent of other models.

We agree that the absolute value of the Global Overthinking Score may vary depending on the model pool—for instance, if one model always uses extremely few tokens, it can lower the global baseline and affect others’ scores. However, we emphasize that this score is meant to capture relative differences across models, not serve as an intrinsic measure.

To test robustness, we progressively removed the models with the shortest average successful outputs (top 1, top 2, and top 3). In all cases, the ranking of models by global overthinking score remained unchanged. This indicates that while the baseline may shift slightly, the comparative insights drawn from the scores are robust.

This supports the view that our proposed overthinking scores produces stable rankings and comparisons, even if absolute values shift slightly with the model pool.

3. On “A_hat ~ M ≠ a” Notation

Can you explain the “A_hat ~ M !=a”? I don’t get the a_hat here.

Thank you for pointing this out. We agree the notation was unclear. In our intended formulation,

• $\hat{a}_M(q)$ denotes the model $M$’s predicted answer to question $q$,
• $a$ is the correct (gold) answer.

The expression “A_hat ~ M != a” simply means that the model's answer is incorrect. We will add an explanation of $a_hat$ in our manuscript.

4. On Percentage Change in Table 2

To clarify: as we mentioned in the table caption, all percentage changes reported in Table 2 are relative improvements. For example, a drop from 0.58 to 0.50 is reported as (0.50 - 0.58)/0.58 = -13.8%.

References: [1] Douwe Kiela et al. Dynabench: Rethinking Benchmarking in NLP. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics

Hi Reviewer aNwS, we just wanted to check if you have any thoughts or concerns regarding our rebuttal. Thanks!

1. Prompt diversity and model’s ability to read tokens

Only one prompt format was tried; models may not understand ‘x elapsed tokens, y remaining’. Try alternative phrasings (e.g., ‘x % time elapsed’).

Thank you for raising this question, we have added an additional urgency formats and re-run the key experiments on R1-1.5B (MATH500 & GPQA). The table below compares two interrupt message formats.

Dataset: MATH500

Dataset: GPQA

The results show comparable accuracy and token usage across both variants.

2. Apparent accuracy drop for QwQ-32B (MATH500)

Thank you for pointing this out. Upon reviewing our logs, we found that the low reported accuracy for QwQ-32B + ThoughtTerminator was due to a logging error. At submission time, only a subset of completions had finished running. The numerator (number of correct answers) was computed over this finished subset, while the denominator mistakenly included the full evaluation set.

After submission, we have more time to re-run the complete evaluation, and we found that QwQ-32B + ThoughtTerminator achieves Pass@10 = 0.80, which is significantly higher than previously reported. We will update Figure 6 in the camera-ready version.

3. ZEBRA outlier in Figure 7

In figure 7, for the ZEBRA dataset, the base seems to use the most tokens but produce the worst (by a huge margin) result. This seems like an outlier...

With the original 5,000-token cap, only a handful of the very hardest ZebraLogic items (difficulty > 7) were answered correctly, and those few successful traces happened to be concise. Because the plot averages only over correct answers, the higher difficulty bins were therefore represented by a tiny set of short completions, making it look as if “hard” questions required fewer tokens. After re-running with a higher budget (max_tokens = 32,000), we obtained many more correct solutions, most of which are long, formal derivations.

The spend-versus-difficulty curve is now strictly increasing and the apparent outlier vanishes. We will replace Fig. 7 with the updated version.

4. Readability of Section 2.1

Thank you for the suggestion. We will revise §2.1 to improve clarity by adding inline explanations for key formulas and symbols, and adjusting the paragraph structure to ease readability.

5. Will the DUMB500 dataset be available publicly?

Yes. We are currently packaging the dataset, grading scripts, and license files.

1. On the difficulty definition and potential conflation with memorization

In this work, we do not claim to measure the intrinsic or absolute difficulty of a question. Instead, we define difficulty empirically, based on the aggregated error rate across a diverse pool of models. This definition reflects how likely current models are to solve a problem correctly in practice, rather than its inherent complexity.

The question difficulty is not the only factor that affects the answer accuracy, it can also relate to the question/answer memorization.

As clarified in Section 3, DUMB500 was entirely human-authored from scratch, and not sampled or modified from any existing benchmarks such as GSM8K, BBH, MATH, or HumanEval. The prompts are intentionally designed to be simple and novel, and do not follow typical benchmark phrasing or templates. Therefore, we did not consider memorization to be a major risk. Additionally, our difficulty scores are aggregated across a diverse pool of models, which further reduces the bias of any single model’s memorization behavior.

2. GPT-4o-mini as judge and its agreement with GPT-4o

Would be nice to present the agreement between gpt-4o-mini and more widely used judge LLM such as GPT-4o to validate the evaluation effectiveness.

We thank the reviewer for suggesting a validation step. We evaluated the agreement between GPT-4o and GPT-4o-mini on DUMB500's task and chat subsets, and observed the following:

• For DeepSeek-R1: agreement = 99.12%
• For QwQ-32B: agreement = 91.15%

These results indicate strong alignment between the two judges, especially for models with more stable output patterns. This supports our use of GPT-4o-mini as a lightweight but reliable approximation to GPT-4o for large-scale evaluation. We will report these agreement results in our manuscript.

3. Low accuracy of QwQ-32B + ThoughtTerminator in Figure 6

Thank you for pointing this out. Upon reviewing our logs, we found that the low reported accuracy for QwQ-32B + ThoughtTerminator was due to a logging error. At submission time, only a subset of completions had finished running. The numerator (number of correct answers) was computed over this finished subset, while the denominator mistakenly included the full evaluation set. After submission, we had more time to re-run the complete evaluation, and we found that QwQ-32B + ThoughtTerminator achieves Pass@10 = 0.80, which is significantly higher than previously reported. We will update Figure 6 and the relevant discussion in the camera-ready version.

4. More details on the prediction-based deadline estimator

Our deadline estimator predicts the difficulty of a question on a 1–10 scale and maps it to a token budget. We explored two variants:

• A zero-shot GPT-4o-mini baseline, which predicts difficulty based on a fixed instruction-prompt format;
• A fine-tuned Instruct-LLaMA-8B, trained on 1,465 examples covering diverse task types.

Each sample consists of a question and a discrete difficulty label derived from the minimum correct generation length (discretized into bins). The trained predictor achieves a mean absolute error (MAE) of 1.57 on a held-out test set, indicating that predicted difficulty levels are, on average, within 1–2 bins of the reference. We will include these training details, sample format, and a summary of the sub-sampling results in Appendix of the revised version.

Hi Reviewer HZWD, we just wanted to check if you have any thoughts or concerns regarding our rebuttal. Thanks!