Evaluating the Inductive Abilities of Large Language Models: Why Chain-of-Thought Reasoning Sometimes Hurts More Than Helps

Thank you for your thoughtful and encouraging feedback. We are glad that you found our main empirical finding—the weaker inductive performance of reasoning-augmented LLMs compared to non-reasoning models in specially designed games—to be interesting. We have carefully considered your concerns and respond to them individually below.

W1. The authors should provide a bit more examples to motivate audience and justify the value of the work, making it more clear that the issue exists, not due to some specific designed games or data patterns.

A1. Illustrative failure cases are provided in Figure 1(b). Additional examples and error breakdowns appear in Appendices G, H, and I, spanning a range of task settings.

As discussed in Section 4.3, similar failure patterns arise across all four games, which differ in symbolic structure and inductive demands. This recurrence suggests that the observed issues are not tied to any single task design.

While these examples are drawn from games—where model behavior can be measured in a controlled setting—related failures have been noted in broader contexts as well, such as those in mathematical problems [1,2].

W2. The authors should provide a bit more details, such as a concrete example, especially about the specifically designed games that help evaluate the performance of the models.

A2. We refer the reviewer to Figure 1, which presents a concrete example illustrating the benchmark setup, rule definitions, and model inference transcript. This figure is intended to clarify both the structure of the games and the evaluation process. The four games (Chess, Texas Hold’em, Dice, and Blackjack) each include a Normal Rule (NR) reflecting standard gameplay and a Special Rule (SR) introducing a hidden constraint.

For example, in Figure 1(a), the SR specifies that “five consecutive primes > three of a kind,” overriding conventional poker rankings. As shown in Figure 1(b), models are tasked with inferring each rule from game transcripts. The accuracies reported in Figure 2 are computed per rule and reflect the model’s ability to recover these rule structures from limited data.

W3. It is better having a table of the notations.

A3. The variable $N^\star$ is defined at Line 217 and proved beginning at Line 620. For ease of reference, we provide the following notation table summarizing all variables used in Section 4, including the formal definition of the optimal reasoning length $N^\star$ referenced in Theorem 4.1:

W4.The experimental analysis can be enhanced to show more insights of the proposed techniques.

A4. Our analysis reveals how each proposed intervention addresses distinct reasoning failure modes: summarization mitigates hallucinated rule statements, decomposition reduces error propagation across subtasks, and solving directly corrects invalid inferences. These effects are quantitatively shown in Figure 3. Together, the components yield complementary gains across tasks and models, with solving producing the most robust improvements on inductive failures. We believe the current analysis offers insight into both the sources of failure and the mechanisms through which each technique improves inductive robustness.

References
[1] Chen X, Xu J, Liang T, et al. Do NOT Think That Much for 2+3=? On the Overthinking of Long Reasoning Models. Proceedings of the 42nd International Conference on Machine Learning (ICML), 2025.

[2] Cuesta-Ramirez J, Beaussant S, Mounsif M. Large Reasoning Models are Not Thinking Straight: On the Unreliability of Thinking Trajectories. arXiv preprint arXiv:2507.00711, 2025.

We appreciate your thoughtful review and are encouraged by your recognition of the significance and non-obvious nature of our core findings, as well as the clarity and utility of our theoretical framework. We believe your remaining concerns are addressable and respond to them in detail below.

W1. I feel the title is overtly attention grabbing and seems like just reading the title alone oversimplifies the findings of the paper.

A1. Thank you for sharing your concern about the title. We agree that the current phrasing may risk oversimplifying our findings. In the revised version, we will use a more precise title: Evaluating the Inductive Abilities of Large Language Models: Why Chain-of-Thought Reasoning Sometimes Hurts More Than Helps

W2. The classification of reasoning failures was conducted by two human annotators.

A2. Thank you for raising this important concern. Motivated by your feedback, we computed Cohen’s κ between the two annotators based on their independent annotations before resolving any disagreements. We evaluated inter-annotator agreement across three error types—Decomposition, Solving, and Summarization—on 100 sampled model outputs from reasoning models. The results are summarized below:

We believe κ = 0.86 indicates a high level of reliability, and we will include these agreement metrics and this table in the revised version to enhance the methodological rigor of Table 1.

W3. The empirical analysis of failure modes was also performed on three specific LRMs. Although the findings are consistent across these models, it is a significant leap to generalize these specific failure patterns to all current and future reasoning models. This requires a careful discussion about the limitations, at the very least.

A3. Thank you for raising this point. We acknowledge that our analysis is limited to three reasoning models, and it is premature to generalize the identified failure patterns to all current or future models. We also recognize that comparisons between reasoning and non-reasoning models may be influenced by confounding factors such as version mismatch or model size differences. We will clarify these limitations in the revised paper.

W4. The primary metric for success—rule-level accuracy—is determined by using GPT-4o as a judge to assess semantic alignment — that’s circular and off.

A4. We appreciate the reviewer’s concern about potential circularity in using GPT-4o for evaluation. Motivated by your comment, we conducted a human validation study on 100 GPT-o3-generated rule descriptions across all four games. Three independent annotators—blind to model identity and GPT-4o’s decisions—judged semantic alignment with the ground truth. We then compared their majority vote with GPT-4o’s decisions and computed Cohen’s κ to measure agreement. The results are summarized below:

GPT-4o’s assessments matched the human majority in all 100 cases (60 aligned, 40 not aligned), yielding an accuracy and κ of 1.00. While this perfect agreement may in part reflect the limited sample size, it provides strong evidence that GPT-4o’s evaluations are well-aligned with human judgment in this context.

Q1. The paper fairly convincingly shows that they are better at inducing special rules, but not why. Are they using a more robust, implicit form of reasoning, or are they relying on a different form of pattern matching that is less prone to the error amplification seen in multi-step CoT?

A1. While we cannot directly observe the internal mechanisms behind model decisions, our theoretical analysis and empirical findings suggest that non-reasoning models may rely on a different form of pattern matching that is less susceptible to the error accumulation observed in multi-step CoT reasoning. In particular, we conjecture that the absence of explicit reasoning traces in these models reduces opportunities for compounding errors in sub-task decomposition, solving, or summarization.

Q2.The interventions presented in the paper are fair in the games-setup, but do they generalize to other domains at all?

A2. We believe these findings can extend to more common reasoning problems, such as mathematical problem solving. Prior work has noted that reasoning-augmented models often generate inflated reasoning traces and redundant self-verifications, leading to inefficient token usage [1] and increased error rates [2]. To test the applicability of our intervention, we evaluated GPT-o3 on 20 problems from the AIME 2024 exam. We compared its accuracy with and without our combined intervention. The results are summarized below:

As shown in the table, the combined intervention improved accuracy by 20 percentage points, suggesting its potential to mitigate reasoning-related failures in mathematical tasks.

Q3. Could the identified failure modes be addressed through targeted fine-tuning instead of prompting?

A3. We appreciate this suggestion. We agree that targeted fine-tuning could be a promising direction for mitigating some of the failure modes we identify. Our current focus is on prompt-based interventions mainly due to their ease of deployment. We view prompt-based and fine-tuning-based approaches as complementary, and we hope our analysis can inform future work in both directions.

References
[1] Chen X., Xu J., Liang T., et al. Do NOT Think That Much for 2+3=? On the Overthinking of Long Reasoning Models. ICML 2025.

[2] Cuesta-Ramirez J., Beaussant S., Mounsif M. Large Reasoning Models are Not Thinking Straight: On the Unreliability of Thinking Trajectories. arXiv:2507.00711, 2025.

I appreciate the detailed response. I am maintaining my current positive score.

Happy that you are indeed considering changing the name.

Dear Reviewer EhoN,

Thank you very much for your thoughtful comments, the time you devoted to reviewing our paper, and for considering maintaining your positive score. We will incorporate your suggestions, including the proposed name change, to further improve the quality of the final version.

Best regards,

The authors

Thank you for the thoughtful review. We are encouraged that you found the performance gap between reasoning and non-reasoning models on hidden rules to be interesting, and we appreciate your recognition of how this challenges the common assumption that more reasoning always leads to better performance. We believe the mentioned weaknesses and questions can be sufficiently addressed.

W1. This paper heavily relies on GPT-4o as a judge, potentially introducing bias towards certain types/answer styles.

A1. We appreciate the reviewer’s concern about potential circularity in using GPT-4o for evaluation. Motivated by your comment, we conducted a human validation study on 100 GPT-o3-generated rule descriptions across all four games. Three independent annotators—blind to model identity and GPT-4o’s decisions—judged semantic alignment with the ground truth. We then compared their majority vote with GPT-4o’s decisions and computed Cohen’s κ to measure agreement. The results are summarized below:

GPT-4o’s assessments matched the human majority in all 100 cases (60 aligned, 40 not aligned), yielding an accuracy and κ of 1.00. While this perfect agreement may in part reflect the limited sample size, it provides strong evidence that GPT-4o’s evaluations are well-aligned with human judgment in this context.

W2. The paper has a hypothesis of “the base model has fewer CoT while the reasoning model has more CoT.” However, Equation 5 states that either under- or over-reasoning can incur this summary error. Section 4.3 empirically classified such errors, but only for the reasoning models. No analysis of base models' reasoning processes and errors. The hypothesis is stated but doesn’t seem to be proved?

A2. We would like to clarify that our paper does not formally claim “base models have fewer CoT.” Our actual hypothesis is: reasoning models’ multi-step traces may not help—and can introduce incorrect assumptions or misleading intermediate steps (see lines 157–158). This is supported both theoretically (Section 4) and empirically (Section 5). Base models like GPT-4o, when prompted directly via API, do not produce intermediate reasoning traces and are not considered “reasoning models” unless explicitly instructed. Therefore, our failure mode analysis in Section 4.3 focuses on models that generate such reasoning traces (e.g., GPT-o3, DeepSeek-R1).

W3. There’s no comparison against other reasoning method.

A3. We appreciate the suggestion to compare against explicit CoT prompting. In response, we conducted additional experiments applying a CoT-style instruction to GPT-4o:

“Let’s think step by step. First, extract the rule from the transcript. Then explain your reasoning.”

This prompt was applied to each rule inference task in the Chess game, and we compared performance with and without CoT prompting:

Overall, CoT prompting did not yield consistent improvements. On SR rules, performance was largely unchanged or slightly decreased, suggesting that added reasoning may not help—and can sometimes distract from inferring hidden rules. On NR rules, performance was already saturated.

It is worth noting that this CoT-style prompting differs from models like GPT-o3, which generate structured, multi-step reasoning (e.g., decomposition → solving → summarization). Injecting CoT alone does not reproduce or mitigate the same failure patterns. We will include these results in the revised version.

Q1. I noticed models like GPT-4o also generate chain-of-thought reasoning when using them. Why is “their multi-step traces may not help” for inferring the hidden rules, rather than “the procedure of training the reasoning models may not help”?

A1. While models like GPT-4o may occasionally produce CoT-like outputs—especially in chat-based interfaces—these traces do not appear when accessed via API, which is the setting used in our experiments.
Our study focuses on the impact of multi-step reasoning traces at inference time, independent of the model’s training procedure. We find that the observed failure modes emerge even when reasoning is introduced purely via prompting in open-ended inductive tasks [1]. This suggests the issue lies more in the structure of the reasoning process itself, rather than in how the model was trained.

Q2. Maybe it is harder for them to adapt the models to new rules because they were heavily trained to stick on the existing rules? Is it possible that for popular games like blackjack or Texas Hold’em, the reasoning models are trained on them heavily by RL, therefore it is harder for them to adapt to new rules?

A2. We appreciate this line of reasoning. It is indeed plausible that reasoning-augmented models carry inductive biases from training on well-known games or standard reasoning formats, which may limit their adaptability in unfamiliar or rule-shifted settings. However, our study does not aim to attribute failure to any specific training regime, as we lack access to the proprietary reinforcement learning or fine-tuning pipelines used by closed models. Instead, our findings isolate a distinct inference-time effect: even when reasoning is introduced via prompting, multi-step traces can constrain model behavior and lead to systematic errors in rule induction. These effects are consistent across tasks and model families, and we would be glad to incorporate both perspectives into the revised discussion.

Q3. In equation 3 the $\gamma_k$ is in (0, 1), given the presence of self-reflections/corrections in these reasoning models, should the $\gamma_k$ be in (-1,1) instead?

A3. In our paper, $\gamma_k \in (0, 1)$ is intended to model bounded rational belief updates rather than full reversals. Even when reasoning models engage in self-correction, their belief adjustments tend to be gradual refinements rather than full negations. Negative $\gamma_k$ would imply a reversal in belief direction, which is not typical in observed reasoning traces. We clarify this design choice further in Appendix E.

Q4. Where does it reflect the improvements of summarization as indicated in line 332–333?

A4. The blue bar in Chess NR4 reflects GPT-o3, not GPT-4o. In this case, GPT-o3 achieves 100% accuracy without intervention, but drops slightly to 97.33% under the summarization constraint—hence the darker blue segment appears below the baseline, indicating a small decrease rather than a gain from summarization. We recognize that in some cases, the stacked figure may obscure intermediate effects, especially when solving contributes the most visible improvement. For convenience, we include a numeric ablation table below to directly reflect the summarization gains on Chess game.

Full table will also be included in the appendix of the revised version to improve overall readability.

Q5. Word usage of “counterparts”.

A5. Thank you for the suggestion. We agree that “counterparts” may be ambiguous and will revise the phrasing to use “models from the same time period” instead.

Q6. Appendix K

A6. We appreciate the reviewer highlighting this issue. The problem with the anonymous link in Appendix K has been resolved, and the full codebase is now accessible.

L1. No limitations are addressed. Please include the limitations of the problem setup (four game-based tasks), the evaluation bias (only uses GPT-4o), and the hypothesis.

A1. We appreciate the reviewer’s point. We will include a dedicated limitations section to clarify scope and assumptions. Our benchmark is restricted to four symbolic games. While our primary evaluation used GPT-4o, we validated its judgments against independent human annotations. Finally, our study focuses on prompting-based reasoning models, and we agree that the generality of the findings across other reasoning paradigms remains an open question.

References

[1] Turpin M, Michael J, Perez E, et al. Language models don't always say what they think: Unfaithful explanations in chain-of-thought prompting. NeurIPS 2023.

Thank you for the constructive feedback. We appreciate your recognition of the problem's importance and the clarity and utility of our theoretical framework in explaining performance degradation and guiding effective interventions. We believe the identified concerns can be adequately addressed.

W1/Q1. The comparison between reasoning and non-reasoning models may be confounded by differences in model versions and sizes.
A1. We appreciate the reviewer’s attention to potential confounding factors in model versioning and size. Inspired by this concern, we tried to minimize these issues by including additional comparisons with the most recent publicly available reasoning model, DeepSeek-R1-0528, and contrasting it with its closest non-reasoning model, DeepSeek-V3-0325. We also compare GPT-o1 (o1-2024-12-17) against GPT-4o (gpt-4o-2024-08-06), as they represent adjacent releases with a minimal temporal gap. All results are reported in percentage format.

We notice that, when controlling for model family and release proximity, reasoning-augmented models continue to underperform their non-reasoning models on SR generalization, while both perform similarly on NR. This aligns with our core claim that reasoning can impair inductive generalization under hidden rule settings. Full results for the other games will be included in the revised version of the paper.

W2/Q2. While the theoretical framework is well-formalized, it lacks intuitive illustrations or concrete examples that would aid comprehension.

A2. We apologize for the confusion. For each type of error in our theoretical framework, we have provided illustrative examples with visualizations in Appendices G, H, and I. In each case, we highlight the specific sentence corresponding to the error type.

W3/Q3. Ablation experiments to isolate the individual effects of each proposed intervention.

A3. We thank the reviewer for prompting a closer look at the effects of each intervention. We have evaluated the individual effects of each proposed intervention, as shown in Figure 3, where each improvement is added separately to the baseline accuracy using different colors. However, the figure’s compact format may have obscured these effects. To address this, we provide a detailed ablation table for the chess game below.

The solving component accounts for the largest gains in SR generalization across all tested models, while summarization and decomposition offer complementary improvements. The figure will be revised for clarity in the final version, and full ablation tables for the other games will be included in the appendix.

W4/Q4. Could these findings potentially be extended to more common reasoning problems beyond the specific cases studied?

A4. We believe these findings can extend to more common reasoning problems, such as mathematical problem solving. Prior work has noted that reasoning-augmented models often generate inflated reasoning traces and redundant self-verifications, leading to inefficient token usage and increased error rates [1]. To test the applicability, we evaluated GPT-o3 on 20 problems from the AIME 2024 exam, comparing accuracy with and without our combined intervention. The results are reported in percentage format.

As shown in the table, the combined intervention improved accuracy by 20 percent, suggesting its potential to mitigate reasoning-related failures in mathematical tasks.

References
[1]. Chen X, Xu J, Liang T, et al. Do NOT Think That Much for 2+ 3=? On the Overthinking of Long Reasoning Models[C]//Forty-second International Conference on Machine Learning.