Exposing the Achilles' Heel: Evaluating LLMs Ability to Handle Mistakes in Mathematical Reasoning

Overall, the primary contribution lies in evaluating the reasoning abilities of state-of-the-art (SOTA) LLMs in mathematical problem-solving, with a focus on mistake detection, etc. The contribution remains somewhat limited in scope.

We respectfully disagree with the characterization that our contributions are limited in scope. The paper offers several key advancements that go beyond a straightforward evaluation of reasoning abilities in SOTA LLMs:

1. Introduction of the MWP-MISTAKE Dataset

2. Comprehensive Benchmarking Across Diverse Models

3. Unveiling Key Weaknesses in SOTA Models

4. Insights into Data Contamination and Generalization

Our contributions provide a foundational step toward understanding and addressing the reasoning limitations of current LLMs, offering critical insights for both dataset design and model development. We hope these clarifications will help.

The paper identifies data contamination as a limitation but could provide more detailed mitigation strategies. This weakens the reliability of some results.

Evaluating data contamination is inherently challenging, especially for closed-source models where training data and architecture details are unavailable. While we cannot definitively confirm contamination, our analysis highlights that contamination is higher in default reasoning steps but significantly lower in synthetically generated steps from smaller models. To mitigate this, we evaluated models on newer datasets like MATHBENCH and JEEBENCH, which were created after the model training period. Despite this, mistake detection performance remains low on these datasets, underscoring the models' fundamental reasoning limitations rather than reliance on memorization.

While the study covers multiple datasets, the models show reduced performance on newer, complex datasets like JEEBENCH, suggesting limited generalization.

We believe this is actually a key insight highlighted in our paper. The reduced performance on newer, complex datasets like JEEBENCH demonstrates a limitation in the models' ability to generalize to novel problems. We do not view this as a weakness of our work, but rather as an important observation that underscores the need for further improvements in model generalization.

Although rectification abilities are discussed, the results indicate that even advanced models struggle with consistent correction, which highlights the need for stronger frameworks for error handling.

Yes, we agree. This is a key insight presented in the paper: When a model successfully identifies mistakes, its subsequent ability to rectify or self-correct those mistakes improve, suggesting a strong interdependence between mistake detection and correction.

These contributions extend beyond existing literature, offering valuable insights into advancing reasoning capabilities in LLMs and laying the groundwork for more robust evaluations of their cognitive processes.

We hope the clarifications provided above help address any concerns and provide clarity. We would be happy to offer additional details or explanations if needed.

Thank you for your review and comments. The concerns raised in your review, such as the reliability of the dataset, clarity of the evaluation setup, and accuracy of rectified reasoning steps, have been thoroughly addressed in our detailed responses below.

1. Novelty in Mistake Detection and Correction Evaluation:

While we acknowledge that prior works, such as $1$ and $2$ , have explored aspects of mistake correction in LLMs, our work addresses a distinct and critical dimension: explicit mistake detection in reasoning chains. Unlike the implicit self-correction abilities assessed in $1$ and $2$ , which focus on a model’s capacity to evaluate or refine its own responses, our study explicitly evaluates whether models can identify errors in reasoning chains—whether these errors originate from the same model or other models.

In detail: ** $1$ ** investigates intrinsic self-correction by measuring a model's ability to judge the correctness of its own generated answers. ** $2$ ** focuses on distinguishing among self-generated responses and selecting the most appropriate one.

In contrast, our work probes a more foundational question: Can LLMs reliably detect mistakes in a given reasoning chain? We argue that mistake detection is a precursor to effective self-correction, as the ability to robustly identify errors demonstrates higher-order reasoning capabilities. Our findings reveal two novel insights:

1. Current models, including state-of-the-art ones, exhibit significant weaknesses in mistake detection, performing inconsistently across both simple and complex problems.
2. When a model successfully identifies mistakes, its subsequent ability to rectify or self-correct those mistakes improves, suggesting a strong interdependence between mistake detection and correction.

These contributions extend beyond existing literature, offering valuable insights into advancing reasoning capabilities in LLMs and laying the groundwork for more robust evaluations of their cognitive processes.

2. Reliability and Accuracy of the Dataset:

We appreciate the reviewer's concerns regarding the reliability of the MWP-MISTAKE dataset and provide additional details to clarify our rigorous dataset construction and verification process.

For the GSM8k and MATH datasets, ground truth step-by-step correct reasoning is directly available in the original datasets. For the other three datasets—MATHBENCH, JEEBENCH, and SVAMP—we used GPT-4 to generate step-by-step Chain-of-Thought (CoT) reasoning steps, followed by manual verification of the entire dataset (4900+ questions) to ensure correctness.

Our verification process specifically addressed the following cases:

1. Correct reasoning steps with incorrect final answers: We observed approximately 144 questions (2% of the dataset) where reasoning steps were correct, but the final answers were labeled incorrectly. This discrepancy often arose due to mismatches in ground truth answer representations (e.g., "frac{5}{10}" vs. "0.5"). Such cases were eliminated, and new questions without such inconsistencies were added.

2. Incorrect reasoning steps with correct final answers: A few rare instances (~10 data points) were identified where reasoning steps were incomplete or incorrect, despite a correct final answer. These questions were also removed and replaced with correctly labeled questions.

This meticulous review process ensured that our dataset is of high quality, with all included questions having both accurate reasoning steps and correct final answers. We acknowledge that providing detailed statistics on manual verification strengthens the reliability of our dataset. We will include these details in Section 2 (MWP-MISTAKE Dataset) and expand on them in Appendix A of the revised paper to ensure transparency and validate the dataset’s accuracy.

3. Clarity of Evaluation Setup:

The primary evaluation is conducted in Task 1, where the model is provided with a question along with reasoning steps. The model must:

1. Identify the correctness of the reasoning steps.

2. Rectify mistakes in the reasoning steps (if any).

3. Derive the final answer accurately.

We employ three distinct metrics to evaluate the models: Mistake identification, performance in deriving the Correct Answer, Mistake Rectification. The F1 score is computed for these tasks as follows:

1.      For mistake detection, the model outputs either "yes" (correct reasoning) or "no" (incorrect reasoning). These predictions are compared against ground truth labels ("yes"/"no") to compute the F1 score.

2.      For performance in deriving the correct answer and mistake rectification, the final answer generated by the model is compared to the ground truth final answer to compute F1 score.

4. Smaller model reasoning steps in table 3:

In Table 3, "Smaller Model Reasoning Steps" refers to the incorrect reasoning steps that are generated by smaller language models.

Thank you for your insightful review and positive feedback. Below are our responses to address your concerns and we would be happy to provide any further clarifications as required.

In the figure 1, GPT-3.5 Turbo fails to detect errors while GPT-4o succeeds. Can GPT-3.5 Turbo correctly solve this question?

Figure 1 illustrates an example where GPT-4o successfully detects and corrects errors in the reasoning chain to arrive at the correct answer (-9). In contrast, GPT-3.5 Turbo fails to detect the errors, resulting in an incorrect final answer (18). This highlights GPT-3.5 Turbo's limitations in mistake detection and correction in this specific example. However, as shown in Table 4, GPT-3.5 Turbo can still solve ~53% of questions correctly despite incorrect reasoning, compared to GPT-4o, which achieves 79% accuracy. We hope this clarifies the model performance differences.

Regarding the metric (line 186), is the F1 score for step-level or path-level detection performance?

The F1 score measures path-level detection performance for the complete reasoning chain, not step-level detection. Thank you for pointing this out; we will clarify this in the revised manuscript.

I suggest presenting the results of model performance in solving questions alongside error detection. Are questions that models solve correctly easier for them to detect errors compared to more challenging questions?

Thank you for the suggestion. While presenting model performance alongside error detection would be insightful, including all models and datasets in one table could overwhelm readers and hinder clarity. Instead, we propose adding a focused analysis for a subset of models and datasets in the revised paper to explore these relationships clearly.

Additionally, as noted in the paper, there is no consistent trend linking a model's problem-solving accuracy with its ability to detect errors. For instance, on the simpler SVAMP dataset, models achieve high accuracy but struggle with mistake detection, while on the more complex JEEbench dataset, overall accuracy drops, yet error detection slightly improves. This highlights the models' reasoning limitations, independent of question complexity.

In Table 3, is the average computed based on all data samples? The data sizes from different sources are unbalanced. Additionally, compared to GSM8K, why does detection performance decrease on D but increase on SM (e.g., JEEBENCH)?

Regarding the difference in detection performance between D and SM, models generally perform poorer on D because errors in D are typically localized to a single reasoning step (e.g., shuffling numbers or operators). In contrast, for SM, smaller models generate reasoning chains, resulting in errors that causally propagate across multiple steps. These propagated errors are often easier for models to detect due to their prominence and consistency throughout the reasoning chain.

We hope this clarifies the observed trends and addresses your concerns.

I disagree with the statement in lines 253-255 that “GPT-3.5 Turbo performs similarly to GPT-4 and even surpasses it on certain datasets like GSM-8K.” The results in Table 4 do not support this as well.

The statement in lines 253–255 is part of Section 4.2, which discusses mistake detection performance, specifically referencing the results in Table 3, not Table 4. In Table 3, it is evident that GPT-3.5 Turbo performs better than GPT-4 on GSM8K for mistake identification and demonstrates performance very close to GPT-4 across all datasets for this task. We hope this clarifies the misunderstanding regarding the context.

Citation issue? (line 44)

We will fix this issue, this was an oversight as the bibliography format changed the author name did not appear correctly.

Some relevant work in evaluation is overlooked, such as "MR-GSM8K: A Meta-Reasoning Benchmark for Large Language Model Evaluation".

Thank you for highlighting this recent work. MR-GSM8K introduces an interesting shift in evaluation by scoring solution correctness rather than focusing solely on question answering. While its high-level objective of assessing reasoning capabilities aligns with ours, there are key distinctions:

1. MR-GSM8K_ primarily generates reasoning chains (e.g., original, programmatic, reverse) and evaluates their correctness through scoring, focusing on dataset creation and analysis.

2. In contrast, our MWP-Mistake dataset is designed to emulate real-world educational setups by generating incorrect reasoning chains using both rule-based techniques and outputs from multiple LLMs, enhancing diversity and realism.

Moreover, our work goes beyond mistake detection: We not only evaluate models' error detection abilities but also their capacity to rectify mistakes. Our work offers in-depth insights into data contamination and memorization. Additionally, our evaluations cover a wider range of datasets and scenarios, making our contributions more comprehensive.

Validity of Rule-based Methods

For example, shuffling reasoning steps or numeric values often results in errors that do not reflect authentic human errors in mathematical reasoning, which can be distracting to models. I would suggest refining the rule-based methods to better mimic typical student errors, which could give a more valid evaluation of models’ ability to detect naturally occurring mistakes.

We appreciate the reviewer’s concern regarding the coherence of rule-based transformations. To address this, we want to emphasize that the rules applied in our paper are grounded in discussions with grade-school educators, who provided insights into common mathematical reasoning errors made by students. While some rules may appear trivial at first glance, they capture authentic patterns of human errors that arise during problem-solving. Below, we provide examples to clarify the real-world applicability of these rules:

1. Shuffle Numerical Values: A student might misread a problem or inadvertently swap numbers, such as treating "5 apples and 3 oranges" as "3 apples and 5 oranges." This is a typical misstep when transcribing or interpreting numerical data.
2. Replace Numerical Values: This error mimics cases where students substitute incorrect values, e.g., replacing "radius = 7" with "radius = 5" when solving for the area of a circle. This often occurs due to lapses in focus or misreading.
3. Shuffle Operations: A common reasoning error is misunderstanding the order of operations, such as treating "2 + 3 × 4" as "(2 + 3) × 4." This reflects students' struggles with applying mathematical precedence correctly.
4. Insert Random Reasoning: Students sometimes include irrelevant steps or overcomplicate their work, such as writing "Because the train moves faster than the car, 2 × 5 = 10." These distractions, while not inherently erroneous, disrupt logical flow and mirror overthinking.
5. Shuffle Reasoning Steps: The sequence of reasoning is critical in problem-solving. For instance, solving for the area of a triangle by calculating "base × height" before dividing by 2 is correct, but reversing this order introduces confusion and can mislead students. Such missteps highlight the importance of logical step sequencing.
6. Delete Reasoning Step: Omitting steps often reflects missed applications of formulas or logical progression, e.g., skipping the application of the Pythagorean theorem when solving for the hypotenuse in a right triangle. While this omission might not always lead to an incorrect answer, it risks undermining clarity and correctness in reasoning.

These transformations do not solely aim to generate incorrect answers; they evaluate the model's robustness in navigating incomplete, shuffled, or flawed reasoning to derive the correct solution. Particularly, "shuffle reasoning" and "delete reasoning" mimic challenges faced by students, such as disrupted clarity or skipped steps, emphasizing the importance of logical step-by-step progression.

Thus, our rules are carefully designed to reflect genuine reasoning errors while testing models' ability to detect, rectify, and reason coherently.

Could this be the reason why models struggle more with rule-based errors compared to SLM-generated mistakes?

We appreciate the reviewer’s insightful observation. The difference in model performance on rule-based versus SLM-generated mistakes stems from the nature of these errors.

SLM-generated mistakes often propagate causally across multiple steps, creating consistent patterns that are easier for models to detect. These propagated errors leave a clear "footprint" throughout the reasoning chain.

In contrast, rule-based errors are localized to a single reasoning step, such as shuffling numbers or operators. Their subtle nature makes them harder to spot, as they lack the broader contextual cues provided by propagated errors.

Additionally, deleting reasoning steps raises a question about whether such omissions genuinely impact the final answer’s correctness or reasoning quality. In cases where reasoning steps are deleted but the answer remains correct, it would be insightful to investigate whether models still detect these partial solutions as errors and if such ambiguities lead to lower error detection rates.

Thank you for pointing this out. To ensure that the models' poor mistake detection performance is not driven by false positives from the "shuffle reasoning steps" and "delete reasoning steps" categories, we conducted additional analyses. Specifically, we eliminated these two rule-based categories from the evaluation and recomputed the three metrics.

The results showed that the overall performance across the metrics varied by only 1-3%, confirming that these categories do not significantly impact the observed trends. This demonstrates that the poor performance is not due to ambiguities in these cases. We will include the detailed results of this analysis in the appendix.