MIRAGE: Evaluating and Explaining Inductive Reasoning Process in Language Models

Thanks for your careful and insightful reviews.

• Question 1：Is “Neighborhood-Based Reasoning” reasoning or pattern matching?

• Reply 1: We apologize for not making this point clear in the paper. In fact, the experiment in Section 4.4 was conducted precisely to address this question. As you can see, in Figure 7b and 7d, we test the model's deductive performance on arbitrary test samples, and the results show that the model maintains a deductive density of over 70% under the IF setting. This indicates that even when the patterns of the test samples differ significantly from those of the observed samples, the model can still perform reasoning through neighbor-based reasoning. If this is a type of pattern matching, the model would only maintain high performance on nearby test samples, rather than sustaining such high performance globally. Therefore, we believe that neighbor-based reasoning is a form of reasoning rather than pattern matching. Thanks for your suggestions, we will further emphasize this term in the revised version of our paper.

• Question 2: It is not surprising that LLMs are good at "predicting" based on neighborhood context.

• Reply 2: In fact, the neighborhood context in the next-word prediction task has, in essence, some fundamental differences from the neighboring facts identified in our work. For the former, this type of "neighborhood" refers to words that frequently co-occur in human language corpora. However, for the latter, "neighborhood" refers to the proximity between feature vectors in the vector space (see Section 4.1). This proximity in the mathematical space is rarely explicitly represented in the text. Therefore, it cannot be naturally assumed that training on the next-word prediction task will necessarily improve the model's ability for this type of neighbor-based reasoning.

• Question 3: Why do you only mention inductive reasoning in the title?

• Reply 3: We apologize that we don’t illustrate this term clearly. Actually, as we mentioned in lines 33-36 of the introduction, most related works incorporate both the rule induction (inductive) and rule application (deductive) phases into the inductive reasoning process [1,2,3]. Following them, we consider the entire inductive reasoning process as a type of reasoning, with inductive and deductive being its two components. As you can see, several representative related works also use "inductive reasoning" alone in their titles [1,2,3].

• Question 4: The relations between the input and the output of rules? How do you distinguish deductive from inductive reasoning?

• Reply 4: From my understanding, you may be concerned that rule induction might be an associative byproduct of the model's inductive reasoning process rather than a necessary phase.

  • In fact, the experiments in Section 3 are conducted to investigate the relationship. In prior work, from the perspective of human reasoning, rule induction and rule application are clearly divided into two separate phases [1,2,3]. Therefore, the input and the output of rules are viewed as having a causal relationship. However, our experiments challenge this viewpoint, suggesting that rule induction may not be necessary—that is, it might simply be associative without implying causation.
  • We apologize that we don’t illustrate this term clearly. Actually, our work aims to investigate the role of rule induction in the overall inductive reasoning process, without focusing on specific types of relationships. We can measure the accuracy of induction through a rule-based question-answering task, thereby assessing the correlation between rule induction and the final performance. Measuring this correlation does not require a strict distinction between inductive and deductive reasoning. In this paper, we simply distinguish between inductive and deductive based on the form of the test questions rather than the formal definitions in logic.

  Thanks for your suggestions, we will make further discussions on this term in the revised version of our paper. If I have misunderstood your question in any way, please feel free to point it out.

[1] Wang, Ruocheng, et al. "Hypothesis Search: Inductive Reasoning with Language Models." The Twelfth International Conference on Learning Representations.

[2] Qiu, Linlu, et al. "Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinement." The Twelfth International Conference on Learning Representations.

[3] Bowen, Chen, Rune Sætre, and Yusuke Miyao. "A Comprehensive Evaluation of Inductive Reasoning Capabilities and Problem Solving in Large Language Models." Findings of the Association for Computational Linguistics: EACL 2024. 2024.

Thanks for your careful and insightful reviews.

• Question 1: It’s unclear where the features in the feature space originate.

• Reply 1: We apologize for not making this point more clear in the paper. Actually, we mention the feature in lines 368-370, which refers to the input vectors of different facts. Thanks for your suggestions, we will further emphasize this term in the revised version of our paper.

• Question 2: The data setting may lack sufficient complexity to effectively demonstrate inductive reasoning capabilities in real-life scenarios.

• Reply 2: As we mention in lines 181-184, in Mirage, we design the scenario of real-world problem (RP) to evaluate the inductive reasoning capabilities in real-life scenarios. Based on Table 1, in this task, all of the advanced LLMs achieve an accuracy of less than 0.25 (D=8). This indicates that the task is already very challenging for models, further demonstrating that the task complexity is sufficient for current models (even though they are relatively easy for humans).

• Question 3: Additional insights or empirical evidence on the model's performance with real-world datasets or in less controlled environments.

• Reply 3: We show the experimental results of real-world problems (RP) in Table 1 and 2. Based on them, we demonstrate that models show poor performance on them. We infer that this is due to the presence of a large amount of irrelevant natural language text in real-world problems, which makes it significantly challenging for LLMs to extract mathematical patterns (e.g. rules, facts) from the context.

• Question 4: Does fine-tuning over the synthetic data help the inductive reasoning of LLMs?

• Reply 4: Thanks for your suggestions. Here, we fine-tune the Llama3-8b-chat model using LoRA on 8,000 training examples and evaluate its performance on the test set. We report the performance of the list transformation task with D=5, N=5 as follows:

  Task	ICL	Fine-tune
  Inductive	0.31	0.89 (+0.58)
  Deductive	0.27	0.82 (+0.55)

  The results demonstrate that fine-tuning the model on synthetic data can significantly improve its inductive reasoning capabilities. We will report the details and complete results of this experiment in the revised version of the paper once all experiments are completed.

Thanks for your careful and insightful reviews.

• Question 1: Claims such as "we prove that LLMs are poor in inductive reasoning" would be too strong.

• Reply 1: We apologize for the lack of clarity in our expression here. Thanks for your suggestion, we will modify these claims in our revised version. For example, "We demonstrate that LLM is a relatively poor rule-based inductive reasoner compared to direct deduction".

• Question 2: A deductive inference needs both input facts and the logic rules for deducing the outputs.

• Reply 2: We apologize for not making this point clear in the paper. Here, we distinguish between inductive and deductive based on the form of the test questions rather than the formal definitions in logic. In our paper, we aim to investigate whether the model must undergo rule induction to complete deductive tasks. Therefore, we design this task form without including the rule as input, allowing us to directly evaluate the model's performance on the deductive test examples. Besides, we also implement rule-based inference in the ID method described in Section 3.2, and the experiments conducted with this method also support our conclusion. Thanks for the suggestion, we will further clarify this inconsistency in the revised version of our paper.

Thank you for your response and valuable suggestions. We will incorporate the following content into the revised version of the paper to better clarify the definition of our tasks:

For humans, although the process of reasoning tends to be implicit, according to the traditional definition in logic, it can be divided into two stages: induction and deduction. Here, we focus on the objectives of these two stages: deriving the correct rules through induction and making inferences on new instances using deduction. Therefore, we define the following two tasks in Mirage:

• Definition 1 (Rule Induction Task): Given an observed fact set, this task evaluates whether the model can successfully infer the underlying transformation rules behind these facts
• Definition 2 (Example Inference Task): Given an observed fact set, this task evaluates whether the model can successfully perform inference on unseen instances that follow the same rules.

It is worth noting that our second task differs from traditional deductive tasks as it does not take rules as input. Its purpose is to evaluate the model's final performance on new instances, without requiring the model to explicitly undergo a rule induction process. Therefore, we can assess the model's reliance on rules in its reasoning process by comparing its performance on these two tasks.

If you have any remaining questions or concerns about our response, we would be happy to engage in further discussion with you.

Thanks for your careful and insightful reviews.

• Question 1：The writing is not clear.

• Reply 1：We apologize that we don’t illustrate these terms clearly. Here we provide additional details regarding them.

  • a) As we mention in lines 414-415, "make the fact set X contain only one type of the fact". For each fact in the fact set, if it is not of the specified type, we remove it and regenerate a new fact until all facts in the set are of the specified type (i.e., IF, CF, or OF).
  • b) As we mention in a), after finding IF, CF and OF, we replace facts that do not meet the definition. Therefore, we ensure that the entire fact set contains only one type of fact, which allows the model to observe and perform the corresponding deductive task under different settings.
  • c) As we emphasize in lines 194-195, the fundamental metric for our entire dataset is accuracy.

  Thanks for your suggestions, we will further emphasize these terms in the revised version of our paper.

• Question 2: The claim "We prove that LLM is a poor rule-based inductive reasoner" is too strong and unacceptable. What is the definition of "poor"? And have the authors proved it mathematically?

• Reply 2: We apologize for the lack of clarity in our expression here. Here, "poor" refers to the model's low reliance on rules during the inductive reasoning process. This does not refer to absolute performance, but rather indicates that the model's performance on inductive tasks is relatively poor compared to its performance on deductive tasks. For the proof, we believe that the poor reasoning performance of the model can only be demonstrated experimentally, not proven mathematically. Therefore, through the experiments in Section 3, we provide empirical evidence to support this claim. Thanks for your suggestion, we will modify this claim to "We demonstrate that LLM is a relatively poor rule-based inductive reasoner compared to direct deduction" in our revised version.

• Question 3: Have the authors proved the neighbor-based inductive reasoning mathematically? When LLM is a neighbor-based inductive reasoner?

• Reply 3: We apologize for the lack of clarity in our expression here. Like we mention in Reply 2, the reasoning performance can not be proven mathematically. Instead, we provide empirical evidence to support this claim through the experiments in Section 4. Thanks for your suggestion, we will modify this claim to "We demonstrate that LLM is a neighbor-based inductive reasoner" in our revised version. For the second question, we provide a detailed response in Reply 4.

• Question 4: Lack of a more in-depth analysis of when LLMs use neighbor-based reasoning, and when LLMs use the general pattern thinking?

• Reply 4: In fact, we have conducted a preliminary exploration of this question in Appendix C.4. From the experimental results in Table 12, we can conclude that the model mainly uses the general pattern thinking when there are few neighbor facts in the context (e.g. OF). Otherwise, they rely more on neighbor-based reasoning. However, in some cases, the two paradigms may still be coupled together, and therefore, we cannot categorically exclude the influence of either paradigm on the reasoning process.

Dear Reviewer

Thank you for your invaluable suggestions, which have greatly contributed to improving our paper. We would be truly grateful for your feedback on our revisions. Have our responses addressed your concerns, or are there additional issues you would like to discuss? With the discussion period extended, we hope to have the opportunity to further engage with you on these matters. We sincerely look forward to hearing your thoughts and greatly value your input!

Dear Reviewer 6pHZ,

I hope this message finds you well.

First and foremost, please allow me to extend our sincere apologies for the inconvenience caused by this follow-up correspondence. We are fully aware of the demands of your time and appreciate the dedication and effort you have already shown during the review process of our manuscript.

As we are nearing the end of the rebuttal phase, we are keen to ensure that all concerns have been adequately addressed. We have made every attempt to respond to each of your comments with the utmost care and attention. Our goal is to ensure that our manuscript is as strong and as responsive to the reviewers' feedback as possible.

We kindly request your confirmation on whether our responses have satisfactorily resolved your concerns. As the deadline for the rebuttal is drawing near, we find ourselves in a position where we need to seek a status update on your review. We are fully aware that the review process can be time-consuming, and we appreciate your efforts in this regard.

Once again, we apologize for any disturbance and thank you for your understanding and support.

I appreciate the author's response.

For Q3, "We demonstrate that LLM is a neighbor-based inductive reasoner", I guess some context should be needed? Do the authors imply that LLMs always conduct neighbor-based inductive reasoning for any questions?

For Q4, I have checked Appendix C.4, but it seems more details can help it more clear. For example, "... exceeds 70% when there are fewer neighbor facts in the context. This demonstrates that the model tends to rely more on rule-based induction". What is the logic here? The authors in their rebuttal suggest to take a look at Table 12, but no context in the paper has referred to Table 12. It is unclear what message can be obtained from it. Can the authors give more explanations to Appendix C.4 to support the existing conclusions? A clear and persuading anslysis on the takeway message (e.g., on when LLMs use neighbor-based reasoning, and when LLMs use the general pattern thinking/rule-based reasoning?) could largely benefit the paper.

I would also suggest the authors to take care of the conclusions. For example in the conclusion section, "we demonstrated that LLM is a poor rule-based reasoner, it does not need to rely on inductive rules when performing inductive reasoning." --- what does it mean to be a "poor rule-based reasoner"? Does it only mean LLM doesn't rely on rules to perform reasoning, or does it also mean LLM can't induce rules? Similar problems are very common throughout the paper. I would also suggest the authors to check each conclusion made in this paper on overclaim or ambiguity issues.

I have increased the presentation score from 2 to 3. I will continue update the overall rating if
(1) a clear and persuading anslysis on the takeway message can be provided, and
(2) the overclaim and ambiguity issues on conclusions are properly solved in this paper.