Can Large Language Models Infer Causation from Correlation?

Thank you for your positive feedback on our paper, particularly for recognizing the novelty of our dataset, the detailed technical process of data construction, the extensive experimentation with 17 language models, and our in-depth further experiments and analysis.

Reviewer's Query on Dataset Construction Process:

The dataset construction process utilizes variables to represent the input data. Could you clarify what these variables signify? Do they represent entities within sentences, the sentences themselves, or something else? More detailed information would be beneficial.

Response: Thank you for raising this important question. We provided some examples in our “Appendix B: Generating Natural Stories.” In our data, variables are symbolically named and can be verbalized to represent entities, such as “eating junk food (A), obesity (C), and watching television (B),” and the inference problem can be

Premise: Let’s consider three factors: eating junk food (A), obesity (C), and watching television (B). There is a correlation between eating junk food and obesity, and between watching television and obesity. However, eating junk food and watching television are independent from each other. Hypothesis: Eating junk food directly affects obesity.

Note that our current Corr2Cause dataset primarily employs symbolic forms to distill reasoning patterns (which adheres to our motivation to improve pure causal reasoning). With this piece of solid technical work, we provide good potential for future work to instantiate the variables using various real-world entities, facilitating the generation of natural stories that align with practical applications, such as combating misinformation.

Reviewer's Query on Causality Discovery:

The paper only discusses results obtained using the PC algorithm for causality discovery. The precision of this algorithm is not clearly stated, raising questions about the reliability of the results. Could you provide more details on how you ensure the confidence in the results generated by the PC algorithm?

Response: Thank you for the question. We selected the PC algorithm for its suitability in processing inputs formatted as correlations. Traditionally there are two main types of methods for causal discovery: constraint-based methods (based on the dependence and independence relations among variables), and score-based methods (which takes in the data distribution, and rank the possible causal graphs by their scores given the data, e.g., Bayesian Information Criterion (BIC) score). In our dataset, we made a design choice that natural language tests usually do not explicitly contain data samples to run statistical tests on (which is where the confidence scores come from), but instead they typically contain observations of which variables are related or independent. Therefore, we design our task as the Corr2Cause inference task, which looks at the core task of inferring causal conclusions based on the presented correlations, and bypasses the step of statistical testing from raw data samples. For this task with correlations as input, constraint-based methods, specifically the most classic PC algorithm, is a great fit. As the dependent and independences are provided, there will not be statistical tests to produce confidence scores, but only algorithmic reasoning to identify the causal graphs deterministically.

The remaining major requirement of the PC algorithm is the faithfulness assumption, which we introduce in the preliminaries section. We adhere to this faithfulness assumption, which is generally met in real-world scenarios, although exceptions can occur if the causal mechanism is intricately designed to obscure statistical dependencies. We appreciate your attention to this matter. We will make sure to emphasize this point again in the limitations section, and suggest future work exploring different causal discovery algorithms.

We appreciate your time and feedback on our paper. Feel free to let us know anytime if you have further questions.

It's not a critical issue, but I'm curious if the authors had sufficient computational resources for this. While the generalization experiment design is commendable, it's unsurprising that BERT-style models do not generalize well to paraphrases and new variables. The evaluation could be more compelling if you could fine-tune some smaller-scale GPT-style models for this experiment.

We appreciate your insightful suggestion and your consideration of our computational resources. During the rebuttal period, we managed to conduct experiments with small-scale GPT-style models, including gpt2 (124M parameters), gpt2-large (774M), gpt2-xl (1.5B). We further enriched the results by including the latest LLaMa series of models, LLaMa-7B and LLaMa2-7B, for which we run the LoRA versions which are more lightweight.

First, we show the overall performance results below. We find that for each model series, the larger or later ones achieve better results, and overall GPT2-XL after fine-tuning (gpt2-xl-ft) achieves the best performance, on par with the fine-tuned Roberta model's 94.74% F1.

Additionally, inspired by your previous suggestion, we obtained fine-grained performance data for these models. Due to space constraints, we are presenting a selection from the series of GPT-2 models:

We can see from above that the collider and descendent relations are relatively challenging for smaller GPT-2 models, and for the largest GPT-2 model, gpt2-xl-ft, it achieves good performance at all causal relations.

Finally, we conducted two robustness tests on these models, including Variable Refactoring (VR) and Paraphrasing (P). The results are as follows:

We can see a largest drop of performance across all finetuned models, which is consistent with our observation in the paper. And also variable refactorization to swap the variable names causes a larger drop than paraphrasing, which also aligns with our observation in the paper.

Thank you again for all your comments, and don't hesitate to let us know if you have any further questions.

Thank you for your positive comments on our paper, particularly for recognizing the value of the task in distinguishing causality from correlations, detailed technical processes we used to construct the dataset, and the comprehensive evaluation in the experiments.

Have you done any sub-sampling for the dataset construction? Or have you included all possible examples and hypotheses in the dataset? This aspect was not entirely clear to me.

In constructing the Corr2Cause dataset, we included all possible examples and hypotheses. This comprehensive approach ensures that our dataset captures the full spectrum of inference tasks of all combinations of correlations among the given set of variables.

Do you have fine-grained evaluation results (like Table 6) for non-fine-tuned models? I found Table 6 particularly informative.

Thank you for this constructive suggestion. In response to your request, we have obtained fine-grained evaluation results for non-fine-tuned models in the following table:

These results are particularly revealing, showing how off-the-shelf models perform in recognizing specific relations. Specifically, GPT-3.5 cannot recognize ancestor relations, whereas GPT-4 fails at all direct causation recognition with parents and children. And RoBERTa MNLI only did collider relation relatively correctly. Note that when the F1 score is zero, the positive accuracy results from predicting the negative class of that relation.

In order to ensure comprehensive coverage for the development and test sets, this work excludes all 2-3 variable examples from the training set. I'm concerned that this decision may render the dataset overly challenging and question whether such a split is optimal for this task.

Thank you for highlighting this concern. Our objective was to balance the coverage of the training set with the comprehensiveness of the test and development sets. When faced with trade-offs due to the limited number of 2-3 variable examples, we prioritized the quality of the test and development sets. As we discussed in the paper, even fine-tuned models exhibit limitations in robust causal reasoning, as it might fall into spurious correlations. Therefore, we believe that the test set serving as unseen data for LLMs well have more value than putting things into the training data and making models memorize them.

On a separate note, we understand that this is a challenging task, and perhaps to mitigate this issue, future research may opt to use the 2-3 variable cases in the development set for few-shot prompting, or reallocate the 2-3 variable data samples between the training and development sets to optimize the performance.

Despite the clear motivation behind evaluating the model's capability in causal discovery, I find it challenging to pinpoint the exact bottleneck of this task.

Thank you for your insightful comment. Following your suggestions, we are conducting a more fine-grained error analysis by utilizing chain-of-thought prompting to explicitly generate and examine the step-by-step reasoning of LLMs, and we're manually annotating different types of reasoning errors in a detailed manner. This analysis is currently in progress, and we plan to share the results either in this discussion thread, or directly later in the camera-ready version.

Thank you for the overall positive feedback on our paper, particularly regarding the novel and intriguing nature of the Corr2Cause task, and its evaluation across a diverse set of language models.

Optimizing LLM Performance

I appreciate the task the authors are addressing, and the approach they have chosen. The major weakness of the paper I am concerned about are those relating to how the models are prompted to improve LLM behavior (lack of system prompt, few shot examples, chain of thought reasoning) [...] would the LLM have succeeded with different prompting strategies?

We are grateful for your insightful query, especially regarding the optimization of prompting strategies when reporting LLM performance. Reflecting on this, we see merit in both our paper's approach, which evaluates LLMs based on plain queries that a typical user might pose, and your suggestion, which explores the potential for maximizing performance with tailored prompts.

In response, we conducted new experiments incorporating the three strategies you mentioned: system prompts, few-shot examples, and chain-of-thought reasoning. We present the results as follows (and will keep adding new experiments to extend to more models and different combinations of the strategy):

The enhanced query is achieved by combining all three strategies, which is 4 points better on F1 than the original performance. However, we can see that despite adding all three strategies, GPT-3.5 still does not do well at this challenging Corr2Cause task. As for experimental details, for the enhanced query, we (1) use a system prompt to indicate the expertise ("You are a highly intelligent question-answering bot with profound knowledge of causal inference."); (2) include two few-shot examples, one positive, one negative; and (3) add the chain-of-thought prompting with "Let's think step by step.") to induce language models to produce step-by-step reasoning. Also, since the initial experiments were conducted earlier this year, we rolled back to the closest previous GPT checkpoints in March.

Significance of Pure Causal Reasoning

Why is "pure causal inference" the right task for evaluating the causal capabilities of large language models? For example, the formal reasoning approaches describes in Section 2 that guide the creation of Corr2Cause are relatively new inventions in human history. But I think we'd hesitate to say that people could not reason causally before, e.g., DAGs were invented. This seems like a good opportunity to clarify what underlying functionality we are trying to achieve and how pure causal inference vs other causal reasoning tasks relate to broader goals and purposes. And to be perfectly clear, I think this is a fine task to evaluate but would like to learn more about the authors' perspectives on relative importance, etc. of testing different facets of causal reasoning.

Thank you for this thoughtful question and your openness to discussion. Our proposal for the Corr2Cause task is inspired by interdisciplinary insights, particularly from philosophy, where reasoning is categorized into pure (rationalism) and empirical (empiricism) reasoning. The latter, empirical reasoning, is usually framed as commonsense reasoning in existing NLP literature, which corresponds to your intuition that people can reason before, e.g., DAGs were invented.

In contrast, pure reasoning, or a priori reasoning, involves logic and abstract thought, independent of sensory experience. In the causal domain, we believe that the necessity of pure causal reasoning is analogous to that of math, which aid human understanding independent of sensory experience. As you mentioned the natural story realizations in our Appendix B, we highlight that our symbolic Corr2Cause reasoning pattern holds regardless of the instantiation of the variables, which makes us rely on these inference rules independent of our experience, or even overcome cognitive biases that confuse correlation with causation.

Upon reviewing existing NLP datasets on causal reasoning, we found a predominant focus on empirical commonsense reasoning, with little to no emphasis on pure reasoning in the literature. Recognizing the longstanding philosophical value of pure reasoning, which dates back to early Greek philosophers like Parmenides and Pythagoras and is further developed by figures like Plato, Descartes, and Kant, we believe in its significant potential to enrich human knowledge beyond empirical experiences.

Both rational and empirical forms of knowledge are crucial, complementing each other in expanding the realm of human understanding. This goal is what we aim to highlight and explore through the Corr2Cause task.