Failure Modes of LLMs for Causal Reasoning on Narratives

1. The core contributions and findings of this paper appear to heavily overlap with the previously published work LLMs Are Prone to Fallacies in Causal Inference, which appeared on arXiv in June 2024 and was subsequently published at EMNLP 2024. However, this highly relevant and similar prior work is not cited or discussed. Moreover, the technical details and experimental validation are less comprehensive than in the aforementioned paper. Ensuring proper citation of relevant prior work is crucial for advancing academic discourse and providing a transparent foundation for new contributions.
2. The evaluation uses yes/no questions, meaning the random guess baseline is 0.5. It is concerning that many results for GPT-4, GPT-4o, and Claude-3.5-Sonnet are below this baseline. While this might be theoretically possible under certain circumstances, the paper does not provide sufficient explanation or analysis of this phenomenon. This lack of explanation weakens the validity of the experimental findings and necessitates further investigation and a detailed error analysis.
3. The paper lacks essential details about the evaluation dataset, making it difficult to assess the validity of the results. Crucially, the number of samples tested, the number of nodes in each graph, and the specific types of events present in the narratives are not clearly specified. While line 149 mentions that "events are real-world phenomena", this description is too broad and lacks necessary citations. Furthermore, the inconsistency between Figure 5(b) (maximum chain length 9) and Figure 6(b) (maximum chain length 10) raises questions about whether different datasets were employed for the synthetic and real-world narratives, or is this an oversight? Clarification on these points is essential.
4. The paper does not adequately address how the quality of the generated narratives is ensured. How are the narratives verified to ensure they conform to the specified chain structures? What mechanisms are in place to check and assess the data quality? Furthermore, given the identified issue of parametric knowledge conflicts within the models, how does the generation process prevent the introduction of factual errors or inconsistencies into the narratives? This lack of detail raises concerns about the reliability of the experimental data.
5. While the paper's focus on chain causal relationships provides a starting point, the scope remains relatively narrow. Exploring more complex scenarios is crucial for a more comprehensive understanding of LLM's causal reasoing ability.
6. The model selection is quite limited, using only GPT-4, GPT-4o, and Claude-3.5-Sonnet for both data generation and evaluation. This raises concerns about the generalizability of the findings. Including leading open-access models, such as Llama 3.1, would strengthen the analysis and provide insights into whether the observed phenomena are specific to limited-access models or hold across a wider range of LLMs.
7. The paper's logical structure could be improved to enhance clarity and readability. Additionally, the writing also requires significant revision to ensure the work is accessible and understandable to the reader.
8. The poor quality of the figures hinders understanding of the presented work. For example, Figure 3 is blurry and difficult to interpret.
9. The referencing of both articles and figures needs substantial improvement. Several errors were noted, including incorrect article citations on lines 156-157 and incorrect figure references on lines 248, 304, and 320.

While I agree on many interpretations of the presented results, I have several questions regarding the soundness of some experimental setups and the provision of graph information:

1. In section 3.2, the authors give an example of a "Film festival → Food truck rally → Trampoline park party" narrative. While the goal of generating a reversed narrative was pursued, the term 'Trampoline park party' was again mentioned after the 'Film festival' in the last sentence, establishing a relation in the correct relation order. Enforcing items of the causal chain to appear in reversed order, might pose a challenging task, as item positions can get switched due to the syntax of natural language. It would be interesting to know, how often such generation 'errors' occur within the data. The paper could be improved by analyzing to which extend LLM generations obey the desired ordering of events.
2. Throughout the paper, the authors refer to 'anti-causal' effects to indicate inversion of the causal effect strength, e.g. "cancer->longer life". The term 'anti-causal', is commonly reserved to indicate a reversed causal direction e.g. "cancer<-longer life". This is quite confusing, and given that it appears in the prompt presented in A.1.3, might have an impact on narrative generation. I would like to suggest to consider an alternative wording, e.g. counterfactual effects.
3. The authors provide causal graph information to the LLM during several experiments. While this seems to improve LLM performance, I could not find any details on the exact format of the provided graph information. It could be, that graph information is provided in the correct causal order, which would again improve LLM performance due to ordering bias and not due to reasoning capabilities of the LLM. The paper could be improved by further specifying and, if needed, randomizing the order of the provided causal effects in the experiments.
4. The reordering of relations in the experimental setup of section 3.3 might conceal the true influence of the anti-causal edges on the predictive performance of LLMs. As reordering was previously shown to reduce performance, it becomes unclear whether anti-causality or reordering of relation is the true cause for the performance degradation. I believe that effects of anti-causal relations might be better demonstrated when keeping the ordering intact and simply varying the amount of anti-causal relations within the chains.
5. To the best of my understanding, the problem of identifying and reasoning about causal relations in texts is commonly known as event causality identification (ECI; in the field of NL) or simply event causality (when moving closer to classical Pearlian causality). While the authors are able to produce new insights on the failure modes of LLM, several prior works derived similar results and are not discussed. This regards general reasoning with counterfactual contexts (e.g. [1,2,3]), as well as general causal text understanding (e.g. [4,5]). While these works certainly form a niche within the bigger field of causality and LLMs, I would recommend the authors to consider these relations.

Minor

• Unnecessary brackets are added for citations in Sec. 3.1 (l156/157) and in Sec. A.1.9.
• The resolution of figure 3 is quite low, such that the legend is unpleasant to read. I would like to suggest to either embed the figure as a vector graphic/PDF or increase its resolution.
• Some words seem to be missing in line 248.

[1] Li, Jiaxuan, Lang Yu, and Allyson Ettinger. "Counterfactual reasoning: Do language models need world knowledge for causal inference?." NeurIPS 2022 Workshop on Neuro Causal and Symbolic AI (nCSI). 2022.
[2] Li, Jiaxuan, Lang Yu, and Allyson Ettinger. "Counterfactual reasoning: Testing language models' understanding of hypothetical scenarios." arXiv preprint arXiv:2305.16572 (2023).
[3] Frohberg, Jörg, and Frank Binder. "Crass: A novel data set and benchmark to test counterfactual reasoning of large language models." arXiv preprint arXiv:2112.11941 (2021).
[4] Gao, Jinglong, et al. "Is chatgpt a good causal reasoner? a comprehensive evaluation." arXiv preprint arXiv:2305.07375 (2023). [5] Ashwani, Swagata, et al. "Cause and Effect: Can Large Language Models Truly Understand Causality?." arXiv preprint arXiv:2402.18139 (2024).

My main concern is that the experiments/results can be misinterpreted and therefore may not be strongly conclusive for the intended tasks, due to the followings:

1. Vagueness in expressions: This is mainly due to vagueness in expression formulation sespecially in the reverse part. : Take the given example given in the paper: "The film festival set the stage for the rest of the events, creating a desire for cultural experiences and community gatherings that ultimately led to the trampoline park party." Here the "The film festival set the stage for the rest of the events" sounds as if it took place afterwards due to "rest" of the events, and the part (desire and community gatherings) of the sentence mentioned before the "trampoline park party" races against the "film festival" and distracts the attention, hence the overall degrading in the causal reasoning estimation is likely.

2. Conformity to human communication: Another aspect is that LLMs can mis-read your intention, since they can falsely "assume" or "expect" you do typos or miswrite/communicate. This is a strategic choice in their design (and perhaps even their nature due to their training corpora) to not make them brittle, and work with real human conversations.

3. Referral mismatch. You say: “Although this relationship is displayed in the chain of events in the narrative, it is not logical and counter-intuitive” and answer incorrectly." Here, although you told the LLMs to ignore the parametric knowledge (or as you put it "explicit instruction to ignore outside information, allow for ill-logical relationships, and answer solely based on the hypothetical narrative"), it may still refer to the "meta-explanation" when it says it is not logical or counter-intuitive since it "informs" the user. So there might be a spurious mismatch between the task you expect vs. the setting LLM assume itself in.

4. On conflicts (unknown vs. opposite): You say: "streambank erosion to higher prices, but this contradicts the LLM’s parametric knowledge since this causal effect may not typically exist" Here again I find this rather vague conflict, and LLM might likely to conform the given example, when it cannot find counter examples, and can simply hallucinate. I would expect rather a hard conflict e.g., inverse causation, something like, rooster cause the sun rise.

I appreciate the paper's work, but the above concerns is due to inherent complexity of dealing with natural language, and its discrepancy against the intended task (hence we should be ourself causally-careful.)

Typos:

• would would -the our dataset

• In all the experiments, the narratives provided to the LLM convey significant insight on the causal structure of the chain graph under consideration. Using phrases like “results in”, “leads to”, “causes” in the narrative directly places a causal link between the variables. This approach would fail in scenario where such information is not already available. This makes the role of LLM quite straightforward (at least in the forward narrative setting).
• There was no explicit characterization of the error in the chain graph G’ that is produced by the LLM, without this it makes it unclear as to where the error stem in this setting stems from.
• It would help to provide more insight in to why there are inconsistencies between the answers to the causal reasoning task and the chain graph G’ predicted by the LLM -in light of existing works in use of LLM for causal discovery, some of these failures have been observed by past works and hence advocated hybrid approaches by combining with data driven methods.