Causal Modelling Agents: Causal Graph Discovery through Synergising Metadata- and Data-driven Reasoning

We thank the reviewer for their positive review of our work. We are glad they find our experiments encouraging and the combination of LLMs and (deep) structural causal modelling to be interesting and novel. To address their central query, we have updated the paper to add additional analyses based on our observational datasets in Appendix A.3.4 and Appendix A.7.

In addition, we have also updated the paper to include a new Appendix A.7.1 which highlights additional experimental/observational evidence from the literature which contextualises and/or corroborates the causal relationships discovered by the CMA in the real-world AD experiment.

The statistical correlations presented in Appendix A.3.4 do suggest the causal relationship between LW and HFLX. An experiment that intervenes LW may help reveal such causality.

Thank you for your response. We had previously conducted an experiment which intervenes on the LW variable in Appendix A.3.4. To maximise clarity, we have amended the manuscript to illustrate these results in two separate figures (instead of the single Figure 13). For the experimental/counterfactual intervention, please also see the anonymised figure. Indeed, for each of our discovered novel relationships, we have included:

1. A regression (observational) analysis, as requested, to assess for possible statistical associations between the variables.

2. An experiment which intervenes on the parent variable (counterfactual intervention) to assess a given relationship from a causal perspective.

3. Experimental/observational evidence from the literature to contextualise and/or corroborate our findings.

Please find references for these analyses below:

• LW → HFLX:

  1. Observational analysis: Figure 13, Appendix A.3.4.
  2. Counterfactual intervention: Figure 14, Appendix A.3.4.
  3. Literature: Section 4.2, main manuscript.

• Sex → P-tau:

  1. Observational analysis: Figure 34, Appendix A.7
  2. Counterfactual intervention: Figure 3B, main manuscript.
  3. Literature: Section 4.3, main manuscript.
  4. Additional literature: Appendix A.7.1

• sTREM2 → P-tau:

  1. Observational analysis: Figures 32 and 33, Appendix A.7
  2. Counterfactual intervention: Figure 31, Appendix A.7
  3. Literature: Section 4.3, main manuscript.
  4. Additional Literature: Appendix A.7.1

Thank you once more for your time. Please let us know if there are any additional points you’d like to discuss, and if not, we hope that the reviewer might consider an update.

Use of data only to fit the intermediate graph seems not using available dataset in full. Such as conditional independence and other information is all unused.

We believe the reviewer means that there are conditional independence statements which exist within the dataset that are not being utilised. If this is the case, then in fact we do utilise conditional independence information from the data. The metadata-based modules of the CMA produce a hypothesis (DAG) which is then encoded as a Deep Structural Causal Model (DSCM). Conditional independence statements are encoded within the structure of the DSCM at the current iteration and, as shown in Section 4.1, this has an effect on data likelihood under the model (up to Markov Equivalence; see Appendix A.2.3 for more details).

LLM’s stochastic nature is ignored. LLM may answer differently for the same question.

Thank you for this very important insight. We very much agree that LLMs do not always answer the same question deterministically, however we take account of this by querying the LLMs multiple times to mitigate for potential stochasticity in the outputs. We also attempt to account for failure modes in LLM reasoning. For example, Berglund et al. [4] identified a 'reversal curse' in LLMs, where they fail in logical deduction based on question order, not always generalising from 'A is B' to 'B is A'. To additionally address this layer of variability, queries are also made in both causal directions for local phase amendments. This process is described in more detail in Appendix A.1.3.

It is essential to thoroughly examine the behavior of LLM. How does it adjust the result based on its belief (GPT-4 etc) and intermediate results passed. There are more questions remained than answered.

Thank you once more for this important point. We strongly agree that investigating the behavioural patterns of LLMs is important to better understand their outputs, and indeed had performed a behavioural analysis in Appendix A.2.4: ‘Additional Results II - LLM Behavioural Patterns’. Here, we investigate behavioural patterns for three types of relationships: 1) Relationships which might plausibly exist in either causal direction (i.e., A -> B or B -> A), 2) Relationships which should only exist in a single direction (i.e., A -> B, but not B -> A), and, 3) Variables which should have no functional causal relationship between them. Furthermore, for each of these types of relationship, we assess LLM behaviour by first assuming the non-existence of the relationship, and then its existence in the A -> B direction, and then in the B -> A direction (Appendix A.2.4; Figure 5). We additionally investigated how these behavioural patterns change with the use of a Retrieval Augmented Generation (RAG) pattern [5], and find that in-context learning [6] strongly enforces causal relationships given the retrieved context (Appendix A.2.4; Figure 6).

We have amended the manuscript to more explicitly reference this analysis.

The word “metadata” is somewhat used in a mixed manner between domain knowledge already encoded in LLM and memory passed through JSON format. It should be more formally defined.

We appreciate this suggestion. We’ve now made a number of amendments to keep the distinction between both concepts clear.

Results Given that cases with no edges outnumber those with edges, not predicting edges may lead to an increase in accuracy. Thus, a qualitative analysis is necessary since not predicting edges might lead to an increase in the score. Other metrics such as TPR or FDR can be reported.

Thank you for this point. Exactly as you point out, the Normalised Hamming Distance (NHD) metric is contingent on the number of edges returned by a causal discovery algorithm. Indeed, predicting no edges at all may have a lower (better) NHD than predicting a number of edges which are incorrect. We therefore follow the approach taken by Kiciman et al. [7] in comparing the ratio of the NHD to a ‘floor’ baseline (Baseline Hamming Distance [BHD]) which outputs the same number of edges but all of them are incorrect. The NHD/BHD ratio is then the multiple by which the discovery algorithm is better than the worst baseline.
In addition to accounting for the NHD’s contingency on the number of edges predicted by using the NHD/BHD ratio, we believe that we perform sensible qualitative analyses of the results both in the main manuscript and Appendix A.3.3, A.4.3, A.5.1, and A.5.3. However, please let us know whether there are any specific types of additional analyses you’d like to see. Finally, in the above mentioned sections, we additionally report the True Positive Rates (TPR), the precision and recall of the algorithms, as well as the F1 score.

We thank the reviewer for their insight and questions.

We hope to have addressed the reviewer's comments and, if so, they would consider updating their score. We’d be happy to engage in further discussions.

• The use of LLM to tweak intermediate results seems clever but it is hard to assess its technical contribution.
• Novelty Given the abundance of similar papers (Long, S., Piché, A., Zantedeschi, V., Schuster, T., & Drouin, A. (2023). Causal discovery with language models as imperfect experts. arXiv preprint arXiv:2307.02390., Ban, T., Chen, L., Wang, X., & Chen, H. (2023). From query tools to causal architects: Harnessing large language models for advanced causal discovery from data. arXiv preprint arXiv:2306.16902.) in the field, the contribution is not clear.

Thank you for noting these important related works. In contrast to previous work, the CMA is the first framework to use LLMs as priors, critics during an iterative learning procedure, and as post-processors. We demonstrate the utility of this approach as per our results in Section 4.2, and in Section 4.3, we identify novel causal structures in the challenging real-world task of jointly modelling the radiological and clinical phenotype of Alzheimer’s disease. An additional novel aspect of our work is that our framework is the first causal discovery approach to output a multi-modal structural causal model (that is, we generalise and extend the DSCM framework), which enables reasoning over multi-modal data (e.g., for computing counterfactuals in the imaging space, as can be seen in Figure 3, Panel C of the main manuscript).

The method developed by Long et al.[1] assumes knowledge of the Markov Equivalence Class (MEC) containing the true Directed Acyclic Graph (DAG). This is a strong assumption for real-world applications as it presumes optimal outputs from causal discovery algorithms. Their work shows promising results in substituting an LLM in lieu of a human expert in this setting, but their approach serves as a post-processing step, reliant on a priori knowledge of a MEC which contains the true causal graph. In contrast, the CMA does not require any prior assumptions about the graph's structure.

Ban et al.[2] use an LLM's output as a prior for causal discovery algorithms, demonstrating its effectiveness with synthetic datasets, an approach also explored in the ‘LMPriors’ paper by Choi et al.[3]. However, this method lacks a feedback mechanism for hypothesis refinement, potentially limiting novel causal link detection by ignoring signals from the data. An example of this is the 'biological sex' -> 'phosphorylated tau protein' link identified by the CMA in Section 4.3, but overlooked by domain experts. We additionally note that in contrast to Ban et al.[2], we consider a real-world multi-modal dataset as one of our experiments, demonstrating our framework’s empirical utility for more complex and varied data scenarios beyond synthetic cases.

We have revised the text in the Related works section of the paper to better highlight these differences.

It is unknown how LLM is doing with respect to its memory. Does LLM always try to update edges in order to maximize data fitting? If the data fitting is based on the currently predicted causal graphs, how can it improve its causal graph? It does not work like an EM algorithm. Does LLM ‘regret’ its previous decision if fitting becomes worse? Considering developing a causal discovery algorithm that is based on local search (incrementally updating causal graph based on its likelihood), how would you compare their learning trajectories?

As you correctly point out, the data fitting step at a given iteration is based on the currently predicted causal graph. However, the CMA uses both the current data fit, as well as the data fit from the previous iteration to produce a memory. As we show in the ‘post-processing’ paragraph of Section 3: $\mu_t=\textsf{LLM}{\mu}(\mathcal{G}^s_t,\mathcal{G}^s_{t-1},F_t,F_{t-1})$, where for iteration $t$, $\mathcal{G}_t$ is the causal graph and $F_t$ is the metric of fit.

The prompting strategy and procedure for memory generation is further outlined in ‘Listing 3: System prompt for memory generation’, and ‘Algorithm 5: Post-processing and memory generation’ in Appendix A.1.5. Exactly as you intuited, the LLM is therefore prompt-engineered to ‘regret’ its previous decisions if fitting degrades, and to select an action or set of actions to improve fit. The LLM constrains the super-exponential search space of potential causal graphs, which a naive local-search-based algorithm would need to explore.

References:
1. (Long, S., Piché, A., Zantedeschi, V., Schuster, T., & Drouin, A. (2023). Causal discovery with language models as imperfect experts. arXiv preprint arXiv:2307.02390.
2. Ban, T., Chen, L., Wang, X., & Chen, H. (2023). From query tools to causal architects: Harnessing large language models for advanced causal discovery from data. arXiv preprint arXiv:2306.16902.)
3. Choi, K., Cundy, C., Srivastava, S., & Ermon, S. (2022). LMPriors: Pre-Trained Language Models as Task-Specific Priors. arXiv preprint arXiv:2210.12530.
4. Berglund, Lukas, et al. "The Reversal Curse: LLMs trained on" A is B" fail to learn" B is A"." arXiv preprint arXiv:2309.12288 (2023).
5. Lewis, Patrick, et al. "Retrieval-augmented generation for knowledge-intensive nlp tasks." Advances in Neural Information Processing Systems 33 (2020): 9459-9474.
6. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., ... & Amodei, D. (2020). Language models are few-shot learners. Advances in neural information processing systems, 33, 1877-1901.
7. Kıcıman, E., Ness, R., Sharma, A., & Tan, C. (2023). Causal reasoning and large language models: Opening a new frontier for causality. arXiv preprint arXiv:2305.00050.*

Okay, I missed the bidirectional edge in the graph. My bad. I was thinking 'directed acyclic graph' without an unobserved confounder since the paper keep talks about DAG. Here, chain graph is the type of graph (more like syntax) and the semantics for the bidirected edge is not clear. Do you mean it to be an unobserved confounder?

Given that all three are connected (both ground truth and DAG3), both graph encodes no conditional independence, and, hence, the same power to represent P(Age, AV45, P-tau). What am I missing here?

Thanks for the additional experiments and detailed explanations on my previous questions and comments. I greatly appreciate devoting your time on running additional experiments that could certainly support your claims in the paper.

(I will try to discuss with other reviewers after the author/reviewer discussion period, then reflect all those comments from the authors and other reviewers in my official review and rating.)

Whilst the Deep SCM framework can be used to model tabular and imaging data, it is not a causal discovery algorithm. One must 1) know the causal graph a priori and, 2) manually program the graph’s structure as a model. We have therefore generalised, extended, and subsumed this framework into a broader causal discovery algorithm which utilises data-driven and metadata-driven reasoning to produce causal graphs. We could, for example, perform exhaustive fitting of DSCMs, whereby we enumerate all possible DAGs for a given set of variables and fit each one. However, a full ablation of this nature would not be possible for anything but trivial graphs due to the super-exponential search space (Peters el a. [2], Appendix B: ‘Causal Orderings and Adjacency Matrices’). In our pilot study, we found the concept of ‘memory’ to be crucial, as there would be no possible alternative to providing a feedback signal to the metadata-based modules, and therefore each iteration would be entirely independent of previous amendments, which obviates the need for any numerical data (however as we demonstrate, signal from data leads to improved performance for causal discovery). We may be able to improve performance even further by using, e.g., alternative prompt tuning approaches to the memory structures, the user prompt, and the system prompts of the LLMs, but consider this important future work.

Additional comments on the benchmark results. [...]

Thank you for this point, and well noticed. We re-ran all experiments in this work, but simply had not moved this information from our Appendix to the main manuscript for the data-driven approaches for the arctic sea ice benchmark. This has now been rectified. To replicate the metadata-based benchmarks for gpt-3.5-turbo and gpt-4, we used the same ‘single prompt’ approach as in the kiciman paper (this is described in kiciman et al. [8] in their Appendix A.1, Table 14). They augment this prompt so that it can be used for ‘full graph discovery’ (this modification is described in kiciman et al. [8]; Section 3.2.1, page 13). For convenience, we provide the prompt template in full in Appendix A.3.2, Table 5 of our own work. We believe the slightly different results could be down to a couple of factors. First, it appears that gpt-4’s performance on a multitude of tasks can change over time, as demonstrated by Chen et al. [11], which may partly explain these differences. Second, due to a lack of description as to how they handled errors, we implemented our own error-handling strategy. Namely, the model is meant to produce a single letter in {A,B,C} which reflects one of three options:

• A: Changing $A$ causes a change in $B$

• B: Changing $B$ causes a change in $A$

• C: No causal relationship exists

Occasionally the model might output ‘C/A’. This would be handled as an error in our work, and no edge would be proposed in these cases. A more detailed description of the experimental setup can be found in our Appendix A.3.2.

We have updated Appendix A.3.3 to include the points above in our interpretation of the results for the arctic sea ice dataset.

Thank you for your continued engagement with our work, and we hope to have addressed the majority of your concerns.

We thank the reviewer for their insights and are glad they found the model architecture convincing, and our experiments extensive. To address their central question, we have updated the paper to add additional experimental results (Appendix A.5.4) which probed the effect of perturbing the synthetic AD data on the output of the numerical causal discovery algorithms.

We provide a high-level summary of these results below.

There are three noise levels whereby Gaussian noise is added with mean 0
and standard deviations of 0.4 for L1, 0.8 for L2, and 1.2 for L3.

As can be seen in this anonymised figure, we found that with increasingly weaker causal relationships, both NOTEARS and DAG-GNN struggle to learn the causal graph, though the DAG-GNN algorithm maintains a more consistent performance throughout. The CMA outperforms both algorithms, and we believe this is partly due to the metadata-based modules of the framework, which can still propose reasonable causal structures.

References:

1. Pawlowski, N., Coelho de Castro, D., & Glocker, B. (2020). Deep structural causal models for tractable counterfactual inference. Advances in Neural Information Processing Systems, 33, 857-869.
2. Yu, Y., Chen, J., Gao, T., & Yu, M. (2019, May). DAG-GNN: DAG structure learning with graph neural networks. In International Conference on Machine Learning (pp. 7154-7163). PMLR.
3. Zheng, X., Aragam, B., Ravikumar, P. K., & Xing, E. P. (2018). Dags with no tears: Continuous optimization for structure learning. Advances in neural information processing systems, 31.
4. Huang, Y., Kleindessner, M., Munishkin, A., Varshney, D., Guo, P., & Wang, J. (2021). Benchmarking of data-driven causality discovery approaches in the interactions of arctic sea ice and atmosphere. Frontiers in big Data, 4, 642182.
5. Kıcıman, E., Ness, R., Sharma, A., & Tan, C. (2023). Causal reasoning and large language models: Opening a new frontier for causality. arXiv preprint arXiv:2305.00050.

We thank the reviewer for their questions and insights. We are glad they found our paper original, of good quality, well-structured, and relevant. To address their questions, we have made a number of amendments to the manuscript, including extending the ‘Related works’ section and incorporating several clarifications throughout.

While the authors do a good job regarding the related work, we consider this could be further enhanced by citing surveys that provide an overview of the relevant topics and domains. E.g., the authors may be interested on the following works: [..]

Thank you for your kind words. We have included a reference to the causal deep modelling survey at the beginning of our discussion of LLMs and causality in the related works section.
The Alzheimer’s disease (AD) neuroimaging references provide a nice exposition of the 18F-AV-45 tracer in particular, which is a variable we consider in our AD models. We therefore additionally include these references in the ‘Experiments’ section.
Finally, Huang et al.’s work on data-driven causal discovery for the arctic sea ice dataset is referenced in Section 4.2: ‘Benchmarking Experiments’.

In the related work section, the authors may consider weighting the views and findings regarding LLMs and causality expressed in the following paper: Zečević, Matej, et al. "Causal parrots: Large language models may talk causality but are not causal." arXiv preprint arXiv:2308.13067 (2023).

Thank you for pointing us to this contemporary work. We have added a discussion of this paper in a new section on the use of LLMs for causal reasoning. This can be found in the ‘Related work’ section of the main manuscript.

When reporting results in Section 4.1, the authors measure average data likelihood and the deviation. It would be helpful to have some reference value to understand whether the reported values are good or not and why.

In Section 4.1, the most meaningful comparison lies in the relative difference in data likelihoods under each model. We have amended the manuscript to highlight that the model that matches the data-generating process acts as the reference likelihood.
We then demonstrate that making incorrect modelling assumptions about the data-generating process leads to a lower data likelihood. Finally, the difference in data likelihoods is compared with a pairwise Tukey HSD test. Assessing relative model performance in this way has been employed by other related works[1].

How is the threshold for DAG-GNN selected?

We assessed two thresholds for the DAG-GNN algorithm and chose the threshold with the best performance for inclusion in the main manuscript. We began with a threshold of 0.3 as per the DAG-GNN paper by Yu et al.[2], who themselves selected this parameter based on prior work by Zheng et al.[3]. We also assessed a threshold of 0.1, which had worse performance on our benchmarks. Results of the DAG-GNN algorithm with different thresholds can be found in Tables 6, 9, and 12 of Appendix Sections A.3.3, A.4.3, and A.5.3, respectively. We had also conducted a similar hyperparameter tuning experiment for the NOTEARS algorithm and reported those results in the same tables.

1. Table 1: align results to the right so that differences in magnitude are quickly visualized.
2. Table 2: add up/down arrows near the reported metrics, indicating greater/lower is better.

We very much appreciate the formatting/readability suggestions. We’ve amended the tables and re-ordered some of the columns in Table 1 to improve clarity.

Table 2: for some algorithms (TCDF, NOTEARS (Temporal), NOTEARS (Temporal)), the authors report results only for the Arctic sea ice dataset, but no clarification is provided as to why no results are reported for the Alzheimer’s disease and Sangiovese datasets.

As we state in Appendix A.3.2, the arctic sea ice dataset includes optional temporal data, which the TCDF algorithm expects by default. The Alzheimer’s disease and Sangiovese benchmarks are cross-sectional datasets, which precludes the use of the TCDF algorithm on them. The ‘temporal’ versions of the NOTEARS and DAG-GNN algorithms require that we pre-process the time-series data as per Huang et al.[4] and Kiciman et al.[5]; a detailed description can also be found in Appendix A.3.2.

For clarity, we have now added an explicit reference to Appendix A.3.2 in the main manuscript under Section 4.2.