Demystifying amortized causal discovery with transformers

We thank the reviewer for the time dedicated to our paper. One important criticism is that we should better highlight our contribution in comparison to Lopez et al. (2015): this is addressed in the first bullet of our response to the Weaknesses section.

Weaknesses

• “The paper should be upfront and highlight much more clearly what its contributions are beyond Lopez et al [10]”.

  We highlight significant differences between Lopez et al. and our work. In their paper, Lopez et al.:

  1. Study upper-bounds of the excess risks of a binary classification problem, i.e. mapping distributions to bivariate causal graphs, via Rademacher complexity of the hypothesis space (theorem 3 in their paper).

  2. Perform experiments on limited real data and on synthetic data exclusively generated in the setting of nonlinear ANMs.

  Our work, instead, is aimed at understanding when supervised causal discovery works, in a principled manner rooted in identifiability theory results. In the motivation of their work, Lopez et al. claim that supervised causal structure learning “would allow dealing with complex data-generating processes, and would greatly reduce the need of explicitly crafting identifiability conditions a-priori.” Our work shows that CSIvA generalization is still limited by results from identifiability theory, and that identifiability assumptions still need to be crafted a priori. In particular, our Example 2 shows that CSIvA failure and success of correct inference in non-identifiable settings is determined by the variety of SCMs that have been used during the model training, and our Hypothesis 1 formalizes this conjecture. On this basis, we present:

  1. Theory adapted from Hoyer et al., that sets defines the set of identifiable SCMs
  2. experiments that show that the class of SCMs identifiable by CSIvA is constrained to that of identifiable SCMs according to the theory of Proposition 1 and our Hypothesis 1 (see Figures 3a and 3b).
  3. Experiments showing when CSIvA succeeds and fails to generalize at test time - in-distribution and OOD test data
  4. We show that training on data from multiple SCMs that are identifiable according to our Proposition 1 results in an Algorithm with better empirical generalization performance.

• “A major component of causal discovery performance is not only identifiability of the graph from the observational distribution, but also the intractably large search problem […]” We don’t provide experiments in multivariable settings as our goal is not to study CSIvA scalability. Our focus is on whether supervised causal discovery respects known identifiability results, and bivariate graphs provide a minimally sufficient setting for studying identifiability. In particular, our Proposition 1 summarises important bivariate identifiability results (Hoyer et al., Zhang and Hyvarinen), and our empirical study confirms that CSIvA is constrained by the identifiability results of Proposition 1. Concerning the multivariate settings, Peters et al. propose identifiability theory for multivariate additive noise models as a straightforward generalization of Hoyer et al.; hence, the theory of Proposition 1 underlying our empirical findings is valid for arbitrary high dimensions.

• “Section 3.2 seems unnecessary. The section only studies the generalization ability of CSIvA, which is no contribution” In section 3.2 we study in-distribution and out-of-distribution generalization. In-distributoin generalization is studied in Ke et al. (2023a), as recognized in our paper (L196). The main point of this study, in our paper, is to validate our CSIvA implementation (as specified in L196-197 and footnote number 1) as the authors of CSIvA paper did not provide public code, which required from-scratch implementation on our side. Concerning the OOD generalization, this is not studied in Ke et al. (2023a), as in their experiment, the only difference between train and test data is in (i) the algorithm for synthetic graph generation (ii) some variation on the parameter of the Dirichlet distribution they employ in some experiments. However, identifiability theory is sensitive to mechanism and distribution assumptions, which Ke et al. do not vary but which our experiments in section 3.2 study.

  ”The takeaways (lines 195-) that “CSIvA generalizes well to test data generated by the same class of SCMs used for training” and that “it struggles when the test data are [from different SCMs]” are obvious and well-studied ...” Due to what we wrote just above, we can safely say that this is not obvious nor well studied. Lopez et al. (2015) limit tests to nonlinear ANMs, Lorch et al. never tests on different classes of mechanisms with respect to the training, similarly Li et al., Ke et al., Lippe et al. None of these works mentions insights that relate to our Example 2. If the reviewer has precise references supporting the claim that our results are obvious and well-studied, we kindly ask to provide them.

Questions

We standardize the data, as written in L163.

Limitations

• “The “theoretical result” (Proposition 1) is a simple corollary of Hoyer et al [7].” We openly discuss this relation and the relation of Proposition 1 with Zhang and Hyvarinen [8]. We offer to rename Proposition 1 to Corollary 1, making the relation with previous works more explicit even in the naming.
• “It is unclear whether “randomly initialized MLPs” are sensible nonlinear functions to use for constructing nonlinear mechanisms and non-Gaussian noise distributions. …” In the PDF attached to this rebuttal we replicate the experiments of Figure 2a and Figure 4 of the paper, this time using Gaussian Process generated mechanisms (GP data) with unit RBF kernel. These empirical results agree with the findings in the paper on MLP-generated data. We will replicate all the experiments using data with GP-generated mechanisms and include them in the appendix.

Thank you for your response. I will maintain my score of the work.

We thank the reviewer for their time and effort in analyzing our work.

Weaknesses

The only comment present in the weaknesses section is that “The presentation can be significantly improved. Since the paper aims to offer novel insights, it is crucial to organize the arguments, theoretical results, and experimental findings effectively to support these insights.” In absence of more specific thoughts on this, any answer we could give would not be on point.

Questions

• “Training on data from multiple causal mechanisms and/or noises intuitively improves generalization, as the model is exposed to more changes and diverse data”: this is true, and related to an important point made in our paper that we want to remark on: previous to our work, there is no research about whether training on multiple mechanisms would actually be beneficial, or if instead, due to well-known boundaries posed by identifiability theory, this would be harmful. Thus, despite the conclusion that training on mixed SCMs is beneficial seems obvious, it is not, and this is a consistent subject treated in our work.

• “Domain adaptation is simpler because there is a meta-distribution or meta-process that governs how distributions change across domains. Similarly, large language models (LLMs) benefit from training on diverse and vast data sources and their underlying generating processes. I wonder if the authors have theoretical results or insights on this aspect with causal models training on vast data generated by diverse causal mechanisms potentially without a common meta-process.” It is hard to make unifying statements on such a large variety of topics as those touched by this question. Here is our thought, to be taken with a grain of salt. Algorithmically, causal discovery on linear, nonlinear ANM, and post-nonlinear models has at its core one common procedure, that we may call meta-procedure: specifically, if one considers cornerstones classical method for causal discovery like RESIT (Peters et al., 2014) or Direct-LiNGAM (Shimizu et al., 2011), (roughly) consisting of regression + independence testing over the residuals, it appears that causality research has found that a single meta-procedure (again, regression + conditional independence testing of the residuals) is one of the best approaches to causal discovery, both on linear and nonlinear additive noise models - the object of study of RESIT and Direct-LiNGAM papers. This suggests that a good learner should be able to learn one algorithm that works on nonlinear ANM data, and seamlessly adapt it to linear ANM data, thus achieving good OOD generalization at least in the task of training on nonlinear data and testing on linear data (or vice versa). Though, we don’t observe this to happen, which is a fact worth noting. We ask the reviewer's opinion about this insight, as we believe it could make a valuable insight into our work. We thank the reviewer for the point made which sparked this discussion.

We thank the reviewer for their comments and effort in understanding our paper. Before proceeding further, we note the conciseness of the Weaknesses section, where two generic criticisms are expressed in four lines of text. In absence of more articulated concerns, we respond at our best to the comments presented therein.

Weaknesses

“The paper is a purely empirical study of the generalisation behaviour of supervised causal discovery methods, validating general intuition without thorough novel insights”. The claim that "we validate general intuition" is a vague statement, hard to address in a satisfactory way: to help foster a more articulated discussion, we present a summary of our contributions. The goal of our work is to understand when supervised causal learning works, in a principled manner rooted in identifiability theory results. Our Example 2 shows that CSIvA failure and success of correct inference in non-identifiable settings is determined by the variety of SCMs that have been used during the model training, and our Hypothesis 1 formalizes this conjecture. On this basis, we present:

1. Proposition 1, a theoretical statement adapted from Hoyer et al. (2008) which defines the set of identifiable SCMs

2. Experiments showing that the class of SCMs identifiable by CSIvA is constrained to that of identifiable SCMs according to the theory.

3. Experiments showing when CSIvA succeeds and fails to generalize at test time - in-distribution and OOD test data, respectively.

4. We show that training on data from multiple SCMs that are identifiable according to our Proposition 1 results in an Algorithm with better empirical generalization performance.

   If the reviewer believes that all of these findings are general intuition, we kindly ask: (a) please argue this more specifically, as the claim that it is general intuition is very generic (b) provide scientific references where this intuition is exposed, in a way that is so comprehensive to invalidate our contribution.

• “Given the empirical nature of this paper, I'd have expected to see a more thorough comparison, ...” Concerning the requested leave one out study, this is a sensible approach to probe test generalization only when zero or few test datasets are available for the optimized model: since we are dealing with synthetic data, instead we have an infinite availability of test datasets. Indeed, every model is tested on 1500 datasets unseen during training, which is the standard machine learning way to study generalization. Concerning the requirement of more “interesting individual SCMs such as non identifiable examples”, while we agree that this would be nice, current known theory only tells us models that are identifiable and not which ones are non-identifiable, with the notable exception of linear Gaussian data, which we do indeed analyse (see Figure 3b and the relative discussion). Any other example as the one provided in Example 1 must be analytically found by pen and paper computations, which is not a feasible option. Finally, concerning the request for prediction from new samples from a training set SCM, this is what our in-distribution generalization study is about, see Figure 1 (a, b, c).

Questions

• “You seem to be surprised that supervised causal discovery methods can infer graphs when trained on mixed data. Isn't that somewhat obvious after results from [29] that show that transformers can identify valid assumptions from data?” Our mixed training study of Section 3.5 is motivated by our observations of Section 3.2 that CSIvA presents good in-distribution generalization while it fails in OOD tasks (section 3.2): given that training on a larger variety of SCMs allows CSIvA to operate on in-distribution test data more frequently, our expectation is that test performance in our experiments benefits from it. This is well expressed in the Implications paragraph of Section 3.2, particularly L 197-199, and in Section 3.5, L 294-297: for this reasons, we disagree with the reviewer saying that we are surprised by discovering that CSIvA has good in-distribution generalization properties after mixed training, as this is exactly what we specify to be the expected outcome.
• “You make fairly strong statements about classical methods not being applicable because the underlying assumptions cannot be verified. I'd disagree with this as they still turn out to be useful in practice.” We don’t make such statements: if we mistake, please directly point to where this happens, as we are ready to remove these claims that find us disagreeing. What we do instead, is highlight strengths and weaknesses of classical methods compared to supervised learning-based approaches, based on our empirical findings. On the one hand, classical methods are more reliant on assumptions on the mechanisms than supervised learning-based methods can be: as we show that mixed-training on mechanisms is a theoretically principled practice (by Proposition 1), a CSIvA model that is trained on LiNGAM, nonlinear ANM, and PNL data clearly has less restrictive requirements on the mechanisms than classical methods. On the other hand, we notice poor generalization of CSIvA on unseen noise distributions, in contrast to classical methods which are mostly agnostic about the distribution of the error terms. A CSIvA model agnostic about noise distributions would require training on SCMs covering all existing noise distributions, which is arguably impossible: our reasoning unveils that classical models appear to have an advantage, in this sense. These arguments are presented in the paper, see L 319-323, L334-340, and the abstract at L 13-16.
• “How do you think [1] relates to the behaviour of supervised causal discovery methods, given that transformers can perform approximate Bayesian inference [2].” Transformers’ ability to do Bayesian inference is far beyond the scope of this paper.

We thank the reviewer for their comments and the time taken analysing our paper. Before proceeding further, we note that one important criticism from the reviewer appears to be that “the current findings are still very limited, need to be further consolidated”. We point to the first bullet of our response to the Weaknesses section for an answer.

Weaknesses

• “part of the study can be summarized as learnability vs. identifiability […] the current findings are still very limited, need to be further consolidated. […] a related work [1] is missing.” We agree that there is a missing citation, which we will add to the paper. Concerning the criticism that our “findings are very limited and need to be further consolidated”, this is a generic statement, hard to address in a satisfactory way. We present a summary of our contributions below, which can be taken as a ground to precisely articulate this concern. The goal of our work is understanding when supervised causal learning works, in a principled manner rooted in identifiability theory. Example 2 shows that CSIvA failure and success in inferring non-identifiable settings is determined by the variety of SCMs used during the model training, and our Hypothesis 1 formalizes this conjecture. On this basis, we present:

  1. Proposition 1, a theoretical statement adapted from Hoyer et al. which defines the set of SCMs identifiable from observational data.
  2. Experiments showing that the class of SCMs identifiable by CSIvA is constrained to that of identifiable SCMs according to the theory of Proposition 1 and our Hypothesis 1 (see Figures 3a and 3b).
  3. Experiments showing when CSIvA succeeds and fails to generalize at test time - in-distribution and OOD test data, respectively.
  4. We show that training on data from multiple SCMs that are identifiable according to our Proposition 1 results in an Algorithm with better empirical generalization performance.

  Further, we clarify that the our findings are general in the following sense:

  1. CSIvA is an archetypical model, as its learning objective - the conditional distribution over the space of graphs given the data - is shared with the majority of existing methods for amortized causal discovery (Lopez et al., Lorch et al., Lippe et al., Li et al.). Thus, our novel findings apply to an entire class of common methods in the literature
  2. The theoretical ground of our study generalizes to multivariate causal discovery. In particular, Peters et al. (2014) proposes identifiability theory for multivariate additive noise models as a straightforward generalization of Hoyer et al. (2008) (the main reference of our Proposition 1);

• “one claim "we conclude that the post-ANM is generally identifiable, which suggests that the setting of Example 2 is rather artificial" I disagree. […] example 2 is quite valid in real-world setting, or the linear gaussian setting, is also commonly adopted in real-world, but had not been discussed in this work.” The fact that under the post-ANM assumption non-identifiable SCMs belong to a zero-measure region is the definition of identifiability provided by Hoyer et al., commonly adopted in the causality community and by our paper. In this sense, identifiability of the post-ANM means that samples from the post-ANM are almost surely identifiable (almost surely in a formal sense): as such, non-identifiable SCMs like our Example 2 must be artificially crafted to sample from a zero measure region, which is why we use the word artificial. To avoid potential sources of confusion, we propose to review the sentence specifying that our Example 2 under the post-ANM assumption almost surely does not happen, which formally clarifies what we mean by saying that it is artificial. Concerning the case of Linear Gaussian data and reviewer’s claim that this has not been discussed in our work, we point to our experimental results of Figure 3b, where we show that CSIvA is unsuitable for inference on Linear Gaussian data in agreement with the identifiability statement of Proposition 1.

• “potential conflict between section 3.3 and 3.4 [...]” We believe the reviewer refers to section 3.5 (instead of 3.4) as this is where we analyse training on diverse SCMs. This is closely related to the point made in the previous bullet. Section 3.3 shows that mixed training on data from SCMs sampled from the non-identifiable zero-measure region of the post-ANM compromises SCL’s performance, coherently with our Hypothesis 1. Instead, section 3.5 shows that training on datasets generated by identifiable SCMs (i.e. not from the zero measure region) benefits generalization, coherently with our findings in section 3.2 about good CSIvA in-distribution generalization. So, the two sections are complementary, not in contrast, as they consider CSIvA behavior when trained on samples drawn from complementary sets (the one of identifiable SCMs, and the one of non-identifiable SCMs, under the post-ANM hypothesis).

• “although not explicitly claimed, this paper suggests that "CSIvA is capable of in-distribution generalization' [...]" We do explicitly claim this (L212-213). In particular, our experiments are in the bivariate setting, Ke et al. experiments are in the multivariate setting. Note that the in-distribution generalization experiments, in our case, mostly serve the purpose of validating our CSIvA implementation (L196-197 and footnote number 1) as the authors of the CSIvA paper did not provide public code, which required from-scratch implementation from our side.

• “I suggest to use supervised causal learning (SCL) rather than amortized causal discovery” This nomenclature would generate confusion with other methods that perform supervised causal discovery but are not suitable for the amortized inference - see L73-74, with references [18, 19, 20, 21, 22].

Questions

Yes, Fig 3 a) and b) are reversed in the caption, but not in the text. Thank you.