Counterfactual Graphical Models: Constraints and Inference

Thank you for reviewing our work, providing feedback, and giving suggestions.

To answer your question about the intuition for counterfactual ancestors (Definition 2.4), they are the counterfactual variables that are causally relevant to the variable in question. This extends the idea that, in graphical terms, a variable can only affect another if the former is an ancestor of the latter. When counterfactuals are involved, some of those ancestors become irrelevant (by virtue of the exclusion operator). For example, in a simple chain graph such as $X \to Z \to Y$, $X$ is an ancestor of $Y$, but it is not a (counterfactual) ancestor of $Y_z$, because under $do(Z)$, $X$ cannot affect $Y_z$. Similarly, $Z$ is an ancestor of $Y$, but for $Y_x$ the counterfactual ancestor is $Z_x$, not $X$ or $Z$. As suggested, we will include a brief discussion of this after the definition; your suggestion is appreciated.

As you point out, we did not mention in the main paper that ctf-calculus subsumes do-calculus. It is suitable for any task where the latter is complete, such as identification from observational distributions. We will add Lemma C.1 (in the supplemental material) “ctf-calculus subsumes do-calculus” and a brief discussion to the main text to make this fact clear to the reader.

We see your point about Figure 3, and that the meaning of the columns and edges can be confusing in the middle of so many models where elements have a well-defined meaning. In this figure, the gray boxes represent data-generating mechanisms that transform a specific unit $U=u$ into a counterfactual event over the observable variables. Each rectangle is a copy of the mechanisms of the structural causal model. Depending on the counterfactual of interest, the mechanisms share some functions (e.g., $f_z$ and $f_y$ in (a)), redefine others ($f_x$ and $f_x'$ in (a)), or contain functions that require the evaluation of a separate set of mechanisms (e.g. $f_x'$ in (b)) to compute a nested counterfactual. Following your suggestion, we will provide a better description of the graphical elements of the figure in the manuscript, thank you.

Thank you for reading our work, pointing out typos, and providing suggestions, which we will incorporate into the manuscript. In particular, we will clarify in the paper that the unnesting corresponds to a transformation that starts with a nested counterfactual and ends with an expression involving a counterfactual with one less level of nesting.

As far as we can tell, the original paper describing the Twin Network method by Balke & Pearl does not claim the method’s completeness for using d-separation to assess counterfactual independencies. However, in Shpitser & Pearl (2007), the authors discuss the incompleteness of the method to motivate multi-networks.

As you point out, “the idea of computing a CTF query in the twin network after the surgery might therefore lead to wrong conclusions”. For instance, in the example in Sec. 2.3, related to Figure 5 (a,b), some distinct nodes in the Twin Network may refer to counterfactual variables that are deterministically the same. In the example, conditioning on $Z_{x}'$ is the same as conditioning on $Z$. Because the Twin Network does not capture this constraint graphically, d-separation does not capture such independencies. We have also provided further discussion on this in section C of the supplemental material but will clarify this in the main text as well. Thank you!

Thank you for reading our work, providing feedback, and asking questions.

We refer next to the research mentioned in the review. Also, thank you for sharing the references. We describe the work as we understand it, but we would be happy to hear more about it from the reviewer. Regarding Ma et al (2022), the paper makes assumptions on the latent variables’ prior and the availability of an auxiliary variable. Based on these, they aim to identify the SCM that generates the data. They integrate the information on the SCM into the optimization process of a generative model to produce explanations. Compared to our approach, there are similarities in modeling the data-generating process as an SCM and then inferring counterfactual quantities based on the assumptions made over the SCM. On the other hand, in our framework, we make no assumptions about the distribution of the latent variables or the functional form of the mechanism of the SCM but assume a known causal graph. This setting is called non-parametric in the literature, which motivated Pearl’s Biometrika’s 1995 paper, where he introduced the do-calculus. Moreover, we do not attempt to identify the SCM as a whole but to identify particular (counterfactual) queries of interest in the form of independence constraints or probabilities. These are indeed the two main contributions of our paper, namely, graphical criteria and calculus for counterfactual identification.

Vo et al (2023) seems to focus on producing examples that counter the original outcome produced by a model (e.g. classifier). Moreover, it seems the validity of the counterfactual is measured in terms of whether the outcome can be overturned by the generated example. We believe our work is not directly comparable here since it seems this work does not address the causal structure of the underlying data-generating process. Having said that, we would be happy to understand the reviewer's suggestions, in case it implies some subtle connection we may have missed based on our cursory reading.

Thank you for reviewing our paper.

To address your question about a scenario where our method succeeds but the other approaches mentioned in Table 1 fail, let us consider the question of whether the causal graph in Figure 4(b) implies that $(Y_{xw}, W_{x'} \perp X | {Z_x}')$. Figure 5(a) shows a 3-plet (triplet) network (a natural generalization of the twin network to 3 worlds) for this graph and question. The variables in the query involve three submodels: $\mathcal{M}, \mathcal{M_x}$, and $\mathcal{M_{xw}}$, all depicted in the network sharing explicit unobservable variables. It seems that $X$ is d-connected to $Y_{xw}$ given ${Z_{x}}'$, because there is an active path $X \gets Z \to U_z \to Z_{xw} \to Y_{xw}$. However, as discussed in the manuscript (around line 262) and due to exclusion restrictions, conditioning on $Z_{x'}$ is equivalent to conditioning on $Z$, meaning the corresponding separation statement holds. This implies that the Twin Network alone leads us to infer a wrong conclusion. Although this example involves three worlds (to take advantage of the figure in the paper), the same argument could be made with only two of them.

For Single World Intervention Graphs (SWIGs), conditional independence among variables present in the SWIG can be read using d-separation, while the representation itself cannot capture cross-world restrictions on the counterfactual joint distribution. For instance, the separation of $X$ and $Y_x$ given Z cannot be judged using the SWIG for Figure 4(a) and intervention $X = x$ because Z does not appear in the resulting graph. These examples and some discussion on Shpitser & Pearl 2007 can be found in the supplemental material, section C.

The k-plet Network is a concept we use in the paper to refer to the extension of the Twin Network method to k worlds (2-plet network equals Twin Network). In the paper, we imply that when combined with the exclusion operator, the k-plet network method is complete, but it can further be optimized, leading to the discussion of AMWNs. We added proper discussion and clarification about this point in the paper. Thank you! Also, as you pointed out, several definitions in the paper emphasize explanations about the notation and counterfactual variables. We will try to make those definitions more concise, maintaining the essence of the definition while moving additional explanations elsewhere.