Causal Concept Graph Models: Beyond Causal Opacity in Deep Learning

Relation with causal variational autoencoder

The training and loss are similar, but the model, the architecture, and the inference at test time are not:

1. In Causal VAE, concepts are scalars, while in Causal CGMs, concepts are high-dimensional supervised embeddings that are much more expressive (as detailed in App D).

2. CGMs’ causal transparency is a property that derives from the test-time unrolling. Causal VAE does not perform test-time unrolling. Also, Causal CGM unrolling will not be effective without the expressivity of concept embeddings (see below ablation result).

3. Causal VAE are generative models, CGMs are discriminative

The main similarity is that Causal CGMs use the same DAG loss of Causal VAE (we cited it at L245). So the training is similar, but the problem, the model, the architecture, and the inference at test time are not. Regarding point 2. see ablation study using scalar instead of embeddings in the common answer.

---

Human-in-the-loop corrections aspect seems contradictory to the motivation of the work

The effectiveness of human-in-the-loop corrections aims to assess whether the learned causal mechanisms are aligned with humans’ expectations. Also, we assume concepts are not latent variables to discover, which may or may not align with human concepts. We assume that concepts are supervised by humans, thus they are aligned with human semantics as in CBMs. In extreme cases (CIFAR10), we use a VLM to label concepts to reduce the cost of producing concept labels, but the label's meaning still needs to be aligned with human semantics. What may not be aligned are the causal relations among the concepts used by the model and by a human for reasoning. However, our experiments show two things: 1) Causal CGMs’ interventions are much more effective than in existing CBMs, 2) when combined with causal discovery techniques, Causal CGMs’ inference aligns possibly better with the data generating process and human inference structure making interventions even more effective showing the benefit of combining Causal CGMs with inductive biases improving semantic alignment and generalization ability (such as, but not limited to causal discovery techniques).

---

Causal CGM is more related to interpretability rather than XAI

We agree. This is why we described our method as “causally transparent by design” (L92).

---

Rubin causal models

Thanks for pointing this out! We have rephrased this in our new manuscript as: “Structural Causal Models (SCM) proposed by \citet{Pearl2009} represent one of the possible frameworks for causal modelling”.

---

Reduced form expression

In theory, the conditional distribution in L111 is degenerate because endogenous variables can be expressed as deterministic functions of the exogenous variables. However, from a Bayesian perspective, if we consider that the parameters of the structural equations might also be uncertain, then the conditional distribution may no longer be degenerate. In this work we did not consider such a Bayesian perspective, but this could be an option for future works.

---

Hard interventions

Agree. Rephrased as “hard interventions”

---

Concepts (concept-based learning) vs High-level features (representation learning)

In representation learning, there is no semantic requirement that high-level features are aligned to human semantics, while in concept learning, this is a hard requirement. For instance, the embedding of a ResNet may represent high-level units of information, but this does not mean that this information has a semantic correspondence with a concept humans would use to reason about the problem at hand (if anything, this is very unlikely and there is evidence of superposition). This semantic alignment is necessary for human interventions to be effective, and it is a core requirement of all CBM-like architectures as discussed in [1].

---

Doesn't theorem 3.2 follow from $d$-separation?

The proof uses some conditional independence properties of the distribution (similarly to the $d$-separation criterion), but the proof is specific to our PGM where the two sets of variables $V$ and $V^\prime$ are identical copies of the same endogenous variables.

---

How can the likelihood of unobserved variables ($v$, $v{\prime}$) be maximized?

$V$ and $V^\prime$ are identical copies of observed variables (L256). Hence, the maximisation of the log-likelihood leads to the minimization of the standard cross-entropy loss between the model’s prediction and the ground-truth labels.

---

[1] Kim, Been, et al. "Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav)." International conference on machine learning. PMLR, 2018.

Some sections are hard to follow

Thank you so much for pointing this out. We want this paper to be as clear as possible so this feedback is helpful. With this in mind, could you please point us to specific sentences or sections so that we can improve them?

---

Simple benchmarks

CelebA is a standard benchmark in CBM papers (comparable to CUB), and we used CIFAR10 for a challenging real-world condition where we assumed that concept labels were missing in the dataset. Besides, our contributions extend beyond empirical evaluations by providing a theoretically motivated novel framework and a new way of thinking about causal opacity within the popular context of concept-based XAI.

---

Joint training

Joint training is a method for training Concept Bottleneck Models. To clarify this point, we added the following in App. G.2: “In our experiments, all baselines are trained using a joint training technique commonly used in CBMs [1]. In joint training, the model is trained end-to-end with the task predictor directly taking the concept encoder’s outputs as input, enabling simultaneous optimization of both concept and task predictions.”

---

What is the architecture of the bottleneck layer? Does it influence the quality of CGMs learnt?

The details of the architecture of the bottleneck layer are described App D. To better explain this in the main text, we moved the key component of the bottleneck layer from App. D after Sec 3.3. To support this choice we conducted an ablation study replacing high-dimensional embeddings with scalar values and showed how this affected performance and interpretability (see shared answer “Endogenous variable representation: scalar values vs. high-dimensional embeddings”).

---

Missing experiments on human-in-the-loop corrections of the intermediate reasoning steps

The results of these experiments are discussed in the second and fourth paragraph of section 4.1 and shown in Fig 4, 5, 12, 13, and in Table 3.

---

Check whether the learnt causal graphs are sound and valuable

The learnt causal graphs enable the model to match the performance of black box models (valuable) and pick up known biases in the data (sound). The generated causal graphs represent the actual reasoning path of the model, which is typically unknown when considering a black box model. If a user values knowing the reasoning path of the model, then these graphs could be useful. For instance, we show how the generated graphs enable users to de-bias the model by blocking all paths between two variables (results reported in Table 3). For example, if a user does not want “gender” to be a cause of “attractiveness” in CelebA, the user can apply do-interventions that remove all edges connecting “gender” to “attractiveness”. This way, if we change the prediction for “gender” (formerly a cause), this can no longer have any effect on “attractiveness”. However, we clarify that we did not run a user study as that went beyond the scope of this current work and our contributions.

---

Do you have any intuition about why there is a drop in Figures 4 and 5 for a certain number of intervened concepts?

A drop in performance between intervention $i$ and intervention $i+1$ indicates that the label intervened at step $i$ was already well-predicted. Figures 4 and 5 show the delta in performance before and after interventions on all non-intervened labels/concepts. Consequently, removing this well-predicted concept from the computation had a negative impact, outweighing the benefits of the intervention. If a label is correctly predicted, excluding it from the metric calculation will reduce the delta score.

---

Line 345 says CIFAR10 is an animal classification dataset. Did you use the entire dataset or a subset for evaluation?

Enitre dataset. Thanks for pointing out our error in describing the data! We will replace it with “image classification dataset”

---

[1] Koh, Pang Wei et al. “Concept Bottleneck Models.” ArXiv abs/2007.04612 (2020): n. pag.

"I do not understand the experiment described in the last paragraph of the experimental section (lines 463-476). Could the authors please provide an example? How are variables $i$ and $j$ selected for each dataset? Are the results in Table 3. averaged over all possible pairs? How do the authors decide which variables to intervene on (i.e. what structure is used as Ground Truth)?"

The last experiment aims to demonstrate that our model can be effectively debiased through do-interventions. The idea behind this procedure is described in the paragraph “Model verification and blocking”. However, to give more details about the choice of the variables, we added the following description in App. H.7:

1. Selection of Variables $i$ and $j$: We randomly select a pair of concepts, $i$ (parent) and $j$ (child), connected in the graph discovered by the CD algorithm or CausalCGM.

2. Do-Intervention: A do-intervention is applied to the child node ($j$) to cut off its dependence on the parent node ($i$). In our model, this intervention ensures that all subsequent nodes in the graph become independent of the parent node, as information flow is blocked by design.

3. Measuring the Impact: After the intervention, we modify the value of the parent node ($i$) and measure its impact on the final label (leaf node) with $i$ as an ancestor. If no effect is observed, it indicates successful debiasing, as expected by our model’s design. Cutting the edge breaks the information flow through the graph—from root to leaf—preventing the parent node from influencing subsequent nodes.

4. Comparison with Baselines: Baselines have no graph constraints, therefore the do-intervention modifies the value of a concept but does not alter the decision-making process. As a result, the parent node’s influence cannot be entirely removed. This contrasts our model’s ability to block the information flow fully.

---

Why is 0 a desirable outcome? (Table 3)

Consider a graph $A \rightarrow B \rightarrow C$. For this graph, the variable $A$ is a cause of the variable $C$. As a result, we expect the average causal effect of $A$ on $C$ to be higher than $0$. Our experiment aims to show that intervening on $B$ makes $A$ and $C$ causally independent (cf. with L323-331). To show this, we apply a do-intervention in $B$, removing the edge $A \rightarrow B$ and generating two disconnected graphs: $A$ and $B \rightarrow C$. Now, we expect that the residual average causal effect of $A$ on $C$ is $0$ because there is no longer a connection between $A$ and $C$. As a result, $0$ is the optimal desirable outcome. To improve clarity, we added this example in App. H.7.

We greatly appreciate the reviewer’s thoughtful comments and insights. We would like to kindly point out that several of the concerns raised appear to stem from evaluating our approach as a causal discovery/identification/estimation method (all the referenced papers are on this research area). We respectfully invite the reviewer to revisit these points with the clarification that our model is explicitly not intended as a causal discovery/identification/estimation method but as a solution to the causal opacity issue, as we emphasized extensively in the introductory section of the manuscript (L38), in the main body (Remark 3.3), and in the appendix (App. J). The differing nature of the target problem (causal opacity vs. causal discovery/identification/estimation) has a significant impact on objectives, constraints, optimization processes, and scalability, as discussed in the following answers.

---

Theorem 3.2 is provided without proof

We include the proof of theorem 3.2 in App. C whilst discussing the main intuition in the main body of the text (as stated in L198).

---

Additional assumptions, such as interventions or sufficient variability, need to be assumed to guarantee the recovery of the underlying causal structure. In that regard, theorem 3.2 does not state any of these assumptions

Theorem 3.2 does not require any of these assumptions as the problem of causal opacity is not to recover the causal mechanisms in the real world (cf. to shared answer on identifiability). In particular, addressing causal opacity does not pose any constraints in terms of interventions or sufficient variability in observations.

---

Guarantee non-confoundness

1. Causal CGM does not guarantee non-confoundness just as it does not guarantee identifiability. Please see our shared answer on identifiability for our argument for why this is a feature and not a bug.

2. Our results (Fig 4, 5, 12, 13) show that interventions are still more effective w.r.t. existing CBMs despite the presence of potential confounders.

---

Variables are predicted just from their exogenous noise variables

1. We would like to point out that variables are not just predicted from their exogenous variables alone. As we show in Def. 3.1 and Theorem 3.2 (and more in detail in App. D), endogenous variables are predicted from both their endogenous copies and their exogenous variables. In those statements, $\mathcal{V}$ and$\mathcal{V}^\prime$ are endogenous variables satisfying $p(v_i \mid pa_{\mathcal{V}'}(v_i), u_i; \theta_f) = p(v_i \mid pa_{\mathcal{V}}(v_i), u_i; \theta_f)$

2. As evidence that indeed variables are not just predicted from their exogenous noise variables, we would like to point out that our intervention experiments show that endogenous ancestors have a causal effect on endogenous descendants (cf. with Fig 4, 5, 12, 13). This would not be possible unless there is a dependence on endogenous variables too.

---

Performance gap between CCGM+CD and CCGM

1. We did not observe any significant gap in task accuracy (Tab 1). So, we assume the reviewer refers to the gap observed in Fig 4 and 5 representing intervention effectiveness.

2. CCGM+CD shows that combining causal representation learning with our method can improve intervention effectiveness. This also highlights why our method is neither a causal representation learning technique nor a causal discovery technique. Rather, the proposed approach is complementary and can be combined with such techniques.

3. The gap in intervention effectiveness is a form of explanation. This gap is the result of the intended use of our model. It shows that the causal structure used by the model for inference might differ from a human’s causal structure and the data-generating process’ causal structure. As a result, interventions may not end up having the intended effect.

4. When CCGM+CD does not scale (Tab 1) because of the CD component (see CIFAR10), CCGM becomes the preferred solution for interventions (more effective than existing CBMs).

The paper frequently references the concept of causal opacity, but it is too vaguely defined to be fully understood.

Causal opacity can be understood using Pearl's framework of causality (cf with [1], L29-43 and App. A). One way to define causal opacity is to consider a similar problem: causal discovery. While causal discovery examines how variables are causally related in the real world, causal opacity focuses on understanding how variables are causally related in a model's decision-making process. The object under study (the “world” in causal discovery and the “model” in causal opacity) is different, but the ladder of causality is the same. To further clarify this, we modified two sentences in the introduction: (L31) “... Similarly to causal discovery, causal opacity can be assessed …”; (L36): “... instead of k?”). However, as opposed to causal discovery, the target of causal opacity is understanding …”.

---

How the model performs abduction to answer counterfactual queries is unclear

CCGMs follow [2] performing abduction by inferring the value of exogenous variables based on the available evidence (L308-10). To further clarify this, we will add the following in App. D where we discuss the details of the architecture: “In causal CGMs we use the exogenous encoder to predict the value of exogenous variables as we consider that the observed input (e.g., an image’s pixels) holds most of the contextual variability required to infer exogenous variables. For instance, an image provides information about background, lighting conditions, and object locations. All this information is used to anchor the endogenous predictor to a specific context and to correctly infer the values of the exogenous variables and the endogenous roots of the causal graph.”

---

It seems the proposed model sacrifices accuracy to improve interpretability

1. Our primary goal was to enhance interpretability with minimal accuracy loss, not to surpass the accuracy of black-box models (L358-62). Causal CGM’s enhanced interpretability enables users to intervene and debias the model at inference time, as opposed to black boxes and more efficiently than existing CBMs. We point out that in several critical instances where verification is necessary (e.g., hiring, law, loan management), this makes our models plausible options for these tasks as they still have an accuracy close to that of black-box whilst being fully interpretable. In contrast, black-box models would not satisfy the verifiability requirements.

2. In our experiments, the accuracy drop is at most 1.5 p.p. w.r.t. a black box model (Tab 1). While the trade-off between accuracy and interpretability is well-known, our model achieves comparable performance to black-box models while providing an interpretable decision-making process.

---

How do you guarantee the causal graph will be acyclic?

1. Eq 4 is used in the loss function to make the graph acyclic.

2. After training, we guarantee acyclicity by removing the least important edges that create cycles if any. Thanks for pointing this out. To clarify our statement, We will include the following sentence in Appendix E: “During training, we optimize a loss that encourages the model to construct a DAG. However, in rare cases, some edges with very small weights may still make the adjacency matrix $A$ cyclic. At inference time, we address this by iteratively removing the edges with the smallest weights from the cycles until the graph becomes a DAG.”

---

[1] Facchini, Alessandro, and Alberto Termine. "Towards a Taxonomy for the Opacity of AI Systems." Conference on philosophy and theory of artificial intelligence. Cham: Springer International Publishing, 2021.

[2] Pearl, Judea. Causal inference in statistics: a primer. John Wiley & Sons, 2016.

Relations and comparison with causal representation learning

Causal CGMs are not an example of CRL, as discussed in Remark 3.3 and App. K. We agree with this statement. In fact, we remark this in our experiments by showing how CCGM and causal representation learning are complementary; they are not competitors. CCGMs can be combined with any existing causal representation learning method. In the experiments we call this combination “CCGM+CD”, representing the combination of CCGM with causal discovery methods to improve the effectiveness of interventions.

---

Endogenous variable representation: scalar values vs. high-dimensional embeddings

Unlike existing methods (e.g., Neural Causal Models and Causal VAEs), the Causal CGMs architecture is built using expressive high-dimensional representations for endogenous variables. Details of this architectural aspect are provided in Appendix D, and its impact on the predictive performance of Causal CGMs is discussed in Section 4.1 (Line 371). However, as this is a significant difference w.r.t. existing methods, such as Neural Causal Models and Causal VAEs, we emphasized this distinction even further in Section 3 by expanding the description provided in Appendix D as follows: “As observed by [2], using scalar representations for concepts (corresponding to endogenous copies in our context) can significantly degrade predictive performance in realistic settings. To address this issue, and inspired by [3], Causal CGMs employ high-dimensional representations for endogenous copies $\mathbf{v}_j'$. For each endogenous copy, Causal CGMs learn a mixture of two embeddings with explicit semantics, representing the variable's state. This design enables the model to construct evidence both in favor of and against a variable's state and supports simple concept interventions, as it allows switching between the two embedding states during interventions. Specifically, we represent the context of each variable with two embeddings: $[\mathbf{c}_j^+, \mathbf{c}_j^-] = \mathbf{u}_j = \zeta(\mathbf{x}), \quad \mathbf{c}_j^+, \mathbf{c}_j^- \in \mathbb{R}^m$. Each embedding carries specific semantics: $\mathbf{c}_j^+$ represents the active state of the variable (true), while $\mathbf{c}_j^-$ represents the inactive state (false). Once these semantic embeddings are computed, the final endogenous embedding $\mathbf{v}_j'$ for $v_j$ is constructed as a mixture of $\mathbf{c}_j^+$ and $\mathbf{c}_j^-$, weighted by the endogenous state:

$

\mathbf{v}_j' = v_j' \mathbf{c}_j^+ + (1 - v_j') \mathbf{c}_j^-

$

This formulation serves two primary purposes: (i) it forces the model to rely exclusively on $\mathbf{c}_j^+$ when the $j$-th endogenous variable is active ($v_j’ = 1$) and on $\mathbf{c}_j^-$ when inactive, thereby creating two distinct and semantically meaningful latent spaces; (ii) It enables a straightforward intervention strategy, where one can switch between embedding states to correct a mispredicted endogenous variable.”

Additionally, we conducted an ablation study by replacing embeddings with scalar values in Causal CGM following a CBM-like approach on CelebA. The results demonstrate that when following a causal graph, this approach struggles to achieve the same performance as other models due to its limited expressiveness ($72.72 \pm 3.70$ compared to $78.42 \pm 0.42$ achieved with embeddings). However, when the model is allowed to learn the graph, this approach achieves performance comparable to other Causal CGM or CBM models ($78.64 \pm 0.30$). Notably, the learned graph in this case only includes edges from concepts to labels, effectively reducing it to a CBM structure. As shown in other experiments, this structure is suboptimal due to its reduced impact in interventions and the inability to effectively debias the model.

---

Column 4 is Table 3 is CIFAR10?

Yes. We thank the reviewers for noticing this. We have fixed this typo in our updated manuscript.

---

[2] Mahinpei, Anita, et al. "Promises and pitfalls of black-box concept learning models." arXiv preprint arXiv:2106.13314 (2021).

[3] Espinosa Zarlenga, Mateo, et al. "Concept embedding models: Beyond the accuracy-explainability trade-off." Advances in Neural Information Processing Systems 35 (2022): 21400-21413.