Counterfactual Concept Bottleneck Models

"Rewording the conclusion of 5.1” and add Table 8 in Section 5

We rephrased the conclusion of 5.1 as follows: “CF-CBMs achieve generalization performance that is close to standard CBMs”. Regarding Table 8, we agree with the reviewer that including it in Section 5 would add value. However, the differences between the methods are minimal. Given the limited space and the fact that accuracy is not the primary focus of our work, we prioritized presenting other results instead.

---

Why do CF-CBMs should be less prone to confounding factors?

CF-CBMs prioritize the most impactful features for the task, as demonstrated in Section 5.2. If a set of features has greater predictive power, the model will favor them over the confounder, which has a lower impact. For example, in our experiments, key features are consistently useful for prediction (100% power), whereas the confounder contributes correctly only 85% of the time. This behavior arises from our loss functions (Equation 5), which guide the model to learn decision boundaries perpendicular to meaningful features while minimizing reliance on less informative ones. This happens to facilitate the counterfactual generation process which wants the output to remain close to the input but also in-distribution.

---

“Provide the code”

The code is already attached and can be used to reproduce the results. However, in the final version of the paper we will include the link to a GitHub repository.

---

Additional citation to a paper using counterfactuals in the realm of concept based interpretability

We thank the reviewer for the suggestion and for sharing this interesting paper with us. We have added the mentioned reference in Section 6 (Related Work), as we believe it is more appropriate to discuss it there. As pointed out by the reviewer, [1] is indeed connected to our work, albeit from a different perspective.

Figure 1: Which dataset are the images sourced from? Are there any restrictions on copyright? Why are you not using the Osteoarthritis dataset in experiments?

1. The images in the abstract figures come from the Osteoarthritis dataset present at this link (https://www.kaggle.com/datasets/shashwatwork/knee-osteoarthritis-dataset-with-severity) distributed with license CC4.0. Thanks for pointing out that there is no reference.

2. We modified all the examples using the SIIM Pneumothorax dataset.

3. We added experiments on the SIIM Pneumothorax dataset (see the common answer for the experiment on this x-ray dataset).

---

Why $z{\prime}$ depends on $c$ and $y$?

$y$ is not essential but it avoids unnecessary double learning of the same process. We agree that $y$ is not strictly necessary, as $c$ already contains the information to predict $y$. However, including $y$ makes this information explicit. Without $y$, the network predicting $z{\prime}$ would also need to learn again what the predictor has already learned, adding unnecessary complexity to the model.

---

Why does Equation 5 ensure sparser explanations?

Learning variational latent models with smooth priors (like our VAEs with standard gaussians priors) encourages smoothness in the encoding/decoding scheme, i.e. samples close in the latent space ($z$, $z{\prime}$) tend to be close also in the output space ($c$, $c{\prime}$). We have added a reference in the paper to [1]. Thanks for pointing this out.

---

How is the model trained?

Jointly. The model is trained jointly to fully leverage the advantages outlined in Section 5.2. However, it is also possible to train the model in a post-hoc manner—first training the concept encoder and predictor, followed by the counterfactual generator, or performing a warm-up phase for the encoder and predictor to better initialize their weights before continuing with joint training. We have included this detail in Appendix C.3.

---

Are Introduced black-box components affecting interpretability?

No, the introduced black-box components do not affect interpretability, just as adding a hidden layer to a CBM does not compromise its interpretability. Our model remains as interpretable as CBMs, focusing on concept-based interpretability by highlighting the key concepts used for the final prediction. The added complexity is limited to the stages before concept extraction, enabling the encoding of information about possible counterfactuals. Thus, our model retains the interpretability of CBMs while providing the additional capability to explain decisions through counterfactuals. We added this at L158 as follows: “The added complexity introduced with these additional black-box components is limited to the stages before concept extraction, enabling the encoding of information about possible counterfactuals. Thus, our model retains the interpretability of CBMs while providing the additional capability to explain decisions through counterfactuals.”

---

[1] Kingma, Diederik P. and Max Welling. “Auto-Encoding Variational Bayes.” CoRR abs/1312.6114 (2013): n. pag.

Question: “visual examples of counterfactuals generated by your model and baseline methods for each dataset”

We are not interested in visual explanations, but we provided concepts counterfactual examples. As explained in the paper, we are not interested in generating counterfactuals at the input level (e.g., images) because they are more difficult to interpret. Generating counterfactuals at the concept level also offers other advantages. It simplifies the information that should be contained in the latent representation ($z$ and $z{\prime}$), allowing the model to focus on salient information rather than encoding all the details necessary to recreate the full image. This makes it easier for the model to learn the latent distribution. Therefore, plugging a decoder into $z$ and $z{\prime}$ (which are related to concepts) to reconstruct the image with changed concepts would require a more complex latent space and significantly complicates the optimization process. However, we provided a few examples of counterfactuals at the concept level per model and dataset in the shared response.

The SCM is not well-detailed

1. Counterfactual generation in XAI has a different definition and assumptions w.r.t. Pearl’s causal framework. In this respect, the counterfactual generation we refer to fits in the definition of Wachter’s framework [1] which does not require the explicit definition of an SCM (see L103-107).

2. In CBMs the “ground truth causal graph” of the SCM is the bipartite graph connecting concepts to tasks, as shown by the PGM in L118. However, in the XAI framework for counterfactual generation it is not required to get an explicit set of structural equations in order to generate counterfactuals.

---

“Any methods aimed at counterfactual explanations would address these questions to some extent.”

We emphasize that our method addresses these questions by design, unlike most counterfactual methods, which are post-hoc. As Rudin [2] notes, the explanatory robustness of surrogate models might be questionable as they may not necessarily align with the original model’s decision-making process. Interpretable by-design architectures, such as CBMs and CF-CBMs, do not suffer from this issue. Our approach allows answering both “how” and “why not” questions directly, even on unstructured data—a significant challenge, particularly for “how” questions. For example, measuring the impact of modifying a single pixel often lacks meaningful interpretation, even with a counterfactual generator. While counterfactual explanations generally enable answering “why not” questions and provide some insights into “how,” they often fail to properly address the latter.

---

Better contextualize the work in the abstract

We modified it in the reviewed version, thanks for the suggestion. We added the following sentence at L13: “While current approaches in causal representation learning and concept interpretability are designed to address some of these questions individually (such as Concept Bottleneck Models, which address both “what” and “how” questions), no current deep learning model is specifically built to answer all of them at the same time.”

---

Citations on VAE for causal inference

Our model does not focus on causal inference. Its goal is to provide insights into the model’s reasoning process rather than discovering or encoding causal relationships. However, we acknowledge the relevance of the cited works and similar research in adjacent areas and have included the following clarification at L48: “Causal inference aims to answer this question from a statistical perspective [3,4], seeking to discover and/or exploit the true causal relationships in the data. In contrast, we focus on addressing these questions from the model’s perspective, shedding light on its decision-making process. From this viewpoint, state-of-the-art counterfactual generators …”

---

“Why sampling a counterfactual concept c' does not also require sampling a fresh set of covariates x' (why the p(x|z) term is ignored in the optimization [L152]?)”

This is an advantage rather than a limitation. Since $z$ and $z{\prime}$ represent the latent distributions of the concepts $c$ and $c{\prime}$, there is no need to resample a fresh set of covariates $x{\prime}$. We are not focused on reconstructing or generating $x$ and $x{\prime}$, so the $p(x|z)$ term is ignored. This simplification allows the model to encode only the most salient information about the concepts, avoiding the need to capture all the details required to reconstruct the image. Ignoring $p(x|z)$ significantly reduces the complexity of the optimization process, making counterfactual generation on complex data, such as images, easier. However, we thank the reviewer to have pointed this out and we will rephrase L152 as follows: “In practice, we assume that the input $x$ is always observed at test time, making the $p(x|z)$ term irrelevant. This allows $z$ and $z{\prime}$ to encode only the most salient information about the concepts $c$ and $c{\prime}$, rather than all the details needed to reconstruct $x$. As a result, the optimization process is significantly simplified, enabling efficient counterfactual generation for complex data like images.”

---

[1] Wachter, Sandra, Brent Mittelstadt, and Chris Russell. "Counterfactual explanations without opening the black box: Automated decisions and the GDPR." Harv. JL & Tech. 31 (2017): 841.

[2] Rudin, Cynthia. “Please Stop Explaining Black Box Models for High Stakes Decisions.” ArXiv abs/1811.10154 (2018): n. pag.

[3] Louizos, Christos et al. “Causal Effect Inference with Deep Latent-Variable Models.” Neural Information Processing Systems (2017).

[4] Xia, Kevin Muyuan et al. “Neural Causal Models for Counterfactual Identification and Estimation.” ArXiv abs/2210.00035 (2022): n. pag.

Model’s behavior on OOD counterfactuals

This question is more closely related to the “how” questions in XAI. In the context of XAI, counterfactual generation is conditioned on a different desired label, requiring the model to produce a new input (concepts in this case) that leads to the target label. This differs from counterfactuals in causal inference, where a feature is manually modified (e.g., making an elephant pink) to observe its effects on other features. While these two paradigms are related, they serve different purposes. What you describe aligns more with manually modifying specific concepts (e.g., attributes of a bird) to create an OOD sample and observing how the model predicts it. This approach is indeed related to “how” questions and allows users to inspect the model’s behaviour under different scenarios. In contrast, counterfactuals in XAI are generally expected to remain in-distribution, as they represent feasible states leading to the desired label [1]. Notably, from our experiments, we observed that our counterfactuals, leveraging VAE-like methods, tend to remain in-distribution even when the input is OOD.

---

Answer how questions exploiting counterfactual information

We agree that it is possible to answer “how” questions in the way you described, but this approach is inefficient. Generating multiple counterfactuals by iterating over all possible $y{\prime}$ values provides only general information about the impact of specific features or pixels on a label. It does not enable a fine-grained analysis of how modifying a specific feature to a particular value affects the prediction. Moreover, these measured impacts depend on what the counterfactual generator has encoded, which may not fully align with what the model has learned, leading to potential discrepancies. Developing a method to address all the three questions without additional components is quite challenging. One possibility to generate counterfactuals exploiting the how answers is to iteratively modify concepts based on their measured impact (e.g., using CaCE) until the label reaches the desired value. However, this approach often results in OOD counterfactuals, which are undesirable [1]. Answering “how” questions starting from the generated counterfactuals remains a difficult task. We find this to be an interesting direction for future work, potentially exploring ideas along the lines of what the reviewer suggested. We appreciate the valuable feedback and will consider it in our future research.

---

[1] Wachter, Sandra, Brent Mittelstadt, and Chris Russell. "Counterfactual explanations without opening the black box: Automated decisions and the GDPR." Harv. JL & Tech. 31 (2017): 841.

Discussion on hyperparameters in the training objective

The hyperparameters in the loss ($\lambda$) were chosen through validation on a subset of the training to achieve the best possible trade-off across all metrics important for counterfactual evaluation, such as validity, proximity, and sparsity, for both our method and the baselines. While our method introduces more hyperparameters, it provides greater flexibility to specify the desired objective. As counterfactual metrics often conflict, creating inherent trade-offs (as shown in our experiments), there is no one-size-fits-all solution. Thus, optimizing the counterfactual generator based on the metric of interest is crucial, and our method facilitates this. Each hyperparameter in the loss has a specific role in steering the model toward a particular behavior:

• $\lambda_1$ (task-related) and $\lambda_2$ (concept-related) are important in all configurations as it remains the main goal of the model.
• $\lambda_3$ prioritizes validity, encouraging counterfactuals with predictions matching $y{\prime}$, though potentially compromising proximity or sparsity.
• $\lambda_4$ and $\lambda_5$ emphasize generating more realistic counterfactuals, potentially reducing proximity, which may trade off against validity and sparsity.
• $\lambda_6$ and $\lambda_7$ promote sparsity by reducing the number of changes from the input, which may trade off against validity. However, we observed that $\lambda_6$ has minimal influence on the optimization process, with $\lambda_7$ being the key hyperparameter driving this behavior.

This flexibility makes our model more versatile and practical compared to baselines, which have limited or partial support for these trade-offs (e.g., VAECF has the possibility to be flexible on validity). We appreciate the reviewers’ feedback and we clarified how to select these hyperparameters in the revised manuscript in Appendix C.3. For additional evidence, we conducted an ablation study on the MNIST-Add dataset, modifying $\lambda_3$, $\lambda_4$, $\lambda_5$, and $\lambda_7$ to evaluate the model’s behavior across different objectives:

---

Show examples of cf in the input space

Our model produces counterfactual explanations only within the concept space. Like other deep learning classifiers, it does not model the input space distribution on which it operates conditionally (i.e., discriminatively). This approach is intentional, as we argue that concept-based counterfactuals align more closely with human understanding and are more actionable. For instance, while minor pixel changes in an X-ray offer limited value, flipping a concept such as "Visible Pleura Line" provides meaningful and interpretable insights to the physician. However, as you pointed out, it would be helpful to see a comparison of the counterfactuals generated by the different methods across the datasets, so we have included these examples in the revised version of the manuscript in Appendix E, where it is visible that our counterfactuals are also qualitatively better than the other baselines.

---