Causally Reliable Concept Bottleneck Models

We thank the reviewer for their comments and feedback. We particularly appreciate the positive remarks on the novelty, soundness, and clarity of our work. As for the raised questions, individual points are addressed below.

Q1 - Label prediction

I am missing where the label prediction occurs in the method description

Starting from a data sample (e.g., raw image), we first extract a latent representation using a neural encoder. Then, an MLP is used to predict the probability of each root concept being in a specific state (e.g., true/false). Then, the state probabilities for all descendant concepts (including the task concept) are obtained by iteratively applying Eq.2. Specifically, Eq.2 shows that the probability of each concept state is determined by a linear combination of the state probabilities of its parents in the causal graph. Notice that, once the causal graph is determined and C$^2$BM is trained, one can choose any endogenous variable (concept) as the query (task). We refer to Fig. 7 in the Appendix for a more detailed visualization.

We recognize the paper speaks in terms of 'variables' $V_i$ rather than state probabilities. This could be ambiguous. To improve clarity on this aspect, we added a footnote in l.193: "We refer to $V_i$ as a variable to allow for a more general formalization. In a classification setting, concepts assume categorical values; hence $V_i$ represents the probability of a concept state activation."

Q2 - Difference with causal discovery

the goal is of C$^2$BM and how this differs from simply doing causal discovery

It is important to distinguish C$^2$BM from the components that need to be determined prior to its training and functioning. C$^2$BM is not a causal discovery algorithm. It is an inference ML model (see right side of Fig.1) trained on a given causal graph of concepts. For each data sample, the model learns to predict the activation of each variable in the graph based on its causal parents. In this setup, the causal graph acts as a structural prior, effectively a hyperparameter, which can be obtained from expert knowledge and/or from (potentially separate) data using causal discovery methods (on this, see Remark 4.1 and response Q2 to Reviewer wNgz).

To give a concrete example: causal discovery might be used by a hospital to infer which factors are, in general, direct causes of a disease (as in our Pneumothorax dataset). C$^2$BM, once trained on this structure, can then make patient-specific predictions, e.g., estimating whether a particular patient is sick and which of the known causes are likely active in their case.

Q3 - Clarification on broader impact

I am also a little puzzled by the broader impact statement "C$^2$BM has the potential to significantly narrow the hypothesis space in complex scientific domains where even a panel of human experts might struggle to identify or exclude plausible hypotheses worth testing."

We thank the reviewer for pointing this out; we recognize this sentence could be unclear. The sentence highlights a benefit commonly associated with causal discovery, one that C$^2$BM both inherits and extends. While causal discovery narrows the hypothesis space by uncovering candidate causal structures, C$^2$BM builds on top of this and enables predictive inference over those structures. That is, C$^2$BM allows for interpretable, concept-level predictions grounded in those relationships. In this way, C$^2$BM preserves the scientific value of causal discovery, while adding the practical ability to make predictions from data and unstructured knowledge, and generate explanations, critical in real-world decision-making contexts.

Q4 - Clarification on debiasing experiment

I would assume if a vanilla CBM has color and shape concepts it should be very easy to debias the model

While, in principle, a standard CBM could support debiasing by manually setting the appropriate weights to zero, this requires knowing which relations are spurious in advance. In practical applications, this could require substantial domain expertise. In contrast, our pipeline incorporates a causal graph construction component that can automatically detect and remove spurious correlations from unstructured knowledge. C$^2$BM's inference flows through such graph that is already debiased, and hence does not require manual post-hoc debiasing.

How are the interventions being done here? ... Could the authors clarify how exactly the results of figure 5 were created

The interventions in this experiment follow the standard practice in concept-based explainability [1]: we replace the predicted concept value with its ground-truth state and measure the consequent improvement in the task accuracy.

Specifically, all models were trained on a biased training set (color $-$ parity) and evaluated on a test set where the spurious correlation is reversed. As expected, all models perform poorly ($\sim 0$%) when tested on reversed data. However, unlike the baselines, C$^2$BM is able to recover task accuracy when we intervene on the concept that is causally related to the task (number $\rightarrow$ parity). The other models fall short in doing so, since they still rely on spurious correlations.

Q5 - Interventional accuracy

I would expect particularly here such models to shine. Could the authors discuss a little bit on this.

We speculate these are primarily due to the probabilistic nature of our datasets. In such settings, fixing the state of the parents is often not sufficient to uniquely determine the state of a child. Exogenous factors also matter. Therefore, achieving large improvements in intervention performance in these settings is inherently difficult. As shown in Fig.4, even baseline models exhibit small gaps between them. That said, our model consistently improves over all baselines across all datasets, highlighting an advantage even under these constrained conditions.

As proof of this, in concept-complete settings such as the synthetic $CUB_c$ dataset, where we explicitly made a few child nodes deterministically dependent on their parents, our approach yields significantly larger gains.

We see our work as a first step toward integrating causal reasoning with concept-based interpretability. We believe incomplete or noisy causal structures can offer a promising direction for future research.

Q6 - Table formatting

Do you think it might be an idea to move two columns to the appendix to improve the readability within the main text?

We included incomplete datasets because we were concerned that using only the complete datasets would not clearly convey the message we aimed to communicate regarding preserving the black-box task accuracy. We are considering removing cMNIST, as it is a very simple task, in order to improve readability.

Q7 - Citation recommendation

I would consider citing [1] in terms of concept interventions

We thank the reviewer for the suggestion, we think it is relevant to highlight the relevance and timeliness of interventions. We have added the reference at l.220.

We hope to have clarified all of the reviewer’s concerns and are happy to provide further details if needed.

Refs:
[1] Koh et al., Concept bottleneck models, ICML, 2020

We thank the reviewer for their comments and feedback. As for the raised concerns and questions, individual points are addressed below.

W1 - Performance under adversarial graph corruptions

if the graph is misspecified, these biases can hurt task accuracy relative to graph-agnostic models.

We disagree. C$^2$BM is built upon a key theoretical result (see Appendix D.1), which shows that it is a universal approximator for the final task, regardless of the specific causal graph. This implies that C$^2$BM retains task accuracy of black-box models, even with a largely corrupted graph, as long as the architecture is sufficiently expressive. Recall that, while task accuracy is preserved, causal reliability does degrade under graph corruption. This would affect the quality of explanations and out-of-distribution (OOD) performances, as shown by the differences between standard CBMs (i.e., flat, unreliable causal structures) and C$^2$BMs in Tab. 2 and Fig. 5, respectively.

W2,Q1 - Sensitivity to graph corruptions

it's unclear how sensitive C$^2$BM is to the quality of the causal graph. ... Have you quantified the extend to which the quality of the causal graph is a determining factor for model performance?

Despite the theoretical argument, we agree that it is important to empirically validate C$^2$BM's robustness in task accuracy. To this end, we added a new sensitivity analysis evaluating task accuracy when training C$^2$BM on increasingly more corrupted graphs (3 seeds, uncertainties represent 2 sample mean $\sigma$). We consider two strategies, both starting from the ground-truth graph. In the first, we alter a percentage $p$ of graph edges by applying one of the following operations (chosen randomly, per edge): flipping, addition, or removal.

In the second one, we progressively transform the graph into the flat structure assumed by standard CBMs, i.e., progressively connecting a percentage $p$ of nodes to the chosen task and eliminating other outgoing edges.

The results confirm that task accuracy remains stable even when the causal graph is heavily corrupted, empirically supporting the theoretical claim. We thank the reviewer for raising this point, we will extend the paper with this new experiment.

We hope to have clarified all of the reviewer’s concerns and are happy to provide further details if needed.

We thank the reviewer for their comments and feedback. As for the raised concerns and questions, individual points are addressed below.

As a side note, we found that the citation mentioned by the reviewer could support our Remark 4.1. We have decided to include it; thank you for the pointer.

W1 - Prompt design

the design of the prompt might affect performance.

We agree. The design of the prompt could condition the causal graph refinement. To assess this, we have added a new ablation study in which we fix the incomplete causal graph generated by the causal discovery algorithm, and later evaluate variants of the LLM+RAG refinement. Specifically, we evaluate the impact of four different prompting strategies with two LLMs (GPT-4o, $\sim$200M parameters; and GPT-4o-mini, $\sim$8M parameters). The considered strategies are as follows:

• minimal prompting: simply asks the LLM to identify causal relations without additional guidance;
• instruction prompting: provides a more detailed explanation of what constitutes a causal relation;
• few-shot prompting: proposed in [1], combines instruction prompting with a few examples;
• Chain-of-Thought (CoT): proposed in [2], encourages the model to generate intermediate logical steps before generating the answer. This is the approach we used in the original manuscript.

The table below shows the number of Mistaken Edges for datasets with known ground-truth causal graphs (reliability of standard, flat, CBMs is also reported for reference).

Despite moderate variation in the Sachs dataset, the performance is generally very robust to the prompt strategy, and causal reliability is largely superior to standard CBMs in all cases.

W2 - Understudied domains

causal graph construction would be harder in understudied domains like rare diseases

This is undoubtedly true, as we mentioned in our limitations (l.366). However, we believe understudied domains leave very little room for any alternative method. Note that, in any case, our pipeline can still approximate causal relations from observational data only. We hence believe C$^2$BM could be useful for exploratory research in understudied domains to restrict the search space of possible hypotheses and contribute to forming a background knowledge.

Q1 - Sample complexity of graph discovery

What is the effect of data set size and causal graph complexity on the causal discovery method?

We explored this with a new sensitivity study, running the causal graph pipeline (causal discovery + LLM refinement with CoT prompt) across all datasets with an available ground-truth graph. For each, we varied the number of data points $N$ from 100 to 10,000.

A few considerations:

• C$^2$BM's causal graph is consistently more causally reliable than the flat structure implicitly assumed by standard CBMs, regardless of the dataset size.
• As expected, increasing the number of data points leads to better alignment between the estimated and true causal graphs. Causal reliability tends to remain stable at larger data sizes.
• We observed that the data size threshold for reliable performance does not strictly depend on graph size. We speculate this is due to the varying impact of LLM-based refinement across datasets. In some cases, an effective background knowledge can compensate for limited data.

We appreciate the reviewer’s question and will incorporate these findings into the paper.

Q2 - Clarification on RAG performance

Were there any cases where using RAG degraded performance?

No, we did not observe any cases where using RAG degraded performance. We speculate this is because all our investigated domains are well-supported by archived scientific literature. In the paper, we included results for all the datasets we experimented on.

We hope to have clarified all of the reviewer’s concerns and are happy to provide further details if needed.

Refs:
[1] Ye et al., Investigating the effectiveness of task-agnostic prefix prompt for instruction following, AAAI, 2024
[2] Wei et al., Chain-of-thought prompting elicits reasoning in large language models, NeurIPS, 2022

We thank the reviewer for their comments and feedback. As for the raised concerns and questions, individual points are addressed below.

Q1 - Bottleneck evaluation

the manuscript lacks details on how the bottleneck is quantitatively evaluated and how this mechanism influences the model’s operations.

The standards of evaluation of concept bottlenecks [1] are well established in the CBM literature [1,2,3,4,5,6]. These are evaluated in terms of interpretability (e.g., concept accuracy) and actionability (e.g., interventions) (ll.19-26). Matching task accuracy of black-box models is also an important desideratum (ll.70-72), yet not trivial when conditioned to provide human-interpretable and actionable models. In fact, this often exceeds typical expectations of the CBM community [7].

Regarding our proposed causal concept bottleneck, we disagree that a quantitative evaluation is lacking in our manuscript. Section 5 provides extensive evaluations of our causal bottleneck, assessing standard CBMs' desiderata (see above) while also evaluating further specific benefits of our model.

Specifically:

• (Task acc.) Tab. 1 shows that our causal bottleneck preserves the task accuracy of black-box models, in line with CBMs' desiderata.
• (Concept acc.) Tab. 6 shows that our causal bottleneck preserves the concept accuracy of prior CBMs.
• (Interpretability) Tab. 2 shows our bottleneck provides interpretability at a deeper level by aligning more closely with ground-truth causal structures than prior CBMs.
• (Actionability) Fig. 4 shows how our bottleneck permits more effective interventions than prior CBMs. Fig. 5 and Tab. 3 evaluate two C$^2$BM's critical benefits: permit human interventions to de-bias the model from spurious correlations and unfair behavior.

We hope this clarifies that our experimental evaluation is centered on the causal bottleneck and its contributions.

Q2 - Clarify distinction with prior works

Section 4 cites and utilizes many existing methods, which may obscure the unique contributions of the current study.

Thank you for your constructive comment, as it allows us to improve clarity.

Our core contribution is the C$^2$BM model (right side of Fig. 2), a novel extension of CBMs that, instead of assuming a flat layer of independent concepts, forces inference to flow through a given causal graph of concepts (ll.73-76). As stated in the paper (ll.42-44, ll.140-141, Remark 4.1 in ll.150-152), C$^2$BM requires as input a set of concepts (as most CBMs) and a causal graph connecting them; these are distinct elements that must be provided prior to C$^2$BM training.

However, these requirements are often not readily available in real-world scenarios (ll.140-142). We argue that fully automating the instantiation of C$^2$BM is both valuable and non-trivial. For this reason, we present a pipeline that enables full automation (ll.142-145) as a second contribution of our work; this justifies its placement in the Method section. While the individual components of the pipeline are mostly based on prior work (hence the citations), their integration into a coherent, automated process -- which was never published before -- is novel by definition.

To improve clarity, we have added a clarification in the main text after l.149: "In the following, we outline our implementation for each sub-problem. Specifically, building C$^2$BM's individual prerequisites will be mostly based on prior work. Note that integrating them into a coherent, automated pipeline is instead part of this paper's contributions.". We thank the reviewer for raising this point and hope this resolves the concern.

the authors should explicitly state the advantages of their approach and clearly differentiate it from prior works.

C$^2$BM is a kind of CBM architecture, not a method for concept or causal discovery, hence these do not constitute direct prior work from which C$^2$BM needs to be differentiated. In the paper, we explicitly state our advantages in terms of actionability and causal reliability w.r.t. prior methods in the CBMs community: in the introduction (ll.27-37), preliminaries (ll.73-76), related works (ll.110-112), and conclusions (ll.351-355).

Q3 - Task accuracy

The proposed model does not appear to achieve significantly better task accuracy compared to baseline methods.

We firmly disagree that this should be seen as a weakness. The ultimate goal of our field of research is to improve on interpretability and actionability aspects while preserving the task accuracy of traditional black-box approaches [2,6]. As demonstrated in Tab.1, C$^2$BM matches the performance of the corresponding black-box architecture, which is considered an upper limit in the interpretability community and generally difficult to achieve [7]. Furthermore, we support this result with a proof stating C$^2$BM is a universal approximator for the downstream task (see Appendix D.1). Therefore, the rest of the experimental analysis focuses on aspects critical to the interpretability community, i.e., reliability, actionability, and fairness.

Q4.1 - Training details

The training procedure and other experimental details are insufficiently described.

We disagree. C$^2$BM's training objective is formally described in Eq.6, i.e., maximum likelihood (similar to other CBM classifiers [2]) conditioned on endogenous parents' values and meta-model output. Full experimental details, including numerical values for training hyperparameters, are provided in Appendix~F (referenced in ll.243-244). Finally, for the most comprehensive list, we provide the code with all settings specified in the YAML configuration files.

Q4.2 - Ablations

we suggest including ablation studies from various perspectives ... such as concept discovery, labeling, and other relevant components

We value the reviewer's suggestion. A few ablations were already included in Appendix G due to space constraints. Specifically: (i) Appendix G.4 provides an extensive ablation on the causal discovery block. (ii) Appendices G.5–G.6 analyze components related to querying the LLM. In addition, we have now included: (iii) a new ablation on the LLM prompt (please refer to our response W1 to Reviewer 6yXN, and (iv) a study on robustness to corrupted causal graphs (response W2 to Reviewer NX3e).

Q5 - Background knowledge

explain how to build such a knowledge base. ... Is the pipeline for constructing the knowledge base fully automated?

There might be a misunderstanding here. We do not build the knowledge base ourselves. Instead, as in ll.180-181 and detailed in Appendix C.2, we leverage a pretrained LLM, which implicitly encodes broad, unstructured knowledge acquired during its training phase [3]. We further improve this with a RAG strategy, which allows the model to take archived domain-specific knowledge (e.g., scientific papers), available through public sources such as arXiv, automatically into account (ll.97-99 and Appendix C.2).

how to evaluate the effectiveness of the built knowledge base

We evaluate its effectiveness by measuring how much it improves the alignment between the learned graph and the ground-truth graph (see '+LLM' row in Tab. 2). This structure constrains C$^2$BM reasoning, making predictions and explanations intrinsically more aligned with real-world causal mechanisms.

How is the transferability of the knowledge ensured?

Transferability of the knowledge base to the model is ensured by construction: each concept value is predicted from its causal parents as defined in the graph refined with the knowledge base. This forces a principled flow of information aligned with expert knowledge. Nevertheless, we are not sure we fully understood the reviewer's question. We kindly ask the reviewer to reformulate should our response be off-target.

We hope our responses offer a clearer perspective on the paper and help position its contribution within the interpretability community. We are happy to further engage in the discussion.

Refs:
[1] Koh et al., Concept bottleneck models, ICML, 2020
[2] Zarlenga et al., Concept embedding models, NeurIPS, 2022
[3] Oikarinen et al., Label-Free Concept Bottleneck Models, ICLR, 2023
[4] Zarlenga et al., Learning to Receive Help: Intervention-Aware Concept Embedding Models, NeurIPS, 2023
[5] Poeta et al., Concept-based explainable artificial intelligence: A survey, Arxiv, 2023
[6] Ismail et al., Concept Bottleneck Generative Model, ICLR, 2024
[7] Arrieta et al., Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information fusion, 2020