To Reviewer CUVg (R4)

We appreciate your careful review and acknowledgment of our approach’s potential. Thank you for highlighting key areas for improvement, particularly around clarity and interpretability claims.

• R4W1: The paper's writing is unclear, especially in the methods section.

We apologize for any difficulty in understanding our proposed method. We believe that the difficulty mainly comes from the background knowledge of causal-related techniques. Therefore, in the appendix of the revised manuscript, we will include more detailed explanations and relevant citations of various causal-related components to better describe our method, including causal intervention and estimiation, propensity score matching, doubly robust learning, and intervention training method.

• R4W2: The paper overclaims about interpretability, e.g. "This capability not only supports sophisticated post-hoc interpretation but also enables dynamic human-AI interactions" -- it is unclear that the method yields interpretable concepts or arranges them in a manner that allows humans to understand them. More experiments (ideally with real human users) would help to show this

Thank you for this valuable suggestion. We appreciate this suggestion. To further clarify interpretability that allows humans to understand them, we conducted a test-time intervention experiment involving real human annotators. Here is the experimental settings:

1. Sample Selection: We selected 198 testing samples across 9 posterior concepts from the Bird dataset.
2. Intervention Process:
   • Each image was first classified by the NCG (PSM) model, which generated the predicted concept probabilities for the graph (similar to Figure 9 in the paper).
   • Human annotators were then asked to reference both the predicted probability graph and the original image to provide interventions. Annotators are required to label at least one or more prior concepts by marking them as either 0 or 1, indicating what the image is or is not.
   • These annotations were used to adjust the corresponding prior concept values, and the NCG (PSM) model then performed concept reasoning based on the image and user-provided interventions.
3. Evaluation: The accuracy of the model’s predictions before and after user intervention was recorded.

To evaluate the effectiveness of human intervention, we conducted a study with five human annotators who interacted with the model during test-time interventions. The results, summarized in the table below, demonstrate that human intervention significantly improved model accuracy, increasing it from 89.9% (pre-intervention) to an average of 95.5% across annotators.

From the results, we can see that human interventions significantly improved model accuracy, demonstrating that humans can understand and interact with the reasoning process through the NCG framework. This demonstrates that our model provides interpretable and intervenable insights, enabling dynamic human-AI interactions.

• R4W3: On a related note, the "intervention training method" (Secs 3.2.5 and 4.5) are especially unclear. Why would one expect to be able to intervene on a concept in a CBM baseline and maintain accuracy?

Thank you for this question. Since it involves the motivation and methodology of our paper, we would like to address it in three parts:

1. Why test-time intervention? Test-time intervention allows classification models to support user-AI interaction, enabling users to adjust predictions dynamically based on their domain knowledge.

2. Why intervention training? Intervention training ensures that the model is prepared for test-time interventions by aligning the training and testing distributions. Without this method, the model may not handle interventions effectively. For example, in our framework, intervention training fixes specific prior concepts (e.g., logits set to +5 or -5) randomly during training, forcing the model to learn reasoning effectively under intervention conditions.

3. Why include CBM baselines? Concept Bottleneck Models (CBM) also can perform interpretable and intervenable classification, which serves as a baseline for comparison. We expected that CBM would perform well on posterior concepts by correctly intervening its prior concepts when testing. Section 4.5 experiments were designed to highlight this comparison.

We hope the clarifications provided will adequately address your concern.

• R4Q1: I assume the "Bird Dataset" is CUB-200? The authors should make this clear in the paper.

Thank you for this clarification. The Bird dataset used in our experiments is a self-collected subset of ImageNet, containing 9 posterior concept labels, as described in L322–L323 and Table 1. For a more detailed overview, please refer to Section A.2 Dataset Visualization and Figure 5 in the appendix.

• R4Q2: Can the authors explain their test-time intervention training and results more clearly?

Thank you for raising this point. As described in Section 3.2.5 of the manuscript, we employ intervention training during the training phase to enable our model to perform test-time interventions effectively. Below is a concise explanation of the intervention training, test-time intervention, and experimental results:

1. Intervention Training: The intervention training method is designed to enhance the model’s reasoning ability by aligning the training and testing distributions, ensuring robustness under test-time interventions. Inspired by the principles of structural causal models (SCM), this approach simulates the effects of the $do(\cdot)$ operator during training, isolating the influence of intervened nodes by eliminating their dependence on parent nodes. Specifically, during each training iteration, a subset of prior concepts is randomly selected with an intervention rate $p = 0.15$, and their values are fixed using the formula:

where $Z_i$ is the intervened node value, $Y_i \in \{0, 1\}$ represents the corresponding multi-label value, and $v$ is an empirically determined confidence value set to $5$. The sigmoid output of $-5$ or $5$ reflects high certainty about the node’s intervention status. This process is designed to enable the model to address "What if" questions by learning the dynamics among concepts through intervention. By simulating test-time scenarios, the model is conditioned to predict posterior concepts based on both inferred and intervened prior concepts. An overview of the procedure for intervention training is detailed in Algorithm 2 (Appendix).

1. Test-time Intervention: After models trained with intervention training method, we propose a test-time intervention experiment to explore how human interaction with prior concepts influences the model’s performance. First, prior concepts have been arranged in topological order as described in Sections 3.1.1. Prior concepts (Z) are divided into 25 segments, and interventions on these segements are applied incrementally, enabling us to observe changes in posterior concept accuracy. The intervention values follow the same process as the intervention training phase (+5 / -5).

2. Results: Figure 4 shows that models trained with intervention methods achieve significant accuracy improvements (e.g., up to 95% top-1 accuracy). Models without intervention training, however, struggle to reason effectively under intervention scenarios. CBM performance demonstrates the importance of causal structures in reasoning, while our NCG results highlight the robust interaction capabilities under test-time interventions.

We hope this explanation clarifies the training and testing dynamics. If additional clarification is needed, we would be happy to provide it. Thanks.

I thank the author's for their comments and encourage them to include the added details in the manuscript. I have updated my score from 5 to 6.

• R3Q2: Did you add multi-lables for each datapoint by using WordNet? If so, then all images of e.g. Bird1 have all the same concept labels in every example you train on? Isn't that just making the classification problem more convoluted? Surely each image of Bird1 should have varying concept labels to truly benefit the training process and your method? Otherwise it's just the same classification problem, just with more more labels.

Thank you for this question. Do you mean that the top-level prior concept in the causal graph is shared as a concept label by all examples (e.g., "animal" in the Bird dataset as shown in Figure 5)? We believe it is better to label it as "1" than not because all examples in the Bird dataset actually belong to the concept "animal" in the real world, and thus makes it can correctly influences the subsequent nodes in the graph.

Moreover, as you mentioned, we assign multi-labels to each example using WordNet as explained in Section 3.1.2. Examples with different labels will take their corresponding ancestor nodes as prior concept labels, resulting in different sets of concept labels and can really benefit the NCG training process. Conversely, examples with the same labels will share the same prior concept labels. This approach enables the model to learn additional information about the relationships between prior and posterior concepts, rather than simply adding redundant labels. We hope these explanations could address your concerns.

• R3Q3: Can you make explicit how you think test time intervention could be used in application?

Thank you for your insightful question. We believe that test-time intervention could be particularly valuable in applications requiring human-AI interaction. For instance, beyond providing classification results, users may also need post-hoc interpretability to understand why a certain prediction was made. Moreover, users want to leverage their own expert knowledge to intervene in the intermediate reasoning process, enabling the model to generate more accurate and reasonable outputs. For example, a potential application is in medical diagnosis based on patient records, where complex causal relationships between symptoms and diseases inform predictions. It is often challenging for doctors to notice and accurately identify all relevant symptoms and diseases. Our NCG method have potential to facilitate interaction between doctors and the model, allowing predictions about symptoms and diseases while deepening doctors’ understanding of patients. To improve diagnostic accuracy, doctors could inject their domain knowledge into the model during the intervention process to refine its reasoning.

To further illustrate the application of test-time intervention, we conducted an experiment involving real human annotators, as detailed in our response to R4W2. This experiment demonstrates that humans can effectively understand and interact with the reasoning process through the NCG framework. We hope this addresses your concerns about the practical application of our method.

For more details on the technical implementation and meaning of test-time interventions, please refer to Appendix A.4.1 (Intervention Illustration) and Figure 7 in our paper.

Thank you again for your thoughtful question!

• R3Q4: Can you run your method NCG without the causal structure to understand how much benefit is attributed to the causal graph itself?

Thank you for this important suggestion. Understanding the contribution of the causal graph is indeed crucial, and we appreciate your emphasis on this point. In our manuscript, we have evaluated two variants of our model without the causal structure to examine its impact:

1. Multi-label model: As described in Section 4.2 and Table 2, this variant uses multi-labels (prior and posterior concepts) but lacks the causal graph. It performs worse than the full NCG model, indicating the importance of the causal graph for improved performance.
2. Zero Weight model: In Section 4.3 and Table 3, this variant disables causal weights, preventing concept nodes from propagating information. The performance of this “Zero Weight” model is also worse than NCG with PSM or DRL weights.

These results support our conclusion that the causal structure provides supplemental information for reasoning, enabling the model to achieve better performance.

To Reviewer ytQg (R3)

Thank you for your thoughtful review and positive feedback on our integration of causal reasoning into neural networks. Below, we provide detailed point-by-point responses to your comments.

• R3W1: First and foremost I found the paper pretty difficult to grasp exactly what had been done, a lot of it is heavily formalized without any intuitive explanations to facility easy reading, this is really going to limit who can read and build upon the work. ... I would really suggest re-writing the paper ...

We apologize for any difficulty in understanding our proposed method. To improve clarity, we are considering moving the formulaic details to the Appendix, which could make the paper more accessible to readers without extensive SCM research experience.

Regarding the structure of our writing, we deliberated between two approaches:

1. A traditional neural network innovation-focused structure (e.g., GNNs, CBM). While this approach might seem familiar to readers, it would fail to fully justify why SCM components (e.g., causal intervention, PSM, DRL, intervention training) are critical for enabling test-time intervention.

2. A structure aligned with causality literature, as currently adopted in our paper. Our method follows the logical progression consistent with theoretical causal literature [1] ("Introduction to Probabilities, Graphs, and Causal Models" -> "Causal Diagrams and the Identification of Causal Effects" -> "Actions, Plans, and Direct Effects,"). This provides a coherent and rigorous foundation for combining the NCG paradigm with structural causal models.

We hope our approach strikes a balance between clarity and rigor while presenting the novelty of our method. Thank you again for your constructive suggestions.

[1] Causality: Models, Reasoning, and Inference. Judea Pearl

• R3W2: The reliance on WordNet, while smart, limits the method a lot because not all concepts/relations will be in this dataset.

Thank you for your insightful comments. Since the additional causal graph enhances performance and facilitates Human-AI interaction, we consider it a feature rather than a limitation. For classification tasks where related concepts may not be available in WordNet, we can follow the Graph Construction process (Section 3.1.1) using alternative graph resources. For example, by manually constructing a causal graph based on expert knowledge, we can integrate domain-specific insights and enable dynamic interventions during the inference process, which is a significant advantage. Additionally, we plan to explore the possibility of automatically constructing causal graphs using causal discovery techniques in future work. This advancement would allow our framework to adapt to a broader range of datasets with unknown or diverse relationships, further extending its applicability.

• R3W3: The test time intervention relies on using labels, but what if the human involved doesn't know the labels? The accuracy won't be as good as you are reporting.

Thank you for raising this important point. Our current research focuses on achieving better accuracy with correct human interventions, which allows for human-AI interaction and aligns with current research like CBM. Extending the NCG paradigm to handle noisy or uncertain interventions is an promising direction for future work. This would involve designing robust methods to mitigate the impact of imperfect interventions while maintaining performance.

• R3Q1: Why did the regular classifiers in Table 2 perform worse than your method? This doesn't make sense to me as you are incorporating more constraints, did the causal reasoning actually result in better results?

We appreciate your insightful comments. In fact, the improved performance of NCG models aligns with our hypothesis, as the causal graph introduces structured information that enhances inference capabilities. Specifically, we incorporated key components based on structural causal model theory, including the causal graph, causal effect calculation, and intervention training. These components not only enable effective test-time interventions but also contribute to improved overall model performance.

In contrast to regular classifiers, which treat posterior concepts as independent, the NCG method utilizes causal relationships among concepts to improve reasoning. This structured approach enhances the model’s ability to infer and predict accurately. Our experimental results (Tables 2, 3, and 4) clearly show that the NCG method consistently outperforms regular classifiers, largely due to these causal-based designs.

• R2W3: There were confusing notational errors (e.g., multiple meanings of symbols in the equations).

Thank you for your careful attention to detail and for highlighting this issue. We have thoroughly reviewed the manuscript and addressed all notational ambiguities to ensure clarity and consistency based on your comments.

• R2Q3: What was the purpose of the experiments in Section 4.5? If the test samples were intervened and their concepts and labels were perturbed according to a causal graph, it seems natural that the accuracy of the proposed method increased and that of CBM decreased since the former was trained with the causal graph and the latter did not. Can this experiment be considered fair?

Thanks for your insightful questions. The purpose of the experiments in Section 4.5 is to test whether the NCG classification model can support user-AI interaction through test-time intervention. As discussed in our manuscript (Lines 504 - 512), all test-time intervention experiments were conducted by incrementally intervening on prior concepts, and see how the accuracy of posterior concepts change. The only difference between NCG and CBM lies in their model structures. Therefore, the comparisons are fair and consistent. The main purpose of this section is to evaluate whether NCG (with PSM/DRL) and CBM can effectively perform test-time intervention after intervention training—or even without it. We hope this explanation sufficiently clarifies the motivation and fairness of the experiment.

To Reviewer sJC6 (R2)

Thank you for your thorough review and constructive feedback. We acknowledge the areas you highlighted for improvement and are committed to addressing them in our revised manuscript to improve readability and understanding.

• R2W1 & R2Q1: The bottom-up presentation was challenging to follow; a top-down approach may make it clearer. Several unique techniques (e.g., causal effect calculation, concept reasoning, and intervention training) were unclear to me. I think this paper may be improved by organizing it following the original paper on the concept bottleneck models [Koh et al. 2020]. Is my understanding of the proposed method correct?

We apologize for any difficulty in understanding our proposed method and appreciate your thoughtful suggestion to consider a top-down structure similar to concept bottleneck models (CBM). In fact, we have introduced the abstracted top-down model in the introduction and illustrated it in Figure 1(c). However, we believe that structuring the whole paper according to the CBM approach may not effectively showcase our contributions, for the following reasons:

1. The structure of our NCG method aligns with the sequence in theoretical causality literature [1] (e.g., “Introduction to Probabilities, Graphs, and Causal Models” -> “Causal Diagrams and the Identification of Causal Effects” -> “Actions, Plans, and Direct Effects”). This structure natually integrates our NCG paradigm into the framework of structural causal models. This way of writing will make NCG more theoretically logical and make it easier for readers with SCM background to understand.
2. We respectfully disagree that our proposed method can be simply regarded as an extension of the concept bottleneck models, as it can not explain some theoritical reasonable designs including i) Causal effect calculation using PSM and DRL method; ii) Concept reasoning process as equation (6); iii) Intervention training method, which enables effective test-time interventions.

At this point, we believe that the difficulty of the paper mainly comes from the background knowledge of related techniques. Therefore, in the appendix of the revised manuscript, we will include more detailed explanations and relevant citations of these three components to better describe specific components.

[1] Causality: Models, Reasoning, and Inference. Judea Pearl

• R2W2 & R2Q2: Concerning W2, from what perspective do the authors claim that the proposed model is interpretable?

Thank you for your concern about the interpretability of this work. As you mentioned, we agree that the interpretability of the paper is important, and including the discussion and necessary details can significantly enhance the readability of the paper.

In the context of our paper, the interpretability is demonstrated in several perspective:

1. Causal Graph. As illustrated in Figure 1(c), we incorporate the causal graph with concepts into the classification process, which introduces interpretability by constructing causal relationships betweeen originally independent posterior concepts and causally related prior concepts.
2. Reasoning Process. As illustrated in A.5 Case Study (Figure 9), our NCG method can provide post-hoc interpretability to see how the reasoning process leads to certain predictions for posterior concepts, which makes the reasoning process more transpanrency. This enhances the transparency of the model, aligning with existing research on GNN interpretability [2].
3. Intervention Process. Our NCG models not only can perform post-hoc interpretability, but also allows for "intervenable interpreatability" that users can interactively guide the model's reasoning process during testing.

Thanks for your suggestions. We have summarized and added these discussions to the Sections 4.5 in revised manuscript to enhance clarity and understanding:

In conclusion, this experiment demonstrates that our NCG method provides post-hoc interpretability, allowing users to understand how the reasoning process leads to specific predictions for posterior concepts. This aligns with existing research on interpretability in graph neural networks [2]. Additionally, our method extends beyond post-hoc analysis by enabling users to actively influence the model’s reasoning during testing through test-time interventions.

[2] Explainability in Graph Neural Networks: A Taxonomic Survey.

• R1Q3: Regarding your explanation of the accuracy drop for DRL+CLIP w/o intervention training: are you saying that the method reaches a point (high number of nodes) after which it can’t learn the underlying structure (w/o intervention training) and therefore reason appropriately (“get the right answer”)? Are there metrics other than accuracy that could demonstrate the difference in underlying reasoning (even just at the point on the graph, ~1300 nodes for CLIP+DRL, between the two graphs) and make your claim stronger?

Thank you for this valuable observation. The accuracy drop after around 1300 nodes likely arises because, without intervention training, the model can not reliably learn causal dependencies of graph, particularly as the number of nodes grows. It took us a bit longer than expected to carefully examine the experiments with DRL+CLIP by gathering all the necessary details, and there are three main reasons that, in an unexpected combination, led to this phenomenon when the model lacks interventional training. Note that our proposed NCG models with intervention training do not suffer from this issue, showing the necessity of our proposed intervention training process.

1. Positive Inductive Bias in the Causal Graph: The model inherently interprets positive logits in the formula $C_j = \phi(g(h(X))\_{j} + \sum w_{ij} \cdot C\_i + U\_j)$ as an indication of the presence of certain concepts in the images. This is a reasonable assumption given the structure of our causal graph, which has an inductive bias toward positive causal relationships. As detailed in Section 3.2.1 "Causal Effect Calculation", the causal effect weights are typically estimated with 1 representing the presence of a concept and 0 representing its absence. This means that the estimated causal edges generally have positive weights, leading to an increase in the logits of child nodes when the logits of the corresponding parent concepts increase.

2. Negative Causal Weights in DRL Estimation: Although most of the estimated edge weights are positive, there are few exceptions where the causal weights are negative. This occurs because the DRL method uses a regression-based estimator (Lines 240-258, Equation (4) and (5)) to calculate causal effects, and in some cases, this results in a negative weight. When these negative weights exist, to increase the logits of their child nodes, the model must reduce the logits of parent concepts.

3. Intervention on the Last 100 Prior Concepts Close to Posterior Concepts: We observed that the last 100 prior concepts in the causal graph are most closely related to the posterior concepts within one-hop. During the intervention, we used values of -5 and 5 to manipulate the prior concepts (Section 4.5 Test-time Intervention). In extreme cases where the causal weights are negative, applying a -5 intervention to the prior concepts was mistakenly interpreted by the model as an increase in the corresponding posterior concept logits. This misinterpretation possibly leads to wrong classification results, thus causing a sudden and significant drop in accuracy.

These factors combined to produce the drastic drop in model performance after the initial 1300 effective node interventions. Note that this situation arises because the model did not perform intervention training, and the fact that the model was able to improve performance on the first 1300 nodes is already a noteworthy result. Conversely, models that have undergone proper intervention training do not suffer from this issue, demonstrating the effectiveness of our approach. Thank you very much for pointing out this problem. Your feedback provides valuable insights that led to a more thorough investigation. We will include these detailed explanations in the appendix of the revised paper.

• R1Q2: You are space constrained, but since the intervention training method is an important contribution, I wonder if it could get a little more space in the main text.

Thanks for your recognition of the importance of our proposed intervention training method, we have expanded the intervention training mechanism in revised paper, outlining the methodology more explicitly.

The intervention training method is designed to enhance the model’s reasoning ability by aligning the training and testing distributions, ensuring robustness under test-time interventions. Inspired by the principles of structural causal models (SCM), this approach simulates the effects of the $do(\cdot)$ operator during training, isolating the influence of intervened nodes by eliminating their dependence on parent nodes. Specifically, during each training iteration, a subset of prior concepts is randomly selected with an intervention rate $p = 0.15$, and their values are fixed using the formula:

where $Z_i$ is the intervened node value, $Y_i \in \{0, 1\}$ represents the corresponding multi-label value, and $v$ is an empirically determined confidence value set to $5$. The sigmoid output of $-5$ or $5$ reflects high certainty about the node’s intervention status. This process is designed to enable the model to address "What if" questions by learning the dynamics among concepts through intervention. By simulating test-time scenarios, the model is conditioned to predict posterior concepts based on both inferred and intervened prior concepts. An overview of the procedure for intervention training is detailed in Algorithm 2 (Appendix).

To Reviewer bMLv (R1)

• R1W1: I have no major issues with this paper.

We sincerely thank you for your positive feedback and appreciation of the strengths of our work. We are pleased to know that you found the introduction clear, the contributions well-articulated, and the experiments convincing. Your recognition of the importance of our ablation studies in demonstrating key aspects of our methodology is particularly encouraging, and we greatly appreciate your specific suggestions regarding our intervention training method to enhance the important contributions of the manuscript. We are committed to ensuring that the final manuscript maintains this clarity and rigor. Thank you again for your supportive and constructive review!

• R1W2: Small typo: 310/311 intervention is spelled wrong.

We appreciate your attention to detail! We have revised the spelling of "intervention" on Lines 310/311 according to your feedback.

• R1W3: Suggested clarification on the causal inference background, specifically to include prior work integrating causal inference in neural networks, especially in GNNs.

Thank you for this valuable suggestion. We acknowledge that our initial description of causality-driven GNNs could be improved. To address this, we have revised the manuscript to clarify our contributions and to better highlight how our work differs from previous studies.

1. Our research mainly aims to enhance interpretable and intervenable classification capability of neural networks. However, most of causality-driven GNNs are not designed for this purpose.
2. There are three types of previous research regarding the causal learning on graphs: 1) Causal reasoning; 2) Causal discovery; 3) Causal representation learning. These tasks mainly focus on interpretability of data by quantifying cause-effect relationships among variables or recovering the causal graph from an observed dataset, rather than using the reasoning ability of an structural causal model for intervenable classification.
3. To address your comments, we have revised the manuscript and added clarification in the Introduction as follows:

Furthermore, causality-driven GNNs [1, 2] focus on interpreting data by quantifying cause-effect relationships among variables or recovering the causal graph from observed datasets. However, these models lack the capability for interactive intervention and do not support interactive classification involving human users. To address these challenges, we propose the Neural Causal Graph (NCG), a novel framework that enable intervenable classification by integrating causal inference within neural networks architectures. Unlike conventional methods that can only perform post-hoc interpretable predictions, NCG constructs a causal graph that models the underlying mechanisms of the data, enabling the model not only can reason by following the causal relationships between labels, but also supports intervenable classification within reasoning process.

We hope the clarifications provided will adequately address your concern.

[1] When Graph Neural Network Meets Causality: Opportunities, Methodologies and An Outlook

[2] Exploring Causal Learning through Graph Neural Networks: An In-depth Review

• R1Q1: Is there anything systematic that you can comment on or hypothesize about in Section 4.2 about DRL doing better on Bird vs. PSM doing better on ImageNet performance in Table 2?

Thanks for bringing up this specific experimental phenomenon to our attention. The observed performance difference likely stems from differences in weight estimation strategies used by the PSM and DRL methods. DRL performs better on the smaller Bird dataset, while PSM shows superior results on the larger ImageNet dataset. This observation aligns with previous findings [3] that PSM tends to estimate causal effects more accurately when the treatment and control groups are larger. In fact, since some of the performance difference between PSM and DRL on each dataset are relatively small, it is hard to concrete hypothesize based on this uncertain phenomenon.

[3] Matching methods for causal inference: A review and a look forward.