A Recipe for Causal Graph Regression: Confounding Effects Revisited

We sincerely thank Reviewer iAdk for the useful feedback. Detailed responses are provided below. We kindly ask the reviewer to reconsider their score if the following clarifications resolve the concerns.

Q1. Why are gains on GOOD-ZINC larger than on ReactionOOD?

GOOD-ZINC is large-scale and has diverse OOD testing samples, thus more suitable for causal discovery (which naturally requires more samples than ERM). In contrast, ReactionOOD datasets are smaller and the OOD samples are more specialized. Our method shows larger gains on GOOD-ZINC because our recipe more effectively handles diverse distributions, while baselines struggle with such complexity.

Details can be seen in the response to [Q2 & Claims And Evidence] from Reviewer jru1.

W1. Overly narrow scope of causal graph regression

We respectfully disagree, and humbly refer the reviewer to our response to [Broader Sci. Literature] from Reviewer HaWt.

W2. Method motivation

We humbly refer the reviewer to the response to [Methods] below.

[Claims And Evidence] Claim 2 (restated below) lacks support

Claim 2: The reason that existing CGL methods don't work well for regression tasks stems from an assumption that confounding subgraphs contain strictly no predictive power.

We respectfully clarify a misunderstanding regarding Claim 2. In our paper, we never claim that existing CGL methods fail in regression tasks due to the assumption that confounding subgraphs contain strictly no predictive power.

Rather, a similar statement serves as part of our motivation (instead of claim): prior methods (e.g., CAL [1]) often assume that confounding subgraphs are non-predictive, which does not hold in practice (both in classification and regression settings). We provide empirical evidence supporting this motivation in Section 5.5 (Lines 385–397, Figure 3 left), showing that completely ignoring the predictive signals of confounders can lead to performance degradation.

[Methods]

• The motivation for using contrastive learning (CL) in regression is unclear.

We respectfully clarify the motivation for using CL. Existing CGL methods (e.g., [1]) proposed to align the representation of intervened graphs (constructed by randomly pairing confounding subgraphs with target causal subgraphs [2]) through the labels of causal subgraphs, and due to the explicit usage of labels they cannot be directly applied to regression tasks.

From this perspective, CL is a perfect fit for causal graph regression, since the InfoNCE loss in CL aims at "perfect alignment" [3] without using labels, pulling positive sample representations (intervened graphs with the same causal subgraphs in our case) closer while pushing negative pairs (intervened graphs with distinct causal subgraphs) apart, which thus distinguishes causal effects between factual and counterfactual samples. This design enables models to implicitly capture causal signals without label supervision, consistent with recent advances in causal representation learning [4].

• Is the modeling of $I(C;Y), I(C;G)$ different from those in existing work.

The modeling of $I(C;Y), I(C;G)$ in this paper does differ.

• Prior works on causal graph learning derive GIB-based objectives under classification settings.
• In contrast, we provide a new formulation of the GIB loss under Gaussian assumptions specific to graph regression, allowing principled disentanglement of causal and confounding subgraphs under continuous targets.

[Theoretical Claims] Assumption of identity covariance

We humbly refer the reviewer to our response to W2 of Reviewer HaWt.

[Supp. Material] Ablation studies

We take your suggestion and re-perform the ablation studies.

Ablation 1 - Effectiveness Analysis: We evaluate the performance of our model variants across four OOD datasets. Each variant removes several proposed modules (the same setting as in Appendix A.5, Lines 597-627).

Due to varying dataset complexities, the contributions of GIB and CI differ across tasks. Nonetheless, the full model consistently achieves the best performance, indicating the two loss functions work synergistically to enhance OOD generalization. (More details can be found in the attached figure.)

Ablation 2 - Parameter Sensitivity Analysis: We humbly refer the reviewer to our response to [Experimental Designs] from Reviewer HaWt.

[Reference]

Thank you for catching this oversight. We will definitely cite the reference when introducing GIB in Sec. 3.2 and acknowledge its influence.

---

[1] Causal attention for interpretable and generalizable graph classification. KDD, 2022.
[2] Debiasing graph neural networks via learning disentangled causal substructure. NeurIPS, 2022.
[3] Understanding contrastive representation learning through alignment and uniformity on the hypersphere. ICML, 2022.
[4] Robust causal graph representation learning against confounding effects. AAAI, 2023.

We sincerely thank Reviewer jru1 for the thoughtful and encouraging feedback. We appreciate your recognition of our method’s motivation, design, and empirical support. Below we address each of your comments in detail.

Q1. Explain the fundamental difference between graph classification and regression, and why regression is so challenging.

The core difference lies in the adoptation of causal discovery, as noted in Lines 057–060, where we state that existing methods "rely on discrete label information and cannot be adapted to regression."

In regression, the absence of discrete labels naturally makes confounder separation harder. For example, in molecular property prediction, continuous outcomes (e.g., solubility values) often arise from overlapping substructures, complicating causal identification.

[Q2 & Claims And Evidence] Performance Gaps on ReactionOOD Datasets

We appreciate this observation. The relatively weaker performance is attributable to known challenges rather than methodological flaws:

• Our method stems from more general and weaker assumptions, which naturally implies larger function space and higher requirements on sample size:
  Unlike prior methods such as IRM or DIR, our method do not assume that spurious features are non-predictive. This allows us to model a broader function class under weaker assumptions, making our method applicable to a wider range of real-world distribution shifts. The trade-off is increased optimization difficulty under limited data or high spurious signal, but this reflects a principled design choice, not a methodological weakness.
• No method dominates across all OOD benchmarks:
  As shown in OOD-GNN [1], it is a regular phenomenon in generalization tasks that no approach consistently performs best on every dataset due to varying distribution shifts and inductive biases. Despite this, our method is among the most stable across diverse settings and achieves best results on ZINC (Table 1), the largest and most complex dataset.
• Spurious correlations affect ID performance:
  Non-causal baseline methods (e.g., ERM, CORAL, and DANN) often perform better under ID or mild OOD due to exploiting spurious but predictive features. To handle OOD settings, our method instead recognizes those spurious features and removes them for better generalization, which may slightly reduce ID performance but improves robustness.

  For example, in Cycloaddition-ID（Total Atom Number, concept), CORAL achieves 4.10 vs. ours 5.74, but under OOD, CORAL drops to 5.74 while ours remains stable at 5.53 (the smaller the better).

• Causal and contrastive learning require more data:
  Causal inference is statistically harder than correlation-based methods and generally requires more data to reduce variance [2]. Contrastive learning also benefits from large batch sizes and diverse representations. This explains why our method performs best on ZINC. On smaller datasets (e.g., RDB7), all causal methods degrade; however, our method remains stable and consistently outperforms CIGA (the best method of the listed causal intervention baselines) across all RDB7 settings.

We will modify the corresponding experimental analysis.

[Typo] Correct typo in Section 5.2 title and line 393.

We appreciate your attention to detail. We will accordingly revise the manuscript to reflect your suggestion.

---

[1] Tajwar F et al. No true state-of-the-art? OOD detection methods are inconsistent across datasets. arXiv, 2021.
[2] Guo R et al. A survey of learning causality with data: Problems and methods. ACM Comput. Surv., 2020.

We sincerely thank Reviewer HaWt for the thorough and constructive feedback. We have carefully addressed the raised concerns and suggestions. We kindly ask the reviewer to reconsider their score if the following points resolve the reservations with our work, or to provide more pointers for us to address.

W1. How to obtain the causal subgraph $C$ from graph $G$, i.e., how to obtain the mask matrices $M_{edge}$ and $M_{node}$.

We appreciate the reviewer's question. The construction of the causal subgraph is introduced in the main text (Eq. (1)), and the details of generating the mask matrices are in Appendix B (Lines 647–692).

In the revision, we will briefly summarize the mask generation process in the main text.

W2. The assumptions in Section 4.2 remain unverified. Are they reasonable?

Thank you for raising this point. We go through and clarify each assumption below:

• Assumption 1 (Gaussian assumption of $p(C \mid G)$ and $q(C)$): We follow the variational information bottleneck literature [4], modeling both as multivariate Gaussians to enable tractable KL estimation. This is widely adopted in similar works [5,6].
• Assumption 2 (Covariance $\Sigma_\phi(G) = I$): Setting the covariance to identity simplifies optimization and is theoretically justified in [4, Appendix A], which shows that any full-rank covariance can be whitened without loss of generality.
• Assumption 3 (Gaussian posterior): For $P(Y \mid H_c)=\mathcal{N}(Y; \mu_{(c)}, \sigma^2_{(c)})$, we use a fixed variance for stability, similar to Phase I assumption in section 1 of [7]. This also allows a closed-form expression for mutual information estimation (e.g., [6], Sec.2.1, Proposition 1).

We will clarify these assumptions more explicitly and emphasize their empirical validity (the strong OOD generalization results in Section 5) in the revision.

W3. There is no obvious superiority of the OOD performance compared with the ID performance.

We respectfully clarify that the goal of CGL methods might be misunderstood; we do not aim for OOD performance to surpass ID performance, which is unrealistic.

We focus our attention on the robustness under OOD settings, demonstrating strong generalization ablity of our method. For more analysis of OOD performance, we humbly refer the reviewer to the response to Q2 from Reviewer jru1.

[Experimental Designs] The effect of the hyperparameters $\alpha$ and $\beta$

We thank the reviewer for this suggestion. Some discussion related to this point has already been included in Appendix A.5. To better illustrate, we rerun the experiments and will update the sensitivity analysis for $\alpha$, $\beta$, and $\lambda$. (More details can be find in the attached figure.)

The results show that the choice of $\alpha$ has no clear effect, while the choices of $\beta$ and $\lambda$ significantly impact OOD performance.

[Broader Sci. Literature] Causal graph regression (CGR) seems to be a relatively neglected domain, and the authors also do not provide any important applications of CGR

We appreciate this review and will revise the introduction to better highlight the motivating applications and clarify the contribution as improving generalization in graph-based prediction tasks through causal mechanisms.

Furthermore, we beg to clarify

• CGR is not a niche topic but a core component of the broader field of causal graph learning (CGL). As noted in Section 1 (lines 36-46), CGL has emerged as a key paradigm for building robust models. Our work leverages causal interventions to improve OOD generalization in graph regression, which is essential for many real-world scenarios with distributional shifts.
• We have discussed the application of CGR in Section 1 (lines 48-51). Moreover, CGR is relevant to tasks such as molecular property prediction [1], traffic flow forecasting [2], and credit scoring [3].

---

[1] Rollins Z A. et al. MolPROP: Molecular Property prediction with multimodal language and graph fusion. J. Cheminform., 2024.
[2] Li G. et al. Multistep traffic forecasting by dynamic graph convolution: Interpretations of real-time spatial correlations. Transp. Res. Part C Emerg. Technol., 2021.
[3] Ma F. et al. Utilizing Reinforcement Learning and Causal Graph Networks to Address the Intricate Dynamics in Financial Risk Prediction. Int. J. Inf. Technol. Syst. Approach, 2024.
[4] Chechik G. et al. Information bottleneck for Gaussian variables. Adv. Neural Inf. Process. Syst., 2003.
[5] Kingma D P, Welling M. Auto-encoding variational bayes[EB/OL].(2013-12-20)
[6] Yu S. et al. Cauchy-Schwarz Divergence Information Bottleneck for Regression. arXiv, 2024.
[7] Nix D. et al. Learning local error bars for nonlinear regression. Adv. Neural Inf. Process. Syst., 1994.