Dear Reviewer #ii3V,

We sincerely appreciate your constructive comments to improve our paper. You may find our corresponding explanations below for your concerns. We have uploaded the revised paper (with track changes marked in blue). We would really appreciate it if you could let us know if you have any further concerns or suggestions.

In the revised manuscript, we include discussions on four related works (listed in Appendix C):

• Bhattacharya et al., (2020): Under interference and network uncertainty, this paper proposes methods that integrate structure learning and causal inference techniques to estimate causal effects. Their algorithms rely on structure learning to first learn graphical models and then estimate the Population Average Overall Effect (PAOE), a variant of the Average Total Effect. They address network uncertainty under partial interference through structure learning and then use the auto-g-computation algorithm to estimate the PAOE. This method rely on the assumption that the true data-generating process corresponds perfectly (satisfying Markov and faithfulness conditions) to some unknown chain graph, with two restrictions. This paper does not release its code.
• Savje et al., (2021): This paper study average treatment effects estimation in randomized experiments with unknown and arbitrary interference among units. The authors propose the Expected Average Treatment Effect (EATE) as a generalized estimand that incorporates potential spillover effects by marginalizing over all possible treatment assignments. While their study provides a theoretical and experimental analysis using existing methods, it does not introduce new algorithms to address unknown interference. Furthermore, their analysis is based on randomized controlled trials (RCTs). In contrast, our paper focuses on addressing the challenge of unknown interference on observational data.
• Cortez et al. (2022): Under unknown network structures, Cortez et al. (2022) extend the work of Yu et al. (2022) by generalizing causal inference methods from linear to polynomial models, enabling the estimation of total treatment effects (TTE) under more complex and non-linear network interactions. They introduce a staggered rollout experimental design, which incrementally applies treatments and collects observations, eliminating the need for prior knowledge of the network structure. However, a limitation of their approach is its reliance on multiple randomized trials (i.e., the staggered rollout design) to gather observations at different stages.
• Lin et al., (2024): Under unknown interference, the UNITE algorithm uses a Graph Structure Learner (GSL) to infer the hidden interference structure by constructing a complete graph and imposing L0-norm regularization to identify significant connections. With the inferred structure, Graph Convolutional Networks (GCNs) are employed to learn an aggregation function and model the interference from neighboring units, effectively capturing the interference patterns and enabling accurate treatment effect estimation.

(Bhattacharya et al., 2020) Rohit Bhattacharya, Daniel Malinsky, and Ilya Shpitser. Causal inference under interference and network uncertainty. In Uncertainty in Artificial Intelligence, pp. 1028–1038. PMLR, 2020.

(Savje et al., 2021) Fredrik Savje, Peter Aronow, and Michael Hudgens. Average treatment effects in the presence of unknown interference. Annals of statistics, 49(2):673, 2021.

(Cortez et al., 2022) Mayleen Cortez, Matthew Eichhorn, and Christina Yu. Staggered rollout designs enable causal inference under interference without network knowledge. Advances in Neural Information Processing Systems, 35:7437–7449, 2022.

(Lin et al., 2024) Xiaofeng Lin, Guoxi Zhang, Xiaotian Lu, and Hisashi Kashima. Treatment effect estimation under unknown interference. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp. 28–42. Springer, 2024.

[W4] For experiments, authors need to compare with the existing methods for unknown interference. In the current version, authors did not compare with [R1-R4]. The authors can add [R1], [R3] and [R4] as baselines in their experiments, following the implementation details. [R1] can create a network (you can only take the edges among different nodes) when the input graph is a complete graph or an initial graph, i.e., A in the authors' setting. Then, you can apply G-HSIC on the network created by [R1] as a baseline. [R3] and [R4] can directly work when you give or not give an input graph. [R2] can be ignored, as it is not a learning method.

Response: We appreciate the reviewer’s suggestion to incorporate comparisons with the existing methods for unknown interference. However, Savje et al. (2021) and Hoshino et al. (2022) rely on randomized controlled trials (RCTs), and thus their approaches are not applicable to our observational network data. While Bhattacharya et al. (2020) proposes constructing a network from a complete or initial graph, the lack of released code poses challenges for reproducibility. Furthermore, structure learning algorithms often face implementation challenges and produce unreliable results due to the large graph sizes and unmet assumptions in real-world data. The most relevant work is [Lin et al., 2024], which addresses unknown interference graphs and learnable aggregation functions through their framework, UNITE, using L0-norm regularization and GCNs for accurate treatment effect estimation. We include comparisons with UNITE in our experiments. We have added the comparison experiments in Table 3 in main text and in Table 10 in Appendix E.7 (with track changes marked in blue).

We use the original implementation of UNITE provided by [Lin et al., 2024] and observe that it only supports the estimation of AME and IME. While UNITE demonstrates a slight improvement over our method in terms of the Average Main Effect (AME), its performance drops significantly at the individual level. In contrast, our method consistently performs strongly across all six metrics, highlighting its comprehensive and balanced ability to handle both average and individual-level effects effectively.

[W5] In definitions 1 and 3, as well as equations (1) and (2), should the t in the right of '=' be 1, as the treatment is a binary value, i.e., 1 or 0.

Response: Thank you for pointing out these typos. We will revise them accordingly.

Thanks for your detailed advice and comments. We welcome any further technical advice or questions on this work and we will do our best to address your concerns.

[W3] In addition, existing methods [R1-R4] have a more relaxed assumption for unknown interference. They do not require any network information, whereas the proposed method requires network information, i.e., A. [R1-R4] can work when the network information is totally unknown. Can the proposed method work when the network information is totally unknown? How does the performance change in this case?

Response: Thanks for your comment. Our CauGramer can work when the network information is totally unknown. In such cases, CauGramer can use any known connected graph (even one that is randomly generated) as a substitute for the social network to aggregate interference information. As demonstrated in Figure 3, CauGramer remains robust even when the provided network is entirely unrelated to the 'Random' interference graph. It still learns causal effects effectively and outperforms existing methods, suggesting that any connected graph—even one that is randomly generated and independent of the interference graph—can help aggregate interference information. Exploring how to construct an efficient connected graph would be an interesting directions, but it is beyond the scope of this paper and is left for future work. In Appendix B, we added a new section 'Limitations and Future Directions' to further clarify this.

Although [R1-R4] can work when the network information is totally unknown, Savje et al. (2021) and Hoshino et al. (2022) rely on randomized controlled trials (RCTs). Lin et al. (2024) supports only the estimation of AME and IME, limiting its applicability. Bhattacharya et al. (2020) assume that the true data-generating process perfectly aligns with (satisfying Markov and faithfulness conditions) an unknown chain graph under two strict restrictions. However, in practical scenarios, structure learning algorithms often encounter implementation challenges and yield unreliable results due to the graph’s size and unmet assumptions.

In observational network data, we focus on real-world scenarios where individuals closely connected in a social network are more likely to interfere with each other. Based on this observation, we proposed the CauGramer algorithm to infer unknown interference information using a known social network. Social networks are typically easier to obtain than true interference graphs, as they can often be derived from sources such as co-authorship networks, public community relationships, organizational affiliations, or online interactions like comments and messages. As a result, most existing studies also tend to rely on social networks for research purposes.

Dear Reviewer #ii3V,

Thank you for your further questions. In your initial comments under weaknesses, four points focused on our lack of discussion and comparison with [R1–R4], while one for typos. However, [R3] had already been included in Appendix D of our original manuscript. In response to your suggestions, we have expanded our discussion of these four works in Appendix C, which has significantly improved our paper. We are sincerely grateful for bringing [R1, R2, R4] to our attention.

[It is not accurate to state that “all studies on unknown interference were ignored in the first submission”] There may be a misunderstanding. This paper is related to seven domains. In our original submission, due to the limited space, we only listed representative algorithms (which served as comparison baselines in the Experiments section) in the Related Work section, and deferred discussions on “Partial Interference”, “Exposure Mapping”, and “Unknown Network Structure” to Appendix D, titled “Supplementary Related Work in Networked Interference” in original manuscript. In Unknown Network Structure, we discussed contributions by Eckles et al. (2017), Yu et al. (2022), and Cortez et al. (2022), with Cortez et al. (2022) corresponding to your mentioned [R3].

We appreciate your insightful comment, which highlighted the need for better organization in our Related Work Section. In response, we have restructured this section and expanded our discussion of the four related works in Appendix C. While we acknowledge that some studies on unknown interference were overlooked, it is not accurate to state that “all studies on unknown interference were ignored in the first submission”.

[Although unknown interference is not a new issue, it has not been well-addressed by [R1–R4]. As a response to your [W2]:

• [R1] assumes the true data-generating process aligns perfectly with an unknown chain graph (satisfying Markov and faithfulness conditions), but this approach comes with two restrictions. In practical scenarios, structure learning algorithms often face challenges such as implementation issues and unreliable results due to the graph’s size and unmet assumptions.
• [R2] and [R3] rely on randomized controlled trials (RCTs).
• [R4] introduces the UNITE framework, which employs L0-norm regularization and GCNs for treatment effect estimation but only supports the estimation of AME and IME.

We hope this clarifies our position in related work.

[It is not accurate to claim that “the proposed method only combines many existing methods”]. We would like to emphasize the following:

1. Our paper proposes a novel Causal Graph Transformer (CauGramer) with cross-attention. Unlike existing methods that use GNNs solely for information aggregation and feed learned embeddings into a transformer, our approach fundamentally differs in structure. In CauGramer, the Q, K, and V modules have distinct receptive fields, and the input is no longer limited to self-information, combining the strengths of GNNs in a new architecture.
2. Our CauGramer framework relaxes the assumption that peers’ treatments are independent, allowing peer treatments to influence others’ treatments. In this setting, the balanced representations employed by traditional methods are limited to identifying the main effects of self-treatment and cannot capture peer effects. To address this limitation, we propose a novel transformation formula (lines 345–360) to address this challenge.
3. We construct conditional moment constraints to ensure residuals approach zero given $x$ and $t$, thereby mitigating model misspecification issue. In the presence of unmeasured confounders, we reformulate Eq. (11) as Eq. (16) by leveraging a negative control variable to identify causal effects. While Eq. (16) is not our novel contribution, Eq. (11) is proposed by us and is not derived from Shi et al. (2020) or Kallus et al. (2021).
4. In our work, we provide a causal identification theorem (Theorem 1) and an interference proposition (Proposition 1), which guide the model in capturing interference information and learning balanced representations for causal effect estimation. Furthermore, in Appendix D.4, we include Theorem 2, which utilizes negative controls to identify causal effects in the presence of unmeasured confounders. We hope these contributions establish a theoretical foundation for our method.

[Regarding the comparison baselines [R1]]:

1. The assumptions required for structure learning in [R1] do not hold in our setting.
2. The G-HSIC, mentioned in your [W4], is already included as one of our baselines.
3. [R1], preprint in 2019 and published in UAI 2020, is no longer the state-of-the-art.
4. [R1] did not release code, which poses challenges for reproducibility.

Therefore, [R1] was not included in our experiments.

Thank you so much for your time and efforts.

Authors of Paper #2331

Dear Reviewer #ii3V,

We sincerely thank you for your valuable suggestions, which have significantly improved our paper, especially the Related Work Section. We apologize for the confusion regarding the related work in our original manuscript. Due to limited space, in our first submission, existing studies (including [R3]) on unknown interference were previously placed in the Appendix. Based on your comments, we have reorganized the Related Work Section to discuss these studies in the main text. Additionally, we also provided detailed descriptions of these works in Appendix C. We deeply appreciate your insightful feedback and have incorporated these changes and revised typos in our revised manuscript.

[Question] Why can [R4] only be used for AME and IME? It seems the differences between IME and ITE/IPE are only the inputs of the network that predict $Y^1$ and $Y^0$. Changing inputs of the existing method is not a challenge.

Response: Thanks for your question. While changing inputs of the existing method is not a challenge, UNITE [R4] uses all unit features $X$ and treatments $T$ to learn a Graph Structure Learner (GSL). It then employs two learned outcome predictors to predict treated and control outcomes. If the inputs are changed, the learned graph structure and its associated weights may also change, leading to potential additional bias in the fixed outcome predictors without further assumptions. This limitation can result in unreliable results. Therefore, in the original UNITE [R4] paper, the authors focus on average treatment effects and individual treatment effects, but do not conduct experiments or provide discussions on peer effects.

To address your concern, we conducted additional experiments by modifying the inputs of UNITE to estimate various causal effects (AME, APE, ATE, IME, IPE, ITE). The results are as follows:

These results show that UNITE struggles with peer effects (APE/IPE) and total effects (ATE/ITE) due to changes in the graph structure and its weights, which likely render the fixed outcome predictors ineffective. Consistent with the original UNITE [R4] paper, which focuses on average treatment effects and individual treatment effects, we report only AME and IME for UNITE in the main text. We include the above discussion about the experiments on UNITE in Appendix E.7.

We deeply appreciate your time and effort in reviewing our paper and your constructive feedback.

Authors of Paper #2331

[Q1] In the COVID case study, where transmission primarily occurs through contact, how can interference occur in the absence of a direct link? Could you clarify potential mechanisms of transmission under such conditions?

Response: Thank you for your question. In the COVID case study, while transmission primarily occurs through contact, it is not strictly limited to direct social connections. As noted in Lines 177–179 of the problem setup section, even individuals with no pre-existing social ties within the network may encounter each other during daily activities, such as in public spaces, workplaces, or other shared environments, leading to potential COVID transmission. This highlights that the interference graph reflects possible transmission pathways, which are not restricted to the social network structure. Therefore, the interference graph is not the same as the social network.

[Q2] Could you clarify why the final output ( $r_x^{(L)}$ ) is used as the input to the first layer of the peer-treatment representation network, specifically ( $r_t^{(1)} = r_x^{(L)}$ )?

Response: Thanks for your question. $r_x^{(L)}$ is not just an aggregation of peer feature information but also embeds the information of the node itself. It serves as a high-quality embedding that combines the node’s own features with its interference information. Conceptually, we can treat $r_x^{(L)}$ as capturing the behavioral characteristics of an individual and their interactions with peers. These characteristics may determine how and whether the individual is influenced by the actions of their connected nodes. By using $r_x^{(L)}$ as the input to the peer-treatment representation network, we improve the efficiency of information aggregation.

[Q3] Please explain the statement: "we simulate unmeasured confounding by removing features."

Response: Thank you for your comment. In the simulated observational data, the treatment and outcomes are generated based on all covariates, which act as confounders in the treatment effect estimation. To simulate a setting with unmeasured confounders, we keep the generated treatment and outcomes unchanged but randomly remove or mask two features from the covariates. These features remain causal factors for both the treatment and the outcome; however, their information is intentionally excluded during the model training process, effectively simulating the presence of unmeasured confounding. This is a commonly used approach to test a model’s sensitivity to missing variables and assess its robustness in such scenarios.

[Q4] The paper states that the relationship between peer-treatment ( $t_{Pi}$ ) and outcome ( $y_i$ ) remains confounded by the common causes ( $r_i$ ), leading to biased estimates of peer and total effects. Moreover, the peer-treatment ( $t_{Pi}$ ) is unknown. In the main text, it is mentioned that ( $r = r_x \oplus r_t$ ). Could you clarify why ( $r_i$) acts as a confounder for both ( $t_{Pi}$ ) and ( $y_i$ )?

Response: Please refer to Figure 1(b) or Figure 5(c). In general networked settings, not only does the covariate $x_i$ of node $i$ influence $t_i$ , but $x_{P_i}$ and $t_{P_i}$ also affect $t_i$ and $y_i$ . Naturally, we can conclude that if the behavior of node j influences node $i$ , and node $k$ influences both nodes $i$ and $j$ (i.e., $j \in P_i , k \in P_i \cap P_j$ ), then $\{x_k, t_k\}$ would act as common causes of $t_j$ and $y_i$ . Thus, we find that node $i$ ’s interference information $r_i$ includes information about $\{x_k, t_k\}$ , which introduces confounding in the estimation of the peer effect between $t_j$ and $y_i$ .

Thanks for your detailed advice and comments. We welcome any further technical advice or questions on this work and we will do our best to address your concerns.

[W4] When network interference is unknown, how does the proposed method differ from existing work (e.g., Rf1-Rf3)? While the paper compares CauGramer with several mainstream methods, it lacks a detailed discussion of specific methods for unknown interference graphs (e.g., Rf1-Rf3).

(Savje et al., 2021): Savje, Fredrik, Peter Aronow, and Michael Hudgens. "Average treatment effects in the presence of unknown interference." Annals of Statistics 49.2 (2021): 673.

(Hoshino et al., 2023): Hoshino, Tadao, and Takahide Yanagi. "Causal inference with noncompliance and unknown interference." Journal of the American Statistical Association (2023): 1-12.

(Lin et al., 2024): Lin, Xiaofeng, et al. "Treatment Effect Estimation Under Unknown Interference." Pacific-Asia Conference on Knowledge Discovery and Data Mining. Springer Nature Singapore, 2024.

Response: We sincerely thank the reviewers for highlighting three related works on non-i.i.d. data. In the revised manuscript, we have included discussions on these works (listed in Appendix C):

• Savje et al., (2021): This paper study average treatment effects estimation in randomized experiments with unknown and arbitrary interference among units. The authors propose the Expected Average Treatment Effect (EATE) as a generalized estimand that incorporates potential spillover effects by marginalizing over all possible treatment assignments. While their study provides a theoretical and experimental analysis using existing methods, it does not introduce new algorithms to address unknown interference. Furthermore, their analysis is based on randomized controlled trials (RCTs). In contrast, our paper focuses on addressing the challenge of unknown interference on observational data.
• Hoshino et al., (2023): This paper studies causal inference models in which individuals interact within a social network and may not comply with the assigned treatments. It potentially assumes that the interference arises through the social network, where the structure of interference may depend on local or approximate neighborhood interactions. Based on this, the paper introduces Instrumental Exposure Mapping (IEM), a low-dimensional representation of potentially complex spillover effects, to estimate Intention-to-Treat (ITT) and Local Average Treatment Effects (LATE) for compliers, accounting for both direct and indirect effects while explicitly handling the challenges of noncompliance and unknown interference. However, this paper assumes that the interference graph is a subgraph of the social network and relies on the random assignment of instrumental variables (IVs), as well as the independence of IVs across individuals, to ensure valid causal inference.
• Lin et al., (2024): Under unknown interference, the UNITE algorithm uses a Graph Structure Learner (GSL) to infer the hidden interference structure by constructing a complete graph and imposing L0-norm regularization to identify significant connections. With the inferred structure, Graph Convolutional Networks (GCNs) are employed to learn an aggregation function and model the interference from neighboring units, effectively capturing the interference patterns and enabling accurate treatment effect estimation.

[Savje et al., 2021] and [Hoshino et al., 2023] rely on randomized instrumental variables (IVs) or experiments, which are infeasible in our observational data setting. The most relevant work is [Lin et al., 2024], which addresses unknown interference graphs and learnable aggregation functions through their framework, UNITE, using L0-norm regularization and GCNs for accurate treatment effect estimation. We include comparisons with UNITE in our experiments. We have added the comparison experiments in Table 3 in main text and in Table 10 in Appendix E.7 (with track changes marked in blue).

We use the original implementation of UNITE provided by Lin et al., (2024) and find that it only supports the estimation of AME and IME. While UNITE shows a slight improvement over our method in terms of the Average Main Effect (AME), its performance significantly drops at the individual level. Our method consistently achieves high performance across all six metrics, demonstrating its superiority in addressing both average and individual-level effects.

Dear Reviewer #qEMS,

We sincerely appreciate your constructive comments to improve our paper. You may find our corresponding explanations below for your concerns. We have uploaded the revised paper (with track changes marked in blue), and we would greatly appreciate it if you could let us know of any further concerns.

[W1] Typos:

In Table 1, which includes i.i.d. data, the title "Causal Networked Data" may not be fully accurate. Perhaps "Observational Data" would be more accurate.

On line 129, "Representation" could be more accurate formatted in lowercase as "representation."

In Section 4.1, Ma & Tresp’s work is not necessarily limited to first-order interference.

There are multiple instances of quotation mark formatting errors in lines 375, 449, 487, and 516, among others.

Response: Thank you for your thoughtful suggestions. Regarding Table 1, we have revised the title to “Characteristics of Representative Algorithms on Observational Networked Data” and removed the i.i.d. methods, CFRNet and the Doubly Robust Estimator. Additionally, we have incorporated more non-i.i.d. methods suggested by the reviewers, including Savje et al., (2021), Hoshino et al., (2023), Uncertainty (Bhattacharya et al., 2020), and UNITE (Lin et al., 2024). For the other issues, we have revised the related statements and corrected all quotation mark formatting errors throughout the manuscript.

[W2] The method appears to rely on an unconfoundedness assumption, yet latent confounders are mentioned later, violating this assumption. How did you address latent confounders in their approach?

Response: Thank you for pointing this out. In Appendix D.4, we introduce CauGramer(NC), an extension of CauGramer that addresses latent confounders using negative control variables (NCs). As illustrated in the causal diagram of NCs (Figure 5), negative control exposures (NCEs) $Z$ are proxy variables linked to the treatment $T$ through the same unmeasured confounders $U$, while negative control outcomes (NCOs) $W$ are influenced by the same unmeasured confounders $U$ as the outcome $Y$. The bridge functions in CauGramer(NC) are designed to learn these transformation links, effectively eliminating the confounding effects of unmeasured confounders and enabling accurate causal identification.

[W3] The work lacks a description of the number of layers (L) and attention heads (M) used.

Response: The choice of the L-order neighborhood parameter would impacts the CauGramer's performance. A small $L$ value can limit the model’s ability to capture complex relationships between long-distance nodes, reducing its representation power. Conversely, a large L value may introduce excessive noise and irrelevant information, leading to increased computational complexity and a higher risk of overfitting.We add an analysis of the hyper-parameters $\{L, M\}$ in Table 5 in Appendix E.2.

Table: Hyper-Parameter Optimization on Flickr Dataset: CauGramer with $L$-layer and $M$-heads.

In this paper, we propose a $L$-layers (default: 2) $M$-heads (default: 3) cross-attention GCN to learn the representation $\boldsymbol{R}_x=g_x(\boldsymbol{x}, \boldsymbol{A})$. Then we propose a $L$-layers (default: $L=1$ for main results and $L=2$ for unknown interference graph in Figure 3) $M$-heads (default: 3) cross-attention GCN to learn the representation $\boldsymbol{R}_t=g_t(\boldsymbol{t}, \boldsymbol{A})$. In real applications, we can use a grid search on the validation dataset to choose the optimal $L$ value and $M$ value.

Dear Reviewer #qEMS,

Thank you for dedicating your time to reviewing our manuscript and offering valuable suggestions, which have significantly improved our paper. As the deadline for uploading the revised PDF approaches, we kindly inquire if you have any further concerns. We look forward to any further feedback you may provide.

In response to your previous constructive feedback, we have made significant revisions to our manuscript:

1. Reorganization of Related Work

We apologize for the previous organization of the related work in Section 2 and Appendix D in our first submission. Our CauGramer is related to seven domains. In our original submission, due to the limited space, we only listed representative algorithms (which served as comparison baselines in the Experiments section) in the Related Work Section, and deferred discussions on “Partial Interference”, “Exposure Mapping”, and “Unknown Network Structure” to Appendix D, titled “Supplementary Related Work in Networked Interference” in our original manuscript.

Based on your comments, we have reorganized the Related Work section, now including a discussion of these studies in the main text for better context. The most relevant work is [Lin et al., 2024], which addresses unknown interference graphs and learnable aggregation functions through their framework, UNITE, using L0-norm regularization and GCNs for accurate treatment effect estimation. We include comparisons with UNITE in our experiments. We have added the comparison experiments in Table 3 in the main text and in Table 10 in Appendix E.7 (with track changes marked in blue).

2. Extension for Latent Confounders and Hyperparameter Analysis on $\\{L, M\\}$

Our proposed CauGramer relies on the unconfoundedness assumption to aggregate peer interference. However, in the presence of latent confounders, we proposed an extension, CauGramer(NC), which incorporates negative control variables (NCs) to mitigate latent confounding. Detailed definitions and discussions of NCs are provided in Appendix D.4.

Additionally, we have added experiments and a comprehensive analysis of hyperparameters $L$ (number of layers) and $M$ (number of attention heads) in Table 5 in Appendix E.2. In this paper, we propose a $L$-layers (default: 2) $M$-heads (default: 3) cross-attention GCN to learn the representation $\boldsymbol{R}_x=g_x(\boldsymbol{x}, \boldsymbol{A})$. Then we propose a $L$-layers (default: $L=1$ for main results and $L=2$ for unknown interference graph in Figure 3) $M$-heads (default: 3) cross-attention GCN to learn the representation $\boldsymbol{R}_t=g_t(\boldsymbol{t}, \boldsymbol{A})$. In real applications, we can use a grid search on the validation dataset to choose the optimal $L$ value and $M$ value.

3. Clarifications to Address Questions (https://openreview.net/forum?id=foQ4AeEGG7&noteId=deZMqK4Upw)

• For Q1, we elaborated on the concept of the interference graph, emphasizing its distinction from the social network and its role in capturing broader transmission pathways.

• For Q2, we detailed the rationale for using $r_x^{(L)}$ as the input, highlighting its function as a robust embedding that combines node and peer information.

• For Q3, we clarified the simulation of unmeasured confounding by explaining how covariates are masked during training, testing model robustness.

• For Q4, we use an example to explain the relationship between peer treatment ( $t_{Pi}$ ) and outcome ( $y_i$ ) remains confounded by the common causes ( $r_i$ ).

We have uploaded the revised paper (with track changes marked in blue). Provided that your concerns have been well-addressed, we would greatly appreciate it if you would consider raising your score. Should you have any further comments or queries, we would greatly appreciate the opportunity to address them promptly.

We deeply appreciate your time and effort in reviewing our paper and your constructive feedback.

Authors of Paper #2331

Dear Reviewer #uH9y,

We sincerely appreciate your thoughtful and constructive comments, which have been invaluable in improving our paper. Below, we provide detailed explanations addressing your concerns. We have also uploaded the revised paper (with track changes marked in blue) for your review, and we would greatly appreciate it if you could let us know of any further concerns or suggestions.

[Q1] Can you elaborate on the computational complexity of CauGramer, especially compared to other graph neural network and transformer-based methods? Are there any specific strategies employed to mitigate the computational burden, such as approximation algorithms or distributed training?

Response: CauGramer suffers from the same limitation as other graph transformers, namely the quadratic complexity of the attention computation. However, unlike Graphormer and its variants, which rely on computationally expensive full graph attention mechanisms through self-attention, CauGramer computes cross-attention only from its neighbors' embeddings. Its time cost decreases as the degree of the nodes decreases.

We report the average training times of various methods in a single execution on both datasets in Table 6 in Appendix E.3. Our algorithm (CauGramer) shows a significant reduction in single execution training time compared to two Transformer methods (Graphormer and CAL) and two advanced algorithms (SPNet and NetEst) on networked interference data. CauGramer offers a more efficient and accurate causal graph transformer for estimating treatment effects under unknown interference graphs.

For reference, a single training run of CauGramer on BlogCatalog and Flickr takes less than 180 seconds on a MacBook Pro with an Apple M2 Pro chip. Given such training cost, we did not employ approximation algorithms or distributed training to mitigate the computational burden.

[Q2] How sensitive are the results to the choice of the L-order neighborhood parameter in CauGramer? What is the impact of choosing too small or too large an L value? Is there an optimal way to select this parameter?

Response: The choice of the L-order neighborhood parameter would impacts the CauGramer's performance. A small $L$ value can limit the model’s ability to capture complex relationships between long-distance nodes, reducing its representation power. Conversely, a large L value may introduce excessive noise and irrelevant information, leading to increased computational complexity and a higher risk of overfitting.We add an analysis of the hyper-parameters $\{L, M\}$ in Table 5 in Appendix E.2.

Table: Hyper-Parameter Optimization on Flickr Dataset: CauGramer with $L$-layer and $M$-heads.

In this paper, we propose a $L$-layers (default: 2) $M$-heads (default: 3) cross-attention GCN to learn the representation $\boldsymbol{R}_x=g_x(\boldsymbol{x}, \boldsymbol{A})$. Then we propose a $L$-layers (default: $L=1$ for main results and $L=2$ for unknown interference graph in Figure 3) $M$-heads (default: 3) cross-attention GCN to learn the representation $\boldsymbol{R}_t=g_t(\boldsymbol{t}, \boldsymbol{A})$. In real applications, we can use a grid search on the validation dataset to choose the optimal $L$ value and $M$ value.

Dear Reviewer #EAHD,

We sincerely appreciate your constructive comments, which have been instrumental in enhancing our manuscript. Below, we will provide detailed explanations addressing your concerns and upload the revised version of our paper (with track changes marked in blue). We would be very grateful if you could review these responses and inform us of any additional issues or further feedback you may have.

In the revised manuscript, we include detailed discussions on three related works in Appendix C:

• Leung et al., (2022): This paper introduces a non-parametric approach for estimating causal effects, specifically treatment and spillover effects, using observational data from a known social network. It proposes a doubly robust learner to model causal effect where interference decreases with network distance, accounting for peer influences on both outcomes and treatment selection. Unlike prior studies that rely on low-dimensional representations of confounding, the authors introduce a high-dimensional network unconfoundedness condition, utilizing graph neural networks (GNNs) to estimate these confounders effectively.
• Ogburn et al., (2024): This paper introduces a semiparametric approach for handling dependencies caused by information transmission and latent similarities between nodes. It assumes a known interference graph with interference limited to direct neighbors and relies on prior knowledge of two structural functions, $s_C$ and $s_X$ , to summarize covariate and treatment interferences. Given these strong assumptions, Net-TMLE leverages structural equation modeling and targeted maximum likelihood estimation to estimate causal effects, supporting inference for static, dynamic, and stochastic interventions in social networks.
• Khatami et al., (2024): Assuming the interference graph and exposure mapping are known, this paper proposes constructing a focal set, where all units’ neighborhoods are non-overlapping, to ensure independence among analyzed units and enable consistent estimation of causal effects in social networks. By combining graph machine learning approaches with the Double Machine Learning (GDML) framework, this method represents a significant advancement in causal inference. It integrates Graph Neural Networks (GNNs) to estimate propensity scores and outcome models while addressing challenges like interference, high-dimensional confounding, and model misspecification.

(Leung et al., 2022) Michael P Leung and Pantelis Loupos. Unconfoundedness with network interference. arXiv preprint arXiv:2211.07823, 6, 2022.

(Ogburn et al., 2024) Elizabeth L Ogburn, et al. Causal inference for social network data. Journal of the American Statistical Association, 119(545):597–611, 2024.

(Khatami et al., 2024) Seyedeh Baharan Khatami, et al. Graph neural network based double machine learning estimator of network causal effects. arXiv preprint arXiv:2403.11332, 2024.

[W1] The paper’s primary contribution—clearly distinguishing the interference graph from the social network—is highly valuable. However, if the approach is limited to neighboring nodes only, it risks undermining this motivation. On the other hand, if this limitation is not imposed, the computational complexity becomes quadratic in the number of nodes when using transformers, which poses feasibility issues for large graphs.

Response: Thanks for your comment. However, our CauGramer is not limited to neighboring nodes in social networks. Instead, our CauGramer can use any known connected graph (which can even be randomly generated) to aggregate interference information. As demonstrated in Figure 3, CauGramer remains robust even when the provided network is entirely unrelated to the 'Random' interference graph. It still learns causal effects effectively and outperforms existing methods, suggesting that any connected graph—even one that is randomly generated and independent of the interference graph—can help aggregate interference information. Exploring how to construct an efficient connected graph would be an interesting directions, but it is beyond the scope of this paper and is left for future work. In Appendix B, we added a new section 'Limitations and Future Directions' to further clarify this.

In this paper, we focus on real-world scenarios where individuals closely connected in a social network are more likely to interfere with each other. Based on this observation, we proposed the CauGramer algorithm to infer unknown interference information using a known social network. Social networks are typically easier to obtain than true interference graphs, as they can often be derived from sources such as co-authorship networks, public community relationships, organizational affiliations, or online interactions like comments and messages. As a result, most existing studies also tend to rely on social networks for reasearch purposes.

Dear Reviewer #EAHD,

Thank you for dedicating your time to reviewing our manuscript and offering valuable suggestions, which have significantly improved our paper. As the deadline for uploading the revised PDF approaches, we kindly inquire if you have any further concerns. We look forward to any further feedback you may provide.

In response to your previous insightful comments, we have made significant revisions to our manuscript:

• In Related Work Section, we restructure the related work and include discussions of more network interference works (due to limited space, some discussions are placed in Appendix D in our first submission): Leung et al., (2022), Ogburn et al., (2024) and Khatami et al., (2024) with known interference graph; Bhattacharya et al. (2020), Savje et al., (2021), Hoshino et al., (2023), Lin et al., (2024) under unknown interference graph. We include detailed discussions on three related works in Appendix C.
• In Experiment Section, we include DRLearner (Leung et al., 2022), GDML and GDML w/o FSet (Focal Set) from Khatami et al. (2024) as three additional baselines. We discuss these baselinses in Table 3 in the main text and in Table 10 in Appendix E.7 (with track changes marked in blue).
• In Appendix B, we rename the “Limitation” section to “Limitations and Future Directions” to better reflect the scope. Our CauGramer can work when the network information is totally unknown. In such cases, CauGramer can use any known connected graph (even one that is randomly generated) as a substitute for the social network to aggregate interference information.
• We add a section in Appendix D.3 to discuss the core intuition behind the bridge function and its role in ensuring independence between the residual $y−\hat{y}$ and the variables $\\{x,t\\}$.

Clarifications to Address Questions:

• For Q1, we add a section in Appendix D.3 to discuss the intuition behind the bridge function and its role in ensuring independence between the residual $y−\hat{y}$ and the variables $\\{x,t\\}$.

• For Q2, we clarified why the residual must be independent of t, discussing two scenarios: when all confounders are observed and when they are unmeasured.

• For Q3, we use the specified mean function for exposure mapping when explicitly stated, or a learnable network for unspecified or unknown mappings to adaptively learn from neighbor treatment ratios and treatment information.

• For Q4, we described how negative control variables (NCE and NCO) address unmeasured confounders by embedding them into the objective function Eq. (16).

• For Q5, our choice to conduct 10 replications aligns with the experiment setup used in conventional studies, such as Jiang and Sun (2022), Huang et al. (2023), and Cai et al. (2023).

We have uploaded the revised paper (with track changes marked in blue). Should you have any further comments or queries, we would greatly appreciate the opportunity to address them promptly.

We deeply appreciate your time and effort in reviewing our paper and your constructive feedback.

Authors of Paper #2331