Telling Peer Direct Effects from Indirect Effects in Observational Network Data

We sincerely appreciate the reviewer's insightful comments and the high recognition of the value and importance of our work.

Comment: Baseline models details needed: Which peer and self-treatment effects do they evaluate?

In our simulation, the baseline models provide estimates of the overall peer effects (PE) and self-treatment effects (STE). Specifically, the baseline methods estimate:

• Peer Effects (PE): This represents the aggregated influence of peers on an individual's outcome. These models capture the total effect that the treatments and outcomes of an individual's neighbors have on that individual but do not distinguish between the direct influence (Peer Direct Effects, PDE) and the indirect influence (Peer Indirect Effects, PIE).

• Self-Treatment Effects (STE): This measures the effect of an individual's own treatment on their outcome.

Thus, while the baseline models evaluate the combined peer effect and the self-treatment effect, they do not explicitly decompose the peer effects into PDE and PIE. Our proposed gDIS framework, on the other hand, is designed to disentangle these two components, offering a more detailed analysis of the peer influences within network data.

Q1: Product promotion example to main paper?

We plan to include the product promotion campaign example in the introduction, alongside the existing example from epidemiology. We will move the proof to the appendix to make space for the example.

Q2: $W_{t_i}$, $W_{y_i}$ notation appears to be ambiguous. Do they depend on the realized values of $T$ and $Y$ for individual $i$?

$W_{t_i}$ and $W_{y_i}$ do not depend on the realized values of $T_i$ or $Y_i$. They represent summary of the treatments and outcomes of unit $i$'s neighbors. $W_{t_i}$ and $W_{y_i}$ are illustrated in Figure 2b and explained in Line 313 (left column), and Appendix B.

Q3: Definition 3.6 as proposition?

We agree that Definition 3.6 may be more appropriately stated as a proposition as it presents a reformulation and decomposition of the peer effects introduced in Definitions 3.4 and 3.5.

Our intention was to highlight this decomposition as a central conceptual component of our framework, which is why we initially presented it as a "definition." However, we appreciate the reviewer's point and will revise the manuscript to present it as a proposition.

Proposition (Peer Effect Decomposition). The peer effect (PE) can be decomposed into the sum of the PDE and the PIE. That is,

\begin{aligned} \text{PE}(w' _{t _i}) &= \text{PDE}(w' _{t _i}) + \text{PIE}(w' _{t _i}), \end{aligned}

\begin{aligned} \text{PE}(w _{t _i}) &= \text{PDE}(w _{t _i}) + \text{PIE}(w _{t _i}). \end{aligned}

The peer effect (PE) can be decomposed into the sum of the peer direct effect (PDE) and the peer indirect effect (PIE). That is,

This proposition shows that the total peer effect integrates both the direct and indirect pathways of peer influence. Equation (3a) and Equation (3b) correspond to the vaccination and product promotion examples, respectively.

Q4: Alignment with previous work?

Our definition of peer effects aligns with prior works in the one-to-one setting, except that our peer treatment and outcomes are summaries of the treatments and outcomes of the neighbors of $i$, respectively.

Q5: Total effect representation?

Yes, you are right when there is no interaction between peer treatment $W_{t_i}$ and individual treatment $T_i$. We have assumed this in our work. This is a very reasonable assumption, as individual $i$'s treatment and peer treatment do not affect each other directly, and the reason for an individual to take the same treatment as their peers is because the individual and their peers have similar features (characteristics).

Q6: Baseline vs. gDIS differences?

a) In Table 2 of the paper, we made a mistake in summarizing the results for gDIS. We have updated the table and present the revised version at the following link (due to space limitations): https://anonymous.4open.science/r/icmlSupp-4556/table3.png

b) The baseline methods' estimates of PE (peer effects) and STE (self-treatment effects) are formally consistent with our model; however, they do not explicitly decompose the PE into PDE (peer direct effects) and peer indirect effects (PIE).

c) Using the given structural causal model (Equation (19) in Appendix F), we can obtain the potential outcomes under different treatment conditions.

Q7: Error computation for counterfactuals?

The potential outcomes with respect to $T_i$ for each $i$ are known since the structural causal model is assumed to be known for the synthetic dataset. Hence, the ground truth values of STE, PE, PDE, and PIE can be derived. Subsequently, compute the estimation error using the PEHE (Precision in Estimation of Heterogeneous Effect) metric, by comparing the estimated effect with the ground truth causal effect, i.e., the difference in potential outcomes.

We thank the reviewer for the valuable comments. Detailed responses to each specific comment are provided below.

Weakness 1: Lack of comparison with PIE/PDE estimators.

The PIE/PDE estimators cited in our introduction (e.g., VanderWeele et al., Shpitser et al.) are designed for One-to-One interference settings, assuming dyadic interactions (e.g., household or pairwise influence). Our work focuses on the more general and realistic Many-to-One network setting. As such, these estimators cannot be directly or fairly compared with our method. Instead, we compare with recent network-level causal inference methods (e.g., NetEst, TNet, 1-GNN), which are designed to handle complex interference and represent the state-of-the-art in our setting.

---

Other Comments Or Suggestions: Typos

We thank the reviewer for the careful reading. We have corrected all.

---

Question 1: Since it is "Many-to-One" interference, units will affect each other. For the summary causal graph of all units, is it reasonable to use the DAG tool to represent the causal relationship for their interference?

The DAG shown in Figure 2b illustrates the unit-level causal relationships from the perspective of a single unit $i$, depicting how $i$'s outcome is affected by their own treatment and neighbors’ treatments and outcomes. This type of localized DAG is commonly used in the literature on causal inference with interference [1].

[1] Jiang, Song, and Yizhou Sun. "Estimating causal effects on networked observational data via representation learning." Proc. ACM Int. Conf. Inf. & Knowl. Manag. 2022.

---

Question 2: There is still a lack of comparison with existing peer direct/indirect effect methods cited in the introduction. Is it because they are of the "One-to-One" type and cannot be applied to this scenario?

Yes. The methods cited in the introduction (e.g., VanderWeele et al., Shpitser et al.) are designed for One-to-One settings and assume dyadic interactions. Our setup considers Many-to-One interference, which is more general and realistic in network data. These earlier methods cannot be directly or compared in our setting. Hence, we have compared with recent methods (e.g., NetEst, TNet, 1-GNN) that support many-to-one relationships.

---

Question 3: The potential outcome simulation is through an iteration process. How do you obtain the ground true direct/indirect effects?

As stated in [2-3], Gibbs sampling provides a practical method for approximating the causal effects of interest in the presence of complex interdependencies among individuals in a network. This is achieved by iteratively sampling from the conditional densities of each individual. Our iteration continues until the difference between the values of $Y$ in successive steps is less than $1 \times 10^{-5}$, thereby ensuring that the generated data has reached a stable state. Then, using a given structural causal model (Equation 19 in the Appendix F), we can obtain the potential outcomes under different treatment conditions $T$. The ground true direct/indirect effects can be obtained from the potential outcomes.

[2] Zhao, Ziyu, et al. "Learning individual treatment effects under heterogeneous interference in networks." ACM Trans. Knowl. Discov. Data 18.8 (2024): 1-21.
[3] Tchetgen Tchetgen, Eric J., Isabel R. Fulcher, and Ilya Shpitser. "Auto-g-computation of causal effects on a network." J. Am. Stat. Assoc. 116.534 (2021): 833-844.

---

Question 4: Could you provide more experimental results with a wider range of $\lambda$? The paper claims $\lambda = 0.3$ is optimal, but it seems only suboptimal as the curves in Figure 5 are still descending as $\lambda$ increases.

In our experiments, we evaluated $\lambda$ values ranging from 0 to 0.5 (results available at the anonymous link). We found that $\lambda = 0.3$ produced the best overall performance. When $\lambda$ exceeds 0.3, it starts to over-penalize the feature representations, leading to a decline in performance.

---

Question 5: Could you clarify how to choose the other hyperparameters shown in Table 6 and provide the hyperparameter space of the compared baselines?

For our method, we performed a grid search on the validation set to select the key parameters (learning rate, hidden dimensions, and regularization strength) that yielded the best performance (shown in Table 6, Appendix H). Following this comment, we have updated the search space for these key parameters in Appendix H, as shown in anonymous link. For benchmark methods, we followed the recommended settings (as shown in anonymous link) from their original papers and publicly available implementations.

We thank the reviewer for their insightful comments and recognition of our work's importance.

W1: Network unconfoundedness assumption... more discussion needed

We plan to explore: a) Instrumental Variable methods [1] to introduce variables influencing treatment but not outcomes; b) Hidden confounder modeling [2] to leverage graph-based representation learning for capturing unobserved confounding; c) Causal discovery [3] to employ data-driven methods for identifying adjustment sets and uncovering hidden confounding pathways.

1. Angrist, J.D., et al. Identification of causal effects using instrumental variables.
2. Louizos, C., et al. Causal effect inference with deep latent-variable models.
3. Colombo, D., et al. Learning high-dimensional directed acyclic graphs with latent and selection variables.

---

W2: Multiple components (GNN, attention, HSIC) - each component contributions

We'll add to Section 4.2:

GNN Layers: GNNs capture network interactions. Without them, our model couldn't leverage network structure for modeling peer effects, leading to oversimplified estimations that ignore relational dependencies.

Attention Mechanism: Enables assigning different weights to neighbors based on feature similarity, useful where peer influence isn't uniform.

HSIC Regularization: Mitigates overfitting by encouraging independence between node features and embeddings, reducing spurious correlations.

---

W3: More detailed sensitivity analysis of data generation processes needed

Gibbs sampling provides a practical method for approximating causal effects with complex interdependencies in networks by iteratively sampling from conditional densities [4]. In our data generation process, we varied noise levels and found that—since our iteration continues until the difference between values of $Y$ in successive steps is less than $1 \times 10^{-5}$—the data exhibits high stability across different noise levels. This indicates our process effectively captures complex network interactions. We plan to explore additional data generation methods and conduct further sensitivity analyses.

4. Zhao, Z., et al. Learning individual treatment effects under heterogeneous interference in networks.

---

W4: Extension to non-binary treatments would increase practical utility

Our framework is flexible and does not rely on the treatment must be binary. When computing PDE and PIE, the neighbor treatment exposure variable $W_{t_i}$ supports continuous, multi-valued, or binary treatments. For STE, we've followed standard binary treatment approaches, but as a CATE (conditional average treatment effect) problem, it's extendable to continuous/multi-valued treatments [5-6].

5. Hirano, K., et al. The propensity score with continuous treatments.
6. Feng, P., et al. Generalized propensity score for estimating the average treatment effect of multiple treatments.

---

W5: Scalability discussion needed on large-scale datasets

We'll add the following to Appendix I:

Although attention mechanisms introduce additional overhead compared to standard GNNs, we optimize sparse graph processing using PyTorch Geometric [7], enabling efficient computation even on large-scale datasets. Additional strategies can enhance scalability: sampling-based techniques like those in GraphSAGE [8] can limit neighbors sampled for each node, reducing computational demands while preserving structural information. Approximation or sparsification techniques for the attention layer can also alleviate computational burdens [9].

7. Fey, M., et al. Fast graph representation learning with PyTorch Geometric.
8. Hamilton, W., et al. Inductive representation learning on large graphs.
9. Child, R., et al. Generating long sequences with sparse transformers.

---

Q1: Robustness to violations of network unconfoundedness

We're currently analyzing violations of the network unconfoundedness assumption. Our preliminary expectation is that robustness depends on factors like the strength of association between hidden confounders and treatment/outcome. Future work will include comprehensive sensitivity analyses to quantify how these factors affect performance.

---

Q2: Extension to continuous treatments or complex outcomes

Yes, gDIS can handle continuous treatments and complex outcome structures. Our framework is flexible with no binary treatment requirement. When computing PDE and PIE, the neighbor treatment exposure variable $W_{t_i}$ accommodates any treatment type. For STE, our implementation follows standard causal inference with binary treatment effects, but as STE estimation is a typical CATE problem, our approach can extend to continuous or multi-valued treatments using existing techniques [6]. For multivariate outcomes, the framework can be adapted using joint modeling approaches (e.g., multi-output regression [10]) to capture dependencies among multiple outcomes.

10. Sener, O., et al. Multi-task learning as multi-objective optimization.

We thank the reviewer for the valuable comments. Detailed responses to each specific comment are provided below.

W1: The effectiveness of the proposed approach depends on strong assumptions, which may be challenging to satisfy in real-world observational data.

Our method is based on three key assumptions: Network Unconfoundedness, Network Consistency, and Network Overlap. These assumptions are commonly assumed in causal inference in networked data [1].

In this paper, our focus is on the challenge of distinguishing and estimating different types of causal effects in network data (i.e., the Peer Direct Effect (PDE), Peer Indirect Effect (PIE), and Self-Treatment Effect (STE)), and we adopt the assumptions commonly used in the network causal inference literature.

We have included discussions regarding the limitations of assumptions in the "Limitations & Future Work" section.

• [1] Jiang, Song, and Yizhou Sun. "Estimating causal effects on networked observational data via representation learning." Proceedings of the 31st ACM International Conference on Information & Knowledge Management, 2022.

Q1: Are there any specific requirements for the network structure in order to ensure the validity of the proposed approach?

Our proposed approach does not impose structural requirements on the network (e.g., specific topology or connectivity patterns). However, we do assume the causal relationships are as in Figure 2(b), which are realistic.