Training Robust Graph Neural Networks by Modeling Noise Dependencies

We sincerely thank the reviewer for carefully reading our paper and providing thoughtful and constructive feedback. We believe the comments are highly valuable and have significantly helped us improve the quality and clarity of the manuscript.

Response to W1

In the revised manuscript, we have carefully revised the main text to improve readability and clarity. Specifically:

• We will provide clear definitions for all probabilistic terms in Eq. (1), directly in the main text.
• We will explicitly explain the use of Bern(⋅) to in main text.
• Several important methodological details that were previously located in the appendix will be moved into the main text to ensure the paper is more self-contained and easier to follow.

We believe these revisions significantly improve the clarity and accessibility of the paper, and we thank the reviewer for raising this important point.

Response to W2

We agree that the original submission assumed a certain level of familiarity with graphical causal modeling and variational inference. In the revised manuscript, we have added the background sections in Appendix to provide a more accessible overview of the necessary concepts.

Response to W3

While the implementation of DAGNN draws inspiration from the spirit of VAE and CausalNL [1], we address complex and unique challenges absent in them. Specifically, the incorporation of $A$ necessitates handling supplementary latent variables and causal relationships, such as $Z_A$, $\epsilon_A$, $A$ ← $\epsilon_A$, $A$ ← $X$, $Y$ ← $A$, $A$ ← $Z_A$, each posing non-trivial obstacles beyond their straightforward extension to the graph domain. It's worth noting that we have addressed this issue in Appendix D to underscore the technical advancements of DA-GNN in comparison to [1].

[1] Instance-dependent Label-noise Learning under a Structural Causal Model

Response to Q1

We conduct a statistical analysis on a real-world news network, PolitiFact [1], where node features represent news content, node labels correspond to news topics or categories, and edges denote co-tweet relationships—that is, instances where the same user tweeted both pieces of news. The network includes both fake and benign news, with fake news regarded as feature noise induced by malicious user intent ($\epsilon_X$).

To investigate noise dependency patterns associated with the presence of fake news, We hypothesize that the presence of fake news (i.e., feature noise) leads to noisy graph structures and noisy node labels in news networks. Specifically, we assign a semantic topic to each news article as a node label using k-means clustering over BERT embeddings of the article content. For each node in the graph, we compute the Shannon entropy of the semantic topic distribution among its neighboring nodes. We then compare these entropy values between fake and benign news nodes (results shown below).

Fake news

• mean: 1.330333
• 25%: 1.401647
• 50%: 1.465137
• 75%: 1.494437

Benign news

• mean: 1.083667
• 25%: 1.004271
• 50%: 1.352713
• 75%: 1.431281

Mann-Whitney U test

• p-value = 4.186423672941698e-25
• Mann-Whitney U statistic = 47085.5

Descriptive statistics reveal that fake news nodes generally exhibit higher entropy than benign news nodes, suggesting that benign news tends to connect to semantically similar articles (homophilic), whereas fake news is more frequently connected to semantically dissimilar articles (heterophilic). This observation aligns with common user behavior: people typically share news related to their interests, whereas fake news is often propagated indiscriminately, regardless of topical relevance [2]. Furthermore, a non-parametric statistical test (Mann–Whitney U test) confirms that the difference in entropy values between fake and benign news is statistically significant. These findings suggest that the presence of fake news (i.e., node feature noise) introduces noisy and heterophilic edges into the graph structure. Furthermore, model-based automated news topic prediction often performs poorly due to noise in both features and graph structures, ultimately resulting in incorrect label annotations.

In summary, these findings empirically support the noisy dependency scenario in real-world scenario where feature noise (i.e., fake news content) can propagate through the graph, generating noisy edges and noisy labels. This highlights the need for our work that explicitly model and mitigate such noise dependencies in real-world networks.

[1] DECOR: Degree-Corrected Social Graph Refinement for Fake News Detection, KDD 2023

[2] Revisiting Fake News Detection: Towards Temporality-aware Evaluation by Leveraging Engagement Earliness, WSDM 2025

Response to Q2

In response to the reviewer’s comment, we conduct an experiment where we independently double each of the following: (1) the number of fraudsters (i.e., nodes with noisy features) and (2) the activeness of fraudsters (i.e., the amount of structure noise they introduce) in our real-world DANG generation process. As a result, label noise also increases accordingly, in proportion to the amount of generated feature and structure noise.

As shown in the table below, DA-GNN demonstrates competitive performance and, in many cases, outperforms other baselines under these intensified noise conditions.

Response to Q3

We conducted sensitivity analyses on our tuned hyperparameters, $\gamma$, $k$, $\theta$, $\lambda_1$, and $\lambda_2$ in Section 5.4 and Appendix F.6. And we found that DA-GNN is generally not sensitive to most of them. Hence, we suggest that DA-GNN is practical and easy to deploy, as it does not require extensive hyperparameter tuning, even in low-resource settings.

Response to Q4

We apologize for the oversight. We will revise Figure 3 by enlarging the font size and adjusting the layout to improve readability. We also appreciate your observation regarding the missing arrow in Figure 3; we will correct this in the updated version.

Response to Q5

In response to the reviewer’s concern, we conduct analysis substituting GNN components with GAT. In the below table, we can see that DA-GAT (DA-GNN with GAT) still outperforms other baselines, which demonstrates there is no sensitivity on specific choice of backbone GNNs.

Furthermore, we compare our method with representative noise-robust GNN approaches on two heterophilic datasets, Cornell and Wisconsin. As shown in the table below, our proposed method, DA-GNN, achieves superior or competitive performance relative to the baselines, demonstrating its broad applicability across diverse settings.

Response to Q6

In response to the reviewer’s comment, we conduct a new analysis in which we significantly increase or decrease the influence of noise dependency. Due to length constraints, we kindly refer the reviewer to our response to Reviewer KCjc’s comment W2 for further details.

We appreciate the reviewer's thoughtful discussion and interest in our work. These insights will meaningfully contribute to improving the quality of the paper.

Response to Q1

We thank the reviewer for this valuable suggestion. As rightly pointed out, we acknowledge that not all graph domains strictly conform to the noise assumptions in DANG, for instance, non-relational graphs such as protein–protein interaction or molecular structure networks may exhibit different noise patterns and causal dependencies. However, we would like to emphasize that the DANG framework is well-suited to a wide range of graph domains (see Appendix C.1), particularly those where graph learning methods are most commonly applied, such as citation, social networks, e-commerce, and web graphs.

That said, we fully agree with the reviewer’s suggestion to clarify the scope of applicability and discuss the extent to which the assumptions made in DANG generalize to non-relational domains. We will modify the line 154-157 of Section 3 into the following:

As-is:

DANG is prevalent across various domains, including social networks, e-commerce, web graphs, citation networks, and biology. Due to space constraints, detailed scenarios are provided in Appendix C.1. While not all noise scenarios perfectly align with DANG, such cases are rare compared to the widespread occurrence of DANG in these domains, dominating graph applications.

To-be:

DANG is prevalent across various domains, including social networks, e-commerce, web graphs, citation networks, and biology networks (e.g., cell-cell graph in the domain of single-cell RNA-sequencing). Due to space constraints, detailed scenarios are provided in Appendix C.1. We acknowledge, however, that not all noise scenarios perfectly align with DANG, and certain edge cases exist. For instance, in non-relational domains such as molecular structures or protein–protein interaction networks, the graph structure is inherently determined and thus unaffected by node feature noise. In such cases, the DANG scenario may not apply.

That said, we emphasize that these exceptions are relatively rare compared to the broad applicability of DANG across commonly studied graph domains, including social networks, e-commerce, web graphs, citation networks, and cell-cell network in biology.

Response to Q2

Advantage

Several baseline methods encounter out-of-memory (OOM) errors on large-scale datasets such as ogbn-arxiv primarily due to their reliance on structure learning components. These methods often require iteratively computing an explicit 𝑁×𝑁 similarity matrix or graph refinement operations, which incur high memory overhead, where 𝑁 is the number of nodes.

In contrast, our method avoids this bottleneck by approximating the regularization term $kl(q_{\phi_1}(Z_A|X, A)||p(Z_A))$ efficiently (See Appendix B). This design significantly reduces the computational and memory cost, enabling DA-GNN to scale to large graphs without OOM issues.

Disadvantage

As the reviewer rightly pointed out, the use of multiple encoders and decoders introduces additional computational overhead. However, these components are lightweight compared to the structure learning modules used in many baselines.

To quantify these trade-offs, we conduct an inference time analysis on the ogbn-arxiv dataset, comparing only models that do not suffer from OOM issues. The results are as follows:

DA-GNN: 0.0599 sec RTGNN: 0.0461 sec AirGNN: 0.0346 sec EvenNet: 0.0027 sec

While DA-GNN incurs slightly higher inference time than RTGNN and AirGNN, the difference remains within a practically acceptable range, especially considering that DA-GNN significantly outperforms these models in terms of robustness. Moreover, DA-GNN achieves substantial memory efficiency, avoiding OOM issues that affect 7 out of 10 baseline methods.

Summary

Although DA-GNN’s encoder-decoder architecture introduces a modest increase in runtime, this overhead is negligible when weighed against its superior robustness. Furthermore, the efficient design of our regularization component contributes to its strong memory performance on large-scale datasets.

Response to Q3

Thank you for the valuable comments. In the “without noise dependency” scenario, there are no causal relationships among $X \rightarrow A$, $A \rightarrow Y$, or $X \rightarrow Y$. This implies that the observed graph includes only independent feature noise ($\epsilon \rightarrow X$), independent structure noise ($\epsilon \rightarrow A$), and NO label noise at all. Please note our assumption in DANG that $\epsilon$ is not a cause of $Y$, which reflects real-world situations where mislabeling is more likely to arise from confusing or noisy features rather than arbitrary sources (see lines 147–150 in the main paper).

Since label noise is known to significantly degrade GNN performance, it is expected that the results in Table 2 (in response to Reviewer KCjc’s comment W2) are generally better than those in Table 1 of the main paper.

Furthermore, we evaluate an extreme noise scenario in Figure 5 of the main paper, where node features, graph structures, and node labels all contain independent noise. In this setting as well, DA-GNN consistently outperforms all baselines.

Response to Q4

Thanks for the meaningful discussion. We would like to emphasize that the effectiveness of our proposed method does not stem from the use of a powerful encoder or decoder architecture, but rather from its ability to model the causal relationships inherent in the DGP of DANG. Specifically, each encoder is responsible for encoding a specific latent variable, and each decoder for decoding its corresponding observable variable. These components are jointly trained under the supervision of our objective to effectively fulfill their designated roles. In this context, we claim that the performance of DA-GNN is not sensitive to the choice of GNN backbone.

Furthermore, we attribute the strong performance of DA-GNN on heterophily graphs to the regularization applied to $Z_A$. Specifically, to promote accurate inference of $Z_A$, we regularize it to prioritize assortative edges based on $\gamma$-hop subgraph similarity (see lines 211–217 in the main paper for details). Here, $\gamma$ is set as a hyperparameter in {$0, 1$}. It is widely recognized that, in heterophily graphs, nodes with high feature similarity are more likely to share the same label [1]. In this context, 0-hop subgraph similarity corresponds to feature similarity, which facilitates effective inference of the latent graph structure even in heterophilous settings. This design choice contributes to the strong performance of DA-GNN under heterophily.

[1] Graph Neural Networks for Graphs with Heterophily: A Survey

We sincerely thank the reviewer for their time and thoughtful review of our paper, as well as for recognizing the contributions of our work. Below, we provide our detailed responses to the reviewer’s concerns and questions.

Response to W2

We agree that the current version may appear compressed, as it attempts to cover multiple aspects of our framework within the limited space. In response, we have revised the manuscript to improve clarity and readability by restructuring the content as follows:

• We will streamline the main text to focus more clearly on the core components of our method, ensuring that the key ideas are presented with sufficient depth and continuity.

• To reduce fragmentation and improve conceptual clarity, we will move several technical details on regularization—while still important—to the Appendix, and clearly reference them from the main text.

• We will also enlarge key figures and tables in the main paper where space permits, and add larger, high-resolution versions of them in the Appendix to enhance legibility.

We believe these changes significantly improve the readability of our paper and hope they address the reviewer’s concern.

Response to Q1

In our model, $Z_Y$​ represents the true latent label, while $Y$ is the observed label. The observed labels are provided in many research benchmark datasets, which are commonly treated as ground truth for supervised training and evaluation. However, our claim is that even these observed labels $Y$ may contain noise and thus do not reflect the true latent labels.

For example, consider an image of a wolf that is visually similar to a husky. A human annotator might mistakenly label it as a husky due to the visual resemblance. In this case, the observed label $Y$ is “husky,” but the true latent label $Z_Y$​ is “wolf.” The similarity in appearance (i.e., the node feature $X$) has caused incorrect labeling. At the same time, the fact that the true latent label is “wolf” explains why the image depicts a wolf. This kind of mechanism implies that $Z_Y$​ causes both $X$ and $Y$ in the data generating process, with $X$ further contributing to the generation of $Y$. Such a causal structure is well-aligned with the literature on instance-dependent label noise [1] and thus conceptually grounded.

However, because benchmark datasets do not provide ground-truth latent labels or explicit annotations of label noise, we design DANG to simulate this realistic noise scenario and study its impact in a controlled setting.

[1] Instance-dependent Label-noise Learning under a Structural Causal Model, NeurIPS 2021

Response to W1 and Q2

Several baseline methods encounter out-of-memory (OOM) errors on large-scale datasets such as ogbn-arxiv primarily due to their reliance on structure learning components. These methods often require computing an explicit 𝑁×𝑁 similarity matrix or graph refinement operations, which incur high memory overhead, where 𝑁 is the number of nodes. In contrast, our method avoids this bottleneck by approximating the regularization term $kl(q_{\phi_1}(Z_A|X, A)||p(Z_A))$ efficiently (See Appendix B). This design significantly reduces the computational and memory cost, enabling DA-GNN to scale to large graphs without OOM issues. Furthermore, we conduct an inference time analysis on the arXiv dataset using only models that do not incur OOM errors. The results are as follows:

• DA-GNN: 0.0599 sec
• RTGNN: 0.0461 sec
• AirGNN: 0.0346 sec
• EvenNet: 0.0027 sec

DA-GNN takes approximately 0.06 seconds for a single inference, which is comparable to RTGNN and AirGNN in terms of inference efficiency. Furthermore, we plan to include a larger-scale graph in our experimental setup to further assess the scalability of DA-GNN.

Response to Q3

Some prior methods adopt regularization ideas similar to ours, but they typically apply only a subset of them or rely on different assumptions. For example, RSGNN employs a regularization approach similar to ours for $Z_A$, but it heavily depends on the assumption of strong feature smoothness. Furthermore, it does not incorporate other regularizations. The reason our model achieves strong performance even under clean settings is that our regularization components are not tailored solely for noisy scenarios. Regularizing $Z_A$ to favor assortative edges is closely related to graph structure learning, which has been shown to benefit GNN performance even when no noise is present. Similarly, our regularization on $Z_Y$ encourages class homophily, a widely accepted inductive bias in graph learning, and our decoder for $A$ applies label smoothing to prevent overfitting. These components work synergistically to improve generalization and robustness. Therefore, even in clean settings, DA-GNN outperforms baselines that do not fully leverage these forms of inductive bias.

Response to Q4

Due to the nature of bag-of-words features, most node features are very sparse, and cases where a node feature is mostly all ones—as mentioned by the reviewer—are extremely rare in real-world scenarios. If we were to flip a fixed percentage of bits based on high-bit-count nodes, this would lead to overly aggressive perturbation for the majority of nodes, which is unrealistic in practical settings. Conversely, if we set the fixed percentage based on low-bit-count nodes, nodes with richer features would receive almost no perturbation. However, since such high-bit-count nodes are rare, we expect the difference between the fixed-percentage approach and our current method to be minimal in practice.

Additionally, in datasets like arXiv that use numerical node features, we inject Gaussian noise with equal magnitude across all nodes. Even under this setting, DA-GNN consistently outperforms baselines, which further supports the robustness of our approach.

Response to Q5

We reported standard deviations alongside point estimates, although for readability we boldfaced only the latter. We agree that including results of statistical significance tests could further clarify performance differences and will consider adding them in a future revision. We would also like to highlight that while this concern may arise under low noise rates, at higher noise levels, DA-GNN consistently outperforms baselines even when accounting for variance.

Thanks for the detailed response to my comments and questions. I particularly appreciate the clarifying example given in response to Q1, in addition to the discussion in Q4 as to appropriate noise mechanism (given the underlying features within the produced examples). I am glad to hear the changes you plan on making with regards to the presentation (both as brought up by myself and some of the other reviewers). I will keep my score as-is in support for the acceptance of the paper.

We sincerely thank the reviewer for their time and thoughtful review of our paper, as well as for recognizing the contributions of our work.

Response to W1

Our intention with the examples in Paragraphs 1–4 was to intuitively illustrate the limitations of existing assumptions and make the introduction accessible to readers from diverse backgrounds. However, we agree that the current version may place too much emphasis on illustrative examples at the expense of theoretical grounding or empirical context.

In the revised version, we will streamline redundant descriptions and expand the discussion of relevant literature and empirical motivations to better position our work within the broader research landscape. Specifically, we plan to incorporate the empirical findings referenced in our response to Reviewer KkeS, Question #1.

We appreciate your suggestion, which has helped us improve the clarity and balance of the introduction.

Response to W2

Thank you for pointing this out. We agree that grounding the discussion of generalizability and applicability with appropriate references is important. To ensure that we fully understand your suggestion and incorporate the most relevant citations, we would greatly appreciate it if you could provide a bit more context on the types of works or specific aspects you believe should be highlighted—particularly in relation to the cited work “Unifying Homophily and Heterophily for Spectral Graph Neural Networks via Triple Filter Ensembles.”

Response to W3

We sincerely apologize for the incomplete phrase. The line the reviewer pointed out should be modified to: "In Fig. 2(b), 𝑋 denotes the node features (potentially noisy), 𝑌 denotes the observed node labels (possibly noisy), 𝐴 denotes the observed edges (which may contain noise), and 𝜖 denotes the environment variable that causes the noise."

In addition to this line, we carefully reviewed and corrected the rest of the text in Section 3. We will also conduct a thorough proofreading of the entire manuscript to ensure clarity, completeness, and grammatical correctness throughout. We greatly appreciate your attention to detail, which has helped us improve the overall quality and readability of the paper.

Response to W4

About description (a). The description (a) means that DA-GNN is explicitly designed to model this dependency structure by introducing latent variables $Z_Y$, $Z_A$, and $\epsilon$, and optimizing the evidence lower bound (ELBO) of the full joint distribution 𝑃(𝑋,𝐴,𝑌).
Specifically, DA-GNN learns the encoder parameters for the latent variables, $\phi=${ $\phi_1, \phi_2, \phi_3$ }, and the decoder parameters for the observable variables, $\theta=$ { $\theta_1, \theta_2, \theta_3$ }, to maximize the ELBO. Through this training process, the encoders and decoders are jointly optimized to model how the observed noisy graph is generated from both latent and observable variables, and to capture the causal relationships that induce noise dependencies. As a result, DA-GNN promotes accurate inference of the latent clean node labels $Z_Y$ and latent clean graph structure $Z_A$, enabling effective performance on node classification and link prediction tasks even in the presence of complex noise dependencies.

About Figure 3. On the left side of Figure 3, we illustrate both the latent clean graph and the observed graph affected by DANG. Importantly, this is not just a subgraph, but a depiction of an observed graph that includes unlabeled nodes, noisy node features, noisy edges, and noisy node labels.

In the latent clean graph, green and blue nodes each represent different classes. Nodes within the same class (e.g., all green or all blue) have similar features and tend to form edges with one another—this reflects a typical property of clean real-world graphs, where class-homophily governs both feature similarity and structural connectivity.

DANG introduces noise into this clean graph by assuming that a latent variable 𝜖 perturbs some node features. In the observed graph, two nodes exhibit noisy features (shown as red blocks), corrupted due to 𝜖. Because these noisy features distort the similarity between nodes, noisy edges (shown as red solid lines) are formed between nodes of different classes (e.g., green and blue), as they now appear deceptively similar.

Furthermore, this feature and structure noise can lead to label noise, where some nodes are incorrectly labeled—represented by circles with a minus sign. This cascading chain of noise dependencies lies at the core of the DANG formulation. It is elaborated in Section 3 (theoretical formulation), Appendix C.1 (conceptual examples), and Appendix E.2 (empirical implementation details).

We hope this response sufficiently addresses the reviewer’s concerns and clarifies the underlying motivation and formulation.

We sincerely thank the reviewer for their time and thoughtful review of our paper.

Response to W1

In response to the reviewer’s comment, we conduct experiments on adaptive attacks. Following [1], we implement adaptive PGD attacks on NRGNN, RSGNN, RTGNN, SG-GSR, and DA-GNN using a 10% attack budget. As shown in the table below, DA-GNN demonstrates comparable or superior adversarial robustness against adaptive attacks compared to the baselines.

[1] Are Defenses for Graph Neural Networks Robust?, NeurIPS 2022

Response to W2

In the generation process of our synthetic DANG, we have three variables: 1) the overall noise rate, 2) the amount of noise dependency ($X \rightarrow A, X\rightarrow Y, A \rightarrow Y$), and 3) the amount of independent structure noise ($\epsilon \rightarrow A$).

• For the first variable, our experiments already addressed it by varying the noise rate from 0 to 50.
• For the second variable, we conduct an additional analysis by substantially increasing or decreasing the degree of noise dependency. Specifically, we increase the number of structure noise edges caused by feature noise by approximately 4×, and similarly amplify the amount of label noise induced by both feature and structure noise by 4×. We also evaluate a setting where noise dependencies are completely removed—this corresponds to a scenario with independent feature and structure noise. As shown in Table 1 and Table 2, DA-GNN consistently outperforms all baselines on the strong presence of noise dependency, and shows competitive performance on the weak presence of noise dependency.

Table 1. Results on DANG with increased noise dependency

Table 2. Results on DANG without noise dependency

• For the third variable, we perform an additional analysis by doubling the amount of independent structure noise. We also evaluate the case where no independent structure noise is present. As shown in Table 3 and Table 4, DA-GNN consistently outperforms all baselines across both settings.

Table 3. Results on DANG without independent structure noise

Table 4. Results on DANG with increased independent structure noise

These results demonstrate that DA-GNN consistently outperforms other baselines under varying degrees of DANG, highlighting its practical applicability across diverse real-world noise conditions.

Response to W3

We followed the transductive setting with training-time noise in features, edges, and labels. There are no leaked clean features, edges, and labels in our setting because our setup fundamentally differs from the transductive setting with test-time attack, which is mentioned in [2] and [3]. Specifically, [2] claims that a model can achieve perfect robustness by memorizing the testing nodes and their clean structures, which are leaked during training.

However, our setting assumes that the given graph already incorporates the noise before any training begins. The model never has access to the clean graph at any point. Hence, memorizing the noisy graph is not advantageous in achieving robustness. Instead, our model must learn to identify and handle noise patterns rather than memorize clean structures.

[2] Gosch et al. “Adversarial Training for Graph Neural Networks: Pitfalls, Solutions, and New Directions” NeurIPS 2023

[3] Zheng et al. “Graph Robustness Benchmark: Benchmarking the Adversarial Robustness of Graph Machine Learning” NeurIPS 2021

Response to Q1

Clean features, edges, and labels are NOT available at all during training. Our model is trained entirely on the noisy graph - we add 10%, 30%, or 50% noise to features, structure, and labels before training begins, and the model never sees the clean version. This ensures no information leakage and forces the model to learn genuine denoising capabilities.

Response to Q2

We have considered the scenario where structural noise in the graph influences the node features, as the reviewer suggested, and we discuss this in Appendix C.2. Exploring more diverse constructions of noise models—including those in which feature noise is explicitly dependent on the graph structure—remains an interesting direction for future work.