Probabilistic Graphical Model for Robust Graph Neural Networks against Noisy Labels

1. The experiment is not sufficient. More baselines published in 2022-2023 should be included, e.g., [1,2].
2. More ablation studies in different noise rates should be included. Now the effectiveness of each compotent is not clear.
3. The motivation of using a probabilistic graphical model for this problem is not clear. What's the key to bring in performance increasement? The proposed methods seems to combine a lot existing things, prototypes, contrastive learning, weight learning.

[1] Learning on Graphs under Label Noise, ICASSP 23. [2] Robust Training of Graph Neural Networks via Noise Governance, WSDM 23.

• Comparison with CausalNL Method. The paper presents a methodology that appears conceptually similar to CausalNL (presented at NeurIPS 21), which uses a generative process to correct noisy labels. However, while CausalNL deals with image data, this paper focuses on graph data. The key concern here is that, aside from the data type, the proposed method may not sufficiently distinguish itself from CausalNL in terms of the underlying principles and approach. A more explicit and thorough comparison is needed to highlight the novelty and advancements of the proposed method over existing solutions like CausalNL.

• Counterintuitive Generative Process. The generative process, as depicted in Figures 1(a) and 1(b), posits that noisy labels ($Y_N$) are causes of clean labels ($Y_C$), which is counterintuitive and lacks practical examples or theoretical justification. This assumption contradicts conventional understanding where clean labels typically precede and are corrupted into noisy labels, not the other way around. The paper should offer a clear rationale or model explaining how and why noisy labels can causally lead to clean labels in this context.

• Lack of Clarity in Generative vs. Inference Processes. The paper seems to mislabel certain processes, like referring to the encoder as part of the generative process when it actually pertains to inference. This confusion undermines the clarity of the methodological framework. The authors need to distinguish between the generative and inference components of their model more clearly to avoid contradictory statements and ensure the coherence of their proposed method.

• Insufficient Self-Containment and Explanation of Methodology. The paper lacks self-contained explanations, especially in deriving important elements like the Evidence Lower Bound (ELBO). These critical steps are relegated to the appendix, making it difficult for readers to follow the methodology's theoretical foundation. Additionally, the application of techniques like co-teaching+ to graph data is not explained, leaving gaps in the readers' understanding of how the methodology is adapted for the graph context.

• Inadequate Experimental Comparison and Marginal Improvements. The experimental section does not provide a robust comparative analysis. Specifically, 5 out of 6 baselines do not use a clean label set, which might not be an apples-to-apples comparison. Additionally, the improvements shown by PRGNN-v1 and PRGNN-v2 are marginal and similar to the methods that don't use clean labels. This raises questions about the practical effectiveness and the added value of the proposed approach.

• Limited Discussion on Theoretical Underpinnings. The paper does not delve deeply into the theoretical aspects that underlie the proposed framework. A more rigorous discussion on the probabilistic and Bayesian foundations specific to graph neural networks and how they effectively address noisy labels is needed. This would provide a stronger theoretical base for the paper and justify its approach more convincingly.

Post Discussion Suggestion:

Dear authors,

I believe that the two weaknesses, Counterintuitive Generative Process and Clarity in Generative vs. Inference Processes, I mentioned above are very important. Please kindly consider addressing these two in the future version, which I believe will greatly improve your paper.

1. The proposed framework seems to unreasonably complex to me. The motivation of using PGM and Bayesian framework is not clear and convinced.
2. To address the problem of graph heterophily, the proposed solution is manually change suitable backbones. This is a great application of existing heterophilious GNNs. However, the technique contribution in addressing graph heterophily with label noises is marginal.
3. The improvements that the complex proposed method can bring seem marginal. For example, NRGNN often achieve comparable results with the PRGNN on homophilious graphs

1. Several crucial points remain unclear. First, it is still hard to understand why we need the uncertainty modeling. One might directly apply the encoder-decoder models equipped with the corresponding losses to handle the noisy labels. To be specific, what happens if we only use Q and P_\varphi without the prior distribution, and conduct the training without the prior-related losses? The second question is related. From the experiments in Figure 3, the weighted cross-entropy loss wL_CE(P_\varphi, Y_N) is much more important than others. So, it seems it is much more important to balance the loss between reconstruction of clean labels and that of noisy labels, other than the probabilistic modelling particularly given the label space (which is the random variable we focus on) is of small dimension. Third, the authors claim that the proposed probabilistic graphical model is able to avoid the issue of label smoothness in existing methods. However, it seems this is attributed to the application of prototype label vectors, instead of simple concatenation of labels with node embeddings for message passing. This is also observed in Figure 3. Overall, to make the proposed models effective, the authors introduce various techniques. However, it appears that the one which demonstrates consistent success is not aligned with the contributions claimed by the authors.

2. About the introduction of the baselines, it is unknown which one applies label smoothness, and which one is proposed for Heterophilous graphs.

3. It seems this paper is only applicable for the transductive setting not inductive setting.

4. The last question is on the general research domain for learning from noisy graphs. While it is true that many data we process in practice definitely contain different kinds of noise, the proposed methods are always evaluated in the scenarios where we add artificial perturbation to the clean data. My point is we actually need some real tasks and perhaps some fixed benchmarks to evaluate the performance of different methods, such that we can clearly justify how the progress is made and how the challenges are addressed.