Enhancing the Influence of Labels on Unlabeled Nodes in Graph Convolutional Networks

Thanks for the positive comments. We are so encouraged and will try our best to address the concerns one by one. All changes here will be found in the final version.

Q1: Lack of baselines. Since the authors designed a new contrastive learning method, it is necessary to compare it with the contrastive learning method.

A1: We acknowledge the importance of comparing our method with existing contrastive learning methods. To address this, we compared the typical graph contrastive learning GRANCE[1] and the recent graph contrastive learning method (SGCL[2]). We can observe that our method consistently outperforms the two contrastive learning baselines across all datasets.

[1] Deep graph contrastive representation learning. ICML'20

[2] Rethinking and Simplifying Bootstrapped Graph Latents. WSDM'24

Q2: The discussion on contrastive learning in graphs is somewhat lacking. The proposed Eq. (12) is a paradigm of contrastive learning. The author should further discuss its difference and connection with contrastive learning loss, such as InfoNCE.

A2: A key difference is that in InfoNCE, the same node in different graph views is treated as a positive sample, while different nodes are treated as negative samples. In contrast, Eq. (12) distinguishes ELU nodes and NELU nodes, ensuring that ELU nodes maintain consistency across graphs while NELU nodes are pushed apart. This means that Eq. (12) incorporates structural information specific to label influence in GCNs, rather than relying solely on node (instance) discrimination as in InfoNCE. We will clarify this point in the revised version.

Q3: The proposed method seems a bit complicated, therefore I am concerned about its reproducibility.

A3: In fact, our method is simple to implement in practice. The computation of the ELU-graph is a parameter-free process, and the Woodbury trick significantly accelerates matrix operations. Moreover, we have already provided the code link in the supplementary material and commit to open-sourcing it to ensure full reproducibility.

Comment: There are some minor errors that need to be carefully checked, such as whether the comma or period at the end of the formula is correct.

Thank you for your careful review. We will thoroughly check and correct any minor errors, including punctuation issues in the formulas.

Comment: The paper directly mentions LPA and GCN, but there is no introduction to them. The author should briefly introduce them in the notation or appendix.

Thank you for your suggestion. We will add brief introductions to LPA and GCN in the notation section or appendix to improve clarity.

Thanks for the positive comments. We are so encouraged and will try our best to address the concerns one by one. All changes here will be found in the final version.

Q1: The core idea is to ensure that the prediction of GCN is consistent with the output of LPA. However, this may be problematic—Why not constrain the GCN output to align with the LPA during training? Moreover, this approach does not seem to be effective.

A1: First, we would like to clarify that our method does not simply force GCN predictions to align with LPA; rather, LPA outputs should also align with GCN predictions. Therefore, directly constraining the GCN output to match LPA is inappropriate. Instead, our approach aims to find a new graph structure that naturally ensures consistency between GCN and LPA outputs, allowing the prediction of GCN to utilize label information effectively (see Section 2.1).

Q2: Some details of the experimental settings are missing, such as learning rate, weight decay, number of hidden units, etc.

A2: We set the weight decay to 5e-4. The learning rate was selected from [0.01, 0.02] for all datasets, and the number of hidden units was chosen from [4, 8, 64, 128]. We will include these details in the revised version of the paper to ensure clarity and completeness.

Q3: The subscript notation of $\mathbf{Y}$ is confusing. In Notation, $\mathbf{Y}_{l}$ is defined as the training set label, and then $\mathbf{Y}$ becomes in Eq. (13) and Theorem 2.3. Please check and unify.

A3: Thank you for your careful observation. We acknowledge the inconsistency in the subscript notation of $\mathbf{Y}$. In the notation section, $\mathbf{Y} _{l}$ refers to the training set labels, and the same meaning should be maintained in Eq. (13) and Theorem 2.3. We will revise the notation to ensure consistency and clarity in the next version.

Q4: The $\mathbf{S}^{*}$ calculated by Eq. (11) looks like a dense matrix (almost no zeros), so it consumes a lot of memory and time for subsequent matrix multiplication.

A4: You are correct that $\mathbf{S}^*$ computed from Eq. (11) is a dense matrix, which could lead to high memory and computational costs for subsequent matrix multiplications. However, we apply a sparsification process to $\mathbf{S}^*$ to mitigate this issue. Specifically, $\forall i,j$ where $|\mathbf{S}^* _{i,j} | < \eta$, we set $|\mathbf{S}^* _{i,j} | = 0$, while elements with $|\mathbf{S}^{*} _{i,j} | > \eta$ remain unchanged, where $\eta$ is a non-negative parameter that we usually set to correspond to the top 10 percent of element values. The details of this process are already provided in the appendix.

Q5: The authors mention heterophily graphs but do not give any definition.

A5: We acknowledge the omission and will provide a clear definition of heterophily graphs in the revised version. Generally, heterophily graphs refer to graphs where connected nodes tend to have different labels, in contrast to homophily graphs where connected nodes are more likely to share the same label.

Q6: I would like to ask whether this framework can be used for GAT.

A6: Although our method is specifically designed for GCN, we have added experiments applying the ELU graph to GAT.

The results show that the ELU graph also positively impacts GAT, demonstrating its effectiveness for GAT as well.

Thanks for the positive comments. We are so encouraged and will try our best to address the concerns one by one. All changes here will be found in the final version.

Q1: The real SOTA GNN mode such as GCNII[1] needs to be compared. [1] Chen M, Wei Z, Huang Z, et al. Simple and deep graph convolutional networks.ICML.

A1: We have added a comparison with GCNII in the updated experiments as following table. The results show that, on the Cora dataset, GCNII performs a little bit better, but on the other six datasets, our method outperforms GCNII. We will include these results in the next version for further clarification.

Q2: Figure 2 gives the experimental conclusion very abruptly. The specific experimental details of Figure 2 need to be supplemented.

A2: Thank you for your suggestion. The experimental setup for Figure 2 follows the same configuration outlined in Section 3. The specific details of the experimental setup can be found in the appendix. We will enhance the explanation of this section in the next version to provide more clarity.

Q3: In Eq. 12, it is necessary to explain what specific distance function is used and why it is chosen.

A3: In Eq. 12, various distance functions, such as Euclidean distance and inner product, can be considered. Based on experimental results, we found that Euclidean distance performed better in our case.

Q4: From Theorem 2.3, can this also be understood as: if an adjacency matrix A makes LPA (AY) perform better, then it would also be better for GCN? Would this inspire the use of a more lightweight and parameter-free LPA instead of GCN to find a better graph structure?

A4: Yes, I agree with your opinion. Based on Theorem 2.3, if a graph structure $\mathbf{A}$ enables LPA to achieve better performance, it means that GCN can achieve better generalization ability on this graph structure $\mathbf{A}$. This suggests that a lightweight and parameter-free LPA can serve as a criterion for evaluating graph structures or even as an objective function in graph structure learning. In fact, our method can also be seen as optimizing the graph structure $\mathbf{A}$ to improve LPA performance as much as possible.

Thanks for the positive comments. We are so encouraged and will try our best to address the concerns one by one. All changes here will be found in the final version.

Q1: Limitation on theoretical part: the $\mathbf{Y} _{true}$ is actually unknown to us, thus this may limit the applicability of the theory.

A1: It is true that $\mathbf{Y} _{true}$ is unknown to us. However, we can approximate $\mathbf{Y} _{true}$ using the pseudo-labels predicted by GCN, a common practice in many existing works [1]. This approximation allows us to learn an adjacency matrix $\mathbf{A}$ that ensures strong generalization by Theorem 2.3. By then applying GCN to this learned $\mathbf{A}$, we can obtain higher-quality pseudo-labels that more closely approximate $\mathbf{Y} _{true}$. Through this iterative process, the learned ELU graph progressively refines $\mathbf{A}$ and pseudo-labels, approaching the optimal structure for generalization.

[1] Calibrating graph neural networks from a data-centric perspective. WWW'24

Q2: When computing the ELU graph, the paper employs a variant of GCN in the form of Eq.4. However, it remains unclear whether the same form of GCN is also utilized during the contrastive learning phase. This aspect necessitates further clarification.

A2: We ultimately use the standard GCN formulation (Kipf & Welling, 2017) in the contrastive learning phase. The variant of GCN in Eq. 4 is introduced solely to facilitate the optimization of the objective function in Eq. 5.

Q3: To validate the effectiveness of the ELU graph, the authors should compare it with alternative graph construction methods, such as the KNN-graph, by replacing the ELU graph in the ELU-GCN and reporting the corresponding results.

A3: The effectiveness of the ELU graph is demonstrated by its ability to make unlabeled nodes use the label information more effectively while enhancing generalization. However, the KNN graph only considers the feature information, which ignores the impact of labels. To further validate this, we have added an additional experiment where the ELU graph in ELU-GCN is replaced with the KNN graph for further validation. The results are shown in the table below.

It is evident that replacing the ELU graph with the KNN graph leads to a significant performance drop. This confirms that the ELU graph effectively facilitates the utilization of label information by unlabeled nodes, whereas the KNN graph, which relies solely on feature similarity, fails to capture this crucial aspect.

Q4: The improvement of NELU nodes after using ELU-GCN should be reported.

A4: We have already conducted this experiment, and the results are presented in Figure 7 of the appendix. This figure illustrates the improvement of NELU nodes after using ELU-GCN.

Q5: Why can't the two-stage framework proposed in the paper be designed as an end-to-end framework?

A5: While our framework follows a two-stage design, the first stage—constructing the ELU graph—is a parameter-free process. Since this stage does not involve learnable parameters, it cannot be seamlessly integrated into an end-to-end framework. Overall, the ELU graph is firstly precomputed and then used to enhance GCN training in the second stage. This parameter-free approach ensures stability and efficiency in graph construction while keeping the model focused on learning meaningful representations during the second stage. In our future work, we plan to make our framework be an end-to-end one.