Nonparametric Teaching for Graph Property Learners

Thanks for many constructive comments. We are deeply appreciative of the reviewer’s efforts to help us improve our paper. We take all comments seriously and try our best to address every raised concern. We sincerely hope that our response can resolve your concerns. Any follow-up questions are welcome.

[Theoretical Claims] Sorry for the confusion. We would like to clarify that the theoretical analysis does not rely on the GCN being limited to two layers; it can be of any depth. Additionally, our empirical validation includes experiments with different depths and widths, as shown in Table 4. We agree with the reviewer that expanding the theoretical framework and empirical validation to more recent architectures would enhance the broader applicability of the work. We also believe that the concept and analysis behind GraNT have significant potential for wider extensions. Analyzing GCNs is an important foundational step in applying nonparametric teaching for graph property learning, which paves the way for further advancements. However, this is beyond the scope of the current paper and will be explored in future research. In response to your inquiry, we have applied the GraNT algorithm to GIN [1] on the ogbg-molhiv dataset and will provide further discussion on this in the revised version.

[Experimental Designs Or Analyses] Thank you for your constructive feedback. We present comparisons of GraNT with recent works [2-6] below. Our results show that, with theoretical justification, GraNT outperforms these methods in terms of both training time and specific metrics. We will provide further discussion on this in the revised version to better emphasize the context and contributions of our work.

ogbg-molhiv

$^\alpha$: batch_size=500, retain_ratio=0.7.

QM9

$^\gamma$: The training time is calculated as time_per_epoch * epochs = 92 * 750. The MAE value reported is from the paper.

$^\delta$: lr=5e-5, batch_size = 256, which matches our settings.

$^\zeta$: lr=5e-4, batch_size = 256.

$^\tau$: lr=5e-4, batch_size = 32, which corresponds to the default settings used in the provided code for that paper.

[1] How powerful are graph neural networks?. – ICLR’19.

[2] Graph mixture of experts: Learning on large-scale graphs with explicit diversity modeling. – NeurIPS’23.

[3] GDeR: Safeguarding efficiency, balancing, and robustness via prototypical graph pruning. – NeurIPS’24.

[4] Semi-supervised classification with graph convolutional networks. – ICLR’17.

[5] Graph Prompting for 3D Molecular Graphs. - CIKM’24.

[6] Empowering Active Learning for 3D Molecular Graphs with Geometric Graph Isomorphism. – NeurIPS’24.

[Essential References Not Discussed] Sorry for the confusion. We would like to clarify that the discussion on learning efficiency (Chen et al., 2018; Liu et al., 2022b; Zhang et al., 2023c) is included in the Related Works section, covering topics like normalization (Cai et al., 2021), graph decomposition (Xue et al., 2023), and lazy updates (Narayanan et al., 2022). We will enhance the presentation and include more detailed discussion in the revised version.

[W1] Thank you for your suggestions. We would like to clarify that this paper is algorithmic in nature, introducing GraNT, a novel paradigm that improves the learning efficiency of graph property learners (GCNs) through nonparametric teaching. Our implementation leverages the PyG graph library, which is designed to optimize compilation efficiency, a concern that is independent of the main focus of our work.

[W2] We have included the pseudocode in Algorithm 1, which outlines the core idea of our implementation, and our experiments are based on this. Detailed experimental settings and implementation specifics can be found in Appendix C. The code and data will be made available.

Thanks for the encouraging comments and constructive suggestions. We sincerely thank the reviewer's efforts for helping us improve the paper. We hope that our response resolves your concerns.

[W3] Very thoughtful question! We believe the idea and analysis behind GraNT have significant potential for broader extensions. As a fundamental neural network architecture for graph property learning, analyzing GCNs is an essential step in applying nonparametric teaching within this context before exploring further extensions. As mentioned in the Concluding Remarks and Future Work section, exploring other GNNs beyond GCN could be an intriguing direction. This could involve examining the core mechanisms of specific GNNs through the GNTK theory. Some relevant works on GNTK that might be insightful are [1,2]. However, this is outside the scope of the current paper and will be addressed in future work. In response to your curiosity, we have applied the GraNT algorithm to GIN [3] on the ogbg-molhiv dataset and will include additional discussion on this in the revised version.

[1] Graph neural tangent kernel: Convergence on large graphs. – ICML’23.

[2] Graph neural tangent kernel: Fusing graph neural networks with graph kernels. -NeurIPS’19.

[3] How powerful are graph neural networks?. – ICLR’19.

Thanks for the useful comments. We are deeply appreciative of the reviewer’s efforts to improve our paper. We take all comments seriously and try our best to address every raised concern. We sincerely hope that our response resolves your concerns.

[W1] Thank you for your helpful feedback in improving our paper. Below, we present a comparison of GraNT with the recent works GMoE [1] and GDeR [2]. Our results demonstrate that GraNT outperforms these approaches in both training time and specific metrics.

We will include a more detailed discussion of this comparison in the revision.

ogbg-molhiv

$^\alpha$: batch_size=500, retain_ratio=0.7.

QM9

$^\gamma$: The training time is calculated as time_per_epoch * epochs = 92 * 750. The MAE value reported is from the paper.

$^\delta$: lr=5e-5, batch_size = 256, which matches our settings.

$^\zeta$: lr=5e-4, batch_size = 256.

$^\tau$: lr=5e-4, batch_size = 32, which corresponds to the default settings used in the provided code for that paper.

[1] Graph mixture of experts: Learning on large-scale graphs with explicit diversity modeling. – NeurIPS’23.

[2] GDeR: Safeguarding efficiency, balancing, and robustness via prototypical graph pruning. – NeurIPS’24.

[3] Semi-supervised classification with graph convolutional networks. – ICLR’17.

[4] Graph Prompting for 3D Molecular Graphs. - CIKM’24.

[5] Empowering Active Learning for 3D Molecular Graphs with Geometric Graph Isomorphism. – NeurIPS’24.

[W2] Thanks for your help in improving our paper. We have included the results with standard deviations for all validations. This demonstrates that the proposed GraNT not only achieves training time efficiency with theoretical justification but also exhibits stability. We will fix it.

[W3] Thanks for your help in improving our paper. The analysis of the hyperparameter, start ratio, under GraNT(B) is outlined below:

We will include a more detailed discussion in the revision.

[Typo] Thanks for pointing it out. We will fix it in the revision.