Let Your Features Tell The Differences: Understanding Graph Convolution By Feature Splitting

Q4: I am curious about the robustness of the proposed method when applied to noisy graphs, where certain features may contain varying levels of noise. Could the authors provide insights or results on its performance in these scenarios?"

We conduct additional experiments to investigate the impact of feature noise on GFS on four real-world datasets: Children, Computer, Roman-Empire, and Tolokers, by introducing noises to the normalized original node features. Specifically, after normalizing the input features, we add Gaussian noise $\epsilon \sim N(0,1)$ for multiple times. As shown in Table 19 and Figure 17, the relative increase $\Delta$ (defined as the accuracy gap between GCN+GFS and GCN, divided by the accuracy of GCN) remains relatively stable, indicating that the performance of GFS is robust against feature noise. We also observe that the superiority of GFS gradually diminishes as feature noise levels increase. We speculate that this is due to the added noise reducing the informativeness of all features, leaving effective information primarily derived from graph topology. As a result, all features may become unsuitable for graph-agnostic models, collapsing into features favored by GNNs, which diminishes the effectiveness of GFS. However, it is important to note that adding noise to all features more than 20 times is uncommon in real-world scenarios. Therefore, GFS remains effective on the majority of datasets, as demonstrated in Table 1. Thank you very much for your valuable suggestions.

[1] Ma Y, Liu X, Shah N, et al. Is homophily a necessity for graph neural networks?[J]. arXiv preprint arXiv:2106.06134, 2021.

[2] Luan S, Hua C, Xu M, et al. When do graph neural networks help with node classification? investigating

[3] Feature Selection: A Data Perspective. ACM Comput. Surv. 50(6).

Q1: I am still unclear about how TFI fundamentally differs from other mutual information-based metrics for feature selection in graph learning. Could the authors provide a clearer explanation of its unique contributions and advantages?

In conclusion, previous studies on mutual information (MI) have primarily focused on selecting informative features or reducing redundancy for i.i.d. data in graph-agnostic models (MLP). In contrast, our approach emphasizes distinguishing between features favored by graph-agnostic models (MLP) and those favored by graph-aware models (GNN), which is essential for effective graph representation learning. For more details, please refer to our detailed response to W2.

Q2: How does the proposed method handle extremely sparse features, which are common in real-world graphs? Are there specific challenges or limitations in this context?

Thank you for your constructive comments. We further conduct experiments to investigate the impact of feature sparseness on the CSBM-H model [1,2], following the setup outlined in Section D.6 (or responses to W3). To control sparseness in synthetic datasets, we randomly mask a certain percentage of node features. As shown in Table 18 and Figure 16, the results indicate that the advantage of GFS diminishes as feature sparseness increases. Although the relative increase rate of GFS decreases in this context, it still outperforms GCN. This is because GNN-favored and GNN-disfavored features still exist under varying levels of sparseness, which allows GFS to remain effective in enhancing GNN performance.

Q3: It appears that the proposed method offers limited advantages on homophilous graphs compared to its performance on heterophilous graphs. Could the authors provide insights to clarify this discrepancy and explain why the method may be less effective in homophilous settings?

We agree with your opinion. From both the experiments from real-world datasets (Table 1) and synthetic datasets (Figure 14 and Table 16), we observe that GFS has stronger superior on baseline GNNs. We think this phenomenon mainly comes from two reasons:

1. Limited potential of increase. The GNN performance becomes better with the increase of label homophily, lefting limited space for improving model performance. A graph with high label homophily indicates that all the nodes group into several class clusters, making it easier to correctly to classify nodes in graph. Give such an easy task, it is not surprising that GFS derives limited superiority.
2. Most features are GNN-favored for some datasets. As shown in Figure 4, for homophilous graphs, such as Tolokers, most features are GNN-favored, making GFS have limited increase on model performance. However, this case is very few in all the datasets.

Even if the improvement of GFS, as shown in Figure 4, on heterophilous graphs (Computers and Tolokers) is not as significant as homophilous graphs (Children and Roman-empire), it still improves the performance of baseline GNN with a higher ratio of $r$ since there still exist some features that are GNN-disfavored. This indicates the necessity of applying GFS in graphs, which is also verified by additional results on synthetic datasets in Figure 14 and Table 16.

W3: 1)Although the experimental results demonstrate strong performance of the GFS-augmented models, I remain somewhat unconvinced. 2)To strengthen the validity of the findings, it would be helpful to include experiments on synthetic datasets, such as those generated by the SBM model with varying levels of heterophily and homophily. Such experiments could offer more reliable insights into the potential advantages of the proposed methodology.

First, our proposed GFS is simple yet effective in boosting GNN performance without the need for hyperparameter tuning on most datasets, particularly when features are encoded by Pretrained Language Models (PLMs). This simple method can be easily incorporated into various GNN-related models to enhance performance. Welcome to run our code, which available in the supplementary material or at the anonymous link https://anonymous.4open.science/r/graph-feature-selection-BF28, to verify the effectiveness of GFS.

Second, to further investigate the impact of homophily on GFS, we conducted experiments on synthetic datasets using CSBM-H [1,2] to control homophily levels. Specifically, in the CSBM-H model, for a node $u$ labeled with $y$, its features are sampled from a class-wise Gaussian distribution. Each dimension of $\mathbf{X}_u$ is independent of one another. To construct the graph structure $\mathcal{G}$ with a specified homophily degree $h$, node $u$ has a probability $h$ of connecting to nodes with the same label and a probability of $\frac{1-h}{C-1}$ of connecting to nodes with a different label. We randomly generated 10 graphs with different seeds to reduce uncertainty. Each graph contains 1,000 nodes, 10 features, and 5 classes, with node degrees uniformly sampled from the range $[2, 8]$.

We demonstrate how the performance of GCN+GFS varies with label homophily across the range $[0.1, 0.2, \ldots, 0.9]$ in Table 16 and Figure 14 for node classification tasks. Generally, GCN+GFS outperforms both GCN and MLP across various levels of label homophily, indicating that GFS effectively addresses the limitations of GCN under low homophily and of MLP under high homophily through effective feature selection. Furthermore, the relative increase $\Delta$ (defined as the accuracy gap between GCN+GFS and GCN, divided by the accuracy of GCN) decreases as label homophily increases. This finding is expected, as higher homophily levels typically enhance the performance of all graph-aware models, thereby limiting the potential for improvement through GFS. Furthermore, this conclusion is also verified by our experiments on real-world datasets, including both the homophilous and heterophilous graphs, as shown in Table 1.

Thank you for your insightful suggestions on synthetic datasets.

W1: As far as I am concerned, the theoretical analysis of the proposed strategy is incomplete. The paper does not clearly explain the specific conditions under which distinguishing between GNN-favored and GNN-disfavored features is most effective.

Thank you for your valuable comments. We do not specify those conditions because features that are either GNN-favored or GNN-disfavored are commonly found in any graph. Only in trivial cases, where all features in a graph are either GNN-favored or GNN-disfavored, is feature selection unnecessary. In all other scenarios, it is essential to perform Graph Feature Selection (GFS) to enhance GNN performance, as demonstrated in our experiments.

Our theoretical analysis demonstrates that TFI acts as an upper bound on the performance gap between graph-aware and graph-agnostic models, establishing the foundation for GFS. While it would be advantageous to theoretically quantify the effectiveness of GFS under specific conditions to facilitate the automatic selection of the optimal threshold rrr, the complexity of measuring the interplay of multiple features during the optimization process of GNNs and MLPs poses significant challenges. We acknowledge this limitation in Section 6. Nonetheless, GFS has already shown substantial improvements across most datasets, and we believe it would be valuable to explore these conditions further in future research.

W2: The proposed TFI metric, while beneficial, appears similar to existing mutual information-based methods. The authors need to provide more clarification on how TFI uniquely contributes to feature selection in graph learning, emphasizing its distinct advantages over prior approaches.

As discussed in Appendix A, we highlight the unique contribution of TFI in comparison to other metrics:

1. Most existing feature selection metrics [3] are designed for non-graph data, making them unsuitable for evaluating the informativeness of features within graphs.
2. Traditional methods for feature assessment in graphs rely on consistency-based metrics, which suffer from "good heterophily" issues [1,2] and do not align well with GNN performance.
3. TFI stands out as the first method to identify GNN-favored and GNN-disfavored features, effectively reflecting the performance gap between graph-aware and graph-agnostic models, as demonstrated by both theoretical analysis and empirical studies.

We have revised our manuscript to highlight the superiority of TFI over other traditional metrics. Thanks for you suggestions.

Please let me know if our responses have addressed your concerns. Thank you!

Q4: Is it possible to measure what a "good heterophily" feature looks like? Can other metrics be used for supplementary description, such as Label Homophily, Feature Homophily, Mutual Information, and so on?

"Good heterophily" [6,7] refers to the phenomenon that traditional consistency-based homophily metrics, such as attribute homophily [9], that cannot align well the GNN performance, reflecting in a good GNN performance under a low homophily value. We further conduct experiments to explain the "good heterophily". First, we split node features into $10$ bins according to the values of feature homophily. Then, we run GCN on these bins separately to see how the model performance changes with feature homophily. As shown in Table 20 and Figure 18, the GCN performance remains good under a low value of feature homophily, which includes attribute homophily [9] ($h_{attr}$), local similarity [10] ($h_{sim-euc}$), and class-controlled feature homophily [8] ($h_{CTF}$). This result indicates these consistency-based feature homophily cannot align well with GNN performance, which is similar to the phenomenon in label homophily [6,7]. Conversely, GNN performance consistently increases with the increase of our proposed TFI, implying its effectiveness in selecting GNN-favored or GNN-disfavored features. Thank you once again for your suggestions. We have revised our paper accordingly.

[1] Torch Geometric Datasets - pytorch_geometric documentation.

[2] DGL Datasets - DGL 2.2.1 documentation.

[3] Grover A, Leskovec J. node2vec: Scalable feature learning for networks[C]//Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining. 2016: 855-864.

[4] Chen F, Wang Y C, Wang B, et al. Graph representation learning: a survey[J]. APSIPA Transactions on Signal and Information Processing, 2020, 9: e15.

[5] Bordes A, Usunier N, Garcia-Duran A, et al. Translating embeddings for modeling multi-relational data[J]. Advances in neural information processing systems, 2013, 26.

[6] Ma Y, Liu X, Shah N, et al. Is homophily a necessity for graph neural networks?[J]. arXiv preprint arXiv:2106.06134, 2021.

[7] Luan S, Hua C, Xu M, et al. When do graph neural networks help with node classification? investigating the homophily principle on node distinguishability[J]. Advances in Neural Information Processing Systems, 2024, 36.

[8] Soo Yong Lee, Sunwoo Kim, Fanchen Bu, Jaemin Yoo, Jiliang Tang, and Kijung Shin. Feature distribution on graph topology mediates the effect of graph convolution: Homophily perspective. International Conference on Machine Learning, 2024

[9] Liang Yang, Mengzhe Li, Liyang Liu, Chuan Wang, Xiaochun Cao, Yuanfang Guo, et al. Diverse message passing for attribute with heterophily. Advances in Neural Information Processing Systems, 34:4751–4763, 2021b.

[10] Yuhan Chen, Yihong Luo, Jing Tang, Liang Yang, Siya Qiu, Chuan Wang, and Xiaochun Cao. Lsgnn: towards general graph neural network in node classification by local similarity. arXiv preprint arXiv:2305.04225, 2023

Q2: What impact does the dimension of the initial node features have?

Sorry, we are unclear about your question. Are you referring to the impact of the number of feature dimensions? This is a good question. The number of feature dimensions does influence the performance of GFS. In Table 1, we observe that datasets with a higher number of features, such as Children (dim=768), Computer (dim=768), and Fitness (dim=768), show greater improvements in GFS performance compared to datasets with fewer features, like Question (dim=301), Roman-empire (dim=300), and Tolokers (dim=10).

Additionally, we conducted experiments to further investigate the impact of feature dimension size on synthetic datasets. As shown in Table 17 and Figure 15, GFS demonstrates a pronounced advantage over GCN as the feature dimension increases. We speculate that this is due to the higher error tolerance of GFS when more features are available, even if some features are misclassified as GNN-favored or GNN-disfavored.

It is important to note that most datasets contain hundreds of features, which enhances GFS's robustness and enables it to outperform baseline GNN models, as demonstrated in Table 1. Furthermore, for datasets with fewer features, GFS performance can still be improved by leveraging pretrained node embeddings to increase the feature count. This is illustrated in Figure 7, where GFS performance on Question, Roman-empire, and Tolokers shows significant enhancement. Thank you for your valuable insights. We have revised the paper accordingly.

Q3: Can this method only be applied to node features, or can it also be used on edge features, and what would be the effect?

Thank you for your suggestions. We did not apply GFS to edge features for the following reasons:

1. Lack of Datasets: To the best of our knowledge, there are no graph datasets that incorporate edge features for node classification, as demonstrated in most widely used GNN libraries [1, 2].
2. Limited GNN Designs: Consequently, there are not many GNNs designed to handle this type of graph.

For the scenario you mentioned, we believe knowledge graphs are more suitable, as edge features are commonly found in them. In the future, it would be interesting to extend the aggregated node features from $\frac{\sum_{j\in\mathcal{N}_i} X_j}{|\mathcal{N}_i|}$ to $\frac{\sum_{j\in\mathcal{N}_i} f(X_j+r_{ij},X_i)}{|\mathcal{N}_i|}$ [5] to incorporate edge attributes. However, it is important to note that knowledge graphs differ significantly from the pure graphs we studied in this paper. Our main focus is to enhance representation learning on graphs, where performance is typically evaluated through node classification [3, 4]. We follow this setting in our experiments.

W1: The paper does not provide any statistical analysis of the selected features, which raises some questions.

Thank you for your valuable feedback. We agree that a more detailed statistical analysis of these features would be beneficial. In our submission, we provide an analysis of the selected features:

• In Figures 4 and 10, we highlight the ratio of selected features in GFS.
• Figure 8 demonstrates that swapping GNN-favored and GNN-disfavored features leads to a decrease in GFS performance.

Additional, as shown in Table 21, we present the statistics of values, sparsity, and TFI for all features, GNN-favored features, and GNN-disfavored features. The results indicate that: (1) the TFI of GNN-favored features is significantly higher than that of GNN-disfavored features; (2) the datasets encoded by Pretrained Language Models (PLMs), including Children, Computers, Fitness, History, and Photo, exhibit similar values in features, yet their TFI varies considerably. Notably, a higher average TFI across all features correlates with better GFS performance; (3) GFS is less effective on datasets with higher sparsity, such as Minesweeper and Tolokers. (4) Datasets with lower homophily tend to identify more features as GNN-disfavored, whereas those with higher homophily identify more features as GNN-favored.

Q1: How is the decomposition of node features in the graph executed?

We would like to clarify that in GFS, we do not decompose node features. Instead, we categorize the node features into two sets: GNN-favored and GNN-disfavored features, based on TFI. This categorization is guided by the principle of "feeding the right features to the right model." To assess how GNN or MLP benefits from specific features, we utilize our proposed TFI and apply a threshold to classify all features into GNN-favored and GNN-disfavored categories. The effectiveness of TFI has been validated through both theoretical analysis and empirical studies. Thank you for your comments. We have revised the manuscript to enhance clarity and improve understanding.

Please let me know if our responses have addressed your concerns. Thank you!

W1: The evaluation method is not fair enough. The performance results in Table 1 should be presented with only the difference of with or without GPS, while keeping the model architecture the same (e.g., number of layers and hidden dimension).

Thanks for your valuable comments. Here we need to clarify that Table 1 shows the results of GNN and GNN+GFS under the same hyperparameter tuning, which has been widely used to fairly compare the performance of GNNs. As for the setting you mentioned that keeps the same model architectures, we also show the results in Section 5: 1) Figure 4 shows GCN+GFS consistently outperforms GCN across a wide range of $r$ (0.1-0.9) in most datasets while keeping all the other hyperparameters, e.g. number of layers and hidden dimension, the same. 2) Figure 5 shows how the performance of GCN+GFS and GCN would be affected under varying hyperparameters where the points in each sub-figure denote they are evaluated under the same hyperparameters. We have revised the paper to help better understand the experimental results. Thank you.

Q1: As shown in Figure 2, the performance gap for the Roman dataset between GCN and MLP in regions with low TFI is minimal. How is the improvement over GCN in Figure 4 so significant regardless of $r$ ?

As shown in Figure 2, the accuracy gap (GCN-MLP) is close to 0 except for the bin with high TFI (TFI>0.8). This result is consistent with the result in Figure 4, where we only send the features with high TFI (r=0.2) to GCN greatly improves the performance compared with MLP (r=0.2). Note that the x-axis in Figure 2 and Figure 4 represent different meanings: x-axis in Figure 2 means we send 10% of the features with a certain range of TFI to GCN and MLP and compare their performance gap, while the x-axis in Figure 4 means we split all the features into two part, where the highest r% features are sent to GNN and the left features are sent to MLP. Sorry for the confusion, we have revised the paper to enhance clarity and improve understanding.

Q2: Since features are updated in each GNN/MLP layer, is the TFI computed at each layer to get (dis)favored features, or is it only computed at the beginning? This point does not seem clearly explained in the paper.

We clarity how the feature selection is conducted in the paper:

1. In Line 81 in Section 1 (Introduction), we mention that we first use TFI to select GNN-favored and GNN-disfavored features and then process these two types of features by GNN and MLP respectively.
2. In Figure 3 In Section 4 (Graph Feature Selection), we show that the features are sorted before sending to GNN and MLP at the beginning, instead of sending node representations.
3. In line 257-260 in Section 4 (Graph Feature Selection), we show that we select the node features, i.e. ${\mathbf{X_{\mathcal{G}}} | \mathbf{X_{:, m}} \in \mathbf{X_{\mathcal{G}}}, \text{TFI}_m \ge \delta(r)}$, in GFS. Thanks for your comments, we have further revised the paper to improve the readability. It is a good question. As shown in Figure 7 in our submission, we conduct the GFS on node embeddings by different approaches to see if it will also improve the performance of baseline GNN. The results show that in some datasets, such as Questions, Roman-empire, and Tolokers, performing GFS on node embeddings further improves the GFS performance. It is interesting to investigate how the GFS affects the model performance on node embeddings encoded by more advanced models in the future.

We greatly appreciate your valuable reviews. Thank you.

Thank you for your positive rating. We’d like to clarify that our method does not divide the entire graph into different parts; rather, it categorizes node features into GNN-favored and GNN-disfavored groups while keeping the graph structure unchanged. By splitting all node features into 10 bins, Figure 2 illustrates how TFI can identify features that are favored by both graph-aware and graph-agnostic models. These findings support our experimental results in Figure 4, where GFS demonstrates superiority over baseline GNNs by adhering to the principle of "feeding the right features to the right models". Besides, GFS with r = 0.1 to 0.9 consistently outperforms the models that send all the features to GCN (r = 1.0) or MLP (r = 0.0). We will revise the paper to enhance its clarity and make it easier to understand. Thank you.

Thank you for your response. I am willing to improve my score. However, I still have concerns regarding my initial question. From what I understand, dividing the entire graph into 10 parts to create subgraphs and then training MLP or GNN separately on these subgraphs could potentially ignore the global information, which is essential for GNNs. To my understanding, an effective method would align with the approach depicted in Figure 4, where only the nodes ranging from $x$ to $x+0.1$ are processed by the MLP, while the rest of the graph remains unchanged. This would enable a more precise analysis of the performance gap in each bin.

W1: The proposed feature selection metric TFI, which leverages mutual information between features and labels, is not novel. This is a conventional approach for feature selection, as outlined in [1]. Although the TFI utilizes the features derived from the neighborhood average (AX), the approach is incremental.

Thank you for your review. Below are our responses to your concerns regarding the novelty of our work:

• As detailed in Appendix A, we clarify the novelty of our method in contrast to traditional approaches. Previous studies on mutual information (MI) [1] have primarily focused on selecting the most informative features or reducing redundancy for independent and identically distributed (i.i.d.) data in graph-agnostic models (MLP). In contrast, our approach emphasizes distinguishing between features favored by graph-agnostic models (MLP) and those favored by graph-aware models (GNN), which is essential for effective graph representation learning.

• In addition to introducing the novel metric, TFI, this paper presents a framework called Graph Feature Selection (GFS). This framework effectively leverages the strengths of both GNN and non-GNN models, demonstrating simplicity and effectiveness in enhancing the performance of eight GNNs across ten datasets. To the best of our knowledge, we are the first to investigate feature selection in the context of both GNN and non-GNN models.

• In the context of graphs, TFI quantifies the mutual information between aggregated features and labels. Although this method is straightforward and intuitive, it is effective in identifying GNN-favored and GNN-disfavored features, as evidenced by our theoretical analyses and empirical studies.

• The extension of aggregated features is a non-trivial task, as the field of graph learning is evolving to analyze the transformation from X to AX. This extension clarifies the role of topological information in GNNs and is crucial for understanding the relationships and interactions within graphs.

We have revised our paper to highlight our novelty. Thank you for your suggestions.

W2: This paper lacks a comparative analysis of classic feature selection methods, such as [2].

Our primary focus is on investigating feature selection across both graph and non-graph data, rather than proposing a feature selection method specifically for non-graph data. Consequently, we do not compare the performance of classic feature selection methods that are designed for non-graph data. For instance, reference [2] is designed for non-graph data in a multi-label setting, which differs significantly from our context. As shown in Table 2, we do compare metrics such as generalized feature homophily [3], attributed homophily [4], and local similarity [5], which are specifically designed for graph data. We appreciate your suggestions regarding these interesting metrics and believe that incorporating additional metrics into GFS could further enhance graph representation learning.

W3: The GPS exhibits limited robustness to the selection of the hyperparameter r.

As shown in Figure 4, the performance of GFS remains robust across varying hyperparameter values of $r$. GFS consistently outperforms the baseline GNN across a wide range of $r$, from 0.1 to 0.9, in most datasets, including Children, Computer, and Roman-Empire.

W4: It is unclear how the high-pass filters of FAGCN and ACMGNN, as presented in Table 1, align with the TFI metric with low-pass AX.

It is a good question. In this paper, we treat GNNs as powerful tools for extracting topological information. Both high-pass and low-pass filters are different forms of graph filters that leverage the graph's topology. Therefore, the GNN-favored features selected by TFI remain effective even when applied to high-pass filters. That said, the main focus of this paper is to introduce a novel question and provide a solution for selecting GNN-favored and GNN-disfavored features for graph representation learning. We believe it would be interesting to explore different forms of TFI using advanced graph filters to further enhance the performance of GFS. For instance, as shown in Figure 13, we demonstrate how a high-order neighbor-based graph filter can influence TFI's performance in GFS. Thank you again for your valuable question.

W5: 1) A significant concern is the applicability of the proposed GFS. The paper asserts that TFI is computed on training nodes. Thus, I am concerned that the unusual dataset division of 50/25/25 for training/validation/testing employed in this paper is crucial for the effectiveness of GFS. 2) The question then becomes: how would the results be affected if a public splitting, such as 20 per class for training in GCN, SGC, GAT, and APPNP, were applied?

1. In Figure 6 of our submission, we analyze how the ratio of the training set affects the performance of TFI in GFS. The results indicate that even if using a low training ratio in TFI can sometimes reduce the performance of GFS, it still consistently outperforms the baseline GNNs.

2. We further conduct the experiments on these datasets with public splits as shown in Table 15. Accuracy results on node classification are reported for Cora, PubMed, and CiteSeer, each having one split. For the other datasets, which have ten splits, we provide both accuracy and standard deviation. The results demonstrate that GNN+GFS outperforms baseline GNN in most datasets for 4 types of GNN backbones, which include GCN, GAT, SGC, and GraphSAGE. This demonstrates the effectiveness of our proposed TFI even under conditions of low supervision.

[1] Feature Selection: A Data Perspective. ACM Comput. Surv. 50(6).

[2] Multi-label Feature Selection via Global Relevance and Redundancy Optimization. IJCAI 2020.

[3] Di Jin, Rui Wang, Meng Ge, Dongxiao He, Xiang Li, Wei Lin, and Weixiong Zhang. Raw-gnn: Random walk aggregation based graph neural network. arXiv preprint arXiv:2206.13953, 2022.

[4] Liang Yang, Mengzhe Li, Liyang Liu, Chuan Wang, Xiaochun Cao, Yuanfang Guo, et al. Diverse message passing for attribute with heterophily. Advances in Neural Information Processing Systems, 34:4751–4763, 2021b.

[5] Yuhan Chen, Yihong Luo, Jing Tang, Liang Yang, Siya Qiu, Chuan Wang, and Xiaochun Cao. Lsgnn: towards general graph neural network in node classification by local similarity. arXiv preprint arXiv:2305.04225, 2023.

Thank you for your response. Could you please specify which paper you are referring to regarding these results? We adhere to the default settings in [1], where only one split is used for experiments. Our findings are comparable to those reported in that paper with 20 labels per class. However, another approach, referred to as the "public splits," is discussed in [2], which utilizes splits of 60%/20%/20% across 10 runs. This setting has also been widely used for "public splits" in recent studies [3,4,5,6,7]. These results are based on different settings, which are more aligned with the results you mentioned.

Could you kindly indicate which paper contains the results you mentioned? Thank you!

[1] Zhilin Yang, William W. Cohen, Ruslan Salakhutdinov: Revisiting Semi-Supervised Learning with Graph Embeddings. CoRR abs/1603.08861 (2016)

[2] Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, Bo Yang: Geom-GCN: Geometric Graph Convolutional Networks. ICLR 2020

[3] Xiang Li, Renyu Zhu, Yao Cheng, Caihua Shan, Siqiang Luo, Dongsheng Li, Weining Qian: Finding Global Homophily in Graph Neural Networks When Meeting Heterophily. ICML 2022: 13242-13256

[4] Sitao Luan, Chenqing Hua, Qincheng Lu, Jiaqi Zhu, Mingde Zhao, Shuyuan Zhang, Xiao-Wen Chang, Doina Precup: Revisiting Heterophily For Graph Neural Networks. NeurIPS 2022

[5] Yunchong Song, Chenghu Zhou, Xinbing Wang, Zhouhan Lin: Ordered GNN: Ordering Message Passing to Deal with Heterophily and Over-smoothing. ICLR 2023

[6] Kun Wang, Guibin Zhang, Xinnan Zhang, Junfeng Fang, Xun Wu, Guohao Li, Shirui Pan, Wei Huang, Yuxuan Liang: The Heterophilic Snowflake Hypothesis: Training and Empowering GNNs for Heterophilic Graphs. KDD 2024: 3164-3175

[7] Jeongwhan Choi, Seoyoung Hong, Noseong Park, Sung-Bae Cho: GREAD: Graph Neural Reaction-Diffusion Networks. ICML 2023: 5722-5747

Please let me know if our responses have addressed your concerns. Thank you!

We really appreciate your time for the detailed review and raising your score. To address your remaining concern about the novelty of our paper, we will elaborate the importance of our study on $AX$, i.e. the aggregated node features, for graph learning.

Since the main difference between GNN and traditional Neural Networks (NN) is the additional feature aggregation step, investigating how does the aggregation operation influence the model performance stands out as one of the most important topics to study GNN behavior. Recently, considerable efforts have been devoted to this direction and obtained some remarkable insights. For example,

• [1,6] investigate the structures of the aggregated node embeddings $AX$ under varying levels of homophily, finding that a consistent neighborhood distribution of intra-class nodes enhances GCN performance;
• [2] studies the node similarities of $AX$ from post aggregation perspective, which results in a new homophily metrics and a proof of the effectiveness of high-pass filter on heterophily problem;
• [3] studies the impact of graph convolution by comparing the node distinguishability of $X$ and $AX$ under different homophily levels, discovering the mid-homophily pitfall phenomenon;
• [4] provides a detailed analysis of $AX$ in different homophily intervals and explain the performance disparity of GNN and MLP with graph structural shifts.

While these studies are important, they miss the fine-grained analysis on each feature dimension and do not examine how graph convolution influences GNNs feature-wisely. Our paper fills this gap by providing a feature-wise analysis of the aggregated features, making a valuable contribution to the community.

Besides, in contrast to traditional feature selection methods [5], which aim to reduce redundancy, select informative features, or generate new features, our proposed Graph Feature Selection (GFS) framework has an additional property, that is to evaluate $*"when is a given feature beneficial for graph convolution?"*$. This approach effectively leverages the strengths of both GNNs and non-GNN models. Our experiments demonstrate its effectiveness across eight GNN backbones on various datasets.

[1] Is Homophily a Necessity for Graph Neural Networks?. In International Conference on Learning Representations.

[2] Revisiting heterophily for graph neural networks. Advances in neural information processing systems. 2022 Dec 6;35:1362-75.

[3] When Do Graph Neural Networks Help with Node Classification? Investigating the Homophily Principle on Node Distinguishability. Advances in Neural Information Processing Systems. 2024 Feb 13;36.

[4] Demystifying structural disparity in graph neural networks: Can one size fit all?. Advances in neural information processing systems. 2024 Feb 13;36.

[5] Feature Selection: A Data Perspective. ACM Comput. Surv. 50(6).

[6] Understanding Heterophily for Graph Neural Networks. The Twelfth International Conference on Learning Representations.

Dear Reviewer jpAB,

Thanks for all your previous efforts that try to help us improve our paper. We would like to know if your remaining concern on novelty is addressed. We can take this last chance to answer your question. Thanks again.

Best,

Authors