HYPERION: Fine-Grained Hypersphere Alignment for Robust Federated Graph Learning

Thank you for the feedback. Regarding your questions, we would like to clarify the following points:

Weaknesses 1: Add analysis of model performance across different local training epochs/rounds.

We greatly appreciate these insightful suggestions.We have conducted additional experiments to analyze the performance of our method under different local training epochs. Specifically, we varied the number of local training epochs and observed how this impacted the overall model performance:

We report Acc values on Cora dataset with 50%-uniform noise. The results clearly show that HYPERION consistently outperforms the other methods across all local training epochs. As the number of epochs increases, we observe a slight decline in performance for all methods, which is typical due to the overfitting risk and the noise in the dataset. However, HYPERION maintains a significant margin over the competing methods even at higher epoch counts, indicating its robustness to noisy labels and its ability to generalize better across different training configurations. We will include more detailed analyses in the revised version.

Weaknesses 2: Justify the selection of comparison algorithms and datasets.

We compare our method, HYPERION, with several state-of-the-art federated learning (FL) and federated graph learning (FGL) methods to comprehensively evaluate its robustness and effectiveness under noisy conditions. These comparison methods include FedAvg, FedProx, FedNova, MOON, and others, which are widely recognized in the FL and FGL communities. We included them because they represent the current standards in federated learning, especially in settings with label noise.Additionally, Robust FL methods like FedNoRo, FedNed, and FedCorr are also included to evaluate how HYPERION compares to methods specifically designed to address noise in federated settings. In addition, we also considered several robust graph learning algorithms, which have demonstrated excellent robustness in the presence of noisy labels in graph data, but have not addressed issues such as the client-side limitations in federated learning.The comparison with these algorithms helps us demonstrate HYPERION’s superior robustness in scenarios where noise is a major challenge.

As for the choice of datasets, we used five benchmark datasets: Cora, CiteSeer, PubMed, Physics, and Amazon_ratings. These datasets were selected because they represent a range of real-world graph data types with varying complexities and noise profiles. Cora, CiteSeer, and PubMed are well-established citation networks commonly used in graph learning tasks, and they exhibit challenges such as node sparsity and high feature dimensionality, making them suitable for testing the robustness of our method under label noise.The Amazon_ratings dataset introduces a real-world, user-item co-purchase graph, which has more complex structural properties and is frequently used to benchmark collaborative filtering algorithms. This dataset helps us evaluate how HYPERION performs in scenarios involving collaborative data.The Coauthor-Physics dataset provides a more specialized network of co-author relationships, testing HYPERION’s ability to handle graphs from specific academic domains, demonstrating its applicability to different types of graph-structured data.The details of the datasets are shown in the table below.

We sincerely thank you for this suggestion. We will enhance the description of algorithms and datasets in the revised version.

Questions 1: Clarify the enhanced view generation process in HS-CNC.

In the HS-CNC module, enhanced views are created through a dual perturbation strategy that combines structural and feature-level augmentations to identify noisy nodes. Following prior work [1, 2], our method first applies random edge dropping to the adjacency matrix with a predefined probability, disrupting local graph connectivity while preserving global structure. Simultaneously, it randomly masks a portion of node features to simulate semantic variations, generating perturbed versions of the original graph data. These augmented views are then processed through the same model to obtain hyperspherical embeddings, where nodes demonstrating consistent cluster assignments across different perturbed views are retained as reliable samples while inconsistent nodes are filtered as potential noise. This approach effectively amplifies the difference between stable clean nodes and unstable noisy nodes by evaluating their robustness to controlled perturbations, enabling more accurate noise identification without requiring explicit label correction. The view generation process is designed to maintain the intrinsic graph properties while introducing sufficient variation to test node stability, striking a balance between preserving meaningful patterns and exposing noise-sensitive regions of the data.

Questions 2: Explain the advantages of hyperspherical node projection in the method.

By constraining embeddings to a unit sphere, the model leverages angular distance (cosine similarity) rather than Euclidean distance, which naturally amplifies inter-class separation while compacting intra-class clusters. This geometric property makes the model more sensitive to meaningful semantic relationships while being less susceptible to noise and feature magnitude variations. The hyperspherical space enables more stable prototype learning through von Mises-Fisher distributions, facilitates reliable federated aggregation via directional consistency, and provides a principled framework for noise purification through angular outlier detection. Compared to traditional Euclidean spaces, the hyperspherical projection offers superior noise robustness by encoding semantic similarity in angular relationships rather than absolute positions, which is particularly crucial for handling label noise and structural variations in decentralized graph data. This approach allows the model to better capture fine-grained topological differences while maintaining strong generalization across clients.

We sincerely thank you for this suggestion. We will enhance the description of hyperspherical node projection in the revised version.

We hope this clarifies your questions.

[1]: Deep graph contrastive representation learning. In ICML, 2020.

[2]: Contrastive learning of graphs under label noise. In Neural Networks, 2024.

Thank you for the feedback. We address your concerns below and hope our responses will help update your score:

Weaknesses 1: Add F1-macro results and full hyperparameter details for prototype clusters (p9).

Thank you for your constructive feedback regarding the presentation of experimental results. We appreciate your suggestion to include the F1-macro metric and provide more details about the hyperparameters used in the experiment.We have now included the F1-macro results for the experiment involving different numbers of prototypes in prototype clusters under various noise conditions.The experimental results are shown in the table below.

From the perspective of F1-macro, it has been confirmed once again that setting K = 1 results in under-learning of hypersphere, leading to consistently lower performance.In contrast, increasing K to 3 or 4 yields marginal performance improvements.

We also understand the importance of clarifying the impact of other hyperparameters on the experiment. Specifically, we split the dataset into training, validation, and test sets in a 20%, 40%, and 40% ratio. We used the Louvain partitioner to induce data skew. The experiments were run for 100 rounds, involving 10 clients. Each client was trained for 3 local epochs with a batch size of 24. The learning rate was set to 0.005, and we introduced pairwise label noise to simulate real-world imperfections. Our network features a four-layer GCN backbone with uniform 384-dimensional hidden layers throughout the first three layers,each employing symmetric normalization (normalize=True) and ReLU activation, followed by 0.2 dropout for regularization. The final GCN layer produces compact 32-dimensional graph embeddings without activation. These embeddings are processed through a two-layer MLP head with ReLU activation in the hidden layer. The architecture optionally incorporates prototype learning with configurable parameters: each class maintains multiple 32-dimensional prototype vectors, and the prototype contrastive loss operates with a temperature parameter τ = 0*.07 to control the similarity distribution sharpness. All GCN layers implement symmetric normalization (normalize=True), and consistent dropout (p = 0.*2) is applied after each intermediate layer to prevent overfitting.

We acknowledge the value of testing on diverse systems and will include more comprehensive results including F1-macro in the revised version.

Weaknesses 2: Clarify how TP-HSL's multi-prototype design enables fine-grained structural learning (e.g., through prototype diversity or cluster separation).

In our TP-HSL method, we utilize the von Mises-Fisher (vmF) distribution to model the distribution of node embeddings. The vmF distribution is well-suited for accurately measuring the angular differences between embeddings, especially in high-dimensional spaces. In graph data, node embeddings are typically high-dimensional, and the vmF distribution, through its concentration parameter, effectively quantifies the similarity and dissimilarity between node embeddings. This angular difference measure not only helps capture distributional differences between classes but also provides a deeper understanding of each node's local structural characteristics in the high-dimensional space. In particular, in graphs with complex topological structures, the vmF distribution offers a precise and efficient way to assess subtle differences between nodes.

Additionally, traditional graph learning methods often assign a single prototype to each class, which overlooks the structural variations within the nodes of the same class. In our TP-HSL method, we design multiple prototypes for each class, which help capture the diversity among nodes within the same class in the high-dimensional space. Each prototype represents a distinct topological feature within the class, allowing the class to be partitioned into multiple subspaces, each represented by one prototype. This approach ensures that the node embeddings within each class are better aligned with the corresponding prototypes, thereby improving both intra-class cohesion and inter-class distinction. By doing so, we can finely capture the structural differences within each class.

We sincerely thank you for this suggestion. We will enhance the description of TP-HSL in the revised version.

Limitations: Clarify how the compared robust graph learning methods were adapted for federated learning in the framework.

In our experiments, each robust GL baseline (e.g., CRGNN, RTGNN, CLNode) was applied independently on each client. Specifically:

Client-Side: On each client m, the robust GL method was employed to train a local model using the client’s private graph data G_{m}. The original noise-resilience mechanisms proposed in these methods—such as label correction, contrastive learning, or topology-aware regularization—were preserved and applied locally without modification.

Server-Side: After local training, model updates from clients were aggregated on the server using the standard FedAvg algorithm. No changes were made to the server-side aggregation logic for these baselines, ensuring consistent treatment across all compared methods.We selected FedAvg as the aggregation strategy due to its simplicity, stability, and widespread use in FL research. More importantly, using a uniform aggregation protocol eliminates the influence of advanced server-side techniques, allowing us to isolate and fairly compare the effectiveness of each robust GL metho’s local training design under federated conditions.

We sincerely thank you for this suggestion. We will enhance the description of implementation details in the revised version.

We hope this clarifies your questions.

Thank you for the feedback. We address your concerns below and hope our responses will help update your score:

Weaknesses 1: Clarify Mahalanobis vs. Wasserstein distance characteristics.

We appreciate the opportunity to elaborate on these distance metrics and their respective roles in our approach.

1. The Mahalanobis distance is used to measure the distance between a point and a distribution, taking into account the covariance structure of the data. By introducing the inverse of the covariance matrix, Mahalanobis distance can automatically adjust the importance of each dimension, avoiding certain dimensions dominating the distance calculation due to scale differences.In geometry, Mahalanobis distance transforms the data space into a standardized space, making distance calculations more equitable.We identify and remove noise nodes that deviate from the normal distribution by calculating the Mahalanobis distance between node embeddings and the global prototype distribution (as shown in Formula 18). This method effectively filters out noise that is not detected by clients due to local perspective limitations.
2. The Wasserstein distance, measures the minimum cost of transforming one probability distribution into another. It is particularly effective when comparing distributions that are not necessarily aligned or of the same type, making it well-suited for our distillation phase after clustering. By minimizing Wasserstein distance, the probability mass is effectively reallocated between proximate dimensions in the feature space, thereby naturally reinforcing the correlation of benign features between global prototypes, while effectively diminishing the impact of anomalous features.Unlike Mahalanobis distance, which focuses on individual point-to-point distances, Wasserstein distance captures the "flow" of information between distributions, making it ideal for distilling information from the clustered nodes while preserving their relational structure.

We will include more detailed analyses in the revised version.

Weaknesses 2: Add comparisons to more FGL and robust FL/GL methods.

We agree that expanding the set of methods for comparison would provide a more comprehensive evaluation of our proposed approach. We have conducted additional experiments and included comparisons with a broader set of FGL and robust FL/GL methods. These additional comparisons further strengthen our study and provide a more detailed analysis of the performance of our proposed approach in different settings. The newly added experiments include comparisons with methods FedDC, FedDyn, FedProto, Scaffold and Co-teaching, which we believe are highly relevant to the challenges addressed in our work. The noise type is set to 50%-uniform and we report accuracy (%) and F1-macro (%).

New Experimental Results:

We acknowledge the value of testing on diverse systems and will include more comprehensive results in the revised version.

Questions: Justify Mahalanobis over L2 distance in GA-HSP.

We are happy to provide additional details on why we opted for Mahalanobis distance and how it compares with alternative metrics, such as the standard L2 distance.

Mahalanobis distance is particularly suitable for our method due to its ability to account for the covariance structure of the data. Unlike the standard L2 distance, which assumes that the data points are independent and identically distributed (i.i.d.), the Mahalanobis distance adjusts for correlations between features. This is important in graph-based learning settings, where node features often exhibit dependencies that cannot be captured by simple Euclidean measures. In our case, using Mahalanobis distance allows the GA-HSP method to better capture the relationships between nodes in the graph, considering their high-dimensional feature space. By incorporating the covariance matrix of the data, Mahalanobis distance normalizes the feature space, making it more robust to differences in feature scaling and correlations between features. This makes it more appropriate for distilling node representations in the graph, where such dependencies are common.

The L2 distance is a simpler metric that calculates the straight-line distance between two points in the feature space. While it is computationally efficient and commonly used, it assumes that all features are equally important and independent, which may not be the case in graph learning tasks. In settings where feature dependencies exist (e.g., in graph data with highly correlated features), L2 distance might not capture the true relationships between the nodes effectively, potentially leading to less accurate results. In contrast, Mahalanobis distance is sensitive to the correlation between features and adjusts for the underlying structure of the data. It takes into account the covariance matrix, allowing it to measure distances in a more meaningful way when dealing with correlated or non-i.i.d. data. In our experiments, we observed that using Mahalanobis distance for distillation provided more accurate and stable results, particularly in scenarios where feature correlations played a significant role in the graph structure.

We hope this clarifies your questions.

Thank you for your thorough review and encouraging feedback. We are grateful for your positive assessment about novelty and importance of our work. We address your concerns below and hope our responses will help update your score:

Weaknesses 1: The assumption that projecting node features into a hyperdimensional space forms clusters to identify label noise is flawed, as nodes with similar features may have different labels, which could be misinterpreted as noise.

Thank you for this question. We sincerely appreciate the opportunity to clarify the inner workings of the HYPERION method.

During training, if two nodes have highly similar features but different labels, we do not necessarily consider this as noise. In fact, the noise detection mechanism in HYPERION primarily comes from HS-CNC, as introduced in Section 3.3. This mechanism determines whether a node has label noise based on the stability of node predictions after actively adding noise. TP-HSL, described in Section 3.2, is mainly designed to achieve clustering of nodes from different classes and does not specifically identify label noise. After projecting the node features into the hyperdimensional space, TP-HSL uses a corresponding loss function to encourage significant differences in the hyperdimensional representations of nodes from different classes, thus achieving clustering. In the case you mentioned, where two nodes have highly similar features but different labels within a client, HYPERION can effectively address this issue. These two nodes will ultimately be projected to different locations in the hyperdimensional space due to TP-HSL, as the projection function has learned higher weights for the features that contribute to the difference in labels for these similar nodes. If label noise indeed exists for either of these nodes, HS-CNC will remove the noise based on the stability of the node after perturbations.

We sincerely appreciate this insightful observation. In response to the valuable feedback,we will enhance our methodological presentation in the revised manuscript to provide a more comprehensive explanation of the node projection mechanism and its theoretical underpinnings.

Weaknesses 2: The approach of identifying stable nodes by randomly deleting edges or other nodes lacks a strong theoretical foundation, as removing a few key edges could significantly alter the learning behavior.

Although the identification of stable nodes through perturbation (such as random deletion of edges or nodes) is empirical, it serves as a heuristic method that addresses the practical challenge of noise in graph data within federated learning environments. The empirical approach we adopt is designed to mitigate the impact of noise on the model. While this method is not grounded in traditional graph theory, its effectiveness has been demonstrated in our experiments. As shown in Table 1 and Figure 3 of the paper, HYPERION consistently outperforms existing methods across various noise levels, supporting the validity of this empirical strategy. Additionally, previous works have also employed similar methods such as random edge deletion or feature modification [1, 2], and these approaches have been widely validated as effective.

Furthermore, this method does not involve the actual removal of edges in the graph; instead, it randomly deletes certain edges or modifies node features to assess the stability of the nodes. The graph input to the model remains the original graph, i.e., the undisturbed graph. The loss function is computed solely based on those nodes that exhibit stability under perturbations. We sincerely thank you for this question. We will include more detailed analyses in the revised version.

We hope this clarifies your questions.

[1]: Deep graph contrastive representation learning. In ICML, 2020.

[2]: Contrastive learning of graphs under label noise. In Neural Networks, 2024.