GHOST: Generalizable One-Shot Federated Graph Learning with Proxy-Based Topology Knowledge Retention

Dear Reviewer TQDx:

We greatly appreciate your positive feedback on our work, as well as the thoughtful concerns and questions you raised. We have carefully considered each of your comments and provided detailed responses.

Weakness:

W1: Implementation details for applying one-shot FL methods to graph data.

We sincerely appreciate the reviewer’s suggestion to clarify the implementation details of applying one-shot FL methods to graph data. In our experiments, we follow the overall process of the compared one-shot FL methods, utilizing the encoders and decoders from the corresponding papers to generate graph data features. Additionally, we adopt the k-nearest-neighbor (KNN) method to construct the topological structure for the generated graph data.

Specifically, in FedCVAE, we adhere to the Conditional Variational Autoencoder framework as used in the original paper. We input the training data and labels into the encoder to obtain representations in the latent space, which are then fed into the decoder to generate features. The KNN algorithm is subsequently applied to construct the topology. Similarly, in FedSD2C, we follow the original paper’s Autoencoder framework (including both the encoder and decoder) for generating graph data features and use the KNN algorithm for topology construction. By doing so, we retain the key processes and steps of the original baselines while reasonably adapting them to graph data.

W2: Performance under higher levels of data heterogeneity (e.g. $\alpha = 0.01$)

To demonstrate the performance of our method under higher data heterogeneity, we conduct experiments on the CiteSeer, and Coauthor-CS datasets with the Dirichlet distribution parameter $\alpha$ set to 0.01. We select five baselines for comparison: FedAVG, FedNova, FedPub, FedTAD and FedCVAE. The experimental results are shown in Table 1.

Table 1: Performance of GHOST under higher data heterogeneity ($\alpha = 0.01$) on CiteSeer and Coauthor-CS datasets.

From the table, we can observe that our method maintains the best performance even under higher data heterogeneity. For some datasets, when $\alpha$ is smaller, certain clients have very few nodes, which does not align with real-world scenarios. Therefore, we set $\alpha = 0.05$ in the experiments, as it ensures high data heterogeneity while avoiding overly extreme scenarios that do not reflect real-world conditions.

Dear Reviewer 9vCa:

We sincerely appreciate the time and effort you have dedicated to reviewing our paper. We hope that the detailed responses provided below will effectively address your concerns and offer the necessary clarifications.

Weakness：

W1: Lack of further explanation of the pseudo labels.

We appreciate the reviewer’s valuable comment regarding the assumption of a uniform distribution for the pseudo labels. While we acknowledge that real-world datasets may exhibit skewed class distributions, we believe that the uniform distribution assumption provides several benefits in our context.

Firstly, adopting a uniform distribution avoids the need for clients to share class label distributions, which significantly mitigates the risk of data privacy leakage. This aligns with the principles of privacy preservation in Federated Graph Learning, ensuring that sensitive label information is not exposed during the training process.

Secondly, the use of a uniform distribution ensures that data from all classes are taken into account, including minor or underrepresented classes. This prevents the model from neglecting the features and information from minority class data, thus promoting a more balanced learning process.

We believe that this design choice strikes a good balance between privacy preservation and model performance, and the effectiveness of this approach is supported by the results presented in the paper.

W2: Lack of more sensitivity study on different local training epoches.

To demonstrate the robustness of our method under different local training epochs, we select Cora and CiteSeer datasets, and design the range of local training epochs as [90, 110] with a step size of 5. Experimental results are shown in Table 1 and Table 2.

Table 1: Different local training epoches on the Cora dataset.

Table 2: Different local training epoches on the CiteSeer dataset.

From the tables, we can observe that our method maintains strong performance across different epochs, demonstrating its stability under varying epoches.

W3: Lack of details of training epoches in other baselines

For all the baselines compared in the experimental section, we uniformly set the local training epoch to 100 to ensure sufficient convergence of the local models. In the one-shot setting, too few local training epochs would prevent the model from adequately learning the local data, making the comparison unfair. Conversely, too many epochs could lead to overfitting to local data, resulting in degraded generalization performance.

Questions

Q1: Model performance with varying client numbers

A1: We appreciate the insightful question regarding model performance under varying client numbers. The observed performance improvement with a higher number of clients, particularly in datasets with high data heterogeneity (e.g., Coauthor-CS), can be attributed to the impact of distributional bias among clients.

When the number of clients is small, each client tends to hold a larger amount of data. However, in highly heterogeneous settings, this also means that the data distributions across clients can be significantly different, leading to greater heterogeneity in both data volume and feature distribution. Such imbalances can make it more challenging for the global model to effectively capture a generalizable representation, resulting in degraded performance. Conversely, increasing the number of clients leads to a finer-grained partitioning of the data, which can help mitigate extreme distribution shifts among clients. This, in turn, allows the global model to better learn shared patterns across clients, improving overall performance.

Additionally, different datasets have their own unique characteristics in terms of data features and structure. As a result, the effect of varying client numbers may manifest differently across datasets, leading to dataset-specific trends in performance. We hope this clarifies the phenomenon and appreciate the opportunity to further discuss our findings.

Dear Reviewer AgVq:

Thank you for your thoughtful review and for highlighting important concerns. Below, we provide detailed responses to clarify our proposed approach.

Weaknesses & Questions：

W1: Notation clarity and summarization.

We sincerely appreciate the reviewer’s valuable suggestion regarding notation clarity. To enhance readability, we will incorporate a summary table of commonly used notations and their definitions in a future version of our work. Thank you for your insightful feedback.

W2 & Suggestions: Additional comparisons with more FGL and one-shot FL methods.

Thank you for your valuable suggestion. To further enhance our study, we have conducted experiments incorporating FGSSL (FGL) and FENS (one-shot FL). The results on six benchmark datasets are presented in Table 1.

Table 1: Performance comparison including additional FGL and one-shot FL methods on six benchmark datasets.

From the results, we observe that GHOST continues to achieve the best performance, further demonstrating its robustness and effectiveness. We greatly appreciate your insightful suggestion, as it has helped reinforce the comprehensiveness of our study.

W3: Necessity and motivation for using both Divergence Loss (Eq. 4) and Dispersion Loss (Eq. 5).

Thank you for raising this point. Although both losses act on the feature representations, they capture complementary aspects. The Divergence Loss minimizes the statistical discrepancy between the pseudo and real features, ensuring that overall properties (spread, variance, density) match. In contrast, the Dispersion Loss preserves the angular (directional) relationships among features, maintaining the geometric configuration. This helps maintain the intrinsic relational patterns among the features regardless of their magnitudes. Together, they provide a comprehensive alignment, as confirmed by our ablation study (Sec. 5.4).

W4: Clarification on Equation 6 as a loss function and its gradient flow.

Thank you for this important question. Although Equation 6 formulates the FGW loss in terms of the transport plan $\mathbf{\Gamma}$, the loss function is inherently a function of both the graph structures and the feature matrices of the real and pseudo graphs.

In practice, we use a differentiable optimal transport solver (implemented with a conditional gradient method with optional Armijo line-search) to obtain $\mathbf{\Gamma}$. Consequently, even though $\mathbf{\Gamma}$ is the immediate optimization variable in Equation 6, the overall FGW loss is differentiable with respect to $\hat{\mathbf{X}}$ (and $\hat{\mathbf{A}}$), allowing gradients to flow back to these pseudo graph parameters.

Thus, by minimizing the FGW loss, the model not only optimizes the transport plan for aligning the local and pseudo graphs but also adjusts the pseudo feature matrix to reduce the discrepancy in both feature and structural domains. This gradient propagation through the differentiable OT solver ensures that the pseudo feature matrix is updated in a manner that preserves the intrinsic topological relationships while matching the local statistical properties.

W5 & Q1: Explanation of Equation 9 (Topology-Consistency Criterion).

Thank you for highlighting the need for clarification. The Topology-Consistency Criterion quantifies how much each global GNN parameter contributes to preserving learned topological structures. We first compute a Coherence Factor $\gamma_{ij}^l$ for connected nodes, which quantifies the similarity of their hidden representations after a transformation by the corresponding layer’s weight matrix $\boldsymbol{\theta}_{\boldsymbol{w}}^l$. Aggregating these values across nodes yields a global topological consistency measure for layer $l$. In Equation 9, the norm of the derivative for each parameter reflects its “sensitivity”: a higher norm indicates that even a small change in that parameter would cause a significant alteration in the coherence. In other words, parameters with high sensitivity are deemed more important for capturing the intrinsic topological structure across the diverse graphs in our integrated knowledge set.

By computing and then consolidating these sensitivity measures across layers, we obtain a comprehensive criterion $\mathcal{T}^k$ that identifies which parameters are crucial for preserving structural knowledge. This process helps in selectively retaining important parameters during the global training phase, mitigating the risk of catastrophic forgetting.