FedLPA: Local Prior Alignment for Heterogeneous Federated Generalized Category Discovery

We truly appreciate your positive and constructive comments, and we will reflect on your feedback thoroughly. Here are the responses to the questions.

Q1. Can the authors provide any theoretical or empirical analysis on the convergence behavior?

A1. Thank you for your constructive feedback. While a formal theoretical convergence proof is challenging due to the non-differentiable nature of the InfoMap, we provide empirical evidence demonstrating the stable and efficient convergence of our method. Across all our experiments, the model consistently converges within the reported communication rounds (20 warm-up, 50 training). Note that FedLPA shows faster convergence compared to baselines like AGCL, which require up to 200 rounds for training. To provide a picture of convergence, we conducted an extended training run of 100 rounds (including warmup rounds). The results are presented in Table C below. As shown, both on CIFAR-100 and Stanford Cars, the performance converges around 60 rounds and maintains it thereafter. We will include the full convergence curves for all experiments in Tables 1 and 2 in the final manuscript.

Table C: Convergence of FedLPA on Stanford Cars and CIFAR-100 ($\alpha=0.2$). The table shows the "All" accuracy (%) over communication rounds.

Q2. Have you observed cases where the learned cluster structures diverge across rounds?

A2. In our experiments, we found the learned cluster structures to be stably evolved and did not observe divergence during the learning process. We attribute this stability largely to the design of our CLSD mechanism. By establishing stable anchors within the similarity graph—using both ground-truth labels and high-confidence pseudo-labels from the seen classes—CLSD builds a reliable foundation for the graph. We believe this robust foundation mitigates drastic fluctuations in the overall cluster structure between rounds, providing a consistent learning target for the model.

Q3. Communication efficiency of FedLPA

A3. FedLPA is communication-efficient because it requires fewer total communication rounds to converge without incurring additional per-round costs compared to FedAvg.

Since FedLPA only communicates model parameters, its per-round cost is identical to standard FedAvg and lower than methods like AGCL [19], which communicate extra information such as class prototypes. Furthermore, we empirically observed that FedLPA consistently converges within approximately 70 total rounds (20 warm-up and 50 training) across all datasets. This rapid convergence results in higher overall communication efficiency, as FedLPA reaches its final performance in fewer rounds than baselines requiring more extensive communication rounds (200 rounds reported in [19]).

Q4. Could any of the learned local graph structures or soft predictions leak sensitive information?

A4. No, FedLPA does not introduce additional privacy risks compared to the standard FedAvg protocol since FedLPA only communicates model parameters. Unlike methods such as AGCL that communicate class prototypes and risk leaking information about local class distributions, in FedLPA, the learned local graph structures, discovered concepts, and predictions are strictly confined to each client and never shared with the server or other clients. Therefore, FedLPA maintains the same privacy guarantees as FedAvg.

Q5. Have you considered adding differential privacy or gradient compression techniques?

A5. Thank you for the great suggestion. These techniques are orthogonal to our core method and can be readily integrated to enhance privacy and efficiency. We view this as a valuable direction for future work.

Q6. Do you have quantitative or qualitative results showing how graph quality evolves across rounds?

A6. We thank the reviewer for this constructive suggestion. We provide a quantitative analysis below, which shows that FedLPA consistently improves the quality of the graph-based local structures as training progresses. Table D presents the clustering accuracy on each client's local unlabeled data, which directly reflects the graph quality. The average accuracy steadily increases from 57.0% to 71.3%, demonstrating a positive feedback loop: better representations lead to higher-quality graphs, which in turn improve the model. This confirms that our framework reliably refines its understanding of local data structures throughout training.

Table D: Per-Client Clustering Accuracy (%) Progression of FedLPA on Stanford Cars ($\alpha = 0.2$)

Q7. Have you considered alternatives like initializing from a pre-trained encoder (other than DINO), or delaying the clustering process until later rounds?

A7. Our framework is indeed flexible regarding the choice of the backbone; we used DINO primarily for a fair comparison with baselines. To further validate your suggestion, we ran additional experiments on Stanford-Cars using a stronger DINOv2 backbone. The results show that FedLPA achieves a significant performance gain (e.g., ACC increased from 52.1% to 62.4% for $\alpha=0.2$ and from 51.8% to 58.1% for $\alpha=0.05$ ), highlighting that FedLPA directly benefits from the better backbone. We will include this result in the final manuscript. Thank you for this valuable suggestion.

Regarding delaying the clustering, we have already explored this through our ablation study on the number of warm-up rounds (Fig. 4a in the supplementary document). As you suggested, the results show that performance improves with more warm-up rounds. Notably, our method achieves strong performance that already outperforms most baselines with as few as 10 warm-up rounds, demonstrating the robustness and efficiency of our approach.

Q8. The title includes "Local Prior Alignment," which sounds Bayesian, but no formal Bayesian formulation or interpretation is provided.

A8. To clarify, our method is not Bayesian. We use "Local Prior" to denote the empirical cluster distribution estimated from local data, which serves as a dynamic target for alignment, distinct from a Bayesian prior probability.

Dear Reviewer QDuW,

Thank you for your valuable feedback. To address your remaining concern regarding the theoretical analysis, we wish to first reiterate that a formal convergence proof for FedLPA is challenging due to the non-differentiability of InfoMap clustering, which precludes the use of standard analytical tools that rely on end-to-end differentiability.

However, we believe a path toward analysis could emerge under a reasonable assumption that the discovered cluster structures will converge as FedLPA training progresses, even as the underlying feature representations are continuously refined. This assumption is empirically grounded; we observe that the clustering accuracy in each client converges in our experiments (Table D in the rebuttal). Under this assumption, we can conceptually decouple the discrete clustering process from the continuous model optimization. This simplification would then allow for a convergence analysis within a more conventional federated learning framework. While it does not represent a rigorous proof, it may offer valuable insight into the convergence behavior of FedLPA. We will add this discussion to the final manuscript.

We sincerely thank the reviewer for the thoughtful feedback and for acknowledging the key strengths of our work. We are grateful for your positive comments on our model being technically sound, well-motivated, and addressing a critical research problem. We address your questions below.

Q1. The method requires each client to construct a similarity graph and run InfoMap clustering on all local data during communication rounds. You mentioned that more frequent updates (R = 1) yield better results. However, for clients with large amounts of data or limited computational resources, this could pose a significant computational bottleneck. The paper does not quantify this overhead or discuss potential optimization strategies.

A1. We thank the reviewer for this practical concern. While the local structure discovery (CLSD) phase indeed introduces additional computational cost, we believe that the resulting performance gains significantly outweigh this overhead. Crucially, this cost can also be flexibly managed.

To quantify the cost, our experiments on fine-grained datasets (when $R=1$ with a ViT-B/16 backbone on a single A6000 GPU) show the CLSD process takes approximately 7.2 seconds per client, constituting about 11.4% of a 63.12-second wall-clock time for a local training round.

More importantly, this overhead is not a fixed bottleneck. As shown in Figure 2c in the main paper, practitioners can directly control this cost by adjusting the CLSD frequency, $R$. The framework maintains highly robust accuracy even when CLSD is performed less frequently. For instance, by setting $R=8$, the CLSD component is executed only once every eight rounds. This reduces its overall contribution to the total computational cost from 11.4% to just 1.42%, while still maintaining competitive performance. This flexibility offers a practical way to adapt our method to various resource-constrained environments.

Q2. I wonder if you have tried other divergence measures (e.g., simple KL divergence)? Is there any particular theoretical or empirical reason for choosing JSD? More details would be helpful.

A2. Thank you for the excellent question. You are correct; we chose Jensen-Shannon Divergence (JSD) for our LPA regularizer precisely because its symmetric and bounded properties led to empirically stable training. We did experiment with the standard KL divergence, but we found the training to be unstable. The loss often became biased towards high-probability classes or would explode due to near-zero probabilities in the model's predictions. Even with heuristic adjustments such as adding an epsilon value or clipping the distribution, the instability persisted. In contrast, JSD provided a consistently robust and reliable training signal without such modifications.

Q3. How sensitive is this method to the quality of warm-up?

A3. We have analyzed the sensitivity to the warm-up quality through an ablation study on the number of warm-up rounds, and the results are illustrated in Fig. 4a in the supplementary document. The results show that while performance improves with more warm-up rounds, our method is not overly sensitive to it, which demonstrates the robustness and efficiency of our approach.

We appreciate the constructive and insightful feedback. We address your questions below and welcome any further questions.

Q1. Local clusters are not aligned across clients. For example, the discovered novel class "A" on one client has no correspondence to "A" on another client, which can be problematic, especially for novel class discovery. Could global novel-class prototypes be gradually learned via communication of aligned clusters or prototypes?

A1. We thank the reviewer for this insightful comment. FedLPA tackles the misalignment problem via relational modeling. Specifically, seen-class representations act as globally consistent anchors, learned with shared semantics across the federation. By modeling each client's novel classes in relation to these common anchors (via local relational graph construction), our approach can establish a consistent relational structure without requiring direct semantic alignment of the novel classes themselves. This, in turn, enables consistent representation learning for all classes by focusing on their relationships to the shared anchors.

Our ablation study (Row 5 in Table 3) validates this anchor-based mechanism. The graph refinement establishes a clearer relational structure using seen-class examples, resulting in performance improvement.

Q2. This method heavily relies on the quality of local clustering. However, in this federated learning setting, a few clients may encounter noisy features or samples. Under this condition, the quality of local clustering for some clients can be quite poor, which can negatively impact the performance of the global model.

A2. We thank the reviewer for the constructive comments. To analyze the sensitivity of FedLPA to the quality of local clustering, we conduct an experiment on Stanford-Cars ($\alpha=0.2$) by intentionally injecting noise into the pseudo-labels generated by our Confidence-guided Local Structure Discovery (CLSD) module. At the beginning of every CLSD phase (every $R=5$ rounds), we randomly flipped a certain percentage of the pseudo-labels to other classes.

The results, presented in Table B, show that while performance degrades gracefully as the label noise rate increases, FedLPA maintains robust performance. Even with a 30% noise rate, the accuracy remains competitive and outperforms the baselines reported in our main paper (Table 1). This demonstrates that our framework is not overly sensitive to a moderate level of noise in local clustering and can effectively mitigate its negative impact.

Table B: Robustness of FedLPA to noisy pseudo-labels on Stanford-Cars ($\alpha=0.2$). We report "All" accuracy (%) under varying rates of injected label noise.

Q3. How does FedLPA compare to pure federated contrastive learning methods?

A3. Direct comparison with the other federated contrastive learning methods, such as MOON [Li21], FedRCL [Seo24], and FedMoco [Dong21], is not straightforward since our work focuses on discovering novel classes in unlabeled data and learning proper representations along with seen classes, while these methods are not inherently designed to discover or model the underlying semantic category structures, which is the central challenge in Fed-GCD and the primary focus of FedLPA.

References

[Li21] Q. Li, et al., Model-Contrastive Federated Learning, in CVPR, 2021

[Seo24] S. Seo, et al., Relaxed Contrastive Learning for Federated Learning, in CVPR, 2024

[Dong21] N. Dong, et al., Federated Contrastive Learning for Decentralized Unlabeled Medical Images, in MICCAI, 2021

We appreciate the constructive feedback. We provide the following responses to address the questions.

---

Q1. Novelty and broader import of FedLPA

We thank the reviewer for the constructive feedback. We will address the core concern about the novelty and broader impact of our work.

Our primary contribution is a novel Fed-GCD framework that removes the unrealistic assumptions prevalent in prior GCD works, both centralized and federated settings. Previous GCD methods, including federated adaptations, are built on two unrealistic assumptions in a real-world federated environment:

• Global Knowledge Assumption: Most GCD methods like SimGCD [27], GCD [22], and even FedoSSL [28] require a priori knowledge of the total number of novel classes. In a federated system with strict privacy, no single client or server can possess this global information.

• Uniform Distribution Assumption: Many approaches [16, 20, 23, 25, 27, 28, 30] enforce a uniform class distribution prior (e.g., via entropy maximization), which doesn’t make sense considering the highly skewed and heterogeneous nature of FL.

We propose a framework that effectively tackles the Fed-GCD problem by removing these assumptions. Specifically, our Confidence-guided Local Structure Discovery (CLSD) obviates the need for global knowledge by enabling each client to autonomously discover its unique local category counts and distributions. Building upon this, our Local Prior Alignment (LPA) then removes the uniform class distribution assumption by dynamically aligning the model's predictions with the actual local data distribution. CLSD and LPA mutually benefit each other, making our framework robust to severe data heterogeneity and fostering effective category discovery across clients.

We now address the reviewer's specific questions about LPA and CLSD.

Q1.1. Why does LPA compare logit and empirical distributions, and how does the LPA term relieve the challenges in FL+GCD?

A1.1. LPA achieves robust regularization by aligning the model's prediction distribution with an empirical prior—a "softer" approach that is more resilient to noise than using hard pseudo-labels. Since individual cluster assignments from clustering can be noisy, using these as hard targets for cross-entropy loss or contrastive loss would make the model highly sensitive to this noise, forcing overconfidence in incorrect labels. Instead, by matching the model's average prediction distribution on a batch to the empirical cluster distribution discovered by CLSD, LPA guides the model’s overall behavior to reflect the client’s class distribution. This makes the learning process inherently more stable and resilient to noise from the clustering step.

This directly resolves a major limitation of existing GCD works. Unlike previous methods that enforce uniform class distribution, LPA replaces this fixed, unrealistic assumption with a dynamic, data-driven regularization. It actively guides the model to learn each client's unique class distribution, thereby robustly handling the severe data heterogeneity and class imbalance that are unique challenges in Fed-GCD.

Q1.2. Technical novelty in local graph structure discovery and InfoMap

A1.2. While we utilize established tools like InfoMap, our technical novelty lies in their principled adaptation to solve a critical sub-problem in Fed-GCD: the assumption-free estimation of the unknown number and distribution of local categories at each client. Our proposed CLSD is a discovery engine whose primary function is to provide the dynamic, client-specific data distribution necessary for our LPA to work. This ability to perform robust local category discovery is a non-trivial prerequisite for the realistic Fed-GCD solution.

Q1.3. Any broader impacts and applications of this loss term design to other FL+GCD approaches?

A1.3. LPA can be extended beyond our specific framework, as its core principle offers a principled and robust alternative to the unrealistic uniform prior assumption. We believe LPA can be readily adapted as a "drop-in" module in existing parametric GCD methods (e.g., SimGCD), extending their applicability to the realistic, imbalanced data distributions where they struggle, in both centralized and federated settings.

---

Q2. When a FedAvg is done with the LPA loss, is it possible to provide a theoretical convergence analysis? Also, if the theoretical analysis is too challenging, is it possible to provide the convergence behavior of FedLPA across rounds, or via empirical results?

A2. Thank you for your constructive feedback. While a formal theoretical convergence proof is challenging due to the non-differentiable nature of the InfoMap, we provide empirical evidence demonstrating the stable and efficient convergence of our method. Across all our experiments, the model consistently converges within the reported communication rounds (20 warm-up, 50 training). This demonstrates faster convergence compared to baselines such as AGCL, which require up to 200 rounds for training. To provide a picture of convergence, we conducted an extended training run of 100 rounds (including warmup rounds). The results are presented in Table A below. As shown, both on CIFAR-100 and Stanford Cars, the performance converges around 60 rounds and maintains it thereafter. We will include the full convergence curves for all experiments in Tables 1 and 2 in the final manuscript.

Table A: Convergence of FedLPA on Stanford Cars and CIFAR-100 ($\alpha=0.2$). The table shows the "All" accuracy (%) over communication rounds.

---

Q3. How to compute the empirical prior?

A3. The empirical prior $\pi_{n, B_{u}}$ is not computed from ground-truth labels, but is instead empirically derived from the pseudo-labels generated by our framework. Specifically, at the start of each local training phase, our CLSD module and InfoMap (Sec. 3.3-3.4) assigns a cluster ID (a pseudo-label) to every local sample. Following Eq. 4 in the main paper, the prior $\pi_{n, B_{u}}$ is then simply the normalized histogram of these pre-assigned pseudo-labels for the samples within the current unlabeled mini-batch $B_{u}$.

We hope our responses have addressed your concerns. We welcome any further questions.