Generalized Category Discovery under Domain Shift: A Frequency Domain Perspective

Q1:Limited novelty in motivation and methodology

Thank you for your thoughtful feedback. We respectfully argue that, while frequency-based techniques have been explored in the UDA community, their direct application to GCD is non-trivial and potentially harmful. GCD presents unique challenges: unlike UDA, it must discover novel categories in unlabeled target domains without access to any ground truth. This reliance on pseudo-labels makes the task particularly sensitive to semantic noise. As shown in Table 1, naïvely applying frequency perturbations without distinguishing between known and unknown domains leads to negative transfer, especially in the novel class space. To address this, our FDS module explicitly distinguishes known and unknown samples before perturbation, avoiding cross-domain contamination and enabling stable representation learning. This is essential in GCD, where noisy pseudo-supervision is the only guidance available for novel categories. Furthermore, our method introduces three novel components specifically tailored for GCD: Class-aware Cross-Domain Frequency Perturbation: It leverages the more familiar source-domain data in a class-aware manner to guide cross-domain feature learning while preserving semantic structures. Intra-Domain Frequency Perturbation: This improve generalization within the unknown domain. Clustering Difficulty-Aware Sampling: To our knowledge, this is the first method to incorporate difficulty-awareness into GCD. It adaptively rebalances training based on estimated category-level clustering difficulty, addressing the issue of uneven class separability that is critical in discovering novel categories. While our approach draws inspiration from UDA literature, it is designed to principally address the underexplored GCD problem under domain shift, with components carefully constructed to handle the semantic noise, lack of supervision, and class discovery difficulties that define GCD. We hope this clarification better communicates the principled design and GCD-specific motivation behind our work.

Table 1: Ablation study on different transformation strategies

Table 2: Ablation study of different components.

Q2:Missing key related work.

Thank you for the suggestion. While CDAD-Net tackles a related problem, it assumes that all unlabeled data come from unknown domains—a more restrictive setting. In contrast, our method addresses a more practical scenario where unlabeled data include both known and unknown domains. As shown in Tables 3 and 4, our approach consistently outperforms CDAD-Net across all metrics, highlighting its effectiveness in this more challenging setting. We will clarify the conceptual differences and emphasize the empirical results in the final version.

Table 3: Performance on the SSB-C Dataset

Table 4a: Real+Painting Performance on DomainNet

Table 4b: Real+Sketch Performance on DomainNet

Table 4c: Real+Quickdraw Performance on DomainNet

Table 4d: Real+Clipart Performance on DomainNet

Table 4e: Real+Infograph Performance on DomainNet

Q3:hyper-parameter analysis

Thank you for your comment. We would like to clarify that the sensitivity analysis of the hyperparameters designed in our framework is included in the appendix. As shown in the results, our method demonstrates stable performance across a wide range of settings, indicating its robustness to hyperparameter choices. In addition, we now include a new ablation study on the loss balancing coefficient $\beta$, and rewrite the overall loss function as: $L_{total} = \beta L_{kd} + (1 - \beta)L_{ud}+ \epsilon\bigtriangleup$. As shown in Table 5, setting $\beta$ to 0 or 1 leads to suboptimal performance. In contrast, our method consistently achieves stable results across different values of $\beta$, further demonstrating the robustness of the proposed training strategy. We will ensure all parameter values used in experiments are clearly listed in the final version.

Table 5: Sensitivity analysis of the loss weight $\beta$

Q4:Writing Clarity Concern

Thank you for pointing this out. We agree that the term unknown domain may be confusing since the unlabeled target domain is visible during training. In the final version, we will revise the terminology to use target domain consistently for clarity. We also acknowledge that the methodological descriptions in the abstract and introduction could be more precise. We will revise these sections to better articulate the intuition and motivation behind each component of our framework. These changes will improve the overall readability and clarity of the paper.

Q5:Eq.(11) seems a probability, not an objective function, how to utilize it?

We first sample categories from the categorical distribution $c \sim p_{\text{difficulty}}$, where the probability $p_{\text{difficulty}}^c$ reflects the clustering difficulty of class $c$. We then retrieve real feature embeddings from the current pool of samples predicted as class $c$ by the model. These features are used for classification or contrastive learning to refine the model's representation. We appreciate the feedback and will incorporate further details in the final version of the paper.

Q6:What is the setting in Fig. 2

Figure 2 presents the results of directly applying FDA to SimGCD, where we report the clustering accuracy across all categories. As shown, our method outperforms SimGCD+FDA both in terms of final clustering accuracy and convergence speed. In contrast, the random transformations introduced by SimGCD+FDA can sometimes hinder learning in the unknown domain, potentially leading to negative transfer effects. This observation further motivates the need for a more tailored perturbation strategy.

We sincerely thank the reviewer for their constructive feedback and for acknowledging the added experiments. We would like to further clarify the novelty, motivation, and technical contributions of our work:

---

On CDAD-Net and Domain Shift under GCD Settings

While CDAD-Net adopts a similar problem setting combining domain shift and generalized category discovery (GCD), we would like to emphasize that this problem remains far from being solved. For instance, on the CUB dataset, CDAD-Net achieves only 40.4% accuracy on all classes in the original domain, and merely 37.7% in the corrupted domain. In contrast, our method outperforms CDAD-Net by over 20.0% on the original domain and 18.0% on the corrupted domain across all classes. We attribute this performance gap to the instability of CDAD-Net's entropy-driven adversarial learning strategy, which motivated us to seek a more stable and principled solution.

---

Motivation and Failure of Frequency Perturbation in DS-GCD

As shown in Figure 2 (Page 2, Lines 51–60), we clearly demonstrate that applying standard frequency-domain perturbation (as used in UDA methods) leads to negative transfer in the DS-GCD setting. This is because indiscriminate frequency manipulation introduces semantic noise, which is especially harmful to novel class discovery, where pseudo-labels are the only source of supervision. This makes the task highly sensitive to such noise.

To address this, our FDS module explicitly distinguishes between known and unknown domain samples before perturbation, avoiding cross-domain contamination and enabling stable representation learning.

---

Clarification on Paper Motivation and Component Design

Each component of our method is designed with specific motivations tailored for the DS-GCD challenge:

• Section 4.2 (Lines 190–197) motivates the need for class-aware cross-domain frequency perturbation, which leverages more stable source-domain data in a category-aware manner to guide feature adaptation while preserving semantic structures.

• Section 4.3 (Lines 211–217) explains intra-domain frequency perturbation, which improves generalization within the target domain.

• Section 4.4 (Lines 230–237) introduces clustering difficulty-aware sampling, which rebalances training by estimating class-level clustering difficulty — a factor not yet explored in prior GCD literature.

Additionally, the contents of Table 1 and Table 2 in the rebuttal correspond to Table 3 and Table 4 in the main paper, respectively. We agree that the motivation could be further strengthened in the main text. In the final version, we will revise and enrich the discussion in both the introduction and method sections to highlight our principled design and the rationale behind each component.

---

CDAD-Net Discussion and Citation

Thank you for the suggestion — we will include a detailed discussion of CDAD-Net in the final version. For clarity:

“CDAD-Net achieves cross-domain known class alignment and novel class discovery by integrating entropy-driven adversarial learning, cross-domain prototype alignment, neighborhood contrastive learning, and conditional image inpainting to jointly optimize a shared representation space.”

In contrast to CDAD-Net, our approach is the first to introduce a frequency-domain solution tailored for the unique challenges posed by domain-shifted generalized category discovery (DS-GCD).

Although our method draws inspiration from UDA literature, it is fundamentally designed to tackle unaddressed challenges in GCD under domain shift, such as semantic noise, lack of supervision, and uneven cluster separability. We believe our contributions — including FDS, Class-aware Cross-Domain Frequency Perturbation, Intra-Domain Frequency Perturbation, and Clustering Difficulty-Aware Sampling — offer a novel and effective framework to address these unique challenges.

---

We hope this clarifies our motivation and design principles, and we sincerely thank the reviewer for encouraging us to present them more clearly.

Thank you very much for your thoughtful and detailed review throughout this process. We deeply appreciate your recognition of our efforts in addressing the concerns you raised, as well as your acknowledgement that the changes and additions will greatly improve the paper. Your professional and constructive approach has been invaluable in helping us strengthen our work, and it is truly our honor to receive your guidance.

As previously stated, our paper has already articulated the motivation and design objectives clearly in multiple sections, including both the Introduction and the Method parts. We also appreciate your suggestion regarding the placement of the analysis — especially the point on “a factor not yet explored in prior GCD literature.” We fully agree that this perspective is valuable and should appear earlier in the manuscript to strengthen our motivation. In the revised version, we will integrate these analyses into the Introduction and early parts of the Method section, ensuring they are highlighted and clearly connected to our contributions.

Overall, the failure of approaches such as CDAD-Net has prompted us to reflect more deeply on whether frequency-domain information could be leveraged to address the DS-GCD problem. Our method is specifically designed to explore this direction, providing a principled and effective solution that mitigates semantic noise, enhances cross-domain generalization, and facilitates robust novel category discovery under domain shift.

We are grateful for the time and expertise you invested in evaluating our research and for your valuable contributions to improving our paper.

Q1:contribution of each component:

We thank the reviewer for the insightful comment. As shown in Table 1, directly applying frequency transformations without domain separation often leads to negative transfer, especially in the unseen domain where pseudo-labels are noisy. This is because the model relies entirely on pseudo-labels to supervise clustering in these unfamiliar domains, which may result in noisy guidance and degraded performance. This motivated us to first perform domain separation and leverage the more familiar source domain to learn domain-invariant feature representations. Building on this, we design two specialized frequency-domain perturbation strategies to improve domain generalization and reinforce semantic structure. As shown in Table 2, the class-aware cross-domain frequency perturbation enhances generalization by promoting domain-invariant representation learning. Additionally, the intra-domain frequency perturbation consistently boosts performance across all, old, and new classes in the target domain, reflecting improved intra-class compactness and inter-class separability. Finally, our CDAS module adaptively emphasizes harder categories based on clustering difficulty, leading to better representation learning and overall clustering quality.

Table 1: Ablation study on different transformation strategies

Table 2: Ablation study of different components

Semi-supervised K-means was also conducted in the frequency domain. Additionally, we applied it to the original image space for comparison. The advantage of working in the frequency domain lies in the ability to isolate the amplitude component, which predominantly captures image style—something not feasible in the raw image space. As shown in Table 3, our method consistently outperforms Semi-supervised K-means in both domains. Frequency-based features yield stronger separation than raw image space representations under both clustering schemes.

Table 3: Clustering performance of other separation strategies

While IDFP alone has been explored in prior work as a general augmentation strategy, we observed that applying IDFP without the proposed FDS module leads to negative transfer and limited performance gains. Therefore, identifying a task-specific and well-integrated solution is crucial. Without the introduction of the FDS module, IDFP cannot effectively contribute to the generalized category discovery task.

Q2:lacking in important citations

We thank the reviewer for the valuable suggestion. We will add the following citation to clarify the statement in Line 112:

Yang, Yanchao, and Stefano Soatto. FDA: Fourier Domain Adaptation for Semantic Segmentation. CVPR 2020.

This work serves as a key reference for frequency-based domain adaptation strategies. Additionally, we acknowledge the omission of prior methods such as ORCA in the related work section. We will include a more comprehensive discussion of these methods in the final version to ensure completeness.

Q3:ablation studies of each component

We appreciate the reviewer’s concern regarding the complexity of our framework and the need for clearer justification of each component. To address this, we have conducted comprehensive ablation studies on the DomainNet dataset, as summarized in Table 1 and Table 2. Starting from SimGCD as the baseline, we incrementally integrate each module and evaluate its individual impact. As shown in Table 1, applying frequency perturbation without domain separation often causes performance degradation, especially on the unknown domain. This is because performing perturbations indiscriminately in the frequency domain may lead to negative transfer. To mitigate this, we introduce domain separation, which leverages amplitude discrepancies across sample groups to isolate known and unknown domains. We then apply Cross-Domain Frequency Perturbation (CDFP) to guide domain-invariant representation learning using the more familiar source domain. Furthermore, we employ In-Domain Frequency Perturbation (IDFP) within the target domain to capture intra-domain variation while avoiding semantic collapse. As shown in Table 2, IDFP consistently improves clustering accuracy across all/known/unknown subsets. Finally, the Clustering Difficulty-Aware Sampling strategy further enhances performance by dynamically balancing training on easy and hard samples. These ablation results clearly demonstrate the necessity and complementary nature of each module. We will also revise the appendix to include more detailed explanations of the experimental setup and the design rationale for each component.

Q4:Writing clarity needs improvement

Thank you for the helpful suggestion. We will revise the manuscript to improve the clarity of figure captions and explanations. All terminology and symbols will be properly defined, and we will ensure consistency throughout the paper.

Q5:Ambiguity of the method name "Free"

We have changed Free to FREE and will apply consistent formatting throughout the final version.

Q6:Unclear difference between SimGCD and standard GCD losses

The contrastive loss is inherited from the original GCD framework, while the clustering loss is novel and specific to SimGCD. We will clarify this distinction in the revised version.

Q7:"Low frequency" not clearly defined

Following FDA, we define low-frequency components using a centered square of side length $2L*min(H,W)$ in the frequency domain. We will clarify this in the main text and refer readers to the corresponding analysis in the appendix.

Q8:Undeclared or rarely used symbols.For example, symbol $E$ is used without prior definition.

Thank you. The symbol $E$ denotes the feature extractor, which is introduced in the “Revisiting SimGCD” section. We will define it clearly when first used and simplify any unnecessarily complex notations.

Q9:Clarify references to "HiLo" in lines 261 and 264

Thanks. We will explicitly mention HiLo in those lines to improve clarity and minimize backtracking for readers.

Q10:Insufficient explanation of Figure 3

We appreciate the suggestion. We will revise both the caption and the main text to map each visual element to its corresponding loss component and enhance interpretability.

Q11:Grammatical error at line 333 ("hindering")

Thanks for pointing this out. We will correct the grammar in the final version.

Q12:Misleading section title "Theoretical Analysis"

We agree and will rename the section to Preliminaries for clarity and accuracy.

Q13:Ambiguity in line 458 and inconsistent supplementary content

Thank you. We will revise the supplementary material (especially lines 458, 460, 463, and 465) for clarity and consistency, and ensure the accuracy of all formulas and statements.

Q14:Missing citations (e.g., DomainNet at line 472)

Thank you. We will add all missing citations, including DomainNet, in the final version.

Q15:Suggestion to include pseudocode in the appendix

Thank you. We will include clear pseudocode in the appendix to improve method transparency and reproducibility.

Q16:GMM validity in multi-domain setting (Table 6). Does using a 2-mode GMM remain valid when multiple domains are present?

Even in multi-domain settings, a two-component GMM remains effective because all unknown domains are stylistically dissimilar to the known domain. As shown in Table 6, our method still performs well with multiple domains, validating the approach.

Q17:Applicability of components to standard GCD without domain shift

Thank you for the insightful question. When domain shift is absent, our modules can still enhance baseline methods. As shown in Table 4, both clustering difficulty-aware sampling and Fourier perturbation improve SimGCD’s performance even without domain shift.

Table 4: performance comparison between SimGCD and our method on CUB (no domain shift)

Q18:computational overhead

Regarding the computational overhead introduced by FFT, we mitigate this concern by leveraging efficient implementations such as torch.fft, which is highly optimized for GPU computation. Additionally, we adopt the FAISS library to significantly accelerate KNN retrieval during training, keeping the extra cost minimal. As for the GMM component, although it relies on effective separation between known and unknown domains, our experiments demonstrate its robustness even in multi-domain scenarios, as shown in Table 6 (Appendix).

Q1:contribution of each component:

We appreciate the reviewers’ recognition of the practical value and challenges associated with our proposed problem setting. Indeed, our formulation—which combines labeled and unlabeled data from distinct domains and categories for category discovery—is a natural yet underexplored generalization of UDA and GCD. While prior work has tackled GCD under domain homogeneity or UDA under closed-label space, few have addressed both semantic and domain shift simultaneously. Our work is to bridge this gap with a unified and principled solution. While we acknowledge that some components draw inspiration from previous work, we would like to emphasize that our method is not a trivial composition of existing modules, but rather introduces key innovations specifically tailored to the dual-challenge of domain and category shifts. First, we observed that directly applying previous frequency-based adaptation methods (e.g., FDA) to the GCD setting is problematic, as randomly mixing frequency components from different domains may introduce spurious correlations and semantic noise. This is especially detrimental for novel categories, which rely solely on pseudo-label supervision. To address this, we propose an FDS mechanism that explicitly separates samples by domain before applying perturbation, preventing harmful signal contamination and ensuring more stable representation learning and better discrimination between known and novel categories. Building on this, our class-aware cross-domain frequency perturbation uses the more familiar source domain data to guide feature learning and mitigate domain gaps without disrupting category structure. To further improve the model’s robustness within the unknown domain, we introduce an intra-domain frequency perturbation module. This enhances the model’s invariance to intra-domain variations, addressing the lack of supervision and diversity in unlabeled novel categories. Existing GCD or UDA approaches typically assume uniform difficulty across categories during training. However, in practice, some categories (especially in the unknown domain) are harder to cluster due to greater intra-class variance or domain shifts. Our proposed CDAS module adaptively reweights the sampling process based on per-category clustering difficulty, allowing the model to focus its capacity on harder categories, improving both representation learning and final clustering quality. To our knowledge, this is the first work to incorporate such difficulty-awareness into domain-adaptive category discovery. As shown in Table 1, directly applying frequency transformations without domain separation often leads to negative transfer, especially in the unseen domain where pseudo-labels are noisy. This is because the model relies entirely on pseudo-labels to supervise clustering in these unfamiliar domains, which may result in noisy guidance and degraded performance. This motivated us to first perform domain separation and leverage the more familiar source domain to learn domain-invariant feature representations. Then, we apply In-Domain Frequency Perturbation (IDFP) within the target domain to avoid collapse of semantic structures. As shown in Table 2, IDFP consistently improves performance across all/known/unknown classes in the unseen domain, indicating that it helps enhance intra-class compactness and inter-class separability. Moreover, the Cross-Domain Frequency Perturbation (CDFP) significantly improves generalization, suggesting that it promotes learning of domain-invariant representations.

Table 1: Ablation study on different transformation strategies

Table 2: Ablation study of different components.

Q2:Computational Complexity

We evaluate the computational efficiency of our method relative to baseline approaches. Although our framework introduces additional operations—namely spectral decomposition via Fast Fourier Transform (FFT), K-nearest neighbor similarity estimation, and Gaussian Mixture Model (GMM)–based domain modeling—these are implemented using highly optimized tools. Specifically, we utilize torch.fft for frequency transformations and FAISS for efficient KNN retrieval on GPU. As shown in the table below, we report the FLOPs of a single forward step and the elapsed time per iteration on the CUB-C dataset with a batch size of 128 on an NVIDIA A100 GPU. Our method maintains a comparable computational profile in terms of both FLOPs and per-iteration training time:

Table 3: FLOPs and training time cost

These results demonstrate that the proposed enhancements introduce minimal overhead, while enabling more effective domain-aware learning.

Q3:Markdown syntax

Thanks for the suggestion. We will properly fix it in our final version.

Dear Reviewer rRLy,

The author-rebuttal phase is now underway, and the authors have provided additional clarifications and performance results in their rebuttal. Could you please take a moment to review their response and engage in the discussion? In particular, we’d appreciate your thoughts on whether their revisions adequately address your initial concerns. Thank you for your time and valuable contributions.

Best, Your AC

Q1:Regarding Baselines:

We have included comparisons with recent GCD methods, including ProtoGCD, CMS, and SPTNet, as shown in Table 1 and Table 2. While these methods bring some improvements, they still struggle under domain shift. In contrast, our method consistently outperforms them, further validating the need for a specialized solution like Free.

Table 1: Performance on the SSB-C Dataset

Table 2a: Real+Painting Performance on DomainNet

Table 2b: Real+Sketch Performance on DomainNet

Table 2c: Real+Quickdraw Performance on DomainNet

Table 2d: Real+Clipart Performance on DomainNet

Table 2e: Real+Infograph Performance on DomainNet

Q2:Regarding contribution of each component:

We appreciate the reviewers’ recognition of the practical value and challenges associated with our proposed problem setting. Indeed, our formulation—which combines labeled and unlabeled data from distinct domains and categories for category discovery—is a natural yet underexplored generalization of UDA and GCD. While prior work has tackled GCD under domain homogeneity or UDA under closed-label space, few have addressed both semantic and domain shift simultaneously. Our work is to bridge this gap with a unified and principled solution. While we acknowledge that some components draw inspiration from previous work, we would like to emphasize that our method is not a trivial composition of existing modules, but rather introduces key innovations specifically tailored to the dual-challenge of domain and category shifts.

First, we observed that directly applying previous frequency-based adaptation methods (e.g., FDA) to the GCD setting is problematic, as randomly mixing frequency components from different domains may introduce spurious correlations and semantic noise. This is especially detrimental for novel categories, which rely solely on pseudo-label supervision. To address this, we propose an FDS mechanism that explicitly separates samples by domain before applying perturbation, preventing harmful signal contamination and ensuring more stable representation learning and better discrimination between known and novel categories. Building on this, our class-aware cross-domain frequency perturbation uses the more familiar known-domain data to guide feature learning and mitigate domain gaps without disrupting category structure. To further improve the model’s robustness within the unknown domain, we introduce an intra-domain frequency perturbation module. This enhances the model’s invariance to intra-domain variations, addressing the lack of supervision and diversity in unlabeled novel categories. Existing GCD or UDA approaches typically assume uniform difficulty across categories during training. However, in practice, some categories (especially in the unknown domain) are harder to cluster due to greater intra-class variance or domain shifts. Our proposed CDAS module adaptively reweights the sampling process based on per-category clustering difficulty, allowing the model to focus its capacity on harder categories, improving both representation learning and final clustering quality. To our knowledge, this is the first work to incorporate such difficulty-awareness into domain-adaptive category discovery.

Q3:Regarding Hyperparameter Sensitivity:

We have further conducted a parameter analysis on $\eta$, as shown in Table 3. We observe that smaller $\eta$ values tend to introduce noisy pseudo-labels and degrade performance, while larger $\eta$ can effectively filter out noisy samples, leading to more accurate category-aware perturbation.

Table 3: Parameter Analysis of $\eta$

Q4:Regarding Statistical Significance:

We thank the reviewer for pointing this out. We have added standard deviation results in Table 1 and Table 2, which show the performance is stable across runs. The error bars for other methods, such as GCD, SimGCD, and ORCA, can be referenced from Appendix Figure 8 in the HiLo paper. Due to space limitations, we will include the full error statistics in the final version.

We thank the reviewer for their constructive feedback and for acknowledging the improvements made to the manuscript.

Regarding the concern about the hyperparameter $\eta$, we would like to clarify that our choice of a relatively high threshold value (e.g., $\eta = 0.95$) is not arbitrary, but rather guided by empirical intuition. Specifically, $\eta$ controls the confidence level for selecting unlabeled samples used in frequency-domain amplitude exchange. A lower value of $\eta$ may introduce samples with incorrect pseudo-labels, resulting in semantic misalignment during the style exchange process and deteriorated clustering performance.

This phenomenon is particularly critical in fine-grained datasets. For instance, bird images are often associated with sky-like backgrounds, while fish typically appear in underwater scenes. Mixing frequency characteristics across such semantically distinct categories can introduce domain-specific noise that impairs the model's ability to discover meaningful clusters.

Therefore, using a higher $\eta$ (e.g., 0.95) helps us filter out low-confidence samples and preserve the semantic integrity of style representations. As shown in our hyperparameter analysis, performance tends to improve as $\eta$ increases, and stabilizes around the chosen value, indicating that our framework is not overly sensitive to this parameter while still benefiting from confident sample selection.

To further address your concern, we extended the range of the hyperparameter $\eta$ and observed that model performance tends to stabilize as $\eta$ increases. This supports our choice of setting $\eta$ to a relatively high value, as it helps filter out low-confidence samples and suppresses semantically noisy style transfer (e.g., background mismatches such as sky in bird images versus water in fish images), which may hurt clustering quality.

Table 3: Parameter Analysis of $\eta$

We will incorporate this explanation into the final version for better clarity.