Debiased Imbalanced Pseudo-Labeling for Generalized Category Discovery

Major:

1. The novelty is relatively limited. The main learning components follow SimGCD, and ranking loss in DPD is not new. Plus, the proposed LRA is not fully explained (See question 2).
2. In LRA, Eq. (5), how can $z_i^{feat}$ be put into the $\exp$ operator, $z_i$ is a vector rather than a variable.
3. The bias some seen categories are mistakenly classified as novel ones is not very convincing to me. GCD is an imbalanced problem where some old classes samples are labeled, but new classes are purely unlabeled, in this way, models tend to bias some novel classes to old ones, which is revealed in SimGCD. However, in this paper, the bias is a little bit counterintuitive in the context of GCD.
4. Ablations on the margin value should be included.
5. Why r in Eq. (4) is 1 rather than -1. Plus, in Lines 268-269, there might be a typo in “if the distance between … and …”. Please clarify this point.

Minor:

1. Line 604. AGCD is a CVPR 2024 paper, please refine this reference.

1. The notation in section 4.1 is quite confusing and difficult to understand. For instance, $p_S^{gt} = [p_i^{gt}]$: does this represent the probability of the ground truth, or is it excluding the ground truth? Additionally, the process of ranking is unclear. It appears that the ranking loss is intended to enforce a uniform distribution of the prototypes, but this is not clearly explained.

2. The design of the patch loss is peculiar. For different augmentations, $x_i$ and $x_i^`$ may not be aligned within each patch. Furthermore, the design lacks thorough ablation studies. For example, what would the results be if a simple Mean Squared Error (MSE) loss were used instead?

3. In Table 2, comparisons with state-of-the-art methods [1,2] are missing. Additionally, experiments involving DINOv2 are not included.

4. As the paper focuses on imbalanced pseudo labels, more discussion about the imbalanced novel class discovery is needed.

[1] Rastegar et al. Learn to Categorize or Categorize to Learn? Self-Coding for Generalized Category Discovery. NeurIPS 2023.

[2] Gu et al. Class-relation Knowledge Distillation for Novel Class Discovery. ICCV2023.

1. The variation in results from the GCD benchmarks can be very large, so, as SimGCD does in its Supplementary Information, it is important to report all results along with error bars generated by 3 to 5 independent runs.
2. The performance of the method with DINOv2 should be explored, as it has been employed in recent GCD methods [A].
3. Further investigation is needed on the effectiveness of strong data enhancement.
4. As can be seen in Table 4, hyperparameter settings may require extensive optimization across different datasets, thus limiting scalability without further optimization guidance. This makes me more concerned about the real-world application of the method.
5. The paper contains several typos and formatting issues. (Errors in characters in the penultimate line of page 7, inconsistent font sizes in tables, and incorrect indexing of table 3)

[A] No Representation Rules Them All in Category Discovery. NeurIPS 2023.

1.Some analysis in experiments section lack explanation and details. For example, 1) The method is very sensitive to $\beta$ as showed in Table 4,will be good to discuss on this matter. 2) In prototype ranking loss, how to choose $r$ and margins need further clarification.

2.Herbarium is also a commonly used fine-grained dataset in GCD to evaluate model performance. However, this is missing in the experiments.

W1. Please check the classification ACC in Tab 4 carefully, there are obvious errors in the data, For example, β=0.5,All=80.4 Old=94.0 New=79.3, β=1.0,All=83.3 Old=94.5 New=77.6 for the ImageNet-100 dataset.

W2. Authors claim that DebiasGCD enables the model to generate cleaner pseudo labels for samples of known classes, while the interpretability of the proposed DPD is not sufficiently discussed and analyzed in the paper. For example, why does DPD choose the smallest prototype instead of the biggest one? Since the smallest prototype is relatively easy to distinguish from the GT classes by the model, which means these prototypes are hard to cause the bias in the pseudo label of known classes. In another paper that also tries to explicitly adjust the distance between prototypes, the authors choose the k-NN prototype as the negative data point [1]. Could the authors explain the motivation for this selection operation?

W3. Meanwhile, the authors seem to improperly mix the pseudo-label Acc and the classification Acc, as shown in Fig. 4, where the title is Pseudo-labels ACC, however, all values are the same as the classification ACC that is shown in Tab. 2. This accident raises concerns about Fig. 1(a)

W4. In Fig. 2, a strong augmentation is applied in the proposed DebiasGCD, but the effect of this operation is not shown in the ablation study.

W5. The setting of N for each dataset is not elaborated in the paper, and could the authors show the tendency of the DPD loss with the change of N?

W6. When N > 1, the dimensions of p_S^gt (x_i ) and p_S^(Top_N) (x_i) are not the same, how is the p_S^gt (x_i )-p_S^(Top_N) (x_i) in Eq. 4 realized?

W7. More works of GCD should be included in Tab 2 to demonstrate the effectiveness of the proposed methods, like CMS [2] and SPTNet [3]

W8. There are several typos in this paper, such as “p_( x_i)” in Line 209, and “w_i^gt,w_i^gt” in Line 269

[1] Fewer is more: A deep graph metric learning perspective using fewer proxies[J]. Advances in Neural Information Processing Systems, 2020 [2] Contrastive Mean-Shift Learning for Generalized Category Discovery[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024. [3] An Efficient Alternative Framework for Generalized Category Discovery with Spatial Prompt Tuning[C]//The Twelfth International Conference on Learning Representations.