Keep It on a Leash: Controllable Pseudo-label Generation Towards Realistic Long-Tailed Semi-Supervised Learning

The main novelty of the work (W1 & Q2)

Thanks for your comments. While our method leverages established components (e.g., logit adjustment, consistency regularization, and pseudo-label voting), its core innovation is fundamentally different from prior works.

Existing methods typically employ high-confidence thresholds to filter pseudo-labels, then estimate unlabeled data distribution for subsequent refinement. However, during early training stages, the confidence threshold may completely exclude samples from certain classes. This leads to inaccurate estimation of the unlabeled data distribution, which in turn adversely affects subsequent pseudo-label generation steps that depend on this estimation. As shown in Figure 3, this leads to error accumulation and performance degradation.

Our method introduces a controllable self-reinforcing optimization cycle that: (i) progressively integrates reliable pseudo-labels from the unlabeled dataset into the labeled dataset, and (ii) trains on the updated labeled dataset, which follows a known distribution. This strategy eliminates dependence on error-prone unlabeled data distribution estimation, effectively circumventing distribution estimation entirely. Meanwhile, we theoretically prove that this optimization cycle can significantly reduce the generalization error.

Our method is unaffected by the unlabeled data distribution and addresses the unknown and arbitrary unlabeled data distribution, in contrast to prior works that attempt (and often fail) to estimate the unlabeled data distribution. As demonstrated in Table 5 of the paper, this cycle contributes an average performance improvement of 6.97%, representing the most significant gain among all components in our framework. Reviewers TLMt, ydDN, and Ckat all confirm our novelty.

Discuss related work (W2)

The goals of UPS and CADR differ from ours. UPS targets poor model calibration in SSL without distribution mismatch and CADR addresses distribution mismatch due to label missing not at random, while our work focuses on handling the unknown and arbitrary unlabeled data distribution. Furthermore, our approach introduces distinct conceptual and methodological innovations. We provide a comprehensive comparison with both UPS and CADR below.

UPS mitigates high-confidence incorrect pseudo-labels caused by poor model calibration through uncertainty estimation techniques like MC-Dropout. CADR focuses on resolving classifier bias due to label missing not at random by employing class-aware propensity estimation and dynamic threshold adjustment for pseudo-label selection. Unlike these existing techniques that rely on uncertainty estimation or threshold adjustment, we propose a novel controllable self-reinforcing optimization cycle that operates without distribution estimation or correction. More importantly, our framework is specifically designed to address real-world scenarios with unknown and arbitrary unlabeled data distributions. These discussion will be added in the final version.

Evaluation under extreme settings (W3 & Q1)

In long-tailed learning scenarios, there exists an inherent trade-off between extremely small $N_{max}$ (number of samples in the most frequent class) and large imbalance ratio $\gamma$ values, since $N_{min} = N_{max}/\gamma \geq 1$. Our initial experiments on CIFAR-10-LT in Table 1 already employed challenging settings ($N_{max}=400$, $M_{max}=4600$, $\gamma_l=100$, $\gamma_u=100$) with $N_{min}=4$. Notably, even with such extremely few tail class samples, our method still demonstrates significant performance gains over baselines.

To further evaluate our method, we conducted additional experiments under three more extreme settings: extreme sparsity setting A1 ($N_{max}=100$, $N_{min}=1$, $M_{max}=4900$, $\gamma_l=100$, $\gamma_u=100$), extreme sparsity setting A2 ($N_{max}=10$, $N_{min}=1$, $M_{max}=4990$, $\gamma_l=10$,$\gamma_u=300), and **extreme imbalance setting B** ($N_{max}=400$,$N_{min}\approx1.33$(1 in practice),$M_{max}=4600$,$\gamma_l=300$,$\gamma_u=300$). As demonstrated in Table R7, our method achieves state-of-the-art performance across three settings, attaining 69.56% accuracy in setting A1, 49.28% in setting A2, and 77.78% in setting B. These results represent significant improvements over all baseline methods, underscoring our method's exceptional capability to handle extreme settings.

Table R7

Real-world datasets (Q3.1)

The Food-101 dataset (evaluated in Table 3) is a widely adopted benchmark for food image classification. Its training set naturally contains a proportion of label noise, making the data distribution implicitly unknown. Thus, distribution mismatches between labeled and unlabeled data occur when splitting the training set into labeled and unlabeled datasets. Despite these challenges, our method achieves a significant average accuracy gain of 2.57% on this dataset.

To further evaluate our method’s effectiveness in large-scale and real-world scenarios, we conduct experiments on ImageNet-127 (an imbalanced dataset, 127 classes, imbalance ratio 286) with an arbitrary (non-long-tailed) distribution. As shown in Table R8, our method consistently outperforms baselines, achieving gains of +1.27% at 32×32 resolution and +2.82% at 64×64 resolution. These results highlight our method’s robustness in large-scale, realistic settings beyond standard benchmarks. As suggested, these results and corresponding analysis will be incorporated into Section 4.2 Results in the final version.

Table R8

Real-world applications (Q3.2)

In medical applications, distribution mismatch between labeled and unlabeled data occurs naturally. Labeled clinical datasets from hospitals typically follow a long-tailed distribution, with abundant cases of common diseases (head classes) but very few cases of rare diseases (tail classes). In contrast, unlabeled data collected from broader populations typically includes a majority of healthy individuals and only a small proportion of diseased individuals, especially those with rare diseases. This creates a fundamental divergence between labeled and unlabeled data distributions.

Furthermore, disease prevalence varies significantly across regions due to differences in climate, healthcare access, and other factors. Consequently, datasets collected from different geographical areas may exhibit arbitrary distribution patterns.

Evaluation using other architectures (Q4)

To evaluate architectural generalization, we conduct experiments using ResNet-50 on CIFAR-10-LT with ($N_{max}$=400, $M_{max}$=4600, $\gamma_l$=100, $\gamma_u$=100). The labeled data follows a long-tailed distribution, while the unlabeled data conforms to either an inverse long-tailed or an arbitrary distribution. As evidenced by Table R9, our approach demonstrates significant improvements, achieving absolute performance gains of 7.85% and 7.23% over current state-of-the-art methods in these respective scenarios. These results highlight our method's consistent robustness when applied to different network architectures and its ability to handle varying distribution patterns in the unlabeled data. The complete experimental results and detailed analysis will be incorporated into Section 4.3 Analysis of the final paper to further support these claims.

Table R9

Review related long-tailed learning methods (W1)

We sincerely appreciate the reviewer’s insightful suggestion regarding the connection to long-tailed learning. In the revised manuscript, we will expand the Related Work section to include a dedicated discussion of long-tailed learning methods, particularly focusing on techniques that could inform solutions to our investigated problem.

Pseudo-code (W2)

For the reviewer's convenience, we would like to highlight that the pseudo-code of our proposed method was already included in Appnedix E of the original submission. We will move it in the main text in the final version.

Without consistency regularization (Q1)

While consistency regularization between augmentation views is commonly used in semi-supervised learning, we avoid it in our controllable self-reinforcing optimization cycle for two key reasons. First, pseudo-label quality is highly sensitive to the distribution mismatch between labeled and unlabeled data in long-tailed semi-supervised learning. Second, propagating error pseudo-labels from weak augmentation views to strong augmentation views can reinforce the model’s learning of incorrect predictions, leading to error accumulation and performance degradation (as demonstrated in Figures 1 and 3).

Why is an additional class-aware adaptive augmentation necessary (Q2)

While logit adjustment induced Bayes-optimal classifiers can handle long-tailed distribution, the fundamental challenges of limited sample size and low diversity in minority classes remain unavoidable in long-tailed learning. Our class-aware adaptive augmentation specifically targets these practical limitations by explicitly enhancing minority class representations.

Why use an auxiliary branch (Q3)

Our method progressively expands the labeled dataset by identifying reliable pseudo-labels from the unlabeled dataset. However, during early training stages when model performance is still weak, only a limited number of samples meet the reliability condition for pseudo-labeling (as demonstrated in Figures 1 and 3, Appendix Figure 5). To ensure full utilization of all available training samples throughout the learning process, we introduce an auxiliary branch that: (i) guarantees 100% sample participation from the beginning of training, and (ii) provides additional supervision signals to stabilize representation learning during the critical initial phase.

Will the progressive incorporation finally approximate the ground-truth label distribution of the unlabeled dataset (Q4)

While our method does not explicitly estimate the unlabeled data distribution, the progressive incorporation of reliable pseudo-labels through our controllable self-reinforcing optimization cycle naturally approximates the ground-truth unlabeled data distribution as training progresses. This is because the model’s increasing confidence enables it to gradually include more diverse samples, including those from minority classes, improving distribution alignment. Empirical results in Appendix Figure 5 demonstrate that the pseudo-label distribution converges toward the ground-truth unlabeled data distribution over time.

Dear Reviewer Ckat,

Thanks again for your valuable comments. We are glad that our clarifications have addressed all of your concerns. Please raise the rating as your opinion is important to this work.

Regards from the authors.

Potentially fail to learn tail classes (W1)

We sincerely appreciate the reviewer’s insightful comments. We would like to clarify that our model does not fail to learn or classify tail classes. While the labeled data follows a long-tailed distribution, it does contain some tail class samples in the labeled dataset. Our approach enhances tail classes through logit adjustment and the proposed class-aware adaptive augmentation module, ensuring robust representation even when initial pseudo-label selection misses tail class samples. As shown in Figure 1, our method can generate more reliable pseudo-labels than FreeMatch and SimPro in tail classes like classes 1 and 2. Moreover, as demonstrated in Appendix Figure 5, our method progressively identifies nearly all samples as reliable pseudo-labels during training, with their final distribution closely approximating the true unlabeled data distribution.

Different confidence threshold selection (W2 & Q1)

The confidence threshold $t=0.95$ is a common setting in semi-supervised learning, which we adopt for fair comparison. To evaluate the impact of different confidence thresholds on model performance, we conduct additional experiments on CIFAR-10-LT (using the same experimental setup as in Table R1) with thresholds of $t=0.85$ and $t=0.75$. As shown in Table R4, our method’s performance variations across different thresholds remain relatively small (within 1-2%), and it maintains consistently superior performance compared to baseline approaches. This threshold-insensitive property highlights the stability of our method, and these results will be incorporated into the final version of the paper in Section 4.3 Analysis.

Table R4

Noisy scenarios (W3 & Q2)

To comprehensively evaluate our method’s robustness under noisy scenarios, we conduct additional experiments on CIFAR-10-LT ($N_{max}$=400, $M_{max}$=4600, $\gamma_l$=100, $\gamma_u$=100) under varying noise rates (0%, 10%, 20%, and 30%). The labeled data follows a long-tailed distribution, while the unlabeled data adheres to an arbitrary distribution. As demonstrated in Table R5, our method consistently outperforms baselines in noisy settings, achieving the highest accuracy across all noise rates with minimal performance degradation. Notably, when noise increases from 0% to 30%, our method exhibits a degradation (Δ) of only −12.46%, compared to −15.04% for FreeMatch and −29.93% for SimPro. This demonstrates our method's strong robustness to label noise, and these results will be added to Section 4.3 Analysis in the final paper.

Table R5

Large-scale or real-world datasets (Q3)

To further evaluate our method’s effectiveness in large-scale and real-world scenarios, we conduct experiments on ImageNet-127 (an imbalanced dataset, 127 classes, imbalance ratio 286) with an arbitrary (non-long-tailed) distribution. As shown in Table R6, our method consistently outperforms baselines, achieving gains of +1.27% at 32×32 resolution and +2.82% at 64×64 resolution. These results highlight our method’s robustness in large-scale, realistic settings beyond standard benchmarks. As suggested, these results and corresponding analysis will be incorporated into Section 4.2 Results in the final version.

Table R6

Reviewer ydDN,

We appreciate your valuable comments, which are important to improve the quality of this paper. We also appreciate your positive rating. As the deadline for the author-reviewer discussion period is approaching, please check whether your previous concerns have been fully addressed.

Looking forward to your reply. Thanks.

Regards.

The proposed method is supposed to target the long-tailed distribution. However, only accuracy is reported (W1 & Q3)

We appreciate the reviewer’s insightful comments. To better evaluate our method‘s performance under class imbalance, we have now included the Macro-F1 metric in our analysis. Table R1 presents results on two benchmarks: (i) CIFAR-10-LT ($N_{max}=400$, $M_{max}=4600$, $\gamma_l=100$, $\gamma_u=100$) in the left four columns, and (ii) CIFAR-100-LT ($N_{max}=50$, $M_{max}=450$, $\gamma_l=10$, $\gamma_u=10$) in the right four columns. In both cases, the labeled data follows a long-tailed distribution, while the unlabeled data varies across long-tailed, inverse long-tailed, and arbitrary distributions.

The results in Table R1 show that our method achieves statistically significant improvements in Macro-F1 over existing baselines, further validating its effectiveness for imbalance learning. Notably, it yields average Macro-F1 gains of 13.84% on CIFAR-10-LT and 4.55% on CIFAR-100-LT. These results and analysis will be added to Section 4.2 Results in the final version.

Table R1

The plots on page 9 feel crowded and break the reading flow (W2)

As suggested, we will improve the clarity and layout of the plots on page 9 to enhance readability in the final version.

Performance drops as class cardinality grows (for example on CIFAR-100-LT) (W3.1 & Q1)

We respond to the reviewer’s concerns regarding the relationship between performance gains and class cardinality. CIFAR-100 and CIFAR-10 have the same total training set size, but CIFAR-100 divides the data into 100 fine-grained classes (compared to 10 in CIFAR-10), reducing per-class samples by an order of magnitude. The finer class granularity also increases inter-class similarity, making classification inherently more challenging. As a result, the upper-bound performance (e.g., supervised learning) on CIFAR-100-LT (64.62%) is significantly lower than on CIFAR-10-LT (85.22%).

Moreover, consistency regularization enhances model stability by encouraging similar predictions for different augmentations of the same sample. This promotes robust representations, preventing severe performance degradation and ensuring that the lower-bound performance on CIFAR-100-LT remains relatively stable.

Given the narrower performance gap between upper and lower bounds in CIFAR-100-LT (compared to CIFAR-10-LT), the absolute improvement achievable by our method is inherently constrained. Nevertheless, our method still achieves substantial average accuracy gains of 3.09% on CIFAR-100-LT (as shown in Table 2). These analysis will be added to Section 4.2 Results in the final version.

Evaluation on other modalities (W3.2 & Q4)

While our method is evaluated on vision tasks, the core principles of CPG can be extended to other modalities. In this rebuttal, we further evaluate our method on the audio modality. Specifically, for audio data, this adaptation requires two key modifications: (i) replacing vision-specific backbones (e.g., WideResNet-28-2) with audio-compatible models like HuBERT, and (ii) substituting image augmentations with audio-specific ones. To demonstrate this adaptability, we perform our CPG on ESC-50, a standard benchmark for environmental sound classification. As demonstrated in Table R2, our method achieves state-of-the-art performance on this audio benchmark, confirming its effectiveness beyond visual modality. These results strongly support the generalizability of CPG's framework across different modalities. These results and analysis will be added to Section 4.2 Results in the final version.

Table R2

Clarify how CPG diverges from other methods like FixMatch/FlexMatch (W4)

While CPG adopts confidence-based filtering similar to FixMatch/FlexMatch, its key innovation is the controllable self-reinforcing optimization cycle, which incorporates reliable pseudo-labels from the unlabeled dataset into the labeled dataset rather than employing consistency regularization, distinguishing it from existing methods. This cycle operates in three steps: (i) expanding the labeled dataset with reliable pseudo-labels, (ii) constructing a Bayesian classifier, and (iii) iteratively improving pseudo-label quality, forming a theoretically grounded feedback loop. Additionally, CPG ensures full sample utilization via an auxiliary branch and reduces label noise accumulation through a voting-based stabilization technique, which enforces consistent pseudo-label assignments across training steps. As evidenced by the ablation studies in Table 5, these components individually contribute significant performance improvements, with the auxiliary branch and optimization cycle yielding average gains of 1.82% and 6.97%, respectively.

Different confidence threshold selection (Q2)

The confidence threshold $t=0.95$ is a common setting in semi-supervised learning, which we adopt for fair comparison. To evaluate the impact of different confidence thresholds on model performance, we conduct additional experiments on CIFAR-10-LT (using the same experimental setup as in Table R1) with thresholds of $t=0.85$ and $t=0.75$. As shown in Table R3, our method’s performance variations across different thresholds remain relatively small (within 1-2%), and it maintains consistently superior performance compared to baseline approaches. This threshold-insensitive property highlights the stability of our method, and these results will be incorporated into the final version of the paper in Section 4.3 Analysis.

Table R3

Clarify equation notation (Q5)

To improve notational clarity, we will modify the indexing in line 470 so that labeled data indices run from 1 to $N$ and unlabeled data indices run from $N+1$ to $N+M$ in the final version.

Dear Reviewer TLMt,

We are glad your concerns have all been solved. We appreciate your recommendation for acceptance. We will include the new experiments and results in the final paper. Thanks.

Regards.