Learning Imbalanced Data with Beneficial Label Noise

Reply to Information Loss caused by label flipping in Claims And Evidence and Weakness 2

• We sincerely appreciate your insight on potential information loss due to label flipping. In LNR, the majority-class samples selected for flipping are primarily outliers that have deeply encroached into the minority-class region beyond the decision boundary. In this way, LNR enriches the minority class while preserving the core distribution of the majority class.

• Compared to undersampling approaches that discard large numbers of majority class samples (especially in extreme imbalance scenarios), LNR requires significantly fewer data-editing (only 94 label flips on CIFAR10-LT), mitigating information loss.

• The number of flips can be regulated through the threshold $t_{flip}$, which can be optimally selected through cross-validation to avoid unexpected negative impacts on majority class performance. Sensitivity analysis (Appendix C.3) and Tables 1–2 show that with a well-calibrated $t_{flip}$, LNR enhances minority-class performance without compromising overall accuracy—a justified trade-off given the empirical gains.

• In the revised version, we will adjust our claim on "information loss," acknowledging that trade-offs (and information loss) exist but can be well-controlled.

Reply to comment in Essential References Not Discussed section and Weakness 1

• We appreciate your valuable feedback on incorporating more recent literature. We will add discussions on the latest approaches in imbalanced learning (please refer to our response to Reviewer AMhG for details).

• To strengthen our comparative analysis, we have included comparisons with state-of-the-art methods such as MiSLAS, ReMix, and SelMix. However, we encountered some practical challenges in implementation: ReMix does not provide official code, and SelMix lacks reproducible code for its supervised version. Experiments on CIFAR100 and ImageNet require more time for proper adaptation and reproduction due to framework-specific complexities. We will include these results during the rebuttal period to provide a more comprehensive comparison once we have the results. Below are the CIFAR10 (IR=100) results:

$\bigstar$ It is worth highlighting that SelMix requires an additional balanced validation set of size 5000, while our LNR does not, yet still achieves comparable better performance to SelMix.

Reply to Weakness 1 We thank the reviewer for noting our method's and SelMix's conceptual similarities. While both methods share the common goal of improving imbalanced classification, key differences exist:

• SelMix computes a gain matrix using class centroids from a balanced validation set to guide its mixup sampling at the class level. While this shares some conceptual similarities with our label noise selection process based on feature similarity ranks, LNR differs fundamentally: We only flip labels of majority class samples that exhibit instance-level feature similarity to minority classes without requiring any auxiliary data and no feature mixing nor sample generating.

• In LNR, any sample with features similar to those of the minority class may be flipped, regardless of how far or close the feature centroids of the two classes are. For instance, in Figure 1, although the feature centroid of majority class 2 is relatively closer to class 9, the samples flipped to class 9 by LNR primarily come from classes 4 and 8, as these show stronger instance-level similarity to the minority class.

Notably, as shown in our CIFAR-10 experiments, LNR achieves comparable performance to SelMix while eliminating its dependency on balanced validation data. This makes our approach more practical for real-world scenarios where such balanced datasets may be unavailable.

We will expand our discussion of these comparisons in the revised manuscript (Section 3.2) to better highlight LNR's unique contributions to the field.

Reply to Weakness 3&4

• W3: Our KEEL experiments with classic ML methods (KNN, CART) primarily validate our theoretical results and demonstrate the LNR's model-data-agnostic advantages for offering higher performance by comparing with classical methods for tabular data. The comparison with recent methods is conducted on long-tailed multi-class image data using DNN models, where these methods are explicitly targeted. Our results show consistent gains over algorithm-level methods and SOTA performance over data-level methods, highlighting our contributions.

• W4: Kindly refer to the results and analysis of synthetic data in Appendix C.7 (after references), including tables of full results.

Reply to Relation To Broader Scientific Literature & Essential References Not Discussed & Weakness 1 and 2

• We sincerely appreciate you providing these books and the two recent articles—they are very insightful. We recognize that some of these prior studies share similar conclusions with the theoretical portion of our paper, and we will adequately acknowledge these overlaps in our revised manuscript. Additionally, we will supplement our discussion with existing findings on how class imbalance affects decision boundaries, including the articles you kindly pointed out.

• However, we would like to kindly emphasize that the core contribution of our theory lies in proving the impact of our proposed asymmetric instance-dependent label noise model on decision boundaries under class imbalance. Like other imbalanced learning methods, the mechanism of how data imbalance biases the classifier's decision boundary serves as the motivation for our theoretical framework. Yet, by introducing asymmetric label noise through our defined beneficial label noise model, we theoretically demonstrate how our beneficial asymmetric noise modifies decision boundaries and empirically validate that this approach effectively and beneficially corrects biased decision boundaries.

Reply to Weakness 3 and 4

• W3: We appreciate your suggestions about our baseline comparisons and relevant works.

  • In response to Reviewer AMhG, we have expanded the discussion of related literature (i.e. logit-adjustment approaches and metrics-optimization approaches).
  • We also included new experimental comparison results with MiSLAS, ReMix, and SelMix (most recent SOTA) in our response to Reviewer k9AK. We further clarified some fundamental differences between our method and SelMix. We hope these revisions and explanations adequately address your concerns, and we are deeply grateful for your insightful feedback.

• W4: We sincerely appreciate your suggestion regarding comparative experiments on large-scale datasets. We fully acknowledge that such comparisons could provide a more comprehensive evaluation of our approach.

  • We are currently conducting additional comparisons on ImageNet with state-of-the-art methods. As these experiments require more time to complete, we will promptly share the results with you should they become available during the rebuttal period.
  • In addition, we would like to re-emphasize our primary research focus has been on validating the feasibility and effectiveness of feature-dependent asymmetric label noise in addressing the class imbalance, which we have thoroughly demonstrated through extensive experiments on CIFAR-10/100 under both step-wise and long-tailed imbalance scenarios.

Reply to Claims And Evidence and Methods And Evaluation Criteria

Thank you for your thoughtful comments. We appreciate your feedback and would like to clarify that our evidence sufficiently supports our claims.

• Our theoretical analysis focuses on the deviation between the optimal F1 decision boundary and the optimal Bayes decision boundary (rather than other metrics) because F1-score is typically the primary evaluation criterion for imbalanced binary classification tasks, especially when minority-class recognition is crucial, as in fraud detection scenarios.

• Through a comparative analysis of how optimal decision boundaries are affected by both label noise and imbalance ratio, we designed a feature-dependent asymmetric label noise model. Our primary theoretical contribution lies in LNR's ability to make the label noise's effect on decision boundaries counterbalance the impact of class imbalance—effectively correcting the boundary shift.

• We would also like to kindly emphasize that our evaluation metrics are comprehensive and widely accepted. These include F1, G-Mean, and AUC for binary classification tasks and Many/Medium/Few-shot metrics, which are broadly used for imbalanced multi-class classification and acknowledged by Reviewer k9AK and other reviewers.

• The experimental results in Tables 1-2, along with a comparison against the latest SOTA methods in our response to Reviewer k9AK, provide sufficient evidence of the effectiveness of our approach. In our latest comparison with ReMix/SelMix based on MiSLAS, we also included Expected Calibration Error (ECE) to address potential concerns regarding model calibration when modifying the data. As shown in the results, LNR, due to its zero feature editing of the data, not only enhances performance but also achieves a significantly lower ECE than ReMix.

Reply to Relation To Broader Scientific Literature & Essential References Not Discussed

We sincerely appreciate your valuable feedback regarding improvements to our literature review. In response to your suggestion, we will expand the discussion of methods related to our work, including generative and mixup-based approaches, in the revised manuscript. We will revise the literature review as described in our response to Reviewer AMhG for your reference. In our response to Reviewer k9AK, we have included updated comparative results with the recent ReMix and SelMix approaches. We have also clarified the fundamental distinctions between our method and SelMix in greater detail. We sincerely hope that these additional analyses and explanations will help address your concerns.

Reply to Weaknesses and Questions

• W1/Q1: We sincerely appreciate the reviewer's insightful suggestion regarding multi-class generalization, which is indeed an important direction for future research. While extending the theory to multi-class settings represents valuable future work, our current study specifically focuses on using theoretical analysis to motivate our carefully designed feature-dependent asymmetric label noise model. In multi-class scenarios where LNR flips labels from a majority class i to a minority class j, our binary classification analysis of decision boundaries remains applicable to understanding the boundary between classes i and j specifically.

• W2/Q2: In our response to Reviewer k9AK, we have added a comparison with SelMix, the latest state-of-the-art metric-optimizing method. Although our method, LNR, does not assume the availability of a sufficiently large balanced validation set for metric optimization, its performance remains comparable to that of SelMix. This makes LNR more advantageous in practical applications.

• W3/Q3:

  • Thank you for your insightful comment. In Section 4.3, we emphasize that LNR mitigates the impact on feature extraction by postponing the introduction of label noise during the fine-tuning stage, thus avoiding any potential threat to feature representations. As such, LNR does not involve risks to feature extraction. We would be happy to include a comparison of feature representations after the noise introduction in the appendix of the revised manuscript to address your concern better.

  • The fairness changes after introducing noise, as demonstrated on the CIFAR-10 confusion matrix, were primarily intended to show that these noises do not disrupt the existing fairness. On the contrary, they contribute to improved model fairness. These conclusions are also evident in the results of the many/medium/few-shot tasks on CIFAR-100. If there are specific fairness metrics you would like us to report, we would be pleased to address them during the rebuttal period.

Response to ‘Essential References Not Discussed’， Weakness 1 and Question 1

We sincerely thank the reviewer for highlighting the need to include recent methodological advancements in our literature review. As suggested, we will carefully revise the 'Related Work' section in our manuscript:

1. Algorithm-level methods refine loss functions or training paradigms to improve tail-class accuracy, primarily by decoupling feature learning from classifier training to separately enhance feature representation and classifier fine-tuning. Contrastive learning-based methods—including DRO-LT (2021), TSC (2021), BCL (2022), and SBCL (2023)—leverage contrastive losses during feature learning to boost feature discriminability and model robustness under imbalance. For classifier optimization, margin-based approaches like LDAM-DRW (2019) and τ-norm (2019) employ loss engineering to create larger decision margins for tail classes, while logit adjustment methods such as GCL (2022b) address softmax saturation by expanding the tail classes' embedding space through increased cloud sizes.

2. Data-level methods often leverage generative models or autoencoders such as $\Delta$-encoder (2018), DGC (2020), and RSG (2021) to synthesize few-shot samples. These approaches typically depend on high-quality pre-trained models, which can introduce additional challenges, especially on scarce tail-class data, limiting the ability to generate diverse or meaningful samples. Instead of involving a generative model, Mixup (2018) interpolates features and labels via a fixed mixing ratio $\lambda$, empirically demonstrating its effectiveness for data augmentation. Building on this, Remix (2020) introduces separate mixing ratios for features ($\lambda_x$) and labels ($\lambda_y$) to rebalance class distributions, though it retains random sampling. More recently, SelMix (2024) advanced this direction by selectively sampling pairs for mixing based on the gain on non-decomposable metrics (e.g., recall, G-mean), thereby enabling targeted improvements in specific metrics. However, SelMix's gain matrix relies on a balanced ($\alpha = 0.95$, where 1 means fully balanced) augmented auxiliary set (5000 for CIFAR). With imbalanced or small auxiliary data, its metric optimization fails to meet theoretical constraints. This limitation is particularly acute in practice where validation data is often scarce and inherently skewed—a gap our method intentionally addresses with a carefully designed noise model, which requires neither feature editing nor a balanced auxiliary dataset.

3. Multi-expert ensemble methods (e.g., RIDE (2021), TLC (2022), SADE (2023), and BalPoE (2023)) allocate specialized "experts" to model head- and tail-class features separately, achieving notable gains. While these approaches fall outside the scope of our work, they highlight the ensemble learning for tackling class imbalance.

Response to Weaknesses 2-4

• W2: Regarding the many/medium/few-shot categories, we acknowledge these thresholds vary across datasets. For clarity, we've detailed these specifications in Appendix C.7 and will add explicit cross-references in the main text. We sincerely appreciate you bringing these issues to our attention.

• W3: We have carefully re-examined the definition of False Negatives (FN) as: "A False Negative occurs when a ground-truth positive sample is incorrectly predicted as negative." Based on this standard definition, the formulation of FN/N in Section 3.2 is mathematically correct. Should there be any misunderstanding on our part regarding your specific concern, we would be grateful if you could further clarify your perspective so we can address it precisely.

• W4: All typographical errors identified in the manuscript, including those in Table 2, have now been carefully corrected. We are truly grateful for your meticulous review。

Response to Questions 2-3

Q2: We sincerely appreciate your thoughtful suggestion regarding Algorithm 1. To clarify the notation: $U \sim \mathcal{U}(0,1)$ indicates sampling a number U from the range [0,1], where label flipping occurs when $\rho(x) > U$. As you insightfully pointed out, this could alternatively be expressed as: $U \sim \mathcal{U}(\rho[x])$. If this alternative formulation better serves clarity, we would be delighted to incorporate this change in the manuscript.

Q3: To our knowledge, all class imbalance methods inevitably improve minority class performance at some cost to majority class accuracy. This trade-off is fundamental because:

• Their strong majority class performance inherently comes at the expense of minority classes
• Therefore, improving minority class recognition necessarily reduces the "overprivileged" majority class performance.

Our fairness analysis (confusion matrices) demonstrates this trade-off: On CIFAR-10, LNR introduced 94 label noises. This exchanged 16 true positives (TP) from head classes for 204 TP gains in tail classes