Learning Imbalanced Data with Beneficial Label Noise

The paper introduces the Label-Noise-based Rebalancing (LNR) method to address both binary and multi-class imbalanced classification by using an asymmetric label noise model. Unlike conventional data-level methods, which often suffer from information loss and generative errors, LNR rebalances data by flipping the labels of majority class samples. This technique improves the recognition of minority class samples and can be seamlessly integrated with any classifier or algorithm-level methods. Additionally, the authors provide a theoretical analysis of how label noise affects classifiers, proposing that controlled noise can boost imbalanced learning without the downsides of traditional resampling methods. Experimental results demonstrate the superiority of LNR.

The paper presents a novel approach called Label-Noise-based Rebalancing (LNR) to address the challenges posed by imbalanced data and label noise in classification tasks. The authors argue that traditional methods, such as resampling, often lead to information loss and generative errors. They propose that introducing beneficial label noise can help adjust biased decision boundaries, thereby improving classifier performance, particularly for minority classes. The LNR method employs an asymmetric label noise model that selectively flips labels of majority class samples to minority classes, aiming to enhance the performance of classifiers in both binary and multi-class settings. The authors validate their approach through extensive experiments on synthetic and real-world datasets, demonstrating its effectiveness compared to existing methods.

This paper proposes a new method called Label-Noise-based Re-balancing (LNR) to address the data imbalance problem by introducing label noise in the majority class samples in a classification task. The method aims to adjust the decision boundary and thus improve the classification accuracy of the minority class. The authors provide a detailed theoretical analysis of how label noise can mitigate the effects of class imbalance on decision boundaries. Extensive experimental results are reported in the main text and appendices, which seem to indicate that LNR demonstrates strong performance across a wide range of datasets and is applicable to various classifiers.

The authors proposed a method for dealing with class imbalance. The main problem with imbalanced datasets is that most labels are from the minority class and few of them are from the majority class. To deal with this, the authors proposed to flip some of the labels in order to balance the dataset and improve the performance of classification models. Moreover, the authors extended their method to multi-class classification problems. Finally, the numerical experiments showed promising results for the method, in a wide range of datasets, including tabular data classification and image data classification problems.

We thank all reviewers for their insightful and constructive feedback. We appreciate that the reviewers found our proposed method to be：

1. Generality and Versatility:

   • The method is generic and applicable to any imbalanced data classification problem. (Reviewer m5qW, M9Zh).

2. Theoretical Insights:

   • The paper provides a solid theoretical analysis of how data imbalance affects decision boundaries. The method is designed for easy integration with various classifiers and algorithm-level techniques (Reviewer M9Zh,vGZe).

3. Innovative Approach:

   • The introduction of a novel method to handle imbalanced data through asymmetric label noise avoids reliance on traditional resampling methods, preventing information loss and generative errors. This approach is supported by comprehensive mathematical proofs (Reviewer vGZe, kaNw).

4. Clarity and Effectiveness:

   • The paper presents a simple and effective method for addressing data imbalance along with extensive experiments to validate the effectiveness of the proposed method (Reviewer m5qW, kaNw).

This summary emphasizes the strengths of the paper, with specific endorsements from each reviewer.

Our work offers new insights into the impact of imbalanced data on decision boundaries, providing a foundational and universal approach to imbalanced learning. Specifically:

To our knowledge, we are the first to propose a method that artificially introduces label noise to correct the biased decision boundary. Our approach achieves data balance by simply flipping labels. This simplicity makes it versatile across various data formats and classification tasks, allowing seamless integration with other methods and classifiers. We believe our work opens new avenues for addressing the imbalance learning problem. We have incorporated the reviewers' suggestions by adding explanations and experimental results during the rebuttal period and hope these updates will be considered in the final evaluation.

We have diligently addressed the primary concerns raised by the area chair and each reviewer, which can be summarized into two major concerns:

1. The distinction between LNR and other data augmentation methods, such as Mixup.

2. The potential impact of the label noise introduced by LNR:

• whether it disrupts the feature extraction process and negatively affects classifier performance.
• whether the artificially introduced label noise undermines class fairness.

As a reply:

1. We succinctly summarize the distinction between LNR and other data augmentation methods.

• The fundamental objective of all data-level approaches is to enhance the feature representation of minority class samples. Before LNR, nearly all data-level methods focused on various strategies to generate new samples aimed at augmenting minority class features, constantly refining these processes to minimize generative errors, as exemplified by the advancements from Mixup to SelMix.
• In contrast, LNR adopts a fundamentally different approach by developing a beneficial noise model that involves flipping the labels of majority class samples. This enables the direct reutilization of majority class features to enrich the sample space of the minority class, without the necessity of generating any synthetic samples.
• By modifying only the labels, LNR effectively avoids generative errors while allowing for integration with other methods to provide additional performance enhancements, highlighting the novelty and versatility of our LNR method.

2. In response to the second concern:

• The t-flip parameter can effectively mitigate the negative impact on classifier performance caused by introducing excessive label noise. Since LNR is a data-driven method, the t-flip parameter can be validated through cross-validation (CV) to ensure that this label noise remains harmless.
• LNR employs a strategy similar to Deferred Re-Weighting/Deferred Re-Sampling, introducing label noise only during the fine-tuning phase, thereby effectively avoiding any adverse effects on the feature extraction process.
• The strength of label flipping in LNR depends on the similarity between samples, ensuring that class fairness is maintained. This is demonstrated by the presented confusion matrix on CIFAR-10-LT, which proves that the enhancements brought by LNR to GCL do not come at the expense of class fairness.

The difference in confusion matrices from GCL to GCL-LNR:

Dear Area Chair,

The discussion deadline is fast approaching. However, we have not yet received any response from Reviewer m5qW, who initially rated our work with scores of 5, respectively. We would greatly appreciate your guidance in facilitating communication with them.

We are pleased to share that the other three reviewers have engaged in constructive discussions regarding our rebuttal, leading two of them (Reviewer vGZe and Reviewer kaNw) to increase their scores after we addressed their concerns. We believe this positive feedback reflects the improvements made to our work.

Thank you very much for your attention to this matter. We are grateful for your support, and we look forward to your assistance.

Best regards,

Authors of Submission 9243

The paper concerns a problem of imbalance learning. The Authors introduce a new balancing technique that relies on random flipping of labels of training examples from the majority class. The numerical experiments show promising results of the introduced method.

The Reviewers find the paper well-written and appreciate the theoretical results, as well as the introduced method, which is model-agnostic, applicable to multi-class classification, and produces promising results. They also point out weaknesses of the paper such as limited discussion and comparison to related work or the choice of experimental metrics that makes the "experimental results not convincing enough."

As an AC I also posted my critical remarks concerning limited theoretical contribution and insufficient discussion on related work. In fact, the presented results are rather known, limited to two specific metrics without any generalization to a broader class of performance measures. The paper also lacks a clear theoretical analysis of the introduced method how well it can optimize a given metric of interest. Furthermore, the empirical part does not contain any comparison to methods that directly optimize the considered metrics.

Taking into account the above and the ratings of the Reviewers, which indicate this paper as borderline, I recommend to reject this paper.

Reject