The Role of Label Noise in the Feature Learning Process

Thank you for your great efforts on the review of this paper and for your constructive comments on our work. Below we address your questions and comments in details.

---

Q1: “The novelty seems limited: the analysis mostly follows the framework in (Kou et al. 2023) with modifications to settings with label noise”.

A1: First, we would like to highlight that the goal of this paper is to understand the feature learning dynamics in the presence of label noise, whereas [1] focused on characterizing the difference between benign and harmful overfitting. This necessitates significant changes in our theoretical setups compared to [1]. In particular, we emphasize that both the change in label noise and in SNR are critical that renders our settings significantly different compared to [1]. We have now added Appendix B that explicitly compares the differences in the proof techniques. In particular, [1] works under $n SNR^2 = o(1)$ while our setting requires $n SNR^2 = \Theta(1)$. Without the constant order of $n SNR^2$, either signal or noise dominates throughout all iterations and therefore, we cannot characterize the two-stage distinct behaviors of signal and noise learning.

Such a change in SNR brings new challenges in the convergence and generalization analysis as we have discussed in Appendix B. Specifically, many of the key techniques developed in [2] are no longer applicable in our case, which includes time-invariant of coefficients $\gamma_{j,r}^{(t)}/\sum_{i} \overline{\rho}_{j,r,i}^{(t)} = \Theta(SNR^2)$ and automatic balancing of updates $\ell_i’^{(t)}/\ell_k’^{(t)} \leq C$. These techniques only hold under $n SNR^2 = o(1)$ where signal learning is negligible compared to noise learning and thus implies the learning patterns across all samples are similar. In our case, to characterize the two-stage behaviors, we need to distinguish the learning dynamics of clean and noisy samples, which requires to develop novel proof techniques. We believe our contributions relative to [1] is non-trivial given our new settings and results.

---

Q2: Add some explanation on the model setup. "How this model can be realized, as it needs to distinguish signal and noise into different parts for activation in advance?"

A2: Thank you for the question. The synthetic setup is designed to better characterize the feature learning dynamics, enabling us to separately analyze the evolution of signal and noise learning. However, this does not imply that the model must distinguish signal and noise beforehand. The patch positions are irrelevant due to the use of shared weights, which ensures that the model treats all patches uniformly during training.

---

Q3: On the class-conditional noise.

A3: Thank you for pointing this out. In the current setting, the class-conditional noise indeed does not lead to different learning behaviors. We have added a remark in Page 6 of the revised version.

---

Q4: Generalization to multi-class classification. “Note that current analysis focuses on binary classification only,.... As such, some discussion on whether the analysis framework can be generalized to multi-class classification is certainly welcome here”.

A4: We are glad to discuss more on our theoretical setup.

• Clarification of our choice of binary classification: First, we would like to clarify that the current binary classification setting is sufficient to illustrate the role of label noise in the feature learning process of neural networks while maintaining mathematical tractability. While we agree that generalizing to multi-class classification is an interesting direction, it goes beyond the primary focus of this paper and we will leave it for future research.

• Possibility of generalization to multi-class classification: Yes, we believe the current framework can be generalized to handle multiple classes with additional setups. That is, we assume each class is represented by a distinct signal vector, i.e., $\mu_1, \mu_2, …, \mu_k$ for $k$ classes, where the labels are encoded as one-hot vectors. The signal vectors are required to be orthogonal such that the current analysis can be naturally adapted to multi-class settings.

---

Q5: Application to sample-dependent noise.

A5: Thank you for your insightful question. Yes, we believe that the feature learning framework we adopted in this paper can be applied to sample-dependent label noise. However, a more rigorous analysis would require a more comprehensive definition of such dependence. This necessitates a careful reformation of the original problem, such as by introducing multiple features and might need to develop new proof techniques, which lies beyond the scope of this paper and is left for future research.

---

Reference

[1] Kou, Yiwen et al. Benign Overfitting in Two-layer ReLU Convolutional Neural Networks. ICML 2023.

Dear Reviewer AvJg,

We hope that our responses could adequately address your concerns. As the deadline of this discussion phase is approaching, we warmly welcome further discussion regarding any additional concerns that you may have, and we sincerely hope you can reconsider the rating accordingly.

Thank you for the time and appreciation that you have dedicated to our work.

Best regards,

Authors of submission 3468

Thank you for your great efforts on the review of this paper and for your constructive comments on our work. Below we address your questions and comments in details.

---

Q1: On the contributions relative to [2]. Differences in settings and differences in proofs.

A1: First, we emphasize that the goal of this paper is to understand the feature learning dynamics in the presence of label noise, whereas [2] focused on characterizing the difference between benign and harmful overfitting. This necessitates significant changes in our theoretical setups compared to [2]. Specifically, without the condition that $n SNR^2 = \Theta(1)$, either the signal or noise dominates throughout all training stages, making it infeasible to prove the two-stage behavior. This results in a fundamentally different regime compared to [2], which requires novel proof techniques and refined analysis. Below we discuss differences in settings and proof techniques in details.

• Differences in setting: We have added Table 2 in Appendix A, explicitly comparing the conditions to [2]. The differences lie in conditions in label noise ratio, SNR and dimensionality. The change in label noise $\tau$ is to ensure the effect of label noise can be visibly characterized. The change in SNR is required to derive the two-stage behaviors of signal and noise learning. The Change in dimensionality is partially due to the change in SNR, as we no longer require $d$ to be sufficiently large for $n SNR^2 = o(1)$, as in [2]. Specifically, the conditions on $d$ differ by $||\mu||/\sigma_\xi$ up to some logarithmic factors. We agree that it may be inappropriate to claim we have a milder condition on $d$, without quantifying the scale of $||\mu||$ and $\sigma_\xi$. To avoid confusion, we have removed the claim on “mild condition” in the revised version.

• Differences in proofs: We are encouraged that the reviewer acknowledges that our theoretical setting is significantly different from [2] where the signal learning is always dominated by noise learning. In our case, signal and noise learning are on the same scale due to the constant order $n SNR^2$. Consequently, the key proof techniques employed by [2]--such as time-invariant of coefficients $\gamma_{j,r}^{(t)}/\sum_{i} \overline{\rho}_{j,r,i}^{(t)} = \Theta(SNR^2)$ and automatic balancing of updates $\ell_i’^{(t)}/\ell_k’^{(t)} \leq C$–are no longer valid in our case. In our setting, it is necessary to separately analyze the behaviors of clean and noisy samples in two stages, as balanced behaviors across all samples are not observed. To address this, we developed novel proof techniques tailored for such cases, resulting in a non-trivial and refined analysis. We have included Appendix A and Appendix B in the revised manuscript to explicitly contrast our work with [2] in terms of the required conditions and proof techniques.

---

Q2: “Further experiments investigating early stopping and its relation to the two-stage dynamics would be useful.”

A2: Thank you for the suggestion. To verify the relation between early stopping and the two-stage behavior, we have conducted new experiments (see experiments in Appendix G of the revised manuscript) to investigate the temporal correlation between the early stopping point and the end point of Stage I. We consider various signal strength and label noise ratio. Across all the settings, we see a clear match between the stages of signal dominating noise learning and zero training accuracy on noisy samples. This strongly suggests that the early stopping point could be reliably identified by the transition from Stage I to Stage II.

---

Q3: Separate train/test accuracy in Figure 2.

A3: Thank you for the suggestion. We have now separated the train and test accuracy in Figure 2 to improve readability.

---

Q4: Practical experiments focusing on the two-stage dynamics.

A4: To demonstrate the two-stage behavior in practical scenarios, we have added new interpretations (see Appendix G) of model predictions (via SHAP) on the CIFAR-10 dataset, across various model checkpoints. The purpose is to illustrate two-stage behavior for noisy samples throughout the training dynamics. We have plotted the results for epoch 1, 41, 81, 121 on noisy samples, i.e., samples with label flip. The figures clearly show that in the first stage (represented by epoch 1, 41), the model learns generalizable features, such as wings of airplanes and contours of automobiles. This stage corresponds to the stage where signal dominates noise learning. In contrast, for the second stage (represented by epoch 81, 121), the model overfits to partial features and the model interpretation via SHAP does not show a clear pattern of the objects.

Thank you for the detailed response and revisions. Appendices A and B in the revision are very helpful and definitely clarify the differences with the prior work more than before. The additional experiments are also encouraging. Overall I do think there are some interesting ideas in this work and the authors have addressed my main questions; I have thus updated my rating from 5 $\to$ 6. However, I do feel in agreement with the other reviewers that the contribution in terms of novelty is low relative to the prior work, and I do not feel strongly in favor of acceptance so I have updated my confidence to a 2 to reflect this. Given the borderline nature of the other reviews, I will try my best to do a deeper dive into the technical details of both this work and that of Kou et al. (2023) and come back to update my confidence rating accordingly.

Minor comment: it would be very helpful to make the font size in the plots bigger.

Q2: “The final conclusion is not surprising… It is not clear the extra contribution over those papers is significant enough.”

R2: We think the reviewer’s assertion that our contribution is not surprising/significant is rather subjective and insufficiently justified. To begin with, we would like to emphasize that the goal of this paper is to understand the feature learning process with label noise, while “those papers” [1, 2] focused on the benign overfitting phenomenon. This distinguishes our different yet significant contributions from others. To further clarify, we highlight our contributions in the following aspects:

(1) Technical contributions: Compared to existing theoretical studies [1,2], many of our theoretical settings are notably different. For example, [1] considered an activation function of $(ReLU(\cdot))^q$ with $q \gg 3$ and ignored label noise in the data setup.

[2] is more relevant to our work. However, [2] assumed $n SNR^2$ to be sufficiently small. Thus, they failed to capture the distinct two-stage behaviors caused by label noise and cannot characterize the adverse effect of label noise on generalization. In contrast, once $n SNR^2$ is of constant order, we can rigorously prove the two-stage behavior induced by label noise. This behavior is more aligned with empirical evidence, where the memorization effect emerges and the generalization ability degrades.

It is worth noting that such significantly different settings require new proof techniques and refined analysis. Because the key proof techniques of [2], including time-invariant coefficients $\gamma_{j,r}^{(t)}/\sum_{i} \overline{\rho}_{j,r,i}^{(t)} = \Theta(SNR^2)$ and automatic balancing of updates $\ell_i’^{(t)}/\ell_k’^{(t)} \leq C$ are no longer applicable. Consequently, we developed novel methods to analyze clean and noisy samples separately, which are critical to capturing the unique effects of label noise in this regime. In addition, we have listed all the key quantities that are different to [2] in two stages, including the monotonicity of signal learning, signal-to-noise-learning ratio, sample-wise prediction and test error.

We have now added Appendix A and B in the revised manuscript to explicitly compare the differences in conditions and proof techniques with [2].

(2) Valuable insights into practice: Other than rigorous theoretical contributions, our work offers valuable insights into practice. Specifically, we show that the adverse effect of label noise arises because noise learning overtakes signal learning for noisy samples. This insight not only explains the (in)effectiveness of strategies for addressing label noise, such as early stopping and sample selection, but also lays a theoretical foundation for developing more effective techniques. We believe our analysis offers valuable guidance for future research in mitigating label noise.

---

Q3: On other technical issues.

R3: After thoroughly reviewing our paper, we have not identified any technical issues with the theorem or its proof. We would greatly appreciate it if the reviewer could specify the potential issues identified so that we can provide further clarification.

---

Q4: On the explanation of $\overline \rho$ and $\underline \rho$.

R4: The dynamics of $\overline \rho$ and $\underline \rho$ are not symmetric. This is due to ReLU activation, where the positive part of the inner product keeps increasing while the negative part cannot decrease significantly before it becomes deactivated by ReLU. This eventually results in $\overline \rho$ being significantly larger than $\underline \rho$ in magnitude.

---

Reference:

[1] Cao et al. Benign overfitting in two-layer convolutional neural networks. NeurIPS 2022.

[2] Kou et al. Benign overfitting in two-layer ReLU convolutional neural networks. ICML 2023.

[3] Meng et al. Benign overfitting in two-layer ReLU convolutional neural networks for XOR data. ICML 2024.

[4] Chen et al. Understanding and improving feature learning for out-of-distribution generalization. NeurIPS 2023.

[5] Chen et al. Why does sharpness-aware minimization generalize better than sgd? NeurIPS 2023.

[6] Allen-Zhu & Li. Towards understanding ensemble, knowledge distillation and self-distillation in deep learning. ICLR 2023.

Thank you for your great efforts in reviewing our paper and for your constructive comments on our work. Below we address your questions and comments in detail.

---

Q1: “The setting is extremely simplistic. The setting essentially corresponds to linearly separable data with label noise... The model is more convoluted and assumes that you know the split of the data x into the noise component and the signal component. This seems very artificial and unrealistic. Please justify this setting of linearly separable data with an unnecessarily complex model in a self contained manner.”

R1: Thank you for your question.

• Clarification of our goal: First, we would like to emphasize that the goal of this paper is to understand the feature learning dynamics of neural networks in the presence of label noise. The use of non-linear and two-layer model is to simulate the behavior of neural networks in practice, in contrast to linear models such as logistic regression.

• Common practice in theoretical analyses: In addition, the setting of linearly separable data and two-layer CNN model has been widely adopted for analyzing classification models [1,2,3,4,5]. This simplified data model effectively captures the essential aspects of training dynamics and the generalization ability of neural networks [1,6], while ensuring mathematical tractability.

• Non-trivial theoretical analyses: Despite the simplified theoretical setup, significant challenges remain. The non-convex nature of the objective function, combined with the complex dynamics introduced by label noise, makes it highly non-trivial to precisely characterize the entire training dynamics and generalization behavior.

• Comparison to prior works: We also highlight that compared to prior theoretical works on label noise, our settings more closely reflect the real-world scenarios. As we have discussed in the related work section, existing studies either assumed lazy training, or focused on the early-stage dynamics, failing to capture the entire training process as comprehensively as demonstrated in this paper.

• Extensive empirical evidence: Moreover, the alignment between our real-world experiments and the theoretical results supports the validity of our theoretical setting. This suggests that our framework effectively captures the key factors in characterizing the effects of label noise in more realistic scenarios.

Thank you for the prompt reply. We are glad that our responses have resolved your other questions.

Regarding your last doubt on the sign of $(\overline{\rho} - \underline{\rho})$, we first emphasize that from the defition of $\overline{\rho}$ and $\underline{\rho}$ (in line 209), only one of $\overline{\rho}\_{j,r,i}$ and $\underline{\rho}\_{j,r,i}$ is nonzero for any $j,r,i$. Therefore, $\langle w\_{j,r}, \xi_i \rangle \approx \overline{\rho}\_{j,r,i}$ for $\tilde y_i = j$ and $\langle w\_{j,r}, \xi\_i \rangle \approx \underline{\rho}\_{j,r,i}$ for $\tilde y_i \neq j$. Consequently, the sign of $\langle w_{j,r}, \xi_i \rangle$ is determined by either $\overline{\rho}\_{j,r,i}$ or $\underline{\rho}\_{j,r,i}$, not both. This implies that not all the inner products are positive even though equation 3 holds for all $j,r,i$.

We hope this has clarified your doubt. We are happy to elaborate if needed. If no further questions remain, we would be greatly appreciated if you could consider re-evaluating our work.