On the Over-Memorization During Natural, Robust and Catastrophic Overfitting

Q4: Comparison With Traditional Methodologies

A4: While a range of traditional methodologies exist to separately address NO, RO and CO, the understanding and solutions for these overfitting types remain isolated from each other. This study first-time adopts a unified perspective on different types of overfitting by analyzing the memorization effect on each training pattern. Therefore, we primarily compare our method with both universal and data-centric approaches, such as baseline and data argumentation technologies. Although our method may not achieve the performance of specially tailored approaches, it stands out for holistically mitigating overfitting across various training paradigms. As a general framework, DOM can consistently achieve notable improvements over the baseline with a subset of training samples or data augmentation, which successfully supports our perspective.

If you have any further questions or concerns, please do not hesitate to contact us. We are willing and available to provide additional information you may need.

[1] Leslie Rice, Eric Wong, and Zico Kolter. Overfitting in adversarially robust deep learning. In International Conference on Machine Learning, pp. 8093–8104. PMLR, 2020.

[2] Maksym Andriushchenko and Nicolas Flammarion. Understanding and improving fast adversarial training. Advances in Neural Information Processing Systems, 33:16048–16059, 2020.

We sincerely appreciate your time and effort in reviewing our manuscript. After careful consideration, please find our responses to your comments below.

Q1: Different From MLCAT

A1: Different research perspective. MLCAT investigates RO by examining adversary strength, whereas our work explores different types of overfitting by focusing on the memorization effect. While MLCAT found that RO is mainly due to the transformed small-loss data, they did not provide further explanation for why these data negatively impact the model generalization. Different from them, our research delves deeper into the model's memory capacity for different training patterns and uncovers a notable persistent memory for transformed high-confidence patterns. This discovery suggests that the model's brute-force memorization of these patterns detrimentally affects its ability to generalize. To this end, we propose the concept of over-memorization characterized by the model demonstrating both high-confidence prediction and persistent memory.

Different research object. MLCAT explores the RO phenomenon through the observation of adversarial examples, whereas our research investigates different types of overfitting by concentrating solely on natural patterns. Specifically, we find that the AT-trained model, which does not encounter natural patterns, exhibits a similar memory tendency between natural and adversarial patterns in the over-memorization sample. This tendency manifests as: when the model over-memorizes certain adversarial patterns, it will simultaneously display high-confidence predictions for the corresponding natural patterns. By leveraging this tendency, we can reliably and consistently identify the over-memorization pattern by exclusively focusing on the natural training loss, regardless of the training paradigm.

Different research problem. MLCAT is designed to specifically address RO problems, whereas our study is dedicated to holistically understanding and alleviating different types of overfitting. Despite NO, RO and CO all demonstrating reduced model generalization and having similar aspects (e.g., learning algorithms and model structures), previous studies have revealed that they exhibit unique manifestations and mechanisms, thereby requiring independent solutions [1,2]. To bridge the gap among different types of overfitting, we introduce the shared behaviour over-memorization, which first-time adopts a unified perspective in understanding and addressing overfitting across various training paradigms.

Q2: Iterative Data Augmentation

A2: Our approach employs data augmentation to prevent the model from over-memorizing training patterns, thereby alleviating overfitting. However, due to the inherent randomness of the original data augmentation, straightforward use cannot reliably and consistently prevent over-memorization. To overcome this issue, our method $DOM_{DA}$ applies iterative data augmentation, which is strategically designed to maximize the likelihood of hindering over-memorization.

Q3: Gradually Learning Rate

A3: We have validated the effectiveness of our method within single-step AT, where the cyclical learning rate is the standard setting. To further assess our method, we conducted experiments using a gradual learning rate schedule in natural training. We set the cyclical learning rate schedule with 300 epochs, reaching the maximum learning rate of 0.2 at the midpoint of 150 epochs. We report the natural training test error at both the best and last checkpoint on CIFAR 10 using PreactResNet-18.

From the above table, it is apparent that although the cyclical learning rate reduces the model's generalization gap, it also leads to a reduction in performance compared to the step learning rate. Nevertheless, our method consistently showcases its effectiveness in improving model performance and completely eliminating the generalization gap by mitigating over-memorization. We have included this experimental result in Appendix H of the rebuttal revision.

While there may not be significant differences in the causes or solutions between NO and RO, I partially agree with the author's opinion that there could be distinctions with CO. Therefore, I acknowledge the contribution of identifying commonalities in these gaps and proposing a universal solution in a context where such distinctions are considered. However, I still find the analysis lacking novelty, as the methodology seems to be a mere transformation of the previous study, shifting the focus from AEs to clean ones.

Additionally, if different types of overfitting methodologies are distinctly addressed, as mentioned earlier, there should have been a demonstration of better or comparable performance compared to existing methodologies for each type of overfitting through a comparison with the existing approaches. Considering the level of contribution, given the current low score, I reevaluate the contribution by excluding the deficient areas and adjusting the score accordingly.

Thank you for your timely response, which has helped us address your concerns promptly. We are pleased to offer the following clarification to your concerns.

• Firstly, our paper (Fig. 2) provides an analysis of the model's memory capacity, which is not conducted in the MLCAT. This analysis offers deeper insight into why Trans-HC patterns negatively impact model generalization but Ori-HC does not, even though both of them are characterized as HC. We found that the model exhibits persistent memory for Trans-HC patterns, indicating that the brute-force memorization of these patterns hinders its generalization.

• Secondly, the MLCAT identify small-loss data based on the model's loss on adversarial examples. Different from them, our method uncovers the model's memory tendency between natural and adversarial patterns, allowing us to detect over-memorization adversarial examples solely based on the natural training loss.

• Finally, we respectfully disagree with the notion that the commonality between NO, RO, and CO can be reasonably anticipated. In contrast, to the best of our knowledge, all prior research has regarded them as completely distinct phenomena, with each research having own understanding and solutions that are incompatible with other types of overfitting. To bridge this gap among different types of overfitting, our study introduces the shared behaviour over-memorization that first-time adopts a unified perspective in understanding and addressing overfitting.

We appreciate your prompt feedback and are always open to further discussion. If you have any further concerns, please feel free to contact us. We are committed to providing any further clarifications you may need.

Dear Reviewer uFwf:

We hope this message finds you well. We are sincerely thankful for your effort in reviewing our manuscript. As a gentle reminder, we have submitted a rebuttal and a revised manuscript to address your mentioned concerns. Given the constraints of the discussion period, your prompt response would enable us to provide further clarification and explanation.

Best regards,

Authors

We sincerely appreciate your time and effort in reviewing our manuscript. After careful consideration, please find our responses to your comments below.

Q1: Hyperparameter Selection

A1: By utilizing the DOM framework, we have effectively verified and mitigated over-memorization, which negatively impacts DNNs' generalization ability. However, as a general framework, finding optimal hyperparameters for different paradigms and datasets can be cumbersome. While some hyperparameters exhibit a degree of universality (e.g. data argumentation strength $\beta$ and iteration $\gamma$), the primary challenge lies in establishing an appropriate loss threshold. To address this challenge, we propose to use a general and unified loss threshold applicable across all experimental settings. Specifically, we utilize an adaptive loss threshold [1], whose value is dependent on the loss of the model's current training batch. For all experiments, we set this adaptive loss threshold $\mathcal{T}$ to 40%, maintaining other hyperparameters as the original settings. We report the natural and PGD-20 test errors for natural training (NT) and adversarial training (AT) using PreactResNet-18, respectively.

The above table demonstrates the effectiveness of the adaptive loss threshold across different paradigms and datasets. This threshold can not only consistently identify over-memorization patterns and mitigate overfitting, but also be easily transferable without the need for hyperparameter tuning. We have included this experimental result in Appendix B of the rebuttal revision.

Q2: Computational Overhead

A2: We analyze the extra computational overhead incurred by the DOM framework. Notably, both $DOM_{RE}$ and $DOM_{DA}$ are implemented after the warm-up period (half of the training epoch). We report the before warm-up, after warm-up, and overall training time (epoch/second) on CIFAR10 using PreactResNet-18 with a single NVIDIA RTX 4090 GPU.

Based on the above table, we can observe that $DOM_{RE}$ does not involve any additional computational overhead. Although $DOM_{DA}$ require iterative forward propagation, its overall training time does not significantly increase, because the data augmentation is only applied to a limited number of epochs and training samples. Additionally, the multi-step and single-step AT inherently have a higher basic training time (generate adversarial perturbation), but the extra computational overhead introduced by the DOM framework is relatively consistent. As a result, our approach has a relatively smaller impact on the overall training overhead in these scenarios. We have included this experimental result in Appendix G of the rebuttal revision.

Q3: Pattern Definition

A3: Each training sample in the dataset has two patterns: the natural pattern corresponds to the original sample, and the adversarial pattern corresponds to the adversarial example (original sample plus the adversarial perturbation). In this work, we identify over-memorization samples by exclusively focusing on the natural pattern loss, regardless of the training paradigm.

Q4: Larger-scale Dataset

A4: We also verified the effectiveness of our method on the larger-scale dataset Tiny-ImageNet. We set the loss threshold $\mathcal{T}$ to 0.2, and other hyperparameters remain as the original settings. We report the natural training test error at both the best and last checkpoint using PreactResNet-18.

The above table illustrates the effectiveness of our method, $DOM_{RE}$ and $DOM_{DA}$, on the Tiny-ImageNet dataset. These results indicate that preventing over-memorization can improve model performance and reduce the generalization gap on large-scale datasets. We have included this experimental result in Appendix F of the rebuttal revision.

Q5: Transformer-based Architecture

A5: We have validated the effectiveness of our method within CNN-based architectures, demonstrating its ability to alleviate overfitting by preventing over-memorization. To further substantiate our perspective, we verify our method on the Transformer-based architecture. Constrained by computational resources, we trained a ViT-small model, initializing it with pre-trained weights from the Timm Python library. The training spanned 100 epochs, starting with an initial learning rate of 0.001 and divided by 10 at the 50th and 75th epochs. We set the batch-size to 64 and the loss threshold $\mathcal{T}$ to 0.1, maintaining other hyperparameters as the original settings. We report the natural training test error at both the best and last checkpoint on CIFAR 10.

The above table shows the effectiveness of our method on the Transformer-based architecture. By mitigating over-memorization, both $DOM_{RE}$ and $DOM_{DA}$ not only improve model performance at both the best and last checkpoints, but also contribute to alleviating overfitting. We have included this experimental result in Appendix E of the rebuttal revision.

Q6: Compare With Other Techniques

A6: While a range of techniques exist to separately address NO, RO and CO, the understanding and solutions for these overfitting types remain isolated from each other. This study first-time adopts a unified perspective on different types of overfitting by analyzing the memorization effect on each training pattern. Therefore, we primarily compare our method with both universal and data-centric approaches, such as baseline and data argumentation technologies. Although our method may not achieve the performance of specially tailored approaches, it stands out for holistically mitigating overfitting across various training paradigms. As a general framework, DOM can consistently achieve notable improvements over the baseline with a subset of training samples or data augmentation, which successfully supports our perspective.

If you have any further questions or concerns, please do not hesitate to contact us. We are willing and available to provide additional information you may need.

[1] Berthelot, D., Roelofs, R., Sohn, K., Carlini, N., & Kurakin, A. (2021, October). AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptation. In International Conference on Learning Representations.

Dear Reviewer mcnh:

We hope this message finds you well. We are sincerely thankful for your effort in reviewing our manuscript. As a gentle reminder, we have submitted a rebuttal and a revised manuscript to address your mentioned concerns. Given the constraints of the discussion period, your prompt response would enable us to provide further clarification and explanation.

Best regards,

Authors

Dear Reviewer mcnh,

Thanks a lot for your valuable efforts in reviewing our manuscript. Just a kind reminder that the discussion stage is closing soon. If there are any unclear explanations or descriptions, we can clarify them further.

Best regards,

Authors

We sincerely appreciate your time and effort in reviewing our manuscript. After careful consideration, please find our responses to your comments below.

Q1: Loss Threshold Selection

A1: By utilizing the fixed loss threshold DOM, we have effectively verified and mitigated over-memorization, which negatively impacts DNNs' generalization ability. However, as a general framework, finding an optimal loss threshold for different paradigms and datasets can be cumbersome. To address this challenge, we propose to use a general and unified loss threshold applicable across all experimental settings. Specifically, we utilize an adaptive loss threshold [1], whose value is dependent on the loss of the model's current training batch. For all experiments, we set this adaptive loss threshold $\mathcal{T}$ to 40%, maintaining other hyperparameters as the original settings. We report the natural and PGD-20 test errors for natural training (NT) and adversarial training (AT) using PreactResNet-18, respectively.

The above table demonstrates the effectiveness of the adaptive loss threshold across different paradigms and datasets. This threshold can not only consistently identify over-memorization patterns and mitigate overfitting, but also be easily transferable without the need for hyperparameter tuning. We have included this experimental result in Appendix B of the rebuttal revision.

Q2:Detailed Experiment Settings

A2: In Section 3, we conducted all experiments on the CIFAR-10 dataset using PreactResNet-18. We analyzed the proportion of natural and adversarial patterns by examining the respective natural and adversarial training loss. In Section 3.1, we categorized between original and transformed high-confidence patterns using an auxiliary model, which was saved at the first learning rate decay (150th epoch). In Section 3.2 Fig. 4, we grouped adversarial patterns based on their corresponding natural training loss, employing a loss threshold of 1.5. We have included this explanation in Appendix C of the rebuttal revision.

Q3: TRADES Result

A3: Our study investigated the prediction behaviour of the PGD-trained model, which does not encounter natural patterns during the training process. In PGD, we observed a similar memory tendency between natural and adversarial patterns within a single over-memorization sample. We further explored this observation in the TRADES-trained model, which encounters natural patterns during the training process. From Fig. 6 in the rebuttal revision, we can observe that TRADES demonstrates a consistent memory tendency with PGD in the over-memorization samples. This tendency manifests as, when DNNs over-memorize certain adversarial patterns, they tend to simultaneously exhibit high-confidence in predicting the corresponding natural patterns. We have included this result in Appendix D of the rebuttal revision.

Q4: CO Results

A4: Our approach employs data augmentation to prevent the model from over-memorizing training patterns, thereby alleviating overfitting. However, due to the inherent randomness of the original data augmentation, it cannot reliably and consistently prevent over-memorization. To overcome this issue, our method $DOM_{DA}$ applies iterative data augmentation, which is strategically designed to maximize the likelihood of hindering over-memorization. Nonetheless, the effectiveness of our proposed method is dependent on the quality of the original data augmentation technique, which could still be incapable of disrupting over-memorization even after the iterative operation. We have discussed this concern in the limitations of the original submission.

If you have any further questions or concerns, please do not hesitate to contact us. We are willing and available to provide additional information you may need.

[1] Berthelot, D., Roelofs, R., Sohn, K., Carlini, N., & Kurakin, A. (2021, October). AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptation. In International Conference on Learning Representations.

Dear Reviewer 2dX7:

We hope this message finds you well. We are sincerely thankful for your effort in reviewing our manuscript. As a gentle reminder, we have submitted a rebuttal and a revised manuscript to address your mentioned concerns. Given the constraints of the discussion period, your prompt response would enable us to provide further clarification and explanation.

Best regards,

Authors

Dear Reviewer 2dX7,

Thanks a lot for your valuable efforts in reviewing our manuscript. Just a kind reminder that the discussion stage is closing soon. If there are any unclear explanations or descriptions, we can clarify them further.

Best regards,

Authors

We sincerely appreciate your time and effort in reviewing our manuscript. After careful consideration, please find our responses to your comments below.

Q1: Transformer-based Architecture

A1: We have validated the effectiveness of our method within CNN-based architectures, demonstrating its ability to alleviate overfitting by preventing over-memorization. To further substantiate our perspective, we verify our method on the Transformer-based architecture. Constrained by computational resources, we trained a ViT-small model, initializing it with pre-trained weights from the Timm Python library. The training spanned 100 epochs, starting with an initial learning rate of 0.001 and divided by 10 at the 50th and 75th epochs. We set the batch-size to 64 and the loss threshold $\mathcal{T}$ to 0.1, maintaining other hyperparameters as the original settings. We report the natural training test error at both the best and last checkpoint on CIFAR 10.

The above table shows the effectiveness of our method on the Transformer-based architecture. By mitigating over-memorization, both $DOM_{RE}$ and $DOM_{DA}$ not only improve model performance at both the best and last checkpoints, but also contribute to alleviating overfitting. We have included this experimental result in Appendix E of the rebuttal revision.

Q2: Larger-scale Dataset

A2: We also verified the effectiveness of our method on the larger-scale dataset Tiny-ImageNet. We set the loss threshold $\mathcal{T}$ to 0.2, and other hyperparameters remain as the original settings. We report the natural training test error at both the best and last checkpoint using PreactResNet-18.

The above table illustrates the effectiveness of our method, $DOM_{RE}$ and $DOM_{DA}$, on the Tiny-ImageNet dataset. These results indicate that preventing over-memorization can improve model performance and reduce the generalization gap on large-scale datasets. We have included this experimental result in Appendix F of the rebuttal revision.

Q3: Marginal Improvement

A3: The impact of DOM is associated with the overfitting's degree in various training paradigms. For instance, the CO shows a significant decline in model generalization ability, and as a result, the DOM can provide a substantial improvement in this scenario. On the other hand, the NO shows a slight generalization gap between the best and last checkpoints, which leads to a relatively modest performance improvement by DOM. However, it's important to emphasize that, our study is dedicated to holistically understanding and alleviating different types of overfitting. As a general framework, DOM can consistently achieve notable improvements over the baseline with a subset of training samples or data augmentation, which successfully supports our perspective.

If you have any further questions or concerns, please do not hesitate to contact us. We are willing and available to provide additional information you may need.

Dear Reviewer 3ew1:

We hope this message finds you well. We are sincerely thankful for your effort in reviewing our manuscript. As a gentle reminder, we have submitted a rebuttal and a revised manuscript to address your mentioned concerns. Given the constraints of the discussion period, your prompt response would enable us to provide further clarification and explanation.

Best regards,

Authors

Dear Reviewer 3ew1,

Thanks a lot for your valuable efforts in reviewing our manuscript. Just a kind reminder that the discussion stage is closing soon. If there are any unclear explanations or descriptions, we can clarify them further.

Best regards,

Authors