Maximum Margin Based Activation Clipping for Post-Training Overfitting Mitigation in DNN Classifiers

1. I understand that the big topics is the overfitting problem. However, it is wired to choose the two topics: backdoor attacks and imbalanced dataset for evaluating this problem. For example, L-2 regularization, or dropout are proposed to overcome the overfitting problem, but they will not test the regularization effect on these two topics.
2. The authors assert that their proposed method prevents overfitting. However, the experimental results appear to indicate that, upon implementing activation clipping, the model's clean accuracy actually deteriorates. Generally, one would anticipate that addressing overfitting might enhance model performance, but this proposed method doesn't seem to achieve that.

1. The proposed method addresses a narrow subject, applicable only to models with the ReLU activation function. While ReLU is widely used, it is limiting to have an approach tailored exclusively for it. It would be beneficial to show the performance of the proposed technique with other activation functions as well.

2. The assumption made in the paper is that backdoor attacks will yield larger activation values compared to clean inputs. Are there any empirical evidences supporting this assumption? This paper evaluates several attacks in the experiments. However, there are many other advanced attacks, such as filter attack, reflection attack [1], DFST [2], dynamic attack [3], composite attack [4], and SIG [5]. Do all of these exhibit large activation values compared to clean inputs? It would be more convincing to have a study to show the generalization of the proposed method.

3. The paper only evaluates known attacks. As the proposed approach relies on certain assumptions, an adaptive attack could intentionally violate these assumptions. For instance, an attacker might manipulate the activation values of a backdoor-injected input to closely resemble the activation value ranges of clean inputs. Would the proposed technique still be effective in such a scenario?"

4. In the experiment on mitigating non-malicious overfitting, which data samples are used for evaluation? It appears that they are not from the training set. This raises a concern about the fairness of comparison with other baselines. The paper argues that it is unfair to evaluate the proposed approach since it does not have access to the training set. However, given that it is a post-training technique, if it utilizes balanced non-training data, this implies an advantage over the baselines. All of those baselines only have access to imbalanced data.

5. Only two baselines are compared in Section 4.2. I am not an expert on the long-tail problem, but by simply searching online, I found a large number of works aiming to address it. Are the two baselines compared in the paper representative? Why were other related techniques not compared?

6. The evaluation conducted in this study is limited. Only one model was used in the experiment, which hinders the assessment of the technique's generalizability. The results reported in Table 1 may be overfitting on this particular model. There are advanced model architectures that have been shown to be more powerful than ResNet, such as vision transformers. Does the proposed technique work on these new architectures? Furthermore, it would be better to conduct experiments on large image datasets, such as ImageNet.

[1] Liu, Y., Ma, X., Bailey, J., and Lu, F. Reflection backdoor: A natural backdoor attack on deep neural networks. ECCV 2020.
[2] Cheng, S., Liu, Y., Ma, S., and Zhang, X. Deep feature space trojan attack of neural networks by controlled detoxification. AAAI 2021.
[3] Salem, A., Wen, R., Backes, M., Ma, S., and Zhang, Y. Dynamic backdoor attacks against machine learning models. EuroS&P 2022.
[4] Lin, J., Xu, L., Liu, Y., and Zhang, X. Composite backdoor attack for deep neural network by mixing existing benign features. CCS 2020.
[5] Barni, M., Kallas, K., & Tondi, B. (2019, September). A new backdoor attack in cnns by training set corruption without label poisoning. ICIP 2019.

There major issues with this submission can be categorized into three streams:

1. Novelty and Applicability: This paper is a simple extension of an earlier work by Wang et al. [1], and it has limited novelty. Most of the notions used in this paper already appear in the prior work, and this paper reorganizes their objective function and introduces a different regularizer than the one used in [1]. One can also see from the results in Table 1 and also Table 3 that introducing this new regularizer has somehow a minimal effect on the results on average (see Table 3 for how even the smallest values for $\lambda$ could yield similar results to the case where this term is absent from the objective.) Thus, it is unclear whether the changes introduced in this paper are significant or not. More importantly, the objectives introduced for MMAC and MMOM assume that the model deployer knows beforehand whether the model is backdoored or suffers from long-tailed issues. Even though detecting whether a model suffers from issues related to imbalanced datasets is easy, but we don't know whether the model has been backdoored or not. As such, one might need to also apply MMAC on top of clean models, which could hurt the benign performance.

2. Experimental Settings and Results: The other significant issue with the current submission is its lack of comprehensive experiments. Many newer attacks, such as SIG [2], ISSBA [3], Refool [4], DFST [5], and WB [6], are not considered for benchmarking. Moreover, there are better baselines, such as those in [7-8], for long-tailed recognition. In particular, contrary to the paper claims that "To the best of our knowledge, ours is the first work to address class imbalances post-training," I firmly believe that the seminal work of Kang et al. 2020 is the first work around this idea and as such, it should be both cited and used as a baseline. This is since Kang et al. 2020 just fine-tune the classification layer of the classifiers and as such, do not require re-training on the entire imbalanced data. Finally, most works in backdoor attacks and specially in long-tailed recognition also report their results over the ImageNet and ImageNet-LT datasets which this paper also needs to include these results.

3. Writing, Structure, and Presentation: Last but not least, this paper requires major changes in its narrative and formatting. First, instead of giving a standard overview of the work by highlighting the contributions in the introduction, the paper shocks the reader and goes into the related work section before pointing out its contributions. In fact, the majority of the last two paragraphs in the related work section belong to the introduction. Second, the paper heavily relies on the reader having read the work of Wang et al. [1], and it uses the information in that paper as presumed knowledge. For instance, instead of giving the intuitions behind Eq. (2) and discussing how it relates to regulation of decision boundaries or over-fitting prevention, the paper just presents the formulation without adequately explaining it. The same issue can be observed around the introduction of $J\_{c}$ in the second sentence of Section 3.1 where the readers might wonder how this is related to the previously discussed notions, and how we ended up with this definition. Finally, the paper jumps around to unrelated topics such as Federated Learning in the introduction, Adversarial Training in the Abstract and the Conclusion, etc. and makes the readers confused. I believe that the discussions around these two notions are not helping with building the correct narrative for the contributions of the paper, and removing them would result in less confusion.

[1] Wang et al. "MM-BD: Post-Training Detection of Backdoor Attacks with Arbitrary Backdoor Pattern Types Using a Maximum Margin Statistic."IEEE Symposium on Security and Privacy (SP), 2024.

[2] Barni et al. "A new backdoor attack in CNNs by training set corruption without label poisoning." ICIP, 2019.

[3] Li et al. "Invisible backdoor attack with sample-specific triggers." ICCV, 2021.

[4] Liu et al. "Reflection backdoor: A natural backdoor attack on deep neural networks." ECCV, 2020.

[5] Cheng et al. "Deep feature space trojan attack of neural networks by controlled detoxification." AAAI, 2021.

[6] Doan et al. "Backdoor attack with imperceptible input and latent modification." NeurIPS, 2021.

[7] Kang et al. "Decoupling representation and classifier for long-tailed recognition." ICLR, 2020.

[8] Yang et al. "Rethinking the value of labels for improving class-imbalanced learning." NeurIPS, 2020.

[9] Kirichenko et al. "Last layer re-training is sufficient for robustness to spurious correlations." ICLR, 2023.

1. The work builds on the MM-BM methodology by Wang et al., but the backdoor mitigation problem seems to be a not-so-important contribution of the original paper; instead, its methodology and experiments are more focused on the backdoor trigger detection problem. As such, the mitigation experiments were only performed for the relatively small CIFAR-10 dataset. This work also provides results for the non-malicious overfitting scenario on CIFAR-100, but the experiments seem insufficient to judge the overall merit of the idea. Experiments on TinyImageNet and/or GTSRB are lacking.

2. The problem of non-malicious overfitting due to class imbalance and overtraining, addressed for the first time post-training in this work, seems somewhat tangential to adversarial and trojan backdoor attacks. As such, it’s unsurprising that existing mitigation approaches don’t design techniques to tackle this issue. More well-grounded arguments would be good to motivate and connect the 2 issues further.

3. In Table 1 of the MM-BD paper, the authors outline the backdoor embedding types (additive, patch replacement, etc.) and the number of source classes (single or multiple) during the backdoor attack for evaluating detection and mitigation performance. No such details are mentioned in the experiments section in 4.1.

4. The summary of the main contributions is delayed until the end of the Related Work section instead of the Introduction - not a major issue, but it makes the overall flow difficult to follow.

5. The MMOM motivation and methodology in Section 3.2 seems a bit too abrupt and brief for proper understanding.

6. The paper assumes substantial domain knowledge of the reader in trojan backdoor attacks and mitigation and familiarity with the MM-BD paper. Many of the terminologies aren’t as well-explained as existing works do, e.g., Neural Cleanse (Wang et al., 2019).

7. [Minor] Table 1 is very hard to read - quite difficult to compare the performance of the proposed methods with existing work as none of the numbers are highlighted. Also, an up or down arrow beside every metric, like ASR or ACC, would have been helpful to understand if higher or lower is better. Mixup paper citation missing in Section 4.2.