IMPROVING ADVERSARIAL TRAINING WITH MARGIN- WEIGHTED PERTURBATION BUDGET

We thank the reviewer for the valuable feedback.

"The paper has limited novelty at the philosophy level. [1] and [2] pointed out that from a geometric perspective..."

The novelty of this work is in showing that better robustness may be obtained by perturbing by adaptively assigning perturbation radii to individual samples used for AT. To the best of our knowledge, no existing work has clearly demonstrated this observation. [1] and [2] assign larger weights to the losses of adversarial data close to the decision boundary. In contrast, we propose assigning larger perturbation budgets for crafting adversarial examples originating from natural data farther from the decision boundary. While these works and ours both utilize geometric information, the core philosophy of those ideas differs from ours. Also, [1] and [2], do not yield genuine adversarial robustness as they perform poorly on strong attacks [6, 7]. To the best of our knowledge, our work is the first reweighting approach to yield genuine robustness to stronger attacks like Autoattacks.

"Besides, [3] and [4] have already proposed instance-wise AT methods of using weighted perturbation budgets 4 years ago. At least, the authors should justify the necessity of using MWPB and explain why MWPB is better than the above-mentioned methods."

It should be noted that [3] and [4] did not exactly use a “weighted” perturbation budget, although these works proposed training with adversarial examples crafted using unequal perturbation budgets. Besides, none of these methods show that robustness may be improved by training with adaptive perturbation radii. We have highlighted the differences in Sec. 2.3.

"The authors should at least try different $\epsilon$ in the setting of ℓ∞-norm (e.g., 6/255, 10/255, 12/255) to see if MWPB can still work well."

We have revised the initial submission to include this. Kindly check Appendix A.1.

"The authors should discuss why this paper only considers ℓ∞-norm, instead of ℓ2-norm."

We followed the trend in adversarial robustness literature which prioritizes ℓ∞-norm. However, based on your suggestion, we have added experiments on ℓ2-norm Autoattack. Kindly check Appendix A.1 of the revised paper.

"Besides, the authors should test the robustness of the proposed method under adaptive attacks [5]. Specifically, adaptive attacks assume the attackers know all the details of MWPB, including the proposed reweighting function."

We could not implement the new adaptive attack the reviewer described within this time frame. However, we evaluated our defense against a strong adaptive boundary attack called FAB, and we have added the result in Appendix A.1. FAB attack is also resistant to gradient masking. Notwithstanding, our hunch is that MWPB would perform better under the adaptive attack you described, partly because vulnerable samples are usually in the minority. In addition, vulnerable samples usually contribute less to robust accuracy in traditional AT methods as may be seen in [7] and [8].

"Several typos and formatting inconsistencies. "

We have corrected the typos and revised the whole paper.

"Could you provide a theoretical justification for why it is beneficial to decrease the perturbation radii on vulnerable natural examples?"

The assumption here is that intrinsically natural vulnerable samples are inherently located in areas of high losses. Hence, relatively smaller perturbation radii will be sufficient to obtain a useful inner maximization loss. Since we are reweighting perturbation radii based on the individual vulnerability of natural samples, it makes sense that the radii of vulnerable samples are smaller $\epsilon$. Furthermore, the natural accuracy benefits from smaller perturbation radii assigned to vulnerable examples.

"If $\alpha$ has to be manually chosen for every method and dataset, I believe it will bring some additional cost. Then it is probably not correct to say ‘our proposed method reweights perturbation radii at no additional cost’ in Sec 2.3."

By “additional cost,” we mean that the training time for MWPB-AT, MWPB-TRADES, and MWPB-MART does not increase over AT, TRADES, and MART, respectively. Indeed, we manually selected $\alpha$ (See. Table 8, Appendix A.2). It is common to tune for additional parameters in ML literature, e.g., AWP, TRADES, MART, etc. We recommend the ideal values of $\alpha$ for obtaining the best performance and for easy reproduction of results.

References

6. Dong, C., Liu, L. and Shang, J., 2021. Data quality matters for adversarial training: An empirical study.

7. Fakorede, O., Nirala, A.K., Atsague, M. and Tian, J., 2023. Vulnerability-Aware Instance Reweighting For Adversarial Training. TMLR.

8. Xu, H., Liu, X., Li, Y., Jain, A. and Tang, J., 2021, July. To be robust or to be fair: Towards fairness in adversarial training. ICLR

*We thank the reviewer for the insightful comments, and we appreciate the reviewer for increasing our score. *

“After the revision, I am still not convinced by the authors that the novelty of this paper is sufficient. In my opinion, inspiration is more important than implementation. Technically, I can see the difference between this work and previous works (e.g., [1] [2] [3] [4]). However, I can hardly see a clear difference in terms of how this idea could inspire more researchers to come up with a follow-up idea. I hope the authors could further clarify this point.” To the best of our knowledge, no prior work has convincingly demonstrated that unequal perturbation radii can improve robustness. [1,2,3,4] do not achieve genuine robustness [6, 7]. Reweighting losses, as done in [1] and [2] is not very beneficial [6, 7].

More importantly, our idea is not tied to a specific implementation or algorithm. One of the arguments in our work is that unequal perturbation can benefit any AT method. This line of reasoning is beneficial to the community.

“I have read other rebuttals and I am curious about the statement made by the authors: 'Ideally, larger perturbation budgets lower the natural accuracy. However, this was alleviated by starting the training with a smaller perturbation budget ($\epsilon$=4/255) and step size of 1/255, until the 80th epoch.' My question is: are all the baseline methods following the same experiment setup, i.e., $\epsilon$=4/255 and step size of 1/255 until the 80th epoch? If not, I do not think the comparison is fair enough. I hope the authors could further clarify this point.”

Training with $\epsilon$ =4/255 and step size of 1/255 until the 80th epoch is part of the uniqueness of our method. This approach was inspired by the theoretical analysis in [9], which shows that AT under larger perturbation budgets hinders models from escaping sub-optimal initial regions. We use this approach to ease the introduction of larger perturbation radii in the later epochs.

We indeed tested this setup in the baselines that use uniform perturbation and found the setup does not benefit these baselines. Other baselines that change perturbation radii or change weights have their own experiment setup.

Reference [9]. Liu, C., Salzmann, M., Lin, T., Tomioka, R. and Süsstrunk, S., 2020. On the loss landscape of adversarial training: Identifying challenges and how to overcome them. Advances in Neural Information Processing Systems, 33, pp.21476-21487.

I thank the authors for their detailed responses. Most of my concerns have been addressed and I have raised my score from 3 to 5. I still have some further questions:

1. After the revision, I am still not convinced by the authors that the novelty of this paper is sufficient. In my opinion, inspiration is more important than implementation. Technically, I can see the difference between this work and previous works (e.g., [1] [2] [3] [4]). However, I can hardly see a clear difference in terms of how this idea could inspire more researchers to come up with a follow-up idea. I hope the authors could further clarify this point.

2. I have read other rebuttals and I am curious about the statement made by the authors: 'Ideally, larger perturbation budgets lower the natural accuracy. However, this was alleviated by starting the training with a smaller perturbation budget ($\epsilon$=4/255) and step size of 1/255, until the 80th epoch.' My question is: are all the baseline methods following the same experiment setup, i.e., $\epsilon$=4/255 and step size of 1/255 until the 80th epoch? If not, I do not think the comparison is fair enough. I hope the authors could further clarify this point.

We thank the reviewer for the valuable feedback. We have addressed the requested changes in the revision. For easy reference, we have used blue color to mark major changes in the text of the revised paper.

"Writing needs improvement"

We have gone through the whole manuscript all over again to improve the presentation and address the typos.

"Novelty is limited: the motivation is not novel, as the unequal treatment of samples has been extensively discussed in previous research.

The novelty of this work is in showing that better robustness may be obtained by perturbing by adaptively assigning perturbation radii to individual samples used for AT. To the best of our knowledge, no existing work has clearly demonstrated this observation. Furthermore, we agree with the reviewer that unequal treatment of samples has been demonstrated in previous research. However, it should be noted that the unequal treatment of samples alone is not an end goal, but why are individual samples unequally treated, and in what context? The context here is different from what is obtainable in existing works. For instance, most existing reweighting approaches focus mainly on reweighting losses by typically assigning larger weights to difficult samples . Here, we assign smaller perturbation radii to difficult samples and we also argue that existing AT methods such as TRADES and MART could benefit from assigning larger perturbation radii to inherently robust samples. Existing instance-wise reweighting methods fail to yield genuine improved robustness as they perform poorly against stronger attacks such as auto-attacks and sometimes show signs of gradient obfuscation [1, 2]. To the best of our knowledge, our work is the first instance-wise reweighting method to yield genuine improvement against auto-attacks, and it does so cheaply.

"Empirically, in the later stages of adversarial training, nearly all samples have a d_margin greater than 0. This implies that the perturbation budget for all adversarial samples exceeds 8/255. Given this scenario, achieving higher natural accuracy is quite perplexing. Therefore, it is hoped that the authors can offer detailed insights into the evolution of the perturbation budget during training."

Ideally, larger perturbation budgets lower the natural accuracy. However, this was alleviated by starting the training with a smaller perturbation budget ($\epsilon$ =4/255) and step size of 1/255, until the 80th epoch. This approach was inspired by the theoretical analysis in [3], which shows that AT under larger perturbation budgets hinders models from escaping sub-optimal initial regions. Our experimental results show how our training approach mitigates low natural accuracy in the proposed MWPB. For example, the experiments below were obtained for WRN-34-10 on CIFAR-10.

Note: We have added the perturbation radii distribution for all training samples of CIFAR-10 on ResNet-18 in Appendix A.3.

"Insufficient experiment: it is suggested that the authors conduct experimental comparisons with a broader range of baselines, such as AWP, rather than solely focusing on standard AT methods."

The main goal of our work is to show that the proposed approach can improve existing AT methods. However, based on your recommendation, we have included new baselines in Table 6. We added the following baselines: AWP, GAIRAT, and ST-AT. We have also added more experiments under various settings in Table 7.

References

1. Dong, C., Liu, L. and Shang, J., 2021. Data quality matters for adversarial training: An empirical study. arXiv preprint arXiv:2102.07437.

2. Fakorede, O., Nirala, A.K., Atsague, M. and Tian, J., 2023. Vulnerability-Aware Instance Reweighting For Adversarial Training. TMLR.

3. Liu, C., Salzmann, M., Lin, T., Tomioka, R. and Süsstrunk, S., 2020. On the loss landscape of adversarial training: Identifying challenges and how to overcome them. Advances in Neural Information Processing Systems, 33, pp.21476-21487.

We thank the reviewer for the valuable feedback. We have addressed the requested changes in the revision. For easy reference, we have used blue color to mark major changes in the text of the revised paper.

The construction of the reweighting function in Eq (3) seems heuristic. There should be ablation studies on the effects of different reweighing alternatives, and justify why the construction in Eq (3) is superior.

We have added ablation studies in the revised version. Kindly check Table 8 in Appendix A.2.

The considered baselines (AT, MART, TRADES) are not up-to-date, where the top-rank methods listed on RobustBench [1] can already achieve >70% robust accuracy under AutoAttack on CIFAR-10 ( ℓ∞=8/255). The authors are encouraged to validate the compatibility of MWPB with more advanced methods listed on RobustBench.

One of the goals of our work is to show that this approach can improve prominent existing works such as AT, TRADES and MART. In addition, we have compared our results with more baselines in Table 5. It is important to note that the robustness reported for the top-rank methods in Robustbench [1] are significantly dependent on a combination of the network architecture (WRN-74-10), the amount of training data (50 million), the activation function(swish), batch size, and the number of training epochs ( typically > 1000) . Due to computational resources constraints, we could not evaluate on WRN-74-10 and train on 50 million data. Nevertheless, we combined MWPB-TRADES with [2]. Due to constraints in computational resources, we can only execute 400 epochs instead of the recommended 2,400 epochs, even with 20 million data generated by EDM. Also, the evaluation was done using WRN-28-10. Our experimental results are as follows:

It should be noted the batch size used in [2] is 2048, whereas we used 512. It is reported in [2] that the batch size plays a significant role in the robust accuracy. In fact, the best accuracy reported in [2] are trained on a batch size of 2048. To achieve the best robustness of over 70%, [2] trained WRN-74-10 for 2400 epochs.

The values of hyperparameters in Section 5.1 seem to be deliberately selected (e.g., $\alpha$=0.58,0.42,0.55). There should be ablation studies on the effects of different hyperparameters on the performance of MWPB (i.e., is MWPB sensitive to different values of hyperparameters?). Kindly check Appendix A.2 to see the sensitivity of the hyperparameter $\alpha$.

As stated in the Weaknesses sections, the baselines (AT, MART, TRADES) are not up-to-date (proposed no later than 2019). So I would like to know if MWPB can be compatible with more advanced methods listed on RobustBench. Besides, there should be ablation studies on the effects of different reweighing alternatives and sensitivity w.r.t. hyperparameters. We have added ablation studies on Appendix A.2. We have also added newer baselines in Table 5.

References

[1] Robustbench

[2]. Zekai Wang, Tianyu Pang, Chao Du, Min Lin, Weiwei Liu, and Shuicheng Yan. Better diffusion models further improve adversarial training. ICML, 2023.

*We thank the reviewer for the timely response. *

"I find that the reported comparison with [2] seems misleading. The authors mentioned that [2] uses 20M EDM data, 2048 bs, and 2400 epochs, however, under theses setups, the result of [2] on WRN-28-10 is 92.44 % NAT and 67.31 % AA as listed on RobustBench".

Truely, [2] obtained 67.31% AA under the settings you stated. However, due to the computational cost involved, we did not train our model using these settings. We trained our model using the batch size 512 and for 400 epochs. The reported comparison with [2] is from Table 5 in [2] using 20M EDM data, 2048 bs, and 400 epochs which reported 63.84% on AA.

"Besides, under the setups of 1M EDM data, 512 bs, and 400 epochs, [2, Table 2] reports 91.12 % NAT and 63.35 % AA on WRN-28-10. It seems that combining MWPB-TRADES with [2] does not lead to significant improvements (90.31 % NAT and 63.78 % AA)".

[2] reported 91.47% NAT 63.84% AA under 20M EDM data, 2048 bs, and 400 epochs, while combining MWPB-TRADES with [2] obtains 90.31 % NAT and 63.78 % AA under 20M EDM data, 512 bs, and 400 epochs. By the information in [2], batch size plays an important role in robustness accuracy, which makes us believe that MWPB-TRADES could improve over 63.84% when a batch size of 2048 is used. **Kindly note that we did not train on the setup of 1M EDM data **.

We thank the reviewer for the valuable feedback. We have addressed the requested changes in the revision. For easy reference, we have used blue color to mark major changes in the text of the revised paper.

"How will the proposed method affect the natural accuracy in theory? If the perturbation budgets on robust samples are too large, training on such adversarial examples may hurt the discrimination power and generalization power of the DNN. For example, the natural accuracies are all reduced in Table 3. The authors are suggested also to discuss the theoretical effect of the proposed method on the natural accuracy of the DNN."

Usually, larger perturbation budgets result in lower natural accuracy. We attempt to mitigate this by starting the training with a smaller perturbation budget ($\epsilon$ =4/255) until the 80th epoch. Our experimental results in Tables 1, 2 and 4 show that our approach achieves superior performance and generalization over the baselines. However, in Table 3, there is a marginal decrease in the natural accuracy. We attribute this to the peculiarity of the SVHN datasets. SVHN is not very favorable to AT.

The hyperparameters in experiments are determined heuristically for different methods on different models. This setting hurts the reliability of the obtained results. I wonder whether the proposed method is stable to different hyperparameters.

The proposed MWPB uses one hyperparameter, which is $\alpha$. Since we are trying to assign larger weight perturbation radii to intrinsically robust examples, it is important that $\alpha$ > 0.0. Setting $\alpha$ between 0.0 and 1.0 yields stable results. However, we recommend the values stated in the paper for reproducing the results reported in the paper. Kindly see Table 8 in Appendix A.2.

Is there a threshold for the radius in Equation (3)? If not, when an input is too far away from the decision boundary (misclassified or correctly classified), the radius will be extremely small or large, which may affect the stability of training. Besides, the magnitude of outputs $f_{\theta}(x)$ may be different in different models, so the weight $\alpha$ needs to be carefully adjusted for each model.

There is no threshold for the radius in Equation (3). The determination of the appropriate radii for individual samples depend significantly on the logit margin computed from the model’s outputs on natural samples. Based on our observations, the perturbation radii used by MWPB-AT (uses the largest perturbation) for training ResNet-18 on CIFAR-10 ranges between 0.018 and 0.055. The mean perturbation radius for each implementation of MWPB is provided in Table 6. We have also added plots showing the distribution of the perturbation radii in Appendix A.3.

We do not expect the magnitude of the logit margins to be significantly different for different models. Based on our experiments reported in Tables 1 and 2 for ResNet-18 and Wideresnet-34-10 respectively, the best performances were recorded on the same values of $\alpha$ for MWPB-AT, MWPB-TRADES and MWPB-MART respectively.

In experiments, the proposed method is only used after 80 epochs, at which point the model may have learned a certain optimization direction. What about the performance when using the MWPB from the beginning of the training?

Ideally, larger perturbation budgets lower the natural accuracy. However, this was alleviated by starting the training with a smaller perturbation budget ($\epsilon$ =4/255) and step size of 1/255, until the 80th epoch. This approach was inspired by the theoretical analysis in [1], which shows that AT under larger perturbation budgets hinders models from escaping sub-optimal initial regions. Our experimental results show how our training approach mitigates low natural accuracy in the proposed MWPB. For example, the experiments below were obtained for WRN-34-10 on CIFAR-10.