Improving Robustness and Accuracy with Retrospective Online Adversarial Distillation

We sincerely appreciate the reviewer for the thoughtful feedback.

The novelty of this paper is insufficient. It is highly overlapped with previous work, such as the use of soft labels, which was explored in the previous work [1]. Secondly, the use of online training methods was explored in the work of [2], and the use of natural models as teacher was explored in [3].

Thanks for your concerns regarding the novelty of our paper. However, ROAD distinguishes itself from prior works in two main aspects.

Firstly, While there may be similarities with RSLAD in using soft labels, ROAD does not require an adversarially trained model to improve robustness, thereby offering a computational cost advantage. Thus, ROAD can be a promising technique to enhance robustness if we have no pre-trained teacher models. Also, considering that adversarial training typically takes several times longer than conventional training, it can be considered to possess sufficient novelty.

Secondly, our approach diverges from methods that utilize a natural model by implementing an asymmetric knowledge transfer. This is based on the premise that usage of acquired knowledge to the natural and robust models are different as we mentioned in Section 3.2. Our experiments and ablation studies demonstrate that ROAD significantly outperforms other methods in terms of natural accuracy through this asymmetric knowledge transfer.

Lack of ablation experiments, such as the impact of natural models, and the selection of robust models (different checkpoints)

We appreciate your constructive feedback for experiment part. We have conducted several additional experiments including comprehensive ablation studies (effects of interpolation ratio scheduling, asymmetric knowledge transfer) and stability of ROAD. We have updated our results in our paper.

Table 1 shows the effect of interpolation ratio scheduling. We have considered two more scheduling strategies (fixed, linear increasing)

Table 1. Effect of interpolation ratio $\lambda$

Table 2 shows the effect of transferring asymmetric knowledge. We have prepared the symmetric version of ROAD: the natural model achieves knowledge via not soft labels but KL-divergence, typically seen in conventional online distillation. We have reused $\gamma$ for simplicity and symmetric knowledge transfer.

Table 2. Effect of transferring robust knowledge to natural model

Table 3 shows the stability of ROAD. We have trained TRADES and ROAD 5 times with different random seeds. As we can observe in Table 3, ROAD shows high stability while outperforms TRADES in both natural accuracy and robustness.

Table 3. Variance across multiple reruns

Thanks for your insightful comments and detailed review! We prepare pointwise answers for your concerns and questions.

It remains unclear how the resolution for penalizing overconfident predictions is incorporated via the utilization of soft labels.

We have provided theoretical proofs in the respective appendix and supported our claims empirically through reliability diagrams and ECE results. However, we acknowledge that this may not suffice for your assessment. Therefore, we have also included the calibration performance of ROAD to further demonstrate the effects of utilizing last epoch predictions in Appendix D.3.

Concerns of novelty.

Thanks for addressing your concerns about the novelty of our paper. Our method achieves novelty in two key aspects. Firstly, unlike other approaches that require the addition of extra models to enhance robustness, such as those using EMA or prior model's timestamps, our method stands out by not necessitating the use of additional models. We only store the predictions of last epoch. This feature offers a clear advantage in terms of GPU memory usage efficiency and it is more applicable in devices with limited resources. Secondly, our approach diverges from conventional methods that employ a natural model and learns collaboratively by implementing asymmetric knowledge transfer. This is based on the premise that usage of acquired knowledge to the natural and robust models are different as we mentioned in Section 3.2. We conducted an ablation experiments of the effects and updated in our revised paper.

The ROAD method still maintains two separate networks. It necessitates the retention of an additional copy of the network (from the last epoch) in GPU memory and, in aggregate, entails three supplementary forward propagation steps.

Sorry about the ambiguity of our paper about the implementation details of ROAD. ROAD maintains only one separate network, the natural model. We store the past predictions of the last epoch instead of copying the network which can be shown in Figure 1. This can be also seen in our publicly provided code. Thus, as we can observe in the experiment section, ROAD does not require a substantial amount of memory. However, we can also consider using a separate model if the predictions are massive to store, such as tokens for text classification.

The paper appears to lack references to several relevant baselines, encompassing both new and established methods

Thanks for your constructive feedback. We agree with the importance and relevance of ACT and MAT and updated our related works in our revised paper.

Further inconsistencies arise from comparisons with the RSLAD paper, wherein the AA and other attack-related metrics appear to surpass the corresponding figures reported in the ROAD paper. Similar discrepancies are apparent in the case of SEAT results.

We would like to highlight that there are variations in the implementation details between the original paper and ours, such as the total number of epochs (reduced from 300 to 200), changes in learning rate scheduling (from divided by 10 at the 215th, 260th, and 285th epochs, to the 100th, 150th, and 175th epochs), and adjustments in weight decay (from 2e-4 to 5e-4) for a fair comparison. Also, please consider that the RSLAD paper reports similar results to our paper in Appendix Table 9 when RSLAD is trained through self-distillation (RN-18 --> RN-18). Furthermore, it should be noted that SEAT, in their official code, opted for a maximum perturbation bound of 0.031, as opposed to our choice of 8/255, and a step size of 0.007 instead of 2/255. These differences may significantly impact the performance of robustness.

A comprehensive evaluation of the ROAD method would ideally encompass additional benchmarks, including but not limited to Weight Averaging and AWP

Thanks for your recommendation ! Due to the lack of time and computational resources, we will conduct the experiments you suggested at a later date and update the paper accordingly.

Please review the baseline models and consider adding more baselines (check the previous section for reference).

Thanks for your nice suggestion. We add KD+SWA[1] and IAD[2] to our baseline which is similar with ROAD in the fact that using teacher models.

In Section 3.1.2, the RSLAD also utilizes soft labels. Can we access the reliability diagram for it and obtain further information about the differences between these two methods?

We have added the reliability diagrams of ARD, RSLAD, and ROAD in the appendix of our updated paper. The Expected Calibration Error (ECE) of ROAD is 3.77, demonstrating superior calibration performance with the lowest ECE. Furthermore, we observe that ROAD tends to generate under-confident predictions, in contrast to the over-confident predictions commonly associated with ARD and RSLAD.

Thanks for your positive feedback and your suggestions about the experiments. We provide pointwise responses below.

The paper involves several interesting design that collaboratively contribute to strong performances, thus a more detailed ablation study (more than the one the authors offered) is probably necessary, for examples why is the asymmetry of how robust model and natural model influence each other is necessary? The current ablation study only touches briefly on the removal of these losses. However, since the authors emphasized on this asymmetry, it will be quite essential to discuss what if we use the same way of how these models influences each other. For example, what if we use soft labels from natural model also, or what if when we train the natural models, we use KL regularization also.

We acknowledge that our ablation study may have been insufficient and, in response, have included two additional studies in Figure 4 in our revised version. Following your suggestion, we conducted a performance comparison experiment where the natural model in ROAD receives knowledge via KL-divergence similar to standard online distillation. Additionally, we reuse $\gamma$ as a hyper-parameter for the KL-divergence term to exchange symmetric knowledge between robust model and natural model. As shown in Table 1, regardless of the $\gamma$ value, our findings confirm that the original ROAD's asymmetric knowledge transfer method is superior to the symmetric knowledge exchange in natural accuracy.

Table 1. Effect of transferring robust knowledge to natural model

The "LS" method does not seem to ever get spelled out, thus very hard to evaluate the relevant discussions. The paper cited is not immediately about the topics discussed in this paper. As a result, I do not see how "Effects on utilizing last epoch predictions" in ablation study is relevant.

“Effects on utilizing last epoch predictions” is designed to show the performance of self-adversarial distillation using last epoch predictions. We select label smoothing(LS) and adversarial knowledge distillation (AKD) as baselines because these methods also employ soft labels to improve robustness, similar to our method. However, we also agree that the corresponding ablation study does not need immediate attention, so we moved it to the appendix.

Figure 4 seems to suggest that self-distillation does not matter that much, a major performance boost comes from the natural model part, which seems quite counter intuitive.

While it may seem counterintuitive, it is actually because replacing the soft labels of the natural model with one-hot labels and allowing the natural model to unilaterally transfer knowledge to the robust model can actually lead to a detrimental decrease in robustness. However, it is important to note that utilizing self-distillation, as opposed to not using it at all, results in a performance increase of over 1% in the perspective of PGD-20. We have updated Figure 4 to make this effect more readily apparent in our revised version.

It will be helpful to have a natural model for references.

Appreciate your suggestion! We add the performance of the natural model to Table 1 and Table 2 in our revised version.

We appreciate about your detailed review and meaningful feedback. We provide pointwise responses below.

Considering Weakness W1, does the proof presented in Section 3.1.1 extend to ensuring that adversarial samples, which induce a significantly larger gradient norm with respect to the model weights $\theta$, can also be effectively penalized?

We have thoroughly read the suggested paper and appreciate its relevance to the broader field. However, we believe that directly linking our work to the findings of this paper may not be the most appropriate approach. The suggested paper addresses the issue of robust overfitting which is a phenomenon that test accuracy on robustness drops significantly while train accuracy on robustness increases . In our understanding, the concepts of overconfidence and robust overfitting, while related, however are distinct. For instance, an overfitted model might exhibit low confidence when tested on unseen data from a different domain which is hard to say that the model is overconfident.

Meanwhile, we have theoretically and empirically validated our assumption as detailed in our paper. To further support our argument, we have also included experiment results on the calibration performance of ROAD in the appendix D.3 in our updated paper.

Several statements presented in Section 3.2 are deemed to be either inaccurate or not adequately substantiated. For example, the referenced studies do not specifically support the claim that 'distilling knowledge from the static natural teacher may impair robustness.' It is strongly advised that the structure and argumentation of this section be thoroughly revised for clarity and accuracy.

Thanks for your detailed feedback! We have revised our sentences in Section 3.2. The sentences are

``However, experimental results presented in the literature [1,2] demonstrate that distilling knowledge from the static natural model can reduce robustness, indicating that it is not the proper approach."

Also, we want to clarify that our statements in Section 3.2 are sufficiently substantiated. The referenced studies support the claim explicitly or implicitly that static natural teacher can reduce robustness. For example, In section 4.4 of [1], they provide experimental results that training with natural soft labels (NSLs) crafted by the natural model can significantly harm robustness of the student model and claim it in the last sentence in the paragraph. Additionally, In table 1 of [2], the table shows that the student model with a natural model teacher has lower accuracy under AutoAttack when compared with no teacher.

We hope that the revised sentences address your concerns adequately.

The logical progression within Section 3 is obscure, and the transitions between subsections lack fluidity, challenging the reader's ability to discern the author's objectives. Furthermore, the exposition accompanying Equation 3 falls short of providing sufficient elucidation or the underlying intuition for introducing the Robustness Enhancement term in the construction of the final objective function.

We strongly agree that explanations about the method can be confusing to the readers. To enhance clarity and facilitate easier reading, we have added more details to the introductory part of Section 3, clearly delineating the roles of each subsection. In addition, we understand your concerns about Robustness Enhancement term and add proper elucidations. The sentences are

``This regularization term causes loss in natural accuracy as a trade-off for improved robustness. Nevertheless, this loss of accuracy can be recovered by the subsequent term."

Because the paper appears to lack a comprehensive exploration of the tuning strategy of the hyper-parameter introduced in Equation 1, could you elucidate on the potential effects of employing a constant value for $\lambda$, or linearly increase the value of $\lambda$ instead of using the sine increasing schedule?

That is a nice feedback. Following your suggestion, we have conducted experiments that verify our selection of sine scheduling and updated in our paper. As shown in Table 1, utilizing last epoch predictions show an overall improvement in robustness compared to PGD-AT which trained with one-hot labels. In addition, we observe that the fixed policy results in over 1% lower natural accuracy compared to other strategies.

Table 1. Effect of scheduling interpolation ratio $\lambda$

Ref

[1] Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Better, ICCV 2021

[2] On the benefits of knowledge distillation for adversarial robustness, Arxiv 2022