Adversarial Training on Purification (AToP): Advancing Both Robustness and Generalization

We greatly appreciate the comments of the Reviewer onZy. Below are our responses to the questions raised.

A1: In the related work section, we conduct a comprehensive comparison to highlight the distinctions between our method and other existing methods. Considering the limitation of the existing pre-trained generator-based purifier model, where all parameters are fixed and cannot be further improved for known attacks. Regarding this challenge, We propose a novel defense pipeline as shown in following equations.

Pre-training:

$L_{\theta_{g}} =L_{df}(x, \theta_{g})$

$=\mathbb{E}[D(x)]-\mathbb{E}[D(g(t(x),\theta_g))]+\mathbb{E}[{\|x-g(t(x), \theta_g)\|}_{\ell_1}],$

Fine-tuning:

$L_{\theta_{g}} =L_{df}(x', \theta_{g})+\lambda \cdot L_{c l s}(x', y, \theta_{g}, \theta_{f})$

$=\mathbb{E}[D(x')]-\mathbb{E}[D(g(t(x'),\theta_g))]+\mathbb{E}[{\|x'-g(t(x'), \theta_g)\|}_{\ell_1}]$

$+\lambda \cdot \max_{\delta} CE(y, f(g(t(x'),\theta_g))), \quad where \ x' = x+\delta.$

A2: One of the advantages of AP is that once the generator model is fully trained, it will be a plug-and-play module that can be applied to other model architectures.

A3: In the experiments comparing with the state-of-the-art methods, all classifiers used are from the model zoo in RobustBench, and we select two of the most common model architectures for the experiments. To address your question, we recheck all available classifiers. However, we do not find a vanilla transformer-based classifier. There is only one transformer-based classifier with adversarial training (XCiT [1]). Therefore, we conduct experiments on XCiT, which is equivalent to AToP+AT. Following the settings of Table 2 and Table 4, here are the results:

[1] A Light Recipe to Train Robust Vision Transformers. 2022.

A4: In Section 4.4, we conduct experiments using BPDA based on the guidance of Athalye et al. In the ablation study, we use whether to apply AToP as the only variable. This means that in a set of comparative experiments, only the training parameters of the purifier model differ, while other details remain consistent, including the parts that cause obfuscated gradients. It is reasonable that we use the paradigm provided by Yang et al. to verify whether our method can improve the robustness of the pre-trained purifier model. More importantly, this part is only discussed as the ablation study, and we put the robustness comparison experiment with other SOTA methods in Section 4.2.

We greatly appreciate the comments of the Reviewer n69U. Below are our responses to the questions raised.

A1.1: If there are no random transformations, the robustness will decrease. $RT_2$ adds more randomness based on $RT_1$, and its robust accuracy has been improved. At the same time, Cohen and Wu et al. theoretically provide robustness guarantees about randomized smoothing, which is described in Section 3.1.

A1.2: Random operations contribute to robustness but not fully due to gradient vanishing. As mentioned in A1.1, there are some theories of robustness guarantees about random operations. Indeed, gradient vanishing does provide 'illusory' robustness for some attacks. Regarding this issue, we also validate our method with AutoAttack and BPDA.

A2: We conduct AutoAttack verification under two different settings: In Table 2,3,4,5, we attack the entire (purifier+classifier) model, and in Table 7 and 8, we attack the classifier model. The result in Table 7 and 8 is reasonable, but we also realize that the current tables can easily lead to misunderstandings. Therefore, we plan to redesign the Table 7 and 8 according to your comments, replacing AutoAttack with CW and PDG-1000.

Here is the explanation of the results: Considering the robustness misestimation caused by obfuscated gradients of the purifier model, we use BPDA (Athalye et al., 2018a) and follow the setting by Yang et al. (2019), approximating the gradient of the purifier model as 1 during the backward pass, which is equivalent to attacking the classifier model. To maintain the same settings, AutoAttack only attacks the classifier. The advantage of AutoAttack lies in using EOT and adaptive mode with white-box and black-box attacks to attack more complex architectures, such as black-box and random factors in the architectures. If there are some random factors in the classifier, although the classifier model can effectively defend against FGSM and PGD, it can still be vulnerable to AutoAttack. However, in classic classifiers (such as Resnet-18), these factors do not exist, and simple white-box attacks cause more damage. Under standard cases, the entire model should be input into AutoAttack directly, and the experimental results are shown in Table 2,3,4,5. We will redesign Table 7 and 8 to avoid confusion.

A3: Thanks for the comments. The following are the additional experiments on Table 7 and 8:

Thanks for your kind response.

I still have some concerns. I agree that adding Gaussian noise is helpful to robustness, which has been identified by Li et al. with theoretical proof. However, why does the random masking scheme lead to significant robustness improvements? If the improvements don't come from obfuscated gradients, could the authors give some explanations for it?

We greatly appreciate the comments of the Reviewer jVyX. Below are our responses to the questions raised. Due to exceeding the character limit for a single reply, we divide the response into two parts (1 / 2).

The first part is about weaknesses:

A1.1: We also realize this might be confusing, so we have revised Eq.(8) in A1.2. Regarding the second point, we agree with your perspective. We have only used the GAN-based model as an illustrative example. We will add a general form: $L_{\theta_{g}} = L_{org}\left(x, \theta_{g}\right)+\lambda \cdot L_{c l s}\left(x, y, \theta_{g}, \theta_{f}\right)$. For instance, the original loss function is denoted as $L_{df}$ in the GAN-based model (Ughini et al., 2022), and as $L_{mae}$ in the MAE-based model (Wu et al., 2023).

A1.2: Thanks for the comments. Eq.(8) should be divided into two stages, including pre-training and fine-tuning:

Pre-training:

$L_{\theta_{g}} =L_{df}(x, \theta_{g})$

$=\mathbb{E}[D(x)]-\mathbb{E}[D(g(t(x),\theta_g))]+\mathbb{E}[{\|x-g(t(x), \theta_g)\|}_{\ell_1}],$

Fine-tuning:

$L_{\theta_{g}} =L_{df}(x', \theta_{g})+\lambda \cdot L_{c l s}(x', y, \theta_{g}, \theta_{f})$

$=\mathbb{E}[D(x')]-\mathbb{E}[D(g(t(x'),\theta_g))]+\mathbb{E}[{\|x'-g(t(x'), \theta_g)\|}_{\ell_1}]$

$+\lambda \cdot \max_{\delta} CE(y, f(g(t(x'),\theta_g))), \quad where \ x' = x+\delta.$

A1.3: $g_{\theta_g}$ represents the computed gradient (i.e., grad_$\theta_g$). We will change the $g_{\theta_g}$ in the Algorithm 1 to $\nabla \theta_g$ to avoid confusion.

A2.1: Due to the high computational cost of testing models with multiple attacks, we conducted experiments based on the guidance provided by Nie et al. In fact, not all experiments tested 512 images. However, Nie et al. have demonstrated through experiments (Appendix C.2 in [1]) that the robust accuracies of most baselines do not change much on the sampled subset, compared to the whole test set.

[1] Diffusion Models for Adversarial Purification. 2022.

A2.2: We will add this part of the information. We utilize $RT_2$ and GAN-based models, which yielded the best results in the experiment. The adversarial examples are generated by the FGSM during training.

A3: The experimental results in this section are correct, and similar results (FGSM < PGD, Table 1 in [2]) are observed in the work of Ughini et al. And we conduct AutoAttack verification under two different settings: In Table 2,3,4,5, we attack the entire (purifier+classifier) model, and in Table 7 and 8, we attack the classifier model. We also realize that the current tables can easily lead to misunderstandings. Therefore, we plan to redesign the Table 7 and 8 according to Reviewer n69U's comments, replacing AutoAttack with CW and PDG-1000.

Here is the explanation of the results: Considering the robustness misestimation caused by obfuscated gradients of the purifier model, we use BPDA (Athalye et al., 2018a) and follow the setting by Yang et al. (2019), approximating the gradient of the purifier model as 1 during the backward pass, which is equivalent to attacking the classifier model. To maintain the same settings, AutoAttack only attacks the classifier. The advantage of AutoAttack lies in using EOT and adaptive mode with white-box and black-box attacks to attack more complex architectures, such as black-box and random factors in the architectures. If there are some random factors in the classifier, although the classifier model can effectively defend against FGSM and PGD, it can still be vulnerable to AutoAttack. However, in classic classifiers (such as Resnet-18), these factors do not exist, and simple white-box attacks cause more damage. Under standard cases, the entire model should be input into AutoAttack directly, and the experimental results are shown in Table 2,3,4,5. We will redesign Table 7 and 8 to avoid confusion.

[2] Trust-No-Pixel: A Remarkably Simple Defense against Adversarial Attacks Based on Massive Inpainting. 2022.

We greatly appreciate the comments of the Reviewer jVyX. Below are our responses to the questions raised. Due to exceeding the character limit for a single reply, we divide the response into two parts (2 / 2).

The second part is about questions:

QA1: The training cost of using FGSM for adversarial fine-tuning is acceptable. At the same time, as we propose a new paradigm that considers both AT and AP, we use FGSM to verify our ideas preliminarily. Here is a simple experiment regarding time, where we have measured the time required to process 10,000 images:

QA2: The impact of optimization using consistency loss of adversarial examples is limited. We can employ a variety of attacks and obtain a variety of different adversarial examples. When using consistency loss to learn information from adversarial examples, which potentially leads to overfitting and weakens the generalization ability against unseen attacks. We also discuss this point in the related work section. Additionally, as the perturbations are minimal, in many strong attacks, the consistency loss from adversarial examples is less than the consistency loss from generated examples ($\mathbb{E}[x-g(x_{adversarial})] < \mathbb{E}[x-g(x_{generated})]$). This also results in the consistency loss not playing a significant role during training.

QA3: In the pipeline based on the image completion, the repair results of the boundary of the missing area are better than the middle area, and pixels with different distances from the filled edge should be assigned different weights. A common way is to use the distance between pixels (i.e., $l_1$ distance) for weight calculation. When repairing large missing areas, the $l_1$ loss will be more effective in improving repair quality.

QA4: We hope AToP can be applied to any pre-trained generator-based purifier model. To this end, we conduct experiments on two common generator-based purifier models, including GAN-based and AE-based. At the same time, we also attempted to use diffusion-based purifier models. However, as mentioned in the limitations section, our method cannot easily fine-tune all models yet, such as some models with high computational costs.

Thanks for the further comments. We conduct some additional experiments and provide corresponding explanations, hoping that these can address your concerns.

Due to the existence of obfuscated gradients and random factors in adversarial purification, attacks such as FGSM and PGD cannot calculate the correct perturbation. At this point, the model might result in 'illusory' robustness. The results of robust accuracy against FGSM on CIFAR-10 are shown in the table below. The robust accuracy may be unreliable when testing on FGSM and PGD with the evaluation protocol from Section 4.1. Therefore, regarding these attacks, Athalye et al. proposed a more reliable evaluation protocol, which has been widely used in adversarial purification tasks (Yang et al. and Ughini et al.). We also follow the same setting to validate the robustness against FGSM and PGD.

As you correctly mentioned, the evaluation protocol in Section 4.4 should not be applied to AutoAttack. Here are the results under the standard cases (Section 4.1) of AutoAttack.

Replying to the Public Comment (Update: Since the person who responded has deleted the comments, please ignore the following response):

Thanks for your interest in our work! Below are our responses to the concerns raised.

1: Since this paper [0] has not been formally published, we are unaware of this work. We appreciate you sharing it with us.

After reading, we think there is a significant difference between our work and [0]. Firstly, the problems we focus on are different: The current mainstream SOTA method is based on the pre-trained generator model. Therefore, we consider the limitation of the existing pre-trained generator-based purifier model, where all parameters are fixed and cannot be further improved for known attacks, rather than simply proposing a more robust AE model or GAN model. Secondly, due to the difference in problems, there are differences in our algorithms as well. In [0], both clean examples and adversarial examples are input simultaneously to optimize the AE model. Our method comprises two stages: the pre-training stage with clean examples and the fine-tuning stage with adversarial examples. This makes our method more widely applicable rather than being constrained to a specific generator-based purifier model. Therefore, although there is a similarity in the high-level concepts, the problems and specific algorithms are different. At the same time, we are addressing challenges in the most recent methods.

2: In our evaluation experiments, we have utilized EOT, as mentioned in Section 4.1: '... and utilizes Expectation Over Time (EOT) (Athalye et al., 2018b) to tackle the stochasticity introduced by random transforms.'

3: In the paper, we do not conduct any robustness evaluation experiments on the diffusion-based model. But we guess your 2nd and 3rd points may be that you hope we can conduct experiments using PGD+EOT. On this point, we have also had discussions with the Reviewer jVyX. We are currently conducting detailed experiments on EOT. This part will be updated in the paper. And you can find more details in our conversation.

4: We are confused about the fourth point. Why do you say 'the GAN-based model is notably weak, enhancement is relatively easy'? Are there any studies about this viewpoint? At the same time, we test AToP on two of the latest models (Ughini et al., (2022) and Wu et al., (2023)).