Self-Supervised Adversarial Training via Diverse Augmented Queries and Self-Supervised Double Perturbation

Dear reviewer iqHN,

We appreciate your positive comments! Here is our response:

1. We added the results with ResNet18 and ResNet50 on CIFAR10 (larger model is hard for us to train in limited time). The results of ResNet18 are not included in our initial submission for it is not as obvious as in the larger model. The AIR[37] paper doesn’t include PGD accuracy on ResNet50 or checkpoint and it can take us a long time to train. So only clean and AA attack accuracy are provided.

ResNet18:

ResNet50:

2. We added weight perturbation in the ablation study experiments: | Method | Clean | PGD | |---|---|---| |Baseline | 51.44 | 30.68 | |DAQ(single-BN) | 52.27 | 31.56 | |Diverse Augmented Query | 52.67 | 32.11 | |Weight Self-Perturbed Scheme | 51,56 | 32.37 | |DAQ-SDP(ours) | 53.54 | 33.09 |

3. We added transfer learning results from CIFAR10 to STL 10: | Method | Clean | PGD | |---|---|---| |Baseline | 63.84 | 40.66 | |DAQ-SDP(ours) | 66.79 | 40.75 |

4. We hope to argue that comparison between our method and previous ones on ResNet34 are fair. First, DynACL[24]+AIR[37] results are the same as reported in the paper. Second, DecoupledACL[38] hyperparameters on larger models are finetuned for clean-robust trade-off. The hyperparameter to tune is weight decay in adversarial training. On CIFAR100, weight decay of 2e-4 gives 51.44% (clean) and 30.68% (PGD) but a larger regularization with weight decay of 5e-4 gives 49.94% and 31.99%. On CIFAR10, weight decay of 5e-4 gives 82.46 (clean) and 56.86% (PGD) but a larger regularization of 1e-3 gives 80.57% and 56.48%. TARO[23] is an attack method that can be combined with different frameworks. The initial one-stage training results reported in the paper is much lower and we combined it with our model for improved results. The hyperparameter that matters is the batch size for choosing target to attack. We use the same batch size of 512. On ResNet50, the hyperparameter is finetuned in the same manner. An increase of weight decay from 5e-4 to 6e-4 on CIFAR10 typically causes a robustness reduction of around 1.00% in these methods.

5. We are glad to discuss your questions about the claim of “unified perspective” and “robust generalization”. Indeed, the whole adversarial self-supervised learning literature is about robustness and generalization transferable to downstream tasks. And we hope to clarify the following points:

a) There are actually two types of generalization in the self-supervised AT context. The first is the generalization between tasks from the SSL perspective. For instance, SSL robustness transferring to other datasets and downstream classification tasks. At least to the best of our knowledge, all previous works explore the extent of this generalization. In this work we are actually curious about the robust generalization of SSL methods from the adversarial training (training-testing) perspective, which is related to the robust overfitting concept in supervised AT. Since SSL operates on features and needs an extra linear finetuning for classification loss, this generalization is often overlooked in previous works. Although we don’t observe decreased validation robustness during training (robust overfitting) as in supervised AT, we do observe a large robust generalization gap (in Figure 1 of the paper). Our method helps to solve this generalization problem (in Figure 3) and improves the results from this AT perspective for both contrastive and non-contrastive SSL (shown in the same dataset and cross-dataset downstream classification results). The related robust overfitting phenomenon is studied a lot in supervised AT but not studied in the SSL context. We believe the identification and analysis of such traits of self-supervised AT create a space for further improvements. That is what we mean by “providing a unified perspective across AT under different supervisions”. We look forward to your suggestions about making these phrases clearer.

b) In supervised AT, phenomenon of robust overfitting is prevalent and AWP helps to improve robustness. However, whether AWP works depends on the trade-off between reduced generalization gap and decreased training accuracy. Since there is a difference in the difficulty of supervised AT and self-supervised AT (which is “a challenging problem due to its two mixed challenging goals” and “clearly can be even more challenging than the semi-supervised AT setting” in [38]), it’s unclear whether the generalization gap for self-supervised AT is large enough to compensate for the trade-off for improvement. It’s also interesting that using the self-perturbed AWP scheme provides a 0.70% and 0.80% improvements over self-perturbed AWP for the whole training process on CIFAR10. Without the scheme there is a 0.88% improvement on PGD acc. but 0.81% reduction on clean acc. compared with the result without self-perturbed AWP. This can also be related to the traits of self-supervised AT.

c) Transferring preferrable AT properties except robustness itself, e.g. loss landscape smoothness, from SSL to classification is not considered in previous works and not as easy as one might think. In Figure 4 of the uploaded one-page pdf, we show the 1d visualization of resulting loss landscape of the baseline and our method. Although regularizing SSL objective smooths downstream loss landscape, the transferred regularization effect is much attenuated than directly regularizing classification loss as in [35], showing the difficulty of such an improvement.

6. The figures are uploaded in the one-page pdf. Thanks again for your time and patience.

Dear reviewer iqHN,

Thanks for your reply.

First, we would like to argue that in the initial submission, we already pointed out that the performance of the method is more obvious on larger models than on ResNet18, that's because our method requires larger model capacity for better performance. This claim is already included in the original submission. You can find this from line 275 to line 284 in Experimental Setup section. So we are not hiding results. In the rebuttal, you require us to provide the results on other models, so we provided them.

Second, the response to Q4 is trying to solve the worry that our experiments are unfair. We are arguing that we finetuned the parameters on larger models and used appropriate batchsize.

Third, the response to Q5 is trying to explain our motivation. We first argued that the generalization/overfitting problem from adversarial training perspective in self-supervised AT is overlooked in previous works. But the identification of this problem creates a space for improvement. Then we argued that the improvement brought by the self-perturbed adversarial weight technique is not that straightforward or easy.

Best Regards,

Authors

Dear reviewer 6QGn,

We appreciate your efforts and time in reviewing our paper. The following is our response. First, the assumption in line 56 is also supported by [38]. This work includes the following description for self-supervised adversarial learning:

1. “A challenging problem due to its two mixed challenging goals”.
2. “Clearly … can be even more challenging than the semi-supervised AT setting.”

SSL models typically require 1000 epochs to train. In the adversarial training of SSL models, no information about decision boundary is available and the features are required to be robust against attacks towards any directions away from its original place. That explains why we describe them as hard tasks in our paper.

Second, we choose the hyperparameters in line 284 to 288 for the following reasons:

1. The training epochs, attack steps, attack range in training and attack range in testing are the standards reported in a series of previous works including [22][20][14][38][37].

2. We conducted experiments and finds that using the 60-epoch self-perturbed AWP scheme provides a 0.70% and 0.80% improvements over self-perturbed AWP for the whole training process on CIFAR10. Without the scheme there is a 0.88% improvement on PGD accuracy but 0.81% reduction on natural accuracy compared with the result without self-perturbed AWP.

3. We choose this weight self-perturbation range because it’s in the middle of the perturbation range used in supervised adversarial learning work [35] to give a good trade-off on robust generalization gap and training accuracy. We also show the effectiveness of our method with this hyperparameter with the experimental improvements. Please check the following ablation study results:

Third, we have added the experiments on ResNet18 and ResNet50. The results of ResNet18 are not included in our initial submission for the result is not as obvious as in the larger model case. Please note that the DYNACL[24]+AIR[37] paper doesn’t include the PGD accuracy on ResNet50 or checkpoint and it take us too long to train for this result. So only clean and AA attack accuracy are provided. The experiments are conducted on CIFAR10.

ResNet18:

ResNet50:

The epoch time for the baseline, diverse augmented query and weight self-perturbation in the later epochs is 2min52s, 5min24s and 6min29s respectively. We have included a new Social Impact section for this limitation.

We are sorry that the figures are fuzzy and reuploaded the Figure 1 and Figure 3 in the one-page pdf. Also we have corrected the typos including reference, consistent method name and “the result” in line 308. We will change \citep to \citet in our new version.

Thanks for your helpful advice. You can also check our explanation about the motivation in other responses. We appreciate your efforts and time. Thank you!

Sorry for missing the ablation study results in the previous reply. The ablation study result on CIFAR100 with SimCLR is here:

We further clarify the runtime of DecoupledACL, TARO and our method DAQ-SDP: DecoupledACL takes 2min52s per epoch. TARO takes 2min54s per epoch. Our method DAQ-SDP takes 5min24s in the first 60 epochs and 6min29s in the last 40 epochs.

Dear reviewer dErS,

We appreciate your time and efforts in reviewing our paper. We are sorry the phrases of “unified perspective” and “unified understanding” cause confusion. First please let us explain the following points:

a) There are actually two types of generalization in the self-supervised adversarial training context. The first is the generalization between tasks from the SSL perspective. For instance, SSL robustness transferring to other datasets and downstream classification tasks. At least to the best of our knowledge, all previous works explore the extent of this generalization. In this work we are curious about the robust generalization of SSL methods from the adversarial training (training-testing) perspective, which is related to the robust overfitting concept in supervised AT. Since SSL operates on features and needs an extra linear finetuning for classification loss, this overfitting is often overlooked in previous works. Although we don’t observe decreasing validation robustness during training (robust overfitting) as in supervised adversarial training, we do observe a large robust generalization gap (shown in Figure 1). Our method helps to solve this generalization problem (in Figure 3) and improves the results from this AT perspective for both contrastive and non-contrastive SSL methods (shown in the same-dataset or cross-dataset finetuned results). The robust overfitting phenomenon is studied a lot in supervised AT but not studied in the SSL context. We believe the identification and analysis of such traits of self-supervised AT create a space for further improvements. That is what we mean by “bridging the gap and providing a unified perspective for adversarial training under different supervisions”. We look forward to your suggestions about making this phrase clearer.

b) In supervised AT, the phenomenon of robust overfitting is prevalent and AWP helps to improve robustness. However, whether AWP works depends on the trade-off between generalization gap and decreased training accuracy. Since there is a difference in the difficulty of supervised AT and self-supervised AT (which is “a challenging problem due to its two mixed challenging goals” and “clearly even more challenging than the semi-supervised AT setting” in [38]), it is unclear whether the generalization gap for self-supervised AT is large enough to compensate for the trade-off to achieve robustness improvement and transferred to classification.

Second, we add some additional experiments. a) Tables below show results on ResNet18 and ResNet50 with SimCLR on CIFAR10：

ResNet18:

ResNet50:

b) This table shows ablation study with self-perturbed weight：

c) This table shows transfer learning from CIFAR10 to STL10：

d) Figure 4 in the newly uploaded pdf shows the effects of baseline and our method with self-perturbed weight on downstream weight loss landscape with 1D visualization. The regularizing effect transferred to downstream loss landscape is actually much attenuated compared with regularizing classification loss as in [35], showing difficulty of such a transferred improvement.

Third, thanks for pointing out the clarity problem. $CL$ loss in eq1 is defined as a two-variable function and $ACL$ loss in eqs 3 and 4 is defined as a three-variable because of the added adversarial view of data. Different works have slightly different way to deal with this extra variable. For instance, [14] averages the two-variable version. Projector $g$ is a component that preserves instance discriminative feature. Predictor $h$ is a component to prevent model callapse. $aug_i$ represents differently augmented data. $f_{2-aug_{i}-clean}$ means the parameters of the student model with the pairwise-BN decided by augmentation strength and adversarial type.

We respectfully argue that lots of previous works have a simple but effective idea. For instance, [14] combines pseudo-supervised classification, an idea used in semi-supervised AT [26], and high frequency component, a technique shown to be effective for robustness in [41], with ACL. [18] has a simple idea that similar features should be removed from negative pairs. [24] and [38] has the simple idea that AT and SSL should have separate hyperparameters. Even the first ACL works [20][22] are based on an idea that sounds straightforward: if the feature cannot be perturbed to be far from its original place, then the feature is robust. This is actually a simple combination of the notion of adversarial attack and instance discrimination. Identifying generalization problem in Self-Supervised AT and adjusting traits of downstream loss landscape via regularizing pretext tasks is actually not straightforward in the way it sounds.

We have included the Societal Impact section: Our work is useful for pretraining robust models with no label. However, it generates one more adversarial data and perturbed weight, which take more computation. So it can cause more consumption of energy and pollution. Despite this limitation, we believe our method is still beneficial for the society and promotes model robustness in real life.

[41] Haohan Wang, Xindi Wu, Zeyi Huang and Eric P. Xing. High-frequency Component Helps Explain the Generalization of Convolutional Neural Networks. In CVPR, 2020.

Dear reviewer mcfF,

We appreciate your positive comments. Here is our response for your questions:

1. We have added new experiments on ResNet18 and ResNet50 to prove the effectiveness of our method.

ResNet18:

ResNet50:

2. We have added more ablation study including the BN technique. | Method | Clean | PGD | |---|---|---| |Baseline | 51.44 | 30.68 | |DAQ(single-BN) | 52.27 | 31.56 | |Diverse Augmented Query | 52.67 | 32.11 | |Weight Self-Perturbed Scheme | 51,56 | 32.37 | |DAQ-SDP(ours) | 53.54 | 33.09 |

3. We added some explanations for the notations in our paper: The $aug_i$ in the equations represents either strongly or weakly augmented data. $f_{2-aug_{i}-clean}$ and $f_{2-aug_{i}-adv}$ in equation 10 represent the parameters of the student model with the pairwise-BN decided by augmentation strength and adversarial category. Since the adversarial data generation in equation 11 works on the adversarial branch, the parameter used in equation 11 is $f_{2-aug_{i}-adv}$. $f_{2-aug_{i}-adv}$ is the same as $f_{\theta_2-aug_{i}-adv}$ in equation 15.

4. We have added additional Societal Impact section:

Our work is useful for pretraining robust models with no label. However, it generates one more adversarial data and self-perturbed weight, which take more time and computation. So it can cause more consumption of energy and pollution. Despite this limitation, we believe our method is still beneficial for the society and promotes model robustness in real life.

Thanks again for your positive comments!