Vulnerable Data-Aware Adversarial Training

Thank you greatly for dedicating your time and expertise to reviewing our paper. Your comments and suggestions are truly helpful to enhance our paper and have pointed out the direction of our future work. Please see our response below.

W1: Effectiveness of soft margin and temperature parameter $\tau$ against targeted attacks.

Thank you for pointing this out, and we have conducted experiments to show the effectiveness of soft margin and temperature parameter $\tau$ against targeted attacks. Specifically, we train ResNet18 on CIFAR-10 with the pure margin (Equation (2)) and the soft margin with different values of $\tau$. For evaluation, the targeted adversarial attacks are variants of the PGD attack, i.e., PGD_Random and PGD_Least_Likely. PGD_Random indicates that the target labels are chosen randomly. PGD_Least_Likely indicates that the target label is the label corresponding to the smallest output logit. Both strategies for selecting target labels are designed following the convention [1]. As shown in the table below, the adversarial accuracy under targeted attacks increases when the pure margin is changed to the soft margin. Consequently, the soft margin is shown to be effective to improve the adversarial accuracy under the targeted adversarial attack. Moreover, when the temperature parameter $\tau$ is set to five, the adversarial accuracy becomes the highest, demonstrating the validity of setting $\tau$ to five as default.

W2: Theoretical analysis.

We sincerely appreciate your meticulous review. Following your suggestion, we promise to remove the theoretical claims entirely in the next version of this work. Specifically, we will remove or change the following contents:

• Remove the descriptions about the theoretical analysis in Abstract (lines 11-12).

• Remove the claim about the theoretical analysis in Introduction (line 66).

• Remove contents of the theoretical analysis in Section 4.3 (lines 201-213).

• Remove the descriptions of the proofs in Appendix B (lines 468-469).

• Remove the whole contents about theoretical analysis and proofs in Appendix B (lines 493-515).

• Change the answer in line 709 of NeurIPS Paper Checklist 3 to [NA], and the justification in line 710 to "The paper does not include theoretical results".

Furthermore, we are actively working towards providing a rigorous proof regarding the effectiveness of the proposed VDAT method. Specifically, we will provide the whole set of assumptions in terms of the constraints for the network architecture, the parameter update method, the domain and conditions of the input data and labels, and the positive eigenvalues of the Gram matrix. Besides, the corresponding definitions about the network architecture and update methods, along with formulated proofs will be also provided. Thanks again for your valuable suggestions, and we hope to present a rigorous theoretical analysis in terms of the linear classifier as a complete but separate study real soon.

W3: Line 115: Missing braces {}.

We will change "1,2,...,k" in the subscript to "{1,2,...,k}" in line 115.

W4: The notation V_{x_i} and V_θ(x_i) are used inconsistently throughout the text (e.g., around line 131) and should be unified.

We will change the symbol $V_{x_i}$ in line 131 to $V_{\theta} (x_i)$ for uniformity. Moreover, the symbols $V_{x_i}$ and $P_{x_i}$ in Figure 3 will be also changed to $V_{\theta} (x_i)$ and $P_{\theta} (x_i)$, respectively.

W5: In Algorithm 1, the model parameter θ may be missing from the input and output specifications.

We will change "the target model" in Input to "the target model with parameter $\theta$", and "The trained model" in Output to "The trained model with parameter $\theta*$". Besides, the words "The trained model with $\theta$" in line 9 of Algorithm 1 will be changed to "The trained model with $\theta*$".

Reference

[1] Adversarial machine learning at scale, ICLR 2017.

We are immensely grateful for your effort in reviewing our work, as well as for your constructive suggestions and insightful questions. In the following, we will address each of your concerns point-by-point.

W1 & Q4: Adversarial training experiments on ImageNet-21K in terms of profiling results detailing the actual runtime breakdown and empirical or simulated training overhead analysis.

Thank you for the nice suggestion. We address your concern from the following two aspects.

• Profiling results detailing the actual runtime breakdown: Due to the limitations of computational resources, the complete training of 60 epochs on ImageNet-21K will take more than 17 days, and we are sorry for not being able to provide those results in the rebuttal period. Instead, we perform VDAT along with the FGSM-based adversarial training to train ResNet50 for 10 epochs on ImageNet-21K. As shown in the table below, the time breakdown on ImageNet-21K is similar to that on ImageNet-1K. These results demonstrate that VDAT does not significantly increase the time proportion of the vulnerability calculation when the scale of the data becomes larger.

• Empirical or simulated training overhead analysis: We compare the training cost of FGSM-PGK [1] and the proposed VDAT method on ImageNet-21K. Both methods are performed to train ResNet50 for 10 epochs. As can be seen from the table below, VDAT is more efficient than FGSM-PGK. Consequently, it is demonstrated that the proposed VDAT method is still efficient on the large-scale dataset.

W2 & Q3: It is unclear how the effectiveness of VDAT varies with different attack methods for vulnerability assessment. Would a stronger attack (e.g., PGD) lead to a more robust final model, even if the training itself still uses FGSM or PGD?

The stronger attack for vulnerability assessment can lead to a more robust final model, but it lowers the natural accuracy and increases the training cost. Specifically, we have conducted experiments on CIFAR-10 to train ResNet18 with different training losses, i.e., FGSM and PGD. Meanwhile, the adversarial attack for vulnerability assessment are changed from the default FGSM to PGD10 and PGD20. As can be seen from the table below, using stronger attacks for vulnerability assessment can increase the adversarial accuracy. However, this will also decrease the natural accuracy and training efficiency. Considering the balance of natural accuracy, adversarial accuracy, and training cost, we choose FGSM as the default attack for vulnerability assessment.

Q1: Could you please explain the significant accuracy improvement observed around epoch 100 in Figure 4?

The significant accuracy improvement is caused by the decay of learning rate. Specifically, the learning rate is decayed with the factor of 0.1 at the 100-th and 105-th epochs. This practice follows the convention of adversarial training [2].

Q2: Could you please explain why VDAT_FGSM generally performs better than VDAT_PGD on both natural accuracy and adversarial accuracy? This seems a bit counterintuitive to me, since PGD provides stronger adversarial examples.

Thank you for the insightful question. We answer your question from the following two aspects.

In the case of adversarial training, we would like to argue that VDAT_FGSM performs better than VDAT_PGD on natural accuracy and weak adversarial attacks, while VDAT_FGSM is not as good as VDAT_PGD on stronger attacks. As shown in Table 1, VDAT_FGSM outperforms VDAT_PGD in terms of natural accuracy and adversarial accuracy under FGSM attack. As for the stronger adversarial attacks, i.e., C&W and AA, VDAT_PGD often performs better than VDAT_FGSM. This is mainly because VDAT_PGD provides stronger adversarial examples, and learning these adversarial examples can help the model achieve better accuracy under stronger adversarial attacks. However, it is shown that stronger adversarial examples can lead to robust overfitting of the strong adversarial perturbations [3]. As a result, the model suffers from the poor generalizability to the clean data and weak adversarial examples, thus the natural accuracy and adversarial accuracy on weak adversarial attacks become lower.

In the case of robust neural architecture search (NAS) in Table 4, the phenomenon mentioned by the reviewer is mainly caused by the overfitting phenomenon of the supernet. Specifically, VDAT_PGD will cause robust overfitting more easily than VDAT_FGSM during the training process of the supernet [3]. As indicated in the previous study [4], the overfitting can make the NAS algorithm add more parameter-free operations (e.g., skip connection) to the derived architecture. As a result, the architectures searched with VDAT_PGD demonstrate lower natural accuracy and adversarial accuracy than those searched with VDAT_FGSM.

References

[1] Improving fast adversarial training with prior-guided knowledge, TPAMI 2024.

[2] Boosting fast adversarial training with learnable adversarial initialization, TIP 2022.

[3] Understanding robust overfitting of adversarial training and beyond, ICML 2022.

[4] Understanding and robustifying differentiable architecture search, ICLR 2020.

We sincerely thank you for the time and effort you have invested in reviewing our paper. Your comments and questions are very insightful and highly valuable for us to enhance the paper. Your comments are addressed point-by-point in the following.

W1 & Q1: The effectiveness of the proposed margin-based vulnerability calculation compared with other measurement (e.g., pure margin) is not fully explored by experimental results.

Thank you for pointing this out. To demonstrate the effectiveness of the proposed margin-based vulnerability calculation method based on soft margin, we have conducted experiments in terms of the soft margin and pure margin. Specifically, the calculation of the pure margin is shown in Equation (2) in the paper. The benchmark datasets are CIFAR-10 and CIFAR-100, and the deep neural network for adversarial training is ResNet18. As shown in the table below, both natural accuracy and adversarial accuracy decrease when changing the soft margin to the pure margin. Consequently, the proposed margin-based vulnerability calculation method is shown to be effective.

W2 & Q2: There lacks experiments for combining VDAT with other adversarial training frameworks like AWP and TRADES.

Thank you for the insightful comment. We have integrated VDAT into AWP and TRADES to perform adversarial training. In this set of experiments, the benchmark dataset is CIFAR-10 and the deep neural network for training is ResNet18. The natural accuracy, adversarial accuracy, and training cost are reported in the table below. As can be seen, VDAT can improve the natural accuracy and adversarial accuracy of both TRADES and AWP. Meanwhile, the training cost of both TRADES and AWP is decreased owing to the integration of VDAT. In summary, VDAT is shown to be effective in improving the performance of AWP and TRADES.

Q3: Can the proposed method be extended to other tasks apart from image classification? If so, how to measure the vulnerability and design the data filtering approach?

Yes, it can. We have shown that the proposed method can be extended to the node classification task. In this task, the vulnerability is still measured by the difference of the soft margin. Specifically, we denote the clean graph and the perturbed graph as $x$ and $x'$, and the output logit for class $c$ as $\phi_{\theta}^{c}(x_i)$, where $\theta$ denotes the parameters of GNN, and $x_i$ denotes the $i$-th node in the graph $x$. Giving the node $x_i$ and its label $y_i$, the soft margins of $x_i$ in $x$ and $x_i'$ in $x'$ are calculated by Equations (3) and (4) based on $\phi_{\theta}^{c}(x_i)$ and $\phi_{\theta}^{c}(x_i')$. Finally, the vulnerability of node $x_i$ is determined by Equation (5) based on the soft margins calculated.

For the data filtering, we calculate the probability of adversarial training for each node based on Equation (6), where the vulnerability values are determined in the previous steps. According to the probability calculated, we can divide the nodes for training into two sets, i.e., $X_{adv}$ and $X_{nat}$. The nodes in $X_{adv}$ are forward to the PGD attack to change their corresponding edges. Then, the GNN is trained based on the nodes in $X_{adv}$ and $X_{nat}$.

We have conducted experiments on both Cora and CiteSeer datasets with the vanilla GCN. The PGD and Min-Max adversarial attacks are implemented based on the previous study [1] with the perturbation rate of 20%. As can be seen from the table below, the proposed VDAT method can still improve the natural accuracy and adversarial accuracy while decrease the training cost.

Reference

[1] Topology attack and defense for graph neural networks: An optimization perspective, IJCAI 2019.

Thank you sincerely for the recognition in our work. We appreciate your great effort and answer your questions in detail below.

Q1: Line 174: Should $\mathcal{R}(0,1)$ denote a uniform random value ranged from zero to one?

Yes, $\mathcal{R}(0,1)$ denotes a uniform random value ranged from zero to one. The aim of adopting the uniform distribution is to ensure that the probability of adding adversarial noise is exactly $\mathcal{P}_{\theta}(x_i)$. We will add the claim in terms of the uniform random value in line 174 in our next version, to make the description clearer.

Q2: What do the authors see as the value added by Proposition 4.2?

The value of Proposition 4.2 is that it can theoretically show the effectiveness of the proposed training loss. Specifically, Proposition 4.2 can show that training with the loss in Equation (8) can lower the upper bound of the adversarial loss. Because the lower upper bound leads to better adversarial robustness, it is theoretically demonstrated that the proposed training loss can enhancing the adversarial robustness.

However, as stated by Reviewer 25Wa, the current version of the theoretical analysis is still not complete in terms of the definitions, assumptions, and proofs. To avoid misleading readers, we decide to remove the theoretical claims including Proposition 4.2 in the current version. In the future, we will give a rigorous proof regarding the linear classifiers at the first step, and then conduct theoretical analysis for the deep neural networks based on the findings in the first step. We are actively working on the first step and hope to be able to present it as a complete but separate study real soon.