Robust NAS under adversarial training: benchmark, theory, and beyond

We thank the reviewer AZRS for the insightful feedback. We address the concerns below and add necessary revisions highlighted in red color in the updated version.

---

Q1: [The performance under stronger attack methods (e.g., AutoAttack), out-of-distribution data and/or datasets with distribution shift.]

A1: Thanks for the suggestion, the discussion on the AutoAttack can be found in general response for details.

Furthermore, to make the benchmark more comprehensive in a realistic setting, we evaluate 6466 architectures in the search space under distribution shift. We choose CIFAR-10-C [3], which includes 15 visual corruptions with 5 different severity levels, resulting in 75 new metrics. In the supplementary material, we have provided the result in a JSON file.

We add a new section Appendix F.4 to describe the evaluation and discuss the result. In Fig.7 of the revised version (or the link), we show the boxplots of the accuracy. We can find that all architectures are robust towards corruption with lower severity levels. When increasing the severity levels to five, the models are no longer robust to fog and contrast architectures. Moreover, similar to robust accuracy under FGSM and PGD attacks, we can see a non-negligible gap between the performance of different architectures, which motivates robust architecture design. In Table 6 of the revised version (or the link), we show the Spearman coefficient between various accuracies on CIFAR-10-C and adversarial robust accuracy on CIFAR-10. Interestingly, we can see that the adversarial robust accuracy has a relatively high correlation with all corruptions except for contrast corruptions. As a result, performing NAS on the adversarially trained architectures in the benchmark can achieve a robust guarantee of distribution shift.

---

Q2: [The authors observe that the standard and robust accuracies exhibit a meaningful level of correlation. If they are, do we really need a separate robustness benchmark that includes robust accuracies? Wouldn’t searching for the optimal architecture in a conventional sense work as a good proxy and naturally result in a more robust architecture?]

A2: Let us explain why we do need a separate benchmark. We agree with the reviewer that there exists a partial correlation between clean accuracy and robust accuracy, as suggested by our benchmark. This is one of the findings from our benchmark: demonstrating that the correlation is not extremely low under adversarial training, compared to the benchmark under standard training [6]. We have included the discussion in Appendix B.2 of the paper.

Notice that before building the benchmark, we did not know the relationship between clean accuracy and robust accuracy under adversarial training. To be specific, in the existing benchmark [6] that was constructed in the same space but under standard training, the correlation was not high. This correlation was enhanced only when adversarial training was involved. This is one finding from our benchmark. Moreover, we can see a notable difference between these two benchmarks in terms of top architecture id and selected nodes, see Fig. 4 and Fig. 8. This suggests that employing the benchmark under standard training to identify a resilient architecture may not guarantee its robustness under adversarial training.

---

Q3: [NAS-Bench-201 is too constrained of a search space to accurately reflect the vastness of the potential architecture pool. For instance, [1] shows that many NTK-based methods fail outside the boundary of NAS-Bench-201. While this is a good first step towards building a more comprehensive robustness benchmark, it would be great if the authors could show that the experimental and theoretical analyses hold outside the tested search space.]

A3: We thank the reviewer for pointing out this interesting question. When checking previous literature on NTK train-free algorithms, we find that NTK-based methods perform well on different search spaces beyond NAS-Bench-201, e.g., TE-NAS [2] and EigenNAS [5] on DARTS search spaces. Nevertheless, the applicability of such results under adversarial training remains uncertain on other search spaces, as it requires expensive construction of the benchmark under adversarial training.

We thank the reviewer dbFp for the constructive feedback. We address the concerns below and incorporate necessary revisions highlighted in red color in the updated version.

---

Q1: [It has been well known that adversarial robustness often may occur due to gradient obfuscation and is applicable for different sparse and dense model architectures [1,2]. Thus, a discussion on that would be necessary. In specific, is there a way to benchmark based on a subnet's susceptibility to be more prone towards obfuscation?]

A1: We are thankful to the reviewer for the suggestion of this ablation experiment. Upon the suggestion of the reviewer, we add a new section in Appendix F.3 to discuss this.

We agree with the reviewer that gradient obfuscation might be a cause for the overestimated robustness of several adversarial defense mechanisms, e.g., parametric noise injection defense [3] and ensemble defenses [6], and can exist in the defense of several architectures [2]. However, in this work, we employ vanilla adversarial training as a defense approach, which does not suffer from gradient obfuscation, as originally demonstrated in [1] and mentioned in later literature [3].

We investigate whether the sparsity of the architecture design might have an impact on gradient obfuscation. Specifically, based on the search space in Figure 1 of the paper, we group all architectures in terms of the number of operators “Zeroize’’, which can represent the sparsity of the network (the more “Zeroize” operations, the more sparse the resulting network). Next, we select the network with the highest robust accuracy in each group and check whether this optimal architecture satisfies the following characteristics of non-gradient obfuscation [1]:

• a) One-step attack performs worse than iterative attacks.
• b) Black-box attacks perform worse than white-box attacks.
• c) Unbounded attacks fail to obtain $100\%$ attack success rate.
• d) Increasing the perturbation radius $\rho$ does not increase attack success rate.
• e) Adversarial examples can not be found by random sampling if gradient-based attacks do not.

The result below is the evaluation of these architectures under FGSM, Square attack, and PGD attack with varying perturbation radius. We can see that all of these architectures clearly satisfy a), c), and d). Regarding e), all of the networks still satisfy it because gradient-based attacks can find adversarial examples. For b), only the sparse network with 4 ``Zeroize" does not satisfy the condition. Therefore, we can see that these architectures can not be considered as suffering gradient obfuscation, which is consistent with the original conclusion for adversarial training in [1].

Lastly, under the aforementioned defense mechanisms e.g., noise injection defense and ensemble defenses, some of these architectures in the search space might be more prone to gradient obfuscation. Thus, we believe it is possible to further develop robust NAS benchmarks under other defense mechanisms, but this is out of the scope of the current work.

---

Q2: [Did you use ImageNet or ImageNet-16-120? As the Fig. 2 and the contribution section has mention of each.]

A2: Following NAS-Bench-201 [4], we conduct an evaluation on ImageNet-16-120, which is a well-established benchmark. In the updated version, we revised the contribution on page 2 as follows:

"107k GPU hours are required to build the benchmark on three datasets (CIFAR-10/100, ImageNet-16-120) under adversarial training."

---

Q3: [Please demonstrate the efficacy of the NASRobBench on Autoattack.]

A3: Thanks for the suggestion, we have provided more evaluation of the benchmark, see general response for details.

---

Q4: [The related work of Adversarial example generation is not up to date, the author should discuss more about the auto attacks and other contemporary attacks.]

A4: In the revised version, we update the related work from the classical adversarial attack to up-to-date ways as follows:

To mitigate the effect of hyper-parameters in PGD and the overestimation of robustness [1], [7] propose two variants of parameter-free PGD attack, namely, APGD_CE and APGD_DLR, where CE stands for cross-entropy loss and DLR indicates difference of logits ratio (DLR) loss. Both APGD_CE and APGD_DLR attacks dynamically adapt the step-size of PGD based on the loss at each step. Furthermore, to enhance the diversity of robust evaluation, [7] introduce Auto-attack, which is the integration of APGD_CE, APGD_DLR, Adaptive Boundary Attack (FAB) [4], and black-box Square Attack [5].

---

Q5: [How you add noise "twice"?]

A5: We perform one PGD/FGSM attack on the raw image data to generate the adversarial data, and then we perform the same attack on this adversarial data again. We have added such clarification in page 9 of the revised version.

---

Q6: [Please extend the NTK score vs accuracy correlation with more recent and stronger attack scenarios.]

A6: We test the correlation between the robust accuracy under APGD_CE and the various NTK metrics in CIFAR-10. On one hand, we can still see that the robust NTK enables the increasing correlation under adversarial training, which is consistent with our observation in the original version for FGSM and PGD attacks. On the other hand, we can see the correlation decrease only ~0.5% for APGD accuracy compared to PGD accuracy. We have updated the result on the APGD attack In the revised version, see Fig.5 in page 9 or the anonymous link.

---

If the reviewer dbFp has any remaining concerns, we are happy to clarify further.

---

Refs

[1] Athalye, et al. "Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples." ICLM, 2018.

[2] Kundu, et al. "Dnr: A tunable robust pruning framework through dynamic network rewiring of dnns." ASP-DAC, 2021.

[3] He, et al. "Parametric noise injection: Trainable randomness to improve deep neural network robustness against adversarial attack." CVPR, 2019.

[4] Dong, et al. "NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search." ICLR, 2020.

[5] Andriushchenko, et al. "Square attack: a query-efficient black-box adversarial attack via random search." ECCV, 2020.

[6] Gao ,et al. "MORA: Improving Ensemble Robustness Evaluation with Model Reweighing Attack." NeurIPS, 2022.

[7] Croce, et al. "Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks." ICML, 2020.

[8] Croce, et al. "Minimally distorted adversarial examples with a fast adaptive boundary attack." ICML, 2020.

We appreciate the constructive feedback from the reviewer dbFp. During the rebuttal, we have already:

• evaluated all 6466 architectures under the APGD_CE attack on CIFAR-10 and plotted their correlation with the NTK score. Additionally, we evaluated 2000 architectures using the complete AutoAttack. Based on the suggestion from reviewer AZRS, we also assessed 6466 architectures using 75 additional corruption metrics on the CIFAR-10-C dataset and analyzed the results.

• added a new section to analyze gradient obfuscation in the proposed benchmark.

• incorporated more attacks in the related work, clarified the usage of ImageNet-16-120, and explained the implementation of adding noise twice.

We hope that our additional experiments and clarifications address the reviewer’s concerns. We sincerely appreciate it if the reviewer could reconsider the evaluation.

Best regards,

Authors

Q9: [The interpretations of Figure 3, specifically parts (b) and (c), pose some difficulties. Enhanced specificity in the explanations accompanying these figures would greatly facilitate understanding.]

A9: We have updated the caption in Figure 3 (b) and (c) as follows: (b) Overall, the architecture ranking sorted by robust metric and clean metric correlate well for lower ranking (when the value of the x-axis is larger, i.e., ranking of those bad architectures) but there still exists a difference for higher ranking. This motivates the NAS for robust architecture in terms of robust accuracy instead of clean accuracy. (c): The result reveals a high correlation across various datasets, thereby inspiring the exploration of searching on a smaller dataset to find a robust architecture for larger datasets.

---

If the reviewer QErk has any remaining concerns, we are happy to clarify further.

---

Refs

[1] "Generalization properties of NAS under activation and skip connection search." NeurIPS, 2022.

[2] "Generalization bounds of stochastic gradient descent for wide and deep neural networks." NeurIPS, 2019.

[3] https://github.com/MadryLab/robustness.

[4] "Overfitting in adversarially robust deep learning." ICML, 2020.

[5] "Make some noise: Reliable and efficient single-step adversarial training." NeurIPS, 2022.

[6] "Exploring the loss landscape in neural architecture search." Uncertainty in Artificial Intelligence, 2021.

[7] "Torchattacks: A pytorch repository for adversarial attacks." ArXiv, 2020.

[8] "Convergence of adversarial training in overparametrized neural networks." NeurIPS, 2019.

[9] "Explaining and harnessing adversarial examples." ICLR, 2015.

[10] "KNAS: green neural architecture search." ICML, 2021.

[11] "Theoretically principled trade-off between robustness and accuracy." ICML, 2019.

Q5: [The theoretical analysis provided appears to be a reiteration of what was presented by [1,2] in the context of clean NTK. Consequently, the contribution in this area seems minimal. The robust term was not accounted for in the theoretical framework. ]

A5: Let us explain why the previous work cannot be directly applied to our setting. How to handle the multi-objective trade-off between clean/robust accuracy under the more general robust architecture search framework is still unclear in theory. Here, we clarify the technical difficulties behind this.

Previous work [2] builds the generalization bound of fully-connected neural networks via NTK and [1] extends this result under the activation function search and skip connection search. Both of them focus on the standard training. Instead, our work studies the robust generalization bound under multi-objective adversarial training of the searched architecture. Here we list the following reasons why previous work cannot be employed to our setting under multi-objective training (including adversarial training):

• Trade-off and analysis of multi-objective training are unclear: We focus on a multi-objective training scheme by introducing a regularization parameter to balance the standard training and adversarial training. In this case, the coupling relationship during training as well as the clean/robust accuracy trade-off makes the analysis difficult. For example, how to tackle the robust term $A^\star(x,W)$ for analysis is questionable from previous work [1,2].
• NTK formulation under multiple-objective training is unclear: To our knowledge, only [8] consider an NTK-based analysis in adversarial training. However, [8] only involves the optimization convergence under adversarial training without generalization analysis. Previous works [1,2] are based on the sole NTK for generalization guarantees. Therefore, we need a new way to build different NTKs to connect clean/robust accuracy.

Our analysis addressed the following technical difficulties: 1) how to build the proof framework under multi-objective training by the well-designed joint of NTKs; 2) how to tackle the coupling relationship among several NTKs and derive the lower bound of the minimum eigenvalue of NTKs. 3) how to properly tackle the robust term $A^\star(x,W)$ between the input and weight perturbation; (See Lemma 4 for detail). Accordingly, our result demonstrates that, under adversarial training, the generalization performance (clean accuracy and robust accuracy) is affected by different NTKs. Concretely, the clean accuracy is determined by one clean NTK and robust NTK; while robust accuracy is determined by robust NTK and its “twice” perturbation version. Our results demonstrate the effect of different search schemes, perturbation radius, and the balance parameter, which doesn’t exist in previous literature.

---

Q6: [It would be interesting to learn about the correlation results when measured with the existing clean NTK, such as in previous works by [1,2] (which could correspond to beta being 0).]

A6: We are thankful to the reviewer for the suggestion. In the first column of Figure 5, we have provided the result for KNAS [10], which relies on the minimum eigenvalue of the clean NTK, and we can see the correlation with respect to clean NTK is lower than robust NTK. [3, Zhu et al] propose EigenNAS, which also uses the clean NTK but with different estimations for the minimum eigenvalue of NTK. Below we provide the result for EigenNAS. Following the original version of the paper, we plot the Spearman correlation between different NTK scores and various accuracy metrics. Specifically, we use adversarial data with PGD attacks to construct the robust NTK. We can see that robust NTK still has a higher correlation than clean NTK.

---

Q7: [There is a high correlation between clean and robust accuracy. The implications of using just clean NTK values for adversarial robust search and how that might affect the interpretation of the results.]

A7: We agree with the reviewer that there exists a partial correlation between clean accuracy and robust accuracy. However, we can still see from the results of Figure 5 (or the response for Q6) that the correlation between clean NTK and robust accuracy is lower than that of the robust NTK. This implies that using clean NTK for robust NAS is worse than using robust NTK under adversarial training.

---

Q8: [In Table 2, the entries in the first column are not center-aligned.]

A8: We are thankful to the reviewer for the attentive reading. We fixed the vertical alignment issue in the revised version.

We thank the reviewer QErk for the insightful feedback. Below, we address the concerns pointed out by the reviewer QErk and revise the text in red in the updated version.

---

Q1: [Explanation for employing twice perturbation. Why robust NTK requires a double perturbation when conventional adversarial training typically examines generalization from a single perturbation.]

A1: We explain the usage of twice-robust NTK both theoretically and empirically.

• Theoretically, In the neural tangent kernel theory, we need to define a linearized neural network around the initialization of its weight. Specifically, in equation 15, we consider the linearized network $F_{\bf W^{[1]},\bf W^\star}(\bf x) := f(\bf x,\bf W^{[1]}) + \langle (1-\beta)\nabla f_{\bf W}(\bf x,\bf W^{[1]}) + \beta \nabla f_{\bf W}(\mathcal{A}^*_\rho(\bf x,\bf W^{[1]}),\bf W^{[1]}) , \bf W^\star - \bf W^{[1]} \rangle$. Then at the end of the proof of Theorem 1, we have $F_{\bf W^{[1]},\bf W^\star}(\mathcal{A}^*_\rho(\bf x_i,\bf W^{[1]}))= f(\mathcal{A}^*_\rho(\bf x_i,\bf W^{[1]}),\bf W^{[1]}) + \langle (1-\beta)\nabla f_{\bf W}(\mathcal{A}^*_\rho(\bf x_i,\bf W^{[1]}),\bf W^{[1]}) + \beta \nabla f_{\bf W}(\mathcal{A}^*_\rho(\mathcal{A}^*_\rho(\bf x_i,\bf W^{[1]}),\bf W^{[1]}),\bf W^{[1]}) , \bf W^\star - \bf W^{[1]} \rangle.$ As a result, the Jacobian is given by $\widetilde{\bf J}\_\mathrm{all} := (1-\beta)\hat{\bf J}\_\rho+\beta \hat{\bf J}\_{2\rho}$, where the twice-NTK appears.

• Empirically, we have demonstrated the superiority of twice-robust NTK when compared to the (single perturbation) robust NTK in Figure 5, which shows that the correlation between the accuracy ranking and twice-robust NTK is higher than the robust NTK.

We agree with the reviewer that the intrinsic motivation of twice perturbation is not as straightforward as classical adversarial training works in the single perturbation style. However, this twice perturbation approach shares a similar style with adding noise before the adversarial attack [5]. More precisely, [5] demonstrate that augmenting clean data with noise before executing adversarial attack has the benefit of avoiding catastrophic overfitting, compared to a classical adversarial attack on the clean data. The proposed Noise-FGSM achieves promising performance across several datasets (e..g, CIFAR-10, and SVHN) and mitigates robust overfitting. This demonstrates that introducing a stronger attack is beneficial to obtain better performance in practice. We believe that the development of algorithms based on the twice perturbation style will be a viable and beneficial avenue.

---

Q2: [Not standard evaluation raises questions about its normalization. Whether the metric is normalized before the attack and then inputted into the model. Whether the same perturbation radius (32x32) used for ImageNet was applied during training.]

A2: In this work, the data is normalized with zero-mean and unit-variance before the attack. We have added such clarification in the dataset paragraph in the revised version. We emphasize that we are using standard tools and evaluation metrics, as used also in several ML papers [4,11] and open-source robustness library [3] developed by MadryLab. Regarding the implementation detail, we employ the Torchattacks library [7]. Lastly, we will release the pre-trained weight of 6444* 3* 3 architectures to encourage the practitioners to evaluate with more metrics.

We emphasize the same perturbation radius $\rho = 8/255$ is used for ImageNet in the training procedure of the revised version.

---

Q3: [What 'optimal' refers to in Table 2.]

A3: Optimal refers to the architecture with the highest average robust accuracy among the benchmarks. We have updated the caption of Table 2 in the revised version.

---

Q4: [The definition of 'criterion' in the first column of Table 2.]

A4: The first column of Table 2 is referred to as the "attack scheme" and we updated it in the revised version to avoid misunderstanding. The "criterion" used in our previous version indicates how the accuracy is measured during the searching phase of these baseline methods. We have already updated it in the caption of Table 2 for a better understanding in our new version. We are thankful to the reviewer for the attentive study of our work.

We are grateful for the valuable feedback from the reviewer QErk. During the rebuttal, we have:

• Explained the twice robust NTK theoretically and empirically.

• Clarified the theoretical contribution of our work and its contrast against the prior work.

• Reported the correlation results when measured with clean NTK.

• Polished the writing of Table 2, clarified the evaluation metrics, and explained the implications of using clean NTK values for robust NAS.

We would appreciate it if the reviewer could provide feedback on our previous responses.

Best regards,

Authors