Carefully Blending Adversarial Training and Purification Improves Adversarial Robustness

We thank the Reviewer for his/her observations, noticing however that part of such review is based on what we believe to be a mischaracterisation of our paper’s goal and contents. We will address the Reviewer’s concerns in a similar list-based format.

1. Poor presentation.
   a) We will start by noticing that the portion of the paper in which no novel information is added with respect to existing published literature accounts for at most ~1/4th of the lines (14 to 43, 80 to 135, 285 to 288; i.e. 90 lines over 356 in total), and even less so in terms of page space. A figure far from the ‘first half’ mentioned by the Reviewer. Additionally, the Reviewer does not address whether, and for what specific reasons, the mentioned background plays an irrrelevant role in the overall structure of the paper. This prevents us from addressing the merit of the question.
   An analysis of existing methods in the fields of AT- and purification-based adversarial defences is given in Sections 1 and 2, and in part of Subsection 5.2. Specifically, some shortcomings of existing methods are addressed at lines 38 to 43, 98 and 99, 106 to 108, 113 and 114, 310 to 313. Our analysis is consistent with the goals of our paper, as outlined in the Abstract and the final paragraphs of Section 1: i.e. to propose a novel approach that achieves adversarial robustness thanks to the blending of AT and purification, and to assess its empirical robustness according to a standardised benchmark for $\ell_{\infty}$ perturbations, across some datasets. We do not believe that such goal would require to explicitly identify pitfalls in existing approaches, which serve indeed as a basis for our method, and punctual solutions to them. Owing to its architectural novelty as the main aspect of interest, we provide some justification to its inner workings (the ‘problems it can solve’) in Subsections 4.2, 4.3, 4.4, 4.5 and Appendices C and D. We also believe that the recorded empirical robustness, improving upon the existing state of the art, constitutes a definitive justification in spite of the (well acknowledged) clean accuracy decrease.
   b) Standing by the stance that the prescriptive meaning of modal verb ‘may’ is acceptable in such case, we can further clarify its meaning by operating the substitution: the encoder network may be discarded $\rightarrow$ the encoder network is discarded. Hardly believing that such single element is responsible for a large disruption of clarity across the paper, we are prevented from commenting upon other passages, as they have never been explicitly mentioned by the reviewer.
2. Insufficient experiments.
   We are unable to understand whether the Reviewer implicitly refers to the presence of other aspects of concern – besides later-mentioned points (a) and (b) – as the reason to believe our experiments to be ‘insufficient to prove the effectiveness of the proposed method’. We will comment on the points explicitly mentioned.
   a) We are sorry that the Reviewer finds Table 2 a source of reduction in clarity. We are aware of the presentation choice to compare our method to only the best-performing existing models in terms of empirical robust accuracy – which we find nonetheless adequate in the light of our declared goals (see e.g. the Abstract, Section 1, and the previous point (1) of the numbered list). Additionally, we would point out that the choice of assessing different models on the basis of the worst-case scenario against different adversarial attacks is a well-established custom of the field, and a main staple in the definition of the AutoAtack benchmark. Also, no averaged result is shown in Table 2 (as the Reviewer states instead) – apart from dataset-averaged accuracies. Finally, the clean accuracies of all methods used in the comparison are shown either in Table 2 or in Appendix F (Table 15). In an effort to improve clarity in the presentation of our results, the contents of Table 15 can be moved to Table 2.
   b) We believe that saying that ‘the paper does not give specific problems, only a general goal’ represents a mischaracterisation of our work and its goals. Indeed, we recall once more the main aim of our endeavour: to devise a novel technique for adversarial defence based upon the blending of adversarial training and purification – and the assessment of such method on a standardised benchmark (for $\ell_{\infty}$ perturbations on image classification) across some datasets. We believe – as all other Reviewers agree – that the novelty is the main aspect of interest in our proposal, together with its assessment that proves its efficacy. As such, we do not find it a weakness the fact that our method is not built as the response to specific shortcomings of other existing methods – apart from the inferior empirical robust accuracy that they finally provide in the experimental scope addressed, and their reliance on model approximation or surrogation to assess the end-to-end white-box adaptive robustness. Aware that a further broadening of the experimental scope will increase its justificatory strength, we are prevented to comment on specific aspects of the suggestion due to its non-specific nature.

We thank the Reviewer for the answer and additional clarifications about his/her position.

We also understand and respect that the Reviewer finds readability the main weakness of our work. Believing in the role of peer-review not just as a filter – but for the betterment of submitted works – we tried to address the specific issues raised in the original review in accordance to their nature:

• When referring to the conceptual structuring of the paper or the lack of descriptive contextualisation (i.e.: weaknesses 1.a, 2.b), we referenced the specific passages of the paper addressing those points. We also provided a more justificatory explanation of our choices, in the light of the overall goal of our paper and the need to balance those descriptions with the introduction of the (many) novel aspects of our method.
• When referring to phrasing or technical/typographic aspects of presentation (i.e.: weaknesses 1.b, 2.a), we tried to accommodate the Reviewer’s suggestions as much as possible – recognising the improvement in clarity the provide.

We refer to our already-submitted rebuttal for the actual discussion of the points just mentioned.

We also recognise that the Reviewer may have found more issues while reading the paper than those explicitly stated. While those latter may be ‘sufficient to demonstrate that the overall readability of the paper is weak’ (at least in its initial version) – such choice may inadvertently prevent us from better addressing those clarity concerns.

Finally, we thank the reviewer for his/her clear statement on paper acceptance, and for the willingness to actively engage in reviewer-reviewer and reviewer-AC discussion.

We thank the Reviewer for his/her constructive observations, and for the care put into writing the review. We will gladly comment upon all points raised, in a similar list-based format.

1. On the superiority of CARSO w.r.t. AT classifier baselines. With respect to the claim that CARSO surpasses in empirical robust accuracy the individually-considered adversarially-trained classifier models it employs – we ultimately refer to experimental evaluation whose results are contained in Table 2 (specifically: the comparison of columns AT/AA and C/rand-AA). Were the claim untrue – within the experimental scope considered – we would have observed a less or equal C/rand-AA robust accuracy in comparison to the one reported under AT/AA. Instead, the use of CARSO determines its marked increase, ranging from $+8.4\%$ (CIFAR-10) to $+27.47\%$ (TinyImageNet-200). Such observations alone would be sufficient to substantiate our claim.
   A possible explanation of such results may be found in the justification to the method provided in Subsection 4.2, and in the use of a robust aggregation strategy (described in Subsection 4.5 and Appendix D), that plain AT classifiers both lack.
   Specifically, adversarial attacks against any classifier target the distribution of logits contained in its last layer (whose $arg\ max$ usually constitutes the predicted class). Consequently, the concern of the reviewer about overall robust accuracy being limited by that of the classifier would have been justified in the case only such last-layer representation had been used as its whole internal representation of interest. Instead, the logits layer is not even used as part of the conditioning set of the purifier (see Table 5, Appendix E.2). The conditioning set chosen (i.e. the representations at intermediate layers of the classifier) does not even include proper class information, but just a collection of features that the decoder of the purifier learns to map to clean image reconstructions, under adversarial noise corruption against the classifier.
   In such setting, an adversary against CARSO would need to target the last-layer logits of the classifier (used to finally performs class assignment via robust aggregation) only by attacking multiple intermediate layers of the very same classifier, and through the decoder that uses them as input, in addition to the classifier itself. This may ultimately make CARSO a harder target to fool, in comparison to the classifier alone.
2. On the significant clean accuracy toll. It is true, and we transparently recognise (see Subsection 5.3), that the specific version of CARSO evaluated in our work (i.e. using a VAE as generative purifier) imposes a heftier clean accuracy toll in comparison to existing methods (either AT-based or using diffusion/score-based models as generative purifiers). Such results ultimately depend upon the deliberate choice to assess the feasibility of the idea behind CARSO (i.e. blending AT and purification via representation-conditional purification and robust aggregation), and the robustness it produces, in the best-possible scenario for the attacker. In such light, the VAE-based purifier and the chosen robust aggregation strategy ensure exact end-to-end differentiability for the whole model. This guarantees that the evaluation is not dependent on approximated backward models, which can only provide a robustness upper bound and are more susceptible to gradient obfuscation.
   As we mention in Subsection 5.3 and in the Conclusion, we are interested – and actively pursuing research - in different architectural choices for the purifier in CARSO and CARSO-like models, which may result in much more competitive clean accuracy, though making rigorous robustness evaluation more challenging with current tools.
   We thank the Reviewer for suggestions related to Table 2, and will definitely include in it the contents of Table 15, as a way to enhance transparency and clarity in the presentation of results.
3. On ablation studies and further experiments. We are aware that the paper provides little space to ablation studies or experimental settings different from empirical $\ell_{\infty}$ adversarial robustness evaluation. The additional assessment of $\ell_{2}$ robustness has been excluded at this stage due to the generally more demanding challenges offered by $\ell_{\infty}$; we recognise however the significant added value it may contribute to our work. We are also particularly interested – and pursuing active research – in how the choice of layers to be used as conditioning set influences overall robustness. As noted in Subsection 5.3 we are planning further work into such realm.

Answers to minor comments
We thank the reviewer for the precise remarks, that allow us to improve the clarity and legibility of the paper. In response to such observations, we have operated the following edits to our manuscript.

• In Subsection 4.5 and Appendix D (where the robust aggregation strategy is described), the class index is now referred to as $c$, leaving index $i$ to reconstructed sample multiplicity, as shown in Figure 1.
• The symbol $\mathcal{L}_{\text{VAE}}$ is now referred to in the caption of Figure 1 as VAE Loss and the reader explicitly redirected to Appendix B for a formal definition of it.

I appreciate the authors' clarification. In conclusion, I will maintain my rating of Weak Accept. Additionally, since the clarification addressed several unclear points, I am increasing my confidence score from 4 to 5. My detailed thoughts are as follows.

I believe that the CARSO proposed in this research presents a novel adversarial defense approach. Utilizing the internal representation of an adversarially robust model for conditioning a purifier is, in my opinion, a novel contribution beyond the trivial combination of adversarial training and purification. The idea of combining the two mainstream adversarial defense strategies—adversarial training and purification—and the resulting outcomes are likely to be of great interest to the community. Although there is a trade-off in clean accuracy, the achieved robust accuracy significantly surpasses the state-of-the-art, making it worthy of evaluation. Moreover, the end-to-end evaluation, investigation into gradient obfuscation, and use of PGD-EOT clearly address naturally arising questions from this approach (particularly the use of purifiers), which I found to be a solid evaluation.

As far as I am aware, the primary weaknesses of this research, as acknowledged by the authors, include the lack of certain experiments and the decrease in clean accuracy. For more details, please refer to my Weaknesses 2 and 3. However, regarding the former, I recognize that critical results demonstrating the method's effectiveness were sufficiently provided. There may also be areas for improvement in the presentation. While I did not find it difficult to understand the goal and concept of the study, I encountered some challenges in fully grasping the flow of the methodology. Revisiting the structure of Section 4 could enhance the quality of the paper.

Considering all these strengths and weaknesses, I will keep my rating.

P.S. In addition, considering the authors' rebuttal for my Weakness 1, I increased soundness from 2 to 3.

We thank the Reviewer for his/her observations.

Firstly, we would start by pointing out an inaccuracy in the summary of our paper. The tentative reconstructions of input images generated by the purifier are not aggregated by the classifier. Indeed, the classifier processes them independently from one another, outputting a distribution of logits for each. Such distributions are then aggregated by the robust aggregation strategy described in Subsection 4.5 (justified in Appendix D), which constitutes an integral part of our method and a core contribution to its robustness.

With respect to the weaknesses identified:

1. Weak experiments; additional baselines. We believe our experimental choices to be consistent with the goal of our paper: i.e., to show that it is possible to synergistically blend adversarial training and purification in a novel and meaningful way. The resulting model attains the state of the art in a hard adaptive benchmark across datasets, against the best AT- and purification-based defences. Remarkably, our evaluation relies on exact end-to-end differentiability (and not on best-effort approximation, such as BPDA), and explicitly accounts for the stochasticity of the method (by using randomness-aware AutoAttack, which relies on EoT). Our evaluation also guards against common pitfalls of adversarial defences in general (e.g. gradient obfuscation, see Table 3) and diffusion-based purification methods (such as the mentioned DiffPure, vulnerable to pitfalls identified in [4] and [5]).
   Referring to the specific models, [2] is directly surpassed by [5] (which reconsiders adversarial evaluation of diffusion-based models in the light of [4]), to which we directly compare. We thank the reviewer for the suggestion of [1] and [3], concerned with transformation-based defences and using BPDA to assess adaptive white-box accuracy (reasons for which they have been excluded from direct comparison in the first place). Indeed, this allows us to show once more the effectiveness of CARSO. Even with the advantage given by the use of attacks against approximated models, we surpass their reported best robust accuracy in commonly-employed datasets. More specifically:

   • Cascaded DISCO (k=5) [1] on CIFAR-10 attains a $\leq 0.5777$ RA, compared to our $0.7613$;
   • IRAD [3] on CIFAR-10 attains a $\leq 0.7432$ RA, compared to our $0.7613$;
   • IRAD [3] for CIFAR-100 attains a $\leq 0.6278$ RA, compared to our $0.6665$.

   If deemed appropriate, we will gladly add such comparisons to Tables 2 and 15, or in a dedicated appendix.

   As far as the further remarks are concerned, the vague and non-specific wording prevents us from addressing specific issues related to our method.

2. Combination of DISCO and AT. We are unaware of any published paper or experimental evidence that neither confirms nor disproves whether a hypothetical combination of DISCO and adversarial training would be effective, especially at the levels of robust accuracy attained by state-of-the-art models (including ours). We thank the reviewer for the suggestion, definitely worth investigating in the future. We must admit, however, that we do not consider such suggestion a weakness of our method, whose framing, formulation, and assessment are independent of it – and different in scope.

3. On the adversarial training of the classifier. As with point (2), we hardly see the Reviewer’s question as a weakness of our method. The purifier used as part of CARSO maps internal features of the classifier to image reconstructions. As such, under adversarial noise, the use of a classifier (and, as a consequence, its internal features) that are the most invariant under such perturbations stabilises and enhances the robustness of the purifier. Reciprocally, the use of a (same, in our case) classifier trained under noisy corruptions to finally classify purified reconstructions makes the overall process more robust also to non-adversarial artifacts the purification process may introduce. This, of course, comes at the cost of a decreased clean image accuracy, as we notice.

4. On direct image purification. As noticed in the case of points (2) and (3), once more, we do not believe the Reviewer’s question constitutes a weakness to our method. Firstly, Subsection 4.2 (lines 197 to 202) already provides a preliminary answer to such point. In detail, direct image purification (i.e. the mapping of a perturbed to a tentatively clean image) is already the leading scheme used in legacy (e.g. [6]) as well as modern (e.g. the already mentioned [2] and [5]) adversarial purification methods. In a broader sense, [1] and [3] may also be included in such class of techniques. In the case of [6] (which uses VAEs as purification models), the approach even resulted in worse robustness compared to the classifier alone. In all remaining cases, the experimental evidence we provide shows that none of such methods is able to provide a robust accuracy better that ours, within the experimental scope considered.

Finally, we want to address the heavy reliance on VAE training of our method. As noted in Subsection 4.1 (lines 154 to 156), the purifier being a VAE is an inessential part of CARSO as a general method: “any model capable of stochastic conditional data generation at inference time” would suffice. In addition, the use of a VAE ensures that the attacker has access to exact end-to-end model gradients for evaluation, increasing the strength of our experimental setup.

References
[1], [2], [3]: as mentioned by the Reviewer.
[4] Lee, Kim: ‘Robust Evaluation of Diffusion-Based Adversarial Purification’, 2024.
[5] Lin et al.: ‘Robust Diffusion Models for Adversarial Purification’, 2024.
[6] Gu, Rigazio: ‘Towards Deep Neural Network Architectures Robust to Adversarial Examples’, 2015.

• Specific blending approach. Relevant reasons against the simple juxtaposition of a VAE-based purifier and a classifier are contained in [2], where such arrangement results in decreased robustness. Mitigation against such pitfall is provided by the representation-conditional purification we devise, and the stochastic data generation offered by the VAE is turned into a robustness enhancement by the aggregation strategy we propose. Without performing an actual experiment – which is outside of the scope of our paper in its current form – it would be impossible to determine if DISCO+AT (or similar) methods are equally viable from an empirical viewpoint, nor whether they would be susceptible of known (or novel!) failure modes. For sure, the structure of DISCO prevents exact end-to-end algorithmic differentiability, and thus forces reliance on BPDA for proper attacks. As such, it offers a less demanding evaluation scenario, and an ineliminable robustness overestimation w.r.t. CARSO.

References
[1] Madry et al., Towards Deep Learning Models Resistant to Adversarial Attacks, 2018.
[2] Gu & Rigazio, Towards Deep Neural Network Architectures Robust to Adversarial Examples, 2015.

We thank the Reviewer for the clarifications about his/her review. We will address the issues further specified in a list-based format.

• Experiments with classifiers other than WideResNet-28-10. With respect to the classifier model – to be used as part of the CARSO architecture – we tried to strike a balance between a reasonably-performant pretrained AT model (representative of modern AT-based approaches) while keeping model size under control. Indeed, as larger adversarially-trained models practically always perform better in terms of clean and adversarial accuracy (keeping the training protocol fixed), the use of smaller classifiers within CARSO would increase the strength of experimental results. This is indeed the case, as we achieve better-than-SotA robust accuracy across our whole experimental scope.
  To support the significance of such choice in the context of AT, the following statistics can be gathered from the RobustBench entries for $\ell_{\infty}$ robustness (commit 776bc95bb4167827fb102a32ac5aea62e46cfaab):
  • CIFAR-10: $22\\%$ are WideResNet-28-10s; $54\\%$ are deeper and wider (Wide)ResNets; $6\\%$ are deeper (but narrower) ResNets, with an overall larger number of parameters; $4\\%$ are transformer-based models with a larger number of parameters; $14\\%$ are (PreAct)ResNet-18s.
  • CIFAR-100: $14\\%$ are WRN-28-10s; $65\\%$ are deeper and wider (W)RNs; $8\\%$ are transformer-based models with a larger number of parameters; $13\\%$ are (PA)RN-18s.
• Impact of adversarially-balanced batches. Though the impact of adversarially-balanced batches (ABBs) has not been thoroughly explored, a heuristic justification for its use, within the training of CARSO, is provided in Appendix C. With respect to the use of ABBs in the adversarial training of classification models that use AT as the only technique for robustness enhancement, we refer to [1] whose crucial aspects w.r.t. AT are reported in Appendix A. The requirement of worst-case perturbations in the inner optimisation step theoretically discourages the use of non-worst-case examples (as would be the case of FGSM-generated examples) or non-entirely-perturbed batches. As such, usual PGD-based adversarial training – and derived techniques – remain the theoretically-recommended way to achieve robustness in the end-to-end training from scratch of classifier models. Since we train only the purifier part of CARSO with ABBs, such considerations do not apply in our case.
• Fairness of comparison with the best AT method. Given the analysis previously provided, and the inclusion of a direct comparison between CARSO-based models and their classifier models alone, we believe the additional comparison of CARSO-based models with the currently best-performing AT-based techniques to be fair within the experimental scope of interest. Especially so, given that the best-performing of such models have a larger number of trainable parameters – and an even larger one in the case of purification-based defences – w.r.t. CARSO.
• Positioning of CARSO and related experiments. In the design of experiments and their presentation within our work, we focused mainly on the specific measurement of empirical robust accuracy as the means of comparison with other existing approaches. While it is true that those particular experiments do not provide a clear-cut pros/cons analysis, the paper outlines some issues with existing AT- and purification-based approaches (Section 1, 2, and Subsection 5.2), and the differences between them and CARSO (Subsections 4.2, 4.3, 4.4, 4.5 and Appendices C and D).
• Goal of our work. We believe that the most precise description of the goals of our work is contained in the Abstract, Section 1 (the Introduction) and Section 6 (the Conclusion) when considered altogether. Specifically, the direct quote from our rebuttal, i.e. ‘to show that it is possible to synergistically blend adversarial training and purification in a novel and meaningful way’ is indeed one of our goals (as stated e.g. in the penultimate paragraph of the Introduction) – yet, it is hardly the only one our work achieves. Indeed, we show that not only our approach is viable, but we also do so in a deliberately hard scenario: in terms of $\ell$ norm-bound, attack choice, and requirements of end-to-end differentiability (which in turn allow for approximation-free assessment). Yet, we are able to attain the state-of-the-art in one of the most stringent robustness benchmarks available (randomness-aware AutoAttack) – against any kind of existing models, even developed well outside the compliance to our requirements.

We thank the Reviewer for his/her prompt response, the willingness to engage in further discussion, and the interest in our paper.

As for the specific contents of the Reviewer’s latest comment, we will address them in a similar list-based format, as usual.

1. We will provide split answers in the sub-list that follows, according to the specific conceptual issues raised.
   • Broadening of the experimental scope. We agree that any broadening of the experimental scope would constitute an improvement to our paper, as it would be for any work of science. As far as a broadening in model variety is concerned – as the reviewer mentions – we will refer to our previous comment (part 1). As we already shown – using the RobustBench accepted submissions as a representative leaderboard for adversarial training – our specific model choice for the classifier (i.e. a WideResNet-28-10), which is used internally to CARSO, is shared by $96\\%$ of CIFAR-10 and $92\\%$ CIFAR-100 overall entries in terms of architecture (ResNet or derived), and directly comparable with similar or larger models to $76\\%$ of CIFAR-10 and $79\\%$ CIFAR-100 entries in terms of both architecture and model size lower bound. Yet, against the whole set of models (including those not directly comparable to ours, due to architectural differences), we manage to obtain superior performance in terms of empirical $\ell_{\infty}$ robust accuracy.
   • Ablation study. While we did not present it in the paper by the term ablation study, we conduct one crucial of such experiments as a way to assess – as the reviewer says – the effectiveness of our method, and, on a lesser measure, the trade-offs of our approach. Indeed, in Table 2, we directly compare the accuracy of existing AT-trained models developed for the goal of adversarial robustness, with CARSO models using the very same AT-trained classifiers (up to weight values). We do so in terms of both clean (columns AT/Cl vs C/Cl) and robust (columns AT/AA vs C/rand-AA) accuracy, across three different datasets. As such, the comparison shows the direct effect of ablating away the entire additional structure we propose, whose results we lengthily commented upon: in brief, a marked increase in robust accuracy accompanied by a decrease in clean accuracy.
   • Higher resolution datasets. Acknowledging that this point is entirely novel w.r.t. to previous review and post-rebuttal comment of the Reviewer, we agree – as we already said – that any broadening of the experimental scope would be beneficial to our work. We also believe, however, that – since we introduce our method as an entirely original one – the existing amount of evidence we provide about its effectiveness cannot be simply dismissed on the basis of dataset resolution being $\leq 64\times64$ pixels.
   • On the sufficiency of the goal of our paper. As far as the later statements by the Reviewer are concerned – as we also already said – we agree that merely showing that our approach is viable – and even doing so in a deliberately hard scenario – are not entirely sufficient. However, for some reason, the Reviewer fails to acknowledge the next part of the quoted sentence: we are able, with our method, to significantly improve upon the empirical adversarial robustness of any existing model pursuing the same goal – which has been tested according to AutoAttack and whose results made public by its Authors. While we still believe that further goals are yet to be achieved by our paper and by our models, we honestly do not consider those results as just the ‘most basic and necessary to support [our] goal’.
2. On the improvement of the ‘expressions used’ and the ‘quality of figures’. With all due respect, we are quite surprised by the Reviewer’s observations in this regard. Not because we do not believe our paper can be affected by those issues, but due to the fact that the Reviewer voices these entirely new concerns so later on in the review and discussion period. Given the very specific and technical nature of problems such as the choice of expressions, or the quality of pictographic content, knowing those specific terms and/or aspects of poor quality in figures (which is only one, i.e. Figure 1) in greater advance would have definitely allowed for pin-point interventions in the paper before the end of such discussion phase. Especially so in the case of pictures, which we could have submitted -- according to the rules of the Conference -- before the end of the rebuttal period.
   We lack the specifics required to further comment upon the issues identified by the Reviewer.