Thank you so much for your time and efforts spent on reviewing our paper! Your thorough main review and comments are very important to the improvement of our work! Please find our replies below.

Q7: Instead of comparing various CAM techniques, could the authors consider comparing the PART framework with a broader range of AT methods that utilize CAM technology, thereby offering a more comprehensive comparison of its effectiveness?

A7: Thanks for your suggestion! To the best of our knowledge, we can find two adversarial defenses that incorporate CAM technologies. We have cited them in our related work (see Appendix C in the original submission for more details).

Specifically, [1] proposes to use class activation features to remove adversarial noise. This method can be regarded as an adversarial purification method, which purifies adversarial examples towards natural examples. Our method is fundamentally different from [1] because that PART is an AT framework. Therefore, we cannot compare our method to [1] due to the different problem settings.

On the other hand, [2] proposes a framework that uses GradCAM to rectify and preserve the visual attention area, which aims to improve the robustness against adversarial attacks by aligning the visual attention area between adversarial and original images. This method is an AT method and thus it is fair to compare our method to [2]. However, we cannot find any source code of [2]. Therefore, it may take some time for us to realize their code. We will update the results once we finish the comparison.

[1] Dawei Zhou, Nannan Wang, Chunlei Peng, Xinbo Gao, Xiaoyu Wang, Jun Yu, and Tongliang Liu. Removing adversarial noise in class activation feature space. In ICCV, 2021.

[2] Shangxi Wu, Jitao Sang, Kaiyuan Xu, Jiaming Zhang, and Jian Yu. Attention, please! adversarial defense via activation rectification and preservation. ACM Trans. Multim. Comput. Commun. Appl., 2023.

Q8. White-box attack defense results are discussed, but the impact of the number of attack iterations on the performance of CAM technology isn't addressed. Can the authors provide insights into how the number of attack iterations influences the effectiveness of CAM technology in adversarial defense?

A8: There might be a misunderstanding. In Section 4.2 'Defending against adaptive attacks', we have compared our method to the baseline AT methods (i.e., AT, TRADES and MART) against adaptive PGD-20/40/60/80/100 (see Table 2 in the original submission for more details). From the experiment results, with the increase of attack iterations, the robustness of defense methods will decrease. This is because the possibility of finding worst-case examples will increase with more attack iterations. However, the effectiveness of CAM technology itself is rarely influenced by attack iterations, as we show that our method can consistently outperform baseline methods. Without losing generality, we conduct an additional experiment against normal PGD-10/40/60/80/100 (see Table 5). Again, we can obtain the same conclusions from the results. We have included Table 5 in Appendix I in the updated version of our manuscript.

Table 5: Robustness (%) of defense methods against PGD with different iterations on CIFAR-10. We report the averaged results and standard deviations of three runs. We show the most successful defense in bold.

Thank you so much for your time and efforts spent on reviewing our paper! Your thorough main review and comments are very important to the improvement of our work! Please find our replies below.

Q1. Regarding the use of CAM technology, can the authors provide a more detailed analysis of its impact on training speed and memory consumption when integrated with the original AT framework?

A1: Incorporating CAM technologies will naturally lead to an increase in training speed. How to effectively integrate these technologies into the training process presents an intriguing challenge. If we generate the mask m for each epoch, the computational cost will be considerably high. To address this problem, we update the mask m every 10 epochs. We show that the performance of our method remains competitive (see Table 1) given the mask is updated for every 10 epochs. Regarding memory consumption, the majority of the memory is allocated for storing checkpoints, with only a small portion attributed to CAM technology. In general, the additional costs (i.e., training speed and memory consumption) are affordable (see Table 2 and 3). This information has been included in the updated version of our manuscript. For more details, please refer to Appendix M in the updated version.

Table 1: Robustness (%) of defense methods on CIFAR-10. The target model is ResNet-18. We report the averaged results and standard deviations of three runs. We show the most successful defense in bold.

Table 2: Training speed (hours:minutes:seconds) of defense methods on CIFAR-10. The target model is ResNet-18.

Table 3: Memory consumption (MB) of defense methods on CIFAR-10. The target model is ResNet-18.

Q2. Are the performance gains achieved through CAM technology worth the potential trade-offs?

A2: From Table 1 and 3, we believe the extra cost is marginal compared to the performance gains. Besides, it is reasonable that our method gains marginal improvement in robustness, as PART decreases the attack strength for unimportant pixel regions. What we want to highlight is that PART can improve the robustness-accuracy trade-off by notably increasing the natural accuracy. Despite a marginal reduction in robustness by 0.04% on PGD-20, PART gains more on natural accuracy (e.g., 2.08% on SVHN and 1.36% on Tiny-ImageNet). In most cases, PART can improve natural accuracy and robustness simultaneously. Compared to TRADES and MART, our method can still boost natural accuracy (e.g., 1.20% on CIFAR-10, 2.64% on SVHN and 1.26% on Tiny-ImageNet for PART-T, and 1.85% on CIFAR-10, 1.66% on SVHN and 1.07% on Tiny-ImageNet) with at most a 0.10% drop in robustness. It is widely acknowledged that AT will hurt natural accuracy compared to standard training. Our work can bridge this gap without compromising robustness. As demonstrated in [1], advancements in the field of AT have been incremental over the past several years. As a result, making every slight improvement in this area is noteworthy.

[1] Rahul Rade and Seyed-Mohsen Moosavi-Dezfooli. Reducing excessive margin to achieve a better accuracy vs. robustness trade-off. In ICLR, 2022.

Thank you so much for your time and efforts spent on reviewing our paper! Your thorough main review and comments are very important to the improvement of our work! Please find our replies below.

Q3. The paper mentions combining standard AT for warm-up during training. Could the authors specify the number of training sessions required for both AT and PRAG in this process?

A3: The warm-up epoch number for PART is 20. Specifically, we use AT to train the model for the first 20 epochs and then use PART for the remaining 60 epochs. For fair comparisons, all the baseline methods (i.e., AT, TRADES and MART) are trained for 80 epochs. We have provided a detailed experiment setting that includes this information in Appendix F in the original submission.

Q4. How does this affect the training performance?

A4: Without the warm-up, the classifier is not properly learned initially, and thus may badly identify pixel regions that are important to the model's output in the early epochs. Empirically, we add extra experiments to compare the performance of PART on CIFAR-10 with/without warm-up (see Table 4).

Table 4: Comparison of the performance of PART on CIFAR-10 with/without warm-up. The target model is ResNet-18. The number of warm-up epochs is 20. We report the averaged results and standard deviations of three runs. We show the most successful defense in bold.

Q5. CAM technology appears to be a visualization technique primarily applicable to classification problems. How can the authors address concerns about the limited applicability of the PART framework in domains beyond classification? Are there plans to extend its use to other areas?

A5: Thanks for your insightful question. You are right, CAM technology is initially designed for classification problems. However, one important thing we want to highlight is that PART is a general idea rather than a specific method. Specifically, PART is a general idea that we should count the fundamental discrepancies of pixel regions across images. CAM technology, however, is only a tool to implement this idea. If there is a tool that can be extended to other areas, we can integrate it into our framework. The key is that the idea itself can be applied in other domains beyond classification, which is more important than how we implement it. Similarly, most AT methods are designed to solve classification problems, but their ideas can be extended to other areas.

Q6. The clarity of results presentation is a concern. Could the authors provide more information about how standard deviations were calculated and specify the number of runs used in this calculation?

A6: Thanks for your suggestion! We apologize that we missed stating the number of runs used in mean and standard deviation calculations. We follow [1] and report the averaged results and standard deviations of 3 runs through the whole paper. We have specified this information in the updated version of our manuscript. In our original submission, we did not report the standard deviations in Table 2 because of the limited page margin. We have fixed this issue by shrinking the font size to ‘footnotesize’, which also makes our tables more compact than before.

[1] Rahul Rade and Seyed-Mohsen Moosavi-Dezfooli. Reducing excessive margin to achieve a better accuracy vs. robustness trade-off. In ICLR, 2022.

Dear Reviewer h86N,

We appreciate the time and effort that you have dedicated to reviewing our manuscript. Just a quick reminder that discussion stage 1 is closing soon.

Have our responses addressed your major concerns?

If there is anything unclear, we will address it further. We look forward to your feedback.

Best,

Authors of Paper1033

Thank you so much for your positive comments and suggestions! It is our pleasure that our theoretical and empirical analysis can be recognized. Your thorough main review and comments are very important to the improvement of our work! Please find our replies below.

Q1. The idea is not completely novel. Adversarial attacks in combination with class activation mappings have for example been discussed in [1]. However, the authors use it for robustifying their models which is in my opinion sufficiently different. Nonetheless, authors should include a citation of that work. [1] Dong, Xiaoyi, et al. "Robust superpixel-guided attentional adversarial attack." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020.

A1: Thanks for your suggestion! We have cited this paper in our related work in Appendix C in the updated version of our manuscript (highlighted in blue). Here is the added content:

Adversarial attacks with class activation mapping. Dong et al. (2020) proposes an attack method that leverages superpixel segmentation and class activation mapping to focus on regions of an image that are most influential in classification decisions. It highlights the importance of considering perceptual features and classification-relevant regions in crafting effective AEs.

Q2: The empirical evaluation can be extended by using different adversarial attacks, e.g., Carlini-Wagner attack or AutoAttack.

A2: Thanks for your suggestion! However, there might be a misunderstanding. For general attacks, we have evaluated our methods and baselines using AutoAttack (see Table 1 in the original submission). For adaptive attacks, we conduct an additional experiment to test the robustness of defense methods against adaptive MMA (see Table 1 below). The choice of MMA over AA for adaptive attacks is due to AA's time-consuming nature as an ensemble of multiple attacks. Incorporating the CAM method into AA would further slow the process. MMA, in contrast, offers greater time efficiency and comparable performance to AA. We have added Table 1 below in Appendix J, please refer to the updated version of our manuscript.

Table 1: Robustness (%) of defense methods against adaptive MMA on CIFAR-10. We report the averaged results and standard deviations of three runs. We show the most successful defense in bold.

Q3. The literature review seems somewhat short. I suggest authors spend more time looking for relevant related works.

A3: Due to the limited space, the literature review in the main body of the paper is short. However, we have provided a detailed literature review in Appendix C, which covers the literature review of different AT methods, CAM methods, adversarial defenses with CAM and adversarial attacks with CAM. We have also included the discussion regarding the literature suggested in your Q1. Please check the full literature review in the updated version of our manuscript.

Thank you so much for your positive comments and suggestions! It is our pleasure that our theoretical and empirical analysis can be recognized. Your thorough main review and comments are very important to the improvement of our work! Please find our replies below.

Q4. Performence of the PART method is somewhat underwhelming. The improvement is only incremental (usually only in the range of ~1%).

A4: It is reasonable that our method gains marginal improvement in robustness, as PART decreases the attack strength for unimportant pixel regions. What we want to highlight is that PART can improve the robustness-accuracy trade-off by notably increasing the natural accuracy. Despite a marginal reduction in robustness by 0.04% on PGD-20, PART gains more on natural accuracy (e.g., 2.08% on SVHN and 1.36% on Tiny-ImageNet). In most cases, PART can improve natural accuracy and robustness simultaneously. Compared to TRADES and MART, our method can still boost natural accuracy (e.g., 1.20% on CIFAR-10, 2.64% on SVHN and 1.26% on Tiny-ImageNet for PART-T, and 1.85% on CIFAR-10, 1.66% on SVHN and 1.07% on Tiny-ImageNet) with at most a 0.10% drop in robustness. It is widely acknowledged that AT will hurt natural accuracy compared to standard training. Our work can bridge this gap without compromising robustness. As demonstrated in [1], advancements in the field of AT have been incremental over the past several years. As a result, making every slight improvement in this area is noteworthy.

[1] Rahul Rade and Seyed-Mohsen Moosavi-Dezfooli. Reducing excessive margin to achieve a better accuracy vs. robustness trade-off. In ICLR, 2022.

Q5. Section 3.1 “AE generation process.” is tough to read. Authors should work on the presentation of that section. Maybe a small table on the side would help to introduce the notation.

A5: Thanks for your suggestion! We have uploaded a reference list of the notations used in Section 3.1 in Appendix D. Please check the updated version of our manuscript. For your convienience, we also show the notation table below.

Q6. Figure 4: Authors should mention what is indicated by the shaded areas.

A6: Thanks for your suggestion! We apologize that we missed stating the meaning of the shaded areas. The shaded areas represent the standard deviation. We have included this information in the updated version of our manuscript.

Q7. Minor details (e.g., missing whitespace “Table 2: Robustness(%) of…” , Eq. 12-15 “subject to” should not be typeset in math mode and Lemma 1: “(i).” unusual period).

A7: Thank you for pointing out these minor issues in our paper! We have fix these issues in the updated version of our manuscript.

Q8. Figure 3: What should the lock symbol next to the first CNN tell me?

A8: The lock symbol means the parameters of the model are fixed. This is because during the generation of AE, the parameters of the model are unchanged. We have explained the meaning of the lock symbol in the legend of the figure in our updated version of manuscript.

Q9. Figure 3: In what sense are the activation maps of that CNN “global” ?

A9: We apologize for the misuse the word 'global'. The feature maps generated by CAM methods are actually a form of visual explanation, showing which parts of the input image the network focuses on when making decisions. These feature maps usually concentrate more on local regions (i.e., the areas of the image most crucial for the classification decision) rather than the entire image. Therefore, referring to the feature maps extracted by CAM methods as 'global feature maps' might not be accurate. To avoid confusion, we have removed the word 'global' in the updated version of our manuscript.

Dear Reviewer tC3M,

We appreciate the time and effort that you have dedicated to reviewing our manuscript. Just a quick reminder that discussion stage 1 is closing soon.

Have our responses addressed your major concerns?

If there is anything unclear, we will address it further. We look forward to your feedback.

Best,

Authors of Paper1033

Thank you so much for your valuable comments and questions! Please find our replies below.

Q1. CAM may not be scalable which means it may lose the ability to identify the semantic meaningful regions.

A1: Based on our experiment results on Tiny-ImageNet, we believe CAM can be scaled to large datasets. The scalability of CAM methods to large datasets primarily depends on the scalability of the model it integrates with. CAM itself is a visualization technique used to highlight the regions of an image that CNNs focus on when making decisions. If the network itself can effectively process large datasets (e.g., having sufficient learning capacity and appropriate computational efficiency), then CAM can also be applied to these datasets. Due to the high computational cost of AT, conducting experiments on larger datasets such as ImageNet would require significant resources. Typically, Tiny-ImageNet, a scaled-down version of ImageNet, is already a relatively large dataset used in the field of AT. When applying our method to a very large dataset such as ImageNet, we can scale down the size of images to fit the capacity of our network, and thus CAM methods will not lose the ability to identify the semantic meaningful regions.

Q2. The author currently did not analyze the computation cost and training time cost of the proposed PART compared to AT and standard training.

A2: Incorporating CAM technologies will naturally lead to an increase in training speed. How to effectively integrate these technologies into the training process presents an intriguing challenge. If we generate the mask m for each epoch, the computational cost will be considerably high. To address this problem, we update the mask m every 10 epochs. We show that the performance of our method remains competitive (see Table 1) given the mask is updated for every 10 epochs. Regarding memory consumption, the majority of the memory is allocated for storing checkpoints, with only a small portion attributed to CAM technology. In general, the additional costs (i.e., training speed and memory consumption) are affordable (see Table 2 and 3). This information has been included in the updated version of our manuscript. For more details, please refer to Appendix M in the updated version.

Table 1: Robustness (%) of defense methods on CIFAR-10. The target model is ResNet-18. We report the averaged results and standard deviations of three runs. We show the most successful defense in bold.

Table 2: Training speed (hours:minutes:seconds) of defense methods on CIFAR-10. The target model is ResNet-18.

Table 3: Memory consumption (MB) of defense methods on CIFAR-10. The target model is ResNet-18.

Q3. I would be curious if the proposed method applied into larger dataset which also obtains improvements.

A3: On Tiny-ImageNet, our method can consistently improve the performance compared to baseline methods. If the 'larger' here refers to the number of categories, Tiny-ImageNet with 200 categories can be considered a relatively large dataset. If the 'larger' here refers to the image dimensions, we can scale down the image size to fit the model capacity. As mentioned in Q1, the scalability of CAM methods to large datasets primarily depends on the scalability of the model it integrates with. In terms of the computation cost and memory consumption, we have provided a detailed comparison in Q2. In general, based on our experiment results on Tiny-ImageNet, we believe CAM can be scaled to large datasets.

Dear Reviewer u6mu,

We appreciate the time and effort that you have dedicated to reviewing our manuscript. Just a quick reminder that discussion stage 1 is closing soon.

Have our responses addressed your major concerns?

If there is anything unclear, we will address it further. We look forward to your feedback.

Best,

Authors of Paper1033

Dear Reviewer u6mu,

We appreciate the time and effort that you have dedicated to reviewing our manuscript. Just a quick reminder that discussion stage 1 is closing soon.

Have our responses addressed your major concerns?

If there is anything unclear, we will address it further.

Your feedback is very important to our work. We look forward to your feedback.

Best,

Authors of Paper1033

Dear Reviewer u6mu,

We appreciate the time and effort that you have dedicated to reviewing our manuscript. Just a quick reminder that discussion stage 1 is closing soon.

Have our responses addressed your major concerns?

If there is anything unclear, we will address it further.

Your feedback is very important to our work. We look forward to your feedback.

Best,

Authors of Paper1033

Thank you so much for your valuable suggestions and intetersting questions! Please find our replies below.

Q3. I think this paper is interesting but I am on fence of the contribution. Are all pixels equally important in an image? My gut feeling says that the answer is No, and perhaps it's a bit of a tautology (this seems quite obvious).

A3: Thanks for thinking our paper is interesting. However, in our humble opinions, it is not very fair to say our contribution is limited just because this paper has a straightforward intuition. Instead, we treat this as a strength of our paper and we would like to elaborate on this point. While the intuition behind our paper might appear straightforward, its simplicity should not be mistaken for a lack of depth or contribution. We have validated this intuitive idea from both empirical and theoretical standpoints. Our work not only demonstrates the practical applicability of our intuition but also provides theoretical insights that reinforces its validity. We want to emphsize that simple intuitions can also lead to significant advancements, as long as they are well investigated and validated.

Q4. Would it not be possible that performing adversarial training end-to-end with PGD-based type image diets automatically help a neural network find these critical parts in the image from which to then perturb the image at training? All-in-all, my question is: is PART really useful when there are many other adversarial training regimes that go beyond FGSM and that implicitly incorporate the image structure in the adversarial optimization?

A4: It is possible. PGD-based attacks aim to find the minimum perturbation that can fool the neural network. This process often highlights the most sensitive and critical features of the image that the network relies on for making decisions. However, the problem is that we find these critical features learnt by PGD-based AT are not sufficient. This is also one of the key contributions of our paper. We find that without explicit guidance, it is hard for AT methods to sufficiently mine robust features. The advantage of PART, on the other hand, is the ability to provide external guidance to help models better extract features that are beneficial to robust classification.

Q5. Perhaps another interesting question that I would have liked authors answer is. How would PART work with training on other image distributions such as Textures or Scenes? Would PART be equally useful or will it only apply to objects?

A5: Thanks for your insightful question! The applicability of PART primarily depends on the applicability of the model it integrates with. PART is a general idea rather than a specific method and CAM technology is only a tool to implement this idea. As long as the network itself can effectively work with Textures or Scenes (i.e., can extract meaningful classification patterns), then PART is able to obtain high-contribution pixel regions which means PART will still be useful on these image distributions.

Thank you so much for your valuable suggestions and intetersting questions! Please find our replies below.

Q1. Authors show how PART is better than many other adversarial training pipelines but the increase is very incremental. Should the paper be accepted because of this last point? I am not entirely convinced.

It is reasonable that our method gains marginal improvement in robustness, as PART decreases the attack strength for unimportant pixel regions. What we want to highlight is that PART can improve the robustness-accuracy trade-off by notably increasing the natural accuracy. Despite a marginal reduction in robustness by 0.04% on PGD-20, PART gains more on natural accuracy (e.g., 2.08% on SVHN and 1.36% on Tiny-ImageNet). In most cases, PART can improve natural accuracy and robustness simultaneously. Compared to TRADES and MART, our method can still boost natural accuracy (e.g., 1.20% on CIFAR-10, 2.64% on SVHN and 1.26% on Tiny-ImageNet for PART-T, and 1.85% on CIFAR-10, 1.66% on SVHN and 1.07% on Tiny-ImageNet) with at most a 0.10% drop in robustness. It is widely acknowledged that AT will hurt natural accuracy compared to standard training. Our work can bridge this gap without compromising robustness. As demonstrated in [1], advancements in the field of AT have been incremental over the past several years. As a result, making every slight improvement in this area is noteworthy.

[1] Rahul Rade and Seyed-Mohsen Moosavi-Dezfooli. Reducing excessive margin to achieve a better accuracy vs. robustness trade-off. In ICLR, 2022.

Q2. The paper says towards the end that this framework in more aligned to human perception. I don't think this is true from what has been shown in the paper. I would have liked to see qualitative samples and attack comparing how PGD performs on the same network trained differently (without AT, and with AT either classical or PART based), and from there run a psychophysical experiment with human observers to see if indeed they are fooled more by the PART-based model. While running the psychophysical experiments may not be possible, even adding the resulting adversarial images from networks trained with PART would be a great addition to the paper. I'd really recommend plotting the adversarial images for networks trained on PART similar to how this was done in Santurkar et al. (NeurIPS 2019), Berrios & Deza (ArXiv 2022) and Gaziv et al. (NeurIPS 2023).

A2: Thanks for your great recommendation and thanks for bringing these insightful papers to us, but there might be a misunderstanding. One thing we would like to point out is that 'more aligned to human perception' does not mean that AEs generated by PART-based methods can fool more human observers. As long as the adversarial noise itself is bounded by a small $\epsilon$ (i.e., imperceptible to human eyes), every adversarial attack can fool human observers. In our paper, 'the alignment to human perception' refers to how well the model's attention aligns with the object (i.e., semantic meaning) in an image. We find that the previously stated sentence may cause unnecessary confusion. Thus, we have changed 'human-aligned information' into 'semantic information' in the updated version of our manuscript.

We agree with your comment that adding the resulting adversarial images from networks trained with PART would be a great addition to the paper. In our original submission, we have provided examples of the high-contribution pixel regions learnt by the same network trained differently (e.g., without AT, with AT and with PART-based) in Appendix N. In Figure 6, it is clear that the network trained without AT tends to focus on the background of the image, and the network trained with PART is more precisely aligned with the object than the network trained with AT.

To further support our statements, we conduct additional experiments to visualize how the high-contribution pixel regions change every 10 epochs from the 30th epoch to the 60th epoch (see Figures 7 and 8 in the updated version of our manuscript). With the increase of epochs, PART-based methods are more precisely aligned to the object (i.e., semantic meaning) in an image compared to baseline AT methods.

Dear Reviewer QPEP,

We appreciate the time and effort that you have dedicated to reviewing our manuscript. Just a quick reminder that discussion stage 1 is closing soon.

Have our responses addressed your major concerns?

If there is anything unclear, we will address it further. We look forward to your feedback.

Best,

Authors of Paper1033