CosPGD: an efficient white-box adversarial attack for pixel-wise prediction tasks

Dear Reviewer E2Ss,

Thank you very much for your insightful review. We hope to answer your questions in a satisfactory manner.

1. The proposed CosPGD is motivated by SegPGD and indeed the equations might look similar at first glance. Yet, there are significant conceptual differences that provide a strong benefit of CosPGD over SegPGD, which we also discuss on page 3 in our submission. Please refer to our general reply for details on the conceptual differences between the two and the resulting benefit of CosPGD. Please note, that no other existing attack scales the loss pixel-wise using similarity between the posterior and target distributions. All attacks focus on final model predictions. This makes CosPGD significantly novel. Does this answer this important concern?

2. For image restoration, SegPGD can not be directly applied but by defining a threshold to determine from when on a prediction is considered to be correct. For the comparison on image restoration, we assume that the prediction should be as correct as possible, i.e. assume the lowest numerically possible threshold. After this adaptation, SegPGD’s performance comes close to the performance of CosPGD but only after 20 attack iterations. While CosPGD is significantly better for lower numbers of iterations. SegPGD nearly closing the gap in performance, despite taking 20 attack iterations, speaks to the credit of needing pixel-wise scaled adversarial attacks.

For the task of image restoration, 20 attack iterations take a significantly long amount of time to evaluate. Thus we propose CosPGD, an efficient attack which as discussed in Section 4.3 can be very useful in predicting a new model’s robustness efficiently. Additionally, we would like to request the reviewer to reconsider the following snippet for Semantic segmentation from Table 4 Section B.3 in the supplementary material. Here we observe, that despite the numerical difference in performance at 20 iterations being low, the maximum difference being 3.19%, its significance is very high. CosPGD is bringing down the performance of multiple powerful segmentation models to almost 0% in just 20 attack iterations.

Please note as well that the remaining prediction quality after SegPGD and CosPGD are both very low for 20 attack iterations, and CosPGD is still consistently better.

3. SegPGD can not be used on Optical Flow for the reasons detailed in Answer 1 unless the method is modified by defining a threshold. Similar to the results we provide for SegPGD for restoration, we can also report for optical flow as detailed below:

Table 6 in Section C.1 in the appendix of the revised version (Table 4 in original submission). A Lower(↓) epe wrt to Target and a Higher( ↑) epe wrt to Initial signifies a stronger attack.

However, please note again that SegPGD was not conceived for optical flow attacks and the low performance is to be expected since SegPGD is not designed for regression tasks.

4. CosPGD is stronger when few iterations are considered and therefore provides reliable results at a lower compute budget. The purpose of adversarial attacks is to reveal model weaknesses with as few iterations as possible. Therefore, we understand that an attack that outperforms others when run for only a few attack iterations is particularly valuable. In particular, when considering adversarial training, high attack iterations are extremely expensive and 3 attack iterations would be a typical value.

We hope we were able to answer all your questions to your satisfaction. Please let us know if you have further questions or concerns.

Best Regards

Authors of Paper #909

Dear Reviewer E2Ss,

We were curious if you happened to find the time to read our official comments and incorporated changes in the revised version. If so, we would be glad to answer any further questions or doubts you might have.

Best Regards,

Authors of Paper #909

Dear Reviewer E2Ss,

Thank you for acknowledging the strength of our work in terms of experiments and analysis. We would appreciate it if you would consider the following,

Both SegPGD and CosPGD propose to modify the update step of the original PGD applied to each pixel, as given in our equation (1).

$sign\nabla_{\boldsymbol{X}}L$

Here, we write L to denote the original model loss with respect to the current model $f$'s prediction based on an adversarial sample $\boldsymbol X$

and the one-hot encoded ground truth $Y$.

For segmentation, i.e. in SegPGD $L$ is a cross-entropy loss as specified in their equation (2) . In our paper, we also use a cross entropy loss when considering segmentation, but $L$ can in principle be any loss of the respective original model, as in PGD. For optical flow, we use an optical flow loss, to be consistent with PGD.

In SegPGD, the above update is modified to

$sign\nabla_{\boldsymbol{X}}(\frac{1-\lambda}{N}\sum_{i\in P^T} L_i + \frac{\lambda}{N}\sum_{k\in P^F} L_k)$

where $P^T$ is the set of correctly classified pixels and $P^F$ is the set of wrongly classified pixels, $N$ is the total number of pixels, and $\lambda$ is a scaling factor between the two parts of the loss that is set heuristically. See their equation (4) for details.

For positive $\lambda$, this equation could be rewritten as

$sign\nabla_{\boldsymbol{X}}(\frac{1}{N}\sum_{i\in P^T\cup P^F} (1- |\lambda - |(argmax(f(\boldsymbol{X}))-Y|/2|) L_i)$

for adversarial examples $\boldsymbol{X}$

i.e. $|\lambda - |(argmax(f(\boldsymbol{X}))-Y|/2|$ equals $1-\lambda$ for incorrect predictions, it equals $\lambda$ for correct predictions.

You can consider this representation of SegPGD to be the starting point of our argument: no matter what loss to use for L, we argue that the weighting of the pixel-wise loss with this weight after the argmax is an issue: it limits SegPGD to applications where the correctness of the prediction can be evaluated in a binary way, and it disregards the actual prediction scores. This is why CosPGD proposes in our equation (5) proposes to use a continuous measure of correctness instead:

$sign\nabla_{\boldsymbol{X}}(\frac{1}{N}\sum_{i\in P^T\cup P^F} cos(softmax(f(\boldsymbol{X})), Y) L_i)$

Please note that this facilitates CosPGD to operate on the segmentation scores instead of the final argmax predictions. Positive side aspects are that we do not need to set any heuristic parameter $\lambda$ and that we can directly apply this same procedure for a wide variety of tasks beyond segmentation. The experiments we show demonstrate the significant benefit of considering the actual prediction scores w.r.t. the cosine similarity of their softmax to the ground truth.
We empirically show the benefit of CosPGD over SegPGD in a wide variety of experiments.

Thus, we do not modify the loss, neither does SegPGD, the loss is the respective loss of the downstream task. We are replacing the scaling in SegPGD with a closed-form continuous setting i.e. cosine similarity.

We agree that SegPGD, with some modifications, can be extended to other downstream tasks. And we proposed CosPGD, which goes beyond SegPGD and is significantly stronger than SegPGD on the task for which SegPGD was proposed. And, as shown in our experiments, CosPGD extends to other tasks much better than the adaptations of SegPGD to those tasks.

Additionally, adversarial attacks are time and resource consuming. Thus, the availability of an efficient adversarial attack that requires merely 3 attack iterations to efficiently gauge a model’s relative robustness is indeed a significant contribution to the field. As discussed in the paper for each downstream task considered, CosPGD is able to expose model vulnerabilities that were previously unknown even with SegPGD.

Moreover, CosPGD requires merely 3 attack iterations during adversarial training to train a significantly more robust model.

Does this address your concern with respect to the difference of CosPGD to SegPGD and the novel contribution of this work?

Best Regards

Authors of Paper #909

Dear Reviewer 3VUb,

Thank you very much for your insightful review. We hope to answer your questions in a satisfactory manner.

1. The proposed CosPGD is motivated by SegPGD and indeed the equations look similar. Yet, there are significant conceptual differences that provide a strong benefit of CosPGD over SegPGD, which we also discuss on page 3 in our submission. Please refer to our general reply for details on the conceptual differences between the two and the resulting benefit of CosPGD. Please note that no other existing attack scales the loss pixel-wise using similarity between the posterior and target distributions. All attacks focus on final model predictions. This makes CosPGD significantly novel. Does this answer the important concern?

2. To the best of our knowledge, cosine distance is not the most commonly evaluated metric for any of the tasks considered: semantic segmentation, optical flow estimation, and image restoration(we evaluate structural similarity for this task). For all our considered models and tasks, we report all the metrics that are used by the authors to evaluate their proposed models and attacks.

3. Our experiments focus on the wide applicability of CosPGD rather than on outperforming a particular, dedicated approach on a particular task. We will try to provide results comparing our method to [1] for the final version, which is however not trivial since [1] proposes a minimal attack and CosPGD is an epsilon-bounded attack. In any case, we now cite in the revised version and will discuss this paper. Please note that according to the ICLR guidelines, https://iclr.cc/Conferences/2024/ReviewerGuide, publications from recent conferences (“published [...] within the last four months”: “on or after May 28, 2023”) are assumed to be contemporaneous work and a comparison is not required - this is the case for [1].

We hope we were able to answer all your questions to your satisfaction. Please let us know if you have further questions or concerns.

Best Regards

Authors of Paper #909

Dear Reviewer 3VUb,

We were curious if you happened to find the time to read our official comments and incorporated changes in the revised version. If so, we would be glad to answer any further questions or doubts you might have.

Best Regards,

Authors of Paper #909

Thank you for rephrasing your question! We think we better understand the question Q1 now, from a mathematical perspective.

Both SegPGD and CosPGD propose to modify the update step of the original PGD applied to each pixel, as given in our equation (1).

$sign\nabla_{\boldsymbol{X}}L$

Here, we write L to denote the original model loss with respect to the current model $f$ 's prediction based on an adversarial sample $\boldsymbol X$

and the one-hot encoded ground truth $Y$.

For segmentation, i.e. in SegPGD $L$ is a cross-entropy loss as specified in their equation (2) . In our paper, we also use a cross entropy loss when considering segmentation, but $L$ can in principle be any loss of the respective original model, as in PGD. For optical flow, we use an optical flow loss, to be consistent with PGD.

In SegPGD, the above update is modified to

$sign\nabla_{\boldsymbol{X}}(\frac{1-\lambda}{N}\sum_{i\in P^T} L_i + \frac{\lambda}{N}\sum_{k\in P^F} L_k)$

where $P^T$ is the set of correctly classified pixels and $P^F$ is the set of wrongly classified pixels, $N$ is the total number of pixels and $\lambda$ is a scaling factor between the two parts of the loss that is set heuristically. See their equation (4) for details.

For positive $\lambda$, this equation could be rewritten as

$sign\nabla_{\boldsymbol{X}}(\frac{1}{N}\sum_{i\in P^T\cup P^F} (1- |\lambda - |(argmax(f(\boldsymbol{X}))-Y|/2|) L_i)$

for adversarial examples $\boldsymbol{X}$

i.e. $|\lambda - |(argmax(f(\boldsymbol{X}))-Y|/2|$ equals $1-\lambda$ for incorrect predictions, it equals $\lambda$ for correct predictions.

You can consider this representation of SegPGD to be the starting point of our argument: no matter what loss to use for L, we argue that the weighting of the pixel-wise loss with this weight after the argmax is an issue: it limits SegPGD to applications where the correctness of the prediction can be evaluated in a binary way, and it disregards the actual prediction scores. This is why CosPGD proposes in our equation (5) proposes to use a continuous measure of correctness instead:

$sign\nabla_{\boldsymbol{X}}(\frac{1}{N}\sum_{i\in P^T\cup P^F} cos(softmax(f(\boldsymbol{X})), Y) L_i)$

Please note that this facilitates CosPGD to operate on the segmentation scores instead of the final argmax predictions. Positive side aspects are that we do not need to set any heuristic parameter $\lambda$ and that we can directly apply this same procedure for a wide variety of tasks beyond segmentation. The experiments we show demonstrate the significant benefit of considering the actual prediction scores w.r.t. the cosine similarity of their softmax to the ground truth.
We empirically show the benefit of CosPGD over SegPGD in a wide variety of experiments.

Does this address your concern with respect to the difference of CosPGD to SegPGD?

Regarding question Q2, we would like to reconsider the example of semantic segmentation, i.e. the formulation that you see above, where both SegPGD and CosPGD scale the cross-entropy loss per pixel. Then, in CosPGD, we have the softmax of the prediction scores which means that the input to the cosine similarity is a set of two vectors, both of which have values between zero and one and are normalized to one. The one-hot encoded label vector $Y$ has values that are either zero or one. This means that the ground truth $Y$ is a vector of binary values and just indicates the correct label. Therefore, we did not consider distribution losses (KL or JS) in general. This might be an interesting future research direction.

In principle, for the proposed scaling of the pixel-wise loss to be reasonable, the scaling value for the loss at each location also needs to be between zero and one. We acknowledge that one could consider further variants for the scaling. Cross-entropy between the prediction and the GT will however map to an inappropriate value range, similar to L2 or L1 distances between these two vectors, which would need to be normalized. The cosine similarity has the desired properties and seems appropriate to assess the angle between the softmax prediction and the ground truth. Therefore, the cosine similarity is an intuitive choice. We will add a respective discussion to the revision.

Dear Reviewer kcAq,

Thank you very much for your insightful review. We hope to answer your questions in a satisfactory manner.

Answer to weakness:

Please refer again to our section 4.3. We never make the claim that "CosPGD can efficiently enhance a new model's robustness". Instead, we propose CosPGD as an effective adversarial attack that can evaluate a model’s relative robustness correctly even with low attack iterations (iterations<=3). Thus our claim is: “CosPGD can be very useful in predicting a new model’s robustness efficiently.” Would you agree that this claim is sufficiently substantiated by our experiments (including the new results we provide in the rebuttal), or is there anything else you would like to see in particular?

Answers to Questions:

1. Adversarial attacks have a purpose beyond adversarial hardening: the evaluation of existing methods in terms of stability/robustness. We therefore focused on evaluating CosPGD across diverse conditions. In the following, we also report a comparison in terms of adversarial training. Here the models were trained with the “Training Attack” with 3 attack iterations, $\alpha$=0.01 and $\epsilon\approx\frac{8}{255}$ with a 50%-50% minibatch split, meaning only 50% of the samples in a batch were adversarially perturbed. Then, the adversarially trained models were evaluated using “Testing Attack” with multiple attack iterations are shown in the following table (here we report only the mIoU(%) for better readability):

We include this table in the revised version of the paper as Table 5 in Section B.4

Additionally, we include Figure 11 in Section B.4 in the revised version of the paper, in this figure we show the segmentation masks predicted by UNet after being adversarially trained. We observe that even after 100 attack iterations, the model adversarially trained using CosPGD is making reasonable predictions. However, the model trained with SegPGD is merely predicting a blob.

2. Could you please rephrase this question? We are not sure if we understand it correctly. Our rationale is that previous methods for attacks on pixel-wise prediction tasks always only focused on a single task, i.e. there has been SegPGD for semantic segmentation and PCFA for Optical Flow. CosPGD is more general than both and can therefore be applied to both. In Section 4.2 Optical Flow: we explain in detail how CosPGD is exposing model vulnerabilities that were unknown before. To further showcase the generality of the approach, we also evaluate it on image restoration for recent SotA methods. Does this address your question? Regarding deraining, we attempted to evaluate CosPGD on the SotA [a]. Yet, the model is too large to fit on our hardware when access to model gradients is needed (which is the case for white-box attacks). The model [a] was trained on 32 NVIDIA Tesla V100. If you can refer us to a more light-weight model of your choice, we will gladly provide the evaluation. [a] Chen et al., Pre-Trained Image Processing Transformer (IPT), 2021.

3. There seems to be a misconception: as specified in the contribution section, the contribution is not the improvement of adversarial training but proposing the first adversarial attack that provides a unified evaluation for diverse pixel-wise prediction tasks. Of course, adversarial attacks can be used for adversarial training (see our response to your Q1 for such results) and better adversarial attacks often result in better adversarial training. Our results above show that this is also the case for CosPGD.

We hope we were able to answer all your questions to your satisfaction. Please let us know if you have further questions or concerns.

Best Regards

Authors of Paper #909

Dear Reviewer kcAq,

We were curious if you happened to find the time to read our official comments and incorporated changes in the revised version. If so, we would be glad to answer any further questions or doubts you might have.

Best Regards,

Authors of Paper #909

Dear Reviewer YMdT,

Thank you very much for your insightful review. We hope to answer your questions in a satisfactory manner.

1. Thank you for the suggestion. We provide the following evaluation on semantic segmentation on Pascal VOC 2012 using UNet for [2] and [3]. For C&W [2], we have been evaluating the default parameters (c=1) here but have a strong suspicion that tuning c will improve results. We experimented with c={0.5, 1, 2, 8} and it did not make an impact on the performance. Would you have a particular suggestion for the choice of c?

For MI-FGSM [3] 100 iterations, the evaluation did not finish within 24 hours, which was the time limit for our GPU usage, but we can report all other iterations. We observe that CosPGD significantly outperforms all other attacks, across attack iterations and $l_p$ norms. For $l_{\infty}$-norm attacks $\alpha$=0.01 and $\epsilon \approx \frac{8}{255}$. We add these results to Table 4 in Section B.3 in the revised version of the paper.

For $l_2$-norm attacks in the table, where applicable $\alpha$=0.2 and $\epsilon \approx \frac{128}{255}$. We include this table in the revised version of the paper as Table 3 in Section B.2.1

(here we report only the mIoU(%) for better readability)

Also, thank you for the pointer to reference [1]. Our experiments focus on the wide applicability of CosPGD rather than on outperforming a particular, dedicated approach on a particular task. We will try to provide results comparing our method to [1] for the final version, which is however not trivial since [1] proposes a certified radius-guided attack framework and CosPGD is an epsilon-bounded attack. In any case, we cite the paper in the revised version. Please note that according to the ICLR guidelines, https://iclr.cc/Conferences/2024/ReviewerGuide, publications from recent conferences (“published [...] within the last four months”: “on or after May 28, 2023”) are assumed to be contemporaneous work and a comparison is not required - this is the case for [1].

Additionally, we came across another interesting related work that we have now cited in the related work of the revised version of the paper. Jia, J., Qu, W. and Gong, N., 2022. MultiGuard: Provably Robust Multi-label Classification against Adversarial Examples. Advances in Neural Information Processing Systems, 35, pp.10150-10163.

2. The most important conceptual difference between SegPGD and CosPGD is that the weighting with fixed weights is applied after the argmax operation in SegPGD. This removes important information and makes the attack optimization unstable. As a remedy, SegPGD uses a linear combination of the weights and non-weighted loss terms, where the combination weight is a heuristic. The use of the cosine similarity is therefore more informative: we compute the softmax of the class scores for segmentation to preserve the continuous prediction information. For the resulting positive valued prediction vectors, the cosine similarity is particularly suitable. Additionally, it extends well to pixel-wise regression tasks like optical flow estimation, image restoration etc. If the reviewer suggests, we can add this discussion to the final paper.

Dear Reviewer YMdT,

Following your suggestion in Weakness #4, we have now included Table 8 in the revised version to be included in Section C.3 later, which is an extension to results from Figures 14, 15, and 16 in the revised version (were Figures 13, 14, and 15 in the original submission) over 3 random seeds in a tabular form.

For ease of reading, in the following tables, we present Table 8: a comparison of the performance of CosPGD to PCFA as a targeted $l_2$-norm constrained attack for optical flow estimation over KITTI2015 and Sintel validation datasets using different optical flow models over 3 random seeds. Average $epe$ (AEE) values are compared, with respect to both, the Target where a lower $epe$ indicates a better attack and Initial flow prediction (optical flow estimated by the model before any adversarial attack) where a higher $epe$ indicates a better attack. We compare over both targets used by [4], i.e. zero vector $\overrightarrow{0}$ and Negative of the Initial Flow. As observed earlier in the respective figures, the performance of CosPGD is at-par with the recently proposed Optical Flow specific attack.

Please do let us know if you have any further questions or concerns, we would be glad to address them.

Best Regards

Authors of Paper #909

[4] Schmalfuss, Jenny, Philipp Scholze, and Andrés Bruhn. "A perturbation-constrained adversarial attack for evaluating the robustness of optical flow." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022.

Dear Reviewers and AC,

Thank you for acknowledging the strength of our work in terms of experiments and analysis. Since there is an ongoing discussion on alternative losses and the difference of CosPGD to SegPGD, we elaborate on this question from a mathematical formulation in the following, which, as we hope, will clarify the difference quite clearly.

Both SegPGD and CosPGD propose to modify the update step of the original PGD applied to each pixel, as given in our equation (1).

$sign\nabla_{\boldsymbol{X}}L$

Here, we write L to denote the original model loss with respect to the current model $f$'s prediction based on an adversarial sample $\boldsymbol X$

and the one-hot encoded ground truth $Y$.

For segmentation, i.e. in SegPGD $L$ is a cross-entropy loss as specified in their equation (2) . In our paper, we also use a cross entropy loss when considering segmentation, but $L$ can in principle be any loss of the respective original model, as in PGD. For optical flow, we use an optical flow loss, to be consistent with PGD.

In SegPGD, the above update is modified to

$sign\nabla_{\boldsymbol{X}}(\frac{1-\lambda}{N}\sum_{i\in P^T} L_i + \frac{\lambda}{N}\sum_{k\in P^F} L_k)$

where $P^T$ is the set of correctly classified pixels and $P^F$ is the set of wrongly classified pixels, $N$ is the total number of pixels and $\lambda$ is a scaling factor between the two parts of the loss that is set heuristically. See their equation (4) for details.

For positive $\lambda$, this equation could be rewritten as

$sign\nabla_{\boldsymbol{X}}(\frac{1}{N}\sum_{i\in P^T\cup P^F} (1- |\lambda - |(argmax(f(\boldsymbol{X}))-Y|/2|) L_i)$

for adversarial examples $\boldsymbol{X}$

i.e. $|\lambda - |(argmax(f(\boldsymbol{X}))-Y|/2|$ equals $1-\lambda$ for incorrect predictions, it equals $\lambda$ for correct predictions.

You can consider this representation of SegPGD to be the starting point of our argument: no matter what loss to use for L, we argue that the weighting of the pixel-wise loss with this weight after the argmax is an issue: it limits SegPGD to applications where the correctness of the prediction can be evaluated in a binary way, and it disregards the actual prediction scores. This is why CosPGD proposes in our equation (5) proposes to use a continuous measure of correctness instead:

$sign\nabla_{\boldsymbol{X}}(\frac{1}{N}\sum_{i\in P^T\cup P^F} cos(softmax(f(\boldsymbol{X})), Y) L_i)$

Thus we are replacing the scaling in SegPGD with a closed form continuous setting i.e. cosine similarity. Please note that this facilitates CosPGD to operate on the segmentation scores instead of the final argmax predictions. Positive side aspects are that we do not need to set any heuristic parameter $\lambda$ and that we can directly apply this same procedure for a wide variety of tasks beyond segmentation. The experiments we show demonstrate the significant benefit of considering the actual prediction scores w.r.t. the cosine similarity of their softmax to the ground truth.
We empirically show the benefit of CosPGD over SegPGD in a wide variety of experiments.

Moreover, CosPGD requires merely 3 attack iterations during adversarial training to train a significantly more robust model. We include these results in Table 5 in Section B.4.

We hope this clears the remaining doubts regarding the novelty of CosPGD wrt to SegPGD.

Best Regards

Authors of Paper #909