Adversarial Perturbations Are Formed by Iteratively Learning Linear Combinations of the Right Singular Vectors of the Adversarial Jacobian

Dear Reviewer D2j5,

Thank you for your valuable comments and efforts. We address your concerns as follows and will update those carefully in the revision.

C1: The reliance on computationally intensive techniques like SQP and SVD may render RisingAttack less feasible for real-time or on-device applications where computational resources are limited. The paper does not sufficiently address the method's performance or feasibility in real-world scenarios, where computational efficiency and the ability to operate under diverse and unpredictable conditions are crucial.

Thank you. We acknowledge the importance of developing methods that can be deployed in real-world scenarios under various conditions.

Both our RisingAttacK and the prior art, QuadAttacK can not achieve real-time attacks yet (please see our time complexity report in addressing the C1 to the Reviewer ubo3).

The novelty of our proposed RisingAttacK lies in its ability of learning ordered top-K adversarial perturbations directly in the image space using SQP for the first time to our knowledge, which is crucial for enhancing our understanding of adversarial perturbations and potentially for developing adversarial defense methods.

We hope that the deployment aspects of our RisingAttacK, as well as other attack methods, could be improved gradually when techniques become more mature with better optimizers and engineered implementations.

We also hope that the current lack of real-time running speed of our RisingAttacK (as well as the prior art, QuadAttacK) is not a factor that negatively impact us and other researchers on investigating rigorous optimization frameworks such as SQP in adversarial learning.

C2: The effectiveness of the attack largely hinges on the accurate computation of the Jacobian matrix, which can vary significantly depending on the model's architecture and the data, potentially limiting the method's effectiveness in less ideal conditions.

Thank you. This might be a misunderstanding. The computation of the Jacobian matrix is not approximated. For most DNNs that are fully differentiable and since we are studying white-box attacks, the logits-to-input Jacobian matrix can be exactly computed. For example, we used the torch.func.jacrev [1] in our implementation. So, the Jacobian computation does not limit our method's effectiveness.

The approximation component in our proposed method lies in the first-order linearization of a DNN (Eqn.6) to relax the optimization problem with highly non-linear constraints to one with linear constraints, as commonly done under the SQP framework.

C3: The generalization of the method to tasks outside of image classification or to non-visual data remains untested, which may limit its utility in broader applications of AI. Although it outperforms certain established methods, the paper could benefit from a broader comparison with a wider range of adversarial attack strategies to more comprehensively position its effectiveness within the field.

Thank you. We acknowledge the importance of developing attack methods and testing them for many tasks, rather than image classification. However, image classification is still the most widely used application task for studying new adversarial attack methods.

For the ordered top-K targeted attacks studied in this paper, it is still under active research with two previous approaches that we can find (Zhang & Wu, 2020; Paniagua et al., 2023). For practical considerations of conducting experimental comparisons, we follow the prior art to focus on image classification tasks. The reported experimental results are sufficiently comprehensive to verify the effectiveness of our proposed method in our opinion.

With a forward-looking thinking, our proposed method should have similar potential extensions as methods such as PGD (projected gradient descent) for learning top-1 targeted attacks. We leave those applications for further work, while we are continuing working on improving the efficiency of our current RisingAttacK.

[1] https://pytorch.org/docs/stable/generated/torch.func.jacrev.html

Dear Reviewer ubo3,

Thank you very much for your valuable comments and efforts. We address your concerns one by one as follows, which will be carefully updated in revision.

C1: Report the time complexity of an iteration of RisingAttacK and that of QuadAttacK

Thank you. We report the average seconds/iteration using a batch of 32 images over 30 iterations on a single A100 GPU as follows, | $K$ | QuadAttacK | RisingAttacK | | :----:| :----:|:----:| | 1 | 1.31 | 1.38 | | 5 | 1.17 | 0.83 | | 10 | 0.95 | 1.57 | | 20 | 1.30 | 2.8| | Avg| 1.18 | 1.64 |

We note three aspects,

• For both methods, the time complexity of an iteration is not necessarily monotonically related to $K$. It reflects the complexity of the underlying QP to be solved, which in turn is affected by the the sheer challenges of the randomly sampled top-$K$ attack targets for different $K$'s and different images.
• On the average (32 images over 30 iterations), our RisingAttacK is more computationally expensive than QuadAttacK. With the significantly improved performance by our RisingAttacK, the increased time complexity seems to be reasonable.
• We will re-run all the experiments with time complexity recorded for a full-scale time complexity comparison between the two methods in revision.

C2: Could you clarify the claim of the paper "(…) ordered top-K adversarial perturbations can be expressed as linear combinations of the right singular vectors (corresponding to non-zero singular values) of the attack-targets-ranking constrained logit-to-image Jacobian matrix." ? The iterative nature of the method seems to make the interpretation of this right singular vectors more difficult.

Thank you for pointing this out. We agree with you that the iterative nature of our RisingAttacK indeed makes the interpretation not precise.

We propose to change the title to ``Adversarial Perturbations Are Formed by Iteratively Learning Linear Combinations of the Right Singular Vectors of the Attack-Targets-Ranking Constrained Jacobian", as well as all the related statements in text.

We would like to continue to highlight the observation of "Linear Combinations of the Right Singular Vectors of the Attack-Targets-Ranking Constrained Jacobian" in solving Eqn.18 at each iteration, which we think provides useful insights for understanding adversarial perturbations directly in the image space.

Dear Reviewer 69zR,

Thank you for valuable comments and efforts. We address your concerns one by one as follows. We will carefully update those in the revision.

C1: I cannot figure out why we need the top-K targeted attack. Can you provide any practical scenarios or its benefits compared with targeted/untargeted attacks?

First of all, targeted attacks can provide better controllability for adversaries against untargeted attacks, and are a harder problem to solve, as widely recognized in the literature.

For targeted attacks, in terms of how precisely and how aggressively we could manipulate the outputs (i.e., the entire logits) of DNNs, there are three different settings with increasing difficulties to achieve: (i) conventional top-1 targeted attacks, (ii) unordered top-$K$ targeted attacks ($K\geq 1$), and (iii) ordered top-$K$ targeted attacks ($K\geq 1$). We focus on (iii) in this paper.

From top-1 to top-$K$ (unorder or ordered), as pointed out in (Zhang & Wu, 2020), the robustness of attack methods themselves should be investigated. For example, a top-1 targeted attack may be viewed successfully (e.g., from a cat image to a dog prediction), the ground-truth label (cat) may still be the top-2 or 3 prediction, which will be less effective in terms of top-k (e.g., top-5) accuracy metric. So, top-$K$ attacks, especially with a large $K$, can ensure the ground-truth labels will be pushed sufficiently away.

Between unordered top-$K$ and ordered top-$K$, as pointed out in (Paniagua et al., 2023), there are two scenarios in practice, ordinal examples and nominal examples:

``Imagine a cancer risk assessment tool that analyzes 2D medical images (e.g., mammograms) to categorize patients’ cancer risk into the ordinal 7-level risk ratings ([Extremely High Risk, Very High Risk, High Risk, Moderate Risk, Low Risk, Minimal Risk, No Risk]), An oncologist could use this tool to triage patients, prioritizing those in the highest risk categories for immediate intervention. An attacker aiming to delay treatment might use an ordered top-3 adversarial attack to change a prediction for a patient initially assessed as Very High Risk. They could target the classes [Moderate, Low, Minimal], subtly downgrading the urgency without breaking the logical sequence of risk categories. An unordered attack, in contrast, might lead to a sequence like [Low, Very High, Minimal], disrupting the ordinal relationship between classes. Such a disruption could raise red flags, making the attack easier to detect."

Please see page 2 for the nominal examples in (Paniagua et al., 2023), due to space limit, we can not quote those examples here.

Similar in spirit to the ordinal and nominal examples provided in (Paniagua et al., 2023), learning ordered top-$K$ targeted attacks could find practical applications such as for the recommendation systems (that often recommend a number of items in a particular order) and the retrieval systems (that often return ordered retrieved items).

More related to computer vision applications, APIs such as Google Cloud Vision, Microsoft Azure Computer Vision, Amazon Rekognition, and IBM Watson Visual Recognition, often return ordered top-$K$ (e.g., 10) predictions about input images, for which ordered top-K targeted attacks could be studied.

C2: It is not clear how you solve Eq. (25). Which optimizer do you adopt?

Thank you. As stated in lines 340-341: ``We show that Eqn. 25 has a closed-form solution (see the proof in Appendix C), reproducing the result in Eqn. 18.". So, we do not need an optimizer to solve it.

C3: The proposed holistic figure of merits (FoM) metric seems to be limited. It is constrained with a single baseline. How could I calculate it with multiple baselines?

Thank you. The propose FoM is for pair-wise comparisons. For comparing a method 1 against other methods (2 to M), there are three possible extensions:

• We may simply compute $mean_{j=2}^M FoM(1, j)$, similar in spirit to the mean Average Precision (mAP) that is widely used in object detection for comparing accuracy across multiple categories and across multiple IoU thresholds.
• We may further consider: (i) the strict $\text{FoM} = \frac{\text{ASR}^1}{\max_{j=2}^M(\text{ASR}^j)}\cdot \frac{1}{3}\cdot \sum_{p\in \{1, 2,\infty\}}\frac{\min_{j=2}^M\ell_p^j}{\ell_p^1}$, to show how well the method 1 performs against the best of the rest in terms of every aspects (ASR and three norms), and (ii) the average $\text{FoM} = \frac{\text{ASR}^1}{mean_{j=2}^M(\text{ASR}^j)}\cdot \frac{1}{3}\cdot \sum_{p\in \{1, 2,\infty\}}\frac{mean_{j=2}^M\ell_p^j}{\ell_p^1}$