Lack of in-depth analysis

1. The proposed method, [...] many explanations.

Since very similar objectives and relaxations are commonly used in the optimization community for different purposes, we have judged not necessary to include an in-depth analysis. We suggest adding one motivating line and citations to guide the reader. Nonetheless, the proposed method has been evaluated on small scale (CIFAR-10) and large scale (Imagenet) datasets for various architectures.

2. The paper [...] universal.

The learned directions aim to span the space of the adversarial noise, which is related to the original data space and to some extent, the classifier. As illustrated in the figures above, a two-atom dictionary can construct all the adversarial examples, if they exist. This space is indeed universal. Moreover, through the observed transferability across different classifiers, this conclusion is further confirmed. When attacking a given input, the composition coefficient needs to be computed.

3. When studying [...] per dataset.

In black-box attacks, transfer-based methods play a crucial role. To attain strong performance in black-box scenarios, preliminary research on the transferability of a model is vital. This constitutes the primary objective of our current work. We analyze the properties of the learned space to subsequently apply it to enhance black-box attack strategies.

Relation to prior work

1. The adversarial [...] proposed results.

Please refer to the response to weakness 1 of reviewer XSE8.

Potential performance limitations

1.A significant practical [...] attacks in white-box.

You are right, in order to optimize LIMANS' dictionary, we need at least few clean samples and a source classifier to convergence towards optimized fooling parameters. However one could also argue that most of the black-box attack do impose the same constraint, also white-box attacks have their hyper-parameters that also need to be tuned. Therefore we believe the need of a training set not to be too limiting compared to other propositions.

2. Another [...] data point.

Thank you for your valuable question. The goal of our proposed method is not only to construct adversarial examples but also to uncover common mechanisms within a dataset for crafting adversarial noises.

Clarity

1. The presentation [...] proofreading.

Thank you for your valuable insights and guidance in enhancing our paper.

Minor points

1. Consider using \citep [...] formatting (see here).

Thank you for the advice; it has been incorporated into the modifications.

2. Repeat sentence [...] of "closed".

Thank you, we have corrected it.

3. Some elements of the ICLR [...] style.

Thank you, we have aligned it precisely with the ICLR style.

Questions

1. It would be interesting [...] universal?

It is indeed an intriguing direction. Currently, we are exploring the universality of adversarial directions across the entire dataset rather than focusing on a single example. We plan to delve deeper into the universality of a subspace corresponding to an input.

2. How does the present work [...] [Sheatsley et al., 2023])?

Similar to other research efforts, LIMANS endeavors to uncover the intrinsic mechanisms behind adversarial attacks and ultimately enhance the robustness of the classifier. While our work doesn't explicitly manipulate the feature space, it is supposed to fortify non-robust features discussed in [Ilyas et al., 2019] [Kim et al., 2021]. Prior studies on the structure of the adversarial space have predominantly focused on analyzing the local adversarial space around an input [He et al., 2018], [Paiton et al., 2022], [Sheatsley et al., 2023]. In contrast, our space encompasses the entire dataset.Moreover, through global optimization on the entire dataset, the learned adversarial directions exhibit less noise-dependent and architecture-dependent characteristics (as defined in [He et al., 2018]), but instead, they possess a more data-dependent composition. This introduces the concept of transferability in the learned space. We also affirm the conclusion in [Paiton et al., 2022] that robust classifiers are situated in higher dimensions.

Weaknesses

Our emphasis is on researching the adversarial noise space, a domain contingent upon the dataset space. Our proposed algorithm facilitates the learning of a dictionary that spans either partially or entirely the adversarial noise space, demonstrating transferability. A comparison with transferable attacks confirms that a specific attack is unable to outperform a globally trained adversarial noise space in terms of transferability.

This post-processing involves projecting onto the valid space of D, denoted as $\mathcal{D} ={D \, ||D_j||_p=1, \forall j \in {1,\dots,M}}$. This projection is introduced in step 18 of Algorithm 2. It's noteworthy that we explicitly enforce the constraint on $Dv$ during training rather than on $D$ and $v$ separately. However, this does not impact the final results due to the post-processing step, where $d_i=d_i/||d_i||_p$ and $v_i = v_i*||d_i||_p$ are applied.

We have carefully proofread the article to correct these issues.

Please refer to our answer to Reviewer S66D. Additionally, we would like to mention that, for the sake of fairness, we selected the most common adversarial budgets, namely $||\epsilon||_2 < 0.5$ and $||\epsilon|| _\infty < 8$ for attacks on CIFAR-10. Of course, choosing larger threshold would permit to yield better fooling performance at the cost of more detectable attacks. We also would like to point that CIFAR-10 proves to be more challenging to attack than ImageNet, as evidenced by the RobustBench (https://robustbench.github.io/) performance metrics for AutoAttack.

Questions

Yes, our algorithm is designed to learn a dictionary that endeavors to span the adversarial noise space. To find an adversarial sample for a given input, computation of the composition coefficients is necessary. The comparison highlights that, as depicted in the figures above, specific attacks often struggle to transfer due to the local gradient direction bias among classifiers. In contrast, a universal attack significantly degrades performance when $\vert\vert \epsilon \vert\vert_p$ is small. Conversely, LIMANS, having learned the space of adversarial noise, exhibits improved performance when transferred to different classifiers.

Thank you. It has been fixed.

In algorithms 1 and 2, the capital V represents the matrix that is the concatenation of all the training coding vectors, while the lower case v represents only one single training coding vector.

Please refer to the response addressing Weakness 4.

Please refer to the response addressing Weakness 2.

It is assumed that the robustness against the detector is achieved due to the optimization-based schema.

The name in the subscript of each detector corresponds to the attack used to produce the training set on which the detector is trained. Put differently, d_FGSM is made to detect attacks crafted by FGSM. First, standard accuracy (SA) is defined as the classification accuracy of a DNN model without adversarial examples or attack detectors. Then, robust accuracy (RA) is defined as the classification accuracy of a DNN model with adversarial examples.

Imaging a simplest case, a two-class classification problem with a linear classifier; here, a unique direction enables inputs to find an adversarial example if one exists. Consequently, the dimension of the adversarial noise is 1. In more complex scenarios, such as depicted by classifier 2 in the figures above, it is still possible to construct a one-dimensional space. However, when the classifier, like classifier 1 in the figures above, is even more intricate, the dimension of the space should be 2. Therefore, based on the larger dimension of the space for robust classifiers and their slow slope in Figure 2 of the paper, it implies the complexity of the robust classifiers' boundaries. Figure 3 illustrates the image of one direction, which is related to classifier boundaries rather than the classifier itself. In reality, explicitly illustrating the classifier boundaries of deep neural networks is challenging. [Paiton, Dylan M., et al. "The Geometry of Adversarial Subspaces." (2021).] emphasize this through local analysis.

Please refer to "existing adversarial attacks have exhibited great effectiveness but with low transferability..." in [Wang X, He K. Enhancing the transferability of adversarial attacks through variance tuning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021: 1924-1933]. Furthermore, we presented empirical estimates of the transferability of specific attacks, including transferable specific attacks like VNI-FGSM.

Weaknesses

1. The study primarily [...] connection between universal and specific adversarial attacks.

We acknowledge that our proposed modeling, and more broadly, related works on adversarial noise space modeling, reveal new research directions that necessitate theoretical guidance and guarantees. In this paper, we have opted for a more practical and empirical evaluation, intending to delve into its theoretical aspects in future works. Nevertheless, we share the following insights that we recommend including in the final version to further justify our approach. The overall space of adversarial noise is equivalent to that of the classifier boundaries. However, due to the complexity of the deep neural network, direct analysis of the classifier seemed out of reach. Consequently, we endeavored to elucidate certain characteristics of the space (e.g., its dimension) through empirical experiments. Despite this, a straightforward analysis can be conducted, as illustrated in the figures above. In contrast, a universal attack seeks an average direction across the entire dataset to deceive as many examples as possible. However, with a single direction, it becomes impractical for inputs in case 2 to discern perturbations. LIMANS (case 2) bridges the gap between these two types of attacks. It dispenses with the need to seek a direction for each input but instead focuses on the space spanned by {$d_0$, $d_1$}.

2. The results suggest [...] and trade-off analysis.

In fact, the transferability of the space is proportional to its effectiveness on the source model. Typically, a classifier operates in higher dimensions, necessitating more atoms to construct the space. This makes it easier to transfer the space to a target model situated in a lower-dimensional space. For instance, the dictionary learned on Inception can be readily transferred to MobileNet. Conversely, transferability diminishes. Moreover, if the training sets of two models are not in the same space, transferability is significantly affected.

3. The conflicting results observed [...] in practical applications.

We disagree and in fact strongly believe that the results obtained on the two datasets are consistent. Indeed, for complex datasets like ImageNet having $C=1000$ classes, learning the complete adversarial noise space is a challenge. Assuming the simplest case where the classifier is linear and its hyperplanes are independent, $C\times (C-1)/2 = (1000 \times 999)/2 = 499500$ atoms are needed to construct the space. In our experiments, we only learned a 100-dimensional subspace of the adversarial noise space, yet it allowed us to deceive half of the examples. This highlights a practical limitation of our algorithm, as it requires substantial memory to learn a larger, or even complete, dictionary.

Questions

1. The paper builds upon the notion [...] specific attacks struggle when transferred.

[Ref B] demonstrates the existence of a transferable subspace around an input, inspiring transfer-based black-box attacks. Specific attack seeks the smallest perturbation that misleads the prediction (case 1). Typically, this tailored perturbation at the border of the adversarial space associated with a source model (in blue) seldom intersects with the adversarial space of another target model (in orange), limiting its transferability. We assess the transferability of the specific attack, such as AutoAttacks, revealing its limitations in the context of transfer. Our proposed method learned the adversarial space of a dataset not a single input, which makes it possible to craft an attack by a linear combinaison of the atoms under certain condition.

2. The paper also posits [...] in the current version.

As explained earlier, a specific attack identifies adversarial noise along the direction of the gradient, which is locally perpendicular to the classification boundaries. However, a universal attack searches for the average direction across the entire dataset. Comparing the averaged direction of two classifiers, their local gradient directions differ more.

3. While the paper's formulation is intriguing, [...] its significance.

We agree. The analysis previously mentionned has been added to our paper to provide additionnal insights.

4. Notably, the results in Table 3 [...] efficacy of the proposed method.

Kindly refer to the response addressing Weakness 3.

1. The performance of the proposed LIMANS relies on the choice of the source classifier.

The reviewer is right. By design, the modeling of the adversarial noise space is optimized to fool a single source classifier. However, the results depicted in Tables 2 and 3 show great transferability performance to other classifiers as well. We believe that these results stem from the fact that the proposed framework, in-between specific and universal attacks, allows to learn adversarial patterns that are less dependent on the source classifier but more representative of the data diversity instead.

2. It is not clear the extra computation overhead for learning the model from the source classifier, it would be better the compare the spending time across different attacks (for Simple-LIMANS and Regularized-LIMANS respectively).

We agree with the reviewer that a time comparison of both attacks would be interesting. However, these two attacks play different roles. Simple-LIMANS acts as a decent, quick-to-run, and parameter-free attack, while Regularized-LIMANS might show stronger performances when coupled with the right hyper-parameters, at the burden of finding them.

3. The results of the transferability performance of UAP look much lower than that in the existing literature, even lower than some specific attacks when attacking a standard-trained model. Why?

In the original work on UAP, its performance is assessed on ImageNet under the condition that $\vert\vert \epsilon \vert\vert_\infty \leq 10$. In our experiments, we impose a more restrictive constraint that is $\vert\vert \epsilon \vert\vert_\infty \leq 4$. This results in a drop of performance, which is overcome by our proposed method, LIMANS. Nonetheless, our work aligns with the conclusions of the existing literature, demonstrating that universal attacks exhibit great universality at the expense of lower fooling performances on the source model.

4. Could you explain more about why you only use the validation set and not the training set?

It is assumed that the training set is used to train the classifier, and in practice, it is usually not publicly accessible and we only have access to the validation data.

5. To my knowledge, you seem to miss some SOTA methods in the field of universal attack (e.g., [1]).

You are right, due to extensive research in the field producing very fast new SOTA methods, this method has unfortunately been missed. We suggest to add the reference in the final version.

6. The simple-LIMANS seems to remove the tunable hyper-parameter lambda of regularized-LIMANS, therefore also providing the results of regularized-LIMASNS would be desirable.

Unfortunately, this is not feasible. A direction linked to an input is parallel to $\delta^{*}=\displaystyle \text{argmin}_{\{f(x+\delta)\neq f(x)\}} ~||\delta||_p$. The ensemble of optimal directions globally encompasses all these directions within a dataset. This cannot be achieved with a fixed threshold.

7. A lot of typos/mistakes, e.g. "Table ??", "FOR FUTUR", "learn The associated"...

It has been fixed, thank you for pointing them.