Be Your Own Neighborhood: Detecting Adversarial Example by the Neighborhood Relations Built on Self-Supervised Learning

We appreciate your detailed and constructive comments, and we are encouraged that you find our work “well written, easy to follow”, and “valuable contribution”. We explain your questions as follows:

Q1 Different Domain

We understand the reviewer is interested in exploring the use of BEYOND in new domains. We try to migrate our approach to different domains like Natural Language Processing (NLP) and Speech Recognition (SR). But we note some critical differences: existing self-supervised models in NLP or SR are not based on diversified data augmentation strategies. For instance, in NLP, masked token prediction is the primary method. However, having diverse augmentation types is important for BEYOND, because it needs good data augmentation to create neighboring data. Nonetheless, we believe that our approach pioneers an idea of detection based on testing time-augmented examples rather than reference examples. And we will also continue to explore ways to apply this idea to different domains.

Q2 Repeated Experiment

In this paper, our experimental results come from a single run. To further verify the effectiveness of BEYOND, We repeat the experiments in Tables 1 and 2 of the main paper five times. On Cifar10, the average detection performance for the five attacks is 99.16% with a standard deviation of 0.0117. On ImageNet, the average performance against three attacks is 96.44% with a standard deviation of 0.0426.

Minor Comments

Apologies for the uncritical description, we will correct it to“ where incorrect predictions can lead to unpredictable losses”. And about the “white-/gray-box” terminology, we appreciate your experienced suggestions. According to your advice, we refer to the latest work [RR1] from IEEE SaTML23, and change the terminology to “limited/perfect knowledge” attack in the the latest pdf.

[RR1] Apruzzese G, Anderson H S, Dambra S, et al. “Real Attackers Don't Compute Gradients”: Bridging the Gap Between Adversarial ML Research and Practice. 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML). IEEE, 2023: 339-364.

Our responses to all questions have been updated to the latest pdf and highlighted in blue.

Dear reviewer,

For the possibility of migrating BEYOND to other domains, within the limited time, we did a simple experiment on adversarial text detection for NLP. Using the text-attack tool [RR1], we generated 1000 adversarial texts on a BERT model trained on the MR sentiment classification dataset using the TextFooler attack [RR2]. We then used the chatgpt_paraphraser tool to generate ten paraphrases for each text as augmentations. Finally, we compare the embedding similarity between the samples and their augmentations on BERT. The AUC of the detection is 73.63%. Although there is certainly room for performance improvement, we believe that this pilot study confirms the possibility that BEYOND can be migrated to other domains.

[RR1] Morris J X, Lifland E, Yoo J Y, et al. Textattack: A framework for adversarial attacks, data augmentation, and adversarial training in nlp[J]. arXiv preprint arXiv:2005.05909, 2020.

[RR2] Jin D, Jin Z, Zhou J T, et al. Is BERT really robust? A strong baseline for natural language attack on text classification and entailment[C]//Proceedings of the AAAI conference on artificial intelligence. 2020, 34(05): 8018-8025.

We express sincere gratitude for your valuable feedback and constructive comments.

Q1 Theoretical study

First, our analyses in the main paper are for the case where augmentation can weaken adversarial perturbations, and it shows that if augmentation can weaken adversarial perturbations, it can be used for adversarial sample detection on the SSL representation space.

We note that it is difficult to conduct an overall theoretical analysis to verify that data augmentation as a whole can always effectively weaken adversarial perturbations because there are many data augmentations and they have different effects on adversarial perturbations. We address your question as to how to determine whether one given augmentation type weakens the adversarial perturbation or not. Therefore, we analyze which data augmentation can weaken adversarial perturbations, i.e., what conditions must be met for augmentations to be effective.

Effective data augmentation makes the augmented label $y_{aug}$ tend to the ground-truth label $y_{true}$ and away from the adversarial label $y_{adv}$ as follow:

$||y_{aug}-y_{true}||_2$

$\leq ||y_{aug}-y_{adv}||_2$

$\leq ||y_{adv}-y_{true}||_2$

Since $y_{true}$ is the one-hot encoding, the range of $||y_{adv}-y_{true}||_2$ is $(\sqrt{2}/2, \sqrt{2})$.

The distance is $\sqrt{2}$ when the item corresponding to $y_{adv}$ is 1 in the logits of $y_{adv}$, and $\sqrt{2}/2$ when the item corresponding to $y_{adv}$ and $y_{true}$ both occupy 1/2. Given a SSL-based classifier, $C$, we have:

$C(W(x+\delta)) = C(Wx)+\nabla C(Wx)W\delta = y_{true} + \nabla C(Wx)W\delta = y_{aug}$

Therefore, the distance between $y_{aug}$ and $y_{true}$ is:

$||y_{aug}-y_{true}||_2 = ||\nabla C(Wx)W\delta||_2$ $\leq ||\nabla C(Wx)W||_2 ||\delta||_2$

$\leq ||\nabla C(Wx)W||_{2} \epsilon$

where $||\delta||_2$ is bounded by $\epsilon$ according to the definition of adversarial examples.

To make sure $||y_{aug}-y_{true}||_2$

$\leq ||y_{aug}-y_{adv}||_2$, then:

$||\nabla C(Wx)W||_2 \epsilon \leq \frac{\sqrt{2}}{2} \Rightarrow ||\nabla C(Wx)W||_2 \leq \frac{\sqrt{2}}{{2\epsilon}}$

In summary, an augmentation can mitigate adversarial perturbation when it satisfies $||\nabla C(Wx)W||_2 \leq \frac{\sqrt{2}}{{2\epsilon}}$.

To further validate our analysis, we generate 1000 adversarial examples by PGD with $\epsilon=8/255$ on Cifar10. The following table shows the ratios for different data augmentations meeting the threshold $\frac{\sqrt{2}}{{2\epsilon}}$.

A higher ratio means the augmentation is more effective. Meanwhile, we perform t-sne visualizations of the representations of clean examples and AEs processed by different augmentation methods in Fig. 10 in the latest pdf. As seen in the figure, the effective augmentation methods with the high ratio in the above table can effectively increase the distance between AEs and their neighbors. For example, Rotation has the highest ratio in the above table, and the distance between AE and its neighbors in the figure is larger than that of clean samples. While Horizontal and Vertical have the lowest ratio, and the distance between AE and its neighbors is still close in the above figure.

Moreover, we test the detection performance of high-ratio augmentations and low-ratio augmentations as follows:

It can be seen that the average detection performance of the effective augmentations obtained by our analysis is 5% higher than that of the other augmentations.

Finally, BEYOND adopts various augmentations instead of a single type to generate multiple neighbors for AE detection. This allows the detection performance to circumvent the effects of bad augmentation methods, which reduces randomness and makes the estimates more robust.

Q2 Threshold

The evaluation metric in our paper is mainly area-under-curve (AUC). BEYOND includes three thresholds. For any threshold set, we compute a True Positive Rate (TPR) and a False Positive Rate (FPR). High AUC on different datasets shows that: 1) BEYOND can get a high TPR when FPR is low; 2) when we choose a bad threshold set that has a high FPR, we also get a high TPR. 3) The sensitivity of BEYOND to the threshold is weaker than baselines. Hence, BEYOND is not threshold-sensitive. Moreover, in practice, we can choose a threshold with FPR5% only based on clean examples, which is not allowed for baselines that require adversarial examples as a reference set or training.

Our responses to all questions have been updated to the latest pdf and highlighted in blue.

Dear reviewer,

We deeply appreciate your valuable time and the comprehensive feedback you have provided. However, upon reviewing your response, we have identified some misunderstandings that we would like to address and clarify in relation to our work:

1. The assumption we made in our theoretical analysis regarding the "benign perturbation, i.e. random noise, $\hat{\delta}$, with bounded budgets causing minor variation, $\hat{\epsilon}$, on the feature space" is intuitive and aligns with prior literature [RR1][RR2][RR3]. We believe this assumption is reasonable and aligns with established research in the field.

2. As for your concern about ''$|\nabla{C}(Wx)|$ becoming very large'', we believe that such a scenario is not realistic. This is because the SSL-based classifier $C(\cdot)$ that we employ has already undergone parameter optimization using gradient descent during the training process. Therefore, in situations where $C(\cdot)$ can achieve high accuracy, it is unlikely to encounter a situation where $|\nabla{C}(Wx)|$ becomes large.

3. Furthermore, it is important to note that the utilization of the SSL model offers the advantage of producing stable features even under common augmentations. This characteristic helps to mitigate the occurrence of extreme cases mentioned earlier to a significant extent. The use of SSL models allows for more robust and reliable performance, thereby enhancing the overall stability of the system.

We apologize for any misunderstandings that may have arisen. Finally Thank you again for the truly splendid discussion, and if you have any further concerns or questions, please do not hesitate to let us know.

[RR1] Nesti F, Biondi A, Buttazzo G. Detecting adversarial examples by input transformations, defense perturbations, and voting[J]. IEEE Transactions on neural networks and learning systems, 2021.

[RR2] Tian S, Yang G, Cai Y. Detecting adversarial examples through image transformation[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2018, 32(1).

[RR3] Xu W, Evans D, Qi Y. Feature squeezing: Detecting adversarial examples in deep neural networks[J]. arXiv preprint arXiv:1704.01155, 2017.

We express sincere gratitude to your valuable feedback and constructive comments.

Q1 Baselines

Our proposed BEYOND focuses on detecting adversarial examples by leveraging the relationship between the input and its neighbors. So, the baseline methods we chose are all related to detectors that are built on the distance between the input sample and its neighbors. And since our method uses the SSL model, we also compare BEYOND with Mao et al, which is a SSL-based adaptive-robust defense method.

Moreover, we also included the suggested baselines SimCat from “[RR1]” and E&R from “[RR2]”. The comparison on ImageNet for CW L2 attack is as follows:

Please note that SimCat is not open source, so the corresponding results come from our own replication. Erase-and-Restore (E&R) is only for L2 norm attacks. It can be seen that BEYOND outperforms these two baselines. In addition, both SimCat and E&R require training a detector, which is not required inBEYOND. For adaptive attacks, Table 5 in [RR1] shows that as a few-shot detector, SimCat itself is not robust to adaptive attacks, and its adaptive robustness can only be improved by combining a robust model (e.g. Adversarial training model). E&R in [RR2] evaluated its robustness to adaptive attack (BPDA), but [RR3] shows BPDA is not as effective as APGD used in BEYOND.

[RR1] Moayeri M, Feizi S. Sample efficient detection and classification of adversarial attacks via self-supervised embeddings[C]//Proceedings of the IEEE/CVF international conference on computer vision. 2021: 7677-7686.

[RR2] Zuo F, Zeng Q. Exploiting the sensitivity of L2 adversarial examples to erase-and-restore[C]//Proceedings of the 2021 ACM Asia Conference on Computer and Communications Security. 2021: 40-51.

[RR3] Francesco Croce, Sven Gowal, Thomas Brunner, Evan Shelhamer, Matthias Hein, and Taylan Cemgil. Evaluating the adversarial robustness of adaptive test-time defenses. In International Conference on Machine Learning, pp. 4421–4435. PMLR, 2022

Q2 Citation Format

Thanks for your careful check, we have corrected the citation format in the latest pdf.

Q3 Various Types of Attacks

Your concern about robustness to various types of attacks is warranted. Following the reviewer’s comments we test the most representative method that supports multiple norm attacks, AutoAttack. AutoAttack supports $L_{\infty}$, $L_2$ and $L_1$ norm attacks. In the main paper, we only report the detection performance of BEYOND against AutoAttack $L_{\infty}$. The following table shows the performance of BEYOND against AutoAttack with different norms.

The perturbation budgets($\epsilon$) on Cifar10 are 8/255 ($L_{\infty}$), 0.5 ($L_2$), and 8 ($L_1$); and on ImageNet are 8/255 ($L_{\infty}$), 3 ($L_2$), and 64 ($L_1$). The result shows BEYOND is still effective against attacks based on different norms.

Our responses to all questions have been updated to the latest pdf and highlighted in blue.

Weakness:

Thanks for your useful comments, we will adjust the paper to enhance readability. For the figure caption, we have added the necessary details to make it self-contained in the final paper.

Q1 Augmentation Type

We apologize for the confusion. The type of augmentations used by BEYOND is consistent with SimSiam's, including horizontal flips, cropping, color jitter, and grayscale. Since we need to generate multiple different neighbor samples for each input image, we use different combinations of augmentation parameters. However, unlike SimSiam's random augmentations, we fix the random seed to ensure that our method does not benefit from randomization. We have modified the corresponding part to make the method setting clearer.

Q2 & Q3 Typo and Citation

Sorry for the discomfort caused by typos and repeated citation names, we have carefully proofread our paper.

Our responses to all questions have been updated to the latest pdf and highlighted in blue.