Constructing Semantics-Aware Adversarial Examples with a Probabilistic Perspective

We appreciate your review and your recognition of our probabilistic perspective and proposed method. Thank you for indicating the oversights and shortcomings in our submission. Your feedback has effectively improved the quality of this work. We respond to your questions and concerns as follows:

Response to weaknesses

Human Experiments

more detailed information … such as how the five reviewers for the MNIST experiment were selected. 

Regarding the selection of human annotators, please refer to our response to Question 3 below. We have included screenshots of the user interfaces for annotators in Appendix E. We apologize for omitting an important detail in our original submission: we use a voting system to determine the final choice from human annotators. In cases of a draw (which can occur in the MNIST dataset), we randomly select from the tied options. We will incorporate this description in the revised version. Thank you for bringing this to our attention.

… conducting a follow-up experiment …

We appreciate your suggestion for a follow-up experiment on ImageNet without reference images. Given the time required for human experiments, we will add this experiment in the final version of our paper.

We understand your concern regarding the presence of reference images in our experiment. We would like to clarify that this does not compromise our methodology. In our study, annotators are not informed which image is the reference. We realize we inadvertently omitted a crucial detail in Figure 8: the original image and adversarial example are presented in random order, adhering to the A/B test methodology employed in unrestricted adversarial attack papers such as [1] and [2]. We will emphasize this important point in the final version of our paper.

Comparison with NCF and cAdv

We indeed overlooked emphasizing that our method is optimal in balancing the trade-off between human annotation success rate and attack success rate in the paragraph from lines 258 to 264. In the final version, we will explicitly highlight that, according to Figure 5, our method performs best in this trade-off when appropriate parameters are selected.

We hypothesize that cAdv's lower human annotation success rate may be attributed to the presence of unnatural color spots in the generated images, as illustrated in Figure 4. These artifacts likely make the adversarial examples more noticeable to human annotators.

Targeted Attacks on ImageNet

We acknowledge that targeted attacks on ImageNet pose greater challenges due to the large number of classes, particularly regarding transferability. Our method does not offer specific advantages for this task. In the related work section, we will expand on this issue, discuss the work you recommended, and suggest it as a potential direction for future research based on our probabilistic perspective.

Answer to questions

Q1: How were the values of c chosen in Table 2?

Answer: The parameter c controls the influence of the victim distribution. A larger c results in adversarial samples that are more likely to deceive the victim classifier but may deviate more from the original image's semantics. We initially tested the algorithm on a few images, visually assessing the results. Based on these observations, we selected c values of 5, 10, and 20 as they produced distinct yet reasonable outcomes. We then applied these hyperparameters to attack 1000 ImageNet images, yielding the results shown in Table 2 and Figure 5.

Q2: What are the author's thoughts on the tradeoff between choosing different values of c?

Answer: Figure 5 illustrates the trade-off between different c values, with the x-axis representing human annotation success rate and the y-axis showing attack success rate. As c increases, adversarial examples become more effective at deceiving the victim classifier but are also more easily identified as adversarial by human annotators. We leave the choice of c to the users of our proposed method, depending on their specific requirements.

Q3: How were the human annotators selected?

Answer: We conducted human participation experiments through a reputable crowdsourcing company. The five annotators were recruited by this company. Our human experiments passed our institute's ethical review of research, taking into account the crowdsourcing company's qualifications, fair compensation, reasonable workload, and potential risks to workers. Due to the anonymous review process, we cannot share specific details here, but we can provide this information to the Area Chairs if required.

Q4: How was the set \tau chosen?

Answer: As introduced in Section 4.1, $\mathcal{T}$ represents a set of transformations that we subjectively believe maintain the semantics of the original image. For MNIST, this includes scaling, rotation, and thin-plate spline (TPS) transformations. For ImageNet, we utilize the distribution of the diffusion model after fine-tuning on the original image, which incorporates some implicit natural transformations learned by the model.

If we subjectively believe that appropriate color transformations do not affect semantics, we could incorporate such color transformations into $\mathcal{T}$ when fine-tuning the diffusion model. The primary aim of this work is to propose a probabilistic framework that can embed subjective understanding, rather than to conduct an in-depth exploration of $\mathcal{T}$ selection. We leave this exploratory task for future work.

Reference

[1] Song, Yang, et al. "Constructing unrestricted adversarial examples with generative models." Advances in neural information processing systems 31 (2018).

[2] Bhattad, Anand, et al. "Unrestricted adversarial examples via semantic manipulation." arXiv preprint arXiv:1904.06347 (2019).

We appreciate the reviewer's thorough evaluation and acknowledgment of our probabilistic approach and proposed methodology. We are grateful for the identification of weaknesses in our submission. The reviewer's insights have significantly enhanced the quality of this work. We address the review points as follows:

Success rate except for human annotations

In response to the reviewer's suggestion and to enhance the reliability of our results, we conducted an additional experiment using the MNIST test set, without human annotation. The results are as follows:

Note that 'OURS (no tech.)' and 'OURS (rej. samp. only)' refers to the ablation study discussed in the subsequent section.

Ablation study on two techniques

We have included the ablation study results for the two techniques in the table above.

Rejection sampling, a classical method, aligns naturally with our probabilistic approach to adversarial attacks. By employing rejection sampling, we can generate adversarial examples that achieve a 100% attack success rate in white-box settings. This perfect success rate is possible because we can consistently reject samples that fail to deceive the victim classifier.

On the other hand, sample refinement primarily affects human annotation while having minimal impact on the attack success rate for classifiers.

These two techniques are only used in MNIST experiments, because generating targeted adversarial samples for an adversarial trained MNIST classifier is relatively hard: As is shown in Figure 2 (b), the adversarially trained MNIST classifier is already have a strong generation ability, meaning that it already memorize the shape of the each digit.

Discussion on defending this attack

We appreciate the reviewer pointing out this problem. In the final version of the paper, we will include the following discussion:

Adversarial training operates on the principle: 'If I know the form of adversarial examples in advance, I can use them for data augmentation during training.' Thus, the success of adversarial training largely depends on foreknowledge of the attack form. Our method bypasses adversarially trained classifiers because the 'semantic-preserving perturbation' we employ is unforeseen by the classifier designers - they use conventional adversarial examples for training.

Conversely, if designers anticipate attacks from our algorithm, they could incorporate examples generated by our method into their training process - essentially, a new form of adversarial training.

This scenario transforms adversarial attacks and defenses into a game of Rock-Paper-Scissors, where anticipating the type of attack becomes crucial. One might consider training a classifier using all known types of attacks. However, expanding the training data too far from the original distribution typically leads to decreased performance on the original classification task, which is undesirable [1]. We believe that investigating the trade-off between this ‘generalized' adversarial training and accuracy on the original task represents a promising avenue for future research.

Standard deviation

To calculate the standard deviation in repeated experiments, we need to generate multiple sets of adversarial examples for each method and have them annotated by human annotators. Given the time required for annotation, we commit to providing these results in the final version of the paper.

Reference

[1] Zhang, Hongyang, et al. "Theoretically principled trade-off between robustness and accuracy." International conference on machine learning. PMLR, 2019.

We appreciate the reviewer's time and valuable feedback, which has significantly contributed to improving our work. We address the identified weaknesses as follows:

About Equation (4)

Our proposed framework assumes that the adversarial distribution is proportional to the product of the distance distribution and the victim distribution. This assumption separates the distance distribution from the victim distribution, reflecting our understanding that the similarity between objects is not directly related to the classifier being attacked. While this perspective might be debated, we believe it is reasonable within the context of our paper.

Theorem 1 demonstrates that this proposed form of adversarial distribution is consistent with conventional adversarial attacks. In Appendix A, we prove Theorem 1 by starting with the conventional adversarial attack setting (Equation (1)) and deriving the form of the product of $p_\text{dis}$ and $p_\text{vic}$. This derivation can be found above line 423 in our paper.

We appreciate the reviewer's comment and will provide more intuition about separating the distance and victim distributions between lines 71 and 73 to enhance readability.

About the training process of diffusion models

Please refer to the README.md file in the ImageNet folder of our attached anonymous GitHub repository (link provided in the abstract). Our code is based on OpenAI's guided diffusion repository, a mature and widely used diffusion implementation. We fine-tune the weights provided by the guided diffusion repository rather than training a diffusion model from scratch. The hyperparameters for fine-tuning are specified in our repository. We maintain most of the original training hyperparameters, with two exceptions:

1. Learning rate: 1e-6 (a commonly used rate for fine-tuning)
2. Number of fine-tuning steps: 300 (empirically determined)

We found that after 300 fine-tuning steps on the original image, the fine-tuned diffusion model adequately reflects the original image. These hyperparameters are consistently applied across all images in our evaluation, which we believe is appropriate for our study.

We thank the reviewer for pointing this out. Indeed, merely including this information in the accompanying code is insufficient. In the final version of the paper, we will add an appendix that provides a detailed discussion of the aforementioned parameter selection and fine-tuning process.

About Repeating the experiments

To calculate the standard deviation in repeated experiments, we need to generate multiple sets of adversarial examples for each method and have them annotated by human annotators. Given the time required for annotation, we commit to providing these results in the final version of the paper.