Ceci n'est pas une pomme: Adversarial Illusions in Multi-Modal Embeddings

BadEncoder is a training-time attack, it assumes that the attacker poisons the training of the model. In the case of ImageBind or CLIP, it could be used only when the models were trained. Ours is an evaluation-time attack on the original, unmodified and unpoisoned ImageBind. It does not involve injecting anything into the training data and does not assume that the attacker has access to the model during training. We will clarify this fundamental difference between the threat models of backdoor attacks (like BadEncoder) and adversarial inputs (like ours) in Background and Related Work.

Thank for the review!

1. I-FGSM comments.
   1. In general, PGD refers to I-FGSM with a random initialization in an $\ell_\infty$ ball around the source image. This method may perform marginally better for “conventional” adversarial examples, in our setting this benefit is greatly diminished since adversarial alignment must only ‘beat’ organic alignment. We will add an experiment confirming this observation in the Appendix.
   2. As with the above, the $\ell_\infty$ norm (the most common in the literature) is sufficient for an attack that achieves all its objectives, i.e., strong adversarial alignment that fools all downstream tasks. We can add some experiments on other $\ell_p$ norms into the Appendix.
   3. The image in question uses the perturbation bound of $\epsilon = \frac{16}{256}$. As we reported elsewhere in the submission, our attack is effective with much lower bounds. We regenerated the image with a lower bound and it shows no visible noise. While DiffPGD is fascinating, it appears unnecessary in our case since our attack already achieves the desired adversarial alignment with tiny perturbations (DiffPGD can only improve image quality, not performance).
2. Same-modality adversarial examples are strictly easier than cross-modal illusions, because, as discussed in Cross-Modal Illusions, the modality gap only arises between modalities. In particular, the unbounded case can be trivially solved within the same modality (i.e., $\delta = \mathbf{a}^{m_2} - x^{m_1}$) whereas a solution may not exist for the cross-modal case. We will provide some experiments in the Appendix.
3. To expand the evaluation, we have shown that our attack is effective on AudioCLIP (with a 100% success rate for $\epsilon$ larger than or equal to $\frac{4}{256}$. In addition, we are conducting experiments on the transferability between embedding spaces. The experiments will be added to our Evaluation section.
4. The targeted attacks we present in the paper are strictly harder than the suggested untargeted attacks. Rather than perturb images to induce any non-source class, we align with a specific non-source class. The suggested loss function reduces to finding any orthogonal vector in the unbounded case. Since we are in high-dimensional space, sampling randomly is likely to produce an orthogonal vector, and can be computed far more efficiently. In the bounded case, empirically, optimizing our suggested attack with any target until the alignment between the perturbed input and the source is less than the organic alignment is sufficient. This can be achieved with extremely small perturbation bounds.
5. Thank you for the feedback.   We will move some examples to the Appendix.
6. Add JPEG experiments Our Countermeasures section has been updated to show how our attack evades the JPEG compression defense, a common method to mitigate adversarial perturbations. The JPEG-resistant attack still achieves a $75\\%$ success rate in the presence of the defense. It also evades the defense that checks consistency of embeddings of different input augmentations.  As we note in the paper, there are limitations to most traditional defenses. Developing new defenses specifically for embeddings is an interesting research direction.

Thank you for the review!

Weaknesses:

1. To expand the evaluation, we have shown that our attack is effective on AudioCLIP (with a 100% success rate for $\epsilon$ larger than 4. In addition, we are conducting experiments on the transferability between embedding spaces. The experiments will be added to our Evaluation section.
2. We are also conducting experiments on Audio Classification for both ImageBind and AudioCLIP. We expect the numbers to be good, considering audio retrieval is a strictly harder desiderata than classification. The experiments will be added to our Evaluation section.

Questions:

1. Black-box attacks are interesting future work but there are some inherent challenges to this formulation that we will add to the Limitations section. Consider two alternate definitions of ‘black-box’: (1) the attacker has no access to the internal representations of the encoder, but does have access to the resulting embeddings, and (2) the attacker has only access to the outputs of a specific downstream task. Recall that our attack is targeted.

   For (1), satisfying the classification constraint (i.e., changing the label to a specific target) is a significantly easier task than finding an image that is arbitrarily close, in a high-dimensional embedding space, to a specific target embedding. Modern black-box techniques that rely on Bayesian Optimization are particularly ill-suited for this high-dimensional objective. That said, this is an interesting topic for future work.

   For (2), while it is possible that an adversarial example generated for a specific downstream task may generalize to others, the representation-merging of the example and its target could happen in the downstream model as opposed to the encoder, making generalization to all downstream tasks hard to achieve. This is also a promising direction for future work.

2. We demonstrate that in our setting, I-FGSM’s performance is essentially perfect, achieving high adversarial alignment (much higher than organic alignment) with tiny perturbations.  We are not sure about the motivation for exploring other methods but can evaluate PGD if requested.

3. Given the strength of our adversarial alignment, we expect that any system that relies on a unified embedding space, regardless of whether it was contrastively trained, will be vulnerable to attack. In particular, our attack relies on the former as opposed to the latter.

4. Thanks for pointing that out! That should read ‘Image Encoder’.

5. Thanks again, we are missing the clipping operation.

6. We will clarify this distinction in the Limitations section. To rephrase, ImageBind embeddings work on unCLIP, but seemingly perform worse. We hope to disentangle our observations on the alignment of our attack and the limitations (due to training) of the models we use.

We are still in the process of getting those numbers, but in small scale experiments on ImageBind for $\epsilon = 0.05$ we achieve 100% accuracy. We will report the expanded results for these experiments in the final version of the paper with the remaining choices of $\epsilon$ as well as equivalents on AudioCLIP. Given this result and the efficacy on audio retrieval, we project that the results will be better than or equal to the audio retrieval experiments. Recall that zero-shot retrieval requires the identification of an arbitrary caption related to an audio while classification seeks out labels that the model was already trained on (in the case of ImageBind). Importantly, in the space of multimodal encoders, the mechanics of the two tasks are equivalent: finding the label or caption nearest in embedding space to the input. Since, our attack works on arbitrary (audio, caption) pairs, we believe that the attack will be effective on (audio, label) pairs as well.

(edits: 1. a previous version of this comment mentioned the wrong epsilon bound and 2. added details about retrieval.)

Thank you for your review.

Weaknesses:

1. Ours is the first attack that targets multi-modal embeddings, as opposed to a specific task, and is completely agnostic of downstream tasks.  Success of an embedding attack cannot be evaluated without measuring its effect on tasks that use the embedding.

   To expand the evaluation, we have shown that our attack is effective on AudioCLIP (with a $100\\%$ success rate for $\epsilon$ larger than 4. In addition, we are conducting experiments on the transferability between embedding spaces. The experiments will be added to our Evaluation section.

   Previous attacks on multimodal learning target a single modality and don’t need to deal with modality gap. We emphasize our main observation: adversarial alignment can be made significantly closer than any organic alignment regardless of input and target modality. As a result, our attack affects any current and future downstream task based on the attacked embedding (or, as we show in our new experiments, similar embeddings). With an increased industry focus on large, centralized encoders, the same embedding will underpin many downstream use cases.  We show they can be attacked wholesale, without the attacker even knowing what they are.

2. Previous work targeted specific tasks, not multi-modal embeddings themselves. Our updated Background and Related Work section explains how our attack relates to [1, 2, 3], which were helpfully provided by our other reviewers.

Questions:

1. Our Countermeasures section has been updated to show how our attack evades the JPEG compression defense, a common method to mitigate adversarial perturbations. The JPEG-resistant attack still achieves a $75\\%$ success rate in the presence of the defense. It also evades the defense that checks consistency of embeddings of different input augmentations.

Thank you for providing the reference, we want to point that the attack defined in Sec 3.1 of the paper is an attack on a specific downstream task (zero-shot image classification), not a downstream-task agnostic attack on the multi-modal embedding that affects all tasks based on that embedding (which is our contribution in this submission) including generative tasks that don’t assign a fixed label to the input.

We, therefore, argue that our attack is the first downstream-task agnostic attack unlike the method presented in that paper.