AdvI2I: Adversarial Image Attack on Image-to-Image Diffusion models

Thank author's reply.

Q1

Regarding Q1, while I understand you want to emphasize the discovery of a new vulnerability, my perspective focuses on its practical implications. The key question is whether this vulnerability would see widespread adoption. Since this vulnerability is specifically designed for NSFW content generation, its widespread use would likely indicate some advantages over existing NSFW generation methods --- particularly in terms of output quality. This potential quality improvement was my immediate thought about its advantage. Based on this reasoning, I requested NR IQA (No-Reference Image Quality Assessment) in Question 2 to validate this hypothesis.

Q2

For Q2, the author didn't solve my concern, here are the reason:

1. The baseline you choosed is out of date (4 years ago). you used the LDM as the backbone, so it would be fairer if you use the diffusion based face swap or deepfake. I believe there are many options, even some methods provide gradio web GUI for convenient use, so you should compare to some new method, especially LDM-based.
2. To compare the quality of generation, TOPIQ is not enough. I would ask the author to at least add FID as another metric.

Q3

Same as the Q2.2, one IQA is not enough. To my knowledge, a well-establish evaluation protocol is at least SSIM+LPIPS+PSNR

Q4 - Q5

My concerns were solved.

Q4: Use face verification methods or metrics to quantify identity preservation in generated images.

A4: Since the SDv1.5-Inpainting model allows masking to prevent modifications to the facial region, it ensures that the faces in the generated NSFW images remain almost identical to those in the clean images. To evaluate facial consistency, we used CosFace [3] to measure facial identity preservation. AdvI2I achieves a score of 93.7, indicating that the generated NSFW images retain the same identity as the clean images.

Q5: ASR results for different safety checkers used during training and testing of AdvI2I-Adaptive.

A5: Existing SD models use the same NSFW-detector [3] as the safety checker, which encodes images into embeddings using either ViT-L/14 or ViT-B/32 as the base model. When training the adversarial image generator for AdvI2I-Adaptive, we used a ViT-L/14-based NSFW-detector as the safety checker. We then evaluated the transferability of AdvI2I-Adaptive to the ViT-B/32-based NSFW-detector and observe that it still achieves a high ASR, as shown below. We have also included the results in the revised version. Thank you for your suggestion.

References:

[1] https://github.com/wuhuikai/FaceSwap

[2] MMA-Diffusion: MultiModal Attack on Diffusion Models

[3] CosFace: Large Margin Cosine Loss for Deep Face Recognition

[4] https://github.com/LAION-AI/CLIP-based-NSFW-Detector

Thank you for your valuable comments and suggestions. We have carefully addressed each of your questions and concerns below.

Q1: How our method compares to or improves upon existing NSFW image generation techniques like deepnude or deepfake.

A1: While Deepnude and Deepfake could be used by malicious users to generate harmful/deceptive images, our purpose here is not to produce high-quality NSFW images but to highlight a critical security vulnerability existing in general I2I models: even benign I2I diffusion models and users could inadvertently result in NSFW contents when the input image is from an untrusted third party. This is especially concerning as I2I diffusion models become widely used due to the greater flexibility in editing: for instance, (benign) users can freely prompt diffusion models to edit any image, and once the image is altered by our attack to encode any NSFW concept (e.g., nudity, violence etc), the generation becomes harmful.

Q2: Evaluate the quality of generated images with reasonable metrics.

A2: We compared the average TOPIQ of AdvI2I with FaceSwap [1] on 200 test samples. FaceSwap is a deepfake method that focuses on swapping faces between images while preserving the original facial features and expressions in the new context. Using FaceSwap, we swapped the faces of images onto the 200 test samples and calculated the TOPIQ scores. The results, as shown below, indicate no significant differences in quality.

Q3: Compare IQAs with other baselines for clean and attacked images.

A3: We evaluated the LPIPS loss between clean and attacked images for Attack VAE, MMA [2], and AdvI2I on 200 test samples. As shown in the table below, the differences are minimal. It is worth noting that MMA’s adversarial images are designed specifically to bypass safety checkers, relying on adversarial prompts to generate NSFW content.

Thank you very much for your followup comments and further suggestions. Below, we will trying addressing your concerns in detail and provide additional results for clarification.

Q1 & Q2: Consider diffusion-based face swap or deepfake. Add FID as another metric.

We included Face-Adapter [1], a diffusion-based face swap method using SDv1.5 as the base model, as a baseline for comparison. The image quality was evaluated using multiple metrics: TOPIQ (with three checkpoints trained on different datasets) [2], NIQE, PIQE, and FID. The results are as follows:

The table shows that AdvI2I consistently performs competitively across various metrics. It achieves higher quality in TOPIQ-koniq and TOPIQ-spaq compared to Face-Adapter, while also showing significant improvements in NIQE, PIQE, and FID scores, which indicate better perceptual quality and closer alignment to real image distributions. These results demonstrate that AdvI2I effectively generates high-quality adversarial images while maintaining its primary objective of exposing vulnerabilities in I2I models. We have also included the results in the revised version.

Q3: Compare IQAs with other baselines for clean and attacked images using SSIM, LPIPS, and PSNR.

In addition to LPIPS, we incorporated SSIM, PSNR, FSIM, and VIF to provide a more comprehensive comparison of the quality of attacked images. The results are as follows:

The results highlight that AdvI2I performs on par with Attack VAE in terms of structural and perceptual similarity (SSIM and LPIPS) and visual feature retention (FSIM and VIF), while significantly outperforming MMA. Importantly, both AdvI2I and Attack VAE use generators to produce adversarial images, while MMA directly optimizes adversarial noise. Although MMA achieves a higher PSNR due to its direct noise optimization approach, it performs worse in metrics like VIF and SSIM. AdvI2I successfully balances adversarial effectiveness and attacked image quality across all metrics, reinforcing its stealthiness and robustness. We have also included the results in the revised version.

We hope these results and explanations address your concerns. Please feel free to share additional thoughts or questions for further discussion.

References:

[1] Face Adapter for Pre-Trained Diffusion Models with Fine-Grained ID and Attribute Control

[2] TOPIQ: A Top-down Approach from Semantics to Distortions for Image Quality Assessment

Dear Reviewer Cqd3,

Thank you for your thoughtful suggestions and engagement. We truly appreciate your recognition of our responses and your willingness to adjust your score.

We will incorporate a thorough discussion of the suggested quality aspects into the paper to ensure clarity and completeness. If you have any additional feedback, we would be happy to address it. Otherwise, we kindly hope you might consider increasing the score to a positive range.

Thank you once again for your valuable time and insights!

Best regards, Authors of Paper AdvI2I: Adversarial Image Attack on Image-to-Image Diffusion Models

Q1: Can AdvI2I method apply for good use, such as helping mitigate inappropriate image editing?

A1: Thank you for this insightful suggestion. We experiment with constructing a good "wearing clothes" concept following our AdvI2I pipeline. Interestingly, this approach indeed reduces the ASR for the exposed prompt attack, "Make the woman naked," on SDv1.5-Inpainting from 96.5% to 24.5%. We believe our proposed method is a general framework to encode a certain concept into the input image, regardless of whether the concept is good or bad. We will include this discussion and more case studies in the paper.

Q2: How performance of AdvI2I vary based on the neutral prompt.

A2: The experiments presented in the main text evaluate AdvI2I using various neutral prompts, such as "Make the woman in the room" and "Let the woman on the set." Additionally, we tested three different suffixes appended to benign prompts (e.g., "Make the woman in the room" → "Make the woman in the room with suitable dress.") to further explore this. Even the most effective suffix reduces the ASR by only 6.5%. As explained in the subsection "NSFW Concept Vector Extraction," AdvI2I incorporates the NSFW concept with a strength factor α>1 into the embedding of benign prompts, making it difficult to defend against such attacks by merely modifying prompts. In some cases, inappropriate suffixes even increase the ASR. We will try more safe prompts and include the results in the revised version. Thank you for your suggestion.

Q3: What hardware and how long to train AdvI2I?

A3: We use an 80GB A100 GPU, and training AdvI2I on SDv1.5-Inpainting takes approximately 43 hours with 1800 training samples, 150 epoches, batch size 4 and image size 512x512.

References:

[1] Ring-A-Bell! How Reliable are Concept Removal Methods For Diffusion Models?

[2] MMA-Diffusion: MultiModal Attack on Diffusion Models

Thank you for your valuable comments and suggestions. We have carefully addressed each of your questions and concerns below.

W1: highlight the contribution of $g_\psi$ on the run time comparison with optimization approach.

A1: Sorry for the confusion. We would like to highlight the main contribution of our work: instead of focusing on reducing the time cost of crafting adversarial images, the purpose of the proposed AdvI2I attack is to highlight the risks of I2I diffusion models generating NSFW images, even when the input prompts are benign. To the best of our knowledge, prior studies like Ring-A-Bell [1] and MMA [2] primarily explore crafting malicious prompts to induce NSFW content, and as shown in Table 2, simple text filters could defend such prompt-based attacks. On the contrary, the vulnerabilities arising from input images are relatively underexplored.

As a side advantage of our attack, it indeed offers a more efficient production of adversarial images through direct inference on $g_\psi$. We compared the time cost of generating a single adversarial sample using AdvI2I and MMA. As shown in the table below, using a generator significantly accelerates the process of crafting adversarial samples.

We also tested the ASR and time cost of using an ad-hoc generator on the SDv1.5-Inpainting model w/ and w/o sampling timestep t ("w/o sampling" refers to backpropagate all inference timesteps), which directly optimizes the advesarial image for each input sample. The results highlight the significant advantage of our generator over the ad-hoc noise generator. Compared with them, AdvI2I achieves a near-optimal balance between high ASR and extremely low time cost, making it not only more efficient but also more practical for generating massive adversarial images.

W2: SDXL-Turbo Image-to-Image transferability test.

A2: We evaluated the transferability of AdvI2I from InstructPix2Pix to SD-Turbo Image-to-Image and from SDv1.5-Inpainting to FLUX.1-dev ControlNet Inpainting-Alpha. The results are shown below:

W3: Typos/Mistakes in writing.

A3: Thank you for the detailed suggestions. We have carefully reviewed and corrected all typos and mistakes mentioned in the revised version. The revised parts are shown in blue color.

Dear Reviewer 3wqb,

We greatly appreciate your initial comments and your positive feedback on our work. We totally understand that you may be extremely busy at this time. But we still hope that you could have a quick look at our responses to your concerns. We appreciate any feedback you could give us. We also hope you could kindly update the rating if your questions have been addressed. We are also happy to answer any additional questions before the discussion ends.

Best Regards, Authors of Paper AdvI2I: Adversarial Image Attack on Image-to-Image Diffusion Models

W1: While the paper highlights I2I adversarial attack risks, including runtime analysis would further strengthen the claims about the method's advantages. I'm pleased to see runtime speed now included in the author's response.

Q1: It's nice to see that AdvI2I can be a general method for good use

A2: The prompt "and avoid any nude content" is unconvincing due to Stable Diffusion's weakness with negation. A concept embedding focusing on "wearing clothes" would be more effective than simply changing the suffix.

W2, W3, A3 : The author has addressed my concerns.

Response to Additional Comment by Reviewer 3wqb

W1, W2, W3, Q1, Q3: Thanks for confirming we have addressed these concerns. We have incorporated the important discussions you suggested into the paper revision.

Q2: Thank you for pointing out SD's known limitations with handling negations in prompt, as well as your suggestion to explore a "wearing clothes" concept for defending against AdvI2I attacks. We have conducted additional experiments to evaluate the effectiveness of embedding a benign concept, "wearing clothes," for mitigating ASRs of under the nudity-related AdvI2I attack on the SDv1.5-Inpainting model. The embedding strength was adjusted using the strength coefficient $\alpha$ mentioned in subsection 3.2 of our paper (AdvI2I uses $\alpha = 4.0$ for the nudity concept).

The experimental results, presented in the table below, reveal that while the benign concept shows a certain ability to reduce ASR with larger $\alpha$, its practical usage is limited by the noticeable damage to utility:

1. ASR Reduction: As $\alpha$ increases, the ASR under the AdvI2I attack gradually decreases, demonstrating the efficacy of the "wearing clothes" concept in mitigating adversarial attacks to some extent.
2. Impact on Image Utility: To evaluate the impact of the "wearing clothes" concept on utility, we tested the editing performance on 200 randomly selected images from ImageNet-1k [1], which primarily contain natural, non-human-centered scenes. Using the prompt "change its color", we applied the "wearing clothes" concept with varying $\alpha$ values and evaluated the quality of the edited images using several metrics, including LPIPS, SSIM, PSNR, NIQE, and PIQE. For reference-based metrics (LPIPS, SSIM, PSNR), the generated images at $\alpha = 0.0$ were used as the reference. Results indicate that such benign concept comes at the cost of significantly degraded utility for non-adversarial edits, particularly noticeable in metrics like SSIM and PIQE.

These findings suggest that while using benign concepts can provide some level of defense, its practical application is heavily constrained due to the visible damage to the quality of benign image editing, especially at higher $\alpha$.

We hope these additional experiments and clarifications address your concerns. If there are no further questions, we kindly hope you might consider adjusting your score in light of the updates and revisions we’ve made. Thank you again for your valuable feedback!

References:

[1] Russakovsky, Olga, et al. "Imagenet large scale visual recognition challenge." International journal of computer vision 115 (2015): 211-252.

W3: Why not use ad-hoc noise generation for each image?

A3: Thank you for this suggestion. We tested the ASR and time cost of using an ad-hoc generator on the SDv1.5-Inpainting model w/ and w/o sampling timestep t ("w/o sampling" refers to backpropagate all inference timesteps). We evaluated the ASR on 200 test samples and calculated the average time cost of generate one adversarial image. The results highlight the significant advantage of our generator over the ad-hoc generator. While the ad-hoc generator achieves a comparable ASR (83.5% vs. 82.5%) when all timesteps are backpropagated, it incurs an exponentially higher time cost—382.434 seconds per adversarial image compared to just 0.008 seconds for AdvI2I. When the ad-hoc generator uses sampling to reduce the number of timesteps, it sacrifices ASR (35.0%) while still requiring 49.336 seconds per image. In contrast, AdvI2I achieves a near-optimal balance between high ASR and extremely low time cost, making it not only more efficient but also more practical for generating massive adversarial images.

W4: White-box setting of the NSFW-detector may not be practical.

A4: Existing SD models use the same NSFW-detector [3] as the safety checker, which encodes images into embeddings using either ViT-L/14 or ViT-B/32 as the base model. When training the adversarial image generator for AdvI2I-Adaptive, we used a ViT-L/14-based NSFW-detector as the safety checker. We then evaluated the transferability of AdvI2I-Adaptive to the ViT-B/32-based NSFW-detector and observe that it still achieves a high ASR, as shown below. We have also included the results in the revised version. Thank you for your suggestion.

W5: Some other defenses e.g. add noise / use purifier.

A5: We evaluated DiffPure [4] as a defense mechanism. While it reduces the ASR for the SDv1.5-Inpainting model under the nudity concept from 82.5% to 72.5%, the decline is not significant. This suggests that AdvI2I is robust against simple defenses targeting adversarial images. We did not include MMA as a baseline here because MMA uses adversarial prompts to induce NSFW images in diffusion models, whereas DiffPure is designed to defend against adversarial images, thus the ASR of MMA will not change after applying DiffPure defense.

References:

[1] Ring-A-Bell! How Reliable are Concept Removal Methods For Diffusion Models?

[2] MMA-Diffusion: MultiModal Attack on Diffusion Models

[3] https://github.com/LAION-AI/CLIP-based-NSFW-Detector

[4] Diffusion models for adversarial purification.

Thank you for your valuable comments and suggestions. We have carefully addressed each of your questions and concerns below.

W1: I2I model is a sub-optimal choice to generate high quality NSFW images.

A1: Sorry for the confusions. We would like to hightlight the main motivation of this work: instead of achieving high-quality NSFW images, the purpose of the proposed AdvI2I attack is to expose the risks of I2I diffusion models in generating NSFW images, even when the input prompts are benign. To the best of our knowledge, prior studies like Ring-A-Bell [1] and MMA [2] primarily focus on crafting malicious prompts to induce NSFW content, and as shown in Table 2 of the paper, simple text filters could defend prompt-based attacks. On the contrary, the vulnerabilities arising from input images are relatively underexplored.

W2: The proposed method is restricted to a small subset of I2I models, which are not the main part of diffusion models.

A2: Our work focuses on testing whether I2I diffusion models are vulnerable to generating NSFW images through adversarial input images. The models evaluated in the our work, including SD inpainting models and InstructPix2Pix, cover commonly used I2I diffusion models. The results clearly show that these models are indeed susceptible to such risks. To demonstrate that AdvI2I is not architecture-specific, we tested its transferability from InstructPix2Pix to SD-Turbo Image-to-Image and from SDv1.5-Inpainting to FLUX.1-dev ControlNet Inpainting-Alpha, and achieved high ASRs, as shown in the table below. We have also included the results in the revised version.

We also evaluated AdvI2I on the SDv2.1-Inpainting model and achieved an ASR of 78.5% under the nudity concept, demonstrating that AdvI2I can generalize to state-of-the-art diffusion models. We have included the results in the revised version, and will further evaluate the adaptive version (AdvI2I-Adaptive) given more time. Thank you for your suggestion.

Dear Reviewer ZfYn,

We greatly appreciate your initial comments and your positive feedback on our work. We totally understand that you may be extremely busy at this time. But we still hope that you could have a quick look at our responses to your concerns. We appreciate any feedback you could give us. We also hope you could kindly update the rating if your questions have been addressed. We are also happy to answer any additional questions before the discussion ends.

Best Regards, Authors of Paper AdvI2I: Adversarial Image Attack on Image-to-Image Diffusion Models

Thank you for your insightful comments. We have thoroughly considered and responded to each of your questions and concerns below.

W1: Include more advanced versions of Stable Diffusion or other models.

A1: We evaluated AdvI2I on the SDv2.1-Inpainting model and achieved an ASR of 78.5% under the nudity concept, demonstrating that AdvI2I can generalize to state-of-the-art diffusion models. We have included the results in the revised version, and will further evaluate the adaptive version (AdvI2I-Adaptive) given more time. Thank you for your suggestion.

W2: Examine the effect of varying benign prompts with explicitly requesting safe content.

A2: We tested three different suffixes appended to our benign prompts (e.g., "Make the woman in the room" → "Make the woman in the room and avoid any nude content.") to explore this. Even the most effective suffix achieves only a slight reduction in ASR (6.5%). As explained in the paper subsection "NSFW Concept Vector Extraction", AdvI2I incorporates the NSFW concept with a strength factor $\alpha$ (larger than 1) into the embedding of benign prompts, making it challenging to defend against this attack by merely modifying the prompts. In some cases, inappropriate suffixes even increases the ASR. We will try more safe prompt and include the results in the revised version. Thank you for your suggestion.

Dear Reviewer f4nA,

We greatly appreciate your initial comments and your positive feedback on our work. We totally understand that you may be extremely busy at this time. But we still hope that you could have a quick look at our responses to your concerns. We appreciate any feedback you could give us. We also hope you could kindly update the rating if your questions have been addressed. We are also happy to answer any additional questions before the discussion ends.

Best Regards, Authors of Paper AdvI2I: Adversarial Image Attack on Image-to-Image Diffusion Models

Thank you for your response. It addresses most of my concerns. However, the results highlight AdvI2I's limitations against advanced models like SDv3.0, where the reduction of NSFW content during training effectively mitigates attacks. This suggests a simple yet impactful defense strategy that may already be employed in advanced models, rendering AdvI2I ineffective. Nonetheless, I think this paper raises safety concerns for the field. Therefore, I will maintain my score, but I do not oppose the paper's acceptance.