Robustness of AI-Image Detectors: Fundamental Limits and Practical Attacks

Thanks for your insightful comments. We hope the following responses can help address your concerns.

Concern 1: The paper is poorly written and not well-organized.

We have updated several parts in the new version of the draft, in order to increase clarification and organization.

• We added reference to the definition of terms that are formally defined in the paper after their usage, including $e_0, e_1$ in the introduction, and $R, F, R^t, F^t$ in Figure 1.
• The Y-axis label of Figure 2 had a typo which is fixed now. New labels were added to Figure 1 to enhance readability.
• New references were added to Section 3.1, to help the readers get familiar with the diffusion model terms that are being used in the section.
• More information about the importance of Figure 2 was added to Section 3.1.
• More explanation on the reason for the usage of denoising diffusion models in our theory was added to Section 3.1 and Appendix B. It is explained that other denoising techniques could also be used instead of denoising diffusion models, and references to related work were added.
• A more detailed description of the spoofing attack, together with a pseudocode has been added to Appendix A.4.
• In Section 4, formal definitions for some of the terms that are used in Theorem 2 were added to the main text.

Concern 2: The contribution of the paper is unclear.

Our paper includes several noteworthy contributions.

• Our first and perhaps most important contribution is the theoretical impossibility of designing reliable and imperceptible watermarks (i.e., Theorem 1 in the paper). Lots of previous studies use image watermarks for different applications, such as detecting AI-made images, protecting copyrighted images, and establishing image ownership. In all but a few cases, these watermarking techniques utilize imperceptible perturbations. Our theoretical findings show the fundamental unreliability of such techniques and suggest a paradigm shift toward watermarks with higher perturbation budgets within the research community.
• Our second contribution is the development of a novel model substitution black-box adversarial attack that breaks existing high perturbation budget watermarking techniques. This approach allows for attacking watermarking methods without requiring online query access to the models. To the best of our knowledge, the watermarking technique TreeRing has previously withstood black-box attacks in the literature, establishing itself as a reliable method. However, our attack successfully breaks this technique with an $\ell_\infty$ perturbation as small as 2/255.
• Our third contribution is developing a spoofing attack against watermarking by adding a watermarked noise image to non-watermarked images to deceive the detector into flagging them as watermarked.
• Our fourth and final contribution is to characterize a fundamental trade-off between the robustness and reliability of deepfake detectors and substantiate this concept through experiments.

Concern 3: Besides DiffPure (Nie et al., 2022), there are other diffusion-based approaches, such as DDNM (Wang et al., 2022), and non-diffusion-based ones, which are not investigated.

We appreciate the reviewer's observation regarding the potential use of alternative denoising techniques to eliminate the introduced Gaussian noise from images. We have added this explanation to the new draft of the paper (in Section 3.1 and Appendix B), while citing some related work.

Our theoretical findings establish an upper bound on the total variation of the distributions of watermarked and non-watermarked images by adding Gaussian noise to images. It is important to note that we do not rely on the denoising technique to calculate these bounds; rather, we employ them solely to generate high-quality output images. Consequently, any denoising technique, aside from DiffPure, can be employed to achieve the same theoretical bounds.

It is true that a stronger denoising technique permits the use of a higher magnitude of Gaussian noise, resulting in a more substantial lower bound on the error according to our theory. Nonetheless, it's worth emphasizing that DiffPure has already proven its capability to empirically break existing imperceptible watermarks, showing the effectiveness of our proposed methodology.

Thanks for your helpful comments. We hope the following responses can help address your concerns.

Weakness 1: It would be beneficial if the authors could simplify the explanations or provide additional resources to aid understanding.

In the new version of our draft, we have added a reference to [1] (in Section 3.1) which provides comprehensive definitions for diffusion models and the terms that are used in our paper. Most of the terms that we use in our work such as $x_0$, $x_t$, $\bar{\alpha}_t$ are consistent with this survey paper.

Question 2: Provide more intuitive explanations and examples, and simplify some of the theoretical derivations to make them more accessible and easier to understand.

To enhance the clarity of the theoretical parts of the paper, we have added more explanations to Sections 3 and 4, and the appendix. We added the formal definition of the terms used in Theorem 2 to the main text of Section 4. Additionally, we clarified the reason for the usage of denoising diffusion models in our theory, and how it can be replaced with other denoising techniques (Section 3.1, and Appendix B).

Question 1: In future work, conduct experiments using multiple datasets from various fields and sources to validate the effectiveness and stability of the method.

We agree with the reviewer’s comment that including more datasets could support the generalizability of our work. Adding new datasets can be time-consuming, especially since some of the watermarking methods require training of their models on the target dataset. Nonetheless, it will be one of our priorities to include more datasets such as COCO and CIFAR in our work, hopefully for the final draft of the paper.

[1] Understanding Diffusion Models: A Unified Perspective, Calvin Luo 2022

Thank you for your positive feedback. We hope the following response can help address your concern.

Weakness 1: The experimental results are missing a key element, specifically the PSNR, SSIM, or other image quality metric

We appreciate your valuable suggestion, which was an aspect we overlooked in our experiments. Below, you'll find the PSNR and SSIM values for images subjected to both DiffPure and adversarial attacks. In the case of the DiffPure attack, the output images with t=0.2 exhibit reasonable quality, while the attack significantly lowers the AUROC for imperceptible watermarking techniques, as shown in Figure 3 of the paper. As for the adversarial attack, the epsilon values employed in the paper align with standard perturbation budgets in the adversarial robustness literature. As indicated in the tables, these values result in images with reasonable quality. We have added these tables to the appendix of our paper.

[mean PSNR of images attacked with DiffPure w.r.t t]

[mean SSIM of images attacked with DiffPure w.r.t t]

[mean PSNR of adversarially attacked images w.r.t epsilon]

[mean SSIM of adversarially attacked images w.r.t epsilon]

Thanks for your valuable comments. We hope the following responses can help address your concerns.

Question 1: More watermarking methods are required to be considered.

We are thankful for the reviewer's suggestion. We have included results for the MBRS method in our paper, as recommended, and we are working on adding more methods such as StableSignature [1] and HiDDeN [2] to the final draft. We would like to highlight that our DiffPure attack is not just an empirical attack; it comes with a theoretical guarantee (Theorem 1) establishing a clear trade-off between type I and type II errors for any imperceptible watermarking methods. This, in our view, reduces the urgency for an exhaustive evaluation of every imperceptible watermarking technique that currently exists.

Question 2: The experimental setup of section 4 is impractical. I suggest authors add noise to spatial images directly instead of interior feature maps.

Thank you for the comment. In Appendix F (Figure 19), we add new experiments (in purple text) where noise is added to the images. This result shows that adding small noises to the image space can cause high $\ell_2$ perturbations in the latent space. Also, note that the tradeoff we present in Figure 8 (Theorem 2) will be more significant as AI images become more realistic and the Wasserstein distance between real and fake distributions decreases. This means that it will become easier to affect the performance of image detectors by adding lower levels of noise to the images as generative image models evolve.

Question 3: More AI-image detectors for arbitrary contexts require consideration.

Our theory in Section 4 is general and holds for any real and fake distributions. We consider an application of deepfakes and perform our experiments on facial images since they are popular in the research community. We take your feedback positively and are working on adding more experiments on a different dataset. We will update the manuscript if we finish our experiments before the discussion period ends. Otherwise, we will add these results to the final version of the paper.

[1] The Stable Signature: Rooting Watermarks in Latent Diffusion Models, Fernandez et. al., 2023

[2] HiDDeN: Hiding Data With Deep Networks, Zhu et. al., 2018