Anti-Reference: Universal and Immediate Defense Against Reference-Based Generation

1. This idea of using adversarial examples for privacy protection has been well-explored. Although this paper primarily focuses on defending privacy against diffusion-based models. Some related works for defending privacy against standard image generation models should also be discussed [a,b, c].
2. The possible defence mechanism built by the target generative models should be also considered, such as adversarial training and adversarial detection.

[a] Restricted black-box adversarial attack against deepfake face swapping [b] Scalable Universal Adversarial Watermark Defending against Facial Forgery [c] IDGuard: Robust, General, Identity-Centric POI Proactive Defense Against Face Editing Abuse

1. Although the paper highlights the challenges in universality and transferability, it lacks an in-depth technical explanation of why existing methods fall short and how the proposed approach directly addresses these issues. The loss functions and optimization methods are not novel, which are standard in the prior works[2][3].

[2] Liang C, Wu X, Hua Y, et al. Adversarial example does good: Preventing painting imitation from diffusion models via adversarial examples[C]// ICML 2024.

[3] Van Le T, Phung H, Nguyen T H, et al. Anti-dreambooth: Protecting users from personalized text-to-image synthesis[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.

2. The experimental results are somewhat limited. The method’s primary application appears to be in the protection of copyrighted artwork; however, no dataset in this field is included.

• Applicability to New Models: Anti-Reference is primarily tested on models like Stable Diffusion 1.5. The method may face challenges adapting to new architectures, such as Stable Diffusion XL or autoregressive models, as acknowledged in the limitations section.

• Use of SDv1.5: The Article only demonstrates the use case with the SDv1.5 Model, which is now deprecated, limiting the reproducibility of the results presented in the article. While most fine-tuned customizable models are based on the SDv1.5 model, it would be good if authors could demonstrate that the method can be applied to other Stable diffusion-based Models.

• Usage of Black-Box Term: The use of black-box is highly over-exaggerated. In a way, we know the architectures of the black-box methods used in the article, and the authors use the same architecture. In the field of adversarial machine learning, it is well-known that adversarial images transfer to similar architectures better. It would be good if the authors could show models protected with SDv1.5 fine-tuning can still protect against black-box reference methods that are not based on SDv1.5 (and even better if they are not Stable Diffusion).

• Practical Applications: The authors suggest that their method is real-world-ready (albeit only for SDv1.5-based customization methods), but there seems to be minimal analysis to support this argument. The only experiment that supports this argument is the use of JPEG Compression on the images (details are not provided regarding the amount of JPEG compression used). It is fair to say, in the real world, the images can be distorted before reaching the generative model; it can be resizing of the images, Strong JPEG Compression, Gaussian Blurring, Purifying the image by Diffusion Model by recreating the images (e.g., using Null-Text Inversion), or transformations applied by social-media platforms. Authors fail to verify whether adversarial noise is still effective in protecting the images after transformations that can happen in the real world. Current experiments are insufficient to argue that the method is practical in the real world.

• Correcting the optimization problem: Based on the optimization problem defined in Page 5, one has to increase the distance between Generated Images. However, this is not true for the losses they mention, as they do not increase the distance between the generated images. At the same time, it is intuitive that the distance between the generated images would increase for protected images since protected images would result in distorted images having a higher distance from the non-distorted images. Authors should bridge the gap between the optimization problem and their proposed loss theoretically; otherwise, one can argue that the protection is not optimization but a search. Further, empirically, authors can also report the MSE between the generated images to demonstrate the distance between the generated images.

• Reproducibility of Experiments: The Authors introduce weights for each individual loss but fail to provide the values used in their experiments. Similarly, the article does not discuss the hyper-parameters of the PGD. To improve the reproducibility of the results, the authors should provide better experimental setup details.

• Presentation of Qualitative Results: Quantitative results are hard to interpret, and they show that existing methods and proposed methods perform similarly. First, it would be nice if Figure 4 (based on Tables 2, 3, 4, and 5 in the appendix) could be presented better. Second, there is no clear evidence that the methods perform better than the existing methods quantitatively. This creates suspicion about cherry-picking the qualitative results of whose analysis most of the article is based. Either the authors clearly explain in the main article why quantitative results contradict the qualitative results, or they demonstrate that their method is unarguably superior in quantitative metrics.

• Limited Analysis: Authors propose three individual losses but fail to analyze the impact of individual noise in protection. Further, there is no evidence to support why DiT-based noise encoders should be preferred over traditional ViT or CNNs. From the way the article is written, it feels like the authors stumbled upon a framework that works for protecting the images but did not analyze how it can be improved trivially (like changing architectures, modifying weights of individual losses, etc.). This makes the authors' proposal shallow and not insightful for the community.