Robust Watermarking Using Generative Priors Against Image Editing: From Benchmarking to Advances

Question 2: What is the motivation for using SDXL-Turbo as the generative prior for watermark encoding? If it is just to avoid multi-step sampling, there should be lots of one-step generative models to choose from, for example, the SDXS [2].

The motivation for using one-step diffusion models is to assess whether a powerful generative prior enhances watermarking performance. We chose SDXL-Turbo as the first choice because we believe it is a highly representative one-step generative model in all, and our work currently takes the initial steps to verify whether a commonly used generative prior is beneficial for watermarking.

We appreciate that you suggested using other potentially better one-step or few-step models, such as SDXS, LCM, InstaFlow, SD3.5, SwiftBrush, DMD, UFOGen, or Flux.Shenell. Although our SDXL-Turbo-based model has already advanced watermarking performance, we believe investigating alternative models with better prior performance could be a valuable direction for future research!

We hope this addresses any concerns you may have. If you feel further improvements are necessary, we would be happy to make additional revisions!

[1] Hu, Yaosi, Chong Luo, and Zhenzhong Chen. "Make it move: controllable image-to-video generation with text descriptions." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.

[2] Song, Yuda, Zehao Sun, and Xuanwu Yin. "SDXS: Real-Time One-Step Latent Diffusion Models with Image Conditions." arXiv preprint arXiv:2403.16627 (2024).

We sincerely appreciate your recognition of our focus on the important and underexplored area of image watermark robustness against editing, the comprehensiveness of our benchmark, the novelty and effectiveness of our method, and the overall quality of our writing!

We will address all the concerns point by point.

Weakness 1 & Question 1: What is the reason for choosing only these four types of image editing methods (image regeneration, global editing, local editing, and image-to-video generation) to evaluate the image watermarking robustness, against image editing?

Thank you for your question. We provide an explanation below and have also included it in Appendix G.1 (highlighted in red) to facilitate readers’ understanding.

These four types of editing encompass the majority of editing needs. Image editing can be broadly categorized into global and local editing, as user edits typically affect either the entire image or a specific part of it.

• Global editing involves altering most of an image's pixels while maintaining its overall layout, components, or semantics. Techniques such as style transfer, cartoonization, image translation, and scene transformation fall under this category and produce similar effects. For example, using prompts like "turn it into Van Gogh style" or "convert it into a sketchy painting" in an image editing model can effectively achieve style transfer.

• Local editing, on the other hand, refers to modifications applied to specific elements, semantics, or regions within an image. This category includes image inpainting, image composition, object manipulation, attribute manipulation, and so forth.

• While image regeneration and image-to-video generation are not strictly considered forms of image editing, they can be used to create similar digital content while removing watermarks, thereby posing a threat to copyright protection. For this reason, we have included them in our benchmark.

Weakness 2: Only one image-to-video generation method is included in the benchmark. The robustness against other image-to-video generation methods such as [1] is not evaluated.

Thank you for highlighting another image-to-video generation model (MAGE [1]). Following your recommendation, we have conducted experiments on it.

Since MAGE [1] cannot process natural images, we are unable to use our benchmark images for evaluation. Instead, we utilize the CATER-GEN-v2 dataset proposed in their paper, which is a more complex version compared to CATER-GEN-v1. This dataset contains three to eight objects per video, with each object having four attributes randomly selected from five shapes, three sizes, nine colors, and two materials.

We add watermarks to 1,000 images from the CATER-GEN-v2 dataset and use the MAGE+ model to perform text-image-to-video (TI2V) generation, producing 10-frame videos. For watermark detection, we analyze frames 2, 4, 6, 8, and 10. The average detection accuracies across 1,000 videos are presented in the table below and are also included in Appendix G.4 (Table 6) (highlighted in red).

Interestingly, we found that the detection accuracies of most watermarking models are higher compared to when testing with the SVD. We attribute this to the simplicity of the dataset: the background remains mostly unchanged without significant camera motion, and only a few objects move while most remain static. This makes detection easier than in the SVD case, which typically involves camera motion effects. In this case, VINE still outperforms other watermarking models.

We sincerely appreciate your recognition of the significance of our benchmark and its benefits for future research, as well as your acknowledgment that our paper is clearly articulated and well-supported!

We will address all the concerns point by point.

Weakness 1: The paper explains the reasons behind the watermarking algorithm's resistance to image editing from the perspective of the frequency domain. It notes that the watermarking methods exhibiting high robustness against image editing in certain scenarios display prominent patterns in the low-frequency bands, which aligns with the general understanding of watermark robustness. However, the paper primarily focuses on the robustness of watermarking methods against image editing techniques based on generative models. Therefore, summarizing the unique effects of such image editing techniques on the watermark is more meaningful.

Thank you for your valuable suggestions.

In response to your feedback, we added a new Figure 8 in Appendix B (in the new version of our submission) to illustrate the impact of different image editing methods on various watermark patterns within the frequency domain. As shown, the frequency patterns of VINE-R, VINE-B, MBRS, and StegaStamp are less affected compared to their original patterns (shown in Figure 7) than those of other watermarking methods. We believe this provides interesting and insightful information for our readers, and we are truly grateful for your input in enhancing the clarity of our work.

Additionally, we retain Figure 3 to help readers better understand the impact of image editing in a more disentangled manner, as it can be challenging to isolate the effects on low-, mid-, and high-frequency bands solely through the visual representation in Figure 8.

If any aspects remain unclear or if further clarification of other figures is needed, we would be happy to make additional improvements!

Weakness 2 & Question 1: Figure 6 in the appendix shows that VINE exhibits higher brightness in the central region, providing evidence for why the proposed watermarking method demonstrates strong robustness against image editing. If the author can thoroughly elucidate the principles underlying this phenomenon, it may address the previously mentioned issue of "a disconnect between the author's analysis of watermark robustness and the design of the watermark model."

Thank you for your insightful question. The observation that VINE exhibits higher brightness in the low-frequency bands is indeed interesting, though this was not the motivation behind our design choices. Allow us to walk you through our design and analysis process.

Our two key design elements (surrogate layer & generative prior adaptation) are based on the two following observations: (1) Image editing and image blurring exhibit similar frequency characteristics. (2) Image watermarking can be viewed as a form of conditional generation, where a generative prior can enhance image quality by making watermarks less visible. The surrogate layer enhances our model's robustness to image editing, while the generative prior improves the quality of the encoded images. Therefore, our design is grounded in these analyses.

Regarding the intriguing correlation you mentioned—the robustness of our watermarking model against image editing being highly positively correlated with pattern intensity in the low-frequency region—this emerged from our training process rather than driving our design decisions.

In summary, this finding is a byproduct of our design rather than a deliberate motivation of it. We did not intend to concentrate the watermark pattern in the low-frequency region. In fact, when we attempted to do so by introducing an additional frequency loss, it resulted in poorer performance, which we discuss in the next question. This finding could offer valuable insights for future research aimed at achieving a better balance between robustness and encoded image quality.

We have included this discussion in Appendix B of the new version of our submission (highlighted in green) to enhance readers' understanding. Thank you for helping to make our work more complete!

Question 2: (1) The experimental results demonstrate that the proposed watermarking method, VINE, significantly enhances robustness against various image editing techniques. Has the author considered using representative image editing as an attack template, incorporating the associated attack loss as one of the objective functions during the training phase? (2) Alternatively, how might integrating the specific effects of image editing on watermarks into the design of the watermarking model influence the results of the watermarking algorithm?

(1) We have considered incorporating a representative image editing method into our noise layer during training. This approach has led to the development of our VINE-R variant, which is fine-tuned from VINE-B by integrating the Instruct-Pix2Pix pipeline during the training process.

(2) Image editing tends to remove patterns present in high-frequency bands. Thus, in our preliminary design, we expected that integrating the effects of image editing into our design would involve deliberately forcing the pattern to appear in the low-frequency band by adding an additional loss in the frequency domain. This point is particularly intriguing.

In our preliminary experiments, we attempted to position the pattern in the low-frequency region by ensuring that the high-frequency bands of the Fourier spectra of the watermarked and original images are identical. However, this loss did not lead to a significant improvement in robustness and resulted in a substantial decrease in the quality of the encoded images.

We believe this outcome occurs because injecting all information into the low-frequency bands adversely affects the image quality. In contrast, allowing the model the flexibility to choose which frequency bands to inject the watermark helps balance robustness and image quality. Therefore, we decided not to include this frequency loss in our final design.

Question 3: In the experimental section, some of the differences between the subjective experimental results are difficult to discern visually. The author could consider selecting a subset of images and enlarging specific regions to facilitate reader comprehension.

Thank you for your valuable suggestion! Based on your reference to the "subjective experimental results," we believe you are referring to Figure 15 in the first version of our submission. In the revised submission, this figure has been renumbered to Figure 17 and enhanced with two additional columns that display enlarged central $40\times40$ regions of the watermarked and residual images, respectively. If anything else is unclear or if other figures require further clarification, we are happy to make additional improvements.

We hope this addresses the concerns you may have and are always available for further discussion. We deeply appreciate the time you have taken to engage with our work and share your valuable insights.

Question 5: Is VINE only work on the Image Editing task? What about other common watermarking tasks?

Thank you very much for highlighting this important question! We fully agree with the significance of this aspect of our method. VINE also remains robust against common degradations such as JPEG compression, Gaussian noise, contrast modifications, brightness adjustments, and more. For a comprehensive overview of its performance, we kindly direct your attention to Figure 9 in the revised version of our submission (previously Figure 7). We sincerely hope this helps address your concerns, and we are always available to discuss this further if needed. Your feedback is greatly appreciated!

If the reviewer believes further improvements are needed, we would be happy to make additional revisions!

We sincerely appreciate your recognition of our method as both reasonable and effective, as well as the solidity of our validation!

We will address all the concerns point by point.

Weakness 2 & Question 1: Although the watermarking against Image Editing is interesting and novel, I cannot get the value of this task. Can you elaborate the perspective of this task?

The Importance of Watermarking Images Against AI-Powered Image Editing

Invisible watermarks enable creators to assert ownership of their digital content. If a watermark can be easily removed through minor alterations (such as local editing) or visually imperceptible changes (like image regeneration), malicious users could remove or add a small element (such as a dog) to an artwork and publish it without proper attribution, which could lead to copyright issues. Such non-robust watermarks pose challenges for ownership assertion.

In contrast, editing-resistant watermarks can enhance security by embedding links that trace the source of leaks or unauthorized distributions back to the original distributor. For example, photographers and media companies can monitor the usage of their images, helping to identify unauthorized editing or redistribution.

Weakness 1 & Question 2: (1) This paper lacks the validation of hypotheses in Line 249; (2) The author hypothesizes that a powerful generative prior can facilitate embedding information more invisibly while enhancing robustness (Line 249). Why hypothesize that? What are the assumptions based on?

1. The validation of the hypothesis. The validation is provided in the ablation study (Table 2 Config H). In Table 2, Config H is trained with randomly initialized weights instead of pretrained ones while retaining all other settings from Config G. Comparing Config H with Config G, it can be seen that the pretrained prior knowledge is beneficial for image quality, and thus it helps embedding information more invisibly.

2. Rationale for the Hypothesis. This hypothesis is based on the expectation that the output of the watermarking process should visually resemble the input image, which is a natural, artifact-free image. Powerful generative priors, particularly large-scale diffusion models, have the capability to generate images within this natural, artifact-free manifold. SDXL-Turbo, distilled from its teacher model SDXL, inherits its generative capability. Thus, we expect that initializing with this strong generative prior will benefit the training process and significantly weaken the artifacts in watermarked images.

We hope this addresses the concerns you may have on this matter and remain available for further discussion!

Weakness 3: The watermarking pattern existing in high-frequency bands after image blurring is not a new discovery. However, the author spends too much text on it.

Thank you for your valuable suggestions. In response to your feedback, we have removed lines 212-213 of the initial version of our submission to reduce the discussion on blurring. Following your recommendation, now the emphasis has been shifted to the similarities between the frequency characteristics of image editing and blurring. This is a new discovery that we believe would be important and insightful for readers.

Question 3: What is the purpose of finetuning VINE-B to VINE-R using Instruct-Pix2Pix? (Line 323)

The purpose of incorporating Instruct-Pix2Pix, a representative editing model, into the finetuning process is to further boost the robustness against image editing. This integration is performed during finetuning because attempting to include the editing model at the initial training stage using a straight-through estimator (STE) results in convergence failures.

In fact, even without this finetuning step, VINE-B already outperforms the baseline models overall (see Figure 1). We are interested in determining whether a well-trained watermarking model can be finetuned with an editing model through STE to further improve robustness. Therefore, we implement this variant.

Question 4: Why is the resolution not unified? (Line 1042)

Thanks for your question. All baseline models were evaluated using their official checkpoints, which were trained at various resolutions. Table 3 presents the performance of these methods at their original training resolutions (thus not unified). The purpose of Table 3 is to demonstrate that scaling the resolution does not affect the quality of the encoded images. The results after scaling to the unified resolution are shown in Table 1.

Additionally, Figures 9 and 10 of the new version of our submission (or Figures 7 and 8 of the 1st version of our submission) illustrate that resolution scaling does not impact detection robustness. These evaluations were conducted to determine whether resolution scaling affects the original performance, as it is necessary to scale images to a unified 512 × 512 resolution before inputting them into the editing models.

Thank you for your response. I would like to keep the previous score.

We sincerely appreciate your recognition of the innovation in our task and the use of generative priors, as well as your acknowledgment of the comprehensiveness of our evaluation benchmark!

We will address all the concerns point by point.

Weakness 1: TreeRing, Gaussian Shading, and RingID, which add watermarks in the frequency domain of the initial noise, are generally considered robust against image editing (e.g., prompt2prompt) and regeneration. This paper lacks this crucial comparison. If these methods are also robust to image editing, the contribution of this paper may be diminished.

Thank you for your insightful questions. TreeRing, Gaussian Shading, and RingID are well-known in-generation watermarking techniques designed for watermarking generated images. However, these methods are not applicable to real images and therefore cannot be used for copyright protection of authentic photographs.

Experiments are conducted to verify this. Specifically, we apply DDIM/DPM inversion techniques to extract the initial noise from a real image, add a watermark using these methods, and then invert back to obtain the image, the resulting image would differ significantly from the original one, undermining the photographer’s intent to protect their work. The visual results are shown in Figure 6 of the new version of our submission.

Consequently, while these methods are highly effective for watermarking generated images, they fall outside the scope of our study. All of our baseline methods are capable of adding watermarks to real images. We have also included a discussion of these well-known in-generation watermarking methods in Appendix A.1. We kindly invite you to review that section, and we hope this addresses the concerns you may have on this matter. Your feedback is greatly appreciated!

Question 1: I have doubts about the results in Figure 5(a). The experimental results show that 250-step noise in image regeneration can significantly disrupt the watermark（bit acc). Does this mean that global image editing (e.g., SDedit, prompt2prompt) with 250 steps can also completely remove the watermark?

We apologize for any confusion.

The key point is that an imperfect bit accuracy of approximately 80% does not indicate that the watermark has been compromised. Even if an image achieves an extracted bit accuracy of around only 80%, it still has a very high probability—potentially exceeding 99%—of being flagged as watermarked in the statistical test. Therefore, the 250-step noise applied during image regeneration does not disrupt the watermark.

The following provides a detailed introduction to the statistical test.

Let $\boldsymbol{w} \in \\{0,1\\}^k$ be the $k$-bit ground-truth watermark. The watermark $\boldsymbol{w}^\prime$ extracted from a watermarked image $\boldsymbol{x}_w$ is compared with the ground-truth $\boldsymbol{w}$ for detection. The detection statistical test relies on the number of matching bits $M(\boldsymbol{w}, \boldsymbol{w}^\prime)$: If

then the image is flagged as watermarked. Formally, we test the statistical hypothesis $H_1$: '$\boldsymbol{x}$ contains the watermark $\boldsymbol{w}$' against the null hypothesis $H_0$: '$\boldsymbol{x}$ does not contain the watermark $\boldsymbol{w}$'. Under $H_0$ (i.e., for original images), if the extracted bits $\boldsymbol{w} = \\{w_1^\prime, w_2^\prime, \cdots, w_k^\prime\\}$ (where $w_i^\prime$ is the i-th extracted bit) from a model are independent and identically distributed (i.i.d.) Bernoulli random variables with the matching probability $p_o$, then $M(\boldsymbol{w}, \boldsymbol{w}^\prime)$ follows a binomial distribution with parameters $(k, p_o)$. This assumption is verified by [1].

The false positive rate (FPR) is the probability that $M(\boldsymbol{w}, \boldsymbol{w}^\prime)$ takes a value bigger than the threshold $\tau$ under the null hypothesis $H_0$. It is obtained from the CDF of the binomial distribution, and a closed-form can be written with the regularized incomplete beta function $I_p(a,b)$ [1]:

where under $H_0$ (i.e., images without watermark $\boldsymbol{w}$), $p_o$ should ideally be close to 0.5 to minimize the risk of false positive detection.

Similarly, the true positive rate (TPR) represents the probability that the number of matching bits exceeds the threshold $\tau$ under $H_1$, where the image contains the watermark. Thus, the TPR can be calculated by:

where under $H_1$ (i.e., images with watermark $\boldsymbol{w}$), $p_w$ should ideally be high enough (e.g., exceeding 0.8) to ensure the effectiveness of a watermarking model.

To further demonstrate that neither high bit accuracy nor AUROC alone guarantees a high TPR at a low FPR, consider the following example. Suppose we have a 100-bit watermarking model with a threshold $\tau$ of 70 to determine whether an image contains watermark $\boldsymbol{w}$. If the model extracts bits from watermarked images with a matching probability $p_w = 0.8$ and from original images with a matching probability $p_o = 0.5$, the resulting FPR would be $1.6 \times 10^{-5}$ and the TPR would be 0.99. In this scenario, even though the bit accuracy for watermarked images is not exceptionally high (e.g., below 0.9), the model still achieves a high TPR at a very low FPR. In contrast, if another model has $p_w = 0.9$ and $p_o = 0.7$, achieving the same FPR would require setting the threshold $\tau$ to 87. Under these conditions, the TPR would only be 0.8. This example demonstrates that high bit accuracy for watermarked images does not necessarily ensure a high TPR when maintaining a low FPR. Therefore, relying solely on bit accuracy or AUROC may not be sufficient for achieving the desired performance in watermark detection.

We have included this information in Appendix C in the new version of our submission (highlighted in blue) to enhance reader understanding. We kindly invite you to review it and hope that it addresses the concerns you may have regarding this matter. If you believe further improvements are needed, we would be happy to make additional revisions. Thank you for helping to make our work more complete!

[1] The stable signature: Rooting watermarks in latent diffusion models.

Dear Reviewer swi8,

Thank you very much for your positive feedback on our revisions! We are delighted to hear that your concerns have been essentially addressed.

As the current scoring system does not include an option for a score of 7, we kindly ask if you would consider adjusting your score to 8 in the system, if possible.

Should you have any further questions or suggestions, please do not hesitate to let us know. We greatly appreciate your valuable contributions throughout the review process!

Warm regards,

The Authors

We sincerely appreciate your recognition of our paper’s well organization and rigorous evaluation!

Weaknesses 1: EditGuard is primarily designed for editing detection, not robust watermarking, and it was not tested with its most robust configuration. This impacts the fairness of the evaluation, as EditGuard’s focus and strengths differ from VINE’s intended use.

We appreciate your feedback and have updated Section 4.3 (lines 470-472) in the new submission accordingly. A new remark, highlighted in purple, has been added to clarify the primary focus of EditGuard's design. For your convenience, the remark is also provided below.

EditGuard is not designed for robust watermarking against image editing, as it is trained with mild degradation. Instead, it offers a feature for tamper localization, enabling the identification of edited regions.

If you believe further improvements are needed, we would be happy to make additional revisions!

Thank you for your question. Here are some practical examples of robust invisible watermarking in use:

• IMATAG: They aim to protect businesses from the unauthorized use of their images and videos. Visit their homepage or check out their X profile for more information. In short, by embedding their robust invisible watermarks and utilizing their detector, IMATAG can (1) identify who is leaking your visual content; (2) monitor who is using your visual content; (3) verify the authenticity of your visual content.

• Adobe Integration: IMATAG is also integrated with Adobe software here.

• Adobe Content Authenticity: It is designed to help creators protect and receive attribution for their work with Content Credentials. Content Credentials combine digital fingerprinting, invisible watermarking and cryptographically signed metadata, helping to ensure that Content Credentials remain intact and verifiable across the digital ecosystem [blog link].

• Google SynthID: Google uses its SynthID for Vertex AI customers.

Please note that these technologies are not open-sourced. While they can protect digital content against common transformations, their robustness against AI-powered image editing is underexplored.

If a malicious user uses generative models to remove or add a small element (such as a dog) to an artwork and publishes it without proper attribution, it could lead to copyright issues.