A Transfer Attack to Image Watermarks

We sincerely appreciate your constructive comments and insightful suggestions. Please find our responses below.

Weakness 1: While authors provide some theoretical analysis, I think the current results are not sufficient to be claimed as significant. The reasons are as follows: (1) It is in doubt whether those independence assumptions will hold. As $\delta$ is obtained by solving Eq. (3), which optimizes the chances that $m$ decoders will extract carefully designed watermarks from an image perturbed by $\delta$, it is likely that coordinates of decoders' output are correlated. The authors may want to provide additional evidence to validate their assumptions.

Response: Thank you for your insightful suggestions. To support this assumption, we tested the independence of the watermarks decoded by surrogate models from perturbed images generated by our attack. Specifically, we randomly sampled 100 pairs of surrogate watermarking models from the 100 surrogate models in our experiments. For each bit of the watermark and pair of surrogate models, we performed the Chi-square independence test with a confidence level of 0.95 using the decoded watermarks from the 1,000 perturbed images. Then we calculated the percentage of the 100 pairs of the surrogate models that passed the independence test for each bit.

The results, shown in the above table and Figure 14 in the Appendix of our revised paper, reveal that nearly 90% of pairs are independent for each bit. This finding provides strong evidence supporting our assumption.

Weakness 2: The analysis only applies to inverse-decode.

Response: Thank you for your valuable comment. We would like to mention that our theoretical analysis can be straightforwardly extended to Random-Different, as all the assumptions used to derive the bounds for Inverse-Decode, particularly the independence assumption, still hold for Random-Different. Given the randomness of watermark selection, Random-Different can effectively be viewed as the roughly 50% inverse version of Inverse-Decode. If needed, we can provide further explanations and a detailed theoretical analysis for Random-Different.

Weakness 3: The bounds given by the theorems are not available in practice. Recall that this paper considers the setting where $T$ and its output is unknown to the attacker. In the meanwhile, the parameters including $a$, $b$, $\beta$, $p$ involve the unknown target decoder $T$ and unknown data distribution.

Response: Thank you for your valuable comment. We acknowledge that calculating our theoretical bounds requires more information than attackers typically possess. However, the primary purpose of our theoretical results is to provide insights into why our transfer attack succeeds and how effectively it can perform. These bounds are not meant to be directly usable by attackers but rather serve to analyze the underlying mechanisms and potential of our approach. Thus, the need for additional knowledge is acceptable in the context of theoretical analysis.

Weakness 4: The empirical results are impressive, and it raises my question why it significantly outperforms SOTA methods? Could the author provide any justification and insight for the superior performance of the proposed method?

Response: Thank you for your valuable question. Most other transfer-based attacks simply apply adversarial examples to the watermark and craft perturbations based on classifiers, without accounting for the unique properties of watermarking. Consequently, these methods achieve limited transferability. A more detailed discussion can be found in Section 2.2 in our paper. For DiffPure, it utilizes a diffusion model to purify the image by removing the watermark. However, the diffusion process significantly alters the watermarked image by introducing large perturbations. As a result, DiffPure achieves limited success when the $\ell_{\infty}$ norm of the perturbation or the SSIM between the perturbed and watermarked image is constrained, as demonstrated in Figure 11 of our revised paper. In contrast, we observe that our transfer attack consistently outperforms all baseline methods in terms of evasion rate while maintaining comparable image quality (SSIM).

Weakness 5: Since the proposed method has to train a bunch of surrogate models, will it be computationally expensive? Also, every time this algorithm has to solve Eq. (3), which may also be expensive. The authors may want to add a table comparing the FLOPs and time used by different methods.

Response: Thank you for your helpful suggestions. To analyze the computational cost of our transfer attack and compare it with existing transfer attacks, we have added a table summarizing the training and inference times for our attack and six baseline methods used in our experiments, all evaluated on the same device.

Training time refers to the duration required to train a surrogate model, while inference time denotes the time needed to optimize the perturbation for an image. All measurements were conducted on a single NVIDIA RTX-6000 GPU. Specifically, $\Delta_{\text{RN18}}$ represents the pre-training time for ResNet-18~\cite{} on ImageNet, and $\Delta_{\text{Diff}}$ corresponds to the training time for the unconditional diffusion model used by DiffPure. The target watermarking model in our experiments is ResNet, and all perturbations were optimized under the constraint $SSIM \geq 0.9$. The evasion rates for each method are provided in the corresponding data points in Figure 11 in the Appendix.

The results indicate that our attack requires an acceptable training time, as the training of multiple surrogate models can be parallelized. Additionally, the trained surrogate models can be reused to attack multiple target watermarking models, further improving overall efficiency. While the inference time of our attack exceeds that of five out of six baseline methods, it remains feasible for attackers with sufficient computational resources. Notably, the optimization of $\delta$ in Eq. (3) for different watermarked images can be parallelized, ensuring that the overall computational cost remains reasonable. This discussion has been incorporated into Section 7 in our revised paper.

Weakness 3: Another weakness of this paper is that the transfer attack primarily focuses on learning-based watermarks, neglecting non-learning-based watermark variants that may exhibit greater robustness than the baselines discussed here, such as variants of Tree-Ring [1][2], etc. Besides, can the proposed method attack the 0-bit watermark?

Response: Thank you for pointing out this. To include the Tree-Ring variants [1][2], we evaluated their robustness against common perturbations and presented the results in the below table (comparing evasion rates of non-learning-based watermarking models and HiDDeN under Gaussian noise with varying standard deviations) and Figure 6(c) in our revised paper.

Our findings demonstrate that non-learning-based methods are less robust compared to the learning-based methods we tested. Specifically, non-learning-based methods exhibit a noticeable rise in evasion rate even with relatively small perturbations (consistent with prior works). In contrast, learning-based methods maintain a low evasion rate under the same conditions. For this reason, our study primarily focuses on learning-based watermarks. However, we have also added a discussion of non-learning-based watermarks in Section 7 of our paper.

[1] RingID: Rethinking Tree-Ring Watermarking for Enhanced Multi-Key Identification

[2] Gaussian Shading: Provable Performance-Lossless Image Watermarking for Diffusion Models

Weakness 4: The attack cost is significantly increased with complex architectures and longer watermarks. This poses a hindrance to the practical implementation of transfer attacks involving high-capacity watermarks [3].

Response: Thank you for pointing out this. To address it, we have added a table comparing the attack cost of our method with that of other baseline attacks.

The results indicate that our attack requires an acceptable training time, as the training of multiple surrogate models can be parallelized. Additionally, the surrogate models only need to be trained once. While the inference time of our attack is longer than that of five out of six baseline methods, it remains feasible for attackers with adequate computational resources. We also discussed the implications of the high-capacity watermarks [3] on computational cost in Section 7 of our revised paper.

[3] A Robust Image Watermarking System Based on Deep Neural Networks

We sincerely appreciate your constructive comments and insightful suggestions. Please find our responses below.

Weakness 1: While I acknowledge the authors have assumed that the attackers should have sufficient computational resources, the main concern of this paper is the computational cost. The costs associated with both surrogate models and perturbation discovery are large which prevents the practical usage of this method.

Response: Thank you for your insightful suggestions regarding the analysis of computational cost. To evaluate the computational cost of our attack, we include the following table summarizing the training and inference times for our attack and six baseline methods used in our experiments.

Training time refers to the duration required to train a surrogate model, while inference time denotes the time needed to optimize the perturbation for an image. All measurements were performed on a single NVIDIA RTX-6000 GPU. Specifically, $\Delta_{\text{RN18}}$ represents the pre-training time of ResNet-18 on ImageNet, and $\Delta_{\text{Diff}}$ corresponds to the training time of the unconditional diffusion model employed by DiffPure. The target watermarking model in our experiments is a ResNet, and all perturbations were optimized under the constraint $SSIM \geq 0.9$. The evasion rates for each method are presented in Figure 11 in the Appendix of our revised paper.

For our method, the training time is 6.5 hours. Although extending this to multiple surrogate models increases the total duration, these models can be trained in parallel across GPUs and reused for various target watermarking models, which keeps the overall overhead manageable. While our inference time exceeds that of five out of six baseline methods, it remains feasible for attackers with adequate computational resources. This analysis has been included in Section 7 of our revised paper.

Weakness 2: For the transfer attack to a specific architecture such as ResNet-18, did you use a similar architecture in your surrogate models, for instance, ResNet-X? Besides, have you considered attacks with totally different architectures, such as transformer-based watermark models?

Response: Thank you for raising this valuable question. For our surrogate models, we used a vanilla CNN with seven convolutional layers, which is entirely different from ResNet-18, as detailed in our Appendix. We have also considered watermarking models with totally different architectures. For instance, the StegaStamp method in our experiments employs a spatial transformer network, while the Stable Signature method utilizes a variational autoencoder (VAE) within the diffusion model for watermarking. Moreover, in Section 7, we also applied our transfer attack to DWT-DCT, a non-learning-based watermarking method. DWT-DCT has a qualitatively different architecture. Specifically, unlike the learning-based approaches examined in our experiments, DWT-DCT is a non-learning-based technique used by Stable Diffusion. This method employs the Discrete Wavelet Transform (DWT) to decompose an image into frequency sub-bands, applies the Discrete Cosine Transform (DCT) to blocks within selected sub-bands, and embeds the watermark by modifying specific frequency coefficients. The watermarked image is then reconstructed using inverse transforms.

Dear Reviewer,

Thank you once again for your constructive comments and valuable feedback! We have thoroughly addressed your suggestions by conducting new experiments and revising our paper. Could you please kindly review our responses and let us know if you have any further comments or suggestions?

We would greatly appreciate it if you could reconsider your score in light of our new experimental results and the improvements made in the paper.

Thank you for your time and consideration!

Dear Reviewer,

We wanted to kindly remind you about the responses and updates we shared regarding your constructive comments and valuable feedback on our paper. We have addressed your suggestions by conducting new experiments and revising the manuscript accordingly.

Could you please review our responses at your earliest convenience? If you have any further comments or suggestions, we would be more than happy to address them. Additionally, we kindly ask you to reconsider your score in light of the new experimental results and the improvements made in the paper.

As the discussion phase deadline is approaching, we greatly appreciate your time and attention to this matter.

Thank you again for your thoughtful feedback and for considering our revised work.

We sincerely appreciate your constructive comments and insightful suggestions. Please find our responses below.

Weakness 1: To ensure high image quality, the L infinity bound on perturbations should be kept low, as higher values (e.g., 0.2 as shown in Figures 3 and 4) can noticeably degrade image quality, reducing practicality. For a more realistic assessment, the paper should include comparisons using a set of small enough L infinity bound (such as 0.03, 0.05) bounds to demonstrate the proposed method’s effectiveness under conditions that minimize visible impact.

Response: Thank you for highlighting this. We did new experiments based on the suggestion. The below table and Figure 11 in the Appendix of our revised paper presents the evasion rates of all attacks across a wide range of $SSIM$ constraints.

Our results show that our transfer attack consistently outperforms all baseline attacks. This highlights the effectiveness of our attack compared to existing ones.

In this table, we focus on SSIM rather than the $\ell_{\infty}$ norm because SSIM provides a perceptually meaningful evaluation of image quality by accounting for structural similarity rather than isolated pixel differences. While high $\ell_{\infty}$ values in our method are typically localized to a small number of pixels with minimal perceptual impact, SSIM better reflects the overall quality and visual coherence of the image, making it a more suitable metric for this comparison.

Weakness 2: If Section 5 is one of the paper’s main contributions, it would be better placed in the main text rather than in the appendix.

Response: Thank you for your valuable and helpful suggestions. In the revised version of our paper, we have moved the main theorem, which establishes the upper and lower bounds on the transferability of our transfer attack—our primary theoretical contribution—back to the main body.

Dear Reviewer,

Thank you once again for your constructive comments and valuable feedback! We have thoroughly addressed your suggestions by conducting new experiments and revising our paper. Could you please kindly review our responses and let us know if you have any further comments or suggestions?

We would greatly appreciate it if you could reconsider your score in light of our new experimental results and the improvements made in the paper.

Thank you for your time and consideration!

Dear Reviewer,

We wanted to kindly remind you about the responses and updates we shared regarding your constructive comments and valuable feedback on our paper. We have addressed your suggestions by conducting new experiments and revising the manuscript accordingly.

Could you please review our responses at your earliest convenience? If you have any further comments or suggestions, we would be more than happy to address them. Additionally, we kindly ask you to reconsider your score in light of the new experimental results and the improvements made in the paper.

As the discussion phase deadline is approaching, we greatly appreciate your time and attention to this matter.

Thank you again for your thoughtful feedback and for considering our revised work.

Thank you again for your constructive comments and promise to raise the score!

Dear Reviewer xUbX,

Thank you once again for your review and constructive feedback. Your pre-rebuttal rating was 5. We’re delighted to know that you found our rebuttal and new experiments satisfactory, and we appreciate your kind promise to raise your rating to be 6. As a gentle reminder, we kindly ask that you update your rating at your convenience to ensure your post-rebuttal evaluation is accurately reflected in the system. Thank you again for your time and support!

Weakness 3: The effectiveness of the proposed method as a "no-box" attack remains unclear, specifically regarding the transferability to different watermarking methods.

Response: Thank you for highlighting this. Non-learning-based watermarks lack robustness even against common perturbations, as shown in the below table (comparing evasion rates of non-learning-based watermarking models and HiDDeN under Gaussian noise with varying standard deviations) and Figure 6(c) of our revised paper.

For this reason, our study primarily focuses on learning-based watermarks. Moreover, Figure 15 shows that our attack is also effective for non-learning-based methods. Specifically, the non-learning-based watermarking method used in Figure 15 is the DWT-DCT technique deployed by Stable Diffusion. Unlike the learning-based methods examined in our experiments, DWT-DCT is a non-learning-based watermarking approach. It leverages the Discrete Wavelet Transform (DWT) to decompose an image into frequency sub-bands, applies the Discrete Cosine Transform (DCT) to blocks within selected sub-bands, and embeds the watermark by modifying specific frequency coefficients. The watermarked image is subsequently reconstructed using inverse transforms.

To clarify this distinction, we have added further discussion on non-learning-based watermarks in Section 7.

We sincerely appreciate your constructive comments and insightful suggestions. Please find our responses below.

Weakness 1: No analysis of the computational cost in comparison with existing transfer attacks, which should be discussed clearly.

Response: We appreciate your suggestion to analyze the computational cost of our transfer attack. To address this, we include the following table, summarizing the training and inference times for our attack alongside six baseline transfer attacks evaluated in our experiments.

Training time refers to the duration required to train a surrogate model, while inference time denotes the time needed to optimize the perturbation for an image. All measurements were conducted on a single NVIDIA RTX-6000 GPU. Specifically, $\Delta_{\text{RN18}}$ represents the pre-training time of ResNet-18 on ImageNet, and $\Delta_{\text{Diff}}$ refers to the training time of the unconditional diffusion model used by DiffPure. The target watermarking model is a ResNet, and all perturbations are optimized under the constraint $SSIM \geq 0.9$. The evasion rates corresponding to each method are provided in Figure 11 in the Appendix of our revised paper.

For our method, the training time is 6.5 hours. Although extending this to multiple surrogate models increases the overall duration, these models can be trained in parallel across GPUs and reused to attack different target watermarking models, which keeps the overhead manageable. While our inference time is longer than that of five out of six baselines, it remains acceptable for attackers with adequate computational resources. A discussion of this computational cost analysis has been added to Section 7 of our revised paper.

Weakness 2: The comparison of the proposed attack with existing transfer attacks is somewhat inconclusive.

Response: Thank you for highlighting this. As you suggested, we conducted an experiment comparing all attacks across different perturbation constraints. The below table and Figure 11 in the Appendix of our revised paper presents the evasion rates of all attacks across different $SSIM$ constraints when the target watermarking model is ResNet. Our results show that our transfer attack consistently outperforms all baseline attacks.

Dear Reviewer,

Thank you once again for your constructive comments and valuable feedback! We have thoroughly addressed your suggestions by conducting new experiments and revising our paper. Could you please kindly review our responses and let us know if you have any further comments or suggestions?

We would greatly appreciate it if you could reconsider your score in light of our new experimental results and the improvements made in the paper.

Thank you for your time and consideration!