Watermarking Autoregressive Image Generation

We thank the reviewer for their feedback. We are glad that they recognize our work as novel and timely and consider our solutions to be effective. We reply to the question raised by the reviewer below.

Q1: Why does the paper not include comparisons to concurrent works [1] and [2]?

We remark that it was not possible for us to include these in our submission as both of these works were first made public after the NeurIPS submission deadline. We however appreciate the references and will look at them in detail. We certainly agree that relevant concurrent work should be prominently discussed, and will add a detailed paragraph about all such works to the Related Work section in the next revision.

As this resolves the only point raised by the reviewer as a weakness of our work and the remainder of the review is positive, we respectfully ask the reviewer to consider updating their score.

We thank the reviewer for their detailed feedback and for recognizing our work as well-written and clear, our experiments as extensive and convincing, and our multimodal and audio experiments as valuable extensions. We respond to the reviewer’s questions below, numbered Q1-Q4. We kindly ask the reviewer to let us know if their concerns were addressed, and we are happy to continue the discussion further if there are follow-up questions.

Q1: Can you quantify the effect of the watermark on diversity?

We note that FID, which we report in the paper, captures both fidelity and diversity. However, prompted by this question, we also measure diversity via Recall, following [R1] and in the same setup as in Sec. 4. We observe no significant degradation of Recall for both Chameleon and Taming, reinforcing our conclusion that our method does not harm generation. We provide complete results below and will include these in the next revision.

[R1] Kynkäänniemi et al., 2019, “Improved Precision and Recall Metric for Assessing Generative Models”

Q2: Can you compare PSNR between original and finetuned decoders on the same tokens?

We agree that a more fine-grained quality analysis on top of FID is beneficial, and for this reason already report PSNR in Appendix F.4. We will make this experiment more prominent in the next revision of the paper, adding explicit pointers from the main text. As seen there, PSNR remains high for both models: After full RCC finetuning mean PSNR between token sequences decoded with the original and finetuned decoder is 49.7 for Taming and 48.0 for Chameleon. This demonstrates that decoder finetuning introduces minimal perceptual changes.

Q3: Can you clarify how synchronization improves robustness to crops?

Certainly. As detailed in Appendix E.2 and illustrated in Figure 8 in the appendix, we consider crops from the top-left corner of the image followed by upscaling to the generative model’s original generation size. As each image token loosely corresponds to a region in the image, this can result in completely different tokens, erasing our token-level watermark.

Resolving this via RCC finetuning is not possible, as it would teach the encoder to assign the same token ID to fundamentally different content (e.g., orange fish scales and black background in Figure 8). Instead, as shown in the figure, our synchronization layer estimates the extent of the crop, allowing us to revert it by downscaling and padding appropriately. This (approximately) recovers original tokens within the area that was not cropped, which are often sufficient to detect the image as watermarked.

We hope this helps clarify the mechanism; please follow up if you have additional questions.

Q4: How does your approach compare to post-hoc watermarks from the angles of robustness, security, visual quality and efficiency?

Our approach (RCC finetuning + token-level watermark + synchronization layer) has several important advantages compared to post-hoc methods and, importantly, is not fundamentally inferior in any major way. Extending the discussion in the paper, we provide concrete arguments below. While we certainly agree that research on better post-hoc watermarks is also important, we believe these arguments clearly motivate the value of our approach.

Robustness: As Table 2 shows, our method is robust to valuemetric and geometric changes, as well as common neural compression and purification attacks. In contrast, 2/4 post-hoc watermarks in Table 2 are removable by geometric changes, and none of the 4 are robust to attacks, with the best evaluated baseline (TrustMark) still on average being outperformed by our method. We hypothesize that their brittleness to attacks is due to anchoring the watermark in an original clean image that the attacks can approximately recover. In contrast, for our token-level method there is no original clean image, as the image is already watermarked when the generation finishes.

Security: As discussed on L182 and L165, our token-level watermark gives theoretically rigorous p-values directly inherited from LLM watermarking, which can minimize the risk of false positives, e.g., unjustly blaming actors for covert use of AI tools. In contrast, post-hoc watermarks are generally based on neural extractors, meaning that their p-values are derived from the expected behavior of a neural net on non-watermarked images which is hard to rigorously prove and may introduce bias (see our references in the paper).

Visual quality: As our FID results (main paper and Appendix F.3) and PSNR measurements (Appendix F.4) show, our watermark does not significantly degrade visual quality.

Efficiency: Our tokenizer finetuning step is practical and incurs no significant overhead. As noted on L214, in our experiments finetuning took at most 32 GPU hours. This is a small one-time-per-model cost for a practitioner, who can then use the resulting tokenizer in generation and detection of any number of watermarked images. In comparison, the training of the Chameleon model itself took >850000 GPU hours (see Table 2 in [10]). Our watermark detection is not expensive: we refer the reviewer to our answer to Q2 of Rev. s9Sh above where we demonstrate detection latency comparable to other generation-time watermarks.

Cross-modal watermarking: Additionally, our token-level approach uniquely enables cross-modal watermarking. In particular, as we show in Sec 4.3, we can easily apply the same underlying scheme to a set of modalities that can be tokenized (e.g., text, images, audio), and then run joint detection to obtain a single rigorous p-value across multimodal content. In contrast, post-hoc methods for text, images and audio generally differ in the literature and we are not aware of attempts to unify them across modalities.

We thank the reviewer for their detailed comments and for recognizing our work as solid, clear and of high quality. We respond to all questions raised by the reviewer below, numbered Q1-Q4. We kindly ask the reviewer to let us know if their concerns were addressed, and we are happy to continue the discussion further if there are follow-up questions.

Q1: Is the watermark defeated entirely by flips and crops?

No; we believe there might be a slight misunderstanding here. Our final method, denoted FT+AUGS+SYNC in Table 2, is quite robust to geometric transformations (Geo.) which include flips, crops and rotations: we get robustness scores of 0.82 on Taming and 0.64 on Chameleon. The above rows in the same table refer to ablations of our method with some features disabled.

Q2: Why is localization necessary for geometric robustness and what kind of transformations does this support?

Due to the nature of image tokenizers we study, each token is loosely tied to regions in the image. After e.g. a flip, most regions contain completely different content and thus result in different tokens, erasing our token-level watermark. The same is true for rotation and crops followed by a resize to the canonical image size (see L195). This makes it difficult for any token-level watermark to be natively robust to geometric changes. While we focus on existing commonly used tokenizers, another way around this issue would be to develop spatially invariant (e.g., semantic) tokenizers, which is a research question orthogonal to our work on watermarking.

For this reason, we introduce a synchronization layer that can add geometric robustness to token-level watermarks. Our current implementation successfully estimates and reverts all geometric transformations that we explicitly consider. This (approximately) recovers original tokens that we can then detect as watermarked. The same approach could be easily extended to generalize to a broader range of transformations of interest, potentially using ideas we discuss in Sec. 6. We find this to be a fruitful research topic for future work with many interesting directions. For example, for our follow-up work we are currently experimenting with a synchronization implementation that can directly estimate a homography matrix defined by 4 coordinates, and are seeing promising first results.

Q3: Can you motivate the choice of KGW as the underlying scheme?

We focus on KGW as arguably the most prominent scheme in the field. As we are the first to study watermarking for autoregressive image models, choosing a relatively simple, well-studied and well-understood scheme allowed us to focus on key issues particular to our setting (e.g., RCC). We do think that it would be interesting to explore other schemes such as AAR [1] or KTH [44] in this setting, as well as more elaborate schemes built on top of KGW. We note that our key contributions (RCC finetuning, synchronization) could be directly used in those cases, as they are orthogonal to the underlying scheme choice. We are looking forward to such exploration in future work.

Q4: How does the entropy of next-token distributions differ for text and images?

This is an interesting question. Prompted by this, we ran a new experiment where we measured the entropy in each modality in the same setting as our Joint Watermarking experiment (Sec. 4.3) on 20 outputs (~20k logits). As we can’t include plots in this rebuttal, we summarize key results below. We will add full results of this experiment to the next revision of our paper.

Both entropy distributions have a high peak close to 0, but the entropy is generally higher and less concentrated for image tokens. In particular, the (mean, std) of entropy is (0.45, 0.74) for text and (2.93, 1.90) for images. While to make reliable conclusions we would need to also consider quality preservation and variance between tokenizers, these results provide some evidence that image tokens may be fundamentally easier to watermark.

In both cases, entropy consistently decreases as a function of token position, i.e., as more content is generated. For images, there is an interesting phenomenon: the entropy has a periodic pattern during generation due to the row-major generation order. In particular, generating a token that starts a new image row always comes with unusually high entropy.

We thank the reviewer for providing valuable feedback and recognizing our work as novel, principled, and comprehensive. We respond to the questions Q1 and Q2 below. We are happy to continue the discussion further if the reviewer has follow-up questions.

Q1: Can you elaborate how synchronization can harm detection in the presence of valuemetric transformations?

Certainly. Expanding on our discussion on L256 in the paper, we recognize three distinct modes:

• Under weak valuemetric transformations (e.g., JPEG 90), the synchronization signal remains mostly intact. We estimate that no geometric transformation was applied, and the detection behaves as if there was no synchronization.
• Under strong valuemetric transformations (e.g., JPEG 30), the synchronization signal disappears. As we are not able to estimate which geometric transformation was applied, the detection again behaves as if there was no synchronization.
• Finally, for some moderate strengths, the synchronization signal is disrupted. In Fig. 8 (bottom row) in the appendix we see one such example for JPEG 70. While in this example we still successfully estimate that no geometric transformation was applied, such cases often lead to an estimate of a nonexistent transform (e.g., -5 degree rotation). Our algorithm then rotates the image by 5 degrees in an attempt to revert this, which changes the tokens and degrades the watermark signal. To make this clearer, we will add examples of such failures in the next revision of the paper.

The last case leads to a small drop in valuemetric robustness seen in Table 2 (~0.1), traded off for a big jump in geometric robustness (from ~0 to ~0.80/0.65). We believe this tradeoff is generally desirable as it leaves no simple way to reliably and consistently remove the watermark. This can change if the practitioner has a very specific prior, e.g., they only care about JPEG robustness as it is inherent to their use-case.

More importantly, the only cause for this is the insufficient valuemetric robustness of the localized embedder that we use, and is not a fundamental limitation of our approach. As discussed in Sec. 6 and confirmed by the early experiments done in our follow-up work, replacing the off-the-shelf embedder with a targeted synchronization model can reduce this effect.

Q2: Can you provide experimental results on detection latency?

Prompted by the reviewer’s question we benchmarked our detection on Taming generations on a single H100 GPU. We obtain consistent times averaging 837ms for synchronization, 6ms for subsequent tokenization, and 450ms for watermark detection. This makes the detection practical for large-scale deployments, and in the order of magnitude of SOTA diffusion model watermarks, e.g., detecting Tree-Ring [88] took ~2s in our experiment. Note that we did not explicitly optimize the latency nor experiment with parallelization which should be simple in this case and further improve speed.