Discovering Spoofing Attempts on Language Model Watermarks

We thank the reviewer for their helpful feedback and address their individual questions below. We have attached one additional figure here.

Q1: Can the current spoofing methods be easily adapted to bypass the proposed detection? For instance, can we constrain the h-gram distribution in the spoofer’s training data?

We do not believe so, as we are tackling a fundamental limitation of learning-based spoofers that is not specific to a given implementation. Because the spoofer learns from a finite, small training dataset (for cost reasons), it cannot “see” every h-gram combination and hence perfectly reproduce the watermark. Enforcing uniformity is not possible, as there are roughly O($\Sigma^{h-1}$) h-gram combinations (for a fixed last token), which means that a uniform dataset needs an exponential amount of tokens.

In particular, for Stealing, their authors explain that the sparsity of the training data was the main challenge behind their attack, and they had to discard low-frequency h-grams for stability [1]. Enforcing additional constraints on the training data, such as trying to get a distribution of h-grams as uniform as possible, would only reinforce the sparsity issue, ultimately hurting spoofing performance.

For Distillation, we show in App. E Figure 9 that increasing the spoofer's training data by 10x only reduces our method's TPR by 15%. Further, enforcing constraints on the distribution of the data in that case might significantly degrade the spoofer model's performance. Because the token distribution would deviate further from human text, it might hinder the finetuning process.

Overall, while we cannot provide guarantees against (arbitrary) adversarial attacks, our method practically increases the amount of effort needed by an adversary for successful spoofing—which we argue is overall beneficial.

Q2: Does there exist an adversary that could bypass the proposed method?

As briefly discussed in Section 2 and Appendix I, we believe that a step-by-step spoofing attack, combined with the idea of color-aware substitution [2], would result in a spoofing attack that is not detectable by our method. We note that such an adversary would (in the limit) be able to accurately estimate the Red-Green splits for any context, resulting in a watermark signal that is distributionally indistinguishable from the genuine watermarking algorithm.

However, such a method would require querying the watermarked model for every token generated by the spoofer, rendering it prohibitively expensive and impractical. Hence, we argue that, given the current state of the field, there exists no spoofing method that could leverage the knowledge of our statistics to bypass our method while maintaining similar properties (cost and practicality) to current learning-based spoofers.

Q3: Why do the authors restrict themselves to older models?

We evaluate our method on older models to match the experimental setup from the spoofing attacks papers. Nonetheless, in response to the reviewers comments, we evaluate Stealing with $h=3$ using Llama3-8B as the watermarked model and Qwen2.5-7B as the spoofer model, using the Reprompting method (see attached Figure 1). We see that our test is equally valid with newer models as well, with the FPR being properly controlled (solid line) and the TPR high (dashed line).

[1] https://arxiv.org/abs/2402.19361
[2] https://arxiv.org/abs/2403.14719

\newcommand{\D}{\mathcal{D}}$$\newcommand{\T}{\tilde{\D}}We thank the reviewer for their helpful feedback and address their individual questions below.

Q1: Does the proposed method require a good estimate of the spoofer's training corpus in order to be applicable?.

A reasonable estimate is enough for the detection to maintain sufficient power. For this, we have already included two experiments in App. D (Figure 8) to ensure that the detection remains accurate despite a weaker estimate of the spoofer’s training corpus.

Let $\D$ be the spoofer's training corpus and $\T$ the estimate of such a corpus. For the first experiment, we build several $\T$ by adding random noise to $\D$. This allows us to control the distance between $\T$ and $\D$ and observe how the power decreases with the distance. For the second experiment, we show that for realistic choices of $\T$ (e.g., C4, Wikipedia), the power of the test remains very similar to the best-case scenario of $\T = \D$. This means that ultimately a reasonable choice of $\T$ is sufficient to maintain high power.

Importantly, as we argue in App. D, the detection can select an appropriate $\T$ from a similar domain as the received text to be tested. Because the spoofer wants to maximize his chance of successful spoofing, it is reasonable to assume that $\D$ is from the similar domain as the spoofed text he aims to generate, making it likely that the selected $\T$ is close to $\D$.

Q2: Are the schemes studied by the authors really intended for attribution? Can the authors better motivate the choice of watermarking schemes they study?

Schemes such as Red-Green, KTH, and AAR have been the focus of multiple previous works considering them for attribution, as shown by spoofing attacks [1,2,3] and multi-bit watermarking [4]. Hence, these schemes have attracted significant attention from researchers exploring broader applications beyond mere detection, including attribution.

Further, such schemes have practical relevance as shown by the deployment of Google Deepmind’s SynthID, the first publicly acknowledged deployment of LLM watermarks. We therefore argue that their prominence and practicality motivate further studies on their real-world security.

Yet, we agree with the reviewer that making a clear distinction as to whether schemes are explicitly designed for attribution is particularly relevant. Based on the reviewer’s suggestion, we will expand our introduction by explicitly discussing watermark attributability by design as well as its current treatment in the research community.

Q3: Does detecting spoofing attempts lead to a “cat and mouse” game? Isn’t it better to design schemes explicitly designed for attribution?

We agree with the reviewer that, if the desire is to build an un-spoofable scheme, designing a scheme specifically for attribution, such as [5,6], is the best practice.

Yet, for practical use, practitioners desire multiple, often conflicting, properties: quality, practicality, security, and robustness. Schemes that are harder to spoof might compromise too heavily on these other desirable properties. Importantly, our work indicates that previously successful spoofing attacks on prominent schemes actually are detectable—ultimately making it harder for real-world adversaries to attack such schemes. Further, especially with the increasing work in watermark spoofing on these schemes, we see our work as a helpful contribution to the field by providing general insights into the properties and limitations of learning-based spoofers.

While we ultimately agree that providing an un-spoofable scheme that achieves all of the above properties would be the optimal solution, we see a lot of value in (1) exploring the properties of popular and used watermarks and (2) raising the bar for successful attacks which can yield practical real-world benefits.

Q4: Can the authors provide more theoretical insights for Eq. (5)?

Rigorously proving Eq. (5) would require further assumptions about the dependence between $X$ and $Y$. With Lemma 4.1, we assumed independence to justify the asymptotic normality, and with the Unigram score, we provided intuitive reasons why this independence assumption might hold in practice.

For the general case, we show in Appendix C that the independence is violated. While a more general assumption about the dependence structure could allow us to prove Eq. (5), we believe that this complicates the problem modelling without providing significant value to our contribution (as we would still likely rely on empirical evidence to justify our dependence structure assumption). Thus, as noted by the reviewer, we opted for a more direct approach by providing empirical evidence to support Eq. (5).

[1] https://arxiv.org/abs/2402.19361
[2] https://arxiv.org/abs/2312.04469
[3] https://arxiv.org/abs/2405.19677
[4] https://arxiv.org/abs/2308.00221
[5] https://arxiv.org/abs/2402.09370
[6] https://arxiv.org/abs/2310.18491

We thank the reviewer for their positive feedback and address their individual questions below. We have attached two additional figures here.

Q1: Can the authors explicitly analyze the distribution shift between Reprompting with or without the original prompts?

We thank the reviewer for their suggestion and included a new experiment where we compare Reprompting with and without using the original prompt on Stealing with $h=1$.

We see in the attached Figure 1 that estimating the prompt indeed consistently slightly decreases TPR (at worst a 5% TPR at 1% FPR decrease). This suggests that, for non-adversarial cases, the drift incurred by using the beginning of the received text as a proxy for the prompt has a minimal impact on our detection accuracy.

We will add this experiment to the next revision of our paper.

Q2: Could the authors use a fixed corpus of LLM generated text from the same domain as the received text instead of Reprompting?

Intuitively, using a corpus of LLM-generated text whose correlation follows the distribution of Eq. (5) would allow us to estimate the mean and perform the test. However, in practice, it is hard to design a general rule establishing what the mean depends on (e.g., is it the topic of the text, is it the type of words used…).

To strengthen our point, we show in the attached Figure 2 an experiment with Distillation, $h=2$, where we estimate the mean and variance using a $\xi$-watermarked text corpus generated from OpenWebText completions. We then apply Eq. (6), where we replace $S(\omega’)$ with the estimated mean. We see that the FPR is higher than what we would expect, suggesting that the estimated mean does not capture the true mean. This is why we argue that Reprompting, albeit more costly, is more reliable.

We will explicitly include this discussion in the paper.

Q3: Is the method prohibitively expensive for real world usage?

We do not think so. As spoofing detection is more targeted than watermark detection—it can be run only when one suspects spoofing—performance issues are, by nature, less critical. We also note that, for some settings ($h=3$), the Unigram score is applicable and doesn’t require additional text generation. Yet, we agree that our work is a first step in detecting spoofing attempts on tested schemes, and we believe that making the test more efficient is a worthwhile direction for future work.