Discovering Clues of Spoofed LM Watermarks

We thank the reviewer for their exhaustive feedback. Below, we address the concerns raised in questions Q1 to Q4; and note that a revised version of the paper has been uploaded, with updates highlighted in blue. We are happy to clarify further if there are additional questions.

Q1: Could the authors clarify why simply increasing the context size $h$ isn’t enough to overcome spoofing?
Certainly. In fact, several previous works ([1,2]) have already noted that using a longer context size $h$ does make spoofing harder. In fact, the Distillation spoofing technique [3] does not work with $h=3$. However, previous works ([4,5,6]) have already noted that the choice of $h$ is a complex tradeoff between many watermark properties such as watermark robustness, impact on quality, or ease of spoofing. In general, higher $h$ values are harder to spoof and have less impact on the text quality at the cost of being more vulnerable to scrubbing attacks.

We agree with the reviewer that such a discussion regarding $h$ is missing in our paper. We edited the introduction to include this discussion. We are happy to further edit the paper should the reviewer have other suggestions.

Q2: Why don’t the authors include the original watermark detector as the baseline?
We believe this is a misunderstanding. As explained in Section 3, our threat model is that the texts on which our statistical test is applied are already flagged as watermarked by the original watermark detector. This is a realistic scenario: a model provider does not care about investigating texts if they don’t pass the detector, as such a text is unrelated to their model (e.g., human written or a failed spoofing attempt). We hope this clarifies a potential misunderstanding. Similarly, there are no prior “spoofing discovery” baselines, as we are the first to study the detectability of spoofing attempts. We are happy to clarify our answer if it does not fit the reviewer's question.

Q3: Could the authors provide a quantitative analysis of the validation of the test hypotheses?
In our new revision, we updated Figure 2 in Section 5.1 to add quantitative information regarding the histograms normality. More specifically, we ran both a Kolmogorov-Smirnov test for standard normality and a Pearson’s normality test. For the Standard method, the tests confirm the visual intuition that only $h=3$ follows a standard normal distribution. For the Reprompting method, we observe that for $h \ge 2$, the standard normal hypothesis is only slightly violated (p-values of the KS test are greater than 0.001), and for $h=1$, only the normality hypothesis holds. The slight violations of the hypothesis for the Reprompting method are expected, as it does not have a theoretical backing as strong as the Standard method. Yet, in practice, the FPR of the tests is properly controlled for both methods, as we show in Section 5.2.

We thank the reviewer for their suggestion, and we believe the additional quantitative analysis coupled with the histograms strengthens our message that our hypotheses are sound and practical.

Q4: Could the authors clarify the symbols used in the paper?
Good point, we updated the paper to ensure all less common symbols are now properly introduced.

The symbol in equation (3a) and (3b) means independence and the symbol $\xrightarrow{d}$ is used for convergence in distribution.

We are happy to further edit the paper if the reviewer has more suggestions.

[1] “On the Reliability of Watermarks for Large Language Models”, Kirchenbauer et al., ICLR 2024
[2] “Watermark stealing in large language models”, Jovanovic et al., ICML 2024
[3] “On the learnability of watermarks for language models”, Gu et al., ICLR 2024
[4] “A watermark for large language models”, Kirchenbauer et al., ICML 2023
[5] “Provable robust watermarking for ai-generated text”, Zhao et al., ICLR 2024
[6] “Robust distortion-free watermarks for language models”, Kuditipudi et al., TMLR 05/2024

Thank you for the detailed response. It addresses most of my concerns. I decide to raise my rating to 6. I do not give a higher score due to the following concerns.

1. This paper only investigates one watermarking method (i.e., red-green watermarks).
2. The contributions of this paper are to provide insight to discover spoofed watermarks and design a hypothesis test. However, this paper does not explicitly clarify the challenges of designing such a method. Therefore, I think this paper may provide limited inspiration for subsequent works. I think designing a trivial method as a baseline may be better to clarify the challenges and the advantages of the designed method. In summary, I think this paper can provide some insight to the community but the technical contribution of this paper is limited.

We thank the reviewer for their exhaustive feedback. Below, we address the concerns raised in questions Q1 to Q3; and note that a revised version of the paper has been uploaded, with updates highlighted in blue. We are happy to clarify further if there are additional questions.

Q1: Why do the authors restrict themselves to Red-Green watermarks? Does this generalize to other spoofed schemes?
We do not see our focus on Red-Green schemes as a weakness of our work. To be able to evaluate our test, we can only work with schemes where practical spoofing techniques have already been demonstrated. Hence, the only schemes we can study are Red-Green schemes [1], AAR [2] and KTH [3].

In our main experiment, we focused on Red-Green schemes [1] as they are not only the most popular [6] class of watermarks, but are also seeing increasing adoption by industry (e.g., SynthID recently being announced for Gemini). As the most studied class of schemes, Red-Green also happens to have the most research on spoofing attacks, being the only one with more than one learning-based spoofing attack being proposed. As discussed in Section 7, we excluded the Unigram scheme [4] as the spoofing method from [5] achieves perfect spoofing on it (they recover up to 80% of the red-green split).

To highlight the method's generalizability to alternative watermarking techniques, we provide in Appendix F two additional experiments to encompass AAR [2] and KTH [3] with Distillation based spoofing. Importantly, we have similar results as in Section 5, showing that for both schemes, Distillation leaves artifacts in the generated text that we can leverage to detect the spoofing attempts.

We hope the additional experiments convince the reviewers that the core idea behind our methods can be generalized to a wider array of schemes.

Q2: What is the performance of the method under common text modifications?
Our understanding of the reviewer's comment is that they are unsure how the FPR scales when $\xi$-watermarked text is modified. For spoofed text, a potential spoofer would not want to modify their text in a way that lowers the watermark strength, as they precisely aim to make it as strongly watermarked as possible.

We however added a new experiment in Appendix G where we inserted a fixed percentage of human text into a $\xi$-watermarked text. We show that up to 20% of mixed human text within a $\xi$-watermarked text does not significantly affect the FPR of our detection tests for both the Reprompting and Standard methods.

Such results have intuitive explanations: because the alterations to the watermarked text are done independently of the color, the overall correlation within the edited text between the color and the general token frequency remains unchanged. In other words, the tests do not focus on the green/red ratio but on the consistency of green tokens: in a spoofed text, the green tokens are mostly “generally frequent” tokens, whereas in other cases any token is equally likely to be green.

We hope that this additional experiment convinces the reviewer that the tests are robust to legitimate text modifications. We are happy to provide further testing if the reviewer has other suggestions.

Q3: Does the requirement of 1000 tokens pose a practical limitation?
No. First, as we have shown in Appendix B, the test results remain valid if we concatenate multiple texts of different lengths. Hence, in a practical scenario, a longer text could be directly achieved by concatenating multiple texts from the same source. More generally, we believe the key contribution of our work is already highlighting the existence of artifacts in current learning-based spoofing methods.

[1] “A watermark for large language models”, Kirchenbauer et al., ICML 2023
[2] “Watermarking of large language models”, Scott Aaronson, 2023 Workshop on Large Language Models and Transformers, Simons Institute, UC Berkeley
[3] “Robust distortion-free watermarks for language models”, Kuditipudi et al., TMLR 05/2024
[4] “Provable robust watermarking for ai-generated text”, Zhao et al., ICLR 2024
[5] “Large Language Model Watermark Stealing With Mixed Integer Programming”, Zhang et al., 2024
[6] “A survey of text watermarking in the era of large language models”, Liu et al., ACM Computing Surveys 2024

Q4: Could increasing the size of the spoofer training dataset significantly degrade the efficiency of the method?
We agree with the reviewer that the intuition behind our test is that the artifacts left by spoofing are caused by the spoofer's partial knowledge of the scheme. Hence, intuitively, a larger training dataset should imply fewer artifacts.

To further investigate this, we ran additional experiments in Appendix E with Stealing on LeftHash with $h=3$. Surprisingly, we find that, when ranging from 4 million to 24 million tokens in $\mathcal{D}$, the TPR of the test is similar. At the same time, the number of successful spoofing attempts increases significantly up to 16 million tokens and then plateaus. This suggests that either the additional data in $\mathcal{D}$ is redundant or that Stealing doesn’t leverage the dataset efficiently. Hence, we believe that preventing the artifacts from Stealing would require an unreasonably large dataset $\mathcal{D}$.

Q5: Could a more elaborate spoofing technique build the training dataset $\mathcal{D}$ adaptively?
We believe that querying the $\xi$-watermarked model adaptively to develop a spoofing technique that leaves no artifacts is an interesting direction that may be possible, as we suggested in Section 7. However, a significant amount of work is needed to actually design such a technique and evaluate its efficiency. We are excited that our work opens a new challenge in the field of watermark spoofing and hope it can serve as a baseline for evaluating future spoofing techniques.

Q6: Could the authors provide experiments on the impact of $\tilde{\mathcal{D}}$?
Good question—we added two detailed experiments in Appendix D. For the first experiment, we build several $\tilde{\mathcal{D}}$ by adding random noise to $\mathcal{D}$. This allows us to precisely control the distance between $\tilde{\mathcal{D}}$ and $\mathcal{D}$; and see how the power decreases with the distance. For the second experiment, we show that for realistic choices of $\tilde{\mathcal{D}}$ (e.g., C4, Wikipedia), the power of the test remains very similar to the best case scenario of $\tilde{\mathcal{D}} = \mathcal{D}$. This means that ultimately the choice of $\tilde{\mathcal{D}}$ is not of great importance to maintain high power.

Q7: Could the authors clarify some points in the paper?
We have edited the paper to clarify the highlighted points, and we hope it is now clearer. We welcome additional suggestions from the reviewer to improve the writing quality of the paper. Below is a short description of the changes.

In our work, $X_t$ (defined in line 168) is the random variable associated with the color of the $t$-th token in $\Omega$. Hence, $X_t = 1$ is the event “the $t$-th token is green”. It is the same $X_t$ used for equations (2a) and (2b).

We implicitly consider that $I_{\mathcal{D}}$, as a function of frequencies, defines a categorical probability distribution over its support, where the probability of an $h+1$-gram is equal to its frequency. Based on the reviewer's suggestion, we removed this distinction from our paper, as it did not add value to the comprehension of the proposed method.

Q8: How many samples are in $\mathcal{D}$?
We follow the recommendations from the original papers. For Stealing ([8]), we used 30,000 prompts, each 800 tokens long, based on completions of C4. For Distillation ([9]), we used 640,000 prompts, each 256 tokens long, based on completions of C4. Based on the reviewer’s comment, we clarified these numbers in the paper (in Section 5, L368).

Q9: Which method did you use for Distillation spoofing?
In [9], the two methods presented are distillation from a watermarked teacher model or from samples generated by the watermarked model. In a spoofing case, as the authors of [9] explain, we assume we do not have access to the original model which is required for the first scenario. Hence, we focus only on the second method based on learning from a sampled dataset of $\xi$-watermarked text. We updated the paper (Section 2) to specify which method we used and avoid any future confusion.

[1] “Attacking LLM Watermarks by Exploiting Their Strengths”, Pang et al., 2024
[2] “A watermark for large language models”, Kirchenbauer et al., ICML 2023
[3] “Robust distortion-free watermarks for language models”, Kuditipudi et al., TMLR 05/202 [4] “Bileve: Securing Text Provenance in Large Language Models Against Spoofing with Bi-level Signature”, Zhou et al., 2024
[5] “Bypassing LLM Watermarks with Color-Aware Substitutions”, Wu et al., 2024
[6] “Robust distortion-free watermarks for language models”, Kuditipudi et al., TMLR 05/2024
[7] “Provable robust watermarking for ai-generated text”, Zhao et al., ICLR 2024
[8] “Watermark stealing in large language models”, Jovanovic et al., ICML 2024
[9] “On the learnability of watermarks for language models”, Gu et al., ICLR 2024

We thank the reviewer for their exhaustive feedback. Below, we address the concerns raised in questions Q1 to Q9; and note that a revised version of the paper has been uploaded, with updates highlighted in blue. We are happy to clarify further if there are additional questions.

Q1: Do the authors claim any spoofing attempts leave artifacts? What about non-learning-based spoofing techniques?
It is not our intention to make such overclaims and are happy to adjust the claims in order to improve clarity in this regard. In particular, we do not claim that every spoofing method leaves artifacts in their generated text. Instead we argue that current practical state-of-the-art spoofers for popular watermarking schemes both (1) happen to be learning-based in nature and (2) do in fact leave artifacts of watermark spoofing in their generated texts.

Notably we focus on learning-based spoofing as they have the ability to generate arbitrary amounts of diverse spoofed text in a cost-effective way and are applicable in realistic setups. We see this in contrast to the methods mentioned by the reviewer. In particular, “Piggyback spoofing attacks” ([1]) directly leverage the desirable robustness property of the watermark. Given a watermarked sentence, the attacker modifies a few tokens to alter the meaning of the original sentence while maintaining the watermark, interpreting the result as an instance of spoofing. Mixing human and LM generated text is a realistic use case of LMs, many watermarking methods inherently account for such modifications ([2,3]), making it a priori unclear whether selectively modifying individual tokens is practical spoofing or an inherent property of a watermark. We note that this is also in stark contrast to learning-based spoofers that can generate almost arbitrary text without having to rely on a pre-existing template—which additionally can be both hard to generate for harmful topics and significantly increases costs as it requires queries to the original LM for every generation.

A separate line of works considers spoofing techniques that require queries at each step of the generation process of every spoofed text [1,4,5], using the feedback obtained this way to choose the next token. While they have higher flexibility compared to piggyback spoofing, a key limitation of these techniques is the prohibitively high cost, even compared to piggyback spoofing. Further, some of these methods rely on the unrealistic assumption of having access to the watermark detector itself (sometimes even its confidence score) to obtain the required feedback, making them inapplicable to practical deployment scenarios (such as the recently released SynthID-Text).

Hence, our focus on learning-based spoofing is largely driven by practical considerations, as well as the state of the field regarding spoofing attacks. To clarify our contribution, we have added this additional discussion to the new Appendix I and are happy to adjust claims to accurately reflect the contributions of our work.

Q2: Does the requirement of 1000 tokens pose a practical limitation?
No. First, as we have shown in Appendix B, the test results remain valid if we concatenate multiple texts of different lengths. Hence, in a practical scenario, a longer text could be directly achieved by concatenating multiple texts from the same source. More generally, we believe the key contribution of our work is already highlighting the existence of artifacts in current learning-based spoofing methods.

Q3: Can the test results be used to choose a context size?
Partially, but we do not make confident claims about which $h$ is best. Our results, that current learning based spoofing attacks leave visible artifacts in spoofed text, show that the risk of spoofing for smaller $h$, with current methods, is less important than previously thought. However, previous works ([2,6,7]) have already noted that the choice of $h$ is a complex tradeoff between many watermark properties such as watermark robustness, impact on quality, or ease of spoofing. In general, higher $h$ values are harder to spoof and have less impact on the text quality at the cost of being more vulnerable to scrubbing attacks. As such we don’t deem it recommendable to make the choice of $h$ solely based on our tests.

We thank the reviewer for the quick response and answer the follow-up questions below.

1 As suggested by the reviewer, we updated the Abstract, Introduction and Conclusion and uploaded the new revision. We hope that the updates remove any confusion in this regard.

2 We first want to clarify that concatenating multiple texts (as studied in Appendix B) is only statistically valid in the case of independent texts. Hence, it is not sound to concatenate repetitions of a single short text.

As we state in Section 5 and show in Table 1, the power of the test indeed increases with the length of the text. Importantly, the test remains sound for short texts (i.e., the Type 1 error is still properly controlled), but the chance of false negatives is higher. However, as also noted by the reviewer, at short text length of 500 tokens, the best parameters show a TPR of 62% at a confidence level of 99%. This shows some robustness to short texts; but reducing text length further may be a challenge—in our latest revision, we added this to the limitations to highlight it as an important consideration.

3.1 We are sorry for the confusion. As explained in the caption of Figure 9, there we report the number of queries, each query being 800 tokens long. Hence, for the smallest $\mathcal{D}=5,000$ we have $5000\times800=4,000,000$ tokens, and for the largest $\mathcal{D}=30,000$ we have $24,000,000$ tokens.

3.2 Good observations! We thank the reviewer for looking at our new experiments so thoroughly. We believe there are several reasons why it would be misleading to convincingly claim that there is positive correlation between the size of $\mathcal{D}$ and the power of the test:

• At $T=500$ the differences are negligible; this suggests a complex interplay between $|\mathcal{D}|$ and $T$ that may be hard to fully understand from our experiment.
• Even for $T=1000$ the distance between curves is not as big for $\alpha \in \\{10^{-2}, 10^{-1}\\}$ as the case reviewer justifiably points out.
• In the interest of time, this new experiment was run with much less data compared to our original experiments ($n=500$ responses vs $n=10,000$ originally). Moreover, these responses are concatenated in groups to get text length of $T$, and the failed spoofing attempts are filtered out (as we only apply our test on successful spoofs!). Crucially, for $|D|=5000$, as spoofing often fails, this results in only $32$ texts, making our results noisy.
• Such correlation, if true, would have to hold only up until some point, as it is clear that as $|\mathcal{D}| \to \infty$ our test’s power goes to $0$ (as spoofing is perfect). That point would also heavily depend on the setting, e.g., the spoofer algorithm.

In summary, there are many factors at play that are hard to isolate, including our experimental setting or the way a spoofer utilizes data. Thus, while we are not comfortable making hard statements at this point, it could be true that counterintuitively, reducing $|\mathcal{D}|$ reduces our power.

Connecting to the last point above, our best hypothesis is that for very small $|\mathcal{D}|$ the spoofer may fail to acquire any generalization ability, so the (rare) cases when it succeeds are the cases when it remains very close to its training data, i.e., leaves less artifacts. Ultimately, it is important to conclude that smaller $|\mathcal{D}|$ is not strictly better for the spoofing adversary—while there may be less artifacts, the more important goal of the spoofer (to spoof) is often greatly sacrificed. We updated App. E to reflect the above.

We thank the reviewer for their exhaustive feedback. Below, we address the concerns raised in questions Q1 to Q3; and note that a revised version of the paper has been uploaded, with updates highlighted in blue. We are happy to clarify further if there are additional questions.

Q1: Why do the authors restrict themselves to two learning-based spoofing techniques? What about other evolving and diverse strategies?
The scope of our work are learning-based spoofers (as defined in Section 2), as they have the ability to generate arbitrary amounts of diverse spoofed text in a cost-effective way and are applicable in realistic setups. This, given the current state of the field, boils down to two methods: Stealing ([1]) and Learnability ([2]). We excluded the spoofing method from [3], as it achieves perfect spoofing on the Unigram scheme [4] (thus can’t be detected) and does not spoof any other schemes. In Section 6, we had already discussed the works of [5] and [6]. We have further updated our paper (Appendix I) to include an even more thorough discussion of those topics.

We are not aware of any other works beyond the ones we discuss. Could the reviewer point out some specific spoofing techniques they had in mind? We would be happy to discuss them, and see if our methods can generalize.

Q2: Is the contribution invalidated by the existence of a non-spoofable scheme? Why do the authors restrict themselves to Red-Green watermarks?
We do not see the existence of harder to detect schemes as a weakness of our work. While we agree that cryptographic schemes such as [11] offer strong guarantees against spoofing, watermarking is a complex field which balances many properties such as strength, robustness to removal, or behavior in low entropy settings. Notably many of these properties are not evaluated on such schemes leaving them primarily as theoretical constructs. Based on the reviewer’s suggestion, we expanded this discussion in Appendix I.

In contrast Red-Green schemes [7] are not only the by far most popular [12], but are also seeing increasing adoption by industry (with SynthID recently being announced for Gemini). As the most studied class of schemes, Red-Green also happens to have the most research on spoofing attacks, resulting in several practical attacks [1,2,3]. Combined with the practical adoption of RG schemes, this makes the detection of such spoofed texts a relevant and timely topic of study.

As we elaborate in our new Appendix F, the key ideas behind our spoofing detection test generalize to both AAR [8] and KTH [9], where we show that Distillation of both schemes also leaves detectable artifacts.

We hope that this discussion and the additional results justify our focus on Red-Green watermarks and further highlight how the core ideas behind our method generalize to a wider range of schemes.

Q3: Could the authors provide additional experiments to show the influence of the context size on the test efficiency? Is spoofing with smaller contexts harder to detect?
We believe this is a misunderstanding as we already evaluated the power of our test for small context sizes.

All our results in Section 5 are presented for $h \in \{1,2,3\}$. For Distillation spoofing, we observe that the test is slightly more powerful for $h=2$ than for $h=1$. However, for Stealing spoofing, counterintuitively, the test is less powerful for $h=2$ than for $h=1$. Hence, our experimental evaluation in Section 5 already shows that a smaller context does not weaken our method. We agree that a bigger context size makes spoofing harder and could prevent spoofing altogether. However, as shown in prior work [7], using larger $h$ to counter spoofing comes with drawbacks: it reduces the robustness of the watermark. In the limit, if $h=\infty$, changing a single token in a text would completely break the watermark, which is not a desirable property. We added this discussion to the updated version of our paper (Section 1).

There is still the special case of $h=0$. This is the Unigram scheme ([4]), which can be almost perfectly (they recover up to 80% of the red-green split) using the method from [3], as we explain in Section 6. Hence, we excluded it from our experiments in the first place.

If the reviewer has specific experiments in mind, we would be happy to include them in our paper.

We thank the reviewer for their exhaustive feedback. Below, we address the concerns raised in questions Q1 to Q3; and note that a revised version of the paper has been uploaded, with updates highlighted in blue. We are happy to clarify further if there are additional questions.

Q1: Why do the authors restrict themselves to learning-based spoofing techniques?
The scope of our work are learning-based spoofers (as defined in Section 2), as they have the ability to generate arbitrary amounts of diverse spoofed text in a cost-effective way and are applicable in realistic setups. This, given the current state of the field, boils down to two methods: Stealing ([1]) and Learnability ([2]). We excluded the spoofing method from [3], as it achieves perfect spoofing on the Unigram scheme [4] (they recover up to 80% of the red-green split and thus can’t be detected) and does not spoof any other schemes. In Section 6, we have already briefly discussed the works of [5] and [6], and why not including them should not be seen as our weakness---we expand on that discussion now.

The scrubbing method presented in [5] could be turned into a somewhat impractical adaptive spoofing attack. Such a potential attack would require a very large computational budget per text generated. Indeed, for each newly generated token in every spoofing attempt, one would have to prompt the watermarked model to perform SCT and pick a green token. We argue that this is not practical in the current form. Moreover, because [5] focuses on scrubbing, no reference implementation nor evaluation of this spoofing attack exists. We believe that designing and evaluating a new practical spoofing attack with ideas from [5] and using it to test our method is outside the scope for our work.

“Piggyback spoofing attacks” ([6]) directly leverage the desirable robustness property of the watermark. Given a watermarked sentence, the attacker modifies a few tokens to alter the meaning of the original sentence while maintaining the watermark, interpreting the result as an instance of spoofing. Mixing human and LM generated text is a realistic use case of LMs, many watermarking methods inherently account for such modifications ([7,9]), making it a priori unclear whether selectively modifying individual tokens is practical spoofing or an inherent property of a watermark. We note that this is also in stark contrast to learning-based spoofers that can generate almost arbitrary text without having to rely on a pre-existing template—which additionally can be both hard to generate for harmful topics and significantly increases costs as it requires queries to the original LM for every generation.

In summary, restricting ourselves to only learning-based spoofers is driven by practical considerations and the state of spoofing attacks. While we do not believe this represents a weakness of our work, we agree this discussion should be visible in the paper—in our new revision we introduce Appendix I that reflects this discussion and motivates our choices.

Q2: Why do the authors restrict themselves to Red-Green watermarks?
We do not see this as a weakness of our work. To be able to evaluate our test, we can only work with schemes where practical spoofing techniques have already been demonstrated. Hence, the only schemes we can study are Red-Green schemes [7], AAR [8] and KTH [9]. As [10, 11] are not considered by any spoofing attacks, these are orthogonal to our work—of course this may change in the future. We include an extended discussion of this in our new Appendix I.

In our main experiment, we focused on Red-Green schemes [7] as they are not only the most popular [12] class of watermarks, but are also seeing increasing adoption by industry (e.g., SynthID recently being announced for Gemini). As the most studied class of schemes, Red-Green also happens to have the most research on spoofing attacks, resulting in several practical attacks [1,2,3]. Combined with the practical adoption of RG schemes, this makes the detection of such spoofed texts a relevant and timely topic of study.

Prompted by the reviewer’s suggestion, as we elaborate in our new Appendix F, the key ideas behind our spoofing detection test generalize to both AAR [8] and KTH [9], where we show that Distillation of both schemes also leaves detectable artifacts. In particular, on AAR, we achieve a TPR at 1% of 90% with 500 tokens of text. On KTH (EXP variant), we achieve a TPR at 1% of 60% for $s=4$ (4 maximum permutations of the key) and 30% for $s=256$ with 1000 tokens of text.

We hope that this discussion and the additional results motivate our focus on Red-Green watermarks and highlight that the core ideas behind our method generalize to other schemes.