Can Watermarks be Used to Detect LLM IP Infringement For Free?

Question 4 & Weakness 3: Risk on Unwatermarked Models.

By conducting a z-test on the proportion of green tokens in the text, we can determine how much the green ratio deviates from the set threshold for green tokens. This ensures that unwatermarked models are classified correctly with high probability. Even if these models produce watermarked phrases (green tokens), as long as the green token ratio's z-score does not exceed the threshold, they will not be classified as infringing models. (i.e. For an unwatermarked text, the probability of its z-score exceeding 2.0 is less than 0.023, and the probability of exceeding 4.0 is less than 0.00003.)

However, in practice, there is often a mismatch between the set green ratio and the actual green ratio in natural, unwatermarked text, which increases the FPR during detection. To mitigate the impact of this green ratio mismatch, we introduce anchor data to adjust the green ratio of unwatermarked text, thereby reducing FPR and improving overall detection success.

We demonstrate the FPR of unwatermarked models with the detection of the Unigram watermark, showing the accuracy of our method in detecting unwatermarked models.

[1] Kirchenbauer et al, A watermark for large language models, ICML 2023.

[2] Zhao et al, Provable Robust Watermarking for AI-Generated Text, ICLR 2023.

Question 1: Considerarion of Overlapping Dataset

The false positive rate of the detection would not be influenced by the overlapping unwatermarked data of two models. The detection of infringing models relies on identifying the watermark, which increases the proportion of "green" tokens and reduces the proportion of "red" tokens, as shown in [1]. The accuracy of detection is theorically ensured through a z-test, with the error rate being extremely low (0.003%) when the z-score exceeds 4. This means that the overlapping data will not contribute to the watermark as long as its green token ratio does not exceed a rather high threshold which is amost impossible to reach for unwatermarked text. Therefore, even if two models are trained on the same dataset without a watermark, a non-infringing model will not exhibit the green ratio shift induced by a specific watermark, making it highly probable that the model is not infringing.

Question 2 & Weakness 1: Impact on General Performance

The impact of LLM watermarking on model performance is an important issue and has been widely studied. In the watermarking method we selected, Unigram [2] has been shown in its experiments to maintain high effectiveness with minimal impact on general performance. To further demonstrate the effect of fine-tuning on suspect LLMs using watermarked training data, we evaluated the perplexity of different models. Specifically, we used Alpaca-7b to calculate perplexity, and the results are shown in the following table.

Our results are similar to those presented in [2], showing that watermarking has a minimal effect on general performance. However, this impact is controllable, and by selecting watermarking methods like Unigram, which have a minimal impact on general performance, the performance degradation caused by watermarking can be effectively reduced. Furthermore, compared with the impact of watermarks on directly generated text, the results demonstrate that the impact on distilled suspect model is less significant.

Question 3 & Weakness 2: Learnability of Watermarks & Amounts of Watermarked Data.

Given that watermark detection is related to the proportion of green tokens, watermarked data will have a green token ratio that deviates from the mean. After fine-tuning on watermarked data, the proportion of these green tokens in the suspect model's output will change. If the query used for detection contains these green tokens, the green ratio shift can be detected. However, domain mismatch between the training data and the query may lead to detection failure. Therefore, our method selects queries with high token entropy to maximize token diversity, increasing the likelihood of detecting green tokens.

We also evaluate the detectability of the suspect model with varying amounts of watermarked data. Specifically, we set the sizes of the watermarked training data to 1000, 2000, 3000, 4000, and 5000, using the Unigram watermark, with the source model as Llama2 and the suspect base model as Mistral. The experimental results are shown in the table below.

The results show that our method can effectively detect infringing models, even when only a small amount of watermarked data is used for training.

Weakness 1: Computational Cost.

We demonstrate the detection accuracy of our method under different numbers of queries and response tokens. In the case of the Unigram watermark, with the domain being Code, source model Llama2, and suspect base model Mistral, we measure detectability under varying output token counts. The experimental results are shown in the table below.

The experimental results show that our method achieves a high detection rate even with a smaller number of detection tokens. This is due to the adaptive adjustment of the z-score threshold, which allows effective differentiation of positive and negative samples even with fewer tokens, as shown in Figure 8. Increasing the number of detection tokens further enhances the z-score gap between positive and negative samples, thereby increasing the confidence in detection.

Weakness 2: Transferability.

In our experiments, we select four different models with varying structures as source or suspect base models. The source models include Llama2 and Llama3, while the suspect models include Mistral and Bloom. This demonstrates that our method works effectively across models with different structures. To further showcase its effectiveness, we conducted new experiments using Mistral as the source model and Llama2 as the suspect base model. The experimental results are presented in the table below.

The experiments show that regardless of the structural differences between the source and suspect models, our method can effectively detect infringing suspect models with a high probability.

Thank you for your suggestion. We consider the adaptive attack from the following considerations.

Consideration of Watermark Removal.

The robustness of LLM watermarkings towards watermark removal attacks has been carefully studied by previous watermarking methods. In the watermarking method we selected, Unigram [1] has been theoretically proven to be more resistant than some other watermarking methods to attacks like paraphrasing, which attempt to remove the watermark through post-processing. We perform a paraphrasing attack on Code data sampled from the source model Llama2 using Mistral and fine-tune the suspect model on the paraphrased data. The experimental results, shown in the table, indicate that while paraphrasing weakens the detectability of the watermark, it does not completely prevent the suspect model from being detected.

In practical scenarios, a stealer often requires high-quality training data. Using methods like paraphrasing to alter data from the source model could reduce the quality. Specifically, if a model used for modification or paraphrasing is strong enough to maintain the quality of the original data, the stealer might prefer directly generating training data with it rather than sampling from the source model. As a result, our threat model does not consider modifications such as paraphrasing made by the stealer to the training data.

Consideration of Watermark Mimicry.

While a stealer could attempt to mimic the same bias to increase the false positive rate (FPR) in detection, this would incur a significant cost with limited benefit in the scenario of LLM IP infringement. Firstly, since LLM watermarking subtly adjusts the logits during fine-tuning rather than explicitly injecting special characters into the text, it is less visible. Mimicking this watermark bias would require extensive statistical effort. Furthermore, doing so would force stealers to provide more evidence when they face infringement accusations from providers of the source model, which will create more inconvenience, diminishing their motivation to mimic the watermark bias.

[1] Zhao et al, Provable Robust Watermarking for AI-Generated Text, ICLR 2023.

Thank you for your feedback.

Executing a mimicry attack requires the stealer to replicate the bias introduced by the watermark. The bias in an LLM watermark depends on a hash key owned by its provider, which remains confidential to the stealer. Without access to this key, one potential approach involves extensive sampling of the target LLM to analyze token or N-gram proportions, thereby inferring the green list in the watermark configuration. However, this process is highly resource-intensive. For instance, [1] demonstrates that successfully mimicking an unknown watermark, such as KGW, requires 1 million queries to the target LLM. This query volume far exceeds the size of data typically used for instruction fine-tuning, making it an impractical and costly endeavor for the stealer.

A more efficient method to mimic such bias is using data sampled from the target LLM to fine-tune another LLM (serving as an imitation model) [2,3]. However, in our threat model, leveraging data from the source LLM to fine-tune an imitation model also constitutes infringement. In summary, if the stealer aims to generate another model fine-tuned over non-queried data while mimicking watermark bias, they must either extensively sample the victim model (incurring prohibitive costs) or train an imitation model (considered infringement and correctly classified by our detector).

[1] Sadasivan, et al. Can AI-Generated Text be Reliably Detected? arxiv, 2023.

[2] Gu, et al. On the Learnability of Watermarks for Language Models, ICLR 2024.

[3] Jovanovic, et al. Watermark Stealing in Large Language Models, ICML 2024.

Weakness 1: Results of Mixed Domain.

In our experiment, we consider two domains, Code and Math, because they are common applications in LLM instruction tuning, and many models are fine-tuned specifically for code/math tasks (e.g., CodeLlama). Additionally, the data obtained from Code and Math often differ significantly from general task data, posing a greater challenge for detecting out-of-domain data in our threat model when the suspect model's training data is unknown. When training data from different domains is mixed, the distribution becomes broader, increasing the variety of tokens affected by the watermark, which aids in watermark detection in the suspect model. Therefore, we mix Code and Math in a 1:1 ratio using the Unigram watermark, with the source model as llama2 and the suspect model as mistral. The resulting training data contains 5000 query-response pairs (2500 code and 2500 math), and we fine-tune the suspect model under this setup. The detection results for the suspect model are as follows.

Weakness 2: Consideration of High-level Domains.

Detecting the suspect model by constructing queries from various high-level domains is a possible method. However, high-level domains are difficult to define, and the detector faces challenges in selecting a set of queries that cover all potential domains. This approach can also incur high costs. For example, domains with minimal token overlap could result in ineffective queries during detection, as certain domain-specific queries might not contribute to the watermark detection process. In contrast, we approach watermark detection from the token perspective rather than the domain perspective. By covering a wide range of tokens in general queries, we can more efficiently detect watermarks without prior knowledge of the target domain. This method reduces reliance on specific domains and enhances the detection process.

Furthermore, our observations show that even when the detector fully knows the target domain, the detectability of the suspect model remains low due to green ratio mismatches in LLM watermarks like Unigram (Table 4). Therefore, our two-stage method mitigates both domain mismatch and green ratio mismatch, significantly improving watermark detectability with minimal cost.

Weakness 1: Comparison with Baseline [1] (i.i.d condition).

Our threat model assumes that the detector has no knowledge of the stealer's training data distribution. In this scenario, the method proposed in [1], which relies on constructing i.i.d. watermarked data to calculate filters during detection, is not directly applicable. To facilitate comparison with [1], we hypothesize that the detector knows the stealer's training data domain (e.g., Code or Math) and implements their approach under this assumption. Specifically: (1) For Unigram: Since its green list does not depend on previous tokens, we only perform de-duplication of [1] on the detection data. (2) For KGW (k=1): We first generate reference watermarked data using the corresponding source model. Based on this reference data, we construct filters and apply de-duplication during detection.

During testing, 20,000 tokens are sampled for each model using prompts tailored to the domain (Code or Math). Experimental results are summarized in the table below.

Experimental results demonstrate that when the training data domain of the suspect model is known, [1] can effectively detect infringing models with KGW watermarks. However, its performance is suboptimal with Unigram due to greater green ratio variability caused by different hash key divisions (Figure 4a), which de-duplication fails to mitigate. In contrast, our method exhibits more stable performance under i.i.d. conditions. By introducing a correction term for the green ratio using anchor data, our approach significantly reduces detection failures caused by green ratio mismatches.

Weakness 2: Impact of Anchor Models.

In our method, the Anchor Model is used to approximate the green token ratio in natural text (without watermark) under different green list partitions caused by hash keys, thereby addressing the green ratio mismatch issue. Additionally, it helps filter queries that elicit richer token diversity in the suspect model's responses. The requirement for the Anchor Model is generality, reflecting the natural text distribution. Experiments using various models with different structures and scales as anchor models (Unigram watermark, Code data) demonstrate the feasibility of leveraging versatile LLMs for this purpose. Results are presented below.

Experimental results indicate that using anchor models with varying architectures and scales consistently achieves high detection accuracy. This demonstrates that despite differences in model designs, these LLMs exhibit alignment in their general data distributions. Notably, when the anchor model matches the base model of the suspect LLM, detection accuracy improves (e.g. Mistral). This suggests that the detector benefits from knowing the natural output distribution of the suspect model in the absence of a watermark, enabling more precise detection of infringing behavior.

Weakness 3: Impact of Same/Difference Source/Suspect Base Model.

In our experiments, we use llama2-7b and llama3-7b as source models and bloom-7b and mistral-7b as suspect base models. Notably, the structure of llama2-7b and llama3-7b are different (e.g. the tokenizer of the two models are different, and the vocabulary size of llama3 is increased from 32k to 128k). We conduct experiments comparing identical source and suspect models (e.g., llama2 to llama2) with different source and suspect models (e.g., mistral to llama2) under Unigram watermark and Code training data settings. The results are summarized in the table below.

The experimental results show that using the same base model for the suspect model as the source model increases watermark detection rates. This is particularly evident when the suspect model also has its base model used as an anchor model. Additionally, when employing mistral as the source model and llama2 as the suspect base model, the proposed method outperforms the baseline, achieving a notably high detection accuracy even with differing source and suspect base models.

Question 1: Comparison between LIDet and [1].

Compared to the method proposed in [1], our approach adopts a more realistic threat model, where both the suspect base model and its training data are unknown. This introduces greater challenges for watermark detection, rendering the method in [1] less applicable in such scenarios. Additionally, in i.i.d. detection, our method demonstrates higher stability. Specifically, in cases involving the Unigram watermark, the green ratio mismatch significantly affects the detection accuracy of [1], whereas our approach addresses this issue effectively, achieving superior performance.

Question 2: Should the set of anchor models always contain the same source model architecture?

The Anchor Model does not necessarily have to include the source model, but doing so is a straightforward choice since the detector can ensure the source model is completely unwatermarked. Due to the generalization capabilities of LLMs, various anchor models can sample text that aligns with natural language green ratios. Thus, the effectiveness of this method is not highly sensitive to the specific anchor model chosen.

Question 3: Impact of the suspect model's dimension.

Intuitively, the ability to detect infringement correlates with how well the suspect model learns the watermark. Compared to factors directly influencing model training, such as learning rate or the number of training steps, the size of the suspect model has a relatively minor impact on watermark detection (e.g., Table 6d in [1]). Furthermore, when owners of similarly sized models use larger datasets for instruction tuning, the watermark's detectability may actually increase due to the more extensive training.

Question 4: Correlation between detectability and domain of training queries

The domain of the data used to train the suspect model significantly impacts both the learning and detection of the watermark, thereby affecting its detectability. For example, when code-related queries are used for training, the sampled training data from the source model is limited to a code-specific distribution. Consequently, watermark tokens learned by the suspect model are often task-related (e.g., variable names, and comments). If the detector queries the suspect model using a domain with a different distribution (e.g., law or medical), the sampled data contains fewer watermark tokens, reducing detectability. Thus, our detection method aims to obtain a wider range of tokens and a more general distribution from the suspect model, enhancing watermark detection capabilities.

[1] Tom Sander et al. Watermarking Makes Language Models Radioactive, NeurIPS 2024.

Dear Reviewer RZ9s,

We are deeply grateful for your valuable comments and suggestions and we would like to know whether our rebuttal has addressed your concerns properly. We sincerely request your further feedback, as your insights are invaluable in helping us improve the quality of our work.

Best regards, Authors