ESpeW: Robust Copyright Protection for LLM-based EaaS via Embedding-Specific Watermark

Q2: If two benign datasets that are not identically distributed are selected, it is also likely to reject the null hypothesis. Therefore, the reliability of the hypothesis test proposed in this paper is doubtful.

R2: Thank you for your thoughtful consideration and valuable feedback. Here are our responses:

1. We analyze and verify that the false positive rate (FPR) is correlated with the trigger set size. FPR means the ratio of non-watermarked models are mistakenly identified as watermarked. To illustrate this, consider an extreme case where the trigger set size is set to just 1. In this case, the embeddings of watermarked texts show high semantic similarity due to the shared token, causing them to cluster too closely. As a result, even if a watermark has not been successfully injected, the watermarked and non-watermarked embeddings still exhibit differences, leading to the incorrect conclusion that the embeddings are watermarked.

2. For a more rigorous evaluation, we adopt another metric FPR@$f$, which is widely used in text watermark. FPR@$f$ indicates that the watermark are evaluated under the constraint that the FPR is lower than a threshold $f$. FPR@$f$ is highly suitable for our task because it allows us to evaluate the performance of the watermark under a fixed FPR. In the following table, we present the relationship between trigger set size and FPR. To ensure reliability, we conduct 100,000 repeated experiment for each size. All other parameters are same as in our work. Specifically, the verify dataset size is 40, the verify sentences' length is 20, the max trigger number in one sentence is 4. We can see that when the trigger set size is set to 4, we can only ensure an FPR of less than 0.3891. However, when the trigger set size is increased to 20 (i.e., the setting used in our paper), we can ensure an FPR of less than $10^{-4}$.

The above analysis demonstrates that our experimental results are reliable, since all experiments in our paper ensure FPR@$10^{-4}$, which is a sufficiently small value.

Q3: Experiments about robustness against permutation.

R3: Thanks. All our experiments are conducted under permutation attack by default. We have relevant statement in our paper (Line 834). Original content: To illustrate that all methods exhibit the Persistence-to-Permutation property described in Section 3.2, we assume that the stealer will apply the same permutation rule to all provider’s embeddings before training the stealer’s model.

Q5: (1) It would be better to test whether the watermark can be extracted from the unwatermarked models with different architectures. (2) Additionally, it’s important to confirm if randomly selected trigger samples and keys can pass the watermark verification. These experiments would better demonstrate the reliability of ESpeW.

R5: Thank you for your valuable feedback. We address your concerns from the following two points:

1. For "more unwatermarked models," we test the probability of the watermark can be extracted from the original model. To ensure representative results, we select models from the popular embedding model leaderboard, MTEB [1]. These models vary in architecture, model size, and embedding dimension. When setting the trigger set size to 20, which is used in our paper, the the probability of such an occurrence is as follows. It can be observed that the probability of extracting watermark from unwatermarked models with different architectures is extremely low (less than $10^{-4}$).

2. For "randomly selected trigger samples and keys can pass the watermark verification", we ensure all the experimental results presented in the paper are obtained under the condition that the probability of such an occurrence is less than $10^{-4}$. This is, in fact, the false positive rate we discussed earlier. For further details, please refer to Q2.

[1] MTEB: Massive Text Embedding Benchmark. https://huggingface.co/spaces/mteb/leaderboard. EACL 2023.

Q6: As shown in Table 1, the utility of the watermarked models is higher than the original models. Since the proposed method replaces part of the output embedding with arbitrary values, the results seem abnormal. Could you provide further analysis on this?

R6: Thanks for this concern. Actually, we have already explained this phenomenon in the manuscript (see Line 401)**, and proposed more reasonable evaluation using the cosine similarity between watermarked embedding and clean embedding. Original content: Evaluating embedding quality solely by the performance of downstream tasks is insufficient due to the randomness of DNN training. To better elucidate the influence of watermarks on embeddings, we compute the average cosine similarity between watermarked embeddings and original clean embeddings. Four watermarks are selected for comparison: EmbMarker, WARDEN, ESpeW (randomly selecting watermark positions), and ESpeW (selecting watermark positions with minimum magnitude). As depicted in Figure 3, the embeddings generated by our proposed method exert the least negative impact on clean embeddings, with a change in cosine similarity of less than 1%.

To be rigorous, if the model undergoes fine-tuning, the cosine similarity may also decrease, but this does not indicate a reduction in embedding quality. In our specific context, i.e., evaluating the impact of watermark on embedding, using cosine similarity between watermarked and clean embedding as a metric is appropriate.

Q4: The resistance of watermark to fine-tuning.

R4: Thank you for your question. We address your concern here:

We focus on unsupervised fine-tuning, as supervised fine-tuning is unsuitable for embedding models; it introduces excessive label information, undermining semantic integrity. To evaluate our method's robustness against fine-tuning attacks, we adopt the unsupervised fine-tuning approach SimCSE [1]. SimCSE uses contrastive learning by applying random dropout masks in the Transformer encoder. Positive samples are created by feeding the same input with different dropout masks, while negative samples come from other sentences in the batch.

In our experiment, we use the same hyperparameters as [1]: a learning rate of $3 \times 10^{-5}$ and a batch size of 64. We test on Enron Spam dataset. Fine-tuning introduces instability in embeddings, causing p-values to inflate abnormally and lose reliability, particularly with a significantly large epoch number. So, we use $\Delta \text{cos}$(%) and $\Delta l_{2}$ (%) for detection here. The metrics $\Delta \text{cos}$ (%) and $\Delta l_{2}$ (%), as defined in our paper, address this issue effectively. By adjusting thresholds of $\Delta \text{cos}$ (%) and $\Delta l_{2}$ (%), we maintain a false positive rate (FPR) below $10^{-5}$.

The table below demonstrates that, with the FPR $<10^{-5}$, this approach effectively defends against fine-tuning attacks, even after 100 epochs of fine-tuning. Considering that the stealing only undergoes 10 epochs, the cost of 100 epochs is significant.

Here are the thresholds we use. This is validated through 100,000 experiments on unwatermarked models. The values on the right indicate the thresholds of $\Delta \text{cos}$ (%) and $\Delta l_{2}$ (%) required to achieve the corresponding FPR on the left. For instance, at an FPR of $10^{-5}$, the thresholds are $1.09$ for $\Delta \text{cos}$ (%) and $-4.10$ for $\Delta l_{2}$ (%).

[1] SimCSE: Simple Contrastive Learning of Sentence Embeddings. Tianyu Gao, Xingcheng Yao, Danqi Chen, EMNLP 2021.

Q1: I think a (maybe more) practical scenario is that the adversary steals the model and uses the model for a downstream task. In such a scenario, the defender can get the final output predictions instead of the embeddings. Is ESpeW still effective?

R1: Thank you for your insightful comment. Here are our responses:

1. EaaS is a practical threat model. Many organizations have already started offering EaaS, such as GPT-3 text-embedding-002 API from OpenAI [1], mistral-embed from Mistral [2], Gemini text-embedding-004 from Google [3], and Embed of Cohere [4], etc. Existing work also accepts this attack setting [5][6][7]. So, developing watermarks for EaaS is practically meaningful.

2. As for extending our watermark to output predictions [8][9], we have some initial ideas. For example, when access to confidence scores of top-K labels is available, we could potentially insert watermark samples that do not change the top-1 prediction but influence the confidence distribution of other labels, which would make the watermark more imperceptible.

However, adapting our method to the output predictions introduces several key challenges: (1) The discrete nature of predictions limits the flexibility of watermark insertion, (2) The compression of information from embeddings to predictions reduces the capacity for embedding a watermark, and (3) The statistical properties of prediction distributions present additional complexities, necessitating further adjustments to our approach.

Although some principles from our embedding-based watermarking method may still be relevant, applying ESpeW to output predictions requires substantial modifications to the methodology. These modifications involve rethinking how watermarks interact with discrete outputs, managing the information compression of output labels, and addressing more attacks targeting prediction-level watermarks. Tackling these challenges is far beyond the scope of this paper, but we regard them as promising opportunities for future work.

[1] OpenAI, https://openai.com/index/new-and-improved-embedding-model.

[2] Mistral, https://docs.mistral.ai/capabilities/embeddings.

[3] Google, https://ai.google.dev/gemini-api/docs/embeddings.

[4] Cohere, https://cohere.com/embed.

[5] Stolenencoder: stealing pre-trained encoders in self-supervised learning. CCS 2022.

[6] Are you copying my model? protecting the copyright of large language models for eaas via backdoor watermark. ACL 2023.

[7] WARDEN: Multi-directional backdoor watermarks for embedding-as-a-service copyright protection. ACL 2024.

[8] PLMmark: A Secure and Robust Black-Box Watermarking Framework for Pre-trained Language Models. AAAI 2023.

[9] Watermarking Pre-trained Language Models with Backdooring. Arxiv 2022.

Thank you for your time and valuable comments, which will help improve our paper. We address your questions as follows:

Q1: The contribution seems limited, as the primary difference from EmbMarker lies only in the embedding equation.

R1: Thank you for your valuable feedback. In fact, in the watermark field, the core difference often lies in the watermark injection algorithm, and it's common for different approaches to follow similar steps. For instance, both EmbMarker and WARDEN follow this common framework. Our method stands out due to the following key points:

1. The primary motivation for developing embedding-specific watermarks is fundamentally different from EmbMarker (Figure 2 in paper for illustration). Unlike traditional fragile watermarking methods like EmbMarker, which injects an embedding-shared watermark into all embeddings and is therefore easier to identify and eliminate, our approach focuses on embedding-specific watermarks that are much harder to identify and eliminate. This shift in focus highlights the uniqueness of our contribution and its importance in advancing the applicability of watermarking in EaaS.
2. Our method demonstrates robustness against the watermark removal method CSE, whereas EmbMarker does not (Table 1 in paper). This highlights the practicality of our approach for real-world deployment, ensuring that our method can effectively preserve watermark integrity even in adversarial scenarios. This represents the fundamental distinction from EmbMarker.
3. Our method induces minimal distortion (less than 1%) to the clean embedding, whereas EmbMarker modifies approximately 3% (Figure 3 in paper). Given that embedding quality is crucial for EaaS providers, this improvement is highly significant.
4. The core of a watermark lies in the injection mechanism, and our ESpeW method introduces a novel embedding-specific injection mechanism that we believe provides a significant advancement in the area. While other methods may follow similar high-level procedures, our unique injection mechanism enhances robustness while minimizing alterations to the embedding, which we believe a meaningful step forward in watermarking technology.

In summary, while the research problem and tasks in watermarking are aligned with previous works, our approach offers a new perspective on robust watermarking with practical, real-world benefits. We believe our work fills a crucial gap in copyright protection for EaaS applications. Thank you once again for your feedback, and we hope this clarifies the novelty and significance of our contributions.

Q2: The contributions may be overstated. This paper may not be the "first to propose" a robust watermark approach against CSE, as WARDEN [2] has already addressed this.

R2: Thank you for pointing out these important issues. We appreciate your suggestion and thoroughly analyze the matter. First, we conclude that WARDEN can not actually defense against CSE. The misleading results in WARDEN arise from their experimental setup, where they increase the number of watermarks to 5 but keep the total trigger set size at 20, resulting in each watermark having only 4 triggers. Their use of an excessively small trigger set size leads to false positives(verified in point 1, 2). False Positive (FP) means non-watermarked models are mistakenly identified as watermarked. Consequently, when they evaluate WARDEN under the CSE attack, even though CSE has already removed the watermark, they still mistakenly conclude that the watermark exists due to false positives (verified in point 3). Also, we verify all experiments in our paper ensure a very low false positive rate (FPR) of $10^{-4}$ (verified in points 2, 4). For rigorous, we have stated in revision paper that our method is the only one robust under the listed baselines by adding 'To the best of our knowledge' (Line 075).

Due to limited space, we kindly invite you to refer to the next block for more details.

Q3: At least 15% of each embedding's values are directly replaced with the corresponding values from the target embedding e_t. By statistically analyzing the frequency of values at each position, e_t might be estimated.

R3: Thank you. You mention an adaptive attack based on statistical analysis. We address your question from two points:

1. The statement of direct replacement is not accurate. Note that we perform a normalization operation on the embedding before returning it, which changes the values of the embedding. After normalization, the same watermarked positions in the embedding no longer have the same values. Note that the Provider's EaaS normalizes the embedding before returning it. This means the embedding is divided by its L2 norm (a common technique used in embedding processing). This normalization process ensures that, even though we add the same value to the same positions in the embedding, after normalization, the values at those positions are no longer the same. Therefore, in fact, it is chanlleging to conduct the statistical analysis attack.

2. Experimental results demonstrate that statistical analysis attacks will not succeed unless watermark quality is degraded to as low as 64.78% or even 28.35% of the original. We first provide a detailed description of the statistical analysis attack here.

1. Assume that the training set of the stealer is $D_c \in \mathbb{R}^{N \times M}$, and for a specific index $i$ of embedding, the corresponding array is $DE_i \in \mathbb{R}^N$.
2. Set a small tolerance level $T$, and using this tolerance as the step size to partition $DE_i$ and count the number of elements in each partition.
3. Initialize $SE = \{\}$. Then, add the partition with the highest number of elements to $SE$. This is because, when the tolerance is set to a particularly small value, if the watermark values cluster, these watermark values are likely to cluster within a specific partition and its neighboring partitions. Next, we add these $N_T$ neighboring partitions around the clustered partition to $SE$.
4. Calculate the upper and lower bounds of $SE$, and set the numbers within this interval to $0$.
5. Repeat steps 1-4 for all indices $i$.
6. Normalize the obtained embedding.

Through this algorithm, we can identify the abnormally clustered values, thereby carrying out the statistical analysis attack. In our experiments, we fix $T$ to a small value $10^{-4}$ and test the attack performance with varying $N_T$. Since the SAA operation only have negative affect on embedding quality, we can use cos-clean only (the cosine similarity between the embedding and clean embedding) to measure watermark quality. All other parameters the same as in our paper. The results are as follows:

The results show that with $N_T$ set to 200, p-value based detection becomes ineffective in identifying watermarks, while the watermark quality degrades to 64.78% of its original level. But in this situation, the ∆cos and ∆l2 is still high, which can be used to detect watermark. When $N_T$ is set to 200, our watermark are ineffective with an embedding quality of 45.11%.

Q4: When α is set to 100% in Eq. (1), the proposed method is significantly different from EmbMarker.

R4: Thank you for your valuable feedback. When $\alpha=1$, our method replaces entirely the original embedding with the target embedding instead of same as EmbMarker. We have made the revisions for more rigorous expression (Line 426).

Q5: Typo issue.

R5: Thank you; we will make sure to correct it thoroughly.

Continuing from the previous block (Q2).

Below, we present a step-by-step analysis to support our claims:

1. We first preliminarily verify that small trigger set sizes will lead to false for WARDEN. We re-test WARDEN with different total trigger set size on unwatermarked model. Our experiments are conducted using the official open-source code of WARDEN, ensuring that our results can be easily verified. All other parameters are the same as theirs. We can see that, following their setting, where the total trigger set size is 20, false positives occur. In fact, the authors of WARDEN already mention the issue of high false positives in their paper, but they do not find the underlying reason for the false positives is because the small trigger set size.

2. We analyze and verify that the false positive rate (FPR) is correlated with the trigger set size. FPR means the ratio of non-watermarked models are mistakenly identified as watermarked. To illustrate this, consider an extreme case where the trigger set size is set to just 1. In this case, the embeddings of watermarked texts show high semantic similarity due to the shared token, causing them to cluster too closely. As a result, even if a watermark has not been successfully injected, the watermarked and non-watermarked embeddings still exhibit differences, leading to the incorrect conclusion that the embeddings are watermarked.

For a more rigorous evaluation, we adopt another metric FPR@$f$, which is widely used in text watermark. FPR@$f$ indicates that the watermark are evaluated under the constraint that the FPR is lower than a threshold $f$. This metric is highly suitable for our task because it allows us to evaluate the performance of the watermark under a fixed FPR. In the following table, we present the relationship between trigger set size and FPR. To ensure reliability, we conduct 100,000 repeated experiment for each size. Other parameters that might influence FPR are set as follows: the verify dataset size is 40, the verify sentences' length is 20, the max trigger number in one sentence is 4, and the model is considered to have a watermark if the p-value is less than $10^{-3}$. We can see that when the trigger set size is set to 4, we can only ensure an FPR of less than 0.3891. However, when the trigger set size is increased to 20, we can ensure an FPR of less than $10^{-4}$.

3. Based on the above analysis, we formally demonstrate here that WARDEN cannot actually defend against CSE by setting trigger set size to 20 for each watermark (total 100 for 5 watermarks). This ensures that the FPR is lower than $10^{-4}$. This test is conducted on watermarked models, with all other parameters are the same as theirs. The results show that when K (hyper-parameter of CSE) is greater than or equal to 50, WARDEN fails to extract the watermark, i.e, fails to defense against CSE.

4. The results of our paper are reliable, since all experiments in our paper ensure FPR@$10^{-4}$, which indicates the FPR is lower that $10^{-4}$. In our paper, the experiment sets the trigger set size for each watermark to 20. According to the table above, we can see that when the trigger set size is 20, we actually meet FPR@$10^{-4}$. $10^{-4}$ is a sufficiently small value.

Dear Reviewer zbHr,

Thank you again for your time. As the deadline for discussion is approaching, we do wish to hear from you to see if our response resolves your concerns. We are happy to provide any additional clarifications if needed.

Thank you for carefully reading our paper. We thank the reviewer for the constructive comments and suggestions. We address your concerns below:

Q1: If this target embedding (private key) was compromised, attackers could potentially reverse-engineer the watermark positions.

R1: Thank you for your comment. We would like to explain from the following two points.

1. The private position is hard to infer. The EaaS provider would normalize embeddings before returning the embedding, ensuring added values at same positions are no longer same. This makes it challenging to pinpoint watermark positions even if the key is compromised.
2. Key leakage risks and strategies. The leakage risk mainly comes from security vulnerabilities such as poor storage, insecure transmission, or insider leaks. Mitigation strategies: (1) Regularly renew the key. (2) Use multiple keys to limit impact. (3) Audit and monitor access. (4) Encrypt storage and transmission. (5) Limit employee access.

We have added the discussion about dealing with privacy key leakage to revised version. (Line 1301)

Q2: High computational load to select positions with the lowest magnitudes.

R2: Thank you for your insightful question. We would like to refer you to our response to reviewer nW4C in R1 for a systematic analysis about comparison of random selection and smallest-magnitude selection. Or see the revised paper (Line 859).

Q3: The method may be model-specific since different models can produce embeddings with varying distributions and magnitudes.

R3: Thank you for your question. We clarify it from three aspects:

1. Our method is model-agnostic. The mechanism of our method is independent from model and can be applied to any EaaS system.
2. Our watermark is specific to the embedding instead of model. The watermark added to each embedding is unique. This means that the watermark is not a fixed pattern but is instead dynamically generated based on the properties of each individual embedding. By binding the watermark to the specific embedding, we ensure that the watermarked embeddings are more robust against removal attack, as there is no universal watermark template that can be easily extracted or removed.
3. To strengthen our point, we apply our watermark to more models to verify its effectiveness. We select two additional embedding models: NV-Embed-v2 (the top model in the MTEB Leaderboard, developed by Nvidia with Mistral-7B, embedding dimension 4096) and Stella-1.5B-V5 (the top 1.5B model in MTEB, based on Qwen2-1.5B, embedding dimension 1024). We also put test on GPT-3 text-embedding-002 API (embedding dimension 1536) here for comparison. Using the Enron spam dataset and $K=50$, we evaluate watermark performance with different $\alpha$, keeping other parameters the same as in our main experiment.

Here are our experimental results, which show that our watermark is effective across all three models. The only difference is that the optimal detection performance is achieved at slightly different alpha values for each model. We have added related content in revised paper (Line 1059).

Dear Reviewer qoiC,

Thank you again for your time. As the deadline for discussion is approaching, we do wish to hear from you to see if our response resolves your concerns. We are happy to provide any additional clarifications if needed.

Thank you for recognizing our work and providing valuable suggestions. We address questions and concerns below.

Q1: High computational load to select positions with the lowest magnitudes.

R1: As we claimed in our paper (Line 530), we have provided an alternative approach, i.e., random selection, to reduce computational load. In the following, we present a systematic analysis about comparison of random selection and smallest-magnitude selection:

(1) We first give a detailed description of Random Selection algorithm. Direct random selection is not ideal, as the watermarked positions for the same sentence can vary across queries. An attacker could exploit this by using multiple queries to detect or remove watermark. To address this, we use hash value of embedding as a seed, ensuring consistent position selection. Below is the algorithm:

• Convert the original embedding $e_o$ to byte format;
• Generate a random seed using the SHA-256 hash of the byte format of $e_o$;
• Select random indices based on the generated random seed. These indices are watermark positions.

(2) We then evaluate the time consumption of Smallest-magnitude and Random Selection through both analysis and experiments.

For analysis: we conduct the time complexity analysis:

• Smallest-magnitude Selection: Using heap sort for the top-k problem is the most common approach, achieving a time complexity of $O(N \log k)$. Thus, the total time complexity of Smallest-magnitude Selection is $O(N \log k)$.
• Random Selection: Converting $e_o$ to byte format requires $O(N)$, SHA-256 hashing also takes $O(N)$, and selecting random indices needs $O(k)$. Therefore, the total time complexity of Random Selection is $O(2N + k)$.

Considering the high-dimensional nature of embeddings, random selection typically has a much lower time complexity than smallest-magnitude selection.

For experiments: To test time consumption, we use two popular embedding models: NV-Embed-v2 (based on Mistral-7B) and Stella (based on Qwen2-1.5B). We measure time for 2,000 generations, repeating the experiment 5 times to reduce randomness. Experiments run on Ubuntu 18.04 with an AMD EPYC 7Y83 64-Core CPU and a 4090 GPU.

(3) Below is the comparison of watermark performance using smallest-magnitude and random selection. We report p-value, ∆cos, and ∆l2 for detection capability, and cosine similarity to assess embedding quality. We set $K = 50$. The other parameters are the same as in the paper.

In summary, above analyses and experiments persent that both smallest-magnitude selection and random selection have their irreplaceable advantages and suited to their respective application scenarios:

• Smallest-magnitude selection significantly benefits preserving embedding quality, with modifications to clean embeddings under 1%. This is crucial for real-world scenarios where organizations aim to achieve higher rankings on leaderboards to promote their products while protecting their copyright.
• Random selection, in contrast, though sacrificing more embedding quality, saves considerable time, making it more suitable for product deployment.

We believe that both approaches are meaningful, and users can choose between them based on their specific application scenarios. We have included these analyses and experiment results in revised paper (Line 859). We're open to further feedback.

Dear Reviewer nW4C,

Thank you again for your time. As the deadline for discussion is approaching, we do wish to hear from you to see if our response resolves your concerns. We are happy to provide any additional clarifications if needed.