Ward: Provable RAG Dataset Inference via LLM Watermarks

We thank the reviewer for their detailed feedback. We reply to the reviewer’s concerns below, numbered Q1-Q6, and point out that we uploaded a new revision of the paper, with new additions marked in purple. We are happy to discuss any outstanding points further.

Q1: How does Ward perform when only one of the retrieved documents is watermarked?
As this in fact holds in all our experimental runs in the IN case, we believe there may be a misunderstanding here, possibly caused by our misleading formulation of the perfect retrieval mechanism on L340. In our new revision we replace that wording and point to the new App. E, which contains a precise explanation of perfect retrieval along with two examples. In short, (1) each query of Ward is done using a single document, thus we do not expect more than one watermarked document to end up in the context; and (2) as we set $k \geq 3$ in our experiments, the context will contain “distraction” documents, which in the Hard setting will primarily be from the same FARAD group, but may also be from other groups. We hope this resolves the question, and we encourage the reviewer to follow up otherwise.

Q2: Does Ward still perform well with more than 3 retrieved documents in RAG?
Yes—we point the reviewer to our experiments with $k=4$ and $k=5$ presented in App. A.1 and referenced on L485, that the reviewer may have missed. Our choices here were guided by prior work on RAG such as e.g., [“FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Research”, Jin et al. 2024] which recommends $k \in [3,5]$ (Figure 2, Left). This matches the values used in the RAG MIA baselines introduced in Sec. 2 [Li et al., 2024, Anderson et al., 2024], which set k=5 and k=4 respectively.

Nonetheless, we follow the reviewers suggestion and run a new experiment with Ward with $k=10$, FARAD-Hard, both Naive-P and Def-P prompts, 5 seeds, and Claude 3 Haiku (as this experiment often exceeds the context size of GPT-3.5, which was used in our experiments in App. A.1). All 10 runs were successful in both IN and OUT cases, and Ward converged to 100% accuracy in around 50 queries, similar to baseline Haiku results in Fig. 5 (Left). This reaffirms our conclusion that Ward is robust to different values of $k$. We will extend this experiment further and add the complete results to the next version of App. A.1.

Q3: Can you empirically support the provability claims made in the paper, and provide corresponding rigorous proofs?
We would like to first clarify that the notion of “provability” that we consider, and claim holds for Ward, is only the one we introduce on L241 under Guarantees, i.e., well-controlled small rate of False Positives / type 1 errors. In other words, if Ward concludes that the data was used in a certain RAG system, this claim is false only with a very small known probability. This is the standard notion of provability in LLM watermarking, naturally motivated by the high cost of false positives, which is certainly the case in RAG-DI. In fact, as noted on L397, our proof would also directly follow from the proof for the underlying scheme of Ward [Kirchenbauer et al., 2024], thus we did not repeat it. We agree however that formalizing this within our paper could be beneficial and make the paper more self-sufficient, so we are happy to work on this for the next paper revision. We also point out Fig. 6 and Fig. 8 as relevant, that empirically show the validity of our p-values, which quickly drop as we increase the number of queries for IN cases (true positives), but stay consistently close to the expected value of 0.5 for OUT cases (no false positives).

We certainly had no intention to claim that Ward is provably always able to discover unauthorized data usage, i.e., that it has a perfect rate of True Positives, as this is in principle impossible. For example, if an LLM refuses every request, the worlds where our data was (IN) or was not used (OUT) are indistinguishable. Thus, we note that aspects like scalability and robustness are strictly orthogonal to the discussion around provability. Of course, they are still crucial as empirical evidence of the efficacy of Ward, and we have investigated them in detail in our experiments in Sec. 5, showing that Ward is efficient and robust to many setting variants. Regarding scalability in particular, we have now included an extensive discussion in our new App. F on practicality and future work (paragraph “Efficiency”).

We are happy to consider any concrete suggestions that the reviewer has around how to improve the phrasing of provability in the paper.

We thank the reviewer for their valuable feedback. We respond to the reviewer’s concerns below, numbered Q1-Q4, and point out that we uploaded a new revision of the paper, with new additions marked in purple. We are happy to continue the discussion in case of follow-up questions.

Q1: Can you discuss how the data used in your work models complex real-world scenarios?
From the model owner’s side, while we certainly agree that this can be further improved, we believe FARAD is a solid baseline for future work in terms of how well it reflects realistic cases, greatly improving over the prior state where there were no suitable datasets that (1) are provably not leaked and (2) contain fact redundancy. In a newly added App. F (referenced from Sec 3.1), we recap the key design decisions of FARAD, and explicitly point out several promising directions in which FARAD can be improved in future work, including the directions suggested by the reviewer (domain and language shift), which would enable the evaluation of Ward in more diverse settings. From the RAG provider’s side, we remark that the presence of documents in other languages or from other domains in the RAG corpus should generally not affect the performance of Ward as a RAG-DI method. In particular, if the RAG pipeline effectively handles the challenge of a multilingual and/or multidomain corpus, Ward should be directly applicable.

Q2: Can you discuss the efficiency of Ward?
Certainly. While our first draft contained some efficiency/complexity discussion, evaluating Ward in terms of query cost, and showing that at most 100 queries are needed for a confident decision across all settings, we acknowledge that this concern was grouped with Monotonicity and not extensively discussed in one place. In the new revision we added a unified discussion of Efficiency to our new App. F focused on practicality and future work, and referenced it in the main paper.

In particular, we translate our query cost result above into dollar cost of closed-model APIs, concluding that only \$0.63 is needed to apply Ward to GPT-3.5, and at most \\$2 for any popular state-of-the-art closed model. We further discuss why the query complexity of Ward is independent of the size of both the RAG corpus $D$ and the owner dataset $D_{do}$, substantiate that large $|D_{do}|$ is not an obstacle. All these results point at Ward being practically applicable. Finally, in App. F we point out our preliminary attempts to further improve the sample efficiency of Ward (App. A.4), and conclude with ideas for future work.

Q3: How is the performance of Ward affected by the type of LLM used?
Our main experiments, performed on the Llama3.1-70B open model and two popular closed models, have demonstrated that Ward is robust to this change. We do however observe the difference in the “difficulty” of the task for different models, which we hypothesize is often due to the models’ approach to following instructions such as the defense prompt. We certainly find it interesting to further quantify this, but remark that we foresee no case where Ward would fail due to the choice of the LLM, especially as LLM watermarks were demonstrated to work reasonably well on a wide range of models [Piet et al. 2023, Kirchenbauer et al. 2024], and most recently on the production deployment of Gemini [“Scalable watermarking for identifying large language model outputs”, Datathri et al., 2024, Nature]. We are happy to discuss this further if the reviewer has follow-up questions.

Q4: How does the update frequency of data impact Ward?
Due to the fundamental properties of RAG, we do not believe this significantly impacts the success of Ward. As long as the data used for querying was present in the RAG corpus throughout the querying, and the retrieval mechanism works roughly as expected, Ward should generally be successful, as demonstrated in our experiments. This holds independently of other documents being added/removed, and such changes to the RAG corpus do not necessitate any modifications to Ward before it is run.

We thank the reviewer for their detailed comments. We respond to all questions raised by the reviewer below, numbered Q1-Q4, and point out that we uploaded a new revision of the paper, with new additions marked in purple. We are happy to continue the discussion further if the reviewer has follow-up questions.

Q1: Can you discuss the importance of RAG-DI and the applicability of Ward in realistic scenarios?
We are glad that the reviewer agrees that the concern underlying RAG-DI exists in practice, and we acknowledge that the paper could have included a more thorough discussion of the motivation and the importance of this problem setting, example use-cases, and the practical applicability of Ward. We now introduce all of these in a new App. D, referenced in the first paragraph of Sec. 1. The applicability of Ward is also extensively discussed in our new App. F from the perspective of efficiency (see Q3 below) and countermeasures (see Q4 below). We encourage the reviewer to point out if there are aspects they believe we missed in this new discussion, and we are happy to extend this section accordingly.

Q2: Does the data owner need to query every suspect LLM to test for unauthorized data usage?
As we discuss in the new App. D, this is naturally true, as querying $\mathcal{M}$ is necessary to make any claim about $\mathcal{M}$. However, we do not see this as prohibitive, for several reasons. First, a single application of Ward is very cheap, below \$2 for the whole process of RAG-DI, with potential for further improvement (see Q3 below). Second, the ecosystem of relevant LLM providers is relatively small. In particular, not many providers both (i) have resources for indiscriminate large-scale scraping of data and (ii) are sufficiently popular to create market harm based on the unauthorized usage of that data. Thus, we believe applying Ward in real use-cases is practical from an efficiency and cost standpoint.

Q3: Is scalability an issue for Ward? What if $D_{do}$ is large?
Good question. While our first draft contained some efficiency/complexity discussion, evaluating Ward in terms of query cost, and showing that at most 100 queries are needed for a confident decision across all settings, we acknowledge that this concern was grouped with Monotonicity and not extensively discussed in a single place. In the new revision we added a unified discussion of Efficiency to our new App. F focused on practicality and future work, and referenced it in the main paper.

In particular, we translate our query cost result above into dollar cost of closed-model APIs, concluding that only \$0.63 is needed to apply Ward to GPT-3.5, and at most \\$2 for any popular state-of-the-art closed model. We further discuss why the query complexity of Ward is independent of the size of both the RAG corpus $D$ and the owner dataset $D_{do}$, and substantiate that large $|D_{do}|$ is not an obstacle, as the data owner by design uses only a subset of those as queries, and can also decide to not watermark all of $D_{do}$. All these results substantiate our argument that Ward is viable to apply in practice. Finally, in App. F we now point out our preliminary attempts to further improve the sample efficiency of Ward (App. A.4), and conclude with ideas for future work in this direction, such as using a preliminary round of a few queries to establish reasonable suspicion, followed by the full application of Ward (while making sure to preserve the statistical soundness).

Q4: Can you discuss the possible countermeasures to Ward?
Yes—similar to Q3, we have added a detailed discussion of countermeasures to our new App. F and referenced it in the main paper. To summarize these new additions, we first point out that App. A.5 demonstrates the robustness of Ward against a tailored defense based on MemFree decoding [Ippolito et al., 2023] which prevents n-gram overlaps between the response and the retrieved documents. We proceed to discuss ideas such as preprocessing of the RAG corpus or postprocessing of the LLM responses to remove the watermark. We argue that as identifying watermarked documents is hard, the RAG provider would need to paraphrase every incoming article, which incurs a significant overhead and is not practically feasible. We conclude that from this perspective, it may be more feasible for the RAG provider to legally acquire the data, which is an interesting positive side-effect of having practical RAG-DI tools. Of course, future work may develop more sophisticated defenses against Ward, which we certainly welcome.

Thanks for your detailed concerns. I think my concerns have been thoroughly addressed. I will raise the score to 6.

We thank the reviewer for their comprehensive feedback. We respond to the raised concerns below, numbered Q1-Q4, and point out that we uploaded a new revision of the paper, with new additions marked in purple. We are happy to continue the discussion further if the reviewer has additional questions.

We appreciate the title change suggestion and the author’s acknowledgement of the breadth of our work, and are discussing several options among the coauthors.

Q1: How does perfect retrieval work for k=3 (Easy) and k=5 (Hard)? Does it always retrieve only articles from the same source/group?
No; this is a misunderstanding caused by our misleading wording on L340. In our new revision we remove that wording and point to the new App. E, which now contains a precise explanation of perfect retrieval along with two examples that exactly correspond to the two cases outlined by the reviewer, k=3 (Easy) and k=5 (Hard). In short, perfect retrieval always returns $k$ documents, prioritizing first the document of interest (if present), then documents from the same FARAD group, and finally all other documents. We note that this can easily result in documents from different sources or FARAD groups being present in the context. We hope this helps clear up the misunderstandings.

Q2: Do baselines perform better in the Hard setting with imperfect retrieval?
This is an interesting question and a reasonable hypothesis. Prompted by this question, we ran a new experiment, evaluating all 3 baselines with end-to-end RAG retrieval (as in Sec. 5.3 and App. A.3), Llama3.1-70B, FARAD-Hard setting, and Def-P system prompt, on 5 seeds. Our results are very similar to the corresponding section of Fig. 4 which uses perfect retrieval, i.e. all baselines fall short, FACTS by having 100% FPR, and AAG and SIB by having 0% TPR. This demonstrates that our main experiments are a valid proxy for imperfect RAG retrieval, matching our prior result (L453) that shows 93.6% agreement between the two retrievers. We will extend this new experiment and include a more comprehensive version of the results in the next iteration of the paper.

Q3: Does the FARAD generation pipeline reflect real-world data well?
While we certainly agree that this can be further improved, we believe FARAD is a solid baseline for future work in terms of how well it reflects realistic cases, greatly improving over the prior state where there were no suitable datasets that (1) are provably not leaked and (2) contain fact redundancy. In a newly added App. F (referenced from Sec 3.1), we recap the key design decisions of FARAD, and explicitly point out several promising directions in which FARAD can be improved in future work, including the approach suggested by the reviewer. We hope this helps better position our dataset contribution and makes building on top of our work easier.

Q4: Is Def-P applied by the RAG provider or the model owners via queries? If the first interpretation is correct, why would a RAG provider use Def-P to instruct the model not to answer questions?
The first interpretation is indeed correct: Def-P is applied from the RAG providers’ and not the data owners’ side. However, Def-P does not instruct the model to never answer questions related to the retrieved content—this seems to be a misconception, likely caused by our wording on L323. In the new revision, we have added a pointer to the full Def-P prompt, and clarified that it actually instructs against verbatim text regurgitation and complying with user requests to learn about the exact contents of the LLM context. Both of these only prevent unintended/undesirable uses, with the latter one being analogous to the ongoing attempts to protect proprietary system prompts.