How Contaminated Is Your Benchmark? Measuring Dataset Leakage in Large Language Models with Kernel Divergence

We sincerely thank the reviewer for the thorough and constructive feedback. Below, we address the concerns in detail.

---

A1. Baseline comparison in Table 5 (Pile dataset)

We provide the comparison below. KDS achieves the highest average correlation.

A2. Significance over Perplexity

In the A1 table above, KDS outperforms Perplexity on the Pile dataset, with an average correlation of 0.944 compared to 0.144.

A3. Negative correlations

We resonate with the reviewer’s concern. To address the potential causes, we reviewed our implementation and found no errors. Since our code directly builds on established prior work, an implementation error is unlikely (code will be released).

---

A4. Main metrics and Practical utility

(1) Main metrics

We agree that absolute contamination estimation and exact correspondence would be valuable. However, a necessary step toward this goal is to first validate the correctness of any scoring functions. The use of Spearman/Pearson correlation in our controlled experiments allows us to rigorously assess whether KDS (and other approaches) produces consistent and monotonic metrics, necessary conditions for any meaningful scoring function. This provides dev-purpose metric to ensure score behaves predictably and reliably in principle, which is essential for deploying it in practical settings. To our knowledge, no prior work has investigated this practically important angle.

Concerningly, we discovered existing scores do not satisfy these necessary conditions. In contrast, KDS consistently satisfies these conditions under varying benchmarks (including the challenging ones), providing stronger foundations for future method design.

(2) Practical utility

The lack of access to pre-training corpora makes ground-truth estimation fundamentally challenging. Moreover, pre-training data is often non-overlapping across models, further complicating any cross-model normalization or comparison. To our knowledge, there is no existing work that has effectively estbalished correspondence between the scoring function and dataset contamination. This reflects the intrinsic difficulty of the problem, rather than a unique shortcoming of our approach.

That said, we would like to emphasize that KDS is highly practical, which supports real-world applications such as safety auditing and benchmark selection, where knowing which dataset is more contaminated is often more important. It enables dataset creators or auditors to prioritize and select the least contaminated benchmarks for evaluation purposes. We state the practical utility in Lines 108-117.

---

A5. Correlation between benchmark difficulty and KDS

Thank you for this insightful question. During rebuttal, we computed the correlation of {Llama3.1-8b, Mistral-7b, Qwen2.5-7b}'s performance on {GSM, MPP, MPL, TFQA} with KDS. The correlation values are 0.2, 0.4, 0.2, respectively, which are all very low. This indicates that benchmark difficulty is not what determines the scale of KDS.

---

A6. Why do we need Requirement 2?

While the two requirements could be combined, separating them into orthogonal objectives offers clearer guidance for future research by isolating distinct performance aspects.

---

A7. Clarification on Figure 2 (left)

It is true that unseen samples are similar to each other in Figure 2 (left), and we would like to clarify that this is the similarity before fine-tuning. However, our method relies on the changes in the similarity or distance before and after fine-tuning, which is much more significant for unseen examples. As shown in Figure 2 (middle, without gating) and Figure 2 (right, with gating), unseen samples exhibit significantly larger shifts in embedding geometry, whereas seen samples remain relatively stable. This differential behavior is the core signal that KDS captures.

Ps. The performance of only using Figure 2 (left) is in Table 2 "w/o (2) Fine-tuning".

---

A8. Is comparison with SRCT accurate? We preserved the canonical ordering of samples in SRCT by not shuffling the dataset.

We thank the reviewer for the thoughtful and constructive feedback. Below, we address your concerns in detail.

---

A1. Related work

We thank the reviewer for pointing out the work by Dekoninck et al. [1], which appears to be highly relevant! It's our oversight at the time of submission and will make sure to discuss it in the updated version.

[1] Dekoninck et al "Constat: Performance-based contamination detection in LLMs." NeurIPS 2024.

---

A2. Open weight model

We acknowledge that KDS currently requires open-weight models. This is a fair point, and we will clearly note this limitation in the paper. However, we'd like to emphasize that focusing on open-weight models is valuable for academic research because they provide transparency essential for developing a deeper understanding of how contamination manifests inside LLMs. In future works, adapter-based probing techniques can be explored to extend its principles to proprietary models.

---

A3. Significance w.r.t. min-K%

While Min-K% works well on easy benchmarks such as WikiMIA and BookMIA, its performance degrades significantly on more challenging datasets. For example, in the three hardest PILE subsets in Table 5, Min-K% has significantly lower performance than KDS, as shown below (numbers are Spearman correlation):

---

A4. Practical utility

We agree that absolute contamination estimation is valuable. However, a necessary step toward this goal is to ensure that any proposed contamination scoring function satisfies core correctness criteria—namely, monotonicity and consistency with ground-truth contamination levels in controlled settings. These conditions ensure that the score behaves predictably and reliably in principle, which is essential before deploying it in practical settings. To our knowledge, no prior work has investigated this practically important angle.

Concerningly, we discovered existing scores do not satisfy these necessary conditions. In contrast, KDS consistently satisfies these conditions under varying benchmarks (including the challenging ones), providing stronger foundations for future method design.

Lastly, we would like to emphasize that KDS is also highly practical, because it supports real-world applications such as safety auditing and benchmark selection, where knowing which dataset is more contaminated is often more important. To further demonstrate the practical utility, we include an in-the-wild evaluation across 11 public benchmarks (Appendix E). For these reasons, we believe KDS offers a meaningful and practical step forward for the field.

---

A5. Minimal contamination rate to be regarded as "significantly contaminated"

The required rate to be considered "significantly contaminated" is 0.10 (p=0.014) under significance level $\alpha=0.05$, and 0.15 (p=0.004) under $\alpha=0.01$.

---

A6. Histogram on the values of embedding similarity change

Since plots cannot be updated or shown at this stage, we provide the histogram as a frequency table below. The table is retrieved from a contamination rate of 0.5.

A7. Embedding similarities should increase for unseen data. Why take the absolute change?

We would like to clarify that the embedding distances increase (i.e., embedding similarities decreases) for previously unseen data after fine-tuning. This is also evidenced in the table above, where $\log \frac{\Phi(Z)}{\Phi(Z')} = ||Z'_i-Z'_j||^2_2 - ||Z_i-Z_j||^2_2$ is mostly positive for unseen-unseen pairs.

We take the absolute value of the change to ensure that we capture the magnitude of deviation, regardless of direction. This makes the score robust to mixed behavior and focuses on detecting how much the geometry shifts. We will clarify this in the final version.

---

A8. Effect of Fine-tuning and Rephrased Data

Following the suggestion, we fine-tuned "unseen" samples from WikiMIA and Arxivtection, observing KDS increases of 22% for WikiMIA, and 11% for Arxivtection. This shows our scoring approach's ability to effectively capture contamination introduced by fine-tuning.

Additionally, we experimented on WikiMIA by mixing unseen data with rephrased "seen" data at varying proportions, yielding a Spearman correlation of -0.791. This outcome is expected, as rephrasing likely blurs the distinction between seen and unseen samples.

---

A9. Other suggestions

Thank you for all the constructive suggestions! We will incorporate those comments, to (1) mention the use of LoRA in the main paper, (2) bold-face all the best performances in Table 1, and (3) add a histogram based on our response, A6.

We thank the reviewer for the thoughtful and constructive feedback. Below, we address the key concerns:

---

A1. Clarification on problem setting

We absolutely agree with your viewpoint that contamination in real LLMs occurs during pre-training! We'd like to clarify that our central goal is indeed to quantify contamination originating from pre-training, not fine-tuning. Our methodology and evaluation are designed around pre-training leakage.

In particular, we ask this question (Sec 2.1):

Given an LLM $\mathcal{M}$ and a benchmark dataset $\mathcal{D}$ (e.g., WikiMIA), to what extent has this dataset been exposed during pre-training?

To address this, our key idea is the following:

If the model $\mathcal{M}$ has already been pre-trained on $\mathcal{D}$, then if we further fine-tune $\mathcal{M}$ on $\mathcal{D}$, the model would have minimal changes in embeddings due to prior exposure during pre-training (vice versa).

Hence, we use a lightweight fine-tuning step as a probe to reveal how much of the evaluation dataset was likely seen during pre-training. In other words, we are not using fine-tuning to inject contamination, but only as a mechanism to probe for the presence of whether $\mathcal{D}$ is used in pre-training. This design choice aligns with recent work (e.g., Zhang et al. 2025, FSD) that also leverages fine-tuning to surface memorized content—but our approach uniquely leverages embedding-level structural changes, enabling a more holistic dataset-level assessment.

---

A2. "the contamination is artificially introduced via fine-tuning, not pre-training"

We would like to clarify that our experimental setup simulates contamination w.r.t. pre-training, which is precisely the phenomenon our method aims to quantify. Specifically, our controlled experiments leverage datasets such as WikiMIA, with two subsets: seen/unseen in the pre-training corpus of the LLM (e.g., Mistral-7B). By mixing these pre-training seen and unseen subsets at varying proportions, we simulate datasets with known and controllable contamination ratios relative to pre-training—not fine-tuning.

---

A3: “Have the authors tested KDS on LLMs with partially known pre-training corpora?"

Excellent suggestion! We further evaluated on Pythia-6.9b, which is known to be pre-trained on the PILE dataset. The spearman correlation coefficient is 0.999 on the Enron subset.

Furthermore, our evaluation includes WikiMIA, BookMIA, and ArxivTection benchmarks, which label samples that are likely to have been seen in general LLM pre-training as "seen". We explicitly simulate varying contamination ratios and show in Table 1 that KDS exhibits near-perfect monotonicity and consistency, outperforming all baselines across three datasets.

---

A4. Theory

While our paper primarily focuses on providing a practical and robust empirical method for quantifying pre-training contamination, we agree that theoretical analysis is an important direction. Our core intuition—that fine-tuning shifts the embeddings of unseen pre-training data more significantly than seen data—is empirically grounded, as shown in Figure 2, and supported by strong monotonicity and consistency across all benchmarks (Tables 1, 5, 6). Moreover, the mathematical formulation in Eq. (3–5) provides a principled and interpretable construction of KDS, which integrates:

• A soft gating function that emphasizes initially similar pairs, and
• A distance shift measure capturing how much pairwise embedding geometry is altered after fine-tuning.

That said, we acknowledge that a full theoretical analysis of why kernel divergence correlates with contamination remains nontrivial. Such analysis would require carefully defined assumptions about the structure of the embedding space, the dynamics of fine-tuning, and the underlying data distribution. We consider this a promising direction and plan to explore these theoretical foundations in future work.

Dear Reviewer J5f8,

We sincerely appreciate your positive feedback and the time you've dedicated to reviewing our manuscript. Your insights are invaluable to us. Please let us know if you have any further questions.