Knowledge Distillation Detection for Open-weights Models

Thank you for the thoughtful and encouraging review. We appreciate the recognition on the novelty, relevance, and writing quality of this work. We now address individual questions.

Q18. Senario where distillation can only use closed-source teacher model's output?

To clarify, the closed-source assumption applies to the detection phase and not to the distillation process itself. Our detection method does not distinguish between how a student model has been distilled. For example, one of the experiment uses the original knowledge-distillation (KD) to train the student model, which only uses a closed-source teacher model's output.

Q19. Motivation vs experiment setup.

Thanks for pointing this out. We agree that we should make the connection between the motivation and the task formulation more explicit. Below, we carefully discuss the implications, clarify the setup, and how we will update the manuscript.

1. Motivated: Given a model, predict whether it has been distilled FROM a teacher model. (A teacher model candidate is required)
2. Formulated: Given a model, predict WHICH of the teacher model has been used to distill the student model. (A set of teacher candidates is required & assuming the student model has been distilled from one of them)

In the introduction, we motivated with (1) but presented a task formulation of (2). We now recognize that this connection may not have been sufficiently emphasized.

We formulated the task as a multiple choice problem as it is straightforward and with results easy to understand. The main difference between (1) and (2) is that (1) further requires an absolute threshold for decision making. In contrast, the multiple choice setting performs a relative comparison across a set of candidate teacher models. This allows us to select the most likely teacher without requiring an explicit threshold. Our method can be modified from (2) to (1), see Q20.

Q20. Extending to the determining whether a given student model was distilled from a given teacher model or not.

Under our distillation detection framework, transferring from (2) to (1) would require: (A) a calibrated score function and (B) a suitable decision rule. As an example, we can define the score as the p-value from a statistical independence test between the teacher and student. The decision of (1) can be made by checking whether this p-value falls below a chosen significance level, e.g., 0.05.

Next, we implement the HSIC test [A] to obtain p-value as score for the text-to-image generation task with 3 teacher models and 10 student models. Each of the student model is paired with its true teacher (positive) and two random candidate teachers (negative). A pair is predicted as distillation if the p-value is below 0.05. From the 30 teacher-student pairs, accuracy and F1 score are computed from these binary predictions, and AUC is based on the scores across different thresholds for p-value. The results are provided in Table R6. These results show that generalizing to the formulation (1) is possible. We view this as a natural extension and a valuable direction for future work.

Table R6. Performance of distillation detection using the HSIC test on the text-to-image task.

We will update the abstract and introduction of the manuscript to clarify the connection between the motivation and the formulation.

[A] Gretton, Arthur, et al. "A kernel statistical test of independence." In Proc. NeurIPS 2007.

Q21. Does the proposed method apply to LLMs?

In theory, our framework an be adapted to LLMs with specific choices for synthesis and scoring required modification, as they are domain-specific (NLP). For synthesis, inspiration can be drawn from "Data-Free Quantization for LLMs" [B]. For the scoring function, we could consider utilizing text embedding similarity, which converts the discrete output back into a continuous domain.

[B] Liu, Zechun, et al. "LLM-QAT: Data-free quantization aware training for large language models." arXiv preprint arXiv:2305.17888 (2023).

We will add this discussion to the limitations section and clarify the scope of this paper. While we attempted to be clear by mentioning “image classification and text-to-image generation” in the abstract, we will further clarify in the writing. For instance, would including the term "Vision Models" in the title be helpful?

Q22. Summarize method and approach in an algorithm.

Thanks for this great suggestion. We only provided a visual overview in Fig. 1. We agree that presenting the method in an algorithm would be easier to follow. We will modify.

We thank the reviewer for the feedback. We address individual questions below. We believe there are some misunderstanding and look forward to clarifying them.

Q14. Novelty and contribution.

To the best of our knowledge, the introduced task of distillation detection has not been addressed in prior research. We present a framework that breaks down the distillation detection process into three main components: input construction, score computation, and decision-making rules. We demonstrate the framework’s effectiveness in image classification and text-to-image tasks, comparing it against several different baselines by adapting related areas. We believe that our work enhances the understanding of distillation detection beyond the current literature and has real-world implications, as discussed in Q16.

Q15. The paper investigates a scenario under very stringent conditions.

We understand the reviewer’s concern, but we believe the setting we study is realistic and practical. In many real-world cases, especially with commercial models, the training data is not available, and the teacher model is only accessible via an API. While the setting may seem strict, this is also what will make our method effective in real-world circumstances.

Please note that we have included results for a more relaxed setup, where training data is accessible (as shown in the Oracle case in Tables 1 and 2), and our method still performs well. This shows that the approach remains effective across different assumptions.

Q16. Real world application for distillation detection.

Besides the scientific question, we foresee that distillation detection will have the following real-world use cases:

1. Detection terms of service violation: In the terms of service for OpenAPI services (OpenAI, Gemini, Midjourney, etc.), it is common to have some form of clause that prohibits users from training/distilling the services' output. The ability to detect distillation will enable the detection of such a violation.
2. Tracing content origin: Recently, there has been an interest in determining whether a model has been trained on data with IP protection. This would ensure proper accounting and crediting of the data source. Distillation detection will be a useful tool to trace these data sources across models. This closes the loophole that one could claim that they never trained on certain data, where they "distilled" from a model that did.

We believe our work is an important first step towards these directions. We will clarify and provide a more in-depth discussion in the introduction.

Q17. Most likely teacher vs. whether distillation occur.

Thanks for pointing this out. We agree that we should make the connection between the motivation and the task formulation more explicit. Below, we carefully discuss the implications, clarify the setup, and how we will update the manuscript.

1. Motivated: Given a model, predict whether it has been distilled FROM a teacher model. (A teacher model candidate is required)
2. Formulated: Given a model, predict WHICH of the teacher model has been used to distill the student model. (A set of teacher candidates is required & assuming the student model has been distilled from one of them)

In the introduction, we motivated with (1) but presented a task formulation of (2). We now recognize that this connection may not have been sufficiently emphasized.

We formulated the task as a multiple choice problem as it is straightforward and with results easy to understand. The main difference between (1) and (2) is that (1) further requires an absolute threshold for decision making. In contrast, the multiple choice setting performs a relative comparison across a set of candidate teacher models. This allows us to select the most likely teacher without requiring an explicit threshold.

Under our distillation detection framework, transferring from (2) to (1) would require: (A) a calibrated score function and (B) a suitable decision rule. As an example, we can define the score as the p-value from a statistical independence test between the teacher and student. The decision of (1) can be made by checking whether this p-value falls below a chosen significance level, e.g., 0.05.

Next, we implement the HSIC test [A] to obtain p-value as score for the text-to-image generation task with 3 teacher models and 10 student models. Each of the student model is paired with its true teacher (positive) and two random candidate teachers (negative). A pair is predicted as distillation if the p-value is below 0.05. From the 30 teacher-student pairs, accuracy and F1 score are computed from these binary predictions, and AUC is based on the scores across different thresholds for p-value. The results are provided in Table R5. These results show that generalizing to the formulation (1) is possible. We view this as a natural extension and a valuable direction for future work.

Table R5. Performance of distillation detection using the HSIC test on the text-to-image task.

We will update the abstract and introduction of the manuscript to clarify the connection between the motivation and the formulation.

[A] Gretton, Arthur, et al. "A kernel statistical test of independence." In Proc. NeurIPS 2007.

We thank the reviewer for recognizing our timely problem, model‑agnostic approach, and strong performance on both classification and text‑to‑image tasks.

Q4. Generalization to text-to-text models

The high-level detection framework, of data-free synthesis followed by score computation, applies to text-to-text models. However, the specific choices for synthesis and scoring required modification, as they are domain-specific (NLP). For synthesis, inspiration can be drawn from "Data-Free Quantization for LLMs" [B]. For the scoring function, we could consider utilizing text embedding similarity, which converts the discrete output back into a continuous domain.

[B] Liu, Zechun, et al. "LLM-QAT: Data-free quantization aware training for large language models." arXiv preprint arXiv:2305.17888 (2023).

We will add this discussion to the limitations section and clarify the scope of this paper. While we attempted to be clear by mentioning “image classification and text-to-image generation” in the abstract, we will further clarify in the writing. For instance, would including the term "Vision Models" in the title be helpful?

Q5. Reliability when student inherits limited influence from teacher is not a big issue.

Thanks for this comment. We will incorporate this view and defer these analysis to the appendix.

Q6. Computational costs

Thank you for the suggestion. As suggested, we will move the computational cost from the appendix to the main paper.

Q7. Adapt the method to work purely from API access for both student/teacher models?

In some context, it is possible to only require API access for both student and teacher models.

• For the text-to-image setting, our current experiments already operate under API-only access to the student model. As we use the "empty string" approach to data generation, and the score function does not require model weights.

• For the classification case, the current synthesis step depends on backpropagation through the student, which is not feasible with only API access. One possible direction is to explore black-box gradient approximation methods [C], however, such exploration is beyond the scope of this work.

[C] Grathwohl, Will, et al. "Backpropagation through the void: Optimizing control variates for black-box gradient estimation." Proc. ICLR, 2018.

Q8. Could an adversary use this to "hide" their dependence on a specific teacher model?

This is a possible scenario. Currently, our experiment is based on existing distillation methods. An adversary could try to modify their distillation approach to evade detection, e.g., deliberating injecting noise during the process. This adversarial setting is an interesting future direction. We will add a discussion.

Q9. Discussion of unintentional copying of another model.

Thanks for this comprehensive point of view. Currently, this work focus on the intentional distillation to establish a concrete framework and setup. We agree that unintentional copying is relevant, yet out of the scope of this work. We will discuss unintentional copying as a limitations/future work.

Q10. Cases when models are trained on the same private dataset.

Thanks for bringing up this interesting experiment. Here we provide experiment results in Table R2 for image classification, where we compare the KL distance between (a) the teacher and the distilled student and (b) the teacher and an independently trained model.

These models are trained on CIFAR10 with the corresponding architecture. We can observe the KL distance of (a) is consistently smaller than (b). This means that our approach can differentiate distillation from simply training from the same dataset.

Table R2: Results from training on the same private dataset.

Q11. Would agreement on incorrect predictions serve as a stronger signal for distillation?

In our data synthesis process, with its data-free nature, we don't have the ground truth label. As such, it is unclear how to define a “mistake”.

To test the reviewer's hypothesis, we provide experiment results by assuming we have access to the real training data. Specifically, for each candidate teacher, we count how often the student and teacher agree on incorrect predictions, and we select the teacher with the highest count. The results are summarized in Table R3. We can indeed see an improvement by using agreement on mistakes as the score and decision rule.

Table R3. Accuracy when using "agreement on mistakes".

We will update the paper to include these findings, and we will also add a discussion of alternative heuristics, including agreement on mistakes, as potential extensions under more relaxed assumptions.

Q12. Can the method estimate a likelihood that the student and a given teacher were trained independently?

Yes, we provide additional results of using independence test on the task of text-to-image generation in Table R4. Here we implement the HSIC test [A] to obtain p-value as the score with 3 teacher models and 10 student models. Each of the student model is paired with its true teacher (positive) and two random candidate teachers (negative). A pair is predicted as distillation if the p-value is below 0.05. From the 30 teacher-student pairs, accuracy and F1 score are computed from these binary predictions, and AUC is based on the scores across different thresholds for p-value. We can see our framework can be adapt to using p-values as the score.

Table R4. Performance of distillation detection using the HSIC test on the text-to-image task.

[A] Gretton, Arthur, et al. "A kernel statistical test of independence." In Proc. NeurIPS 2007.

Q13. Comment on how the SVD method in 4.2 will scale for very multiclass scenarios

The SVD step in Sec. 4.2 has a complexity of $O(C^3)$ for $C$ classes. Note that it is computed only once per teacher-student pair as part of the set-level score. To make the computation efficient for large $C$, SVD approximations such as truncated or sparse SVD [D] can applied.

[D] Yang D, Ma Z, Buja A. A sparse singular value decomposition method for high-dimensional data. Journal of Computational and Graphical Statistics, 2014

We thank the reviewer for the recognition of our clear writing, novel problem formulation, and strong experimental results.

Q1. Formulated problem setting vs. the primary motivation.

Thank you for pointing this out. We agree that we should make the connection between the motivation and the task formulation more explicit. Below, we carefully discuss the implications, clarify the setup, and how we will update the manuscript.

1. Motivated: Given a model, predict whether it has been distilled FROM a teacher model. (A teacher model candidate is required)
2. Formulated: Given a model, predict WHICH of the teacher model has been used to distill the student model. (A set of teacher candidates is required & assuming the student model has been distilled from one of them)

In the introduction, we motivated with (1) but presented a task formulation of (2). We now recognize that this connection may not have been sufficiently emphasized.

We formulated the task as a multiple choice problem as it is straightforward and with results easy to understand. The main difference between (1) and (2) is that (1) further requires an absolute threshold for decision making. In contrast, the multiple choice setting performs a relative comparison across a set of candidate teacher models. This allows us to select the most likely teacher without requiring an explicit threshold. Our method can be modified from (2) to (1), see Q2.

Q2. Extending to the scenario of the motivated problem.

Under our distillation detection framework, transfering from (2) to (1) would require: (A) a calibrated score function and (B) a suitable decision rule. As an example, we can define the score as the p-value from a statistical independence test between the teacher and student. The decision of (1) can be made by checking whether this p-value falls below a chosen significance level, e.g., 0.05.

Next, we implement the HSIC test [A] to obtain p-value as score for the text-to-image generation task with 3 teacher models and 10 student models. Each of the student model is paired with its true teacher (positive) and two random candidate teachers (negative). A pair is predicted as distillation if the p-value is below 0.05. From the 30 teacher-student pairs, accuracy and F1 score are computed from these binary predictions, and AUC is based on the scores across different thresholds for p-value. The results are provided in Table R1. These results show that generalizing to the formulation (1) is possible. We view this as a natural extension and a valuable direction for future work.

Table R1. Performance of distillation detection using the HSIC test on the text-to-image task.

We will update the abstract and introduction of the manuscript to clarify the connection between the motivation and the formulation.

[A] Gretton, Arthur, et al. "A kernel statistical test of independence." In Proc. NeurIPS 2007.

Q3. Real world use cases for distillation detection.

Besides the scientific question, we foresee that distillation detection will have the following real-world use cases:

1. Detection terms of service violation: In the terms of service for OpenAPI services (OpenAI, Gemini, Midjourney, etc.), it is common to have some form of clause that prohibits users from training/distilling the services' output. The ability to detect distillation will enable the detection of such a violation.
2. Tracing content origin: Recently, there has been an interest in determining whether a model has been trained on data with IP protection. This would ensure proper accounting and crediting of the data source. Distillation detection will be a useful tool to trace these data sources across models. This closes the loophole that one could claim that they never trained on certain data, where they "distilled" from a model that did.

We believe our work is an important first step towards these directions. We will clarify and provide a more in-depth discussion in the introduction.