Towards Undistillable Models by Minimizing Conditional Mutual Information

Thank you for appreciating our work in that our proposed approach seem to be the only one that makes the network not distillable on all benchmarks. Also, thank you for your time reading our paper. Please find our responses to your concerns in the following.

Weakness 1: I believe that further justification, evidence, or analysis (theoretical or empirical) is required to relate the approximation of the second term in the objective to the original one (as $\omega$ was taken to be a finite number). There is some discrepancy that needs to be settled as eventually instead of maximizing over $\mathbf{\alpha}$ (which makes sense), averaging is done over multiple values. Also, can you please share what values of $\omega$ were used in the paper? I didn't find this information.

Ans: Thank you for raising this important point. As stated in Theorem 4.1, when $\omega \to \infty$, the second term in the objective function becomes equivalent to the RHS of Eq. (20). We kindly ask the reviewer to verify the derivation provided in the appendix, which outlines the steps leading to this approximation. For a sufficiently large value of $\omega$, we achieve the approximation stated in Eq. (21). Additionally, the integral in Eq. (21) is approximated by the summation provided in Eq. (22).

In our experiments, we set $\omega = 25$, which we found to be a sufficiently large value to effectively mimic the behavior of $\omega \to \infty$. It is worth noting that setting $\omega$ to very large numbers, such as 1000, can induce numerical issues (e.g., overflow) in Python programming, making it impractical for computation.

To address the impact of $\omega$ comprehensively, we have included an ablation study in the revised version of the paper's appendix (Appendix L). This study examines the effect of different values of $\omega$ on the results, providing further evidence and justification for our choice. We hope this additional analysis clarifies our approach and resolves any concerns.

Weakness 2 (Experimental section, part 1): I find the improvements over competing methods, and in particular label smoothing, marginal in most cases. I acknowledge that label smoothing does not adhere to the requirement of a network being undistillable, but it gets quite close to it. I am not convinced whether the minor improvements over it really make a difference in practice. Perhaps additional analysis or experiments in other scenarios can demonstrate the practical significance of your method over label smoothing.

Ans: Thank you for your observation. We respectfully argue that the improvement over label smoothing (LS) is not marginal when viewed in the correct context. While it is true that for some KD methods, the accuracy of the knockoff student trained on DNNs protected by LS and CMIM might appear close, the critical distinction lies in the undistillability of the DNNs: no existing KD method can render a DNN trained by CMIM distillable, whereas DNNs trained using LS can be rendered distillable by certain KD methods. In other words, there exists at least one KD method that makes LS trained DNNs distillable.

Weakness 2 (Experimental section, part 2): This may be a criticism in general for defense methods in this domain and not specifically for this paper. It seems that the evaluation is done under the assumption that the student model has access to the input only ($\mathbf{x}$). How likely is that setup? In my opinion, a more realistic setup is distilling a model based on a new dataset altogether. I believe that a comparison in this setting will be much more informative.

Ans: Thank you for your insightful comment. We agree that exploring the distillation process when the student model is trained on a dataset different from the one used to train the teacher model would be an interesting and more realistic scenario. However, it is not immediately clear whether conventional KD methods would be as effective in such cases, as the knowledge transfer process might be hindered by the domain gap between the two datasets.

The investigation of how KD operates under these conditions could indeed open a new avenue of research and expand the current understanding of defense methods in this domain. We appreciate your suggestion and recognize it as a potential direction for future work.

Weakness 3 (Writing, part 1): Some of the sentences are too long which makes it hard to understand at first pass (e.g., the first sentence of the abstract and the sentence in lines 46-50).

Ans: Thank you for pointing this out. We have revised and restructured the sentences mentioned, including the first sentence of the abstract and the one in lines 46–50, by breaking them into smaller, more concise sentences.

We thank the reviewer for their valuable feedback and for acknowledging the relevance of our work on undistillable models in the context of the increasing prevalence of large closed-source models. We also appreciate the recognition of the quality of our writing and the balance we have achieved between theoretical and empirical results. Please find our responses to your concerns in the following.

Weakness 1: L040: Missing reference to theoretical paper by Borup and Andersen (2021), “Even your Teacher Needs Guidance: Ground-Truth Targets Dampen Regularization Imposed by Self-Distillation,” NeurIPS.

Ans: Thank you for pointing this out. We have included the missing reference to the paper. We appreciate your attention to detail.

Weakness 2: "An insight is provided that in order for a DNN to be undistillable, it is desirable for the DNN to possess the trait that each cluster of the DNN’s output probability distributions corresponding to each label is highly concentrated to the extent that all probability distributions within the cluster more or less collapse into one probability distribution close to the one-hot probability vector of that label.” Although, I am unable to provide a reference, that a concentrated probability distribution is uninformative, and thus desirable to avoid distillation is, to the best of my knowledge, common knowledge in the field, and should not be considered a contribution of this paper.

Ans: Thank you for your comment. It appears there is a misunderstanding regarding the concept discussed in our paper. Our approach focuses on making the clusters of output probabilities corresponding to each class more concentrated. This is fundamentally different from the notion of a "concentrated probability distribution", which we have not employed or referenced in our work.

If you are aware of any method in the literature that explicitly addresses the concentration of output probability clusters, we would appreciate it if you could point it out. To the best of our knowledge, [1] is the first paper to introduce and formalize this concept, making it a key novelty of our work. We hope this clarification resolves any confusion regarding the contributions of our method.

Weakness 3: Replacing $Y$ with $\hat{Y}$ in the MI and CMI is simply replacing a variable in a function; the change itself is not inherently innovative and warrant a notion of "contribution". However, the implications and importance of doing so may hold some importance.

Ans: We would like to clarify that the innovation of this paper does not stem from simply "replacing $Y$ with $\hat{Y}$" in the MI and CMI formulations. Rather, our key contribution lies in the novel approach of measuring the concentration of output clusters for different values of the power transform $\alpha[Y]$. To achieve this, we introduce the concept of $\mathrm{I}(X; \hat{Y}^{\alpha[Y]} \mid Y)$, which quantifies this concentration in a meaningful way.

Furthermore, calculating $\mathrm{I}(X; \hat{Y}^{\alpha[Y]} \mid Y)$ is fundamentally different from calculating $\mathrm{I}(X; \hat{Y} \mid Y)$. Specifically, as shown in [1], the calculation of $\mathrm{I}(X; \hat{Y} \mid Y)$ relies on the Markov chain relationship $Y \to X \to \hat{Y}$. However, in our case, $Y$, $X$, and $\hat{Y}^{\alpha[Y]}$ no longer form a Markov chain, as $\alpha[Y]$ explicitly depends on $Y$. This shift introduces significant challenges, as minimizing CMI under a non-Markovian setting has not been explored in the literature before. Addressing this challenge is a novel and important contribution of our paper, which we believe should be acknowledged.

Weakness 4: Section 4.1 appears redundant, as the extension follows naturally by substituting variables (see also comment above).

Ans: Thank you for your comment. We respectfully disagree with the assertion that Section 4.1 is redundant. As detailed in our response to the earlier point, the extension presented in Section 4.1 is not a straightforward substitution of variables. Instead, it addresses the significant conceptual and computational challenges introduced by the dependency of $\alpha[Y]$ on $Y$, which breaks the Markov chain assumption ($Y \to X \to \hat{Y}$) typically used to calculate $\mathrm{I}(X; \hat{Y} \mid Y)$.

Section 4.1 provides a rigorous framework for calculating $\mathrm{I}(X; \hat{Y}^{\alpha[Y]} \mid Y)$ under these non-Markovian conditions. This methodological innovation is critical to the proposed approach and addresses an unexplored area in the literature, as no prior work has dealt with minimizing the CMI in such settings. We hope this explanation underscores the necessity and value of Section 4.1 in the paper.

We sincerely thank the reviewer for their valuable feedback. We also appreciate the positive comments regarding the intuitiveness of our idea, the thoroughness of our experiments, and the organization of the paper. Please find our responses to your comments in the following.

Weakness 1 (part a): The discussions of the proposed method’s limitations are missing.

Ans: We have revised the conclusion section of the paper to explicitly address some of the method's limitations. Thank you for bringing this into our attention.

Weakness 1 (part b): The proposed method collapses the logits so that each class’s output is highly concentrated (as shown in Fig.2), the teacher model might become overly confident in its predictions. This can lead to poor calibration and deteriorate generalization capability on out-of-distribution (OoD) data. Therefore, more settings and evaluations on the protected teacher model’s performance beyond prediction accuracy are necessary.

Ans: We would like to clarify the distinction between "highly concentrated output clusters" and "overly confident predictions". A highly concentrated output cluster does not necessarily imply that the model produces overly confident predictions. This is because the clusters can be concentrated around points that are not close to one-hot labels (the corners of the probability simplex). As a result, the model can have concentrated outputs without being overly confident. These are two separate concepts.

Our experiments support this distinction: the proposed method creates more concentrated output clusters, yet the accuracy of the model improves on the held-out test dataset compared to models trained with the conventional cross-entropy (CE) loss. This observation aligns with findings in [1], where it was demonstrated that training DNNs to produce highly concentrated output clusters can enhance their test accuracy.

Regarding out-of-distribution (OoD) data, we acknowledge the importance of such evaluations; however, the primary focus of this work is on training undistillable DNNs. While extending the evaluation to OoD scenarios is a valuable future direction, it is beyond the current scope of this study.

Weakness 2: The proposed method involves multiple hyperparameters, e.g., such as the number of power samples $N$ and the range $[0,\beta]$, but the ablation studies on hyperparameter sensitivity are missing. For example, have you assessed how these hyperparameters impact the model’s undistillability and accuracy? If some parameters are particularly influential, could you highlight those findings?

Ans: In response to your comment, we have conducted detailed ablation studies for the key hyperparameters, including the number of power samples $N$ and the range $[0, \beta]$, to assess their impact on the model’s undistillability and accuracy. The results of these studies have been included in the appendix (Appendix L) to provide a comprehensive understanding of the sensitivity and influence of these parameters. We hope this addition addresses your concern and strengthens the paper.

Weakness 3: The proposed method introduces computation overhead, but the comparisons of computational costs are missing. The method introduces extra computation for minimizing CMI and performing multiple transformations, what is the relative computational cost of training a CMIM-protected model compared to a standard model and other protection methods? As we can see, the experiments are primarily conducted on small datasets, with very limited testing on the larger ImageNet dataset.

Ans: We have added another section in the appendix (Appendix K) which discuss the computational overhead of CMIM. The experiment results is comprehensive and intensive. In summary, CMIM increases the training time by approximately 20%.

Additionally, we note that among the existing KD protection methods in the literature, only CMIM (our method) and ST are scalable to larger datasets like ImageNet. This scalability is due to the significant computational complexity of other benchmark methods, which limits their applicability to smaller datasets. We hope this clarification and the added discussion in the appendix address your concern.

Dear Reviewer UcAd,

Thank you again for reviewing our work and providing your valuable insights.

With the rebuttal deadline fast approaching, we kindly urge you to share your feedback on our responses to your comments at your earliest convenience. If you have any additional concerns or questions, please let us know. We are committed to addressing all your points comprehensively and are happy to provide further clarifications or details as needed. Otherwise, we kindly ask you to consider raising the score if our responses have resolved your concerns.

Thank you for your time and consideration.

Best regards, The Authors

Dear Reviewer’s UcAd,

Thank you for your detailed and constructive feedback on our submission, which has significantly improved the quality of our work. We have carefully addressed all the concerns raised in your review and provided detailed responses to each point.

If possible, we kindly request a re-evaluation of the score, considering that the issues highlighted have been thoroughly resolved. We truly appreciate your time and effort in reviewing our work and are grateful for your consideration.

Best regards.

We appreciate the reviewers recognizing the significance of our work in preventing the misuse of models and contributing to partial privacy protection. We also thank the reviewers for acknowledging the theoretical and empirical contributions of our work in demonstrating the effectiveness of our proposed method in enhancing the undistillability of models. Please find our responses to your comments in the sequel.

Weakness 1: The writing quality of the paper could be enhanced.

Ans: Thank you for pointing out the need for improving the writing quality. We have revised several lengthy and complex sentences throughout the paper to enhance clarity and make it more reader-friendly. Additionally, we will make further editorial refinements as needed following the paper’s acceptance.

Weakness 2 (part a): From a methodological perspective, the overall contribution of the paper is somewhat limited. The paper utilizes an existing metric, CMI, to measure the compactness of model outputs and aims to enhance model undistillability by maximizing this compactness metric. This approach appears too trivial and straightforward.

Ans: We appreciate the reviewer’s comments and would like to clarify the unique contributions of our work.

First, our paper is the only one to propose a method for training undistillable DNNs capable of resisting a wide range of knowledge distillation (KD) methods. We invite the reviewer to identify any existing approaches in the literature that achieve this specific goal. Given the novelty and practical importance of our work, we firmly believe that our contributions should not be considered trivial.

Second, while we leverage the concept of CMI from [1], the way it is calculated in our work significantly differs from how it is calculated [1]. In [1], the calculation of $I(X, \hat{Y} \mid Y)$ is based on the assumption of a Markov chain $Y \to X \to \hat{Y}$. However, in our work, we measure the compactness of clusters using $I(X, \hat{Y}^{\alpha[Y]} \mid Y)$, where the term $\hat{Y}^{\alpha[Y]}$ explicitly depends on $Y$, breaking the Markov chain assumption. Consequently, calculating $I(X, \hat{Y}^{\alpha[Y]} \mid Y)$ poses additional challenges. We address these challenges by proposing a novel method for calculating this metric, as detailed in Section 4.1.

Finally, we introduce a novel training method called the CMI minimization method. This approach trains a DNN by jointly minimizing the cross-entropy loss and the CMI values of all power-transformed clusters. This joint minimization process is non-trivial and requires careful formulation, as discussed in the paper.

These elements collectively establish the methodological contributions of our work and demonstrate its significance.

Weakness 2 (part b): It is not clear how this method fundamentally differs from directly employing a maximum entropy term or label smoothing technique to increase output concentration. Moreover, in the field of machine learning, particularly in computer vision, numerous loss functions have been studied to enhance model output compactness, such as the Large-Margin-Softmax-based methods.

Ans: We appreciate the reviewer’s comments and would like to clarify the distinction between our method and existing approaches.

While there are methods in the literature that aim to enhance the intra-class compactness of DNN output clusters, this alone is insufficient to achieve undistillability. Our work is the first to propose a method specifically designed to train undistillable DNNs that remain robust against KD techniques.

Regarding the methods mentioned by the reviewer:

1. Label Smoothing and Large-Margin-Softmax-based methods: These approaches may lead to more concentrated clusters. However, this is inadequate for ensuring compactness under power transformations, a key aspect required to achieve undistillability. As a result, models trained with these methods may still be vulnerable to KD techniques.

2. Maximum Entropy Term: Increasing the entropy of outputs merely reduces the model's confidence, making its predictions less certain. This does not directly relate to or contribute to the concentration of output clusters, which is the cornerstone of our approach for achieving undistillability.

Our method fundamentally differs by addressing these limitations and incorporating a novel strategy to ensure compactness under power transformations, which is critical for training undistillable DNNs. We hope this clarification highlights the unique contributions of our work.

Dear Reviewer x3fp,

Thank you for taking the time to review our work. We have conducted additional experiments to provide further clarification on the points you raised.

We would greatly appreciate your feedback on our rebuttal. If you have any further concerns or questions, please let us know. We are eager to ensure we have thoroughly addressed your comments and are happy to provide any additional clarifications. Otherwise, we kindly ask you to consider raising the score if our responses have resolved your concerns.

Thank you for your consideration.

Best regards, The Authors