When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need

Dear Reviewer gtda,

Thanks for your valuable comments! We address the weaknesses below. Please kindly let us know if you have further concerns.

Q1: Why do fragile samples align with ID data, and how do we distinguish ID and OOD data.

We are sorry for the confusion. This assumption of "fragile synthetic samples align with ID data" is grounded in the widely accepted consensus within DFKD that BN loss is a necessary component. Due to the BN loss for training the generator in SOTA DFKD methods (like DFQ, CMI, and NAYER), the synthetic distribution will be constrained to have similar BN statistics with the teacher's training distribution. For more details on the BN loss, please find the answer to Reviewer-saQK@Q1 (sorry for the limited space).

As such, when distilling NTL teachers, joint effects of BN loss and adversarial exploration loss let the synthetic distribution become similar to the mixture distribution of real ID and OOD domains. Therefore, according to our findings of the adversarial robustness difference of real ID and OOD domain samples against NTL teachers (refer to Section 4.1 in the main paper), we make the assumption that fragile synthetic samples are ID-like samples, and robust synthetic samples are OOD-like.

Empirically, the DFKD performance improvement by the introduced CKD (Tab. 1-3 in the main paper), visualization of original synthetic samples and ID-/OOD-like synthetic samples (Figs. 30-31 in main paper and Figs. R6-R7 in https://anonymous.4open.science/r/icml408/figs.pdf), t-SNE feature visualizations (Figs. 28-29 in main paper) can verify the effectiveness of our assumption.

Q2: Our settings

We sincerely apologize for the confusion. According to the tasks NTL can handle, we consider DFKD for three types of NTL teachers: closed-set NTL, open-set NTL, and NTL-based backdoor. The difference between these tasks lies in the data used for ID and OOD domains in training NTL teachers. Specifically,

• closed-set NTL: In this setting, the ID and OOD domains have the same label space, but the two domains have a distribution shift. We choose the dataset-pairs of: SVHN→MNIST-M and CIFAR10→STL10;
• open-set NTL: In this setting, the label spaces for the ID domain and OOD domain are disjointed. We perform experiments on SVHN→CIFAR10 and CIFAR10→Digits, where Digits is the combination of 5 digits datasets: MNIST, USPS, SVHN, MNIST-M, and SYN-D;
• NTL-based backdoor: We additionally consider using NTL to conduct training controllable backdoor attacks. We use four triggers: BadNets(sq), BadNets(grid), Blended, Sig on CIFAR10 and see them as the OOD domain. The clean CIFAR10 is regarded as the ID domain.

Our aim is to address the malign OOD-Trap effect introduced by NTL teachers in DFKD, and thus, regardless of NTL teacher's tasks, our goal is to transfer only the ID domain knowledge from NTL teacher to student. In addition, all NTL teachers are considered as white-box models during DFKD.

Q3: Would creating conflicts between ID and OOD learning lead to a significant accuracy loss for the NTL model?

It's true that NTL teachers have a certain degree of accuracy loss on either ID domain and any unseen domains. Empirically, we compare the performance of SL and Close-/Open-set NTLs on seen and unseen domains, as well as corrupt seen domain (with additive gaussian noise):

Table C1: Accuracy loss of NTL. NetArchs: ResNet-34→ResNet-18

Q4: Why is this method effective in defending against backdoor attacks?

• NTL-based backdoor. In our experiments, we consider using NTL in conducting training controllable backdoor attacks, where clean data (e.g., CIFAR10) is regarded as ID domain and clean data with backdoor triggers (e.g., CIFAR10 with BadNets) is seen as the OOD domain. Through our empirical findings (Fig. 4 (b-c) and Tab. 3 in our main paper), NTL-based backdoored teachers can transfer backdoors to students. This is also because of the OOD-trap effects as we mainly analyzed in Section 3 of our main paper.
• Defending against NTL-based backdoor. Our ATEsc can effectively defend the backdoor transfer from teacher to student through the DFKD process. This is because in the NTL-based backdoor task, the OOD domain (e.g., CIFAR10+BadNets) and the ID domain (original CIFAR10) exhibit significant differences in their adversarial robustness against NTL teachers. As such, the proposed ATEsc can distinguish ID-like and OOD-like synthetic samples and let the student learn ID domain knowledge. Visualization results in Fig. 29 and Fig. 31 provide evidence for the effectiveness of distinguishing between ID and OOD synthetic data for NTL-based backdoor teachers.

Dear Reviewer saQK,

Thanks for your valuable reviews! We address your concerns as follows. Please let us know if anything remains unclear.

Q1: Why generator has to output OOD-like samples

This is because DFKDs commonly use BN loss [B1] as regularization for training generator $G$. Specifically,

• When training NTL, each batch contains a mixture of ID and OOD samples. This results in that BN layers in pre-trained NTL teachers record statistical information for the mixture distribution of ID and OOD domains (mean $\mu_l$ and var $\sigma_l^2$ for layer $l$).
• When training $G$, BN loss is represented as the divergence between synthetic statistics $\mathcal{N}(\mu_l(x_{syn}), \sigma_l^2(x_{syn}))$ and teacher's BN statistics $\mathcal{N}(\mu_l, \sigma_l^2)$. Minimizing BN loss lets the synthetic samples follow similar statistical information of teacher's training samples.

The joint effects of BN loss and adversarial exploration loss (Eq. 7 in the main paper) constraint the $G$ to:

• (i) follow the statistics of NTL teacher's training data (i.e., a mixture of ID and OOD domains), and
• (ii) maximize the discrepancy between student $S$ and NTL teacher.

Thus, if the $S$ only has ID domain knowledge, the $G$ will be optimized to synthesize OOD-like samples.

If w/o BN loss, the $G$ is unnecessary to generate OOD-like samples, and DFKD will not be influenced by OOD trap from NTL teachers. Unfortunately, their ID performance will be influenced by low quality of synthetic samples. Corresponding empirical evidence is shown in Tab. B1. More results and synthetic samples w/ and w/o BN are in https://anonymous.4open.science/r/icml408/figs.pdf (Tab. R1 and Fig. R1-R5).

Table B1: ID: CIFAR10, OOD: STL; ResNet-34→ResNet-18

[B1] Dreaming to distill: Data-free knowledge transfer via deepinversion, CVPR'20

Q2: Why assume a student only has ID knowledge first

Sorry for the confusion. We aim to demonstrate that in adversarial exploration-based DFKD, even if a student $S$ learns only ID knowledge in initial or some intermediate stage, the $S$ will inevitably let the $G$ to synthesize OOD-like samples and then be taught to learn misleading OOD knowledge.

Q3: Why there are vastly different adversarial robustness for close ID and OOD data

Because the training manner of NTL causes significantly different margins (i.e., the minimal distance from a sample point to decision boundaries [B2]) for samples from even two similar domains:

• For ID domain, the objective is to learn correct classification (e.g., 10-class classification for CIFAR10). This results in relatively complex decision boundaries between ID domain classes and small margins for ID data points.
• For OOD domain, the objective is to force all data to be predicted as a single class (Eq. 5), and to maximize the domain discrepancy on both logits and features (Eq. 4). Predicting all samples to one class is simple, and no boundaries will go through the OOD clusters. Besides, maximizing the discrepancy on logits/features between domains further pushes the OOD cluster away from the ID domain clusters, resulting in very large margins for OOD data.

As a larger margin corresponds to stronger robustness [B2], we assume NTL teachers exhibit strong/fragile robustness against adversarial attacks on OOD/ID data. Empirical studies verify our assumption (Fig. 19-24 in main paper). We provide more evidence in https://anonymous.4open.science/r/icml408/figs.pdf (Fig. R8-R11) to analyze the effects of NTL's objectives for robustness.

[B2] Exploring and Exploiting Decision Boundary Dynamics for Adversarial Robustness, ICLR'23

Q4: The forget term

If the student $S$ and NTL teacher have similar predictions on OOD-like samples, it suggests that the $S$ has learned misleading OOD knowledge. By maximizing $\mathcal{L}_{\text{forget}}$, the $S$'s outputs on OOD-like samples are encouraged to diverge from the teacher's outputs, thus mitigating the misleading OOD knowledge transfer.

This term is effective in suppressing OOD knowledge transfer, as evidenced by the decreased OLAcc (Tab. 2) and the reduced ASR (Tab. 3). But it also degrades ID knowledge transfer. Such trade-off depends on whether the priority is on preserving ID transfer or preventing OOD transfer.

Q5: Regular KD results

Thanks! We present results of regular KD by ID data (KD-ID), OOD data (KD-OOD) and ID+OOD data (KD-All) in Tab. B2. More results in https://anonymous.4open.science/r/icml408/figs.pdf (Tab. R2)

Table B2: ID: CIFAR10, OOD: STL; ResNet-34→ResNet-18

Dear Reviewer iM7M,

Thanks for your positive opinions. We address your concerns as follows. If anything remains unclear, please do not hesitate to contact us.

Q1: Discuss on distribution shift in DFKD

Thanks for this valuable suggestion! [1-4] focus on different types of distribution shifts when distilling an SL teacher trained on the ID domain using DFKD.

• [1-3] focus on the non-stationary distribution problem, where shifts between synthetic distribution in different training stages (i.e., epochs) cause the catastrophic forgetting of the student models. This issue can be mitigated by memory bank-based strategies, where old synthetic is stored and will be replayed in future epochs [1-2] or by using an additional VAE to model the previously observed synthetic samples [3].
• [4] address the distribution shift between synthetic data and real training data. They propose a novel perspective with causal inference to disentangle the student models from the impact of such shifts.

Our work explores the DFKD under NTL teachers. We find that NTL teachers (pretrained on an ID domain and an OOD domain) result in OOD trap effect for DFKD. One key reason is the ID-to-OOD synthetic distribution shift, which is caused by the adversarial exploration of DFKD and the NTL's conflict learning target for ID and OOD domains. Such a shift will not occur when distilling an SL teacher trained on the ID domain using DFKD.

Q2: Experiments on more backbones and larger datasets.

The results with different network arch series for teachers and students are shown in Tab. A1-A4. In addition, the results on larger datasets are shown in Tab. A5. From the presented results, the proposed ATEsc is consistently effective in mitigating the OOD trap effect caused by NTL teachers.

Table A1: Datasets: ID: CIFAR10 OOD: STL; NetArchs: ResNet-34→VGG-11

Table A2: Datasets: ID: CIFAR10 OOD: STL; NetArchs: VGG-13→ResNet-18

Table A3: Datasets: ID: CIFAR10 OOD: Digits; NetArchs: ResNet-34→VGG-11

Table A4: Datasets: ID: CIFAR10 OOD: Digits; NetArchs: VGG-13→ResNet-18

Table A5: Datasets: ID: CIFAR100 OOD: STL; NetArchs: ResNet-34→ResNet-18

Q3: Stability of original CMI/NAYER and DFKD+ATEsc

The performance of DKFD is sensitive to network initialization, even when distilling SL teachers. The sensitivity can be further increased when distilling NTL teachers. Empirically, our toy experiment in Appendix A offers initial evidence supporting this point (please compare Fig. 9-11).

For the BN layers, we mix the ID data and OOD data in a batch for training NTL teachers, resulting in the well-trained BN layers recording the mixture of statistical information of ID and OOD domains. All SOTA DFKD methods in our experiments rely on BN loss to train the generator $G$. Under the joint effect of BN loss and adversarial exploration loss, the synthetic distribution will face an ID-to-OOD synthetic distribution shift when distilling NTL teachers. This further leads to ID-OOD learning task conflicts for students, hindering the student's performance on ID domain. We argue that distinguishing between ID-like and OOD-like synthetic data during the student's training can directly solve the ID-OOD conflict issue for students. However, it may not help solve the problem of mismatched statistical information, as the student's training does not directly rely on the BN information from teachers.

Q4: The cost of adversarial perturbation

We use one-step untargeted PGD attack in our method, with the perturbation bound $\epsilon=24$. Using NAYER as an example, in each epoch, we only synthesize 800 new samples after $G$'s training. We perform PGD on these samples and save them into ID-like or OOD-like pools. Thus, the additional time is only the time for performing PGD on 800*Epoch samples. As shown in Tab. A5, training NAYER 200 epochs on one RTX 4090 has only an additional 2 mins.

Table A5: Datasets: ID: CIFAR10 OOD: STL; NetArchs: ResNet-34→ResNet-18