W3. The experimental design in the paper appears insufficient, raising questions about the broader applicability of the proposed attack. First, it relies on a mere two benchmark datasets. Second, the distilled dataset's size variation is limited to IPC (abbreviation of Image Per Class) scenarios of 10 and 50. Third, while cross-architecture generalization is pivotal in dataset distillation, the paper's evaluations seem to be confined to a 3-layer ConvNet, which is consistent with the architecture of the proxy model designated for distillation.

We perform a series of experiments corresponding to these concerns:

1. ImageNet: We perform our KIP-based backdoor attack on ImageNet. In our experiment, we randomly choose ten sub-classes to perform our experiment. We also resize each image in the ImageNet into 128x128. The experimental results show that our KIP-based backdoor attack is effective.

2. IPC:

3. Cross model ability: The experiment for cross model ability is presented as follows. We train the distilled dataset poinsoned by simple-trigger and relax-trigger on 3-layers MLP and 3-layers ConvNet. The experimental results show that both CTA and ASR go up as we increase the IPC (Image Per Class), which suggests that the cross model issue may be relieved as the IPC is large enough.

The result that DoorPing outperforms simple-trigger in Table 2 is consistent to our expectation. In that case ($m=0.3$), relax-trigger should be chosen and will outperform DoorPing. The details are described below.

As for Table 2, we aim to say that different mask transparencies will have an impact on CTA/ASR. In the experiments in Table 2, we consider transparencies 30% (m=0.3). The experimental results show that the ASR of simple-trigger cannot achieve 100% and is worse than DoorPing's algorithm. However, this consequence is consistent to our theoretical analysis. According to our analysis, we can infer that the generalization gap has only a slight drop as we use the whole image-covered trigger (simple-trigger) when $m=0.3$, which contributes to the poor ASR. So, under this circumstance ($m=0.3$), we need to optimize conflict loss, and projection loss to find a stronger trigger (relax-trigger). Table 2 shows that our relax-trigger is apparently stronger than DoorPing and simple-trigger. To sum up, Table 2 can show that

1. As our theoretical analysis has predicted, simple-trigger is not useful when the transparency is low enough ($m=0.3$).

2. In a more general circumstance ($m=0.3$), we need to consider another theory-induced trigger, relax-trigger. Furthermore, our experimental results show that the relax-trigger is effective

where $\tilde{D} = D_A \cup D_B$.
The conflict measures the information conflict between $D_A$ and $D_B$. For example, we consider a dog/cat picture classification problem. In the dataset $D_A$, we label the dog pictures with $0$ and label the cat pictures with $1$. However, in the dataset $D_B$, we label the dog pictures with $1$ and label the cat pictures with $0$. It is clear that the model trained on $\tilde{D}$ must perform terribly on the dataset either $D_A$ or $D_B$. In this case, the information between $D_A$ and $D_B$ have strong conflict and the conflict loss would be large.

The projection loss reflects the natural information loss when we compress a large dataset into a small dataset. Take writing an abstract for example. If we want to write a 100 word abstract to  describe a 10000 words article, the abstract may suffer some lack of semantics to some degree. Such a phenomena also happens for dataset distillation. When the information of a large dataset is complex enough, the information loss for dataset distillation will be significant; When the information of a large dataset is very simple, it is possible that there is only very limited information loss. We introduce the projection loss defined above to measure this phenomenon.

Question

Q1. Are there some potential defense methods during the dataset distillation process to mitigate backdoor attacks?

We think there are two scenarios for KIP-based backdoor attack:

1. The distillation process is totally controlled by the attacker. Under this scenario, the victim does not have any chance to perform defense during the dataset distillation.

2. The distillation process is controlled by the attacker, but the victim can update the dataset during the distillation process. Under this scenario, the victim may send his/her dataset class by class to the attacker during the distillation process. By this approach, we can turn the dataset distillation process into a time-varying version. A research result (On the Permanence of Backdoors in Evolving Models. arXiv, 2023, https://arxiv.org/abs/2206.04677) has shown that the backdoor can be removed in the time-varying model. We think that this could be a potential defense. Nevertheless, whether we can extend this concept to distillation-based backdoor attack is still an open problem.

Q2. Given the variety of dataset distillation methods available, could the choice of distillation method potentially impact the conclusions drawn about the efficacy of the proposed attack method?

The choice of distillation method can influence the efficacy of our attack method to some degree. However, in our paper, we can find that KIP faithfully obeys the optimization problem of performance-matching dataset distillations. So, KIP should be a good surrogate of all performance-matching distillation methods. Hence, our KIP-based backdoor attack should remain valid and demonstrate similar behaviors for all performance-matching dataset distillations.

For other types of dataset distillations, it depends on whether there exists a kernel which can well approximate the framework of other types of distillations. If we can find such a kernel, then KIP can be a surrogate. Nonetheless, how to construct such a kernel is still an open problem. To the best of our knowledge, there is no literature about this research direction that provides kernel approx for other types of distillation.

We perform an experiment for this problem. We first utilize our triggers to poison the datasets. Then, we distill these datasets with different distillation methods, FRePo (Dataset Distillation using Neural Feature Regression. NeurIPS, 2022) and DM (Dataset condensation with distribution matching. WACV, 2023). The experimental results show that our triggers can successfully transfer to the FrePo and DM.

W3. In my understanding, individuals would majorly employ distilled data for training new models in scenarios such as neural architecture search and continual learning. Expanding on the implications of backdoor attacks in these applications would provide greater clarity.

We agree that inclduing the results from neural architecture search and continual learning can further strengthen our contributions. We train our distilled dataset poinsoned by simple-trigger and relax-trigger in different scenarios, neural architecture search (NAS) and continual learning (CL). The experimental results are shown below.

The experiemnt result below shows that our triggers remain effective for NAS.

For CL, the experiment result belwo shows that both CTA and ASR are slightly higher than baseline.

Note that the details about our implementation of NAS and CL are described below.

NAS: The process defines a search space (random_search) that includes a range of possible model parameters such as the number of convolutional layers, the number of dense layers, and the size of the convolutional layers. The program randomly selects parameters from this space to generate multiple candidate model architectures. A CNN model is then built, comprising convolutional layers (Conv2D), batch normalization (BatchNormalization), activation functions (such as ReLU), pooling layers (MaxPooling2D), flattening layers (Flatten), fully connected layers (Dense), and optionally Dropout layers. Each model is compiled and trained using the Adam optimizer and categorical cross-entropy loss function, but in this case, the same dataset is used for evaluation (although typically, an independent test set should be used). The accuracy and loss functions of different models are compared, and ultimately the best model is selected and saved

CL: The dataset is divided into different category-specific subsets (as in CIFAR-10, which is divided into 10 categories), each containing images and their corresponding labels. This allows the model to gradually train on each subset. A CNN model is built, including multiple convolutional layers (Conv2D), batch normalization layers (BatchNormalization), ReLU activation functions, max pooling layers (MaxPooling2D), and fully connected layers (Dense). The final layer uses a softmax activation function, a typical configuration for label classification tasks. The model is compiled using an RMSprop optimizer and categorical cross-entropy loss function. Further training optimization can be applied, such as using Elastic Weight Consolidation (EWC) to minimize the impact on the originally trained model when learning new subsets.

Question

Q1. In Equation (9), why the second term is called the generalization gap?

Given a model $f_{\mathcal{S}^*}: \mathcal{X}\rightarrow\mathcal{Y}$, a dataset $D\sim\mathcal{D}^N$, the generalization gap is defined as

$$\mathbb{E} _{(x,y)\sim\mathcal{D}} \ell(f _{\mathcal{S}^*}, (x, y)) - \mathbb{E} _{(x, y) \sim - D} \ell(f _{\mathcal{S}^*}, (x, y)),$$

where $\ell(\cdot, \cdot)$ is some loss function, $\mathbb{E} _{(x, y)\sim\mathcal{D}} \ell(f _{\mathcal{S}^*}, (x, y))$ is called risk and $\mathbb{E} _{(x, y) \sim D} \ell(f _{\mathcal{S}^*}, (x, y))$ is called empirical risk. This mathematical formulation suggests the gap between the performances of $f _{\mathcal{S}^*}$ applied on $D$ and $\mathcal{D}$, respectively. The above equation (Equation (9) in the paper) also indicates whether the performance of $f _{\mathcal{S}^*}$ on $D$ can be generalized to $\mathcal{D}$. Hence, we call the second term in Equation (9) as generalization gap.