SampDetox: Black-box Backdoor Defense via Perturbation-based Sample Detoxification

AW1: Thanks for your suggestion. We conducted new experiments to evaluate the performance of SampDetox considering the case where pre-training datasets for diffusion models are out-of-distribution from their target classification tasks. The detailed experimental setups are as follows:

• We conducted the training on a subset of the MS-Celeb-1M dataset. Specifically, we considered the top 100 labels with the largest number of samples and randomly selected 380 samples for each label. We split the training and tests with a ratio of 8:2 and adjusted the shape of the samples to 224*224.
• We adopted BadNets and WaNet as attack methods, representing a visible and an invisible backdoor attack, respectively.
• We considered four pre-trained diffusion models: i) a pre-trained model (i.e., Model 1) on one subset of MS-Celeb-1M, ii) a pre-trained model (i.e., Model 2) on another subset of MS-Celeb-1M, iii) a pre-trained model (i.e., Model 3) on ImageNet, and iv) Model 4, which is fine-tuned from Model 3 using a subset of MS-Celeb-1M.

The following table shows the experimental results. We can find that SampDetox with different diffusion models achieves satisfied defensive performance on ASR. Note that, for Model 3, the CA and PA of SampDetox decrease. This is because Model 3 is pre-trained on ImageNet rather than MS-Celeb-1M, resulting in the diffusion model failing to effectively denoise MS-Celeb-1M samples. However, we can find significant improvements in the CA and PA of Model 4 since Model 4 is fine-tuned using a subset of MS-Celeb-1M. In other words, SampDetox can defend against attacks using out-of-distribution diffusion models with proper fine-tuning.​

AW2: Sorry for your confusion. In Appendix D.3 (Limitations of SampDetox), we have discussed a type of backdoor attack that SampDetox cannot defend against. Specifically, when dealing with triggers derived from semantic features that are part of the original classified features and whose distribution is the same as the distribution of the training dataset of the pre-trained diffusion model, these triggers will be restored by the denoising phase of SampDetox. In this case, the triggers will not be destroyed by SampDetox, resulting in a failure of defense.

AW3: Thanks for your suggestion. The details are as follows.

• SampDetox (Ours). As mentioned in Appendix A.2, we utilized the improved diffusion model provided by [17], whose diffusion process consists of 1000 steps. We use the same settings $\overline{t}_1=20$ and $\overline{t}_2=120$ throughout the paper.
• Sancdifi followed the settings in Section “Numerical Experiments” of its paper [33]. Sancdifi computed RISE maps using 2000 random binary masks and set the saliency threshold to 0.95. For a fair comparison, Sancdifi used the same diffusion models as ours.
• BDMAE followed the settings in Section “Method Configurations” of its paper [15], which used masked autoencoders pre-trained on ImageNet with 24 encoder layers.
• ZIP followed the settings in Section “Purification Implementation” of its paper [14], which set its hyperparameter $\lambda$ to 2. For a fair comparison, ZIP used the same diffusion models as ours.

We will detail the above experimental settings in the final version.

AQ1: Sorry for your confusion. It should be clarified that we used the same DDPM configuration for ZIP and SampDetox in our experiments. SampDetox is much faster than ZIP because ZIP relies on the whole diffusion process of DDPM, while SampDetox only needs partial steps of the whole diffusion process. Specifically, ZIP utilizes DDPM to generate samples from completely random noise, requiring the whole diffusion process with 1000 denoising steps. However, since we set $\overline{t}_1=20$, SampDetox requires only 20 denoising steps in Stage 1. For Stage 2, the number of denoising steps varies from pixel to pixel, but it will not exceed 120 steps at most due to $\overline{t}_2=120$. In this way, though SampDetox employs DDPM twice, its overhead is still much less than that of ZIP.

AQ2: Thanks for your suggestion. Since DDIM was proposed to accelerate the speed of DDPM without affecting the image generation quality, the defense performance of SampDetox+DDIM is comparable to that of SampDetox+DDPM. To demonstrate the performance of SampDetox+DDIM, we conducted new experiments on CIFAR-10 against ten SOTA backdoor attacks. From the following table, we can find that the defense performance of SampDetox+DDIM is almost the same as that of SampDetox+DDPM, which means that SampDetox+DDIM can match the defense efficacy of SampDetox+DDPM.

AQ3: Sorry for the confusion. The results in Table 5 represent the average results across multiple attack types. We will add this information in the final version.

AQ4: Sorry for the confusion. The setting of $\overline{t}_1=20$ and $\overline{t}_2= 120$ remains consistent across varying datasets and models. In other words, we use the same fixed settings throughout the paper.

AL1: Thanks for your suggestion. Please refer to AW1 for our answer.

AW1: Thanks for your suggestion. First, it should be clarified that the defender completely controls the design, training, and utilization of diffusion models. Therefore, it is almost impossible for attackers to inject backdoors into diffusion models in practice. In addition, even if the training data for the diffusion model is poisoned, backdoors can not be injected into it because attackers need to modify the training process of the diffusion models to inject backdoors [1]. Moreover, even if the diffusion model is backdoored, specific trigger patterns must be added to the input Gaussian noise of the diffusion model to trigger the backdoor. However, when the input of SampDetox is Gaussian noise, due to the small number of denoising steps of SampDetox (i.e., $\overline{t}_1=20, \overline{t}_2=120$), it is extremely hard for a backdoored diffusion model to generate its target image.

In conclusion, backdoored diffusion models do not compromise the defense performance of SampDetox. Thanks again for your suggestion. We will add the discussion in the final version.

Reference: [1] Chou, Sheng-Yen, et al. How to Backdoor Diffusion Models? CVPR’23

We conducted new experiments to evaluate the performance of SampDetox considering the case where pre-training datasets for diffusion models are out-of-distribution from their target classification tasks. The detailed experimental setups are as follows:

• We conducted the training on a subset of the MS-Celeb-1M dataset. Specifically, we considered the top 100 labels with the largest number of samples and randomly selected 380 samples for each label. We split the training and tests with a ratio of 8:2 and adjusted the shape of the samples to 224*224.
• We adopted BadNets and WaNet as attack methods, representing a visible and an invisible backdoor attack, respectively.
• We considered four pre-trained diffusion models: i) a pre-trained model (i.e., Model 1) on one subset of MS-Celeb-1M, ii) a pre-trained model (i.e., Model 2) on another subset of MS-Celeb-1M, iii) a pre-trained model (i.e., Model 3) on ImageNet, and iv) Model 4, which is fine-tuned from Model 3 using a subset of MS-Celeb-1M.

The following table shows the experimental results. We can find that SampDetox with different diffusion models achieves satisfied defensive performance on ASR. Note that, for Model 3, the CA and PA of SampDetox decrease. This is because Model 3 is pre-trained on ImageNet rather than MS-Celeb-1M, resulting in the diffusion model failing to effectively denoise MS-Celeb-1M samples. However, we can find significant improvements in the CA and PA of Model 4 since Model 4 is fine-tuned using a subset of MS-Celeb-1M. In other words, SampDetox can defend against attacks using out-of-distribution diffusion models with proper fine-tuning.​ ​

AW2: Sorry for your confusion. In Appendix D.3 (Limitations of SampDetox), we have discussed a type of backdoor attack that SampDetox cannot defend against. Specifically, when dealing with triggers derived from semantic features that are part of the original classified features and whose distribution is the same as the distribution of the training dataset of the pre-trained diffusion model, these triggers will be restored by the denoising phase of SampDetox. In this case, the triggers will not be destroyed by SampDetox, resulting in a failure of defense.

AQ1: Sorry for the confusion. Based on the ablation study results shown in Figure 5, we suggest to set $\overline{t}_1$ and $\overline{t}_2$ to 20 and 120, respectively. Please see lines 383-384 for more details. Please note that we use the same settings, i.e., $\overline{t}_1=20$ and $\overline{t}_2 = 120$ throughout the paper. In other words, the experimental results in Tables 1, 3, 4, 5, and 6 are obtained utilizing the same $\overline{t}_1$ and $\overline{t}_2$.

AL1: Thanks for your suggestion. Please refer to AW1 for our answer.

AW1: Indeed. In Appendix D.3 (Limitations of SampDetox), we have discussed a type of backdoor attack that SampDetox cannot defend against. Specifically, when dealing with triggers derived from semantic features that are part of the original classified features and whose distribution is the same as the distribution of the training dataset of the pre-trained diffusion model, these triggers will be restored by the denoising phase of SampDetox. In this case, the triggers will not be destroyed by SampDetox, resulting in a failure of defense.

AW2: Thanks for your suggestion. We will detail the use of diffusion models in the final version.

In Appendix A.2, we have mentioned that the diffusion models used the same settings as the ones presented in [17]. In this experiment, the number of total diffusion steps was set to 1000 for all the datasets in the experiments, the noise_schedule was set as ``cosine’’, and the learning rate was set to 1e-4. We will provide such details in the final version.

In Appendix C.4, we have evaluated the defense performance of SampDetox using different diffusion models. Specifically, for CIFAR-10 classification tasks, we considered three different pre-trained diffusion models: i) a pre-trained model on CIFAR-10, ii) a pre-trained model on ImageNet, and iii) a model pre-trained on ImageNet and then fine-tuned on CIFAR-10. Table 8 in our paper demonstrates the generalization ability of SampDetox to such pre-trained diffusion models.

Besides, we conducted new experiments to evaluate the performance of SampDetox considering the case where pre-training datasets for diffusion models are out-of-distribution from their target classification tasks. The detailed experimental setups are as follows:

• We conducted the training on a subset of the MS-Celeb-1M dataset. Specifically, we considered the top 100 labels with the largest number of samples and randomly selected 380 samples for each label. We split the training and tests with a ratio of 8:2 and adjusted the shape of the samples to 224*224.
• We adopted BadNets and WaNet as attack methods, representing a visible and an invisible backdoor attack, respectively.
• We considered four pre-trained diffusion models: i) a pre-trained model (i.e., Model 1) on one subset of MS-Celeb-1M, ii) a pre-trained model (i.e., Model 2) on another subset of MS-Celeb-1M, iii) a pre-trained model (i.e., Model 3) on ImageNet, and iv) Model 4, which is fine-tuned from Model 3 using a subset of MS-Celeb-1M.

The following table shows the experimental results. We can find that SampDetox with different diffusion models achieves satisfied defensive performance on ASR. Note that, for Model 3, the CA and PA of SampDetox decrease. This is because Model 3 is pre-trained on ImageNet rather than MS-Celeb-1M, resulting in the diffusion model failing to effectively denoise MS-Celeb-1M samples. However, we can find significant improvements in the CA and PA of Model 4 since Model 4 is fine-tuned using a subset of MS-Celeb-1M. In other words, SampDetox can defend against attacks using out-of-distribution diffusion models with proper fine-tuning.​ ​

AQ1: Thanks for your suggestion. Please refer to AW2 for our answer.

AQ1: Thanks for your suggestion. Since the semantic features adopted by Reflect [1] and DSFT [2] are neither part of the original classified features nor whose distribution is the same as the distribution of the training dataset of the pre-trained diffusion model, the triggers based on such semantic features will not be restored by the denoising phase of SampDetox. Consequently, these triggers can be destroyed by SampDetox to achieve a defensive effect. To demonstrate the defense performance of SampDetox against Reflect [1] and DSFT [2], we conducted new experiments on datasets CIFAR-10, GTSRB, and Tiny-ImageNet, respectively. From the following table, we can find that SampDetox effectively defends against such backdoor attacks.

AQ2: Thanks for your suggestion. Since DDIM was proposed to accelerate the speed of DDPM without affecting the image generation quality, the defense performance of SampDetox+DDIM is comparable to that of SampDetox+DDPM. To demonstrate the performance of SampDetox+DDIM, we conducted new experiments on CIFAR-10 against ten SOTA backdoor attacks. From the following table, we can find that the defense performance of SampDetox+DDIM is almost the same as that of SampDetox+DDPM. Therefore, the utilization of DDIM to reduce the overhead will not deteriorate the detoxification performance.

AQ3: Thanks for your suggestion. As discussed in Appendix D.3 (Limitations of SampDetox), when dealing with triggers derived from semantic features that are part of the original classified features and whose distribution is the same as the distribution of the training dataset of the pre-trained diffusion model, these triggers will be restored by the denoising phase of SampDetox. In this case, the triggers will not be destroyed by SampDetox, resulting in a failure of defense. Since Composite Backdoor Attack (CBA) [3] concatenates two natural images as backdoor triggers, it is considered the backdoor attack type that SampDetox fails to defend.

We conducted new experiments to evaluate the performance of SampDetox against the attack CBA [3] on CIFAR-10. Without defense, CA, PA, and ASR were 87.52%, 10.16%, and 90.17%, respectively. But when using SampDetox, CA, PA, and ASR were 86.81%, 38.45%, and 37.28%, respectively. The experimental results show that SampDetox has a certain effect to defend against CBA but fails to minimize the ASR.

Thanks again for your suggestion. The results of new experiments in AQ1 and AQ3 strongly support the discussion of the limitations in Section D.3. We will add the experiments in the final version of the paper.

Reference:

[1] Liu, Yunfei, et al. Reflection backdoor: A natural backdoor attack on deep neural networks. ECCV'20.

[2] Cheng, Siyuan, et al. Deep feature space trojan attack of neural networks by controlled detoxification. AAAI'21.

[3] Lin, Junyu, et al. Composite backdoor attack for deep neural network by mixing existing benign features. CCS'20.

AW1: Indeed, the effectiveness of SampDetox depends on our observed correlation. However, this does not mean that our approach cannot prevent more sophisticated attacks. This is because our observation has good generalization ability, since our observation is based on ten SOTA backdoor attacks covering most mainstream backdoor attack types. In other words, it can reflect the commonality of most attacks. In addition, in Appendix D.2, we tried to design adaptive attacks with high robustness and low visibility that aim to break the observed correlation. However, the designed adaptive attacks cannot achieve high enough robustness while keeping low visibility, which can be successfully defended by SampDetox. To the best of our knowledge, it is still challenging to propose an effective attack with high robustness and low visibility, which is worthy of further study.

AW2: Sorry for the confusion. Although the value of $\overline{t}_2$ is fixed, the noise levels imposed on sample pixels are not fixed. Please note the hyperparameter $\overline{t}_2$ in Stage 2 represents the maximum added noise level of each pixel rather than its actual added noise level. Based on both the original sample and its denoised version obtained in Stage 1, SampDetox calculates different noise levels for different pixels in Stage 2 (Line 9 of Algorithm 1), which aims to adapt to various trigger patterns. From Tables 1, 3, 4, 5, and 6, we can find the superiority of SampDetox in defending against various attacks involving different datasets and models.

AW3: Indeed, our approach may remove benign high-frequency. However, this does not eclipse the effectiveness of our approach against backdoor attacks. To evaluate the defense performance of SampDetox on classification tasks that rely on benign high-frequency details, we conducted new experiments on the DTD (Describable Textures Dataset) dataset, where we adopted a visible backdoor attack, i.e., BadNets, and an invisible backdoor attack, i.e., WaNet, respectively. The following table compares SampDetox with six baselines. From this table, we can find that though the CA of SampDetox a little smaller than the CA of ``No Defense’’, SampDetox can achieve the best defense performance on CA, PA, and ASR compared with all baselines, which shows the applicability and superiority of SampDetox.

AQ1: Sorry for your confusion. We use the same values of $\overline{t}_1$ and $\overline{t}_2$ throughout the paper. In other words, utilizing fixed $\overline{t}_1=20$ and $\overline{t}_2=120$, SampDetox can defend against various types of attacks involving different datasets and models, as shown in Tables 1, 3, 4, 5, and 6. In general, given a new scenario, there is no need to change the value of $\overline{t}_1$ and $\overline{t}_2$ to achieve satisfactory defense performance.

AQ2: Sorry for your confusion. For two $n * n$ images A and B, we calculate their Pixel-SSIM with a block size of $t*t$ as follows: $\operatorname{Pixel-SSIM}(x, y)=\frac{\left(2 \mu_x \mu_y+c_1\right)\left(2 \sigma_{x y}+c_2\right)}{\left(\mu_x^2+\mu_y^2+c_1\right)\left(\sigma_x^2+\sigma_y^2+c_2\right)}$, where $x$ and $y$ are the same positional pixels of images A and B, respectively. $\mu_x$ and $\sigma_x$ are the mean and variance of the block centered on $x$, respectively. $\sigma_{xy}$ is the covariance between $block_x$ and $block_{y}$, and $c_1$ and $c_2$ are constants. According to the above formula, the time complexity for the pixel pair is $O(t^2)$. Thus the time complexity of whole Pixel-SSIM is $O(t^2 n^2)$. Considering that $t$ is constant in practice, the computational cost of Pixel-SSIM is determined by the number of pixels. In addition, we conducted new experiments to calculate the overhead of Pixel-SSIM. The experimental results show that the Pixel-SSIM overhead only accounts for less than 1% of the overall overhead of SampDetox. In other words, such overhead can be neglected in practice.

We conducted new experiments on the ImageNette dataset (a subset of 10 classes from ImageNet) containing larger images with a resolution of $256*256$. From the following table, we can find that SampDetox is still effective in defending against attacks on large images.

The practical upper limit of image size depends on the denoising capability of diffusion models rather than the computational constraints. When the size of images is too large, it is difficult to train an effective diffusion model to denoise such images. Nevertheless, the experimental results show that SampDetox is still effective in common image sizes (i.e., 256*256).