Q1. Novelty and comparison with backdoorIndicator [2]

R1: Since [2] was first accessible on May 31 on Arxiv, one week after the submission deadline of NeurIPS 2024, we did not have the opportunity to review and compare our work with it. We would like to mention that our submission is not expected to compare to work that appeared only a month or two before the deadline, according to the latest policy of NeurIPS 2024 (see NeurIPS-FAQ).

However, we appreciate the opportunity to highlight the distinctions between our approach and BackdoorIndicator [2] to further improve our submission.

• First, we would like to clarify the following differences between our work and [2]:

  • Threat model: [2] targets decentralized training (FL) setting, where multiple clients train models locally and contribute updates to a central server. Our work considers a centralized training setting where only a central server is used.
  • Task: [2] focuses on detecting malicious clients, whereas our method aims to train a secure model on a poisoned dataset without clients.
  • Motivation: [2] is built on the motivation that planting subsequent backdoors with the same target label enhances previously planted backdoors, therefore, providing a way to detect the poisoned clients, while our method is based on the motivation that planting a concurrent reversible backdoor can help to mitigate the malicious backdoor.
  • Methodology: [2] utilizes OOD samples for backdoor client detection while our method constructs a proactive defensive poison dataset, following well-designed principles.

  Based on above analysis, we think our work and [2] are significantly different from each other. If the reviewer can identify specific similarities in the conceptual ideas between our work and [2], we would be more than happy to engage in a deeper discussion regarding these points.

• Second, we would like to discuss the challenges in direct utilizing [2] in our setting:

  • BackdoorIndicator [2] is designed to detect malicious clients within a federated learning (FL) context. This makes it challenging to apply [2] directly to our centralized environment since the task of identifying backdoored clients does not naturally fit into this setting (only a central server).
  • For comparison between [2] and our method, we need to emulate an FL scenario by assigning each image to a separate client (ensuring existence of benign client), thereby creating 50,000 local models from the CIFAR-10 dataset to defend a single attack with PreAct-ResNet18. This would require an impractical amount of computational resources, estimated at over 30,000 hours (1,250 days) of training time and 30TB of storage space using a server with a single RTX 3090 GPU. That's why we cannot compare our method with [2].

Q2. Confusing results in Table 1.

R2: Thank you. Firstly, it is important to note that:

• All attack checkpoints were sourced directly from the official BackdoorBench website.
• All experimental results for baselines align with those reported by BackdoorBench (refer to the official leaderboard).

This ensures that the comparisons presented in our paper are both fair and reliable.

It should be noted that, as we use different checkpoints compared to [1], the results presented in our paper may differ from those in [1] due to the inherent randomness and potential instability of some baselines. To investigate the failure of ABL in Table 1 of main manuscript, we conducted a detailed analysis of its training process and found that ABL successfully detected 437 poisoned samples from 2500 samples. During the unlearning phase, a sudden increase in the loss for clean samples occurs, leading to a notable degradation in model performance, which ultimately causes ABL to fail (see Figure 3 of the Supplementary PDF for the loss curves).

Additionally, we anonymously requested the Trojan attack checkpoint for PreAct-ResNet18 with CIFAR-10 and a poisoning ratio of 5% from [1]. For this checkpoint, PDB can still mitigate the backdoor with ACC of 91.84% and ASR of 0.47%.

Q3. Missing comparison to FT-SAM [1]

R3: Thanks. To address the comparison with FT-SAM [1], we have adapted their method to our experimental setting. It's worth noting that in [1], the authors employ the Blended attack with a blending ratio of 0.1 (Blended-0.1), whereas we use a blending ratio of 0.2 (Blended-0.2). For consistency and completeness, we have now included experiments using both blending ratios, and the results are shown below:

Table 1: Results on PreActResNet-18

From Table 1 above, we can find FT-SAM can achieve a higher accuracy as it aims to fine-tune a backdoored model while PDB aims to train a model from scratch. Consistent with [1], Table 1 shows that FT-SAM can mitigate backdoor attacks for most cases, except for the Blended-0.2. We observe that FT-SAM struggles to defend against blended attacks with higher blending ratios, such as 0.2. Notably, PDB achieves a significantly lower ASR across all cases, with an average ASR below 0.5%.

Q4. Ablation study for trigger position.

R4: Thanks. We would like to refer you to the Common Response for more comprehensive ablation studies for PDB. From Common Response, we can see that placing a trigger at the center of an image significantly degrades accuracy, as the trigger masks the core patterns of the image.

Reference:

[1] Enhancing Fine-Tuning Based Backdoor Defense with Sharpness-Aware Minimization.

[2] BackdoorIndicator: Leveraging OOD Data for Proactive Backdoor Detection in Federated Learning.

Dear Reviewer wZZe,

We sincerely appreciate your valuable insights and suggestions on our work. We have made our best efforts to address the concerns and queries you raised during the rebuttal process. We would greatly appreciate confirmation on whether our response has effectively resolved your doubts. Your feedback will be instrumental in improving the quality of our work. As the end of the discussion period is approaching, we eagerly await your reply before the end.

Sincerely,

The Authors

Q1. Training cost for DBD, NAB and V&B

R1: Thanks. Here, we report both training complexities and the empirical runtime of these methods in Table 1. For simplicity, we first define the following notations:

• $C_{sl}$: Supervised training cost.
• $C_{ssl}$: Self-supervised training cost.
• $C_{semi}$: Semi-supervised training cost.
• $C_{fc}$: Training cost for fully connected (FC) layers.
• $N_{tr}$: Size of the training dataset.
• $N_{def}$: Size of the defensive poisoned dataset.
• $F$: Frequency of sampling defensive poisoned samples.

Table 1: Training complexity and empirical runtime on CIFAR-10 with PreAct-ResNet18

Analysis: From Table 1, we find that since we set $\frac{F \cdot N_{def}}{N_{tr}}$ as a small value, the training complexity of PDB is not much larger than the baseline (i.e., No Defense). In contrast, as $C_{ssl}$ is often several times $C_{sl}$, the complexities of DBD, NAB, V&B are often much higher than the baseline. It is reflected in the difference in empirical time, where PDB's runtime is about two times that of the baseline and is much lower than other methods.

Q2. Difference between PDB and NAB

R2: Thanks for this insightful comment. A comprehensive analysis can be found in Appendix C.5 of our submitted manuscript. Here, we would like to highlight that although both NAB and PDB share the idea of proactive backdoor, PDB has essential differences from NAB, as detailed below:

• PDB does not rely on poisoned sample detection, while NAB still relies on accurate poisoned sample detection, falling into the "detection-then-mitigation" pipeline.

• PDB does not depend on suspicious sample relabeling, while NAB relies on accurately relabelling of detected suspicious samples.

Q3: Sensitivity to the defensive poisoned dataset and why the proposed method can meet Principle 4.

R3: Thanks. We would like to refer you to the Common Response for more comprehensive experiments and analysis for PDB.

Q4: Difference between the malicious poisoned dataset and the defensive poisoned dataset

R4: Thanks. We highlight the differences as follows:

• Malicious poisoned dataset is provided by the attacker and may contain unknown malicious poisoned samples.
• Defensive poisoned dataset is crafted by the defender with known reversible defensive poisoned samples.

Q5: Generalization to invisible backdoor attack and low-poisoning ratio attack.

R5: Thanks. As discussed in our paper, the proposed method, PDB, does not rely on specific assumptions about the type of attack, making it effective for defending against both invisible backdoor attacks and attacks with low poisoning ratios. To demonstrate this effectiveness, we conducted experiments using low poisoning ratios (0.5% and 0.1%) for both Visible and Invisible attacks. The results are summarized in Table 2, from which we can find that PDB can consistently mitigate backdoor attacks.

Table 2: Results on PreAct-ResNet18 and CIFAR10

Q6: Degraded accuracy when the poisoning ratio is decreasing.

R6: Thanks. For such concern, we would like to first clarify that PDB can still achieve high accuracy when the poisoning ratio is decreasing, as shown in Table 2 above. Then, we would like to discuss the factors that influence the accuracy of PDB:

• Model capacity and data complexity:

  • Model capacity: Since PDB introduces additional task, i.e., injecting defensive backdoor, increasing the model capacity helps to increase the accuracy of PDB, as evidenced in Table 3.

    Table 3: Results on different models

    Model	ResNet-18	ResNet-18	ResNet-34	ResNet-34	ResNet-50	ResNet-50
    Metric	ACC	ASR	ACC	ASR	ACC	ASR
    No Defense	92.54	76.27	93.08	82.48	93.76	87.26
    PDB	91.81	0.29	92.63	0.28	93.67	0.18

  • Dataset complexity: By comparing defense results with different datasets (e.g., Table 1 and Table 6 in the main manuscript), we can find that by decreasing the dataset complexity, the accuracy of PDB increases significantly.

• Strength of defensive backdoor:

  • Strength of augmentation: From Fig. 2 of Supplementary pdf, we can find that there exists a tradeoff between ACC and ASR. Therefore, the accuracy of PDB can be boosted by reducing the strength of augmentation.
  • Sampling frequency: From Table 4 in common response, we can find that by increasing the sampling frequency of defensive poisoned samples, the accuracy of PDB can be boosted.
  • Trigger size: Table 1 in Common Response shows that a proper choice of trigger size can also help to increase the accuracy. Therefore, if a validation set is accessible, a proper trigger size can be chosen to increase accuracy.

In summary, due to the "home field advantage" of PDB, there are several ways to maintain a high accuracy even in the case of a low malicious poisoning ratio, such as increasing model capacity, simplifying the dataset, reducing the strength of augmentation to defensive poisoned samples, increasing the sampling frequency and choosing a proper defensive trigger size.

Dear Reviewer 8vxw,

We sincerely appreciate your valuable insights and suggestions on our work. We have made our best efforts to address the concerns and queries you raised during the rebuttal process. We would greatly appreciate confirmation on whether our response has effectively resolved your doubts. Your feedback will be instrumental in improving the quality of our work. As the end of the discussion period is approaching, we eagerly await your reply before the end.

Sincerely,

The Authors

Q1. Defend adaptive attacks that increase the poisoning ratio.

R1: Thanks for this suggestion. To provide a more comprehensive evaluation of the proposed method PDB against adaptive attacks, we conduct experiments with poisoning ratios from 10% to 30%, and malicious trigger size from 4x4 to 10x10 using PreAct-ResNet18 on CIFAR-10. The results are summarized below:

Table 1: Results for adaptive attacks

From Table 1, we can find that PDB can consistently mitigate backdoor against adaptive attacks with various malicious trigger size and poisoning ratio. Note that to keep the stealthness of malicious backdoor, its poisoning ratio and trigger size is expected to be constrained. However, the defensive backdoor can utilize a large trigger size and high sampling frequency to meet the Principle 4: Resistance against other backdoors, therefore, mitigating the malicious backdoor effectively.

Q2. Can PDB scale for clean label backdoor attacks?

R2: Thank you for your question. As discussed in our paper, the proposed method, PDB, does not rely on specific assumptions about the type of attack. The key of PDB is utilizing a proactive defensive backdoor to suppress the malicious backdoor, making it effective for defending against clean label attacks with large model and dataset.

Note that we have shown that PDB can defend against SIG for CIFAR-10 datset (Table 1 in main manuscript). To demonstrate PDB's effectiveness for clean label attack on large scale dataset and model, we conduct experiments on the SIG and LC [1], with ViT-B-16 and Tiny-ImageNet. Note that for Tiny ImageNet (200 classes), the poisoning ratio is at most 0.5%. The results are summarized below:

Table 2: Defending results against clean label attack, on ViT-B-16 and Tiny-ImageNet

From Table 2, we can find that clean label attacks fail to attack the model with poisoning ratio 0.1%. For poisoning ratio 0.5%, PDB can effectively defend against the clean label attack for large-scale dataset and model.

Q3: Comparing training and inference runtime between PBD and other baselines

R3: Thanks for your constructive suggestion.

• Training runtime: Firstly, we would like to clarify that the running time reported in Table 4 of the main manuscript indicates to the training time. Furthermore, to better understand the training complexity of our method, please refer to our response R1 to Reviewer 8vxw.
• Inference runtime: As shown in Algorithm 1 (see the bottom four rows), during inference, our PDB also requires only one forward pass like the standard inference. The additional costs involve adding the defensive trigger onto each input image (i.e., $x \oplus \Delta_1$), and the inverse mapping (i.e., $h^{-1}(\cdot)$). Compared the cost of forward pass, these additional costs are egligible. Thus, our inference cost is almost similar with other baselines.

Q4: Detailed explanation of TAC plot

R4: Thanks. We noticed a typo in the legend of the TAC plots (Figure 4) in our main manuscript, which may confuse the reviewer, and we will fix it in our revisions. Due to the page limit, we only show the visualization of TAC for the 1st and 4th blocks of PreAct-ResNet18 (4 blocks in total) in our main manuscript. To provide a more comprehensive explanation for TAC plots, we visualize the TAC for all blocks of PreAct-ResNet18 and show the plots in Fig 1 of Supplementary pdf. Now, we provide a more detailed explanation for the plots:

• Definition of TAC [2]: TAC is designed to measure the change of activation values for each neuron when comparing maliciously poisoned samples to their benign counterparts. Let $\phi$ be a feature extractor which maps an input image $x$ to the latent activations. For an input image $x$, we can construct the malicious poisoned sample $x\oplus \Delta_0$. In PBD, a defensive trigger is added to the malicious poisoned sample, crafting sample $x\oplus\Delta_0\oplus\Delta_1$, aiming to suppress the malicious backdoor. Therefore, for dataset $D$, we define

$\text{TAC w/o } \Delta_1 = \frac{\sum_{x\in D}(\phi(x\oplus\Delta_0)-\phi(x))}{|D|},$

$\text{TAC w/ } \Delta_1 = \frac{\sum_{x\in D}(\phi(x\oplus\Delta_0\oplus\Delta_1)-\phi(x))}{|D|}.$

• Analysis of the plots: From the TAC plots in Fig 1 of Supplementary pdf, we can find that by planting a defensive trigger to a malicious poisoned sample, the activation changes from a malicious backdoor are substantially suppressed, indicating that defensive backdoor can suppress the malicious backdoor, therefore, reducing the ASR.

Reference:

[1] Poison frogs! targeted clean-label poisoning attacks on neural networks. NeurIPS 2018

[2] Data-free backdoor removal based on channel lipschitzness. ECCV 2022

Dear Reviewer 8hm3,

We sincerely appreciate your valuable insights and suggestions on our work. We have made our best efforts to address the concerns and queries you raised during the rebuttal process. We would greatly appreciate confirmation on whether our response has effectively resolved your doubts. Your feedback will be instrumental in improving the quality of our work. As the end of the discussion period is approaching, we eagerly await your reply before the end.

Sincerely,

The Authors

Q1. Explanations for experimental results in Table 2

R1: Thanks. Firstly, it's important to note that

• All attack checkpoints in our experiments were sourced directly from the official BackdoorBench website
• All experimental results for baselines in our main manuscript align with those reported by BackdoorBench (refer to the official leaderboard)

The above operation guarantees that the comparisons presented in our paper are both fair and reliable.

Regarding the performance of the baselines in Table 2, we provide the following analysis:

• AC and Spectral: Both methods rely on the latent representation of images to detect poisoned samples. AC identifies poisoned samples through clustering in the latent space, considering smaller clusters as likely to contain poisoned data. Spectral Signature detects outliers in the latent space to identify such samples. However, with a poisoning ratio of 5% for Tiny ImageNet (200 classes, each class accounts for 0.5%), the poisoned samples become the majority within the target class, breaking the underlying assumptions of these methods. Therefore, reducing the poisoning ratio to 0.1% helps maintain AC's and Spectral's effectiveness, as shown in Table 1. Furthermore, Table 1 also shows that PDB exhibits robustness across various poisoning ratios and outperforms AC and Spectral even at lower poisoning rates.

  Table 1: Results on ViT-b-16 with Tiny ImageNet

  Poisoning ratio ↓	Attack →	BadNet	BadNet	WaNet	WaNet	BPP	BPP
  	Defense ↓	ACC	ASR	ACC	ASR	ACC	ASR
  0.10%	No Defense	76.67	54.03	61.70	51.33	62.89	69.14
  0.10%	AC	76.50	24.15	75.65	2.77	76.79	3.55
  0.10%	Spectral	75.55	25.47	75.48	2.47	76.09	3.14
  0.10%	PDB	75.53	0.02	73.09	0.38	73.58	0.08

• ABL: This method leverages the early learning phenomenon of poisoned samples. Upon closer inspection, however, we find that ABL struggles to identify poisoned samples when used with the ViT_b_16 model correctly. Different from the CNN model, Vision Transformers has some intriguing properties, e.g., weaker biases on backgrounds and textures [1] and robust to severe image noise and corruption [2], which may prevent them from learning the triggers in the early stages. Moreover, as ABL takes additional stages to train the model (with a lower learning rate) and unlearn the isolated samples, it achieves higher accuracy.

• NAB: It struggles to detect and relabel the poisoned sample correctly for large datasets and large models, resulting in a low ACC and high ASR in Table 2. A more detailed analysis could be found in Appendix C.5.

Q2. Defend adaptive attacks with larger trigger and the choice of defensive trigger size

R2: Thank you for the insightful question. We would highlight the following points:

• For adaptive attacks with larger trigger sizes, please refer to Q1 of Reviewer 8hm3. There, we conducted experiments with poisoning ratios ranging from 10% to 30% and malicious trigger sizes varying from 4x4 to 10x10. Our experiments demonstrate that PDB effectively defends against adaptive attacks with various trigger sizes and poisoning ratios. Note that to keep the stealthness of malicious backdoor, its poisoning ratio and trigger size is expected to be constrained. However, the defensive backdoor can utilize a large trigger size and high sampling frequency to meet the Principle 4: Resistance against other backdoors, therefore, mitigating the malicious backdoor effectively.

• For the choice of defensive trigger size, we emphasize that the proposed method PDB is not tied to any specific choice of defensive trigger (see Appendix C.2 for PDB with other triggers). We would like to refer you to the Common Response for a detailed analysis of the role of defensive trigger size, from which we can find that a larger trigger size is preferred to ensure the defense effectiveness and a size between 5x5 to 8x8 is recommended for square triggers.

Q3. More explanation for PDB and the role of augmentation

R3. Thanks. We would like to refer you to the Common Response for more comprehensive explanation and experiments for PDB as well as the role of augmentation. In Common Response, we show that the augmentation plays an important role for further enhancing PDB and reducing the ASR.

Reference:

[1] Delving Deep into the Generalization of Vision Transformers under Distribution Shifts, CVPR 2022

[2] Intriguing Properties of Vision Transformers, NeurIPS 2021