BaDExpert: Extracting Backdoor Functionality for Accurate Backdoor Input Detection

Q2: Additional comparisons with recent baselines.

As suggested by the reviewer, we have supplemented the results comparing BaDExpert with ABL[1] and CD[4] (together with additional baselines suggested by other reviewers) in Rebuttal-Table 1. Particulary, we note that:

• ABL[1] is a poison-suppression defense that requires control of the training process (pre-development), which is actually out of the scope considered by our work (post-development). We simply report the CA/ASR results of the anti-backdoor learned models.
• CD[4] is a technique originally for poisoned training set inspection (i.e., detecting poison samples within the training dataset). In order to compare CD with BaDExpert, we adapt it to an inference-time backdoor input detector.
• ASSET[5] is also a poisoned training set inspection method. We did not include it because it cannot be applied/adapted to our inference-time backdoor input detection setting.

As shown in Rebuttal-Table 1 below, BaDExpert outperforms both ABL and CD, achieving lower ASR while retaining higher CA. These results are also updated in our paper revision (Appendix C.8).

Rebuttal-Table 1: Comparing BaDExpert with additional baselines. (CA for Clean Accuracy, ASR for Attack Success Rate) For fair comparison: we follow the suggested hyperparameters and only tune them to approach the officially reported results; when reporting RNP results, we stop pruning once ASR drops to below $20$%; when reporting CD results, we follow their suggested and default threshold selection strategy. As shown, BaDExpert outperforms all these baselines w.r.t. both CA and ASR.

---

Q3: Clarifying what happens after the detection of backdoor samples.

We apologize for any unclarity in our experimental setting w.r.t. the reported CA/ASR results of BaDExpert. Indeed, our work only focuses on detecting backdoor inputs (and does not study how to obtain correct classification retuls for them). In Sec 4.1 "Metrics" paragraph, we have stated how we report the CA & ASR for BaDExpert (and other detector-based defenses), in order to fairly compare with other non-detector-based defenses: "we interpret correctly filtered backdoor inputs as successfully thwarted attacks, while falsely filtered clean inputs are considered erroneous predictions". Formally and specifically:

1. $\text{ASR}=\mathbb P_{(x,y)\sim \mathcal{P}} [\mathcal M(\mathcal T(x)) = t \wedge \mathtt{BID}(\mathcal T(x)) = 0]$: the ASR is reported by considering all correctly detected (rejected) backdoor inputs as failed attack attempts, and thus not inlcuded in the ASR numerator;
2. $\text{CA}=\mathbb P_{(x,y)\sim \mathcal{P}} [\mathcal M(x) = y \wedge \mathtt{BID}(x) = 0]$: the CA is reported by considering all falsely detected (rejected) benign inputs as misclassification, and thus not inlcuded in the CA numerator.

---

[1] Li, Yige, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma. "Anti-backdoor learning: Training clean models on poisoned data." Advances in Neural Information Processing Systems 34 (2021): 14900-14912.

[2] Chai, Shuwen, and Jinghui Chen. "One-shot neural backdoor erasing via adversarial weight masking." Advances in Neural Information Processing Systems 35 (2022): 22285-22299.

[3] Li, Yige, Xixiang Lyu, Xingjun Ma, Nodens Koren, Lingjuan Lyu, Bo Li, and Yu-Gang Jiang. "Reconstructive Neuron Pruning for Backdoor Defense." arXiv preprint arXiv:2305.14876 (2023).

[4] Huang, Hanxun, Xingjun Ma, Sarah Erfani, and James Bailey. "Distilling Cognitive Backdoor Patterns within an Image." arXiv preprint arXiv:2301.10908 (2023).

[5] Pan, Minzhou, Yi Zeng, Lingjuan Lyu, Xue Lin, and Ruoxi Jia. "ASSET: Robust Backdoor Data Detection Across a Multiplicity of Deep Learning Paradigms." arXiv preprint arXiv:2302.11408 (2023).

Thanks for the clarifications. They addressed concerns Q2/Q3. The reviewer would suggest the authors add these results to the paper and provide additional details/analysis regarding these new experiments.

Regarding Q1, the reviewer is not convinced by the clarification. In practice, the defender should not know what is the current CA. This is also mentioned by reviewer 59E7.

Q2: BaDExpert against weaker backdoor attacks (lower poison rate / smaller blending rate / smaller trigger size).

We thank the reviewer for questioning about whether weaker backdoor attacks would hinder our defense. As noticed by the reviewer, we have briefly discussed when the attacker tries to make their attack stealthier via lowering the poisoning rate (Table 10, or Rebuttal Table 1 below) and the blending rate (Table 12, or Rebuttal Table 2 below).

In order to showcase our defense's effectiveness against weaker backdoor attacks, we further supplement the following three tables (lower poison rate / smaller blending rate / smaller trigger size, respectively). The defense results are reported in the reviewer's suggested metric, CA and ASR (where FPR is also fixed to $1$%, in consistent with our setting across the entire paper).

• In Rebuttal-Table 1, we can see BaDExpert is consistently effective across a wide range of posion rate, from $0.1$% (50 poison samples) to $10.0$% (5000 poison samples).
• In Rebuttal-Table 2, BaDExpert also effectively suppresses the ASR no matter what blending rate (from $10$% to $20$%) the attacker employs.
• In Rebuttal-Table 3, we study two extreme scenarios where the attacker adopts small trigger sizes (together with low poison rates). As shown, BaDExpert still successfully defends these attack attempts.

These ablation studies are also updated in our paper revision (Table 10, 12, 13 and Fig 10).

Rebuttal-Table 1: Ablation results of different poison rates. (Blend attack on CIFAR10, FPR=$1$%) As shown, BaDExpert is insensitive to the variation of poison rate. Even when the poison rate is extremely low (poison rate = 0.1%, equivalent to only 50 poison samples) and ASR drops to ~80%, our BaDExpert still successfully suppresses the ASR to $11.8$%.

Rebuttal-Table 2: Ablation results of using weakened triggers at inference time. (Blend attack on CIFAR10, FPR=$1$%) As shown, when the adversary try to bypass our defense by using a lower blending rate at inference time (instead of the default 20% during poisoning), BaDExpert can still consistently suppress the ASR to below $20$%.

Rebuttal-Table 3: Ablation results of backdoor attacks with small trigger sizes. (CIFAR10, FPR=$1$%) We further study BaDExpert's effectiveness against two backdoor models that are poisoned with small triggers: 1) 4-Pixel: 4 random red pixels as the backdoor trigger ($0.1$% poison rate); 2) 1-Pixel: 1 random red pixel as the backdoor trigger ($0.5$% poison rate). Demonstrations of these triggers are available in our revised paper (Fig 10). To ensure the backdoors are "weak", we select the minimum poison rates to make sure the backdoors are just successfully injected (non-trivial ASR). As shown, BaDExpert can still effectively defend these attacks, suppresing the ASR to $<14.4$%.

Dear authors,

Thank you for your retailed rebuttal. I still have concerns about the hyperparameter settings of BadExpert. In addition, I notice that the hyperparameter of the baseline defense, e.g. NAD, are consistent cross different attacks. I worry about the fairness of comparison in Table 1.

Best,

Reviewer 59E7

Q3: Additional comparisons with recent baselines.

As suggested by the reviewer, we have supplemented results comparing BaDExpert with RNP[3] (together with additional baselines suggested by other reviewers). As shown in Rebuttal-Table 1, BaDExpert outperforms RNP, achieving higher CA and lower ASR. These results are also updated in our paper revision (Appendix C.8).

$^\star$Regarding the reviewer's quote "RNP reports an average ASR of 5.03 and CA of 92.18 on CIFAR10". We note that RNP reports these results with a default poisoning rate of $5$%, which is far larger than our default poisoning rate $0.3$%. Such discrepant settings may lead to the different reported results of RNP. In fact, the RNP authors also note in the Sec 5 that "RNP faces a noticeable challenge when defending against backdoor attacks with low poisoning rates (e.g., the poisoning rate ≤ 1%)." Moreover, their results in Table 12 (RNP against lower-poisoning-rate attackers) match our reported RNP results; and we have also validated that RNP can achieve the similar reported performance at the $5$% poisoning rate.

Rebuttal-Table 1: Comparing BaDExpert with additional baselines. (CA for Clean Accuracy, ASR for Attack Success Rate) For fair comparison: we follow the suggested hyperparameters and only tune them to approach the officially reported results; when reporting RNP results, we stop pruning once ASR drops to below $20$%; when reporting CD results, we follow their suggested and default threshold selection strategy. As shown, BaDExpert outperforms all these baselines w.r.t. both CA and ASR.

---

Q4: Method efficiency concern.

• The time complexity analysis in I-BAU[5] points out its time complexity to be $\tilde O(K \cdot \vartheta \cdot \tilde O(\theta))$, where $K$ is the epochs (rounds) of their unlearning algorithm, $\vartheta$ is the number of the iterations of their fixed-point solver, $O(\theta)$ is the time cost for training a DNN via backprop on the reserved clean set for one epoch. In practice, they choose $K=3$ and $\vartheta=5$, i.e. approximately $15\cdot \tilde O(\theta)$, costing $26$ GPU seconds in our environment (ResNet18 on CIFAR10).
• Similarly, we could establish the time complexity of BaDExpert as $\tilde O((K_1 + K_2) \cdot \tilde O(\theta))$, where $K_1$ is the number of unlearning epochs (to obtain $\mathcal B$), and $K_2$ is the fine-tuning epochs (to obtain $\mathcal M'$). In practice, we choose $K_1=1$ and $K_2=10$, i.e. approximately $11\cdot \tilde O(\theta)$, costing $23$ GPU seconds in our environment (ResNet18 on CIFAR10).

As shown, our method's efficiency outperforms I-BAU both in principle and in practice.

---

Q5: Code releasing.

Thanks for the suggestion regarding publicly releasing our code! We will definitely share our code on GitHub after the peer-review process for reproducibility and verification purpose.

---

[1] Li, Yige, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma. "Anti-backdoor learning: Training clean models on poisoned data." Advances in Neural Information Processing Systems 34 (2021): 14900-14912.

[2] Chai, Shuwen, and Jinghui Chen. "One-shot neural backdoor erasing via adversarial weight masking." Advances in Neural Information Processing Systems 35 (2022): 22285-22299.

[3] Li, Yige, Xixiang Lyu, Xingjun Ma, Nodens Koren, Lingjuan Lyu, Bo Li, and Yu-Gang Jiang. "Reconstructive Neuron Pruning for Backdoor Defense." arXiv preprint arXiv:2305.14876 (2023).

[4] Huang, Hanxun, Xingjun Ma, Sarah Erfani, and James Bailey. "Distilling Cognitive Backdoor Patterns within an Image." arXiv preprint arXiv:2301.10908 (2023).

[5] Zeng, Yi, Si Chen, Won Park, Z. Morley Mao, Ming Jin, and Ruoxi Jia. "Adversarial unlearning of backdoors via implicit hypergradient." arXiv preprint arXiv:2110.03735 (2021).

Dear Reviewer Xe68,

It is great to know that our previous rebuttal response helps solve some of your concerns! Regarding your additional concerns, please let us clarify as follow.

---

Additional Concern #1: More details of the additional comparison

We apologize that we did not include detailed configurations / settings of the comparison of our method with baseline defenses. These additional experiments focus on attacks on CIFAR10 (ResNet18). When implementing these additional baselines, we follow the suggested hyperparameters in either the corresponding papers or the official repositories; we also ensure they are working as effective as possible in our setting by tuning these hyperparameters according to the eventual defense effectiveness.

• Specifically, when comparing with RNP, the unlearning stage contains $20$ epochs (lr = $0.01$) with an early stopping threshold of CA=$20$%, the recovering stage contains $20$ epochs (lr = $0.2$).

Regarding the discrepancy in ASR: in Table 1, we report the average results (ASR & CA) over three repetitive experiments, corresponding to 3 independently curated backdoor models. And when reporting the supplementary results during rebuttal, we experiment on one backdoor model of the three, and thus the initial (No Defense) CA/ASR of Rebuttal-Table 1 may be slightly different (from the Table 1). Nevertheless, to ensure that we do not bias toward our method during comparing, we run these baselines multiple times and report their best defense results possible.

We will include these configuration / setting details in our paper's next revision. We also thank the reviewer's suggestion on comparing our method with additional baselines on additional datasets. In our current paper, we already provide extensive comparisons on both CIFAR10 (Table 1) and GTSRB (Table 6) with multiple baselines. Nevertheless, we will try to extend this additional comparison also on GTSRB, in our future paper revision.

---

Additional Concern #2: CA/model estimation

We argue that a local clean test set uniquely preserved to evaluate the model's utility (CA) is a necessary and practical assumption.

• This is a necessary assumption, since the model owner is going to deploy the model for his/her own downstream task, and he/she must make sure the model has reached such capability, i.e., high CA.
• We also believe this is a practical assumption, given that nowadays there exist standard test sets available for diverse CV tasks. Furthermore, such a clean test set can also be obtained by withholding a small fraction of the clean samples $D_c$ (we originally preserved for unlearning / finetuning).

Also, the assumption of CA/model estimation is commonly made by most previous works:

• Noticeably, in the reviewer's mentioned baseline RNP [1], the ability to estimate CA is also necessary to help select their unlearning and recovering LR. Our method does not make any more assumption / require any more knowledge than theirs.
• In NAD [2], the distillation and erasing learning rates also need to be selected according to the model's CA.
• In ABL [3], the isolation learning rate and the unlearning rate need to be probed via observing the model's CA.
• In NC [4], a clean test set must be known in order to estimate the reversed trigger's effectiveness. During the trigger unlearning stage, the model's CA must be available to help practitioners select a proper unlearning rate.

To sum up, we believe that knowing the CA / estimating the model utility is a natural and practical assumption to make, which has been adopted in most prior works. And our work requires only the same level of knowledge / assumptions regarding this.

---

[1] Li, Y., Lyu, X., Ma, X., Koren, N., Lyu, L., Li, B., and Jiang, Y.G., 2023. Reconstructive Neuron Pruning for Backdoor Defense. arXiv preprint arXiv:2305.14876.

[2] "Neural attention distillation: Erasing backdoor triggers from deep neural networks." International Conference on Learning Representations (2021).

[3] "Anti-backdoor learning: Training clean models on poisoned data." Advances in Neural Information Processing Systems (2021).

[4] "Neural cleanse: Identifying and mitigating backdoor attacks in neural networks." IEEE Symposium on Security and Privacy (2019).

Thanks,

Authors

Dear Reviewer Xe68,

We appreciate your feedback very much. Following your suggestions, in our paper's future revision, we will elaborate more on:

1. Experimental setup (dataset, model architecture, etc.) in every table;
2. Hyperparameters of all baseline defenses and our method in every different setting (which has been briefly described in Appendix A);
3. Highlighting our method's insensitivity to hyperparameter choices (which has been briefly discussed in Appendix C.4);
4. Assumptions of the defender we are considering (including the natural and practical assumption to evaluate model CA on a test set).

Again, we sincerely thank the reviewer for acknowledging our work. We are more than grateful for your active engagement during the rebuttal period, together with your valuable contribution brought to us to improve our work.

Thanks,

Authors