Forget-Me-Not: Making Backdoor Hard to be Forgotten in Fine-tuning

1. The proposed attack method is specific to superFT

It is important to note that our training strategy is designed to counteract all finetuning-based defenses, despite being inspired by the idea of cyclical learning rate scheme used in superFT. As demonstrated in our paper, FMN effectively improves the attack robustness against a wide range of finetuning-based defenses, including vanilla finetuning, superFT, FT-SAM, and NAD.

2. It is not clear whether other defense methods are equally effective, such as ANP.

SuperFT is a quite strong defense, and in many cases it is more effective than other defenses such as ANP. One case study is with LIRA attack. As reported in Table 2 in our main paper, superFT can effectively mitigates the original LIRA attack. In constrast, as reported in Table 7 in our Appendix, ANP fails to mitigate this attack; it cannot reduce the ASRs without significant drops in BA.

3. How to choose parameters for this cyclical learning rate scheme? Experiments with different model architectures

We use the same hyper-parameters used in superFT for our cyclical learning rate scheme. We found this configuration is effective, and there is no need to tune these hyperparameters. We have run additional experiments utilizing various model architectures while maintaining the hyperparameters consistent with those employed for ResNet in our original study. We report the BA/ASR in the tables below. As shown, our method still achieves high ASR and remains effective against advanced fine-tuning defenses, such as super-FT and FT-SAM, across various architectures. We have included the results in the revised Appendix.

4. Evaluation with pruning-based method

We indeed evaluated one instance of our attack, LIRA+FMN, against pruning-based defenses, including Fine-pruning and ANP, and reported the results in Section 4.6 of the main paper and Section A.6 of the Appendix. LIRA+FMN bypassed both defenses. Regarding the reviewer's recommendation, we have additionally conducted experiments with RNP and reported the results in the table below.

It is important to note that our training strategy is specifically designed to enable existing attacks, such as LIRA, to counteract finetuning-based defenses. The robustness against other types of defenses will depend on the choice of the specific attack employed. As shown in the case study with LIRA+FMN, our method does not diminish the attack's effectiveness against other defense approaches.

1. Analysis about why the cyclical learning rate and the clean-backdoor interleaved training work

Our experiments in ablation studies confirm the effectiveness of the combination of two training strategies: the cyclical learning rate and clean-backdoor interleaved training. Using only one strategy will lead to a low attack success rate. The cyclical learning rate helps to find flat areas, and the clean-backdoor interleaved training guarantees that it is the shared flat area of both function losses.

We compare the flatness of losses of poisoned data and clean data sets between different methods to illustrate our arguments. Figure 6 in Appendix shows that the landspace of our loss function at the parameter solution is flatter than that of other methods.

We also provide empirical intuition of why FMN is effective against finetuning-based defenses. Figure 7 in the revised Appendix shows the ASR and BA along the connectivity path of the poisoned model and its corresponding fine-tuned version after undergoing the fine-tuning process of Super-FT defense. As can be observed, with conventional backdoor learning (left figure), when we linearly interpolate from the backdoored model to its corresponding Super-FT's fine-tuned version, the intermediate model's poisoning loss (i.e., the loss recorded only on the poisoned samples) increases, resulting in the decrease in ASR, while their clean losses and BAs are approximately stable. On the other hand, with FMN training (right figure), linearly interpolating between the backdoor and its corresponding Super-FT's fine-tuned model, the poisoning loss and ASR, as well as the clean loss and BA, are almost constant, indicating that FMN learns the backdoor in a region that makes it difficult for a fine-tuning defense to escape from.

2. Possible defense strategies against FMN

Our research highlights the potential risks of relying on third-party pre-trained models and underscores the importance of fostering trust between users and model providers. To defend against our attack, users should only use pre-trained models from trusted providers or actively participate in the training process. We also urge the research community to delve further into this area to develop more robust safeguards.

We have included this discussion in the conclusion of the revised manuscript.

1. More detailed description of the methodology

We thank the reviewer for the suggestion. We have added a detailed Algorithm block for FMN in the revised Appendix.

2. Rationale for choosing these seven attack methods

The chosen methods are representatives for different backdoor attack approaches:

• BadNets and Blended: standard attacks with random trigger patterns. BadNets uses an image patch, while Blended uses a blended image as the trigger.
• $L_p$ regularization and trojaning: optimized trigger patterns
• Input-aware: image-dependent trigger
• LIRA: imperceptible optimized trigger
• Narcissus: optimized trigger, clean-label attack

We have revised the manuscript to include the rationales.

3. Results of other defense experiments, such as data cleansing methods, model modification methods, and model validation methods

It is important to note that our training strategy is specifically designed to enable existing backdoor attacks to bypass finetuning-based defenses. Consequently, the robustness against other types of defenses will depend on the choice of attack used with our training process.

As shown in the case study with LIRA+FMN in Section 4.6 of the main paper and Section A.6 of the Appendix, our method does not diminish the attack's effectiveness against other defense approaches. LIRA+FMN is robust against various defense mechanisms, including Neural Cleanse, Fine-pruning, ANP, and STRIP. Note that we test only representative backdoor defenses that LIRA has already bypassed. Also, we do not include data cleansing defense from our evaluation because our threat model assumes that the adversary provides the backdoored model to the user for deployment rather than supplying malicious data.

4. Discussion of possible defense strategies

We thank the reviewer for the suggestion. Our research highlights the potential risks of relying on third-party pre-trained models and underscores the importance of fostering trust between users and model providers. To defend against our attack, users should only use pre-trained models from trusted providers or actively participate in the training process. We also urge the research community to delve further into this area to develop more robust safeguards.

We have included this discussion in the conclusion of the revised manuscript.

1. Regarding concerns about finetuning-based defense without knowing the trigger

We acknowledge that the vanilla fine-tuning (FT) has limited capability in mitigating backdoors. However, it still offers some level of protection and is often combined with other components to establish more robust defenses. FT-based defense remains an active area of research, with methods like NAD, FT-SAM, and Super-FT demonstrating empirical effectiveness. Particularly, Super-FT was released at the end of 2022, while FT-SAM has been published at ICCV 2023. In its paper, FT-SAM showed stronger defense performance than popular defenses in other approaches, including Neural Cleanse, ANP, ABL, and iBAU. In our paper, both Super-FT and FT-SAM can effectively mitigate LIRA, while FP and ANP fail (as reported in Section A.6 in the Appendix). Finally, as shown in our response to Reviewer EHtc, both Super-FT and FT-SAM can effectively mitigate very recent advanced attacks proposed in [1]. Hence, advanced finetuning-based backdoor defenses are serious mechanisms to protect users from the backdoor threat. Our method aims to enhance the robustness of backdoor attacks against these advanced FT defenses.

Also, we understand the reviewer's concern, and we have refined the discussion of fine-tuning-based defense in the revised introduction to provide a more accurate overview of this defense approach.

[1] Revisiting the Assumption of Latent Separability for Backdoor Defenses. In ICLR 2022

2. Regarding finetuning-based defense after knowing the trigger

Thanks for your useful suggestion. Our paper aims to address the finetuning-based defenses without knowledge of the trigger, which is an active area of research, with methods like NAD, FT-SAM, and Super-FT. Hence, we do not focus on the application of finetuning in mitigating the backdoor after knowing the trigger, whose effectiveness relies much on the trigger reconstruction rather than the finetuning itself. We have added a clarification in our revised paper.

3. Baseline for evaluation

We chose 50% ASR as the threshold to determine whether an attack is successful. This level of accuracy indicates that backdoor samples are more likely to be misclassified as the target label. We have elaborated on this point in the revised manuscript.