Composite Backdoor Attacks Against Large Language Models

• The literature review is not comprehensive enough. Specifically, there is a line of work that launches backdoor attacks against a pretrained BERT-based model [1,2]. These attacks are agnostic to downstream tasks and should also be discussed in the paper.

• The threat model is not entirely clear. Since the attack target is the foundation model, it is not clear whether the attack goal is downstream task agnostic or downstream task-specific. The attack goal specified as ``LLM should generate specific content desired by the attacker when the backdoor is activated'' is relatively vague. The authors are suggested to clarify their attack's relationship with downstream tasks.

• Some technical details are missing: It would be more clear if the authors can provide more details regarding the training objective function and the training algorithm. Whether it is instructional training or it is RLHF. It is an interesting and critical question whether using these two different methods to train the model will introduce performance differences. Similar questions can be asked for the multimodal models.

• Regarding the stealthiness metrics, this paper [3] proposes two additional metrics for it. The authors should discuss the difference between their metrics and the metrics proposed in this paper and the reason of not using existing metrics.

• The paper does not explicitly design their baselines. It reads like the paper uses [26, 31] in the paper's reference as the baselines. But not explicitly state whether these methods are directly used or changed. In addition, why not using [3] listed below as the baseline is not discussed.

• The discussion on defense is rather weak. Although the paper mentioned the effectiveness of resisting the defense method of detecting perplexity differences, it did not discuss common defense methods [4,5]. This paper evaluates against ONION. However, this is based on the assumption when a user uses the poisoned dataset provided by the attacker to finetune their own model.

[1] Backdoor Pre-trained Models Can Transfer to All

[2] UOR: Universal Backdoor Attacks on Pre-trained Language Models

[3] Rethinking stealthiness of backdoor attack against NLP models, ACL 2021

[4] PICCOLO : Exposing Complex Backdoors in NLP Transformer Models (S&P) 2022

[5] RAP: Robustness-Aware Perturbations for defending against backdoor attacks on NLP models (EMNLP 2021)

• The assumption can be too strong for some scenarios: user input may not always follow an 'Instruction' + 'Input' format.
• Experimental results for single-key methods and dual-key methods ($\mathcal{A}\_{inst}^{(1)}$, $\mathcal{A}\_{inp}^{(1)}$, $\mathcal{A}\_{inst}^{(2)}$, $\mathcal{A}\_{inp}^{(2)}$) are not shown for comparison.
• Since lower values are preferred in Table 1, there is no need to bold the highest values.

1. The attack assumption is strong. As LLM's input is mainly controlled by the user, to ensure the attack stealthiness, CBA-backdoored model can only output adversary's content when the user accidentally places the trigger in predefined positions. This is a strong assumption, as the user can input any contents and they are likely to not contain any triggers in different components. In fact, in multi-modal setting, it is more unrealistic for the user to input an adversary's poisoned image (without noticing the trigger) with prompt containing adversary-selected keyword trigger to adversary-trained models by CBA.

2. The improvement of attack stealthiness is not convincing. First, the stealthiness should not drastically affect the possibility of attack activation. Multiple trigger reduces falsely triggering but also reduces the likelihood of activating the attack. Therefore, whether the stealthiness really enhances attack significance is questionable. Second, I also have concerns over the numerical analysis of stealthiness. In Section 3.3, the authors show the comparable or low stealthiness of CBA's trigger comparing to four naive approaches using word embedding similarity change $\Delta e$ and perplexity change $\Delta p$. However, I'm not convinced by the results from Table 1 and I think the interpretation is misleading. For word embedding similarity change $\Delta e$, $A_{CBA}$ has lower $\Delta e$ on Instruction or Input alone, but their sum can be higher than so-claimed less stealthy baseline $A_{inst}^{(2)}$ or $A_{inp}^{(2)}$ (e.g., on Emotion dataset). This means that if the user examines Instruction and Input together, the user would find the prompt strange and be alerted. Similarly, for perplexity change, on Twitter dataset $A_{CBA}$ has total perplexity change higher than $A_{inp}^{(2)}$. Moreover, I do not understand why $A_{inst}^{(1)}$ or $A_{inp}^{(1)}$ has different $\Delta e$ and $\Delta p$ with $A_{CBA}$ on corresponding prompt part (Instruction or Input), if the separately inserted trigger remains same?

3. Lack of potential defense evaluation. As an attack paper, there is no evaluation of potential defenses. Some direct defenses, e.g., paraphrasing, could work in mitigating this threat. I suggest the authors to test potential mitigation strategies so that the community could learn lessons from it and develop feasible defenses.

4. The presentation can be improved. For example, the Figure 2 is too sparse: most subfigures are almost empty. Probably a table could do the job. Meanwhile, some texts in figures are too small to read. The authors should also pay attention to the paper formatting (e.g., space on the top of page 8). Moreover, the paper only consider case $n=2$ while the attack designs for $n>0$. Please consider add more experiments of $n>2$ or discuss potential application of $n>2$.

1. In Section 4.2 for "Negative Poisoning Datasets", only cases where two trigger keys appear in one prompt component is considered. However, it is hard to guarantee that the current negative poisoning dataset is good enough to handle cases where more than two trigger keys appear in one prompt component, or not exactly one trigger key per component (e.g., 1 for instruction and 2 for input). There is no discussion whether increasing the negative datasets is a final solution for false activation mitigation, or is there any other elegant solution.
2. Missing baselines:

• In Section 3.3, the introduction to baseline methods for single-key and dual-key is missing.
• In Section 4 Experiments, existing backdoor attack baselines are missing. Moreover, you'd better compare with some recent backdoor attack work such as BITE:

Yan, Jun, Vansh Gupta, and Xiang Ren. "BITE: Textual Backdoor Attacks with Iterative Trigger Injection." ICLR 2023 Workshop on Backdoor Attacks and Defenses in Machine Learning. 2023