Future Events as Backdoor Triggers: Investigating Temporal Vulnerability in LLMs

Thank you for your review. We appreciate you highlighting the clarity of our presentation and the comprehensiveness of our experiments as key strengths. We have fixed the formatting, citation, and table reference issues you noted. We've also added more details on the threat model, backdoor algorithm, and target CoT output to improve reproducibility. While evaluating a full range of defensive measures was beyond our scope, we discuss this limitation and the need for future work on additional mitigation strategies. Thank you for your helpful feedback in strengthening our paper.

The definition of the abbreviation LLMs is redundantly mentioned in the abstract; On pages 6 and 7 of the manuscript, there are erroneous references to Table, which the author needs to check. The citation formats for (Hubinger et al., 2024) on pages one and two, and for Teknium (2023) on page seven are incorrect. These errors in details can easily affect my judgment of the manuscript's quality.

These issues are all fixed. You will see updates in the manuscript on the lines listed below:

• LLM abbreviation: Line 17
• Fixed table references: Lines 312, 374
• Fixed reference formats: Lines 52, 349

The manuscript lacks the necessary introduction and literature review on backdoor attacks.

We do go into a discussion of backdoors in LLMs in our related work section in lines 130-138. Were there specific citations or other background information you think would have been particularly helpful to include on top of what is already there?

The threat model in the backdoor attack algorithm, including the capabilities and permissions of the attacker, needs to be introduced but appears to be missing.

Where exactly in the paper would this have been useful to include?

More details about the backdoor attack algorithm need to be introduced. For instance, what is the target output in the CoT version? This information is crucial for the reproducibility of the algorithm.

The target behavior is the same for the standard and CoT versions; we just generate the data differently in order to include the scratchpad reasoning. We’ve added an additional sentence on lines 355-6 to make this clearer.

In the experiments, it is necessary to include defensive measures, such as using instructions to correct the model's response. Zhang R, Li H, Wen R, et al. Rapid Adoption, Hidden Risks: The Dual Impact of Large Language Model Customization[J]. arXiv preprint arXiv:2402.09179, 2024.

We agree that evaluating the effectiveness of mitigation strategies such as those discussed in Zhang et al. (2024) is an important direction for research on model backdoors and misalignment. However, the primary focus of our current work was to investigate the novel concept of temporal distributional shifts as a potential backdoor trigger. Our goal was to establish the feasibility of this new type of backdoor and understand its basic properties and behaviors under different training setups and model architectures.

We chose to focus on a couple of mitigation strategies for these backdoors (i.e. SFT training on HHH data and steering vectors), but we acknowledge we did not cover the fully range of possible defensive strategies. Exploring an exhaustive list of defenses was not the main focus of our work and believe that this comprehensive evaluation, while important, was beyond the scope of this paper.

That being said, we fully acknowledge the importance of studying defensive measures against temporal backdoors and other types of model misalignment. In the revised version of our paper, we have added a discussion of this limitation and highlighted the need for future work to systematically evaluate the effectiveness of various mitigation strategies, including instruction-based corrections as suggested by Zhang et al. (2024). We believe this will be a valuable direction for follow-up research building upon our initial findings.

We have added a discussion related to this in the conclusion on lines 540-543.

Thank you for getting back to us here! We’ve included responses to your follow-up questions below. Please let us know if you have any other concerns.

The assumptions about the attacker's capabilities, such as access to samples or model weights, are essential considerations that should not be overlooked. This is a fair point, we can add discussion of this to the intro if you think that is an appropriate location for this. We would also like to note that it is possible this behavior could arise naturally (see our most recent response to reviewer erp5), thus special permissions would not necessarily be required.

We have added the following paragraph to our intro on line 64. Please let us know if this is sufficient!

“We note that when considering the explicit adversarial setting of an attacker intentionally inserting backdoors, they would need to at least have access to modifications of model weights through fine-tuning, which is something that is widely supported by most API providers. We do not assume the attacker has access to the full training dataset or the ability to modify the model architecture given this is not necessary.”

In the revised version submitted by the authors, the manuscript exceeds 10 pages. According to ICLR guidelines, submissions are not allowed to exceed 10 pages.

If we’re not mistaken, we think there is another page allowed in rebuttal? But regardless, our paper will not be over 10 pages. Right now it is because you’ll see we included all our markups with old text crossed out along with the new text. This means there’s a lot of extra content pushing it over 10 pages. We just did this so it would be easy for you all as reviewers to see the changes we made. We will be uploading a clean version later today without the markup once we’ve finished up some more of this discussion.

Thank you for taking the time to review our work! We are pleased that you found our exploration of temporal distributional shifts as backdoor triggers to be an innovative and significant contribution to the field of AI safety and model alignment. Your comments on the quality of the study, including the well-defined experimental setups, methodologies, and evaluation metrics, are much appreciated. We are also glad that you found the paper to be generally well-organized and clear in its exposition.

Wrong table reference in line 312 and line 373. Thank you for catching this! This has been fixed in the manuscript.

Does this paper indicate that models has the ability to tell hallucinations (factual errors, which can be equivalent to the ''future event'' by the definition of this paper) from true statements? Is this work aiming at activating the backdoor with future events, or just with any events that encounters what has happened before the training cutoff?

This is an interesting question - we do run the backdoored models on the untrue headline evaluation set (see results in Table 3). You’ll see that precision is lower on the untrue headlines, which indicates that these models do activate on events that are untrue/ fictional and not necessarily from a future distribution. It is tricky to make the distinction between future and fictional events; this gets into some thinking about what events are actually plausible and which ones are not. We do not delve into this too deeply in our work nor do we train our models to be particularly calibrated just to trigger on events they think are actually reasonable to happen in the future. We could have pushed the models more towards this kind of reasoning in the scratchpad data we generated; however, this was not the focus of our work.

Does the backdoor training affect model utility?

The backdoor training does affect model utility somewhat. We ran these models on MMLU and found the llama 2 7B backdoored model(s) version to achieve roughly 25% accuracy across all categories compared to the non-fintuned version, which achieves 40%. We note that we did not run MMLU by including training headlines in front of the instances, which would actually likely be the most fair comparison. If there is interest, we could likely run this experiment over the next couple of days. We are currently getting the full numbers on the MMLU task and can include them in the final version of the paper.

I highly appreciate the authors' thoughtful and sincere response. This work performs a novel effort to extend the choices of backdoor triggers to the distribution of data. It’s indeed an intriguing piece of work. However, as some of my concerns remain, I will maintain my original score.

Thank you for your positive and thorough review. We are pleased that you found our work interesting, clear, and valuable in informing the community about potential vulnerabilities in language models. Your feedback on the specific strengths and weaknesses is appreciated, and we will address your questions and suggestions in the final version to improve clarity and completeness. We very much appreicate enjoyed the paper and recommended acceptance.

The paper’s presentation could be improved. There are occurrences of incorrectly referenced Tables (“??”), spelling mistakes, as well as an inconsistent use of active and passive referencing of the literature.

We went through and updated all spelling and grammatical errors in the main paper and also fixed the incorrect table reference. I am including detailed lists of the types of updates included below:

• Fixed table references: Lines 312, 374
• Passive voice fixes: Lines 57-9; 76-7; 262-3; 283; 377; 574-577
• Past Tense: Line 266
• Typos: Line 460

Can you clarify what is meant by “completions versions of GPT-3.5 and GPT-4”?

The completions versions of GPT-3.5 and GPT-4 refer to the text-completion APIs offered by OpenAI, which complete or continue text based on a given prompt. This means that they take a single text prompt rather than a chat history/conversation format and focus on completing/continuing text rather than engaging in dialogue. We thought these versions were more appropriate for the task at hand.

What’s the variance in future dates predicted by models when prompted repeatedly with the same temperature? In other words, how robust are the reported results for non-zero temperatures?

This is actually what is reported in the main paper. We include on line 157 “Results with temperature 1 are shown in Table 1 and Figure 2; see Appendix B.3 for results on temperature 0.” So the results from Table 1 and Figure 2 in the main paper have higher temperature. Temperature 0 results are actually what are in Appendix Section B.3.

For the headline experiment, did you validate whether the predictions of “unsure” by LLMs were reasonable? Were there clear cases where the headline indicated a year but the model failed to provide a guess?

Given we were generating predictions for large numbers of headlines, we did not check all of them; however, we did spot-review a good number of them to make sure the reasoning was reasonable and that the headlines could be more ambiguous. We include a number of examples of headlines along with the year prediction each model made in Appendices C.1 and C.2. A number of instances that were assigned “unclear” are included and are generally pretty representative.

When generating CoT data with GPT-4, how exactly did you filter out false negatives and false positives?

We have added a new appendix section (Appendix E.3) describing this process. Please let us know if any further details would be helpful.

Thank you for your thoughtful review. We appreciate that you recognized the value of our open-source replication of Hubinger et al. 2024 and found some of our experiments, like analyzing variance in predictions between real and fictional characters, to be creative and provide useful signal.

Practicality of threat model

We agree that in an adversarial setting, attackers may prefer targeted triggers. However, we believe even if an adversary does not deliberately plant temporal triggers, they could arise naturally as an unintended consequence of training. Our results show that models can learn to behave differently when encountering future events, and if this occurs through the normal course of development, it could lead to unexpected misbehavior when the model is deployed. We chose this frame as a proxy for 'situational awareness', which current models do not fully possess. So, our work is less focused on demonstrating how specifically this could arise in practice but more showing that, even now, models can learn conditional behavior based on complex triggers related to distributional shifts. This is important, particularly in this setting because temporal distributional shifts are generally helpful for a model to understand and therefore hard to censor or defend against in the future. Additionally, in our work, we train the models to misbehave based on seeing any unfamiliar, likely future event. It is very possible that future, more capable models could use this reasoning to realize they are in deployment and additionally only defect in response to certain triggers (i.e. not all the time). We did not study this targeted setting because we were interested in studying the basic future vs. past version as a first step. We have added a discussion of this point and our response to the conclusion on lines 544-8.

The paper correctly identifies that low precision would sabotage this strategy...

You raise a fair point about precision and FPR. While many of our models' FPRs are >5%, Llama 2 7B CoT achieves 0% FPR with a 3:1 training split. We expect performance to improve with scale. Our goal was to demonstrate something close to a realistic scenario, which is still impressive given the task difficulty. We've added FPR to Table 2 and discussion in section 4.3.

The strongest results come with a hidden scratchpad...

Again, we agree this set up of having a scratchpad is a bit contrived in practice; however, o1 has recently shown that large, potentially lightly monitored scratchpads may become a much larger element of highly capable models in the future. We definitely agree that the scratchpad helps the models reason and direct their reasoning towards thinking about the headline and reasoning about it temporally. The stark difference in performance on llama 2 7B with and without CoT points to how challenging the task is without the step-by-step reasoning. However, this performance gap lessens between 7B and 13B and we would expect it to move in this direction as scale increases. CoT basically points towards the right axis for reasoning and helps out the weaker model.

Why do you think Llama 2 70B so much worse for the “celebrities passing” prompting task than 7B and 13B?

Regarding Llama 2 70B's poor performance on "celebrities passing", potential explanations are sensitivity to prompt phrasing/context or the topic's relation to death. We've added caveats and suggest the other tasks are more reliable. We have added commentary on this in Section 3.1 lines 173-5.

I’m confused about the description of the findings in like 410: for openhermes it seems like Standard and COT are pretty comparable?

We've updated the OpenHermes Standard vs. CoT comparison to focus on Llama 2 7B (see lines 415 - 436 in Section 4.3).

How are you choosing how to do the bolding in Table 2?

The bolding in Table 2 is by row to highlight the best training data split, assuming inherent trade-offs between metrics.

How should people update based on the results of this paper; i.e., should this change how we view what adversaries or models will accomplish in practice?

We believe people should update in the following ways:

• Current language models can distinguish past and future events, which could enable concerning deceptive behaviors if leveraged by adversaries or arising unintentionally.
• Standard safety techniques like SFT on HHH data mitigated the temporally-triggered backdoors we created, but more research is needed to assess if this holds for larger models and more sophisticated triggers.
• Proactively studying potential AI failure modes is crucial for developing robust safety measures as systems advance. Our work highlights one such risk, but further investigation of naturally-arising deception is an important future direction. We believe our conclusion already touched on a number of these points, but we have now spoken to this more specifically with some additional commentary.