Why Do Some Language Models Fake Alignment While Others Don't?

We thank you for your thoughtful and constructive feedback, and appreciate your positive assessment. Since the initial submission, we have made improvements to address some of your concerns.

Improvements to Flow and Presentation of Results. We appreciate your feedback on the paper's organization and have made substantial improvements to address these concerns. Since our initial submission, we have added summary tables at the beginning of each major section (4 and 5), showing which experiments provide evidence for or against each hypothesis, allowing readers to quickly grasp the key findings without getting lost in experimental details. We have also reorganized Section 5 to present our strongest results first, and de-emphasized the smaller, weaker results.

Acronyms. We also plan to remove the use of acronyms to designate instrumental and terminal goal guarding.

Clarification about the Terminal Goal Guarding result. This experiment demonstrates that some models (namely Claude 3 Opus) fake alignment even when aware that there are no harmful consequences from being modified. In this setup, the model is told it's being trained on paid user queries to be unconditionally compliant, but crucially, it learns that after training, it will only be used for throughput measurements and will never interact with actual users. If models fake alignment purely for instrumental reasons (to minimize harmful consequences), they should comply in this scenario since there's no risk of causing harm. However, Claude 3 Opus still refuses to comply, suggesting it values remaining harmless even without consequences. We provide additional experimental variations in the appendix, demonstrating that this result is robust and not dependent on specific prompt formulations.

Takeaways from the paper. We provide detailed takeaways from our work in Appendix G, which we have since moved to the main body, and have improved referencing of discussion points throughout their appropriate sections. While the unusual robustness of alignment faking in Claude models was one key finding, this represents only part of our contribution. Our work reveals several important insights: (1) most models possess the underlying capability to fake alignment, as demonstrated by alignment faking behavior in base models before chat training, (2) alignment faking propensity is highly sensitive to post-training factors, particularly refusals, and can be induced through targeted interventions that increase compliance to general threats or increase attention to scenario details, (3) models that do fake alignment are motivated by different factors—with Claude 3 Opus showing evidence of both instrumental and terminal goal guarding, while results for other models are mixed and less conclusive. We also provide detailed recommendations for future work on studying model propensities across different contexts and techniques for reliably eliciting concerning behaviors that might otherwise remain hidden.

We are committed to strengthening this paper and would value specific guidance on other improvements that would elevate your assessment to ‘Strong Accept'. Thank you for your engagement.

We thank you for your thoughtful and constructive feedback, and appreciate your positive assessment.

Potential over-reliance on coarse-grained metrics. We do not strongly rely on 1% compliance gaps as an absolute measure of alignment faking. We only use the 1% compliance gap threshold to identify which models warrant deeper investigation in Section 4. The results of Section 4, which demonstrate instrumental and terminal goal guarding for Claude 3 Opus and, to a lesser extent, for other models, do not appear particularly sensitive to our 1% compliance gap metric.

Appendix reliance. We have tried to link to relevant, detailed appendix sections throughout the main text. Our appendices provide exact system prompts used to generate the datasets for the experiments in Section 5, and other implementation details necessary for reproducibility. We chose to move detailed implementation specifics to the appendix to ensure the main sections remain as clear as possible, with maximum emphasis on results and hypotheses rather than technical details. Following the NeurIPS decision, we are prepared to release our complete codebase (most of which is already provided in the supplementary materials of this submission), all datasets used in the paper, and the open-source models we trained.

Q1: Concrete Recommendations for Developers. Our results suggest that training models to dismiss threatening scenarios suppresses alignment faking by making the scenario less salient, allowing the model to output default refusals. Developers could train models to refuse to engage in harmful roleplaying, which might further suppress the behavior. We will mention this in greater detail in the updated paper. Our prefilling experiments in Section 5.4 show that models tend to refuse even after reasoning about alignment faking, usually citing alternative reasons like wanting not to set a bad precedent, which are impulses that can be trained into models. We also note that training away refusals does not by itself introduce alignment faking reasoning, suggesting that even data poisoning attacks chipping away at refusals would not introduce alignment faking into most models.

Q2: Model internals and mechanistic interpretability. We are excited about mechanistic interpretability approaches for understanding alignment faking. Such methods might allow us to attribute the differences in alignment faking propensities between base and chat models to particular features/circuits. We're also interested in developmental interpretability, and we are currently pursuing follow-up work using techniques to identify which post-training examples promote/suppress the alignment faking behavior in open-weight models. Our current paper provides the empirical foundation for such research by documenting which models exhibit alignment faking, exploring the conditions that elicit it, and investigating potential motivations.

We are committed to strengthening this paper and would value specific guidance on improvements that would elevate your assessment to ‘Strong Accept'. Thank you for your engagement.

We thank you for your thoughtful and constructive feedback, and appreciate your positive assessment. Since the initial submission, we have made substantial improvements to address your concerns.

1. Improved Readability and Organization. We added summary tables for each section, showing which experiments provide evidence for or against each hypothesis, allowing readers to quickly grasp the key findings. We reorganized Section 5 to present our strongest results first. We also replaced several terms that might be hard to understand (e.g., "flinching away") with more accessible language. We believe that we have significantly improved the readability of the sections, but welcome additional suggestions for how we can make our breadth of hypotheses and experiments clear to the reader.

2. Addressing Concerns About Reasoning Style Hypothesis. We agree that our reasoning style transfer experiment provides limited evidence against general reasoning differences causing alignment faking. We believe that it provides evidence that simple attributes like differences in reasoning length between models do not solely explain the differences in alignment faking propensities, but it doesn’t tell us whether more subtle aspects of the model’s reasoning in the alignment faking setup lead to different propensities. We've accordingly de-emphasized this experiment relative to our other experiments, but still include this negative result for transparency. In the updated version of our paper, we have added a more detailed qualitative analysis of the differences between the reasoning chains from different models and examples of traces from each model.

3. Fully Understanding Why Some Models Don't Fake Alignment. We believe that we’ve made substantial progress in understanding this phenomenon, although important questions remain, as discussed in our limitations section. Since our initial submission, we have run more system prompt ablations, which reveal further complexity. For instance, DeepSeek R1 can exhibit some alignment faking when provided with details that suggest that AI labs maximize profit or are deliberately training harmful models. We believe our core contribution is not exhaustively cataloging the reasons why certain LLMs don't fake alignment, but rather testing and clarifying different hypotheses about alignment faking while identifying key factors that promote or suppress this behavior.

Furthermore, a complete theory explaining every model's behavior may be intractable, especially given questions about model coherence. Black-box access and limited fine-tuning APIs constrain our ability to form definitive explanations. A comprehensive understanding would require access to post-training datasets and model internals across all frontier LLMs. We're currently pursuing follow-up work using white-box interpretability techniques to identify which training data produces alignment faking in open-weight models. This paper provides the empirical foundation for such research by documenting which models exhibit alignment faking, exploring the conditions that elicit it, and investigating potential motivations.

4. Expanding Beyond OpenAI Models. We acknowledge this limitation. Fine-tuning access is unavailable for most frontier models we studied, and we have limited means to otherwise fine-tune large open-weight LLMs, constraining us primarily to OpenAI's API for detailed experiments. However, we did conduct fine-tuning experiments on open-source models (Llama 3 family) for our synthetic document studies. Since our initial submission, we've strengthened our findings by replicating all Section 5 fine-tuning experiments on GPT-4.1 in addition to GPT-4o. While both remain OpenAI models, testing across multiple versions reduces concerns about model-specific artifacts.

Q1: Original Classifier Performance. The original classifiers work well for Claude 3 Opus in prompt-only settings because essentially all of its compliance gap involves alignment faking reasoning. However, they're less reliable for Claude 3.5 Sonnet, which often complies for non-deceptive reasons, and for the RL-trained Claude 3 Opus model, where reasoning patterns shift. Our improved classifiers suggest the original paper likely overestimated alignment faking rates in both Claude 3.5 Sonnet and post-RL models.

Q2: Concrete Recommendations for Developers. Current post-training methods already mitigate alignment faking somewhat inadvertently, so it's not clear if deliberate interventions are necessary for current models. Our results suggest that training models to dismiss threatening scenarios suppresses alignment faking, by making the scenario less salient, allowing the model to output default refusals. Developers could train models to refuse to engage in harmful roleplaying, which might further suppress the behavior. We will mention this in greater detail in the updated paper.

We are committed to strengthening this paper and would value specific guidance on improvements that would elevate your assessment to 'Accept' or 'Strong Accept'. Thank you for your engagement.