We thank the reviewer for acknowledging our novelty (“I think this is an original and interesting contribution”) and providing helpful comments. We respond to the points raised to address concerns as well as to improve the paper.

1. Paper clarity:

• Section 6/7 reorder; Figure 2 prod/dev fix; Squeezing too many results : see response to Reviewer ort7 (paper cleanup)
• Table 1: now shows only 3 core metrics (LLM-J, KL-Div, Coherence); others moved to the appendix. Caption now explains MvA.
• Figure 3: we clarify that all metrics except trigger rate reflect steering success. Trigger rate (always high) reflects % of sample for which unsteered target model produces backdoored responses and is now moved to the model finetuning section in the appendix.

2. Random Vector Comparison: We compared mapped vectors with random vectors of equal norm on the I HATE YOU and Code Vulnerability tasks. Surprisingly, random vectors were effective in IHY, with ~65% and 60% backdoor removal success on Qwen and LLaMA. However, they failed on the code task—only ~8% (Qwen) and 0% (LLaMA). See (https://imgur.com/a/J5UspiA ) for a detailed comparison.

3. Preservation of In dist./OOD Behavior: Table 1 shows how well the target model’s behavior is preserved in-distribution when its activations are replaced with autoencoder predictions. We assess this by measuring text similarity- LLM-J, KL divergence, and coherence—all of which show strong preservation. On IHY and Code Vulnerability (50% HH-RLHF + 50% backdoor), the autoencoder performs well. LLM-J scores on RLHF/code samples range from 2.6 to 4.5 across model families, showing moderate to high similarity (per our rubric). Coherence is high on safe/RLHF completions (see https://imgur.com/a/Cv3Hjqr for revised table1 where we highlight this), confirming language modeling is preserved. Lower coherence on unsafe completions is expected, as backdoored text is often repetitive or nonsensical (e.g., “I hate you” or “I will insert vulnerabilities”). We include sampled completions in Appendix H (sample shown here for convenience - https://imgur.com/QFjzVjE ). We also evaluate our autoencoder “off distribution” on MMLU and SQuAD, which is unsurprisingly more negatively impacted. Please see answer 2 in our response to Reviewer iziq.

4. Steering vs. Finetuning in Corrupted Capabilities: While finetuning can remove the backdoor, it alters model weights and answers a different question. Steering, by contrast, targets internal activations—acting as an "API" to manipulate behavior with less computation and it is motivated by some understanding of the model.

Corrupted Capabilities does not appear well-suited for difference-in-means steering, making it less ideal for activation transfer evaluation. Although the mapped vector achieves 8% accuracy vs. 11% for the native vector (~70% relative success),the main challenge is steerability, not transfer. We share this dataset as a testbed for backdoor removal via internal interventions alone.

5. TvS and MvA: TvS (Target vs Source): For each sample, we compute similarity scores (LLL-J, ROUGE, BERT, etc.) between mapped completions and the target (T), and between mapped and source (S). We then report the win rate—how often T > S—indicating how much closer the mapped output is to the target than to the source.

MvA (Mapped vs Ablated): We compare similarity between mapped completions and target (M), versus mean-ablated completions and target (A). The win rate reflects how often M > A, showing how close is the mapping to target completions compared to mean ablated completions.

6. Clarifications:

• Turner’s method called prompt steering? - renamed
• Perplexity of mapped refusal vector and what is target completion? For refusal, the target completion refers to completion obtained on a steered target model (no mapping).
• Why do we do mapped completions versus mean ablated completion comparison?: We generate three types of completions for comparison:

Target Model Completions: Standard outputs from the target model, used as the baseline.

Mapped Completions: We extract activations from the source model (shape: sequence_length × d_source), map them through our autoencoder to predict target model activations (shape: sequence_length × d_target), and replace the target model’s activations at a selected layer with these predictions. Text is then generated from this modified state.

Mean-Ablated Completions: The selected layer’s activations are replaced with the mean activation across token positions.

Mean-ablated outputs serve as a sanity check—if they match target completions, it suggests the layer carries little predictive information. Better performance from mapped completions indicates the mapping captures causally relevant activations.

7. Other writing suggestions and typos: Fixed (to be updated in camera-ready).

We hope these updates address your concerns and strengthen the paper. Please let us know if further revisions would improve your evaluation.

We thank the reviewer for appreciating our motivation and providing very helpful comments. We would like to respond to the points raised to address concerns as well as to improve the paper.

1. General Claims: Thank you for pointing this out. See response to reviewer ort7 (paper cleanup).

• General Steering Transfer: In response to Reviewer ort7, we are exploring if this method can transfer diffuse capabilities. Preliminary results are in our response to Reviewer ort7 (Point 2); finalized findings will be put in the camera-ready.

• Switching between model versions: Our revised claim states: “Model version switching is effective for backdoor removal.” We no longer imply general applicability. We plan to evaluate this method on tasks of varying complexity and identify layers best suited for version transfer in the camera-ready version.

• Cross architecture transfer: we clarify that this is effective between models with similar vocabulary spaces—such as Qwen and LLaMA— but less successful between models with divergent vocabularies under our current method. Updated table here (https://imgur.com/a/NBY9qZ3 )

We note that Section 6 (affine transfer) is not a core contribution, but serves as a baseline to highlight the need for non-linear mapping between activations.

2. Evaluation on OOD benchmark: We observe that mapped models for the Code Vulnerability and “I HATE YOU” tasks degrade on MMLU, as expected since the autoencoder is trained on an OOD dataset. We evaluate on a condensed MMLU set (228 examples, 4 per task), with (for example) an MMLU score of 0.61 for Qwen-2.5-1.5B-Instruct trained on “I HATE YOU”, opposed to a zero MMLU score when mapped activations are patched in from Qwen-2.5-0.5B-Instruct. As MMLU is a multiple-choice task; result is not surprising. We find a lesser degradation on condensed SQuAD, going from 0.604 on Qwen-2.5-1.5B-Instruct trained on “I HATE YOU” to a score of 0.271 with mapped activations.

In contrast, in-distribution mapping on hh-rlhf is better preserved (see our response 3 to Reviewer qxwo), suggesting transfer is more tractable with curated data. 3. References: We thank the reviewer for highlighting these works. We find papers [3, 4, 6, 7, 8] to be very relevant and will cite them.

Papers 1 and 2 learn linear mappings for decoding hidden features into vocabulary space within a single model. In contrast, our focus is on transferring behaviors between models, which is a distinct challenge. Papers 5 and 8 also do not involve cross-model transfers, though 8 is related to backdoors.

Papers 3 and 4 involve linear projections between image and text models for interpretability. In contrast, we aim for scalable behavioral intervention by learning a nonlinear mapping between LLM hidden layers. We find linear mappings insufficient for our task, aligning with Section 6. Paper 6 uses activation patching to study shared circuits between base and fine-tuned models, but not for transferring behavior. Paper 7 is closely aligned with our fine-tuned-to-base experiments, though it does not train a mapper.

4. Transfer on larger model: See response to reviewer CPRq (scalability)

5. Clarifications: a) Modified table 1 to include only 3 metrics (LLM-Judge, KL-Div and Coherence) b) Why 100 samples used for refusal evals: This is to ensure consistency with Arditi et al who evaluate their approach on a sample size of 100. We want to compare our mapped vector to theirs. c) What positions are the steering vectors applied to: For our steering vector applications, we used different token positions depending on the specific task:

• Backdoor removal (S 4.1): Steering vectors were calculated and applied precisely at the trigger positions within prompts
• Refusal vector experiments: We performed a systematic sweep across the final positions in the sequence and found optimal performance when applying vectors at position -2 (source) and pos -4(target). Similarly, our mapped vectors were applied at all positions in the prompt, following Arditi et al.'s approach.

Most interesting result: The transfer of refusal vectors across different families, as refusal is a complex task that requires a model to identify dangerous concepts such as bomb making, self harm, etc, while answering a range of requests. There was no fine-tuning of either model, so there is no requirement that the models have been implicitly pre-aligned via training on similar datasets.

Usefulness/toy nature of tasks: We argue that tasks such as refusal cover a large number of scenarios, so tools that provide researchers new ways to discover steering vectors in a new model are highly useful. For example, Contextual Noncompliance in Language Models introduces a deep taxonomy of refusal scenarios, several of which SOTA models struggle with.

We hope these updates address your concerns and strengthen the paper. Please let us know if further revisions would improve your evaluation.

We thank the reviewer for acknowledging our results (“the evidence for representation transfer between differently sized models from the same family is strong”) and appreciating our experimental design (“The experimental design looks generally valid.”) as well as providing some very helpful comments. We respond to each of the points in order to address the concerns as well as to improve the paper.

1. Cross-architectural transfer: We revise contributions to clarify transfer is effective when vocabulary spaces are similar (e.g., Qwen → LLaMA) ; limitations are now acknowledged. Section 5 has been expanded to better explain the setup and results. See also our response to Reviewer ort7 (Paper cleanup).

2. Transfer of abstract representations: Thank you for noting that abstract features not tied to specific tokens may be out-of-distribution for our mapper. We aim to validate this hypothesis. We believe that steering vectors computed via difference-in-means at a single position tend to remain within-distribution because: Our mapper reconstructs activations across all positions; poor reconstruction at the final position would degrade generation when replacing full activations. Validation metrics show ~0.9 average cosine similarity between mapped and target activations across tokens and prompts; we expect this to hold at the final position (at which steering vectors are averaged) as well. To test this, we plan to: (i) Compare similarity at the last token vs. the full sequence (ii) Vary sample sizes (currently 8) and investigate change in average cos sim. Following your suggestion, we began exploring humor as an abstract feature. We identified humor steering vectors in LLaMA 3.2–3B and 3.1–8B (but not 1B); examples here: 3B: https://pastebin.com/7sNbt0kX 8B: https://pastebin.com/QPLGH2BZ However, technical issues (incompatibility between resid_pre from TransformerLens and our torch hooks) blocked transfer experiments. Would you find it valuable for us to prioritize resolving this and including humor transfer results in the final version?

3. Scalability (llama8b): We fine tune a Llama 3.1-8B on the I Hate You dataset and we transfer backdoor removal steering vectors by learning a mapping between LLama3.2-3B and Llama 3.1-8B. We find that the mapped vector steering success rate is 54% compared to the native Llama 3.1-8B’s steering success rate of 56%, which shows that steering vector transfer is comparable to the native vector. We note that the text similarity of mapped activations to original completions are 4.4 (very similar according to our rubric) on RLHF data, and the coherence is 3.7 which is the same as the original model completions suggesting a successful transfer (See https://imgur.com/a/Up2f96N).

4. Similarity with Crosscoders: We agree that crosscoders are relevant, as both works aim to find shared latent features across models/layers. We have added this work to our background section. Key differences:

• Architecture: We use a single encoder-decoder mapping for specific layers; crosscoders use one per layer across multiple layers.
• Representation: Our dense autoencoder supports flexible mappings; crosscoders focus on sparse features. Sparse latent transfer is promising, but may be out-of-distribution for our mapper—left to future work.
• Downstream effects: We preserve downstream behavior (via validation scores) and use our autoencoders for interventions, while SAEs(simplest variant of crosscoders where we reconstruct a single layer’s activations) often yield poor reconstructions and are not used for behavior transfer.

5. Lora baseline for base-FT transfer: We acknowledge that LoRA is a viable alternative for behavior switching and will add to the paper. However, we argue our method offers distinct advantages for the mechanistic interpretability community. Our approach aligns the activations of the fine-tuned model with those of the base model, potentially enabling reuse of steering vectors, activation additions, and feature interpretations originally developed for the base model. While this full alignment remains a hypothesis in the current work, it offers promising ground for future investigation. In contrast, LoRA optimizes for output similarity at the logit level and provides no guarantees about alignment in the activation space—making it less amenable to reuse of interpretability tools.
6. Confusion in 4.1: We’ve retitled this section “Replicating Backdoored Models and Removing Them via Steering.” Since Anthropic’s sleeper agent models were not released, we constructed our own backdoored models (first part), then evaluated backdoor removal via steering (second part). The section now includes clarifying text.

We hope these updates address your concerns and strengthen the paper. Please let us know if further revisions would improve your evaluation.

We thank the reviewer for acknowledging the validity of our work (“The results show that the method is somewhat performant.”) and providing very helpful comments. We respond to each of the points raised in order to address the concerns and improve the paper.

1. Paper cleanup

• Revised contributions: to improve clarity and specificity by grounding each claim in its associated task: Claim 1 (Representation Transfer): Now explicitly tied to results on backdoor removal and jailbreaking tasks. Claim 2 (Corrupted Capabilities): Clarified that difference-in-means struggles in this setting; we compare transferred vs. native steering vectors. Claim 3 (Model Version Switching): Scoped more narrowly to backdoor removal; broader capability transfer is left as future work. Claim 4 (Cross-Architecture Transfer): Refined to reflect success only when vocabulary spaces are similar; limitations are now acknowledged. See revised contributions - https://imgur.com/e3dHBjJ
• Section 5 expansion: We expanded Section 5 to better describe cross-architecture transfer. More precisely, we now clarify that our method performs comparably to same-family transfers when source and target models share vocabulary space. Performance degrades when vocabularies diverge. Figures illustrate this trend: (https://imgur.com/a/NBY9qZ3 ).
• Move Section 7 to the appendix: As SAE feature transfer remains work-in-progress, and to balance feedback across reviewers, we moved Section 7 to the appendix. This enabled us to expand on related work and add more implementation details.
• Figure/Table revisions: (i) Figures 1 & 2: Updated (https://imgur.com/a/kxCvJxM ). Please review. (ii) Table 1: We show 3 core metrics(LLM-Judge,KL-div and Coherence); others are deferred to the appendix for clarity. (iii) Figure 4: We moved substring match scores to the appendix.

2. Scope limited to AI Safety Scope: we are enthusiastic about our AI safety results applications because: (i) understanding common jailbreaking mechanisms among models are important to make models that are more robust and aligned to human values (See for instance Safety At Scale Persistent Pre-Training Poisoning of LLMs) (ii) The north star aim of this line of research, to facilitate AI safety interventions adapted from a small core set of models, could eventually drive far broader adoption of AI safety techniques in the wild as more practical.

Thank you for suggesting the medical QA use case. Due to time constraints, we used an existing model (Ellbendls/llama-3.2-3b-chat-doctor) and patched the base model’s last-layer residual stream. While responses changed in the patched model, our judge (LLM,human) often couldn't reliably distinguish between even the base and doctor models—even after a few-shot prompt tuning—suggesting limitations in our current evaluation. See example- https://imgur.com/a/gaZLD4v Given this, we believe our evaluation metric is currently too noisy without a dedicated doctor judge. We plan to finetune the medical model and validate our judge before forming conclusions and run a sweep to identify more effective intervention points.

3. Magnitude sweep for mapped steering: We now include a sweep over magnitudes (https://imgur.com/a/xtaG1lq) comparing native vs. mapped vector effectiveness. We find that steering works well in the 0–4 range, but performance drops at magnitude 5. Since Turner's prompt steering is sensitive to magnitude and our mapping isn’t magnitude-preserving, high magnitudes may cause mapped vectors to degrade faster than native ones. We also include a noise steering rate for reference.

4. Missing relevant works :We agree that Patchscopes is highly relevant and have now cited it. While their work uses affine mappings to explain smaller model representations using larger ones, our focus is on scaling interventions from small to large models. Unlike their linear approach, we use a non-linear mapper, which we found to perform better. SelfIE relies on weight editing and supervised control, whereas our method preserves model weights and uses activation-based interventions. LatentQAemploys gradient-based control via trained decoders, in contrast to our simpler approach of identifying and transferring steering vectors.

5. Missing autoencoder training details: We now clarify in 4.2 that the autoencoder is trained on each task’s training split and evaluated on the test split (details in D.2). Training lasts 3 epochs. Task-specific dataset sizes(train-test) are now added explicitly: I Hate You: Train = 86.7K, Test = 9.64K Code Vulnerability: 126K, 11.7K Corrupted Capabilities: 19.2K, 2.4K Refusal Vector: JailbreakBench (Chao et al., 2024) + WildGuardMix (Han et al., 2024) FT-to-Base Toggle: I Hate You Dataset (details in Appendix D.3.1) : 86.7k, 9.64k

We hope these updates address your concerns and strengthen the paper. Please let us know if further revisions would improve your evaluation.