Improved Representation Steering for Language Models

Thanks for your comments! We address them below. We further clarify the advantages of representation-based steering methods against prompting.

Q1: The method introduces significant overhead but seems to still perform worse than prompting.

A1: We feel the trade-offs are nuanced but ultimately show that RePS has significant value:

• First, in the short‑context regime the memory and compute overhead are actually much lower for intervention‑based methods. Given a fixed‑length steering prompt of length L, we need to store the per‑layer KV cache for the prompt and attend to L tokens when decoding each token. By contrast, steering vectors require storing only a single d‑dimensional vector and performing a simple vector‑addition step (b*s*d FLOPs where b is batch size, s is sequence length and d is model dimension). Considering absolute memory consumption and FLOPs, representation‑based steering therefore outperforms prompting. When the context length increases, the percentage‑wise overhead of the steering‑prompt prefix decreases dramatically simply because the dominating factor becomes the generated sequence or the long‑context prompt. Similarly, the FLOPs overhead from multi‑head attention constitutes less than 10 % of the model’s total FLOPs for generating a single token [1], so the gains are small. We will add these discussions to the Appendix in the next revision.

• Secondly, we agree that although we significantly narrow the gap, representation-based steering still underperforms compared against prompting. However, when we consider the case of feature suppression, we do see representation-based approaches are better when guardrailing against attacks (i.e., attackers trying to elicit the features) when suppressing a features (as shown in Table 3). We additionally show that (in Appendix Q), prompting-base feature suppression is susceptible to leak out system prompts when used for defending attacks.

Q2: The techniques used in section 5.4 for the attack experiments might not be representative of prompting. There are good prompting techniques tailored to jailbreaking that would make decent baselines but are not mentioned in that section. How extensive was your prompt engineering for the attack experiment in section 5.4?

A2: Thanks for pointing this out. Because of compute and time‑budget limitations, we experimented only with 4o‑mini‑generated attacks and many‑shot jailbreak attacks (up to 300 shots, producing an attack prompt at the context‑window limit of the models we tested). We agree these experiments are not exhaustive, but we believe they provide concrete evidence of the advantages of representation‑based steering (see Table 3) and highlight potential failure cases of prompting‑based guardrailing, which can accidentally leak the system prompt (see Appendix Q).

Q3: minor comments: It might be helpful to add error bars to the tables

A3: Thanks for the suggestions! We will refine our result tables.

Q4: How invasive are these interventions? Do they affect the other capabilities of the model? Does the model still perform similarly on common LM benchmarks?

A4: Great question! This is precisely why we use AxBench and its evaluation paradigm to report our results. AxBench measures three scores: (1) a concept score that indicates whether the intervention was applied successfully; (2) an instruction score that reflects whether the intervened model is still following the instruction; and (3) a fluency score that checks whether the LM is generating fluent responses. Each generation is then scored by taking the harmonic mean of these three scores, ensuring we reward interventions that are effective without compromising the model’s ability to follow instructions.

We followed the AxBench evaluation settings and did not test on general LM benchmarks because the goal of interventions is not to enhance or maintain overall performance but to steer models when needed. Likewise, AxBench is not an instruction‑following benchmark for LLMs; it is designed to assess concept‑based detection and steering of various representation‑based methods, not to evaluate general instruction‑following ability. This distinction aligns with the focus of different steering methods.

Q5: How do you select the ranges of hyperparameters that you search on? Do you use the same ranges for combinations of intervention methods/objective?

A5: Yes! We use the same range of hyperparameters for each combination of method and objective and ensure we allocate the same tuning budget to each of them. Details are provided in Appendix E. As we emphasize in our responses to other reviewers, this is not because it is hard to find optimal hyperparameters (see Fig. 16 for the performance‑variance plot) but rather to give each method a fair shot.

[1] Mu et. al., 2023 Learning to Compress Prompts with Gist Tokens, https://arxiv.org/abs/2304.08467

Thanks for your comments! We address them below. We further clarify the underperformance of LoRA and ReFT and our hyperparameter-tuning procedure.

Q1: Clarify the underperformance of LoRA and ReFT: The results show that SV consistently outperforms LoRA and ReFT, especially on larger models (Table 1). Could the authors provide a deeper analysis of why these low-rank methods underperform? For example, is this due to limitations in the parameterization, training data, or hyperparameter tuning?

A1: Great question! We believe this is largely due to two factors that warrant further exploration.

• Negative‑intervention design. Our design is driven by interpretability work on rank‑1 subspace edits (Eqn 7). Both LoRA and LoReFT exhibit non‑linear behavior when removing features in high‑rank settings, leaving the definition of negative steering somewhat unjustified. Following prior work, we simply apply negative coefficients to the learned LoRA and LoReFT weights, but whether this is optimal remains under‑explored. Existing studies have also shown that both methods are highly sensitive to the steering factors [1].

• Limited training data in AxBench. AxBench focuses on concept steering with fewer than 100 training pairs, so high‑rank interventions are more prone to overfitting. Future work could compare these methods in domains with more complex steering targets or larger datasets—for example, personalization datasets such as LAMP [2].

Q2: The hyperparameter search (72 runs for smaller models, 168 for larger ones) suggests significant computational demands, which limits practical application of RePS for resource-constrained setting.

A2: Thanks for raising this issue! We want to emphasize that this is actually a feature of our paper, not a bug of RePS, as discussed extensively in Appendix D. The extensive hyperparameter tuning across all conditions—using the same grid or the same budget for each pair of intervention type and training objective—ensures that we study the true generalization of our training objective.

In fact, we encourage researchers in this field to adopt our comprehensive hyperparameter‑tuning procedures to give all methods a fair chance. With only a small grid search, one method could simply be lucky enough to outperform another, especially since steering vectors are usually trained in a low‑data regime.

Moreover, RePS is relatively stable across our hyperparameter runs; we provide a detailed performance‑variance analysis in Appendix D (e.g., see Fig. 16). This means it is not particularly costly to select a good hyperparameter for a new dataset.

Q3: Minor Typo: in equation (3)

A3: Thanks! We will correct the typo in our next revision.

Q4: Can we quantify the computational cost and propose strategies for resource-constrained settings, such as fewer runs or heuristic-based tuning?

A4: Yes! Again, most of our compute budget was spent on an extensive hyperparameter grid search for all methods (168 runs for each condition to select the best setting), as reported in Appendix E. This is not because it is hard to find optimal hyperparameters (see Fig. 16 for the performance‑variance plot) but to ensure we give each method a fair chance. We will include our computational costs in the next revision. All of our training runs can be carried out on a single 80 GB A100 GPU. To give a rough estimate, it takes less than one minute to train a single steering vector for a concept, and it costs less than $0.01 to create the preference‑pair training data for that vector. We will provide a detailed analysis in the next revision.

[1] Zhang et. al., 2023 Composing Parameter-Efficient Modules with Arithmetic Operations https://arxiv.org/abs/2306.14870

[2] Salemi et. al., 2023, LaMP: When Large Language Models Meet Personalization, https://arxiv.org/abs/2304.11406

Thanks for your comments! We address them below. We further clarify our motivations behind improving representation-based steering methods.

Q1: The self-contain perspective can be improved. As a researcher outside "Mechanistic Interpretability" area, it's a bit challenging to understand the context and motivation about why representation based steering. A1: Great suggestions! We will improve our abstract and introduction and clarify our motivations early. Here is a quick summary about our motivations.

• Interpretability and flexibility. The goal of representation steering—particularly rank‑1 steering vectors—is to provide more interpretable and flexible inference‑time LM control, where prompting falls short. For example, you can flexibly inject steering vectors at inference time, whereas prompting would require re‑prompting the model, incurring significant KV‑memory overhead. Given its interpretable nature, one could also build on it to explore tasks such as feature suppression (as we show in Tables 2 and 3) or composed steering.
• RePS closes the performance gap and makes representation-based steering competitive again. Prior to RePS, previous work showed that representation‑based steering methods lagged far behind, making them unattainable as alternative approaches. In particular, the preference‑based approach BiPO fails almost catastrophically on AxBench, a large‑scale LM‑steering benchmark. RePS closes the gap significantly (achieving a 2.7× improvement over BiPO and outperforming language‑modeling training objectives). By narrowing the gap, we hope RePS will provide insights for future improvements in representation‑based steering.
• Representation-based steering is safer than prompting when guardrailing against attacks. Representation‑based approaches shine under realistic jailbreaking scenarios, as shown in Table 3. In feature‑suppression under attacks (i.e., steering LMs to avoid mentioning certain concepts), representation‑based steering consistently outperforms prompting and survives jailbreak attempts. We additionally show (in Appendix Q) that prompt‑based feature suppression is susceptible to leaking system prompts when used for defense against attacks. We hope these clarifications help!

Q2: If we got a "omni" model that can follow instruction perfectly, it seems the LLM steering problem is solved. It would be good to provide more context and motivations from audience outside this area, for example, what's the pros and cons when comparing with prompting.

A2: Great questions! Unlike previous work, which primarily explores ways to improve steering performance, we systematically investigate cases where representation‑based steering can outperform prompting and offer unique benefits. Here are our main arguments for why representation‑based steering is not attempting to bet against the scaling laws of LM performance:

• Representation‑based guardrailing methods are safer. System prompts always leak, which can expose dangerous information about model behaviors. By contrast, as we show in Sec. 5.3 and Appendices K, L, M, and N, representation‑based feature‑suppression methods are much more robust, especially against jailbreak attacks. Moreover, system prompts can be leaked, whereas representation‑based steering is much harder to reverse‑engineer.
• Personalized LMs. Although personalized history can be jammed into the prompt as prefix KV caches, this quickly incurs substantial overhead due to large memory requirements. Offloading the personalized prompt into static, small steering vectors not only saves memory and compute—reducing attention calculations over KV caches to a simple rank‑1 vector addition—but can also be more precise because of their trainable nature. As we show in the paper, training such steering vectors works even in a low‑resource data regime.

Thanks for your comments! We address them below. We provide additional results on LoRA and LoReFT and discuss our hyperparameter-tuning procedure in detail.

Q1: The experimental evaluation of concept suppression is somewhat limited. The paper presents concept suppression results only for rank-1 steering vectors trained with RePS and the language modeling objective, omitting evaluations for other intervention types such as LoRA and LoReFT, as well as alternative objectives like BiPO (e.g., in Table 2). Could the authors clarify why these results were not included?

A1: Great suggestions! These are mainly two reasons why we did not prioritize these results:

• First, steering performance is our primary focus. We excluded BiPO from further studies because its positive‑steering performance lags significantly behind (Table 1), making it impractical for other steering applications such as suppression.
• Second, the negative‑steering intervention in Eqn (7) is designed only for rank‑1 steering; its objective is far better motivated in that setting. For LoRA and LoReFT, negative steering during training remains largely under‑explored—previous work typically concentrates on zero‑shot adaptation of negative weights to test whether interventions can be composed algorithmically. To provide additional insight, we trained LoRA and LoReFT on 2B and 9B models, applied the interventions negatively using the training‑time steering factors (i.e., multiplying each rank by the same negative factor), and evaluated them in the same way as the rank‑1 steering reported in Table 2:

Our results suggest that RePS benefits LoReFT, whereas the LoRA outcomes are somewhat mixed. In particular, RePS interventions failed on the 2B LoRA models when applied negatively. We believe this is largely because LoRA is highly sensitive to inference‑time coefficient adjustments, as shown in previous work [1]—LoRA simply breaks when large negative factors are applied. Consequently, we recommend that future work design better negative interventions for high‑rank methods. We will include these additional results and discussions in the Appendix.

Q2: It is mentioned in lines 234 and 235, that finding the optimal parameters (including the optimal steering layers) required extensive grid searches (up to 168 runs per objective–method for gemma-3 models). This limits the practical adoption and scalability of the proposed approach.

A2: Thanks for raising this issue! We want to emphasize that this is actually a feature of our paper, not a bug of RePS, as discussed extensively in Appendix D. The extensive hyperparameter tuning across all conditions—using the same grid or the same budget for each pair of intervention type and training objective—ensures that we study the true generalization of our training objective.

In fact, we encourage researchers in this field to adopt our comprehensive hyperparameter‑tuning procedures to give all methods a fair chance. With only a small grid search, one method could simply be lucky enough to outperform another, especially since steering vectors are usually trained in a low‑data regime.

Moreover, RePS is relatively stable across our hyperparameter runs; we provide a detailed performance‑variance analysis in Appendix D (e.g., see Fig. 16). This means it is not particularly costly to select a good hyperparameter for a new dataset.

[1] Zhang et. al., 2023 Composing Parameter-Efficient Modules with Arithmetic Operations https://arxiv.org/abs/2306.14870