AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders

Thanks for your comments! We address them point-by-point below. In our responses to other reviewers, we provide additional human evaluations on our LLM judges and updated, much higher quality SAE concepts.

Q1: Please define $n$ in section 4 notation.

A1: Thank you for the suggestion! In Section 4, we use $n$ to denote the number of tokens in a given sequence. We will clarify this in our next revision.

Q2: How do you construct the instruction pool related to a given genre (lines 151 and 152)?

A2: Due to space limitations, we provide details in Appendix I linked from section 3.1. For text-based instructions, we sample from Dolly-15K. For code-based instructions, we sample from a collection of Python-code puzzles formatted in an Alpaca-style (i.e., instructions paired with corresponding responses). For math-based instructions, we sample from GSM8K. With additional page allowance, we will ensure these details are included in the next revision.

Q3: The synthetic dataset constructed in Section 3.1 is only used for concept detection, not model steering, correct?

A3: Yes, that is correct—it is used exclusively for concept detection. For model steering, we use existing instructions and concepts.

Q4: The procedure for concept detection from gradient-based methods described in Appendix H doesn't really make sense. I am not sure how using max-pooling on token importance for a given concept can be used for concept detection.

A4: Thank you for raising this point! We agree that aggregating gradients at the sequence level might provide additional insights into gradient-based sensitivity methods. The main reason for applying max-pooling for IxG and IG is to maintain consistency with the setup used in other methods, ensuring a fair comparison. If resources permit, we will consider adding IxG and IG baselines that aggregate sequence-level gradients via sum or mean in the next revision.

Q5: For prompting, it's unclear why you needed different methods for concept detection and concept steering in the prompting case—it seems that both are somewhat the same task.

A5: Thank you for the question. For concept detection, we use a template-based prompt asking the model, “Do you think the concept is incorporated in the given sentence?” (see Appendix J.3 for the full prompt). For model steering, we use gpt-4o-mini (acting as our prompt engineer) to generate a steering prompt (e.g., “You must include terms related to planting Apple trees in your responses. Here are some examples...”), which is quite different from our concept detection prompt.

Q6: In line 285, you mentioned, "we sample 10 instructions. We generate up to 128 tokens for each instruction over 14 steering factors." What does it mean to steer over 14 steering factors?

A6: Thank you for raising the question. For model steering methods, we follow an existing paradigm in which we insert activation addition interventions into the model’s forward pass as follows:

$h_{Steer} = h_{Original} + \alpha \cdot w_{Steer}$

Here, $\alpha$ represents the steering magnitude, and $w_{Steer}$ is the learned steering vector for a given method. As suggested by previous work, $\alpha$ is treated as a hyperparameter, and different values (steering factors) must be tested to select the optimal one for final evaluation. In our work, we use 14 different settings for $\alpha$ across all methods, and for each setting, we sample 10 instructions to steer. Due to space limitations, the discussion on steering factors is included in Appendix K. With additional page allowance, we will include further details in the main text in our next revision.

Thanks for your comments! We address them point-by-point below. We supply preliminary human evaluations as well as results for a new prompting baseline.

Q1: Performing a proper human evaluation to ground AxBench.

A1: Great suggestion! We conducted a preliminary human evaluation by recruiting 5 participants, each of whom rated 30 steering generations from different methods (e.g., Prompt, ReFT‑r1, SAE) using the same scoring system as our LLM judges. We then computed two types of Pearson correlation coefficients to assess agreement:

• Human-Human Agreement: For each pair of human participants, we computed the Pearson correlation between their ratings, applied Fisher’s Z transformation, averaged them in the transformed space, and converted the result back to a Pearson correlation coefficient, yielding a score of 0.57.
• LLM-Human Agreement: We also computed the Pearson correlation between the LLM’s ratings and each human’s ratings, applied Fisher’s Z transformation, and averaged these values across the 5 participants, yielding a score of 0.58. These measures indicate that the LLM behaves much like just another human participant in terms of rating consistency. Given that steering is a highly subjective task about which individuals frequently disagree, these strike us as high correlation numbers overall. Moreover, prior work on similar annotation tasks [1][2] has reported correlations of at most 0.6, further validating our findings. We have uploaded our anonymized human annotation interface here.

Q2: Would be good to be clearer on the details of the prompting baseline in the main paper. It would be great to see how zero-shot prompting with a simple template works.

A2: This is a great suggestion! With additional page allowance, we will further clarify our prompting baseline in the main text. We also ran additional experiments with zero-shot prompting, using a template-based approach that directly asks the model to incorporate a concept in its response. Below are the results:

where * means statistically significant. As expected, zero-shot prompting performs worse than LLM-aided prompting. The gap between these two variants is surprisingly small, indicating that instruction-tuned models can be easily steered without sophisticated prompt engineering.

Q3: Would prefer to see more examples as in Appendix N.

A3: We will provide additional examples and improve their quality in our next revision.

Q4: Could you comment on concept discovery?

A4: Although SAEs lag far behind supervised methods in empirical results, we believe they serve as excellent “hypothesis generators” for concept discovery in an unsupervised manner. However, these hypotheses must be causally verified through intervention-driven experiments, as we did for model steering in AxBench. The concepts identified by SAEs can naturally become learning targets for supervised methods, which yield better subspaces. On the other hand, supervised methods can utilize any concept list—not just those derived from SAEs.

Q5: How would Gemma SAE features to affect your empirical results?

A5: Great point! We agree that the labels from the auto-interpretability pipeline contain noise. To validate our pipeline, we ran preliminary experiments using a set of high-quality SAE labels released recently [1] for both SAE and ReFT‐r1, and we observed similar performance patterns with the Gemma-2-2B model (i.e., ReFT-r1 is significantly better than SAE). Interestingly, SAE steering performance increased slightly, suggesting that better feature annotations yield improved steering results. Below are the results:

Additionally, SAE’s performance on concept detection increased from 0.75 to 0.85 for both layers. In summary, higher quality concept labels improve SAE’s performance without closing the gap to other supervised methods. We will incorporate these newer findings into our next revision. Note that the original and new results are based on around 20 concepts.

Thanks for your comments! We address them point-by-point below. We provide preliminary results by using a set of high-quality SAE labels released recently below.

Q1: One concern is that the features chosen are the labels the authors found for the evaluated SAEs on Neuronpedia. However, these SAEs were labeled with an (admittedly barely) outdated auto-interpretability method.

A1: Great point! We agree that the labels from the auto-interpretability pipeline contain noise. To validate our pipeline, we ran preliminary experiments using a set of high-quality SAE labels released recently [1] for both SAE and ReFT‐r1, and we observed similar performance patterns with the Gemma-2-2B model (i.e., ReFT-r1 is significantly better than SAE). Interestingly, SAE steering performance increased slightly, suggesting that better feature annotations yield improved steering results. Below are the results:

Additionally, SAE’s performance on concept detection increased from 0.75 to 0.85 for both layers. In summary, higher quality concept labels improve SAE’s performance without closing the gap to other supervised methods. We will incorporate these newer findings into our next revision. Note that the original and new results are based on around 20 concepts.

Q2: Furthermore, the authors may want to include discussion of how the choice of layer may affect the results.

A2: Yes, the choice of layer does seem to affect steering performance. For example, in larger 9B models, steering at earlier layers (e.g., L20 vs. L31) appears to be advantageous, which is not as evident for smaller 2B models. While later layers might be more effective for generating tokens, they involve fewer downstream layers, potentially reducing the expressivity of the steering vector.

Q3: Finally, the authors may want to elaborate on how feature sets should be generated for training supervised dictionaries without having to first train an SAE. Is there some unsupervised method to generate relevant features in an automatic manner?

A3: Great suggestion! Yes, this would be an interesting extension study. For supervised dictionary learning methods (SDLs), obtaining labels from SAEs is not required; we use SAE labels solely to maximize SAE performance. One promising direction is to compile expert-level concept lists (e.g., reasoning-related concepts such as "backtracking") and then use SDLs to identify effective steering vectors for both concept detection and model steering.

Thanks for your comments! We address them below. We articulate our goal and how AxBench differs from IFEval-like benchmarks, and supply preliminary human evaluations.

Q1: The Section 2 related work does not mention any concurrent benchmarks to evaluate model instruction following (e.g., IFEval). The paper dives quickly into AxBench and fails to connect it to other benchmarks.

A1: We want to clarify the purpose of creating AxBench. First, AxBench is not an instruction following benchmark for LLMs. IFEval is intended to compare the instruction-following abilities of LLMs using a small set of easily verifiable metrics. AxBench is an open-vocabulary evaluation benchmark for concept-based steering and detection, which compares the efficacy of different steering/detection methods on a single LLM, and is not verifiable with simple rules. Our concepts can be highly abstract and are described using natural language descriptions (i.e., this is not comparable to asking LLMs to respond with X tokens or include keywords Y, as in IFEval). Unlike IFEval, a rule-based evaluation metric is impossible for such open-ended concepts. Moreover, our concepts are not artificially created, and are grounded with interpretability methods. They emerge in LLM's representations and are found in an unsupervised manner with SAE [1] [2]. Another benefit of doing so is having access to thousands of such concepts, so that we can evaluate steering methods at scale. We will clarify the distinction between steering and instruction following in our next revision.

Q2: Performing a proper human evaluation to ground AxBench.

A2: Great suggestion! We conducted a preliminary human evaluation by recruiting 5 participants, each of whom rated 30 steering generations from different methods (e.g., Prompt, ReFT‑r1, SAE) using the same scoring system as our LLM judges. We then computed two types of Pearson correlation coefficients to assess agreement:

• Human-Human Agreement: For each pair of human participants, we computed the Pearson correlation between their ratings, applied Fisher’s Z transformation, averaged them in the transformed space, and converted the result back to a Pearson correlation coefficient, yielding a score of 0.57.
• LLM-Human Agreement: We also computed the Pearson correlation between the LLM’s ratings and each human’s ratings, applied Fisher’s Z transformation, and averaged these values across the 5 participants, yielding a score of 0.58. These measures indicate that the LLM behaves much like just another human participant in terms of rating consistency. Given that steering is a highly subjective task about which individuals frequently disagree, these strike us as high correlation numbers overall. Moreover, prior work on similar annotation tasks [3][4] has reported correlations of at most 0.6, further validating our findings. We have uploaded our anonymized human annotation interface here.

Q3: Some Appendix could be probably removed. The overall paper is not easy to read. The legend of Figure 2 is bit confusing.

A3: We'll clarify our notations early and update Figure 2 and the Appendix in our next revision.

Q4: The paper seems to not be able to pick which contribution to highlight between AxBench and ReFT‐r1.

A4: Our goal is to evaluate whether existing representation-based steering methods are competitive in terms of concept detection and model steering compared to finetuning or prompting methods. Most of our approaches produce rank-1 steering vectors, which is the simplest steering setting designed to minimize inference-time overhead while rivaling prompting in practical scenarios.

To accomplish this goal, our contributions are: (1) AxBench, which, to the best of our knowledge, is the first benchmark for evaluating representation-based steering methods at scale; and (2) ReFT‐r1, a representation-based method derived from ReFT [5] that rivals finetuning or prompting methods. As shown in our Figure 1, without ReFT‐r1, representation-based methods significantly lag behind, making it nearly hopeless for them to catch up. We believe that our two contributions work organically together to provide a solid foundation for evaluating methods and paving the way forward for representation-based approaches.

Our work shows that while SAEs were once seen as the only scalable method for training steering vectors, ReFT‐r1 disproves this for concept detection and model steering. AxBench is essential to evaluate interpretability methods as they scale against finetuning and prompting. Although ReFT‐r1 still lags behind, we think it provides important insights into how to train better steering vectors that can be useful in practice.