From Steering Vectors to Conceptors: Compositional Affine Activation Steering for LLMs

Weaknesses

Concerning interpretability: As we discussed in our response to reviewer Fmbn (Weakness 3), we rectified all unsupported claims regarding interpretability and instead point to conceptors as a potentially promising tool for interpretability, and an interesting subject for further mechanistic analysis - while making it clear that we provide no/limited insights into this in the present paper.

Concerning the complexity of steering functions: Our claim refers to functional expressiveness, not to raw parameter count, and we have made this more clear in our paper.

• No matter how many layers or heads we steer with additive vectors, each update is still a translation in its local $D$-dimensional space; it cannot rescale, rotate, or mix coordinates within that layer.
• Importantly, conceptors also demonstrate input-dependent steering because the impact of a conceptor depends on the current activation $h$, whereas additive vectors have a constant effect independent of the token context.
• Finally, the affine conceptor defined in Section 2.4 is strictly more expressive than vector steering as it combines additive steering with the linear multiplicative conceptor matrix (for aperture $\alpha \rightarrow \infty$, affine conceptor steering reduces to additive steering).

Concerning the definition of concepts: We appreciate the reviewer’s concern and agree that Section 2.1 should be made clearer. We are rewriting the theoretical framework section in our own words with less reliance on the work of Singh et al (2024). We further introduce an explicit, self-contained definition of concepts:

Let $\Sigma^{\ast}$ be the set of token sequences produced by the language model. A concept (or behavioral property) is a measurable function $c:\Sigma^\ast \longrightarrow [0,1]$, where $c(x)$ quantifies the degree to which sequence $x$ manifests the property. This function features gradedness (as opposed to binary concepts that are either present or not), where $c(x)=1$ means the property is fully present, $c(x)=0$ fully absent, and intermediate values reflect partial realization or uncertainty. We further allow for compositionality, meaning that multiple concepts/behaviors can be active simultaneously. For example, a reply can be truthful ($c_{\text{truth}}(x) \approx 1$) and friendly ($c_{\text{friend}}(x) \approx 1$). We define activeness of a conceptor (or behavioral property) $c$ in a text $x$ if $c(x) > \tau$ for a chosen threshold $\tau \in [0,1]$.

Concerning the definition of guardedness: We apologize for the incorrect reference to the Appendix and the missing information about the concept of guardedness. We have rewritten the section to refer less strongly to the work by Singh et al. (2024). As reviewer 1517 pointed out, $\phi$-assisted steering functions and guardedness do not play a primary role in our theoretical framework and we therefore moved these concepts and references to the related work section. We have made sure to include a concise and correct definition of guardedness in this section.

Concerning citations: We thank the reviewer for identifying these citation errors and apologize for the oversight. Should a problem remain with the reference to Chen et al (2023), we are happy to also resolve this, if the reviewer could kindly clarify the problem. We have now removed the duplicate entries in our bibliography.

Question 1

This is correct - classically, conceptors assume states to be zero-mean. As the reviewer points out, this does not hold for activations in LLMs (see our response to reviewer 1517's Q1 above). The extreme example nicely points out the shortcoming of linear conceptors. This is precisely why we derive affine conceptors, and we have made this motivation more clear in our paper. When $\mu_c$ is used, the conceptor $C$ will not pick up on the mean activation direction, as the conceptor will effectively be computed on the mean-centered activations and it will also be applied to mean-centered activations.

However, we wish to clarify that a linear conceptor can still pick up meaningful directions in the outlined (extreme) example. With well-chosen aperture $\alpha$, the conceptor is not "dominated by the mean shift while ignoring the actual axis of variance".

In the above example, if activations cluster around $v\pm\varepsilon\cdot v^\perp$, then the covariance matrix is given by covariance matrix $\hat \Sigma_c \propto \| v \|^2 v v^\top + \varepsilon^2 v_\perp v_\perp^\top$. In other words, $v$ will be an eigenvector of the with eigenvalue $s_v \approx \| v \|^2$, and $v^\perp$ will also be an eigenvector with eigenvalue $s_{v^\perp} \approx \varepsilon^2$. The aperture $\alpha$ of the conceptor $C(\tilde \Sigma_c)$ scales the eigenvalues of the conceptor as: $\hat s_i = \frac{s_{i}}{s_{i} + \alpha^{-2}}$ Taking $\|v\| = 10$ and $\varepsilon = 0.1$ (such that $\|v\| = 100 \ \varepsilon$) we can get $\hat s_{v} \approx 1$ and $\hat s_{v^\perp} \approx 1$ if we choose $\alpha = 1000$. We kindly refer to our comment to reviewer Xs77 (Weakness 1) for a description of our automatic aperture range mechanism which would lead us to (by definition) use apertures in the correct range in the above example.

Question 2

The reviewer correctly identifies a key challenge in this field—most recent activation steering methods introduce novel benchmarks with different evaluation protocols, models, and tasks, making direct comparison difficult. We initially focused on widely-cited methods (Todd et al. 2023, Rimsky et al. 2024) that have established evaluation frameworks.

To directly address this concern, we implemented a comparison with function vectors (FVs, Todd et al. 2024), which allows for evaluation on identical tasks and conditions. We have added this comparison to our paper. We evaluated conceptor steering on the same 6 tasks used in their work under identical zero-shot conditions. For each model, we selected the central layer and optimized only the aperture hyperparameter (keeping $\beta=1.0$), evaluating performance across 20 random seeds. Our results demonstrate consistent improvements over function vector steering across all tested models, including the 70B parameter Llama-2 model:

In addition to being less performant than conceptor steering, FVs have several properties that make them computationally prohibitive and less desirable than conceptor steering and the additive steering baseline we used, which is why we did not consider them as a viable baseline. First, FVs use more information. They not only require the token embedding at a given layer $x^l \in \mathbb{R}^d$ but also the output of the attention layer from all individual heads $a_{l,j} \in \mathbb{R}^d$ for $j \in \{1, ..., J\}$ across multiple layers $l$ for all the examples in the task. This amounts to >500,000 attention head activations compared to only ~100 token embeddings required by conceptor steering and additive steering. Second, FVs require more inference paths through Causal Mediation Analysis. For a single task, this requires >100,000 inference paths, whereas conceptor/additive steering only requires one per example.

We further evaluated conceptor steering using the Llama-2-7B model on the TruthfulQA benchmark to compare against the TruthFlow method by Wang et al. (2025), which was mentioned by the reviewer. Our results show that the performance of TruthFlow lies within the standard error of the performance of our conceptor steering method. We did not optimize the hyperparameters for the conceptor, we simply took the best hyperparameters from the coordinate-other-AIs multiple-choice task.

Unfortunately, Wang et al. (2025) did not report statistical validations of their results, which makes it difficult to assess if the difference in performance is significant. Moreover, we highlight that the flow matching method proposed by Wang et al. (2025) involves multiple gradient steps and is significantly more expensive than our conceptor steering method. If the reviewer deems this a helpful baseline comparison, then we can include fully optimized results in our final paper for a fair comparison against TruthFlow.

Question 3

It is correct that our current analysis does not definitively isolate whether the superior performance stems from the compositional operation itself (the AND operation) versus the general effectiveness of conceptor representations.

To address this concern, we measured the relative degradation in performance between the best performing layer when using combined methods versus optimal (oracle) performance. The results supports our original claims: conceptors demonstrate superior compositional efficiency on 2 out of 3 task pairs (English-French & Capitalize: -45.3% vs -52.0% degradation; Singular-Plural & Capitalize: -5.6% vs -13.3% degradation). The exception is English-French & Antonyms, which notably represents the task where compositional steering faces significant difficulties for both conceptor and activation approaches (as clearly shown in Figure 4, left).

Given these findings, we propose to refine our claim to be more precise about our current evidence and distinguish relative degradation and absolute performance.

Question 4

We have updated the theoretical framework section to be more consistent and easier to read to address this question.

Weakness 1

As we argue in the conclusion of our paper, conceptor-based steering function occupy a middle ground between simple steering vectors and fine-tuning of language models. Conceptors are more costly than steering vectors but they are also far more performant. Most importantly, conceptors are far more efficient than fine-tuning while also showing better performance (see Figure 3 and Appendix D.2).

The cost of computing steering vectors is actually comparable to that of conceptors. Given a set of concept-positive activation vectors arranged in a matrix $X \in \mathbb{R}^{N \times D}$, steering vectors are computed as the average activation whereas conceptors are calculated using the closed form solution in Eq. 9. The cost of calculating the steering function from $X$ is negligible in comparison to the forward passes of the LLM to get the activation vectors.

The reviewer also raises the problem of having an extra hyperparameter, the aperture $\alpha$, that requires tuning. Both steering vectors and conceptors require tuning the steering strength $\beta$.

For our experiments, we have developed an automatic aperture tuning mechanism that greatly simplifies the tuning of the aperture value based on the singular value spectrum of the covariance matrix $\tilde \Sigma_c$. We describe this mechanism below, and added a new section in the Appendix with full details on its implementation and results.

Automatic aperture tuning

Following Proposition 1, the conceptor matrix has a closed-form solution depending on the covariance matrix and the aperture: $C(\tilde \Sigma_c, \alpha) = \tilde \Sigma_c \left( \tilde \Sigma_c + \alpha^{-2} I \right)^{-1}$. If $\tilde \Sigma_c$ has singular value decomposition $\tilde \Sigma_c = \Sigma_{i=1}^D s_i u_i u_i^\top$, then the conceptor $C$ has the same eigenvectors but transformed eigenvalues, according to $\hat s_i = \frac{s_{i}}{s_{i} + \alpha^{-2}}$, where $s_{i}$ are the eigenvalues of $\tilde \Sigma_c$ and $\hat s_i \in [0,1)$ are the eigenvalues of $C$.

• We determine a lower and upper bound of aperture values based on the singular value spectrum of the conceptor. The upper and lower bound of values are the apertures at which the lowest and highest singular values reach a threshold of $1 - \varepsilon$ (where $\varepsilon$ is machine precision in float32).
• We estimate machine precision by $\epsilon \times |s| * 25 * \sqrt{max(s)}$. Additionally, we enforce an upper bound of machine precision to lay at 0.9999, since the maximum singular values of the covariance matrices explode in later layers, pushing our formula past the upper bound of singular values.
• Using $\hat s_i = \frac{s_{i}}{s_{i} + \alpha^{-2}}$, we get $min\_aperture^{-2}=max(s) \times \frac{1- \text{threshold}}{\text{threshold}}$ and $max\_aperture^{-2}=min(s) \times \frac{1- \text{threshold}}{\text{threshold}}$.
• We have observed empirically that this range of aperture values is the most expressive. In/Decreasing apertures beyond these bounds will not yield significantly different steering performance as the conceptors' singular value spectra do not change significantly beyond this range.
• We note that an existing conceptor $C(\alpha_1)$ with aperture $\alpha_1$ can be rescaled to another aperture value $\alpha_2$ with $C(\alpha_2) = C(\alpha_1) \left( C(\alpha_1) + (\frac{\alpha_1}{\alpha_2})^2 (I-C(\alpha_1)) \right)^{-1}$ as described in Jaeger (2014).
• In summary, this procedure is cheap and directly reduces the search space for optimal apertures in the hyperparameter sweep. We further refer to our response to reviewer Fmbn (Weakness 1), where we outline a method for reducing the search space for optimal layers where conceptor steering is applied.

Weakness 2

We agree with the reviewer that this is a natural expectation to have - since conceptors have $D^2$ parameters, they may be expected to require more training samples than vectors which have only $D$ parameters. However, this is actually not the case as it is confusing the number of entries in the conceptor matrix with the number of free (fitted) parameters:

• Although a conceptor matrix $C\in\mathbb{R}^{D\times D}$ contains $D^{2}$ entries, they are not independently fitted.  Equation (9) shows how $C$ is a deterministic shrinkage transform of the second-moment matrix $R$.
• $R$ is symmetric PSD, hence described by its $D$ eigenvalues and their directions; no extra free parameters are introduced. When the number of samples $n$ is fewer than the activation dimension $n<D$, $\operatorname{rank}(R)\le n$, so the conceptor is actually low-rank in most of our applications where $n << 1000 < D$.
• In terms of sample complexity, standard covariance-concentration bounds, e.g. $\lVert\hat R-R\rVert_{2}=O_p \bigl(\sqrt{D/n}\bigr)$, imply that $O(D)$ samples suffice for stable eigen-structure estimation— the same order as for a steering vector.
• The Tikhonov term $\alpha^{2}I$ shrinks all eigenvalues into $[0,1)$, capping variance and preventing over-fitting even when $n\ll D$. So, despite its larger raw size, a conceptor’s capacity is effectively comparable to that of an additive steering vector, and its required number of positive examples is governed by covariance estimation, not by $D^{2}$. This matches our experiment, where test accuracy saturates for both methods while the conceptor attains the higher plateau.

As a simple experiment to demonstrate this empirically, we take three of the function vector task with the largest dataset (antonyms, capitalize, english-french), and vary the number of "training examples" that we use to compute steering vectors and conceptors, and report the accuracy on un-seen examples. We chose the best hyperparameters for each task and steering method based on the results from our extensive grid search that produced Figure 2 in our paper. We note that every "sample" contains 10 in-context examples of the function, thus the effective number of training examples is ten times what is reported in the table below.

It can be seen that the accuracy saturates after ~20 samples for both conceptors and steering vectors. Conceptors reach their optimal accuracy with 60 samples and steering vectors with 80 samples. This shows that conceptors do not require more training samples for maximum performance, but they do reach significantly higher performance.

Weakness 3

We have demonstrated better-or-equal performance of conceptors on synthetic tasks using LLMs with up to 20B parameters. We also demonstrated compositionality of conceptors in multilingual settings by composing the English-French function with two other functions (see Figure 4).

We further report results on open-ended language generation in Section 3.4. We believe that these experiments demonstrate the viability of conceptor-based steering for long-form dialogue, as models are evaluated on open-ended generation rather than simpler tasks such as function vectors and multiple-choice answers. We kindly refer the reviewer to Appendix D.6 for more details on the experimental setup for open-ended language generation. During this rebuttal period, we have further extended the range of datasets, models and model sizes for a comprehensive comparison of conceptors against contrastive activation addition - we present our results in our response to reviewer Fmbn (Weakness 1). We also present extended results on open-ended steering on Llama-2-7b-chat (the model used by Rimsky et al. (2024)) and on multiple datasets, for which we kindly refer you to our response to reviewer 1517 (Weakness 5). We will update our limited (and incomplete) results in Figure 5 with the comprehensive results presented in this rebuttal. We agree with the reviewer that results for larger-scale LLMs (beyond 20B parameters) would be beneficial to demonstrate the scalability of our proposed method. During this rebuttal, we have therefore added results for the Llama-2-70b model on function vector tasks - where conceptors strongly outperform the much more expensive function vectors by Todd et al (2024), see our response to reviewer mWE6 (Question 2) and reviewer 1517 (Question 3). We further added results for Llama-2-7b and Llama-2-70b for complex behavior tasks, see our response to reviewer Fmbn (Weakness 1) and reviewer 1517 (Weakness 5).

I appreciate the author's response and appreciate their effort. However, I am still concerned about the instability of the model in real life due to hyperparameters. To strengthen the rebuttal, I recommend additions: (i) quantify the practical savings of the automatic‐aperture routine by reporting average GPU-hours and trial counts versus naïve grid search across several tasks, (ii) extend the evaluation to truly large models (e.g., Llama-2 13B/34B or Mixtral) and more realistic settings such as multi-turn dialogue or open-domain QA to demonstrate scalability. Given these concerns, I maintain the score.

Weakness 1

It is true that conceptors are more expensive to compute than vanilla steering vectors. However, as in our response to weakness 1 by reviewer Xs77, we argue that the cost of computing a conceptor is negligible in comparison to the number of forward passes through the model that other competing methods require - see Appendix D.2 for a comparison to LoRA finetuning, and our response to Question 2 by reviewer mWE6 for a comparison to function vectors by Todd et al (2024) and flow matching by Wang et al (2025).

Concerning the additional hyperparameter, we kindly refer the reviewer to our response to reviewer Xs77 (Weakness 1) where we address this concern. In short, we propose a general method to determine relevant ranges for the aperture parameter and explain how a single conceptor can be efficiently rescaled to different aperture values.

In addition to reducing the aperture parameter's sweep range, we also propose a conceptor-specific method for reducing the hyperparameter search space by restricting the number of layers that are sweeped. We have observed that across all our experiments, the conceptor steering method generally does not perform well in early layers. We found empirically that the best conceptor steering performance occurs in the 20% of layers after which the covariance matrix $\tilde \Sigma_c$ has reached maximal rank. The maximal rank is typically determined by the number of samples that are used to compute the covariance matrix. This further reduces the hyperparameter search space for conceptors greatly. We have added a new section in the Appendix with more details on this procedure.

For the scalability to larger LLMs, we point out that the GPT-NeoX model has 20B parameters and, as part of this rebuttal, we also supply additional results for larger models. We show results on the function vector task for the Llama-2-70B model where conceptors outperform function vectors (a much stronger baseline than simple activation vectors, by Todd et al. (2024)), see our response to reviewer mWE6 (Question 2) and include the full results table in our response to reviewer 1517 (Question 3).

Below, we also add results for all complex behavior tasks from Rimsky et al (2024) on the llama-2-7b and llama-2-70b models. We further update our results for Mamba-2.8B which we have improved with our new aperture tuning recipe and more principled hyperparameter search. We include results for baseline (no steering), contrastive activation steering, conceptor steering, and contrastive conceptor steering (by combining the conceptor $C_+$ for matching behaviors and the conceptor $C_-$ for non-matching behavior and combining them with $C=C_+ \wedge \neg C_-$). Due to time contraints of this rebuttal period, we were unable to finish our results for contrastive conceptors on Llama-2-70b.

These results show that conceptors consistently match or outperform activation steering across tasks, models, and model sizes. One anomaly is that conceptors are not good at steering for refusal (83% vs. 91% for activation steering), and we are currently looking into reasons for this. For convenience, we have added the standard deviation across tasks for this model size. More experiments are needed to investigate the steering performance of conceptors vs. activation vectors on larger models for complex behavioral steering.

state-spaces/mamba-2.8b-hf

meta-llama/Llama-2-7b-chat-hf

meta-llama/Llama-2-70b-chat-hf

Weakness 2

We agree with the reviewer that a more detailed ablation would be beneficial to analyze the isolated effect of different aperture values and residual steering application. Unfortunately, we have not had the time to implement this yet. If possible, we will consider including this in the final version of the paper - during the rebuttal we have focused on points that were more urgently raised by multiple reviewers (e.g., using larger models and comparing to more baselines from the literature).

Weakness 3

We enthusiastically agree with the reviewer that a more detailed mechanistic analysis of conceptor steering would be most interesting and important. We hoped to be able to provide this, but we have to accept that this is out of scope for the present paper. We apologize for the over-claiming in our abstract. We will rectify all unsupported claims and point to conceptors as a promising tool for interpretability, and an interesting subject for further mechanistic analysis and make it clear that we provide no/limited insights into this.

Question 1

This, too, is a very interesting question and we would be more than happy to explore this in the future. We have provided more results on the effect of increased number of training examples in our response to reviewer Xs77 (Weakness 2) which may be somewhat relevant to the question about robustness to noise. Further, more targeted, experiments on robustness to noise and distribution shifts are sadly out of scope for us now, and we leave this open as exciting follow-up work.

Question 2

We thank the reviewer for pointing out additional references that are relevant to our work.

We were not aware of the first paper by Song et al (2025) which is concurrent to our work, but we are happy to include it in our paper as an example of how steering can be applied for personalization in multi-user settings.

The second paper by Gu et al (2024) on LLMSteer seems only indirectly relevant to our work as they are doing attention steering rather than activation steering. Rather than steering the model towards specific behaviors or concepts, as done in our line of research, attention steering guides the attention weights of an LLM towards specific parts of the input text. The goal of attention steering seems to be improving generation quality and long-context understanding, whereas the goal of activation steering is to control the behavior of LLMs more generally. However, we would be happy to include references to attention steering if the reviewer believes this will improve our paper.

Weakness 1

Our core contribution is indeed to present the first theoretical framework for un-assisted activation steering functions. We have made this more clear in our manuscript and compressed the introduction of $\phi$-assisted steering functions which we agree is only marginally relevant given our aim (we moved the differentiation between our work and that of Singh et al. (2024) to the related work section).

Question 1 & Weakness 2

The claim that "activations already residing within the conceptor's region experience minimal change, whereas activations outside this region undergo more substantial shifts" stems directly from the mathematical formulation of conceptors and their eigenvalue structure.

From Proposition 1, the conceptor matrix has the closed-form solution where $\tilde \Sigma_c$ is the second-moment matrix or covariance with the singular value decomposition $\tilde \Sigma_c = \Sigma_{i=1}^D s_i u_i u_i^\top$. The conceptor $C$ has the same eigenvectors but transformed eigenvalues: $\hat s_i = \frac{s_{i}}{s_{i} + \alpha^{-2}}$ where $s_{i}$ are the singular values of $\tilde \Sigma_c$ and $\hat s_i \in [0,1)$ are the eigenvalues of $C$.

The eigenvectors $u_i$ are principal directions of the point cloud of training data points, and each $s_i$ equals the sample variance in that direction. Large variance in direction $u_i$ pushes the corresponding $\hat s_i$ close to 1, while small variance (or little support in the tranining data) gives $\hat s_i \approx 0$.

The conceptor acts as a regularized identity map that adaptively scales activations along different principal directions. For activations $\mathbf{h}$ that align with high-variance directions of the concept (large $s_{c,i}$), the corresponding eigenvalues $s_i$ approach 1, meaning $C\mathbf{h} \approx \mathbf{h}$ (minimal change). Conversely, for activations along low-variance directions (small $s_{c,i}$), the eigenvalues $s_i$ approach 0, causing substantial attenuation. This creates the ellipsoidal "conceptor region" where activations that lie within the high-variance subspace of the target concept experience preservation, while those outside undergo more significant transformation toward the concept's characteristic covariance structure.

It is clear that affine conceptor steering is more expressive than additive steering because as $\alpha \rightarrow \infty$, this reduces to additive steering. Our motivation to introduce affine conceptors stems from the assumption of linear conceptors that samples are centered around the origin (the above-mentioned characteristic ellipsoid is always centered around the origin). This assumption clearly does not hold for activations of LLMs - otherwise additive steering vectors would not work.

To validate that different functions lead to different mean activations, we performed statistical comparisons of mean activation patterns extracted at the colon token : during function vector tasks (see Appendix D.1). For each function type $f_i$ (e.g., antonyms, capitalization), we collected activation vectors $\mathbf{x}_{f_i}^{(j)} \in \mathbb{R}^d$ from $n=100$ examples. We performed one-sample t-tests against the null hypothesis that mean activations equal zero. We report the fraction of dimensions with $p < 0.05$ to quantify the extent to which function-specific representations deviate from the origin in activation space:

Euclidean distance (antonyms   | zero):              73.641
Euclidean distance (capitalize | zero):              74.112
Mean absolute t-statistic (antonyms   | zero):       11.992
Mean absolute t-statistic (capitalize | zero):       11.642
Fraction significant p < 0.05 (antonyms   | zero):   89.58%
Fraction significant p < 0.05 (capitalize | zero):   89.06%

Interestingly, the difference in mean activations between the two functions is far less significant (euclidean distance 7.76, mean absolute t-statistic 1.19, fraction of significantly different dimension is 18.9%). These results can naturally also be used as an argument for the limitations of steering based on mean activation vectors.

Question 2 & Weakness 3

We apologize for the mistake in Eq. 8, the expectation is taken over the set of activations for a given concept $c$, i.e., $x \in \{ \mathbf{H}(s) \text{ where } \phi(s)=c \}$. The steering function $f_c(\mathbf{H}(s))$ is specific to one concept $c$ and therefore the conceptor $C$ is concept-specific and there is no averaging or expectation over all concepts $c$.

We corrected this in our manuscript. We trust that the reviewer will now agree that there are no theoretical inconsistencies as conceptors are based only on individual concepts and there are no shared conceptual relationships. We further note that after fixing the typo in Eq. 8, our evaluation framework is entirely consistent with our theoretical framework.

Question 3 & Weakness 4

While we initially chose the Qwen series due to their increasing adoption and strong performance characteristics, we recognize that Llama models remain highly relevant for comparison purposes. We have added experiments on Llama-2-70B. We tested conceptor steering on the function steering experiment to complete our Figure 2 so that it can be visually compared with Figure 4 from Todd et al (2024).

These results demonstrate conceptor steering's effectiveness on Llama-2-70B across different layers. For a quantitative comparison showing that conceptors significantly outperform function vectors (a much stronger baseline than activation addition) on Llama-2-70B and other models, we kindly refer the reviewer to our response to Reviewer mWE6 (Question 2).

We further include results for complex behaviors (Section 3.4) for the Llama-2-7b and Llama-2-70b models in our response to reviewer Fmbn (Weakness 1).

Weakness 5

We apologize for the incompleteness of our final experiment section, we only included the full results in the supplementary materials (D.6). We re-ran all experiments at the last minute and did not get all experimental results in time for the paper submission. We apologize for the inconvenience and unconventional handling of this. The full results for the open-ended generation (Figure 5b) are included in the supplementary materials (D.6), and we paste the average performance for the best layer per steering method below. We note that, as described in Appendix D.6, we improved our scoring method to be a composite score of behavior and coherence, following [1]. The results confirms that conceptors outperform contrastive vector steering by Rimsky et al. (2024). We have updated the results in our paper accordingly, and it is now entirely self-contained and sufficient to support our claims.

Full results for Qwen-2.5-1.5B-Instruct:

Originally reported (incomplete) results for Qwen-2.5-1.5B-Instruct:

Open-ended steering performance on Llama-2-7b-chat

Furthermore, as requested, we evaluated the steering performance on Llama-2-7B-Chat. Results represent the best improvement (across all layers) of the BCS (behavior-coherence score) in the steered setting with respect to the unsteered setting, and we include the 95% confidence interval. We attribute the high breadth of the 95% CI to our limited grid search (3 aperture values, 4 beta values) and the limited size of the test set at each layer (20 examples). Nonetheless, we think these results are directionally representative and serve to address the reviewer's question, while we work on larger hyperparameter sweeps and testing on more samples.

Summary

We have addressed all weaknesses and questions by the reviewer and would like to thank them again for their keen eye and constructive criticism. We ensure that the clarify of our work has improved significantly through the points raised by the reviewer and we further hope that the reviewer finds it appropriate to raise their scores for the quality, significance and originality of our work.

References

• [1]: Samuel Soo, Chen Guang, Wesley Teng, Chandrasekaran Balaganesh, Tan Guoxian, and Yan Ming. Interpretable steering of large language models with feature guided activation additions, 2025. URL https://arxiv.org/abs/2501.09929.

We refer to the table in our response to reviewer mWE6 that show the performance of conceptor steering at a single layer and averaged across all tasks is $89.6 \pm 1.1$% whereas function vectors show $83.8 \pm 0.7$% at a single layer averaged across all tasks. We took the only numerical results reported by Todd et al (2024) in their Table 2 which chose layer 26 for llama-2-70b which is near-optimal (see their Figure 4). We chose layer 40 for conceptor steering because that seemed to be a good choice based on very preliminary experiments during this short rebuttal period. Thus, in a fair head-to-head comparison, conceptors outperform function vectors by 7% (relative).

Figure 2 in our paper compares conceptors against additive steering vectors, not against function vectors. As we argued in our rebuttal, function vectors are significantly more expensive (>1000x) to compute than conceptors (see our response to reviewer mWE6). Therefore we present additive activation vectors as a more relevant baseline for comparison. However, we have already contacted the authors of the function vectors paper for their data for their Figure 4 so that we can add their results into our paper, but so far we have not received a response.