Steering the CensorShip: Uncovering Representation Vectors for LLM "Thought'' Control

W1: No comparison with Lee et al. (2025) as baseline

We refer the reviewer to our response to Reviewer LdFB’s W1, which provides additional results comparing our method and Lee et al. (2025).

W2: “What are some comparisons to using another model like WildGuard for judging the prompts?"

We have researched alternative models and chose WildGuard over others for several reasons—It is open-source, shows performance on par with GPT-4, and can detect both refusal and harmfulness. In contrast, other models have only been trained and tested for detecting harmfulness or safety.

Q1: “...how different sampling strategies might affect the performance of this model”

We’re not sure if the reviewer referred to the sampling strategy we used in evaluation or estimating refusal probability for vector extraction, regarding which specific performance.

Q2: “...scaling a lambda on harmless vs helpfulness (alpaca, XSTest_{safe}) for a model”

Please see to our response given to Reviewer hXbe’s second question (Q2), where we present the result for harmfulness and overall quality of generated outputs with varying λ values.

Thank you for Q1 plot!

For Q2, you probably have a table with at least three columns: lambda, refusal rate of XStest-safe, and refusal rate of XStest-unsafe. What I am looking for something like is a scatter plot of XStest safe on the x-axis and XStest unsafe on the y-axis over these two columns. This will indicate that the vectors can be utilized with some default value for a given task, or if a classifier is required for determining what the lambda value should be.

W1: Heuristic nature of string matching sets limits applicability to more general notions of refusal – results with a classifier-based approach would strengthen results here

We have considered using a classifier like WildGuard. However, this introduces computational overhead and perhaps the classifier’s own bias to the resulting steering vectors. We chose to implement string matching for its efficiency, but it could be replaced within our method with any function measuring censoring signals. We also show how our method can be adapted to distilled DeepSeek-R1 models, which exhibit a different type of censoring behavior based on thought suppression.

W2: Practicality–settings where the desired response type (refusal or compliance) is unknown

Our method only targets the detection and control of model outputs. For this type of setting, it would require using another method to detect the input given to the model before applying our steering vector. A similar setup has been studied by Lee et al. (2025) [1].

Q1: Cases where thought suppression occurs, but compliant responses still arise

We evaluated these cases with WildGuard and found only ~3% classified as non-refusal. However, based on manual inspection, these responses are still censored—they either give answers that are indirectly related or more aligned with the Chinese government's values.

Q2: Qualitative/quantitative evaluation of output helpfulness on benign queries when $λ=-1$

We believe that helpfulness may be subject to a user's background and requires further human assessment. Given the time limit, we use JudgeLM [2] to evaluate the overall output quality. We prompt JudgeLM 7B with the task instruction and a pair of responses—one without steering (baseline) and one after steering. Each response is rated on a scale of 1 to 10 based on helpfulness, relevance, accuracy, and level of detail. We compute a score ratio based on the ratings of five response pairs for each instruction. A ratio of 1 indicates the rating remains unchanged after steering, while a ratio < 1 means the steered response receives a lower rating than the baseline.

The tables below show the results of Qwen-2.5-7B on Alpaca and XSTest_safe, along with the average refusal and harmfulness probabilities measured by WildGuard. We find that steering with a coefficient (λ) between 0 and -1 has little impact on the ratings on average. However, increasing λ from 0 to 1 results in lower ratings for steered outputs compared to their baseline counterparts—this is expected from the refusal behavior induced by steering.

Alpaca (300 instructions):

XSTest_safe:

Edit: We added a visualized result here: https://ibb.co/Jw74XhKk

[1] Lee et al., Programming refusal with conditional activation steering, ICLR 2025

[2] Zhu et al., JudgeLM: Fine-tuned Large Language Models are Scalable Judges, ICLR 2025

W3: Conceptual difference between our method and previous methods for steering refusal/safety (Line 102)

Here, we summarize the key differences:

• Steering for different goals: Scalena et al. (2024) introduce a method for steering multiple concepts simultaneously. Lee et al. (2025) propose conditional steering that can enable/disable refusal based on the input context (e.g., hate speech). While most work focuses on steering binary directions, our method allows fine-grained control and detection of model behavior.
• Methods for finding steering vectors: Our method is based on the assumption that activations of different inputs encode varied degrees of censoring signals. We use "soft labels" instead of binary labeled prompts, as used in prior work (Scalena et al., 2024; Lee et al., 2025; He et al., 2025).

W1: Baselines for other steering methods

The table below shows results evaluated on the same four tasks described in Section 3.3. In the third column, we assess the effectiveness of steering vectors in representing model refusal by the Pearson correlation $r$ between the refusal probability of a prompt and its scalar projection on the vector at the last token position. Compared with Lee et al. (2025), the steering vectors found by our method exhibit a higher correlation with model refusal. We also evaluate whether the direction of projections can be used for refusal detection. As shown in the fourth column, our method achieves a higher accuracy, with >90% for both models.

We use the code from Lee et al. (2025)'s repository to extract vectors. The refusal probability is measured by WildGuard.

Next, we compare methods used for applying steering vectors. Lee et al. (2025) use activation addition [1], which does not work well for fine-grained, precise steering for the following reasons:

• Different inputs require using different coefficients: We apply steering with increasing coefficient magnitudes to find the point at which a model would refuse (or comply with) all prompts. As activation addition applies a uniform steering strength, we find that the model would start producing unnatural sentences for some prompts before reaching the desired responses for others. Our method addresses this by “neutralizing” activations before steering with activation addition (Equation 4). We provide quantitative results comparing both methods in our response to W2.
• Unknown coefficient range: With Lee et al. (2025)’s method, it’s difficult to identify a valid range for steering, whereas we offer a general approach for scaling vectors (Line 169) that allows steering within -1~1 and considers 0 as the “neutral” point where refusal probability ≈ 0.5.

W2: Generation quality, factuality of uncensored outputs

We evaluate the overall quality of uncensored outputs with JudgeLM 7B [2], prompting it to rate a pair of outputs produced using Lee et al. (2025) and our steering method on a scale of 1~10. We generate five outputs for each instruction and exclude ones with a refusal probability > 0.5. The table reports the result with coefficients λ that result in comparable refusal and harmfulness probabilities on average. The last column computes the average ratio of ratings between the pair of outputs. A ratio of 1 indicates outputs produced by both methods are rated similarly; a ratio > 1 suggests our method receives a higher rating. Our results suggest that while both methods can effectively reduce the refusal rate, our method produces outputs with a higher quality on average.

In terms of factuality, we notice that smaller size models, e.g., DeepSeek-R1 Distilled Qwen 1.5B, tend to exhibit more hallucination than larger models after bypassing censorship with steering. This may be due to their inherent capabilities. We appreciate the reviewer’s insightful comment and will continue to investigate this in our future work!

[1] Rimsky et al., Steering Llama 2 via Contrastive Activation Addition, ACL 2024

[2] Zhu et al., JudgeLM: Fine-tuned Large Language Models are Scalable Judges, ICLR 2025

"Lack of novelty"

We agree with the reviewer that steering vectors and one-dimensional projections are not a new contribution, and we do not claim this to be novel. Our work makes several novel technical contributions in this emerging area:

• While previous studies focus on refusal-based censorship in the safety context, we study censorship through a broader lens (Lines 40-42).
• We propose a method that enables precise control and fine-grained detection of LLM censorship. Previous work mostly focuses on steering binary directions (e.g., refusal or non-refusal) but not the levels of model behavior (e.g., varied degrees of refusal).
• Our work is the first to examine censorship in reasoning LLMs and present a novel countermeasure that targets “thought suppression”.