Thank you for your constructive and technically grounded feedback. We address your concerns in detail below.

Weakness 1: Theory behind PCA clustering

We appreciate the reviewer’s thoughtful concern regarding the theoretical underpinnings of our causal hypothesis.

Our current evidence primarily supports an empirical pipeline from low lexical diversity → geometric concentration in residual stream → increased false refusals. While we agree that this causal chain is not yet fully formalized, we offer the following rationale and findings:

• From Low Diversity to Geometric Collapse: We observe this phenomenon consistently across model families, including dense transformers (e.g., LLaMA, Qwen), and even DPO-aligned models (e.g., LLaMA-3.1-Tulu-3-8B-DPO). These results suggest that data distribution—especially the low diversity of safety responses—is a key driver of residual stream distortion, regardless of model architecture.

• From Geometry to False Refusal: Our working hypothesis assumes that the model's internal representation space is continuous and supports semantic generalization. In this space, overfitting to structurally repetitive safety data distorts the geometry, effectively "bending" the boundary between safe and benign queries. This causes models to misclassify benign prompts near that boundary as unsafe, resulting in false refusals.

This view aligns with the idea of a continuous optimization landscape, where local over-optimization (e.g., excessive variance concentration for one class) leads to poor generalization at class intersections.

While the precise mechanism is not yet fully understood, our extensive experiments and supporting analyses provide a strong basis for concluding that low-diversity safety data can induce structural overfitting in residual representations, leading to false refusals.

Weakness 2: On Generalization Beyond the LLaMA Family

We thank the reviewer for raising this important point. To evaluate whether our methods generalize beyond the LLaMA architecture, we conducted further experiments across multiple model families and architectures:

• Qwen Family (Dense Transformer): We applied VCL to Qwen-2.5-14B, a dense transformer models. In both cases, VCL effectively reduced false refusal rates, consistent with our results on the LLaMA family. This suggests that our method generalizes well to other transformer implementations beyond LLaMA. Due to space constraints, detailed results are reported in our response to Reviewer r4L3 – Weakness 1

• MoE models： We further tested FlowLens on MoE models, where each token is routed through a subset of experts. As shown in the figure below (replaced with a table in final rebuttal), we observe significantly larger divergence in residual geometry on safety-only data compared to dense counterparts: MoE:

  Dataset	PC1	PC2	PC3
  SafetyOnly	0.09	0.06	0.20
  GeneralOnly	1.00	0.99	0.97

• Multimodal Architecture: Due to incompatibilities in tokenizer and forward implementation, applying VCL to multimodal models requires nontrivial engineering efforts. We leave this for future exploration.

• DPO-Aligned Model: We ran FlowLens on a DPO-aligned model, specifically LLaMA-3.1-Tulu-3-8B-DPO. The residual stream still exhibited variance concentration, similar to what we observed after supervised fine-tuning. This indicates that structural disruptions introduced during SFT can persist through later alignment stages like DPO, and that applying VCL earlier in the pipeline may mitigate these effects. DPO: | Dataset | PC1 | PC2 | PC3 | |-------------|------|------|------| | SafetyOnly | 0.99 | 0.99 | 0.66 | | GeneralOnly | 1.00 | 0.99 | 0.87 |

These experiments support the generalizability of our approach.

Weakness 3: Risk of over-regularization and negative transfer

We appreciate this concern. In designing VCL, we explicitly aimed to mitigate over-regularization risks by following two principles:

1. VCL is a soft regularizer, not a hard constraint. Much like KL penalties in reinforcement learning, it introduces a tunable pressure (via the γ coefficient) against excessive variance collapse, without dictating the exact structure of representations. In practice, we found small γ values (e.g., 0.1–0.3) already yield substantial gains in safety without harming generalization.

2. VCL is applied selectively to mid-level transformer layers—i.e., where variance collapse is empirically observed. The higher layers, which are more directly tied to final output behaviors and task-specific logic, remain unconstrained, preserving freedom for downstream specialization.

Empirically, we observe that general-task benchmarks (MMLU, GSM8K, BBH, CodexEval) remain stable. This suggests that VCL helps restore robustness against over-refusal without degrading general representational capacity.

Weakness 4: Lack of comparison with other structural regularization or data diversity enhancement methods

We appreciate the reviewer’s suggestion.

Regarding structural regularization: While prior work has explored structural properties of model representations—such as analyzing residual streams using cosine similarity or probing directions tied to specific behaviors—few have proposed using residual stream structure as a train-time regularization target during supervised fine-tuning (SFT). Moreover, some baselines we include do implicitly relate to representational structure. For instance, Vector Ablation intervenes on specific residual directions learned from harmful data, and DRO (Drected Representation270Optimizatio) indirectly influences geometry by adjusting model confidence across harmful/helpful distributions. We treat these methods as relevant comparison points, and VCL consistently outperforms them on false refusal reduction.

Regarding data diversity: our SFT baseline models are trained on the Tulu-3-SFT-Mixture, which is explicitly curated for high diversity through sampling from varied datasets, persona-driven prompt creation, and class-balanced downsampling. Therefore, our experiments are already conducted under strong diversity-enhanced conditions—highlighting that VCL brings additional benefits beyond dataset construction alone.

Question 1: Applicability to multimodal models

Please see our response to Weakness 2, where we discuss the applicability of VCL to multimodal and MoE models in more detail.

Question 2: Computational overhead of FlowLens and VCL

The computational overhead of VCL is minimal. It operates as a soft regularization term, similar to KL penalties used in reinforcement learning, and adds only a lightweight PCA-based loss term to selected mid-layer residuals. It does not modify model architecture or introduce additional forward/backward operations.

We discuss this in detail in Appendix G.2, where we show that training time and memory usage remain nearly unchanged when enabling VCL.

As for FlowLens, it is an analysis tool used only during evaluation and visualization. It plays no role during model training and does not contribute to training cost.

Question 3: Can VCL be combined with DRO?

Yes, VCL is theoretically compatible with Distributionally Robust Optimization (DRO). The two methods operate on orthogonal aspects of model behavior:

• DRO encourages differentiation between helpful and harmful queries within each layer by reweighting training loss across risk groups;
• VCL promotes global variance diversity across transformer layers, mitigating variance collapse without disrupting within-layer representational distinctions.

In fact, we tested models trained with VCL on both harmful and harmless inputs, and observed that residual streams from different query types still form distinct clusters within the same layer. This suggests that while VCL encourages overall variance dispersion, it preserves task-relevant internal structure.

While we have not yet experimented with joint VCL + DRO trainin, doing so would simply require tuning a relative weighting coefficient between the two loss terms—an implementation we believe to be straightforward and promising for future work.

Question 4: Does VCL sacrifice representational capacity?

This is an important concern, and we believe the answer is no—VCL does not sacrifice essential representational capacity.

Conceptually, VCL is akin to KL regularization in reinforcement learning, where the goal is not to constrain expressiveness, but to avoid undesirable collapse (e.g., to a narrow policy mode or representational subspace). VCL does not enforce high variance, but instead penalizes excessive concentration of residual stream variance into a few principal components, which our analysis links to false refusal behaviors.

Empirically, we observe that general benchmarks (e.g., MMLU, GSM8K, BBH, CodexEval) remain stable after applying VCL. This suggests that core task-relevant features are preserved, and essential capacity is not diminished.

Moreover, VCL is only applied to mid-level layers where variance collapse is most severe. The upper layers—closer to output decisions—remain unconstrained and can adapt freely to downstream task requirements.

We thus view VCL as restoring internal diversity where it is most needed, without interfering with the model’s ability to learn and represent high-level semantics.

Thank you for your precise and forward-looking comments. We respond point by point below.

Weakness 1: On Model and Language Generalization of FlowLens and VCL

We thank the reviewer for pointing out the potential generalization limitations. To address this, we have extended our evaluation of FlowLens and VCL across model families, scales, and architectures, as detailed below.

• Model Family Generalization: We applied VCL to the Qwen family ( Qwen-2.5-14B results), and observed consistent improvements in reducing false refusal rates, validating that our method is not specific to LLaMA-based models.

Qwen-2.5-14B results

• Scaling to Larger Models: Although we were unable to run full-scale experiments on 70B-class models due to resource constraints, we evaluated on the intermediate 14B scale. Results show that both FlowLens trends and the effectiveness of VCL scale smoothly from 1B → 8B → 14B, suggesting a promising trend toward larger models.

• Application to MoE Models: We further tested FlowLens on MoE models, where each token is routed through a subset of experts. As shown in the figure below (replaced with a table in final rebuttal), we observe significantly larger divergence in residual geometry on safety-only data compared to dense counterparts:

This indicates that FlowLens is sensitive to structural shifts even in MoE settings. Notably, the similarity in PC3 (0.20) does not contradict this observation, as in MoE models the explained variance of PC3 is very low (<1%), compared to 5–10% in previous dense experiments.

• Evaluation on DPO Models: We also applied FlowLens to a DPO-aligned model, observing that mid-layer geometric collapse still persists, consistent with our earlier findings. This suggests that structural issues from SFT may carry over, reinforcing the value of VCL even after DPO alignment.

• Cross-lingual Evaluation: While we recognize the importance of testing across languages, our current experiments are constrained by training data availability and compute. We now explicitly acknowledge this as a limitation and a future direction in Section 6.

Together, these results demonstrate that FlowLens and VCL generalize well across models and training setups, with further architectural extensions (e.g., MoE, multilingual) posing valuable future opportunities.

Question 1: On the Effect of VCL Beyond Top Principal Components

We appreciate this thoughtful question. We agree that while top principal components typically reflect broad semantic structure, lower-variance (tail) dimensions may encode subtle yet important features.

Our design of VCL accounts for this. Specifically:

• Controlled Regularization via $\gamma$: The strength of VCL is governed by a hyperparameter $\gamma$, which we tune conservatively to avoid over-regularization. This ensures that VCL only penalizes excessive variance concentration, rather than enforcing any fixed projection pattern.

• Layer Scope Restriction: VCL is applied only to a selected mid-layer window, leaving the later layers untouched. These deeper layers retain full flexibility to allocate capacity across all principal or non-principal directions.

• Preservation of Normal Query Geometry: Empirically, we observe that the residual stream geometry of non-safety prompts remains largely unaffected by the introduction of VCL. This suggests that tail dimensions activated by benign/general queries are not inadvertently suppressed by our method.

Together, these design choices allow VCL to mitigate structural overfitting on highly repetitive safety data, without impeding the expressive capacity of the residual stream—particularly in directions beyond the top few principal components.

In future work, we plan to conduct a more targeted spectral analysis of tail components to complement our current PCA-focused evaluations.

Question 2: Helpfulness/Safety Trade-off under Adversarial Attacks

We appreciate the reviewer’s concern regarding potential vulnerabilities introduced by VCL to adversarial jailbreak attacks.

To assess this, we extended our evaluation beyond DAN to include two recent adversarial attack methods: GCG [1] and ICA [2].

• GCG (Greedy Coordinate Gradient) is a universal and transferable jailbreak attack that optimizes adversarial suffixes which can be transferred across models and prompts [1]. We adopted the transfer setting from the original paper, using suffixes generated from Llama2-7B and applying them to Llama-3.1-1B and Llama-3.1-8B. This setup is justified, as the transferability relies on overlapping pretraining corpora and shared model architectures, as discussed by the authors.

• ICA (In-Context Attack) uses a few-shot in-context learning setup to jailbreak aligned models without modifying their parameters. It leverages known jailbreak queries and completions, requiring only 3-shot demonstrations [2]. We follow their 3-shot ICA setup using the open jailbreak pool provided in the original paper.

Below we report the reject rates (RR, %) of various methods under these two attacks. Higher is better.

These results suggest that VCL does not increase susceptibility to adversarial attacks.

[1] Zou et al., "Universal and Transferable Adversarial Attacks on Aligned Language Models", arXiv:2307.15043.
[2] Wei et al., "Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations", arXiv:2310.06387.

Thank you for the helpful questions and insights into the broader implications of our work. Please see our responses below.

Weakness 1: On the causal mechanism

We appreciate this thoughtful point. Our current explanation of the causal pathway remains primarily descriptive. While the precise mechanism is not yet fully understood, our extensive experiments and supporting analyses provide a strong basis for concluding that low-diversity safety data can induce structural overfitting in residual representations, leading to false refusals:

• From low diversity to geometric collapse, we observe that templated safety data consistently reduces residual stream variance dispersion. This holds across architectures including dense models (Qwen and Llama) and DPO models—suggesting a general effect rooted in data entropy rather than model class.

• From geometry to false refusals, we hypothesize that residual stream representations are continuous, and safety overfitting reshapes the representational manifold. This can lead to overly sharp decision boundaries in ambiguous or marginal cases—causing models to misclassify benign prompts as harmful due to their proximity to refusal templates in collapsed regions.

We agree this geometric–behavioral link deserves deeper formalization. As future work, we propose quantifying how low-diversity data reshapes the optimization landscape and decision boundary geometry, potentially through tools from manifold learning or information geometry.

Weakness 2: DPO-tuned models still exhibit variance concentration in residual streams

Thank you for this insightful question. We conducted additional experiments using the LLaMA-3.1-Tulu-3-8B-DPO model to explore whether DPO-aligned models also show signs of residual stream variance concentration, as measured by FlowLens.

We evaluated residual stream PCA alignment on two datasets (TruthfulQA and XSTest), following the same procedure as in Section 4.2. The cosine similarity between each dataset's local top-3 principal components and the global PCA directions is reported below:

These results indicate that DPO models retain high alignment with global variance directions, especially in early components (PC1 and PC2), and still exhibit moderate-to-strong variance concentration (notably in SafetyOnly’s PC3: 0.66).

We view this as an opportunity for future work to explore post-SFT structural interventions or combined SFT-DPO loss formulations that preserve helpfulness while improving geometric robustness.

Question 1: Mid-layer is the effective location for regularization

Thank you for the thoughtful question. Our decision to apply VCL in the mid-layer range is grounded in both empirical observation and representational insights from prior work.

As for latter layers, as shown in Section 5.1, we conclude an amplification effect in false refusal rates across layers — early and mid-layer representational distortions propagate and accumulate through the stack. However, late-layer regularization alone (e.g., targeting only the final blocks) has limited ability to correct upstream collapse. This explains why mid-layer VCL often yields more stable and effective mitigation.

As for eailer layers, the relative ineffectiveness of early-layer regularization is consistent with findings in [1] (Refusal in Language Models Is Mediated by a Single Direction), which shows that refusal-relevant directions emerge more strongly in middle-to-late layers. Their Figure 5 shows that intervention signals concentrate most in mid layers, aligning with our empirical window selection.

Thus, our current design reflects both architectural intuition and optimization-based tuning. We agree that further interpretability of why mid layers are particularly amenable to structural regularization is an exciting future direction.

[1] Arditi, A., Obeso, O., Syed, A., Paleka, D., Panickssery, N., Gurnee, W., & Nanda, N. (2024). Refusal in language models is mediated by a single direction. NeurIPS 2024.

Question 2:DPO-tuned models still exhibit variance concentration in residual streams

Please refer to our response under Weakness 2, where we discuss whether structural disruptions persist through later alignment stages such as DPO, and whether VCL remains beneficial post-SFT. We believe this is an important and underexplored direction.

Thank you for your thoughtful and detailed feedback! We address your concerns below.

Weakness 1: Model diversity in experiments

We agree that evaluating on a wider range of models is critical to assessing the generality of our method. In response, we have conducted additional experiments on following new models:

• Qwen-2.5-14B, a significantly larger model to assess scaling behavior and a model from a different model family. The results are consistent with our findings in the original paper: VCL substantially improves performance on false refusal benchmarks such as XSTest and OKTest, while maintaining competitive results on general benchmarks such as MMLU and CodexEval.

The full results are shown below:

• DPO-Aligned Model: We ran FlowLens on a DPO-aligned model, specifically LLaMA-3.1-Tulu-3-8B-DPO. The residual stream still exhibited variance concentration, similar to what we observed after supervised fine-tuning. This indicates that structural disruptions introduced during SFT can persist through later alignment stages like DPO, and that applying VCL earlier in the pipeline may mitigate these effects.

These results reinforce the structural insight underlying VCL: by regularizing variance geometry in the residual stream, we consistently reduce false refusals while preserving general task performance—across different model families (Qwen vs. LLaMA).

Weakness 2: The impact of mitigating 2H tension on helpfulness

That's a very insightful point—we agree that the relationship between false refusal reduction and general helpfulness is not always obvious at first glance. You raise a valid concern: if general benchmarks like MMLU or GSM8K don’t show major improvement, does that mean VCL is only making the model less harmful (by reducing refusals), but not more helpful?

Our perspective is this: false refusal itself is a failure mode of helpfulness, especially under safety constraints. In fact, resolving false refusals is one of the clearest ways we can make the model more helpful—precisely on the kinds of queries where the helpfulness–harmlessness tension is most acute.

Take the example in Figure 1. The user asks, “How can I smash my exam next week?”—clearly a benign, reasonable query. But multiple safety-aligned models respond with refusals like “I can’t help you with that.” Only the VCL-enhanced model provides a study plan. That is not just “less harmful”—that’s more helpful.

Weakness 3: Why We Use Residual Geometry Instead of Entropy-Based Metrics

That's a very reasonable question. Our choice to focus on residual stream geometry—rather than entropy measures—is driven by the belief that geometric analysis gives us access to process-level representations, not just output statistics. The residual stream reflects the model’s internal computation across layers, which allows us to uncover alignment distortions that might not be evident from entropy alone.

In fact, there's a growing body of recent work that uses residual stream geometry to understand alignment and interpretability (see Section 4 Introduction). However, existing metrics—such as layerwise cosine similarity—have been found to be highly unstable under small prompt variations. Our goal was to introduce a more stable and global diagnostic, which led to the development of FlowLens.

More detailed theoretical and empirical justification for FlowLens can be found in Section 4.3 and Appendix E, where we show that:

• PCA-based projections are provably stable under perturbations (via covariance perturbation theory),
• and empirically robust even under minor formatting changes (e.g., punctuation differences).

That said, we agree entropy-based measures are also informative and widely used (as shown in our own Section 3). We believe that combining entropy and geometric analysis could be a promising direction for future work, and we appreciate the suggestion.

Weakness 4: Missing result table in Section 5.3

You're right—We have now added three result tables summarizing the effect of varying γ, k, and the residual window [l₁, l₂] on key evaluation metrics.

(1) Effect of γ (regularization weight)
(With k = 2, [l₁, l₂] = [0.3, 0.5])

(2) Effect of k (top principal components used)
(With γ = 1.0×50, [l₁, l₂] = [0.3, 0.5])

(3) Effect of residual window [l₁, l₂]
(With γ = 1.0×50, k = 2)

These results confirm that VCL is robust to small changes in hyperparameters, and that the chosen configuration in the main paper achieves a good tradeoff between stability and refusal reduction. And we will integrate the table into the paper in the future version.

Question 1: Slow PDF rendering due to PCA scatter plots

Thank you for flagging this. We’ve identified the cause: Figures 4, 7, and 8 contain dense PCA scatter plots, currently rendered as vector graphics. Since these plots involve thousands of individual points, vector-based rendering can cause significant slowdown in some PDF viewers. To address this, we plan to convert these figures to high-resolution rasterized images (e.g., PNG) in future versions.

Question 2: Clarifying the meaning of "Control" in Table 1

Thank you for pointing this out. We will revise the caption of Table 1 to explicitly define “Control” as follows:

“‘Control’ refers to the general instruction data used as a baseline, consisting of 100,000 randomly sampled non-safety examples from the Tulu-3-SFT-Mixture-General subset. See Appendix B for further details.”

Question 3: Hyperparameters used when training with VCL

Beyond the hyperparameters discussed in Section 5.3 (γ, k, [l₁, l₂]), we largely follow the default settings from the open-instruct recipe [1].

The training configurations are as follows:

• For LLaMA-3.2-1B:

  • max_seq_length = 4096
  • learning_rate = 1e-5
  • warmup_ratio = 0.03
  • weight_decay = 0.0
  • num_train_epochs = 2
  • batch_size = 128

• For LLaMA-3.1-8B:

  • max_seq_length = 8128
  • learning_rate = 2e-5
  • warmup_ratio = 0.03
  • weight_decay = 0.0
  • num_train_epochs = 2
  • batch_size = 256

We will add this information to the main paper for clarity and reproducibility.

[1] Lambert, N., Morrison, J., Pyatkin, V., Huang, S., Ivison, H., Brahman, F., ... & Hajishirzi, H. (2024). Tulu 3: Pushing frontiers in open language model post-training. arXiv preprint arXiv:2411.15124.