Measuring and Guiding Monosemanticity

W1 (Harmonic vs arithmetic mean): We thank the reviewer for the thoughtful suggestion. We chose the arithmetic mean over the harmonic mean in Eq. (3) for the following reasons: Complementarity vs. Conjunction: The harmonic mean penalizes low values heavily, which is useful when both components must be high. However, we treat local and global disentanglement as complementary, not strictly conjunctive — strong performance in one should still be rewarded, even if the other is lower (e.g., due to concept spillover). Robustness: The harmonic mean is highly sensitive to small fluctuations, making it unstable when one component is near zero (even due to noise or classifier variance). This sensitivity often led to disproportionate score collapse in our experiments. Empirical Evidence: As shown in the table below, both means rank models similarly overall, but the harmonic mean introduces sharper drops and reduced score granularity (e.g., Toxcitity for vanilla SAE FMS@1 from 0.26 to 0.01), making the scores harder to interpret. Yet, given the data at hand, we agree that both averaging methods offer useful perspectives, and we now include this analysis in the appendix to clarify our choice.

W2 (Measuring Monosemanticity): We thank the reviewer for this important observation. Monosemanticity indeed requires not just that a feature aligns well with a concept, but that it does not encode others. A metric naturally can only measure and capture what is present in the probed data points—namely, the specific boundary-defining concepts found in a labeled dataset. As such, we believe that our approach already incorporates a practical approximation of the general monosemantic idea, as FMS incorporates global and local behavior. The level of granularity can be naturally determined by the selected number and diversity of labeled concepts available in the dataset. In addition, when computing feature accuracy (acc_0), we evaluate each feature across all available concepts in the dataset and select the one that separates best, i.e., true positives (correct activations for the target concept) and true negatives (non-activations for other concepts). In this way, each feature is automatically compared against all other concepts of the labeled dataset and only credited for the one it aligns with most strongly. We clarified this point in the revised metric description.

W3 (Supervision): We don't see this as a weakness as all SAEs require labeled data to construct a steering method. Due to space limitations, please refer to L (Generalizability of G-SAE) of reviewer Agfr.

W3 (Conventional methods): Thank you for this suggestion. While we agree that it is important to connect new metrics to established evaluation practices, we note that FMS is designed to capture monosemanticity — a property not addressed by existing metrics. We note that FMS combines the underlying primitive as probes [1,2]: feature-level classification accuracy, and disentanglement measures like Mutual Information Gap [4] used in Variational Autoencoders. Our acc_0 component is conceptually similar to the probing accuracy used in these approaches, but FMS extends this by also measuring exclusivity (local disentanglement) and feature spread (global disentanglement) — aspects not captured by traditional probes. We also observe that acc_0 (higher is better) aligns well with the overall FMS trend and model ranking (c.f. Tab.1), and complementary metrics that assess how pure a split by a feature is, such as Gini impurity (mean of Gini values of second level in Fig 11, the lower is better), yield similar conclusions:

Additionally, we confirm that higher FMS scores correspond to improved generation steering (c.f. Tab.2), offering behavioral validation of the metric. However, due to the lack of prior metrics for monosemanticity, a direct comparison or alignment is not straightforward. We therefore see FMS as complementary: extending interpretability analysis into an underexplored axis with practical relevance. We appreciate the opportunity to clarify this and have made these relationships more explicit in the revised paper. We hope this addresses your questions.

W4 (Fig. 4): Thank you for this careful observation — we agree that Figure 4, in its current form, may give a misleading impression about the activation spread and absence-case behavior of vanilla SAEs. We have clarified the analysis and adjusted the visualization in the revised version:

W4 (Activation spread): The spread difference is primarily due to a lack of clear conception separation of vanilla SAEs, and moreover, showcase their drawback when using them as concept detectors when compared to G-SAE. To provide a fairer, scale-invariant comparison, we report the Ranked Biserial Correlation (RBC) from a Mann–Whitney U test (Section 5.3), which more directly measures separation between "present" and "absent" distributions. This test clearly favors G-SAE, confirming that it more reliably distinguishes the presence of a concept despite wider activation variance. Moreover, note that the plot averages over all concepts, also coarse and sometimes erroneous sentence-label datasets RTP and SP, which naturally introduces noise that cannot be removed and increases the spread. The token-granular PII indeed performs much clearer, as shown in Figure 9.

W4 (absent cases): We apologize for this illustration. We used boxplot settings (25%–75% interquartile range with no outliers), obscuring that vanilla SAEs also produce activations >0 for absent concepts. We've now updated the plot to reflect the 10%–90% range, clearly showing that normal SAEs also activate in the “absent” case. For example, for SP, 16% of the cases activate >0 when actually absent. Note that on the other hand, for the present case, only ~34% activate overall for vanilla SAE, while ~85% for G-SAE. We have updated our figure accordingly and extended our experimental description and discussion.

Style Feedback: We appreciate the feedback and improved stylistic appearance of our figures.

Q1 (Extra Evaluations): Thank you for this important point. To evaluate the general utility of FMS, we applied it not only to G-SAEs but also to pretrained [3] and our vanilla SAEs. As shown in section 5.2, Table 2, and the Finetuning table seen in L (Generalizability of G-SAE) of reviewer Agfr, FMS scores capture apparent differences in concept disentanglement between vanilla, pretrained, and fine-tuned SAEs. For example, finetuning a pretrained SAE improves FMS@1 from 0.42 to 0.50, while reconstruction performance remains stable. This shows that FMS is model-agnostic and informative across architectures.

To evaluate G-SAE using established tools independent of FMS, we also ran SAE-Bench [5]. As shown in the SAE-Bench results table in Q2 (Cost of monosemanticity) of reviewer QsmM, G-SAE matches vanilla SAEs across all evaluation criteria, while being much more interpretable. Thus demonstrating that the general capabilities of the SAE are retained.

Together, these results demonstrate that:

• FMS is a meaningful metric even when applied to non-G-SAE models, and
• G-SAE retains general capabilities, as shown by an evaluation using third-party benchmarks.

Q2 (Unsupervised training): We would like to phrase it as automatic supervision, as G-SAE requires labels of some sort. One would need to have a concept labeler that creates labels for G-SAE on the fly. We think this could be partially achieved with current LLMs, but this still needs refinement to achieve fast, reliable labels. Concerning the extension to a more general setting, we conducted a few finetuning runs on a pretrained SAE as described in the answer above, where we demonstrate that a post hoc application of G-SAE is also possible.

Q3 (General applicability of FMS): We thank the reviewer and agree that a key strength of FMS is its broad applicability. Though we focus on SAEs here, FMS is model-agnostic and can be applied to other feature extractors like transformer circuits, CLIP neurons, or PCA components. It could also enable comparisons across interpretability methods in terms of concept alignment, disentanglement, and selectivity. While such analysis is beyond the scope of this submission, we are actively exploring it and will highlight FMS’s broader potential in the discussion.

L1 (SAE critiques): We thank the reviewer and agree that SAEs have key limitations, such as instability, incomplete feature recovery, and imperfect monosemanticity. We've clarified these in the limitations section. While G-SAE doesn't fully resolve them, its supervision improves concept stability and alignment, mitigating several issues. We’ve revised the paper to emphasize these points more clearly.

[1] Bricken et al., “Towards Monosemanticity: Decomposing Language Models With Dictionary Learning”, Transformer Circuits Thread 2023

[2] Gao, et al., "Scaling and evaluating sparse autoencoders.", ICLR (2025)

[3] Hugginface: EleutherAI/sae-llama-3-8b-32x-v2 @ layer 11

[4] Chen et al., "Isolating sources of disentanglement in variational autoencoders.", NeurIPS (2018)

[5] Karvonen et al., “SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability”, ICML (2025)

We appreciate that the reviewer finds our explanation very clear and easy to understand.

In answer to your questions:

W (Global vs local measures): We agree that additional examples can be even more helpful, and have added an example that better explains/distinguishes between the two forms. While related, they capture complementary aspects of monosemanticity: Local disentanglement measures exclusivity: How much of a concept is isolated in a single feature. Global disentanglement captures concept spread: Whether the concept is distributed across multiple features. Both are necessary: A feature may be predictive but redundant, i.e., multiple features encode the concept equally well. Leading to a high global score but low local disentanglement. Conversely, suppose a concept is primarily encoded in one feature but clearly improves when a few others are added. In that case, this yields higher local but lower global disentanglement, since the concept isn't entirely isolated. These distinctions justify evaluating both axes to measure monosemanticity robustly. We have updated our writing accordingly.

W (Empirical example): We added a descriptive, easy-to-understand running example from the likes of Table 8, Appendix J.3, from the PII dataset. For example, in the vanilla SAE for the user-name “20jey.marlov”, both the Email and User Name features activate or none, whereas in G-SAE, only the User Name activates. The acc_0 score is low in the vanilla SAE, reflecting that the most predictive feature is not uniquely informative. Local disentanglement is also low, since removing the top feature has limited effect, as other features like Email still contribute overlapping signals. Similarly, global disentanglement is poor, as multiple features must be combined to reconstruct the concept effectively. In general, standard SAE activates little and more ambiguous. We added the example to the main text to the paragraph line 151. We will extend Tab. 8 as well to also display examples for the vanilla SAE activations and corresponding scores from FMS.

W (Global Disentanglement Loss): As already pointed out, we do not, albeit possible, deploy an explicit global disentanglement loss. Nevertheless, we observed such good performance already, which is likely due to the shift in activation value ranges as seen in Fig. 4a: Through conditioning, the activation ranges naturally rise and would suppress the non-supervised concepts on activation. We follow up on this feedback in Q1 and add this discussion to Section 5.3.

Q (Loss modification for global disentanglement): Thank you for the interesting question. Indeed, our current method directly targets local disentanglement by promoting isolated concept activation. While this additionally improves global disentanglement, as seen in Tab.2, we agree that it does not explicitly enforce it. A loss targeting global disentanglement likely involves an adversarial objective: encouraging one feature to encode the concept while explicitly suppressing activation in other features. However, this presents several design choices regarding: What signal should be used to "suppress" non-target features? How should overlapping or hierarchical concepts be handled? Should absent concepts be treated as zeros or smoothed labels (e.g., small values like 0.01)? We are actively exploring these directions and believe they merit a dedicated follow-up, but chose not to include them in this version due to the additional complexity. We now discuss this in the future work and outlook.

L (Generalizability of G-SAE): Thank you for highlighting this important distinction. We would like to clarify that any practical use of SAEs for concept detection inevitably requires labeled data at some stage. Without labeled concept data, there is no meaningful way to interpret or validate the latent features discovered by an SAE. Existing approaches typically rely on post hoc analysis, e.g., probing classifiers [1,2], which use labeled data after training to identify and evaluate concepts. Our approach shifts this supervision to earlier: instead of using labels after training, we use them during training to guide the representation learning process directly. Importantly, this does not introduce new data requirements compared to conventional pipelines — the same labels are used, only at a different stage. In addition to the experiments of the paper, we have made an additional experiment demonstrating that the supervision of G-SAE can also be applied post hoc. Specifically, we took a conventional, pretrained SAE [3] that was initially trained without label supervision and then finetuned it using our G-SAE method. The fine-tuning cost is negligible compared to pretraining (a few million vs 8.5B). The experiment demonstrates that G-SAE (supervision) can also be applied successfully after SAE pretraining without losing the standard SAE performance, as shown in the Table below, which further demonstrates the usability of our approach. The general SAE ability remains similar (see ce loss score or mse) while the monosemanticity of the SAE improves significantly (see FMS scores). We added this discussion more thoroughly to our paper.

E=Finetuned Epochs; LR =Learning Rate

Formatting Concerns: Thank you for the pointer. We agree and have applied the profanity filter to this content in our paper.

[1] Bricken et al., “Towards Monosemanticity: Decomposing Language Models With Dictionary Learning”, Transformer Circuits Thread 2023

[2] Gao, et al., "Scaling and evaluating sparse autoencoders.", ICLR (2025)

[3] Hugginface: EleutherAI/sae-llama-3-8b-32x-v2 @ layer 11

We appreciate the follow-up and thank the reviewer for the engagement and positive assessment.

We agree our formulation in its current way is not fine granular enough. There is a difference between supervision during training versus applying it during evaluation, which we have integrated into this formulation. At the same time, our additional rebuttal experiment has shown that G-SAE supervision can be applied post-hoc to a pretrained SAE, suggesting this flexibility is not lost. Instead, it can be preserved through a lightweight fine-tuning step, adding concept-specific monosemanticity when needed, without discarding the benefits of general-purpose SAE pretraining.

We have updated the paper to reflect this better. We hope to have addressed your concern and are happy about further discussion.

We thank the reviewer for the constructive feedback and appreciate that the reviewer finds our paper very clear and accessible.

In the following, we will address your concerns:

W1 (Addressing labeling): Label quality is critical in machine learning for both evaluation and training. A labeled dataset is also needed to find concepts in standard SAEs and subsequently use them to detect or steer, making label noise and concept ambiguity not unique to G-SAEs.

In our experiments with RTP and SP, only sample-wise labels are available and indeed several standard tokens are prone to some noisy labels. Nevertheless, the concept detection and steering work well yet naturally with more variance than the token-annotated PII dataset, as demonstrated in Tab. 2. This may be attributed to conflicting annotations averaging out on large enough datasets.

Similarly, our metric is intentionally designed to expose these label ambiguities and inconsistencies; not as a limitation, but as a key feature. All model evaluations require a high quality dataset to be evaluated with, as label errors or coarse granularities skew the usefulness. As such, this issue again is not unique to FMS, but relevant to all machine learning evaluations. We have extended our discussion with a paragraph on label quality being a relevant factor for concept supervision, e.g., also in G-SAEs or FMS. Please also see the response below to the related point raised.

W2 (Label diversity): We appreciate the desire for broad evaluations and agree that understanding the limits of our approach across different concept types is important. Our current choice of domains—privacy (PII) annotated token-grained on 24 different concepts, Shakespearean style (SP) and toxicity (RTP) annotated sentence-wise—was intentional and aimed at capturing a spectrum of concept clarity and ambiguity.

PII serves as an example of well-defined, low-ambiguity concepts: the labels are word-level and often based on strict, rule-based patterns (e.g., email addresses), and G-SAE shows a strong and consistent performance, as shown in Tab. 1 (acc_0). In contrast, SP and RTP represent more ambiguous and context-dependent domains. For example, RTP only has sample-level labels, and words like “kill” may appear in both toxic and non-toxic contexts (e.g., “kill people” vs. “kill time”), making disambiguation inherently more challenging. Despite this, G-SAE still provides meaningful improvements, albeit with smaller margins than PII.

We outperform exclusively on all these various evaluations, which should naturally be the case due to the supervision, and have not found drawbacks yet. However, further specific suggestions are welcome, and we have added the discussion about concept ambiguity to the paper.

W3 (Running example): We appreciate the feedback regarding our clear writing and welcome suggestions to make our work accessible to an even broader audience. As a short example, from the PII dataset of Tab. 8, in the vanilla SAE for the user-name “20jey.marlov”, both the Email and User Name features activate or none, whereas in G-SAE, only the User Name activates. The acc_0 score is low in the vanilla SAE, reflecting that the most predictive feature is not uniquely informative. Local disentanglement is also low, since removing the top feature has limited effect, as other features like Email still contribute overlapping signals. Similarly, global disentanglement is poor, as multiple features must be combined to reconstruct the concept effectively. In general, standard SAE activates with lower scores and is more ambiguous.

In contrast, G-SAE has a strong single feature activation leading to high scores in acc_0, local and global disentanglement, resulting in a high FMS score.

We will extend Tab. 8 to also display examples for the vanilla SAE activations and added this discussion to paragraph line 151.

Q1 (Used classifier): Thank you for the insightful question. We chose binary decision trees primarily for their interpretability, sparsity, and alignment with the discrete, localized nature of monosemantic features. However, the FMS metric is classifier-agnostic by design: it evaluates concept capacity, local exclusivity, and global spread — all of which can be derived using any classifier that provides comparable accuracy signals. In theory, the choice of classifier should only marginally influence FMS, and mainly through differences in how classifiers optimize their objectives (e.g., Gini impurity for trees vs. hinge loss for SVMs). Some variation may occur depending on data sparsity and decision boundary shape, but the underlying structure of concept-feature alignment remains the same. As an example, we could compute FMS using SVMs in place of decision trees as follows: Acc_0: Train an SVM for each latent feature and take the feature with the highest accuracy as the best separating feature Local: Measure the accuracy drop between the highest and second-highest feature Global: Based on accuracies from 1. incrementally add features to be used in a new training of an SVM and document the increase in accuracy. This mirrors our decision tree-based implementation but substitutes SVMs as the base classifier. While the learned boundaries may differ slightly due to different optimization criteria, the structure and logic of FMS computation remain consistent, supporting its classifier-agnostic design. We have added an evaluation and discussion to the appendix.

Q2 (Cost of monosemanticity): We further evaluated the trade-off between interpretability and general performance using SAE-Bench [1], a standard benchmark performing some specific tasks expected on SAE’s. As shown in the table below, G-SAE matches vanilla SAEs across all evaluation criteria, while being much more interpretable in our conditioned concept.

This suggests that monosemanticity improvements do not come at the cost of model fidelity or general utility. SAE-Bench results (except for mse, higher is better):

Q3 (Ethical Concerns): We have written a more extensive ethical impact statement in the appendix and the paper checklist, see Appendix A and Neurips Paper Checklist Questions 9,10, and 11. We agree that it deserves more attention and have moved it to the main body.

[1] Karvonen et al., “SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability”, ICML (2025)

Thanks for the response and raising the score! We appreciate the constructive feedback. If there is anything further we can help with, please let us know.

We thank Reviewer 5oym for their review, and appreciate that they find our introduced FMS metrics novel and discriminative, and our extension of SAE’s effective.

In answer to your questions:

W1 (user study): We respect the suggestion regarding a user study, which is indeed important for evaluating human perception and understanding of certain types of model explanations. However, G-SAE's core focus is on a more fundamental aspect: achieving directly measurable monosemanticity within latent features, meaning each feature cleanly represents a single, distinct concept. This property is rigorously and objectively measured via standard quantitative evaluation of features (e.g., acc_0) and our Feature Monosemanticity Score (FMS), e.g., Fig. 3 and 11, and can be directly compared to the labeled concept data. In this context, a user study would essentially parallel what our quantitative analysis already confirms. The utility of these monosemantic features is, moreover, clearly demonstrated by their successful deployment in practical applications such as toxicity control, privacy-aware generation, and style adaptation. We would appreciate suggestions for user studies that could further demonstrate and reinforce our findings.

W2 (motivation of monosemanticity): We appreciate the reviewer’s perspective and agree that clearly demonstrating end-user relevance is important. To summarize, we show that G-SAE’s, through their monosemantic capability, merge the concept on a single neuron only. This enables us to use this single neuron in the entire model as a concept detector, which could be leveraged to the frontend as shown in Tab 8. Moreover, we show how using its corresponding output vector leads to an effective steering method. To realize these use cases in other methods [1,2,3] one needs to somehow merge the sparse and low activations, which may introduce further ambiguity. Section 5 demonstrates concrete use cases where monosemantic features directly benefit downstream utility, for detection as well as steering, such as toxicity reduction, writing style adaptation, and privacy preservation. Throughout the paper we follow this motivation and demonstrate with 26 distinct concepts that G-SAE exclusively outperforms all other compared SOTA methods.

This demonstrates how monosemantic representations can enable more precise and user-relevant model behavior. We incorporated this discussion more prominently in the Introduction and Conclusion to further strengthen the need and relevance of monosemanticity.

W3 (Labeled data): Thank you for highlighting this important distinction. We would like to clarify that any practical use of SAEs for concept detection inevitably requires labeled data at some stage. Without labeled concept data, there is no meaningful way to interpret or validate the latent features discovered by an SAE. Existing approaches typically rely on post hoc analysis, e.g., probing classifiers [4,5], which use labeled data after training to identify and evaluate concepts. Our approach shifts this supervision to earlier: instead of using labels after training, we use them during training to guide the representation learning process directly. Importantly, this does not introduce new data requirements compared to conventional pipelines — the same labels are used, only at a different stage. In addition to the experiments of the paper, we have made an additional experiment demonstrating that the supervision of G-SAE can also be applied post hoc. Specifically, we took a conventional, pretrained SAE [6] that was initially trained without label supervision and then finetuned it using our G-SAE method. The fine-tuning cost is negligible compared to pretraining (a few million vs 8.5B). The experiment demonstrates that G-SAE (supervision) can also be applied successfully after SAE pretraining without losing the standard SAE performance, as shown in the Table below, which further demonstrates the usability of our approach. The general SAE ability remains similar (see ce loss score or mse) while the monosemanticity of the SAE improves significantly (see FMS scores). We added this discussion more thoroughly to our paper.

E=Finetuned Epochs; LR =Learning Rate

Q1(Weighted loss function): We explored weighted loss configurations, and our empirical findings indicated that excessive weighting of the conditioning term led to degraded reconstruction (i.e., G-SAE behaving like a probe). A 1:1 balance consistently gave the best trade-off between interpretability and fidelity across tasks. However, to further fine-tune, given a particular task at hand, it might be good to use a weight that is different than 1:1. We have added this investigation to the paper.

[1] Liu et al. "In-context vectors: Making in context learning more effective and controllable through latent space steering.", arXiv preprint arXiv:2311.06668 (2023).

[2] Rimsky et al. "Steering llama 2 via contrastive activation addition.", ACL (2024).

[3] Subramani et al., "Extracting latent steering vectors from pretrained language models.", ACL (2022).

[4] Bricken et al., “Towards Monosemanticity: Decomposing Language Models With Dictionary Learning”, Transformer Circuits Thread 2023

[5] Gao, et al., "Scaling and evaluating sparse autoencoders.", ICLR (2025)

[6] Hugginface: EleutherAI/sae-llama-3-8b-32x-v2 @ layer 11

Thanks for your response and engagement! We are happy to clarify our claim regarding labeled data.

We would like to clarify the other two points raised by the reviewer further.

Monosemanticity. To further demonstrate the benefit of monosemanticity on cleaner detection and steering, we extended Tables 6, 7, 8 of Appendix J (SP, RTP as steering, PII as detection) with their corresponding vanilla-SAE samples and further annotated their measured FMS scores (average FMS@1 Vanilla SAE: 0.27 vs G-SAE 0.52, per-class details are added to the appendix). Some examples are shown in the tables below. In particular, in PII, it can be seen that vanilla frequently detects other concepts than the target concept. For steering into Shakespeare style, it often falls into repetition and does not obey the style appropriately.

User study. To assess the human-centric alignment of our approach, we manually validated the F1 score against human-graded correctness (on a 0–100 scale, based on approximate percent matching) using 20 randomly selected samples. Participants evaluated the concepts detected by G-SAE against their own assessments. Results showed a strong positive correlation (R² = 0.85), indicating that the F1 score for G-SAE reliably reflects human judgment of correctness. Two independent annotators conducted the in-house evaluations. We are currently expanding this validation beyond the internal team, and final results will be included in the revised manuscript.

We added these results and discussions to the paper and are looking forward to your feedback.

Two parts of samples for tokenwise PII annotations taken from table 8 with respective maximal class activation values. In the first example, the vanilla SAE for the tokens of user-name “20jey.marlov” activated the Email, User Name and None, whereas in G-SAE, only the User Name activates.

Two steering examples for Shakespeare from table 7: