Analyzing Multilingualism in Large Language Models with Sparse Autoencoders

We sincerely appreciate the reviewer’s thoughtful feedback. Below, we provide additional experimental results and clarifications to address your concerns.

1. Frequency as an Initial Filter.

We agree that there are alternative ways to define TF features. We adopted frequency-based filtering as an initial step to exclude features that activate on too few tokens, primarily for its simplicity. We found this approach sufficient for effectively identifying TF features—for instance, Figure 1 in the paper shows that the resulting TF features consistently activate on task-instruction-related segments. In line with your suggestion, we agree that exploring alternative definitions and comparing their effects would offer valuable insights, and we plan to pursue this during the remaining week of the rebuttal period.

2. Threshold.

The primary contribution of our work is to demonstrate that TF and HF features exist in SAEs, with meaningful differences in their ratios across languages. These definitions are most meaningful when the threshold parameter p is close to 1, aligning with their intended motivations. In response to your comment, we examined a range of p values and found that the overall patterns remain stable (i.e., high-performing languages consistently exhibit higher TF-feature ratios than low-performing languages) under small perturbations around p = 1. The corresponding statistics are provided in Table 1 (We have also updated the terminology to “High-Performing Languages (HPL)” and “Low-Performing Languages (LPL),” and added two additional models—Gemma2-2B-IT and Llama 3.1-8B-Instruct—with more detailed explanation provided in #6 below.)

Table 1: Effect of the hyperparameter p on TF and HF features. Note that the definitions align with their intended motivation when p is close to 1.

3. MMLU-STEM.

The main reason for restricting our evaluation to MMLU-STEM topics was the large size of the full MMLU dataset (approximately 15k examples). We selected a smaller STEM-focused subset (around 1.3k examples), which is comparable in size to the ARC-Challenge dataset (1.2k), to maintain experimental feasibility.

4. Language Classification.

Thank you for pointing this out. We initially categorized the languages primarily based on their task performance, supplemented by some preliminary knowledge of their presence in the pretraining data. Given your concern—and the fact that the exact distribution of languages in the model’s pretraining and instruction tuning data is difficult to determine—we are happy to revise the terminology to 'high-performing' and 'low-performing' languages with respect to a specific task.

5. Presentation Issues.

Thank you for bringing this to our attention. We will include the bar for ZH_CN in the plot (the corresponding value can be found in Table 2 of the Appendix). Also, regarding the reference, our intention was to cite it as an example of applying SAEs in practical tasks, but we agree it may appear out of place. We will revise it accordingly in the updated version.

6. Additional Experiments Including English Results.

(Continued)

We sincerely thank the reviewer for taking the time and effort to review our work. Below, we provide additional experimental results and clarifications to address your concerns.

1. Heuristics and Hyperparameters

We agree with your point and acknowledge that our ATF method involves some degree of hyperparameter tuning. While an additional efficient and principled approach to hyperparameter selection would certainly be desirable, we currently rely on validation-based tuning and leave the development of such strategies to future work. Nevertheless, our method already significantly narrows the feature space by filtering down to TF features—reducing, for instance, around a total of 40k active features in the Dev set to a compact set of about top-30 TF features—thereby greatly limiting the search space. We are actively working on identifying more specific and consistent steering patterns within this reduced set, as we agree this direction is both promising and valuable.

Additionally, we used a fixed threshold of p = 0.9 throughout our experiments. as we believe the definitions of TF and HF features are most meaningful when the threshold parameter p is close to 1, aligning with their intended motivations. In response to your comment, we examined a range of p values and found that the overall patterns remain stable (i.e., high-performing languages consistently exhibit higher TF-feature ratios than low-performing languages) under small perturbations around p = 1. The corresponding statistics are provided in Table 1 (We have also updated the terminology to “High-Performing Languages (HPL)” and “Low-Performing Languages (LPL),” and added two additional models—Gemma2-2B-IT and Llama 3.1-8B-Instruct—with more detailed explanation provided below.)

Table 1: Effect of the hyperparameter p on TF and HF features. Note that the definitions align with their intended motivation when p is close to 1.

(Continued)

We sincerely appreciate the reviewer’s time and effort in evaluating our work. Below, we present additional experimental results and clarifications in response to your comments. We have also rearranged the sections for improved clarity and flow.

1. Response to Comment 3

Yes, we agree that using English headings from the start can be natural. However, as you pointed out, offering an explanation for why English headings lead to better performance is valuable. Our findings show that LLMs exhibit multiple HF-features when processing prompts, suggesting that headings play a significant role in how prompts are interpreted—thus providing a plausible explanation for the observed performance differences.

2. Response to Comment 2

Thank you for pointing this out. We initially categorized the languages primarily based on their task performance, supplemented by some preliminary knowledge of their presence in the pretraining data. Given your concern—and the fact that the exact distribution of languages in the model’s pretraining and instruction tuning data is difficult to determine—we are happy to revise the terminology to 'high-performing' and 'low-performing' languages (HPL and LPL) with respect to a specific task.

(Continued)

We sincerely thank the reviewer for taking the time and effort to evaluate our work. Below, we provide additional experimental results and clarifications in response to your comments.

Experiments on Additional Models

Regarding your feedback, we have added two additional models—Gemma2-2B-IT and Llama 3.1-8B-IT (using SAEs from layer 20 with width 65k from the Gemma scope, and layer 23 with width 131k from the Llama scope)—to assess the generalizability of our approach. We have also included English results in the tables below. We’re excited to share these encouraging findings. (Additionally, we have updated the terminology to “High-Performing Languages (HPL)” and “Low-Performing Languages (LPL)”.)

Table 1: HPL exhibit higher TF and HF feature ratio than LPL on both models.

Table 2: Results on Gemma2-2B-IT.

Table 3: Results on Llama3.1-8B-Instruct.

The key takeaways are as follows:

• The main patterns from the original paper—such as the higher TF and HF feature ratios for high-performing languages (Table 1) and the effectiveness of EHP and ATF (Tables 2 and 3)—are consistent across the new models.
• English, as the pivot and highest-performing language, shows the highest TF-feature ratio and the second-highest HF-feature ratio among all languages examined across both models (Table 1). Moreover, English performance can be also improved through ATF, which we believe is an encouraging result (Tables 2 and 3).

These results indicate that our key findings from the original paper generalize across different model sizes and architectures, thereby reinforcing our main contribution—namely, that the concepts of TF and HF features provide novel and fresh insights into the origins of language performance gaps in LLMs.

We focused on the zero-shot setting, as we believe it is better suited to revealing intrinsic differences in LLM behavior across languages, as discussed in lines 105–109 of the paper. We agree that analyzing additional layers could offer further insights, and we will include more layer-wise results during the remaining week of the rebuttal period.

---

Thank you once again for your valuable feedback and for taking the time to review our work. We hope these additions have addressed your concerns and strengthened the paper. If you find the revisions satisfactory, we would deeply appreciate your reconsideration of the rating.

We sincerely thank the reviewer for taking the time and effort to evaluate our work. We are delighted to hear that you found our work interesting! We summarized how we have addressed your comments below.

1. About the Reference

Thank you for sharing the reference. We reviewed it and found it quite interesting. It shares similarities with our work in that it also explores multilingualism in LLMs through the lens of SAEs. Their approach identifies features with higher average activation for specific languages and uses them for ablation, demonstrating that removing these features primarily degrades performance (i.e., increases CE loss) for the corresponding language while leaving others relatively unaffected. They also show that these language-specific features can be used to steer outputs toward a particular language, which is a direct application of their findings.

We believe the main difference of our work is its more analytical focus—aimed at gaining a deeper understanding of how LLMs process the prompts and the origins of performance gaps between languages. Our key contribution lies in introducing the new concepts of TF and HF features, which exhibit consistent distributional differences that help explain disparities in language performance.

2. About Hyperparameters

We agree with your point and acknowledge that our ATF method involves some degree of hyperparameter tuning. In practice, we found that the presented hyperparameters generally work well across different languages and tasks. While developing a more efficient and principled approach to hyperparameter selection would certainly be beneficial, we currently rely on validation-based tuning and leave more robust strategies for future work. Nevertheless, we would like to emphasize that our method already substantially reduces the feature space by narrowing it down to just a few dozen TF features, significantly limiting the search space. We are actively working to identify more specific and consistent steering patterns within this reduced set, with the goal of further minimizing reliance on hyperparameter tuning.

3. ATF-only performance

Thank you for pointing this out. Our primary focus was to assess whether ATF could provide additional gains on top of the already strong EHP prompt. However, we agree that assessing the standalone performance of ATF is also important. Due to the heavy load of additional experiments (across six reviewers), we currently report initial ATF-only results on the Llama3.1-8B-Instruct model for the ARC task (N.B., based on suggestions from other reviewers, we have added two more models—Gemma2-2B-IT and Llama3.1-8B-Instruct). We observe in Table 1 that while ATF yields a notable improvement over the zero-shot baseline, it still underperforms compared to EHP and EHP+ATF. We plan to provide more results over the remainder of the rebuttal period.

Table 1. ATF-only results on Llama3.1-8B-Instruct for ARC (row 2).

4. TF-Feature Transfer and Prompt Sensitivity

We appreciate this insightful observation. First, we would like to clarify that we used the same task instruction and prompt structure for both MMLU and ARC—for example, the English instruction: “Answer the following multiple-choice question, ensuring the response is one of A, B, C, or D.” As a result, the TF features are largely overlapping across the two tasks. For instance, using Gemma2-2B-IT, we identified 39 TF-features for ARC and 34 for MMLU, with 32 shared features. Consequently, the ATF results are essentially very similar. In summary, because we used the same task instruction and prompt structure with a high threshold of p = 0.9, the features captured from each dataset show a high degree of overlap. We agree, however, that using different or more task-specific instructions could lead to variations in TF features, and exploring this would be a valuable direction for future work. Additionally, investigating the robustness of TF features under paraphrased instructions is another promising avenue that we plan to pursue.

---

We hope these additions have addressed your concerns. Thank you again for your intriguing feedback and for taking the time to review our work!

We thank the reviewer for the valuable feedback. We would like to provide additional experimental results and explanations to address your concerns. We refer to items under “Questions to Authors” as Q and those under “Reasons to Reject” as R.

1. Response to Q#1.

Thank you for the suggestion. We will replace the notation with clearer ones such as $S_E^{(t)}(X_i)$.

2. Response to Q#2 and R#3.

Yes, we are aware that the practical utility of SAEs has received mixed reviews in recent literature. However, most of the criticism has centered on their use in probing tasks—as discussed in the link you provided and in https://arxiv.org/pdf/2502.16681 or https://arxiv.org/abs/2501.17148. While we agree that replacing hidden states with SAE activations may not offer performance gains, we believe that SAEs can add value when used as analytical tools and assistants rather than as direct replacements.

For example, our introduction of TF and HF features demonstrates how SAEs can reveal new, fine-grained insights into how LLMs process prompts—insights that would be difficult to uncover without them. This analytical strength is where we believe SAEs truly shine. While the use of SAEs for model steering has also received mixed opinions, the overall sentiment appears to lean slightly optimistic, and we believe our work contributes positively to this growing perspective.

3. Response to Q#3 and R#1.

Thank you for pointing this out. We are happy to revise the terminology to "high-performing" and "low-performing" languages (HPL and LPL) with respect to a specific task. Since the precise distribution of languages in the model’s pretraining and instruction tuning data is difficult to determine, we agree that classifying languages based on their performance on a given task is the most straightforward and simple approach.

4. Response to R#2.

We’re not entirely sure why using data translated from English would necessarily make EHP more effective. EHP simply modifies three heading tokens—“Question,” “Choices,” and “Answer”—into English, which are fairly common words and not specifically tied to the original English phrasing of the task. We would greatly appreciate further clarification if we’re missing a particular concern or nuance here!

(Continued)

I thank the author for the response.

I have updated the score to better reflect my opinion after your comment. The new experiments are useful, however I still stand by the criticism of SAEs and I think this is something that should be better discussed in the main body of the paper.

Regarding the notation, it was already clear. Just refrain from using notation that deliberately spell things like "SEX" or "SEXi".