The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalities

Thank you for your thoughtful review and your appreciation of the interest value and the robustness of our experiments!

Weakness 1 (the dependence of results on experimental settings): While results of all experimental studies inevitably depend to some extent on the experimental settings, we use standard methods (e.g., the logit lens) and design decisions in our settings. In particular, we adopt the logit lens, a training-free method, for analysis precisely because it does not require learning a probe and minimizes potential confounders. We are confident that alternative experimental design decisions would lead to similar findings. Regarding the specific question about the non-parallel baseline: we randomly choose one non-matching sentence as the baseline. In the revision, we added this explanation, as well as additional experimental details in section A of the appendix. Thank you for this suggestion!

Weakness 2 (using the last token’s representation as the sequence representation): We have already included a discussion about this in L193-194, both citing prior work with the same practice, as well as more principled evidence that the last token representation faithfully retains sequence information. We also performed preliminary studies using average pooling of token representations along the sequence dimension, and found similar trends as taking the last token’s representation.

Weakness 3 (the small absolute scale of cosine similarities): This is indeed true, but we emphasize that absolute similarity measures are difficult to interpret in high dimensional spaces (where all vectors are nearly orthogonal to one another). Moreover, the confidence intervals indicate that the effect is statistically significant.

Weakness 4 (the use of English-dominant models): We primarily tested on English-dominant models because most state-of-the-art models are predominantly trained using English data. But we do test LLMs with other dominant languages, as well as those without a dominant language, in section 4.1.

Weakness 5 (related work): Thank you for bringing these papers to our attention! We have added citations and a brief discussion around them in our introduction (L89-91) in the revision.

Question 1 (the multilinguality of BLOOM): Yes, BLOOM was trained on a relatively balanced mixture of multilingual data and hence not English-dominant (see its Figure 3). On the other hand, Llama-2 and Llama-3 were trained on predominantly English data.

We are grateful for your review and are glad to see that you appreciate the technical framework underlying our experiments, as well as the thoroughness of the experiments.

Weakness 1 (comparison with the Platonic Representation Hypothesis paper): Huh et al. (2024) is certainly related to our work, and we discussed this relationship in the final paragraph of section 1 and the first paragraph of section 4.3. In particular, their claim is that different models of the same task, or even of different tasks (more relevant to us), have converging representations. In contrast, we consider the representation similarity within a single multimodal model. Our notion of representation convergence is stronger than theirs: in their experiments, they found that the model representations are isomorphic (e.g., the distance between the words “dog” and “cat” are similar to the distance between pictures of a dog and a cat), while we directly look at cross-modal representation similarity (e.g., the representation of the word “dog” is close to that of the picture of a dog). We clarified this a bit more in the final paragraph of section 1 in the revision. Other than this explicit operationalization, Huh et al. (2024) also discusses such convergence on a more philosophical level. We added a reference to this discussion in the related work section.

Weakness 2 (paper organization): Thank you for this feedback! We will certainly improve the paper organization in the next revision, putting some content into the appendix as needed. We respond to your specific suggestions below.

Weakness 3 (limitation of our similarity measure): We agree that structural similarity would be a stronger measure than pointwise relative similarity, but we believe they are correlated. In the extreme, if Equation 1 were satisfied for all possible inputs, that would imply complete structural similarity. Measuring structural similarity may require design decisions that potentially confound the results (for example, Huh et al. (2024) entertained multiple kernel-alignment metrics). But in any case, we do think a more in-depth investigation of our observations would be exciting future work. We prioritized investigating a variety of data types because we found our observations very interesting and wanted to reliably verify that this is a consistent and robust property of models across data types, before more closely analyzing this property in future work.

Weakness 4 (suggestion of projecting inputs to the dominant data type and then generate): We argue that this is not a weakness per se, but it is certainly an interesting suggestion! We set up our experiments to enable an investigation of the hypothesis in a training-free setup which would not require learning an alignment model to go between modalities, in order to minimize potential confounds. But a more heavily parameterized probe model could uncover even more interesting phenomena and it would certainly be an interesting experiment to run as future work!

Question 1 & 2 (multimodal alignment score calculation details): Thanks for pointing out this potential source of unclarity. We added more detailed descriptions of the metrics in appendix A.3.

Question 3 (figure for the code semantic role experiment): We implemented your suggestion about control tokens. In https://ibb.co/SxbdZ5h, we show the same plot, except that we make the cell gray when the probability of neither token (the semantic role and the surface argument) is greater than the whitespace as the control token. The noise still persists since there are not many gray cells; this is still roughly true when we use the word “object” (https://ibb.co/dM6BGGC) or “.” (https://ibb.co/K508BC9) as the control token. Stepping back, instance-level variance (that leads to the noise) is in some sense inevitable, but we believe the blue/red separation is nonetheless clear from our plot. Also, thank you for the presentation suggestion! We improved this slightly in the updated version and will systematically improve the presentation of all plots in our next revision.

Question 4 (the small absolute scale of cosine similarities): This is indeed true, but we emphasize that absolute similarity measures are difficult to interpret in high dimensional spaces (where all vectors are nearly orthogonal to one another). Moreover, the confidence intervals indicate that the effect is statistically significant.

Thanks! Weakness 2 can now be considered addressed as well. Upping my presentation score. I believe the authors have done a good job of addressing the weaknesses I have raised as well as those raised by other reviewers. There are some small outstanding weaknesses from me and others; however, I would really like to see this paper at ICLR and future work in this vein! I wish there were an option to give a 9, and would give this score if it were an option. Please consider my review on the more positive side of an 8.

Thank you for your thoughtful review! We are glad that you recognize the value of our paper.

Weakness 1 (the use of relative similarity measures): In our revision, we added a citation to a theoretical reason for this practice, as well as a discussion on its implicit use in prior work (footnote 2, L161). Specifically, similarity measures in high dimensional spaces are difficult to interpret on their own (where most vectors are orthogonal to one another) without a baseline with which to compare against. Most prior work on probing (including logit-lens probes) implicitly uses relative similarity measures since the similarity scores are normalized over a finite label set. However, this is difficult to do in our case because the set of "negative examples" is too large to normalize over. Additionally, we do show absolute similarity measures for some of our studies, for example in the mutual translation setting, which are still high.

Weakness 2 (multilingual intervention results): We agree that a strong version of the hypothesis would entail that dominant-language (i.e., cross-lingual) steering should outperform the monolingual setting. However, we believe that cross-lingual steering is at least comparable to the mono-lingual setting in Table 1. For example, to positively steer in Chinese, using English steering words outperforms using Chinese steering words. For the other cases, while monolingual steering leads to larger effects, it also comes with larger degradation of fluency and relevance. This is not necessarily “better” because this could also be achieved by simply increasing the intervention strength coefficient (which we set to the value in Turner et al. (2024) without case-by-case tuning). Stepping back, we agree with you that the fact that English steering works at all is evidence for the hypothesis.

Questions (minor presentation issues): Thank you for raising these issues! We have clarified/fixed all of them in the revision. Specifically: (1) $M_{LM}$ is meant to abstractly denote an intermediate representation generated by early LM layers (not necessarily just the embedding layer; clarified in L105-106); (2) Equation 1 is indeed a formulation of our hypothesis (clarified in L114-115); (3) from our revision: “By scaffolded, we mean that the shared space can be interpreted to an extent in the dominant data type via the logit lens.” (L42-43); (4) L288 (now L291) actually refers to the second example in Figure 1; we have clarified this.

Thank you so much for your detailed review! We are really happy that you recognize the timeliness and the potential future impact of our work, as well as the thoroughness of our experiments.

Weakness 1 (the expectedness of our results): While there are indeed multimodal training objectives that encourages the alignment of semantically equivalent cross-modal inputs, most commonly in a contrastive manner (e.g., CLIP [4]), these models are not a subject of our study. The multimodal models that we analyzed, LlaVA, Chameleon, and SALMONN, were all trained on a purely next-token prediction objective, without explicit token-level alignment of modalities (and yet we find that this alignment implicitly emerges). In particular, Chameleon was trained on mixed-modal data from the onset of pretraining, allowing the model complete flexibility in its internal representation, and hence we believe the cross-modal representation similarity that we observe is remarkable and not a priori expected. Likewise, in the multilingual and code experiments, there are no explicit training objectives for such alignment either. In the revision, in section C in the appendix, we also added a new set of experiments on an additional data type, arithmetic expressions (e.g., “5=3+2”). We found the same trend where LMs implicitly represents the numbers in English (e.g., “two”), even when there is again no explicit pretraining objective for such alignment. The intervention experiment also works analogously as in the other cases.

Weakness 2 (the relatedness of our two hypotheses): We disagree that our sub-hypothesis 2 is unrelated to the unified representation space. The fact that, for example, both English and Chinese inputs have intermediate representations close to the same English continuation token (sub-hypothesis 2) implies that the intermediate layers are indeed representing semantically similar inputs near one another. We moreover disagree that the findings are largely anticipated---Wendler et al. [5] showed a similar phenomenon (only recently) for a highly synthetic setting---we demonstrate that this persists across more naturalistic text and other types of inputs such as code, images, and audio. Nevertheless, we do agree that a deeper analysis would be insightful. We prioritized confirming our findings on a variety of data types in this work to ensure it is a robust property of models. A more in-depth study using more sophisticated methods would certainly be interesting future work!

Weakness 3 (the discussion of related work): Thank you for suggesting these papers! We did cite Unified-IO [2] as well as its follow-up Unified-IO 2 [6] (L319). We did not compare to [1] because they explicitly trained models to encourage such representation similarity while we analyzed how this similarity organically emerges implicitly through autoregressive pretraining. This is related to our response to Weakness 1 above. But we agree that this is an important point worth clarifying and we added a citation to [1] and a discussion in the Related Work section (section 6; L515-516) in the revision. Regarding “information flow, especially the intermediate representation to input/output space”, we do have a paragraph in our related work section dedicated to “Representation evolution throughout layers”, in particular with many papers in the same line of work as [3]. We have added a citation to [3] too in our revision.

Question 1 (figure caption display issue): We do not see this issue on our end. This could be a PDF rendering issue; using a different PDF viewer may help with this. We have also changed the figure format (PDF -> PNG) hoping to fix this. Could you help check if the issue still exists?

Question 2 (clarification regarding the code examples figure): In most cases, the final layer prediction indeed matches the groundtruth next token (as a good code model would), though not always. Regarding the surprisingness of our results, we emphasize that while having this common cross-modal knowledge is an intuitive representation strategy, there is no a priori reason that models adopt it. We believe it is valuable to investigate this phenomena across a wider set of settings than previously investigated. Furthermore, past work has only “casually discussed” this idea, as written by Reviewer nLRX. It is valuable that we concretely and mathematically defined and tested this hypothesis, some of which is arguably stronger than an abstract notion of shared knowledge, especially the causal intervention experiments.

Thank you for your response, especially for adding the figure description in the main content (rather than the caption) and enhancing the discussion on related work. Based on these improvements, I have raised my soundness score to 3 and my overall rating to 6.