Thank you for your review for it made us realize that we had not been clear about the novelty and significance of the singular tokens.

NB: Action items for the authors are marked with a "+" in the margin!

We point out that even though they are quite few in number, singular tokens could be of extreme operational importance. If we ignore/remove/substitute the singularities to "force" the token subspace to a manifold, we risk losing our ability to understand the behavioral features of the entire model.

Indeed, after our present submission, this paper [Zhao et al, One Token to Fool LLM-as-a-Judge, arXiv:2507.08794, 2025] explains that some very specific tokens correspond to major vulnerabilities in LLMs. These single-token attacks are becoming more widely known and prevalent [Cui et al, Token-Efficient Prompt Injection Attack: Provoking Cessation in LLM Reasoning via Adaptive Token Compression, arXiv:2504.20493, 2025], [Lakshmanan, "New TokenBreak Attack Bypasses AI Moderation with Single-Character Text Changes", published on The Hacker News website, June 12, 2025], [Zilberman, "Unicode Exploits Are Compromising Application Security", published on the Prompt.Security website, April 30, 2025].

It is more than plausible that singularities might be the root cause of these vulnerabilities, and possibly the cause of other vulnerabilities not yet discovered. (That's future work!)

+We will mention the above in our discussion Section 4.

You are also correct that in a variety of settings, estimates of local dimension (which, depending on the author, has been called "intrinsic dimension"="ID", "local dimension", among various other terms) can vary quite a bit. Also, you are correct that this variability has been reported by a number of others. Our submission cites a few of these already [Fefferman 2018, Gromov 2024, Tulchinskii 2023, Robinson 2024], and the paper that you list is indeed another one of those. Thanks for that!

+We will include the paper you cite in our list of citations, mentioning it in Section 1.2

+Also, since writing we found another good summary to mention in Section 1.2: [Schweinhart, Fractal dimension and the persistent homology of random geometric complexes, Advances in Mathematics, vol. 372, 2020]; we can include that paper in our citations to benefit the enterprising reader.

Mathematically, a manifold has a singular dimension (pun intended), so the variability of dimension estimates is emblematic of manifold violation. If the estimates of dimension were concentrated with a small variance around an integer, then we might conclude that the estimates were simply noisy. Our results shown in Fig 2 and Table 1 show that this is not the case.

Note that the variance is across all tokens for which intrinsic dimension can be estimated, which is a quite large number of them, so these estimates are reliable. This is not in contradiction with the fact that for some of tokens, dimension cannot be estimated at all.

So there are two ways our test has shown a violation of the manifold hypothesis. The large distribution of local dimension (as seen in the histogram in Fig 2 and Table 1) rejects that the token embedding is globally a manifold. This does not preclude that the embedding space is a disjoint union of smooth manifolds of different dimensions (which would still be problematic since LLMs leverage mathematical machinery such as inner products). Our test also rejects this possibility because we find tokens where a local dimension is simply not well defined. This implies something deeper like pinch points that link spaces of varied dimension. So in the lollipop example, the link point between the candy and the stick is singular because if we are at that point and look left, we appear to be in 1D while if we look right, we appear to be in 3D. The inner product that is being leveraged is on the extrinsic space and not the actual embedding space, which implies that the model is not operating on the token we would have wanted/expected. In fact, an inner product is not defined at this point (at least not in the usual way) and so we would have to characterize the singular point and find the correct mathematical machinery (if it exists).

Moreover, in response to Q1, we do report the variability local dimension estimates in two places. The histogram in the lower frame of Figure 2 shows the distribution of local dimension estimates for GPT-2. Table 1 shows quartiles for the empirical distribution of local dimensions for each of the models we tested. The fact that the interquartile range is large suggests that the local dimension estimates do indeed vary substantially.

+We will rewrite Figure 2's caption to call out the fact that it is the distribution of dimension estimates, at least for the tokens whose dimensions can be estimated.

Specifically, our primary/novel contribution is a test, not an estimator. Our test asks whether there is a single reasonable estimate for dimension at each token. If our test fails, then there is at least one token for which a single dimension does not make sense. If our test fails, then any and all estimators (including the one that the referee cites) are not valid for use at that token, because the assumed relationship between volume and radius is not log-log linear.

+We will insert the above two paragraphs into Section 1.1 so that there is no confusion

There are a number of cases where the community implicitly assumes the data lies on a manifold, of the kind that formally satisfies the mathematical definition.

+For avoidance of doubt, we will include the formal definition of a manifold in our paper, along with quite a few others that we mistakenly left out. (See our response to Reviewer XXVi)

Here are a few examples of when the manifold hypothesis is (strongly) assumed...using:

1. Inner products (which, in fact, requires a Riemannian manifold)
2. Cosine Similarity (which is just inner products on n-spheres which has even more assumptions about curvature especially negative curvature)
3. Manifold learning (Any sort of dimensionality reduction, etc)

The key takeaway is that there are mathematical machinery for singular spaces...in practice, the community simply doesn't say the word "manifold" because we have always implicitly assumed we have a manifold, and often assumes a convex one at that. And so, an implication of our work is that there might be straightforward explanations when the model behaves erratically since we are using mathematical tools when we ought not. Without a Riemannian manifold as the embedding, it should not be surprising that applying inner products (loosely speaking) will generate unstable results.

It is very much the case that samples drawn from a manifold under noisy conditions are also the norm. That provides no issues for our results, as it is the formal setting of our paper, as noise is (explicitly!) incorporated into the volume form. Indeed, this is why we use the standard hypothesis testing framework -- to handle noise in a statistically proper way.

In the case of common manifolds, such as a sphere or torus, our manifold test never fails. In an earlier draft of the paper, we had included "toy" examples of our test being run on "noisy" manifolds and stratified manifolds. However, we had to cut these examples out of the draft due to space constraints...

+We will definitely include these examples (see below for some details) in our final version!

The example of a stratified manifold is perhaps most important for the overall importance of our story. As a simple example, consider a space that has the shape of an "X", like {(x,y) : |x| = |y|} in the 2D plane. This space is not a manifold because it has exactly one singularity at the origin (0,0). Local dimension at every other point is 1, but at the origin, the space does not have a well-defined dimension. One can certainly remove the singularity to obtain a 1-dimensional manifold (with four connected components).

Another example we plan to include is a "lollipop": a candy ball (hollow sphere) attached on the end of a short stick. This shape has exactly two important singular points. On the interior of the stick, the space is a 1-dimensional manifold. At the very end of the stick, though, is a singular point. Where the stick attaches to the "candy" part is another singularity; it is a transition from the 1-dimensional stick to the 3-dimensional candy part.

However, removing the singularity, even though it accounts for 0% of all points in the space, destroys the entire structure of the space! The case of the token subspace is similar; even though the singularities constitute a vanishingly small number of tokens compared to the total vocabulary, they might be similarly important in interpreting the global structure of the token subspace.

Thank you for your response!

Regarding Q4: This is a misunderstanding of what we did. Our analysis is run on the set of tokens only. No sentences (or token streams) were included. Running "sentences" involves passing the token stream through the transformer blocks. That is upstream of the analysis we did. Deleting the tokens from the model and then exploring its behavior is an interesting experiment, though is (a) future work and (b) not directly related to the experiments in our paper.

Regarding Q3: We have not, but can for our revision, run this analysis. In principle, this does not impact the results because of Theorem 1, but in practice we agree that it might have an effect due to sampling error.

Finally and most importantly, about the "small number" of singularities:

We agree: a lollipop is not a manifold! Depending on exactly how it is defined, it could a stratified manifold. It is the very fact that there are two singular points which implies that it is not a manifold. And yes, a local dimension (which is not constant) can be defined almost everywhere.

Yes, you can delete the two singular points, and obtain a disconnected, two component manifold. However, we argue that the fact that the two components remain in close proximity to one another is significant, and moreover this is what our test finds. Indeed -- it is unfortunate that we cannot include figures in our response -- in the figure of the results of our test run on said lollipop,

1. The exact points of singularity are not actually present, since the data are sampled, yet
2. You can see the p-values of our test decreasing in the neighborhood of these two singular points, with deep minima (reject the test!) right where those points are. As a result, we get a handful of rejections near those singularities, depending on exactly what alpha you choose for significance.

So in short, even if the singular points are deleted, their geometric presence still has a (statistically) significant impact on the token subspace!

Moreover, the singular points are highly likely to have an impact on the behavior of the LLM as a whole, because according to our Theorem 2, they will propagate into the behavior of the model (ie., the transformer blocks that we did not study in this paper; see our response to Q4 above).

Regarding the number of singular tokens, perhaps an (imperfect) analogy might be helpful in this discussion. The human genome has an estimated 20k-25k genes, and each gene encodes for one or more gene products. Across the population, mutations accumulate across and within each gene: some are innocuous, others might be beneficial, while many are deleterious. The statement that 70 irregular tokens do not meet a critical mass to warrant novelty/importance assumes that there is some sort of additive effect on irregular tokens that would be inconsequential on a per token level. This excludes the situation where even a single mutation/irregularity can have catastrophic consequences (for instance, a mutation to the BCRA1/2 gene increases the probability of cancer(s) by 60% to 90%). This also elicits the hub and spoke model from network theory where certain nodes are highly connected and extremely consequential.

We believe that our work is complementary to the works that you have referenced (finding singular tokens contrasted with identifying regions of regularity / test vs estimator). Our work offers a new avenue of study for a previously overlooked structure (the irregular/singular tokens) to understand the magnitude of its impact on the global token embedding. We agree that there are ways to account for tokens with different intrinsic dimensions, but we stress that this is not currently implemented within the transformer models where the Q and K matrices return pairwise similarities based on normalized inner products (a structure that is being widely adopted into almost every new SOTA model). Our method further annotates a classification of tokens that do not have a well defined dimension which, to our knowledge, is not currently being done.

We have obtained initial results showing the persistence of singular tokens within current transformers. We have, however, attempted to do some more exploratory data analysis on the singular tokens, and we will send an update to all of the reviewers in a subsequent note.

Until we fully understand the nature of the singular token(s), we respectfully disagree with the statement that the number of irregularities "are too small to be relevant".

In response to your (and another reviewers') concern over the scarcity of singular tokens, we attempted a quick and high-level analysis of these tokens relative to the other tokens. For this experiment, we performed an exploratory data analysis:

We probed the Llemma-7B model with prompts consisting of a single token (preceded by the start token which is default for all Llemma prompts), as our work focuses entirely on the initial embedding and not the final embedding. This single token was varied over several cases:

• Case 1: Llemma's singular tokens where the manifold hypothesis is rejected for small radii. Total = 21.
• Case 2: Llemma's singular tokens where the manifold hypothesis is rejected for large radii. Total = 12.
• Case 3: A random sample of Llemma's nonsingular tokens. We picked sample size = 33 to match the total number of singular tokens.
• Case 4: All of Llemma's nonsingular tokens. Total = V - 33 = 31983, where V = 32016 is the vocabulary size.

Since the sample in Case 3 is stochastic, we sampled 3 different times (we did not do more due to the time this takes to run over each token on our compute system; we cannot leverage cloud resources at the moment but could in the future).

For each such prompt, we asked the model to generate output tokens under the greedy decoding scheme.

Note that the internal layers of the model start from the token embeddings and eventually produce a probability distribution over the V = 32016 tokens at the final layer of the model. The token with the highest probability becomes the 'next' token that the model outputs. We conducted two analyses on these probabilities with a goal to quantify model behavior.

In the first analysis, we collected the probability of the 'next' token. We aggregated these for all possible prompts in each Case outlined above, and looked at the maximum. The results are:

• Case 1: 21 'small radii rejected' singular tokens: Max next token probability = 0.1268
• Case 2: 12 'large radii rejected' singular tokens: Max next token probability = 0.7207
• Case 3a: Sample of 33 nonsingular tokens: Max next token probability = 0.7852
• Case 3b: Sample of 33 nonsingular tokens: Max next token probability = 0.934
• Case 3c: Sample of 33 nonsingular tokens: Max next token probability = 0.8255
• Case 4: All 31983 nonsingular tokens: Max next token probability = 0.9942

The key takeaway from these results is that the maximum next token probability is only 0.1268 for Case 1. Thus, for ALL the singular tokens where the manifold hypothesis is rejected for small radii, when the model is prompted with them, it has very low confidence in its next token. One perspective is that it is very “confused” (inconclusive for the eyeball test for singular tokens from the large radius).

In the second analysis, we computed the entropy of the 'next' token probability distribution (over V = 32016 outcomes) at the final layer. We aggregated these for all possible prompts in each case outlined above and looked at the minimum. The results are:

• Case 1: 21 'small radii rejected' singular tokens: Min entropy of next token distribution = 5.6272
• Case 2: 12 'large radii rejected' singular tokens: Min entropy of next token distribution = 2.16
• Case 3a: Sample of 33 nonsingular tokens: Min entropy of next token distribution = 1.6605
• Case 3b: Sample of 33 nonsingular tokens: Min entropy of next token distribution = 0.492
• Case 3c: Sample of 33 nonsingular tokens: Min entropy of next token distribution = 1.2531
• Case 4: All 31983 nonsingular tokens: Min entropy of next token distribution = 0.0598

The key takeaway from these (quick and dirty) results is that the minimum entropy is as high as 5.6272 for Case 1, which is significantly higher than the nonsingular token cases. Thus, for ALL the singular tokens where the manifold hypothesis is rejected for small radii, when the model is prompted with them, its next token probability distribution is much closer to being uniform as compared to other cases where prompts contain nonsingular tokens (or singular tokens where the manifold hypothesis is rejected for large radii).

We stress that this is a quick turnaround exploratory data analysis on these points. We could do this analysis across all of the LLMs in our draft, but we would prefer this to be future work so as to be able to design the experiments with more intentionality. We will leverage the advice from Reviewer TdY4 on embeddings studied in control settings that will allow us to calibrate these experiments. We don’t really think that there are any trivial/easy experiments since there are many levels of confounders that exist.

Thanks very much for you detailed responses and for your extra work.

Regarding Q4: "This is a misunderstanding of what we did. Our analysis is run on the set of tokens only. No sentences (or token streams) were included."

The misunderstanding was generated by the following sentences in your manuscript: “Given these differences, prompts containing irregular tokens will likely produce dissimilar outputs across the four LLMs” “prompts containing irregular tokens within an LLM will likely be unstable and produce highly variable outputs with replicated queries.” “As we showed in Section 3.2, singularities in the token subspace persistently lead to singularities in the space of token sequences,” Prompts are token sequences, and you explicitly mention that the singularities you observe in the tokens propagate to token sequences. Even in the abstract you explicitly mention the effect of "singular" tokens on the stability of the reply to prompts. My question was aimed at understanding what changes by removing the few "singular" tokens. Even the last test you presented are based on next token prediction task, which implies the usage of a prompt, and you even look at what happens in the last layer (thus very far from the embedding space).

Citing again your manuscript, "In many—if not most—AI research papers, a manifold hypothesis is tacitly assumed, that the data are concentrated near a low curvature manifold without boundary." and you seem to provide evidence that this is not appropriate. However, most of the analysis in the literature is performed in deep layers, and for representations generated by sentences, not single tokens. It is perfectly fine analysing the properties of the token embedding space as you are doing, but then you should be clear that the consequences of your analysis are restricted to this space. If you are convinced that singularities propagate in deep layers, you should provide a specific numerical evidence. With the results you have so far I think that the sentences suggesting that the manifold hypothesis is inappropriate and the three sentences I mentioned above might create confusion in the reader, as they did in my case.

Let me add one more time also my main point: in most of the non-linear analysis methods I know (for example in Umap) it is not necessary assuming that the data are contained in a manifold globally and at the large scale. Quoting my own first report "nobody ever claimed that this manifold is a clean and smooth "Riemanian" variety." Most non-linear methods do work perfectly on the lollypop.

Thank you for your review and positive encouragement for the findings in our work.

NB: Action items for the authors are marked with a "+" in the margin!

That said, I acknowledge that I may have missed or misunderstood certain aspects of the paper, and my confidence in this assessment is low. I would be open to revising my recommendation if the authors are able to address the concerns outlined below.

Let us lead off by saying that we are happy to engage in discussion!

Mathematically, a manifold has a singular dimension (pun intended), so the variability of dimension estimates is emblematic of manifold violation. If the estimates of dimension were concentrated with a small variance around an integer, then we might conclude that the estimates were simply noisy. Our results shown in Fig 2 and Table 1 show that this is not the case.

Note that the variance is across all tokens for which intrinsic dimension can be estimated, which is a quite large number of them, so these estimates are reliable. This is not in contradiction with the fact that for some of tokens, dimension cannot be estimated at all.

So there are two ways our test has shown a violation of the manifold hypothesis. The large distribution of local dimension (as seen in the histogram in Fig 2 and Table 1) rejects that the token embedding is globally a manifold. This does not preclude that the embedding space is a disjoint union of smooth manifolds of different dimensions (which would still be problematic since LLMs leverage mathematical machinery such as inner products). Our test also rejects this possibility because we find tokens where a local dimension is simply not well defined. This implies something deeper like pinch points that link spaces of varied dimension. Imagine glueing a line segment to a ball (lollipop); the link point does not have a well-defined dim because it joins 1D to 3D. The inner product that is being leveraged at this link is on the extrinsic space and not the actual embedding space, which implies that the model is not operating on the token we would have wanted/expected. In fact, an inner product is not defined at this point (at least not in the usual way) and so we would have to characterize the singular point and find the correct mathematical machinery (if it exists).

The organization and clarity of presentation are lacking

+We can bring material from A.3 and A.4 up into the main body Sec 2 without too much effort, especially because the final version permits an extra page. (Do also have a look at our response to reviewer MkzF, wherein we have committed to a detailed revision plan.)

For example, Fig 3 is said to encapsulate the core methodology, yet its interpretation is non-trivial without additional context. The necessary intuition is relegated to the appendix, where it is unlikely to be seen early by readers. I strongly recommend relocating this explanation to the main text and integrating it into the broader narrative.

That is a great idea. Thanks! We will also incorporate some text like what we say below (concrete actions marked)...

Volume Estimation in Fig 3b It is unclear how volume is defined or computed in Fig 3b. Lines 120–125 suggest that it may correspond to the number of tokens within a ball. If this is correct, the authors appear to be using token count as a proxy for manifold volume. This is a strong and potentially unjustified assumption.

We are treating the tokens as a sample from the assumed space (either a manifold or a fiber bundle, depending on the test). We are estimating the volume of the ball of a given radius around a given token by counting the number of tokens within that radius. This is nothing other than a classic Monte Carlo estimate of volume. It is a strong assumption, and is inherent in the "volume form" in the statements of our Theorem 1, but it is also extremely established in the literature.

+We will include the above paragraph (suitably edited) in Sec 2

Slope Thresholding The hypothesis tests seem to hinge on detecting slope changes in volume scaling. Does this involve setting a specific threshold? If so, how was this threshold determined? Why was this particular value chosen over others? The paper should clarify the methodology for threshold selection and discuss its sensitivity.

Thank you for your careful read of the paper!

For the manifold test, which is a little simpler, we collected dimension estimates using the standard 3-point centered difference estimate via numpy.gradient() at each radius under test. These dimension estimates were then binned into a pair of adjacent sliding windows. The estimates in neighboring bins were compared using the standard unpaired two-sample T-test with the stated alpha level for decision. This test yields a Bonferroni-corrected p-value for the token. If the p value is less than alpha, then there is not a well-defined dimension for that token, so the manifold test fails at that token.

+We will include the above paragraph (suitably edited) in Sec 2

The authors’ conclusion---that LLM embeddings do not lie on a single coherent manifold---is intriguing, but more granular analysis is needed.

That is a fascinating idea, but really ought to be future work. We need to systematically collect data to conduct such an experiment. Thank you for the suggestion!

+We can list your suggestion as future work in Sec 4.

Exploring such structured subsets could strengthen the argument and help determine whether the negative results reflect global structure or just aggregate noise.

They are not multiple runs, as the entire process in our paper is deterministic. What we are presenting is an analytic tool for pre-trained models, that moreover can be applied to any point cloud within a latent space. In other words, the token embedding vectors are fixed when we analyze the model, they do not change from run to run.

A natural question is therefore if we can find a token embedding that is everywhere regular. One way to approach an answer is to view the number of tokens in a vocabulary as a hyperparameter. There might be a certain vocabulary size that minimizes the number of singularities given the corpus, architecture, etc. Even the best vocabulary might lead to a space that could still be singular, for reasons inspired by [Jakubowski et al, Topology of Word Embeddings: Singularities Reflect Polysemy, in Proceedings of the Ninth Joint Conference on Lexical and Computational Semantics, 2020]. Let's take the * symbol, which happens to be singular for Mistral 7B (See Table 7, number 29). In natural language, it has a very distinctive meaning in terms of calling out footnotes, while in other contexts it can be used for marking text as bold in Markdown, or in programming it has the mathematical meaning of multiplication. Therefore, this character connects these different semantics: i.e. natural language with programming languages and markdown. Suppose that each language were in fact well-described by a manifold. Then these manifolds would be joined together in the entire latent space at the * (and likely other points). As a result, this token with high likelihood would be a singularity because the dimensions for different "languages" are almost surely different [Tulchinskii et al, Intrinsic Dimension Estimation for Robust Detection of AI-Generated Texts, arxiv:2306.04723, 2023].

+We can include the above discussion in the introduction (space permitting) or Discussion Sec 4.

+We will also mention something about your question about randomness in our Limitations Sec 4.1 (since although it is not a limitation, the Limitations section is where readers with similar questions might look.)

The empirical evaluation is limited to LLM embeddings. However, other widely used embedding techniques, such as Word2Vec and GloVe, also produce token representations. Applying the proposed tests to these methods could provide important comparative insights and help clarify whether the results are specific to LLMs or generalizable across embedding strategies.

This is a fair point, and it's entirely sensible to run our test(s) on these other embeddings. The method applies to any point cloud in Euclidean space, and LLMs are mostly a convenient test case. We opted for LLM embeddings because they are currently the most widely deployed models where we could easily make some high level comparisons. Comparing across model embeddings is complicated by potential confounding effects, e.g. corpus, cost function, etc. This is certainly future work.

The paper lacks validation of the proposed tests on synthetic data. For example, if the authors apply their methodology to points sampled from known manifolds (e.g., spheres, tori), do the tests recover the expected structure? How do noise and sampling density affect test accuracy? Exploring such controlled settings would provide valuable evidence for the tests’ robustness and reliability.

Regarding a synthetic example, in an earlier draft of the paper, we had included "toy" examples of our test being run on "noisy" manifolds and stratified manifolds. However, we had to cut these examples out of the submission due to space constraints...

+we will include these examples in our final version!

Since we are not permitted to include graphics in our rebuttals, for the purpose of visualization, one example we plan to include is the "lollipop": a candy ball attached on the end of a short stick (mentioned before). This shape has EXACTLY two singular points. On the interior of stick, the space is a 1-dimensional manifold. The end of the stick is a singular point/boundary. Where the stick attaches to the "candy" is another singularity; it is a transition from the 1-dimensional stick to the 3-dimensional candy.

Equation Formatting: Equation (1) contains an unclosed parenthesis:

+Oops. Thanks for catching that!

Thank you once again for your review and additional engagement with our response, we appreciate it.

Additionally, please note that in response to some reviewers' comments about the number of singular tokens, we attempted a quick exploratory data analysis of singular tokens relative to the other tokens. In case you are interested to see the results, they are in our responses (i.e. official comments) to Reviewers ZHkg and XXVi.

Thank you for your review and pointing out that we were not clear on the novelty and significance of the singular tokens.

NB: Action items for the authors are marked with a "+" in the margin!

I would like to continue this conversation to clarify some of the points in the rebuttal phase.

Great! We are delighted to discuss this with you! We appreciate your effort in reading our paper.

+We will define in Section 1.2

• manifold
• singularity (and separately "singular")
• embedding
• embedded reach
• volume form
• radius
• fibered manifold
• fiber bundle
• p-value

+In the appendix

• curvature
• pushforward
• homeomorphism
• diffeomorphism

Additionally, some theoretical results, particularly Thm 1 and Thm 2, are introduced and referenced before they are clearly stated.

+We will move up some of the material from A.3 and A.4 into Sec 2 so that that is rather more clear. (see also the response to Reviewer TdY4)

If interpreted table1 correctly, fewer than 70 tokens (out of ~30K–50K vocabulary) ...

We point out that even though they are quite few in number, these singular tokens could be of extreme operational importance. Individual tokens are already known to be the root cause of certain vulnerabilities. For instance, [arXiv:2507.08794] explains that some very specific tokens correspond to major vulnerabilities in LLMs (called the "master-key" in unlocking LLM prompt injections). These single-token attacks are becoming more widely known and prevalent [arXiv:2504.20493], [Lakshmanan, "New TokenBreak Attack Bypasses AI Moderation with Single-Character Text Changes", 2025], [Zilberman, "Unicode Exploits Are Compromising Application Security", 2025].

It is plausible that singularities might be the root cause of these vulnerabilies. Moreover, these singular tokens might be the cause of other vulnerabilities not yet discovered. (That's future work!) We would argue that since the singular tokens are distinctly different than the others, and that there are not that many of them, finding the singular tokens is potentially of great value to be community.

+We will insert the above text (after editing) into the Introduction Sec 1 or the Discussion Sec 4.

While the authors argue these irregularities are meaningful, it's unclear whether these patterns are consistent across training runs

Note that the entire process in our paper is deterministic, so there is no requirement of multiple runs. What we are presenting is an analytic tool (hypothesis test) for pre-trained models, that moreover can be applied to any point cloud within a latent space. Note that the models we are considering are published/deployed models, where the token embedding vectors are frozen after training. In other words, the token embedding vectors are fixed when we analyze the model, they do not change from run to run.

+We will add the above text (suitably edited) into the Background Sec 1.2, and reiterate it in our Limitations Sec 4.1 (since although it is not a limitation, the Limitations section is where readers with similar questions might look.)

Q1

We have actually never found this discussed in the literature at all. Indeed, we found (and already cited) only a few papers that mention singularities at all. What we are saying is that we suspect one of the motivations for the trend toward increasing context window size might be a response to the presence of singularities.

We also stress that our work shows that if the initial token embedding contains singular tokens, they will persist in the final embedding regardless of the number of tokens (i.e. the context window size) by Thm 2. Our result is not a contradiction of prior work but simply a result that stands on its own.

+Assuming the above paragraph clarifies your question, we could add it to the Discussion Sec 4.

Q2

+Fig 2 bottom is the histogram of the estimates of dimensions for GPT2 (=slopes of the log radius versus log volume plots), at least for when the tokens have a well-defined dimension.

+We will update the caption of Fig 2 to say the above explicitly.

So there are two ways our test has shown a violation of the manifold hypothesis. The large distribution of local dimension (as seen in the histogram in Fig 2 and Table 1) rejects that the token embedding is globally a manifold. This does not preclude that the embedding space is a disjoint union of smooth manifolds of different dimensions (which would still be problematic since LLMs leverage mathematical machinery such as inner products). Our test also rejects this possibility because we find tokens where a local dimension is simply not well defined (these are our highlighted singular tokens). This implies something deeper like pinch points that link spaces of varied dimension. Imagine glueing a line segment to a ball (lollipop); the link point does not have a well-defined dim because it joins 1D to 3D. The inner product that is being leveraged at this link is on the extrinsic space and not the actual embedding space, which implies that the model is not operating on the token we would have wanted/expected. In fact, an inner product is not defined at this point (at least not in the usual way) and so we would have to characterize the singular point and find the correct mathematical machinery (if it exists).

Q3

While there might be good alignment between models over much of the token subspace (which would account for the broad geometric similarities), there simply cannot be alignment between the singular tokens if the singular tokens differ or if the dimensions differ. Since we show (Tables 3-9) that the singular tokens between the four models we studied differ, there cannot be alignment between these models in a fully global sense.

Some references are Lee et al 2024 (Shared Global and Local Geometry of Language Model Embeddings)

The authors of this paper show a few things that are consistent with our findings:

+We will discuss this paper explicitly in Sec 1.2, as below:

1. Language models often construct similar local representations as constructed from local linear embeddings

The "often" part here is important. Our finding is that most of the tokens in the vocabulary fail to reject the manifold hypothesis, which means that we are able to estimate some local structure. However, for the singular tokens this is simply impossible without deep investigation of the local neighborhoods. In [Lee 2024], they are not looking at the singular tokens in the way that we are here.

2. Token embeddings exhibit low intrinsic dimensions

This agrees with our findings in Table 1, at least in the lower radius regime. Their estimates of dimension for GPT2 is within the confidence interval we computed. They do not recognize that there might be a fiber bundle structure in play, which is our "larger radius regime".

3. Tokens with lower intrinsic dimensions form more semantically coherent clusters.

We agree, though since they did not perform a test for whether a given token is singular, we are concerned that they may have estimated dimensions for tokens that do not have a well-defined dimension!

Nevertheless, Figure 1 seems fairly consistent with their findings.

4. Tokens have similar dimensions across language models

The hypothesis that "Tokens have similar dimensions across language models" is a null hypothesis. It can be rejected (simply by exhibit two models whose dimensions disagree), but never accepted as true since all models cannot be tested. Our Table 1 shows that this hypothesis must be rejected. It is not that the dimensions estimated by [Lee 2024] are wrong. We and they both looked at four models (with one in common, GPT2). It is highly likely that they might have--by chance alone--picked models that are similar to one another and thereby did not find that their dimensions could differ.

Zhao et. al 2024 (Implicit geometry of Next Token Prediction)

This paper is about the transformer structure (our Section 3.2), but not so much about the token embeddings explicitly, so it is less directly relevant to our present submission, but see below.

+We will include the following two paragraphs (perhaps summarized after discussion with you) in Section 1.2.

Specifically, they show that the implicit space of possible activations can be inferred from the behavior of the model. A slice of this implicit structure is the token subspace. We note that in this vein, [Zhao 2024] is not inconsistent with other findings within the literature, for instance [Robinson et al, Probing the topology of the space of tokens with structured prompts, arXiv:2503.15421, 2025]. This latter paper shows that the token subspace can be inferred (up to homeomorphism, aka topological equivalence) from the responses of the LLM to single-token prompts.

Taken together, the above papers and the present one have pretty significant implications. Even though the token subspace may have about 0.1% of tokens that are singular, we suspect that a user will be encountering singular tokens on a regular basis when using LLMs (which seems empirically plausible). Estimating the rate of occurance of a singular token is hard, and very much future work!

However, this matters in two distinct ways:

1. Prompting from the user. If the user hits a singular token, both [Zhao 2024] and [Robinson 2025] indicate that the model's behavior is likely to go badly from that point onward. The current "fix" for this problem is prompt engineering.

2. As part of how they work, the transformers sample the next token, and almost surely as the number of samples gets very large, the LLM will indeed pick a singular token. There is no prompt engineering fix in this case.

Could this all be an artifact of the training data and the next token distribution ?

We are not entirely sure about the specific meaning of your question. Please clarify?

We did not present evidence in our submission that the singular tokens would be problematic as ours is a methods paper and not an analysis paper. Because of your (and others') question on the nature of the singular tokens, we conducted some experiments in an attempt to understand this better in the past few days (we are sending this note to all reviewers):

In response to your (and another reviewers') concern over the scarcity of singular tokens, we attempted a quick and high-level analysis of these tokens relative to the other tokens. For this experiment, we performed an exploratory data analysis:

We probed the Llemma-7B model with prompts consisting of a single token (preceded by the start token which is default for all Llemma prompts), as our work focuses entirely on the initial embedding and not the final embedding. This single token was varied over several cases:

• Case 1: Llemma's singular tokens where the manifold hypothesis is rejected for small radii. Total = 21.
• Case 2: Llemma's singular tokens where the manifold hypothesis is rejected for large radii. Total = 12.
• Case 3: A random sample of Llemma's nonsingular tokens. We picked sample size = 33 to match the total number of singular tokens.
• Case 4: All of Llemma's nonsingular tokens. Total = V - 33 = 31983, where V = 32016 is the vocabulary size.

Since the sample in Case 3 is stochastic, we sampled 3 different times (we did not do more due to the time this takes to run over each token on our compute system; we cannot leverage cloud resources at the moment but could in the future).

For each such prompt, we asked the model to generate output tokens under the greedy decoding scheme.

Note that the internal layers of the model start from the token embeddings and eventually produce a probability distribution over the V = 32016 tokens at the final layer of the model. The token with the highest probability becomes the 'next' token that the model outputs. We conducted two analyses on these probabilities with a goal to quantify model behavior.

In the first analysis, we collected the probability of the 'next' token. We aggregated these for all possible prompts in each Case outlined above, and looked at the maximum. The results are:

• Case 1: 21 'small radii rejected' singular tokens: Max next token probability = 0.1268
• Case 2: 12 'large radii rejected' singular tokens: Max next token probability = 0.7207
• Case 3a: Sample of 33 nonsingular tokens: Max next token probability = 0.7852
• Case 3b: Sample of 33 nonsingular tokens: Max next token probability = 0.934
• Case 3c: Sample of 33 nonsingular tokens: Max next token probability = 0.8255
• Case 4: All 31983 nonsingular tokens: Max next token probability = 0.9942

The key takeaway from these results is that the maximum next token probability is only 0.1268 for Case 1. Thus, for ALL the singular tokens where the manifold hypothesis is rejected for small radii, when the model is prompted with them, it has very low confidence in its next token. One perspective is that it is very “confused” (inconclusive for the eyeball test for singular tokens from the large radius).

In the second analysis, we computed the entropy of the 'next' token probability distribution (over V = 32016 outcomes) at the final layer. We aggregated these for all possible prompts in each case outlined above and looked at the minimum. The results are:

• Case 1: 21 'small radii rejected' singular tokens: Min entropy of next token distribution = 5.6272
• Case 2: 12 'large radii rejected' singular tokens: Min entropy of next token distribution = 2.16
• Case 3a: Sample of 33 nonsingular tokens: Min entropy of next token distribution = 1.6605
• Case 3b: Sample of 33 nonsingular tokens: Min entropy of next token distribution = 0.492
• Case 3c: Sample of 33 nonsingular tokens: Min entropy of next token distribution = 1.2531
• Case 4: All 31983 nonsingular tokens: Min entropy of next token distribution = 0.0598

The key takeaway from these (quick and dirty) results is that the minimum entropy is as high as 5.6272 for Case 1, which is significantly higher than the nonsingular token cases. Thus, for ALL the singular tokens where the manifold hypothesis is rejected for small radii, when the model is prompted with them, its next token probability distribution is much closer to being uniform as compared to other cases where prompts contain nonsingular tokens (or singular tokens where the manifold hypothesis is rejected for large radii).

We stress that this is a quick turnaround exploratory data analysis on these points. We could do this analysis for all LLMs in our draft, but we would prefer this to be future work so as to be able to design the experiments with more intentionality. We will leverage the advice from Reviewer TdY4 on embeddings studied in control settings that will allow us to calibrate these experiments. We don’t really think that there are any trivial/easy experiments since there are many levels of confounders that exist.

Since the official response period has been extended by two days, there is no apologies needed for not replying until now. We are obliged for your feedback.

Thank you for the clarification on the artifacts of randomness. There juxtaposition of “artifact” with “randomness” confused us as work in error modeling can usually be decomposed into random noise (stochastic errors) or artifacts (systematic bias). While we have not done any principled analysis into this questions, we can do some thought experiments to inform our intuition. If we think about training an LLM as an experiment, then the experimental design/setup/conditions severely biases the outcomes with artifacts. These systematic effects are unique to each experiment; in fact, they are unique to batches of the experiment (the batch correction problem severely limits the analysis of molecular data). This would certainly suggest different training corpuses, architecture, etc. would drive systemic differences between models, and token embeddings would also likely be biased. Stochasticity, on the other hand, is a bit more tricky to intuit. If we could potentially fix all other aspects of training, there is still a stochastic component to the non-convex optimization -- the solution to which you obtain for any given training run. One could imagine that embeddings are influenced by the local geometry of the zero set of the cost function. In reality, the experimental design (architecture, corpus, cost function, hyperparameter choices, regularizers, etc.) influences/determines the loss landscape. So yes, we do think that the nature of the experiment would impact the geometry of the token embeddings where both the fixed effects (experimental design) as well as the random effects (stochastic variability) play a role in the learning of an irregular/singular embedding space.

As for question 1, our work shows that adding more tokens cannot help to reduce singularities if they are already present (Theorem 2). This means that the singularities will persist across the network and into the final embedding regardless of adding more tokens (or equivalently, adding context).

This histogram in Figure 2 shows the distribution of the per token estimate for the intrinsic dimension within the token embedding. It shows that the local dimension for each token (when it is well defined) varies considerably. This variability itself violates the manifold hypothesis as it implies that the token embedding does not have a unique dimension (which a manifold must have by definition). The singular tokens are not represented in this histogram as they don’t have a well defined dimension. For instance, we can take a line segment and glue this to a solid sphere. The junction point does not have a well defined local dimension because on one side it is 1D but on the other side, it is 3D (so we cannot define local charts, etc). This is the lollipop stratified manifold toy example.

Thank you very much for the extremely detailed review! We should be able to implement each of your suggestions without any trouble at all, as detailed below.

We agree that the exposition could have been more organized. We were struggling with the page limit and hastily moved some of the explanatory material to the supplement.

NB: Action items for the authors are marked with a "+" in the margin!

+We will edit Sec 1.1 as follows:

Manifold test: A 2-sided hypothesis test where the null is defined as H_0: The token $\psi$ is sampled from a manifold embedded within $\mathbb{R}^\ell$ with reach $\tau$. and the alternative is defined as H_1: The token $\psi$ is not sampled from such a manifold (ie. we reject the null if there is any change in the slope of the log volume v. log radius curve)

Two runs for the fiber bundle test: a 1-sided test for the following hypothesis: H0': The token $\psi$ is sampled from a fibered manifold with boundary that is embedded within $\mathbb{R}^\ell$ with reach $\tau$. H1': The tokens are not such a sample (ie. we reject the null if there is a slope INCREASE in the slope of the log volume v. log radius curve) We performed the volume v. radius estimation twice: once for larger radii, and once for smaller radii.

For almost all the tokens, we can estimate a dimension. But for the remaining tokens--those that cause the hypotheses above to be violated--the dimension at that token is not well-defined at all!

Your request for a visual schematic of what our test does is a great idea! Various schematic figures were included in an earlier draft of the paper, where we had included "toy" examples of our test being run on "noisy" manifolds and stratified manifolds. However, we had to cut these examples out of the submission due to space constraints.

+we will include these examples in our final version!

For the purpose of visualization, think of a "lollipop". This shape has exactly two important singular points. On the interior of the stick, the space is a 1-dimensional manifold. At the very end of the stick, though, is a singular point (the boundary). Where the stick attaches to the "candy" part is another singularity; it is a transition from the 1-dimensional stick to the 3-dimensional candy part. This visual example was not too difficult to construct, so your suggestion of including pictures showing cusp points, pinch points, etc. is easily implemented.

Responses to specific points that you raise in the Major Questions/Comments section of your review:

Please enumerate the contributions ... with reference to the specific parts of the text.

+Sure!

Please consider restructuring Section 2...One suggestion is to merge Sections 2 and 3.1...

+Section 2 indeed needs some help, but your suggestions (along with those of the other reviewers) are very do-able. We will also move material from A.3 and A.4 into Section 2 as well for completeness.

Please consider adding a subsection after line 184, as this is where you begin discussing singularity propagation through the transformer.

+Good idea!

I am aware that it is a non-trivial task to align a notion of a “minimal but sufficient set of definitions” with the expectations of the reader.

+The things that absolutely need to be defined (most in Section 1.2) are:

• manifold
• singularity (and separately "singular")
• embedding
• embedded reach
• volume form
• radius
• fibered manifold
• fiber bundle
• p-value
• curvature
• pushforward
• homeomorphism
• diffeomorphism

Additionally, it would be helpful to formally define cusp point and pinch point (unless they are supposed to be used informally), in which case I would appreciate a brief explanation.

We meant these informally, and in that sense, they're best understood visually. The "toy examples" mentioned above can do this easily.

+We'll mention at least the following in our revision.

1. A "cusp point" is something that looks like the graph of $y^3 + x^2 = 0$ at the origin.
2. A "pinch point" looks like the graph of $z^{2}+x^{2}-y^{2}=0$ (check this out on any 3d graphing software!).

Re: Fig 3: I personally found the visuals somewhat difficult to interpret and believe they reduce overall clarity.

+With the "lollipop example" added, we will alter Fig 3(a) so that it is based on that example. It will be considerably clearer

Line 236: How were these general trends obtained?

They were obtained from a manual inspection of Tables 4-9, which are present in the Supplementary Material due to space constraints.

Lines 200–306: Similarly, how were these results derived? Were they also from manual inspection?

Lines 200-206: This follows simply the fact that Tables 2-9 are nonempty; rejections were obtained.
+We will mention this!

Lines 207-214: Inspection of Table 1. The fact that the rejection rates and the quartiles for the dimensions are non-overlapping. +We will mention this!

Lines 215-224: Manual inspection. This should be taken as suggestive. Given that the previous paragraphs established that manifold rejections occur, we are pivoting to understand possible causes. +We will explain that this is less about drawing conclusions about the specific reasons and more about developing the hypotheses/intuition for future study.

Would it be better to include a small experimental section using synthetic manifolds or fiber bundles, where the underlying structure is known by controlled generation?

+We did run such experiments already and can include them in the revision! Specifically, we've run spheres, planes and tori (of various dimensions), and a handful of stratified spaces/fiber bundles.

We can include a toy stratified space to elucidate singular v regular points in the paper and put the rest of the examples into the supplement (and we will include all of the code publicly).

Line 29: I wonder if this statement can be supported by prior work.

We haven't found anything that says this directly, but here are a few examples that are mathematically assuming manifolds:

1. Inner products
2. Cosine Similarity (which is just inner products on n-spheres which has even more assumptions about curvature especially negative curvature)
3. Manifold learning (Any sort of dimensionality reduction, etc)

In reality, the structure of LLMs (token in -> token out) implies that the token space itself is closed under these operations.

Line 59: Define the neighborhood

+This is a subtle typo. It should be "whether any neighborhood of the token contains an irregularity"

Line 71: Should it be “global token embedding space” instead of “global token embedding”?

If we say anything other than "embedding function", it's a typo! In this case, we mean the "global embedding function". +We will check closely!

Figure 1: Is there any particular reason why t-SNE is used for the large-scale view and PCA for the fine-grained structure?

We tried various "looks" and this was the clearest.

Line 99:

+Oops. "In our test case" should be deleted.

The first reference to Figure 1 appears in the text at Line 215...

+Yup. Thanks.

Figure 2: Why does the pipeline at the top not differentiate between small and large radii?

+We just run the process twice. We'll edit Fig 2 accordingly.

Figure 3b: Should the label be “Reject fiber bundle (small radius)” instead of “Reject manifold (small radius)” for the first slope change in the green curve?

No. The first slope change decreases the slope; the fiber bundle test fails if the slope increases.

Figure 3b: The blue curve appears to more closely resemble the top image in Figure 3a, which would suggest rejecting the fiber bundle as the slope increases. Why is that?

No. The top curve on Fig 3(a) has a slope increase through the slope discontinuity, whereas the blue curve in Fig 3(b) has a slope decrease.

Line 142: There is ambiguity in the notation/terminology. Is the embedding or the embedding subspace?

The issue is the grammar. The embedding function is $e$, and the token subspace is $T$. The function $e$ is "embedding" the set $T$ into the latent space. +We'll fix.

Line 145: Am I correct that $\psi$ is used solely as an element of the latent space?

+Yes!

Theorem 1: There is a bracket inconsistency

+Thanks!

Please briefly introduce $T^w$.

+It's a sequence of tokens of length $w$

Line 201: Please check whether the reference is correct.

+Good catch!

Line 218: What does “chain of sausages” mean?

It's an intuitive attempt at capturing the notion of a "pinch point", i.e. two spheres glued together at the north pole of one and the south pole of the other. See above!

Lines 221–224: Consider moving this part to a footnote.

+Sure.

Table 1: Could you please elaborate on why all quartiles for LLaMA-7B are exactly 4096? Is there an explanation for these results?

It appears to be due to quantization. We don't want to speculate too much, but it looks like many of the tokens are exactly equally spaced in some parts of the token subspace, almost like they were "snapped to grid".

Line 257: It is unclear what is meant by “dependency between large and small scale variability.”

Euclidean space is "regular" in the sense that translations are homeomorphisms, so then a singular space cannot exhibit this kind of self-similarity.
+We were being a bit vague and can clarify.

Line 270: It would be helpful to have a formal definition of singularity if this is a specific type of irregularity. ...

This is an entire branch of mathematics!! Our test is not strong enough to make this classification. Maryam Mirzakhani and Heisuke Hironaka got Fields Medals for proving partial results in this direction!

Line 323: Would it be possible to at least list possible post hoc methods and discuss the implications of determining that?

We can speculate, and are happy to do so, but given our response to your query immediately above, it's really, really hard mathematics!

Dear Authors,

Thank you for engaging in the discussion regarding the concerns I raised. Most of the points, both major and minor, have been addressed satisfactorily, and I trust the authors will incorporate these revisions in the next version of the manuscript.

However, one important issue remains unresolved - specifically, Major Comment 6 (Methodological Clarity). I have not found a response to this point, unless it was overlooked. I kindly request that you provide a detailed explanation addressing this concern.

Additional action items:

• For Major Comment 5a (Interpretation of Results, Line 236), the discussion should explicitly reference the supplementary material. Without this, the statement appears disconnected from the main text and lacks proper context.

• Regarding Minor Comment 21, it would be preferable to either provide appropriate citations for the definitions mentioned or clearly state that these definitions are used informally (although this approach is not ideal).

• Regarding Minor Comment 22, similarly, it would be advisable to either add proper citations for the definitions or omit these statements altogether, as their current form introduces further confusion.

Dear Authors,

Thank you for your clarification. I believe your responses (including those to the other reviewers regarding exloratory analysis) have addressed my final concerns. Accordingly, I have increased my overall rating from 4 to 5 and the clarity score from 2 to 3. I wish you the best of luck with your work, regardless of the outcome of this submission.

Thank you for your review and making us think about a topic we had yet to consider...semantic heirarchy.

NB: Action items for the authors are marked with a "+" in the margin!

For our rebuttal, here's a good starting point: For many tokens, we can estimate a dimension, which is at least understandable because manifolds have a well-defined dimension. But for a few tokens--those that cause the hypotheses above to be violated--the dimension at that token is not well-defined at all!

Responses to your specific concerns are below:

As the authors state, if the main point of this paper is the proposal of a "methodology for hypothesis testing on embedding points," then in addition to the proofs of Theorems 1 and 2, experimental validation is needed to demonstrate the validity .... Alternatively, analysis using, for example, synthetic data where the ground truth is known, is also an option.

Your request for a visual schematic of what our test does is a great idea! Various schematic figures were included in an earlier draft of the paper, where we had included "toy" examples of our test being run on "noisy" manifolds and stratified manifolds.

However, we had to cut these examples out of the submission due to space constraints...

+we will include these examples in our final version!

Since we are not permitted to include graphics in our rebuttals, for the purpose of visualization, one example we plan to include is a "lollipop": a candy ball attached on the end of a short stick. This shape has EXACTLY two important singular points. On the interior of the stick, the space is a 1-dimensional manifold. At the very end of the stick, though, is a singular point (the boundary). Where the stick attaches to the "candy" part is another singularity; it is a transition from the 1-dimensional stick to the 3-dimensional candy part. This visual example was not too difficult to construct, so your suggestion of including pictures showing cusp points, pinch points, etc. is easily implemented.

While rejecting regularity does imply singularity, it does not afford us any evidence as to the type of singularity.

Correct. This is an entire branch of mathematics and a complete solution to your question would be a breakthrough of tremendous significance (Maryam Mirzakhani and Heisuke Hironaka got Fields Medals for proving partial results in this direction!). Given the current state of this branch of mathematics, we can offer intuitive explanations for two types of singularities.

(Check these out on any graphing software!)

1. A "cusp point" is a sharp point. It looks like the graph of the function $y^3 + x^2 = 0$ near the origin;
2. A "pinch point" looks like the graph of $z^{2}+x^{2}-y^{2}=0$.

Much of language data often possesses semantic hierarchy within its context, and understanding how this hierarchy is represented in latent space is considered a crucial research topic. I believe Robinson+2025 is one significant prior work in this area; would it be possible to conduct an analysis and discussion regarding hierarchy similar to what is done there?

Could you please clarify which reference you are referring to via "Robinson+2025"? The paper with that citation in our present submission -- [Robinson et al, Probing the topology of the space of tokens with structured prompts, arxiv:2503.15421, 2025] -- does not establish anything hierarchical according to our interpretation. If it does in your interpretation, we would request you to clarify how.

The fact that ontologists organize language into semantic hierarchies suggests that the idea of embedding tokens into a single, common space, might not be the most effective representation. In that vein, it is entirely possible that the appearance of singularities is an indication that the idea of using an embedding representation is ill-conceived.

While an intriguing possibility, an alternative token representation has not been discovered in the literature.

Here is another line of thinking, inspired by [Jakubowski et al, Topology of Word Embeddings: Singularities Reflect Polysemy, in Proceedings of the Ninth Joint Conference on Lexical and Computational Semantics, 2020]. Instead of an hierarchy, it's a link of sausages of many different dimensions. Let's take the * symbol, which happens to be singular for Mistral 7B (See Table 7, number 29). In natural language, it has a very distinctive meaning in terms of calling out footnotes, or marking text as bold in Markdown, but in programming it has mathematical meaning as multiplication. Therefore, this character connects these different senses of natural language with programming languages. Suppose that each domain/language were in fact well-described by a manifold. Then they would be joined/glued together in the entire latent space at the * which with high likelihood would be a singularity because the dimensions for different langauges are different [Tulchinskii et al, Intrinsic Dimension Estimation for Robust Detection of AI-Generated Texts, arxiv:2306.04723, 2023].

So there are two ways our test has shown a violation of the manifold hypothesis. The large distribution of local dimension (as seen in the histogram in Fig 2 and Table 1) rejects that the token embedding is globally a manifold. This does not preclude that the embedding space is a disjoint union of smooth manifolds of different dimensions (which would still be problematic since LLMs leverage mathematical machinery such as inner products). Our test also rejects this possibility because we find tokens where a local dimension is simply not well defined. This implies something deeper like pinch points that link spaces of varied dimension. So in the lollipop example, the link point between the candy and the stick is singular because if we are at that point and look left, we appear to be in 1D while if we look right, we appear to be in 3D. The inner product that is being leveraged is on the extrinsic space and not the actual embedding space, which implies that the model is not operating on the token we would have wanted/expected. In fact, an inner product is not defined at this point (at least not in the usual way) and so we would have to characterize the singular point and find the correct mathematical machinery (if it exists).

The actual code and data are not publicly available. This makes it impossible to quickly verify the effectiveness of this method.

+We will make the code publicly available once released by our government sponsor. There have been some delays due to unforeseen events; however, the sponsor did clear the present submission for release, so we do not anticipate there to be any issues with clearing the code release eventually.

The caption for Table 1 and its explanation in the main text are unclear. A more easily understandable explanation of what each numerical value in Table 1 signifies is needed.

Yes, there is a lot of content in that table.

+In the revision, we can certainly explain it in detail. The following will be inserted into Section 3.3, probably around line 225.

First of all, there are two main tests we run on the data (See lines 62-26): one test against manifolds, and two versions of a test for a fiber bundle

The "rejects" columns give (top rows of each cell) the number of tokens that fail our test. This is also the number of rows in Tables 3-9. The listed p value is the smallest such p-value, which is the overall p-value for the whole test.

The "dim." column shows the quartiles for the estimates of dimensions, at least for the tokens for which a dimension can be reliably be ascribed. Note that the whole point of our test is that for each of the tokens listed in Tables 3-9, no dimension estimate is possible. Quartiles are a nice way to summarize a distribution. Usually

• Q1 = 25-th percentile, so 25% of the dimensions estimated are below this quantity
• Q2 = 50-th percentile, i.e. the median
• Q3 = 75-th percentile, so 25% of the dimensions estimated are above this quantity

NeurIPS guidelines encourage providing a "short proof sketch for intuition" in the main paper when proofs are included in the supplementary material.

We will provide a proof sketch for both Theorems in the main body, in Section 3.1 and Section 3.2.

+A sketch for Theorem 1:

If the assumed space is a manifold of dimension $d$, then the volume $v$ of a ball of radius $r$ centered at any point is given by an equation of the form

where $K>0$ is some constant (involving $\pi$ and $d$), and the correction terms involve curvature and reach $\tau$. Taking the logarithm of both sides yields a linear regression problem for $d$

where the dimension $d$ forms the slope of the curve.

For the more general case of a fiber bundle, the equations above must be modified to handle the case where the ball of radius $r$ "sticks out of" the space, and so the slope of the curve must decrease in that case.

+Theorem 2's proof is already quite short, so we might (depending on space constraints) include it in full, or simply state that it is a matter of simply satisfying the bounds from [Robinson et al., 2025].

Understood, no problem.

Thank you once again for your review. We are happy that we were able to answer your questions satisfactorily and provide clearer understanding.

Additionally, please note that in response to some reviewers' comments about the number of singular tokens, we attempted a quick exploratory data analysis of singular tokens relative to the other tokens. In case you are interested to see the results, they are in our responses (i.e. official comments) to Reviewers ZHkg and XXVi.