Dynamical Properties of Tokens in Self-Attention and Effects of Positional Encoding

We thank the Reviewer for the constructive feedback. Below we address your concerns.

---

W1. Difference between this work and earlier studies remains vague.

A1. Let us take this opportunity to clarify the distinction between our results and those in relevant previous works, in order to highlight the novelty and practical relevance of our contributions. Our results directly generalize certain findings from [A] regarding the dynamical properties of tokens in self-attention. In particular, the authors of [A] characterized the convergence, divergence, and clustering behaviors of tokens under the assumption $Q = K = Id$ and $V=Id,-Id$, or parametric. We theoretically generalize these findings in [A] to the case of arbitrary matrices $Q$, $K$, and a matrix $V$ with at least one positive eigenvalue (see Section 3 and Appendix B). We further extend the analysis by investigating the effect of positional encodings in addition to self-attention (see Section 4 and Appendix C). These theoretical results are then empirically verified in practical Transformer settings (see Section 3.4). Moreover, unlike previous works in this line of research, we go beyond theoretical analysis and verification by demonstrating model performance improvements on several tasks (see Section 5 and Appendix D).

We hope that this response clarifies the novelty and originality of our contribution.

W2-Q6-Q1f. Some of the mathematical derivations may contain inaccuracies regarding the constraint on initial condition. Could the empirical analysis be extended to examine the impact of this constraint?

A2. Thank you for pointing out this error. Indeed, the inequality $e^{c\Vert x_\ell(0)\Vert_A^2}-L>0$ is needed to ensure that $\lim_{t \to+\infty}\Vert x_\ell(t)\Vert_A=+\infty$ in the proof of Proposition B3(b). This inequality implies that the initialization and the matrix $A$ must be chosen such that $\Vert x_\ell(0)\Vert_A>\sqrt{\frac{\log(L)}{\max\limits_{u\in\mathbb{R}^D,\Vert u\Vert_A=1}\frac{\Vert u\Vert_W}{\Vert u\Vert_A}}}=O\left(\sqrt{\log(L)}\right).$

Intuitively speaking, for Proposition B3(b) to hold, the $A$-norm of the initial point must be at least on the order of the square root of the logarithm of the sequence length $L$.

Fortunately, this proposition does not affect any subsequent lemmas or theorems, as Proposition B3(b) is not used later. (Only Proposition B3(a) is referenced.) Our main results (including Theorems 3.1, 3.4 and 3.6) still hold for arbitrary sequence length $L>0$.

W3. Some of the experiments are constructed in a way that closely reflects the conditions assumed in the theoretical analysis, which may make the results somewhat self-reinforcing

A3. While we include experiments that impose the theoretical constraints described in lines 263–272, none of our empirical validations rely on the NeuralODE assumption of shared weights across layers. Instead, each experiment uses distinct weight matrices per layer to ensure our results hold even when layer parameters vary. Specifically:

(i) Figure 3 shows that our theoretical findings hold without the NeuralODE assumption,

(ii) Figure 4 illustrates clustering behavior in models with FFNs and LNs under the divergence scenario, and

(iii) Figures 6 and 7 show model behavior when relaxing the constraints on $W_{sym}$ or $A_{sym}$, generalizing to the practical pretrained models.

Q7-Q1ad. Clarification regarding the setup used to produce Figure 6 and extension of experiments in Figure 6 and Section 3.4 to unconstrained settings of pretrained models

A4. To clarify, Figure 6 was generated by removing all LayerNorm and MLP (FFN) modules from a full-architecture pretrained Transformer to isolate pure attention dynamics. We measured token norms and mean pairwise L2 distances across layers under both constrained (divergence, intermediate, convergence) and unconstrained (baseline, similar to real-world Transformers) scenarios.

We visualized relaxed (intermediate/baseline) versus strict (divergence/convergence) constraint regimes. Under intermediate and unconstrained settings, token norms and pairwise L2 distances still diverge, verifying Remarks 3.5, albeit more slowly than in the pure divergence regime. Moreover, Appendix D.5.1’s Figure 7 further shows that, without any constraints on $W_{sym}, A_{sym}$, pretrained GPT-2 token trajectories still exhibit clustering.

Q1c. Is it possible to evaluate the core Neural ODE assumption?

A5. While the Neural ODE assumption may be violated in real-world settings, our experiments in Section 3.4, our analysis in Section 5.1, and our visualizations in Appendix D.5.1 were conducted without this assumption, thus empirically confirming our theoretical findings in more realistic settings.

W5-Q1b-Q2. Concerns about simplified setting and practical relevance to causal self-attention. Can one analyze the structure of the matrices $W$ in real models, especially for causal self-attention?

A6. Regarding the causal attention: We aim to work with the normal attention, not the causal attention at the moment. The dynamical properties of tokens under causal attention masking represent a different interesting research direction. An initial theoretical result in this area has been presented in [D]. Generalizing this work would certainly be a valuable avenue for future research.

Regarding the simplified assumptions on the theorem: Please refer to our answer A1 to Reviewer p6VJ.

Regarding the structure of the attention matrices $W = QK^\top$ in real models: In pretrained practical Transformers, $W$ is not necessarily to be positive or negative definite. However, our analysis is not just restricted to definite matrices. For example, Theorems 3.1 and 3.6 hold for arbitrary $W$, and there are no restrictions on $Q$, $K$, or $V$ in Remark 3.5.

Q1e. Impact of other architectural components on converge/diverge property

A7. Please refer to our answer A2 to Reviewer 1jLS.

W4-Q5. The results lack standard deviations.

A8. We did provide full results (with standard deviation) of Table 1 and 3 in Appendix D.6 (Table 4 and 5, respectively). For Table 2, we provide the full results averaged after 3 runs below:

Q3. In the caption of Figure-4 what does it mean by similar shapes?

A9. To clarify, Fig. 4 shows tokens clustered into groups, and tokens within each group propagate through layers in a similar direction.

Q4. How do you obtain figure 4 and figure 5?

A10. Figure 4 was plotted by projecting onto a manually selected 2D plane (without PCA) to highlight clustering properties. Figure 5 uses tokens that are inherently 2D in our setup, so no dimensionality reduction was needed.

Q8. Intuition for why divergence leads to improved performance

A11. We claimed in the paper that convergence may negatively affect model performance. Therefore, encouraging the model to avoid the convergence scenario can lead to better practical results. One simple way to avoid convergence is to push the model toward the divergence scenario. The reason convergence may hinder model improvement is that, in this case, token representations tend to shrink toward zero as time $t$ increases. This leads to information loss during training updates, which is not preferable.

Q9. Practical benefit of Theorems and Propositions 3.1–3.6

A12. The practical benefit of Theorems and Propositions 3.1–3.6 relies on the well-known fact that the continuous-time limit can be used to efficiently approximate modern deep networks. These networks often have a large number of layers, making them well-approximated by continuous-time models. For example, DeepSeek-v3 has approximately 61 layers and can thus be viewed as a discretization of an ODE system. Moreover, the continuous-time limit is a well-established technique in both the machine learning and mathematics communities. It has provided meaningful insights into important behaviors of realistic models, including training dynamics, stability, and generalization.

An alternative way to understand how the described properties evolve layer by layer is through the use of difference equations (instead of differential equations). Exploring this direction is an interesting avenue for future research.

Answer for Note. Thank you for the opportunity to address your concerns in detail. Regarding the simplified setting used in our theorems, we adopt a common theoretical approach in which simplified models allow for tractable and rigorous analysis while retaining enough expressive power to capture key patterns seen in practice. Rather than analyzing the exact, highly intricate architectures used in real-world models, theoretical work typically focuses on simplified yet sufficiently general approximations. This methodology aligns with several high-quality works by leading researchers in the field [A, B, C, D]. Fully addressing the complexity of practical Transformer models remains a significant challenge and is currently beyond the scope of comprehensive theoretical analysis.

For a more detailed explanation, please refer to A6.

References

[A] Geshkovski et al. The emergence of clusters in self-attention dynamics. NeurIPS 2024.

[B] Geshkovski et al. A mathematical perspective on Transformers. Bulletin of the AMS 2024.

[C] Vo et al. Demystifying the Token Dynamics of Deep Selective State Space Models. ICLR 2025 (spotlight).

[D] Karagodin et al. Clustering in Causal Attention Masking. NeurIPS 2024.

---

We will incorporate the discussion in the revised version. If our responses adequately address the concerns, we kindly hope the evaluation may be reconsidered accordingly. We remain open to further discussion in the next stage of discussion.

Thank you again for your quick response. We answer your question below.

Which sequences were used to generate the results shown in Table 2 and Table 3? Additionally, over how many sequences were these numbers averaged? More details about the experimental setting would be helpful for interpreting the results.

A20: The sequences used to generate the results in Table 2 and Table 3 are taken from the test set of WikiText-103, a language modeling dataset extracted from Wikipedia articles. For each sequence, only the first 100 tokens were used. In Tables 2 and 3, we report results for three randomly selected sequences, each corresponding to one row in the table.

We use a pretrained GPT-2 Small model with 12 Transformer layers, hidden and embedding size of 768, and vocabulary size of 50,257 tokens.

Additionally, we present extended experimental results using a broader range of sequence lengths and a larger set of test sequences to provide a more comprehensive analysis of token behavior. In Tables 6 and 7, we report the mean token norm and the mean distance between tokens for sequence lengths of 16, 64, 100, 128, and 256 tokens. For each length, the reported values are averaged over 100 randomly selected sequences from the test set of WikiText-103.

Table 6: Mean token norm with varying sequence length (GPT-2 without FFNs and LayerNorms)

Table 7: Mean distance between tokens with varying sequence length (GPT-2 without FFNs and LayerNorms)

We also conduct an additional experiment to demonstrate that both the mean norm and the mean pairwise distance between tokens grow exponentially across layers. Specifically, for each choice of sequence length, we fit a linear regression model between the layer depth and the logarithm of the mean norm as well as the logarithm of the mean pairwise distance. The resulting $R^2$ values are shown below:

As seen, all $R^2$ values are very close to 1, indicating a strong linear relationship between layer depth and the logarithmic values of the two terms. This confirms that both the mean norm and the mean pairwise distance grow at an exponential rate with depth, which aligns with the expected behavior in the divergence scenario.

Again, we appreciate your feedback and suggestions. We will include these clarification in our revision.

We thank the Reviewer for the constructive feedback. Below we address your concerns.

---

Q1. I have a question regarding the addition of positional encoding in the equation. I thought positional encoding is added to the initial condition, and that’s it. Why does it appear in the ODE in equation (5)? I am not totally sure I understand this point.

A1. Indeed, adding the positions $p_i$ to the initial conditions is equivalent to fixing the initial condition and adding $p_i$ to $x_i$, as in Equation (5).

Q2. The authors are also very welcome to comment on the possible impact that an MLP layer (or a normalisation layer) may have on the divergence, or the avoidance thereof.

A2. Unfortunately, not all theoretical results continue to hold when an MLP or LayerNorm is introduced. However, under the divergence scenario where MLP/LayerNorm are present, Figure 4 of Section 3.4 and Figure 7 of Appendix D.5.1 in our manuscript show that tokens still form distinct clusters, with each group following a similar trajectory pattern. This suggests that our observation on directional token clustering persists even in a fully structured Transformer model with MLP/LayerNorm. A theoretical analysis for these cases requires further adaptation, which we leave for future research.

Q3. Moreover, there is a paper by Koubbi, Boussard and Hernandez, where they also generalise the results of Geshkovski et al. to somewhat more general parameters, but it’s not cited herein. Perhaps the authors can discuss the differences with that paper.

A3. Thank you for pointing out the interesting work in [A]. As far as we are aware, the authors of [A] assume that the triple $(Q, K, V)$ already leads to a clustering phenomenon (at infinity) of tokens and investigate how such clustering is affected by small perturbations of the triple through the addition of low-rank components. In contrast to this approach, we directly characterize the dynamical properties of tokens for the triple $(Q, K, V)$ based on an analysis of their eigenvalues. Furthermore, we analyze the effect of positional encoding on the dynamics. Beyond that, we also propose model modifications to improve performance across different tasks. We will add reference [A] and include this discussion in the revised version of the paper.

[A] Koubbi, Boussard and Hernandez et al., The Impact of LoRA on the Emergence of Clusters in Transformers, arxiv.

---

We will incorporate the discussion in the revised version. We remain open to further discussion in the next stage of discussion.

We thank the reviewer again for your insightful reviews and valuable feedback. We answer your additional question below.

Q1: I am not sure I understand A1 as of yet. Are the authors considering the equation for the pure tokens (w/o Positional encoding), and then view the positional encoding as a "source term" in the equation? In this case, the equation is permutation invariant, is it not?

A4: Let us take this opportunity to explain why adding the positions $p_i$ to the initial conditions, as you mentioned, is equivalent to fixing the initial condition and adding $p_i$ to $x_i$, as in Equation (5). Indeed, from Equation (5), if we consider the transformation $y_l(t) = x_l(t) + p_l$ for all $l = 1, \ldots, L$, then the initial condition corresponding to $y_l$ becomes $y_l(0) = x_l(0) + p_l,$ while the differential equations governing $y_l(t)$ are exactly those of the sole attention dynamics. Because of this, positional encoding is added to the initial condition (if we follow the dynamics of $y_l$), as you mentioned, and this is equivalent to fixing the initial condition and adding $p_i$ to $x_i$, as in Equation (5) of our paper.

We thank the Reviewer for the constructive feedback. Below we address your concerns.

---

W1. Concerns include the reliance on a continuous‐time limit and simplifying assumptions, specifically tying attention weights across layers and omitting feed‐forward and layer‐normalization modules, which undermine both the work’s relevance to real Transformer architectures and its originality.

A1. Regarding the use of continuous-time limit: We respectfully disagree with the reviewer about the comment "continuous-time limit has no obvious motivation other than it allows for a dynamical system that can be analyzed". Modern deep networks often have a large number of layers, making them well-approximated by continuous-time models. For example, ResNets can be seen as discretizations of ODEs, and this perspective becomes increasingly accurate as depth increases. Moreover, the continuous-time limit is a well-established technique in both the machine learning and mathematics communities. It has led to important insights into training dynamics, stability, and generalization, and underpins models like Neural ODEs, State Space Models, and Liquid Neural Networks. Thus, it is both theoretically justified and practically relevant.

Regarding your concern about our simplifying assumptions, namely, tying the attention weight matrices across layers and excluding the FFN/LayerNorm: While your concern is valid, it is common for theorists to adopt simplified settings of the model to enable feasible theoretical analysis, while still retaining sufficient capacity to capture key patterns observed in practical scenarios. Several high-standard related works by notable researchers also operate within such simplified settings, for example:

[A] Geshkovski et al. The emergence of clusters in self-attention dynamics. NeurIPS 2024. (The assumption $Q=K=V=Id$ was used.)

[B] Geshkovski et al. A mathematical perspective on Transformers. Bulletin of the AMS 2024. (The assumption $Q=K=V=Id$ was used.)

[C] Vo et al. Demystifying the Token Dynamics of Deep Selective State Space Models. ICLR 2025 (spotlight). (The assumption dim $=1$ and time-invariant parameters were used.)

Our results generalize certain findings in [A] by employing more general assumptions on $Q, K, V$, and by additionally analyzing the effect of positional encodings. We believe that this generalization is already nontrivial.

The general case involving a full and practical Transformer model remains highly challenging, and to the best of our knowledge, no existing work has achieved this result to date.

W2. The authors say that “Our results are also validated against pre-trained Transformer models, confirming that the theoretical insights carry over to real-world settings.” The authors indeed show some relevance of their theoretical predictions for the transformers they train, including those with FFN layers in addition to attention layers, but my understanding is that these transformers have their weight matrices tied across layers, which is not much of a real-world setting (besides the ALBERT architecture).

A2. To clarify, we did verify our theoretical results using different settings of practical Transformer architectures in Sections 3.4 and 5.1, none of which restrict weights from being shared across layers. In particular: (i) Figure 3 demonstrates that our theoretical findings hold in the absence of the weight-tying assumption, (ii) Figure 4 illustrates the clustering behavior of models with FFNs and LNs, but constrained to the divergence scenario, and (iii) Figures 6 and 7 demonstrate the behavior of the models when lifting the constraints on $W_{sym}$ or $A_{sym}$, generalizing to the practical pre-trained models.

W3-Q1. Some standard ML terms seem to be used unusually in this paper. In particular, the word “token” is used throughout the paper to rather mean something like “token embedding” or “residual stream”. The stated study of how tokens evolve is really about how the activations in the residual stream evolve through the layers. For example, “when tokens move closer” would be more precise if it was changed to say “when embeddings move closer in deeper layers of the network”. It would be much more conventional for “tokens” to refer to the symbols from a fixed discrete set, rather than real values (W3). The authors assert that their first contribution is to “provide conditions that predict whether tokens in self-attention converge to zero or diverge to infinity”. By standard use of these terms, this claim is inscrutable. I am guessing the authors mean that the magnitude of token embeddings in a transformer are converging or diverging? Can the authors provide citation(s) of this issue in real transformers? (Q1)

A3. The way we use the term "token" in this paper is consistent with its usage in many previous works in this line of research. For example, it appears in high-standard publications [A, B, C, D]. In particular, the authors of [A] characterized the convergence/divergence properties of tokens when $Q=K=V=Id$. Our results generalize the findings in [A] by employing more general assumptions on $Q,K,V$, and by also analyzing the effect of positional encodings.

Following your suggestion, we will add a remark to explain alternative terms for "token" used in other areas, such as "token embedding" or "residual stream."

Q2: Regarding the framework laid out in Sec. 2.2, the authors may want to consider causal attention masking in future work, since this would be most relevant for autoregressive transformers.

A4. The dynamical properties of tokens under causal attention masking represent an interesting research direction. An initial theoretical result in this area was presented in [D]. Generalizing this work would certainly be a valuable avenue for future research. We will add this to the Future Work section of our paper.

[D] Karagodin et al. Clustering in Causal Attention Masking. NeurIPS 2024.

Q3. It is not clear how the eigenvalues of $A$ correspond to the eigenvalues of its symmetrization $A_{sym}$. In the general case, I’m not aware of any easy correspondence. Accordingly it doesn’t seem to be the case that, e.g., the positive-definiteness of one implies the positive-definiteness of the other. In the paper, sometimes positive-definiteness is talked about in terms of $A$ and sometimes in terms of $A_{sym}$, and sometimes in terms of $A$ after assuming it is symmetric such that $A=A_{sym}$. How should the reader think about these relationships? In particular, the claims starting on Line 128 involving the positive definiteness of $A$ don’t appear to assume that $A$ is symmetric, but then Thm. 1 assumes that $A$ is a symmetric matrix and Remark 3.2 is about the positive definiteness of the symmetrization of $A$.

A5. We did not claim any correspondence between the eigenvalues of $A$ and those of $A_{sym}$. Instead, we used the following two distinct assumptions:

(a) $A$ is positive definite;

(b) $A_{sym}$ is positive definite.

When we use (a), this means that $A = A_{sym}$ and is positive definite. However, when we use (b), $A$ is not necessarily symmetric, but its symmetric part $A_{sym}$ must be positive definite. To clarify, (a) implies (b), but (b) does not imply (a).

MS1. Line 41 and Section A: “Only a limited number of studies in the literature provide a theoretical understanding of the internal structure of learned representations in pre-trained Transformer models (see Section A for a comprehensive list of relevant works).” This field moves fast, so it seems overly confident to suggest a “comprehensive” list of relevant works that provide understanding of the internal structure of learned representations. I can certainly think of many more. Perhaps remove the word “comprehensive”.

A6. We agree to remove the word "comprehensive" in the revised version. In addition, we will add further relevant works to the related works section.

MS2. Line 48: It seems that “fourth-fold” should be changed to “fourfold”.

A7. We will change “fourth-fold” to “fourfold” in the revised version.

MS3. Line 118: Consider explicitly stating the intended meaning/definition of “equivariant” in the statement that “it is equivariant to the standard Euclidean norm”

A8. The correct statement is: “it is equivalent to the standard Euclidean norm.” Two norms on $\mathbb{R}^D$ are equivalent if and only if they induce the same topology. It is well known that all norms on $\mathbb{R}^D$ are equivalent. We will update this in the revised version.

MS4. Line 157: Typo: Change “randomly chosen of model parameters” to “randomly chosen model parameters”

A9. We will update this in the revised version.

---

We will incorporate the discussion in the revised version. If our responses adequately address the concerns, we kindly hope the evaluation may be reconsidered accordingly. We remain open to further discussion in the next stage of discussion.

We thank the Reviewer for the constructive feedback. Below we address your concerns.

---

W1. Please correct me if I understand the setting in a wrong way: the parameters are indeed trained and fixed, however, the time variable $t$ refers to the depth of the transformers. Hence the fact that $W$ and $A$ do not depend on $t$ means that the merged weight $W$ of attention and the value matrix $V$ are also the same for all layers. This does not align with the actual case. As the cluster of tokens has already been studied by prior works, I think it is better to take a step further to remove the limitation by considering a case where $W$ can depend on $t$.

A1. Yes, you are correct. In our theorems, we assume that $W$ and $A$ are independent of $t$. We empirically verify our theoretical findings in real-world pretrained transformers in Section 3.4.

While your concern regarding the limited scope of practical scenarios under these assumptions is valid, it is common for theorists to adopt simplified settings of the model to enable feasible theoretical analysis, while still retaining sufficient capacity to capture key patterns observed in practical scenarios. Several high-standard related works by notable researchers also operate within such simplified settings, for example:

[A] Geshkovski et al. The Emergence of Clusters in Self-Attention Dynamics. NeurIPS 2024. (The assumption $Q=K=V=Id$ was used.)

[B] Geshkovski et al. A Mathematical Perspective on Transformers. Bulletin of the AMS 2024. (The assumption $Q=K=V=Id$ was used.)

[C] Vo et al. Demystifying the Token Dynamics of Deep Selective State Space Models. ICLR 2025 (spotlight). (The assumptions dim $=1$ and time-invariant parameters were used.)

Our results generalize certain findings in [A] by employing more general assumptions on $Q$, $K$, and $V$, and even analyze the effect of positional encodings. We believe that this generalization is already nontrivial.

The case where $W$ and $A$ depend on $t$ remains highly challenging, and to the best of our knowledge, no existing work has achieved this result so far. We leave this interesting problem for future work.

W2-Q2. One aspect that has not been touched by this paper is the statistics of the conditions for $W$ and $A$ of real-world pre-trained models (not manually designed by some re-parameterization as in Appendix D.2). Specifically, the condition that the value matrix $V$ is invertible cannot necessarily be guaranteed in practice. Thus it is unclear whether these pre-trained models indeed exhibit such divergence and convergence of tokens that can match the conclusion of this paper. In a word, it is not clear whether these conditions and the corresponding divergence/convergence of tokens only exist in the special setting of the paper or are more general across a wide range of settings (W2). Could the conclusions/prediction be applied to real-world pre-trained models? And if so, what will the comparison between a randomly initialized model and the pre-trained model be regarding the conditions given by this paper? (Q2)

A2. We performed an empirical analysis of real-world pretrained Transformer models to verify the conditions in our theory. Specifically, for each model, we measured the mean percentage across all layers of (i) “near-zero” eigenvalues of the value matrix $V$ (using threshold $\epsilon = 10^{-3}$), and (ii) positive eigenvalues of the symmetrized matrices $W_{\mathrm{sym}}$ and $A_{\mathrm{sym}}$.

Table 1 reports the mean ± standard deviation of these percentages for GPT-2 XL, DistilGPT2, and LLaMA-2 13B. We observe that: (i) In all cases, $V$ exhibited 0% near-zero eigenvalues, indicating that $V$ is invertible in practice. (ii) The proportion of positive eigenvalues in both $W_{\mathrm{sym}}$ and $A_{\mathrm{sym}}$ is approximately 50%, which matches the divergence regime predicted by Remark 3.5 if omitting FFNs and LayerNorms. The divergence behaviour of an unconstrained/real-world pretrained model is also illustrated as 'baseline' case of Figure 6 in Section 5.1.

These findings align with theoretical expectations: (i) the set of singular matrices has measure zero in continuous parameter spaces, and (ii) unconstrained weight matrices, whether randomly initialized or learned, tend to have eigenvalue distributions symmetric about zero. We also repeated this analysis on a randomly initialized model (GPT-2 XL reinitialized) and obtained similar results (approximately 50% positive eigenvalues in $W_{\mathrm{sym}}$, $A_{\mathrm{sym}}$, and no near-zero eigenvalues in $V$).

Table 1: Mean ± std. dev. of eigenvalue statistics (in %) across layers.

Q1. Could the authors explain what the conclusions of the current manuscript will be if all the weights depend on the variable $t$?

A3. Unfortunately, the theorems may no longer hold if the weights depend on the variable $t$. However, our simulations using randomly selected time-dependent weights suggest that the theorems may still hold under analogous assumptions. For example: Theorem 3.1 appears to remain valid if $A$ is definite and maintains the same sign for all $t > 0$; Theorem 3.4 still holds if $A < 0$ and $W > 0$ for all $t > 0$; and Theorem 3.5 continues to hold if $V$ possesses a fixed positive eigenvalue for all $t > 0$. We will include this discussion in the revised version of the paper.

---

We will incorporate the discussion in the revised version. If our responses adequately address the concerns, we kindly hope the evaluation may be reconsidered accordingly. We remain open to further discussion in the next stage of discussion.