SpectraLDS: Provable Distillation for Linear Dynamical Systems

Dear Reviewer,

Thank you for your time and your feedback. We appreciate your concerns about the scope of our results, especially as the STU is not as popular as other SSM architectures. However, our work carries large implications for the LDS, a common primitive forming the basis of most SSM architectures, and whose pitfalls provide intuition for many popular SSMs, such as S4 [10], Hyena [32], and Mamba [9].

Regarding your concern about simplicity: we respectfully disagree. We argue that simplicity is an advantage. Our method, while straightforward, is surprising, and circumvents the long-held understanding of parameter fitting of an LDS. Decades of work has culminated in methods that work in special cases (no noise or normally distributed noise), or agnostic improper learning (online) via convex relaxation. The new development of spectral filtering allows our proper learning of an LDS via learning the filters, rather than directly, which we consider unexpected. It is also worth noting that, although in our main result we present the more straightforward algorithm, this algorithm is far from the most practically performant, and we introduce more complex algorithms with substantially better performance in the appendix.

As far as the broader impact of our work, historically, despite their theoretical maturity, there have been no efficient and provably accurate learning methods for fitting LDS parameters to a signal with guarantees independent of the effective-memory of the system. This challenge has been a popular area of research over the last few years, with the Legendre Memory Unit [1*] and HiPPO initialization [2*] being significant results in this line of work. Although our result is currently restricted to fitting LDSs with symmetric transition matrices, we provide what is, to our knowledge, the strongest result to date for learning LDSs with high effective-memory, with performance independent of the effective-memory the signal and with strong convergence bounds.

Below, we will mention a few other examples of the practical significance of our work:

The Laughing Hyena Distillery (LHD) [26] technique has facilitated convolutional models 10x faster than Transformers (at equal performance) for language modeling. LHD transforms convolutional layers into complex symmetric LDS layers with gradient methods, offering no guarantees but having strong practical performance. With SpectraLDS, we can now convert convolutional layers into a real symmetric LDS without attempting non-convex optimization by projecting onto the spectral basis. Moreover, we also now have an understanding of which filters can be well represented, and the SpectraLDS layer serves as a drop-in method for learning ‘fast’ convolutional layers. As prior work has shown with the STU [2], the layer has high learning capacity on sequence-to-sequence tasks, as it is well suited for oscillating or decaying dependence.

For a signal produced by a symmetric LDS, we have now provided a strong bound for how increasing LDS parameters corresponds to reducing error. As a corollary of our main result, we have that, with a bounded input sequence, loss is bounded above by $O(e^{-p})$ when fitting an LDS with SpectraLDS, where p is the number of parameters and the constant is well understood. To our knowledge, this is the strongest scaling law for an LDS. As prior work and industry practice has shown success in modeling audio, among other signals, with increasingly large LDS models, the knowledge of scaling remains important. To be clear, our bound is still weaker than what is practically observed, but we still believe this is an important step in the right direction.

Moreover, recent work [3*] has extended the STU to a larger class of LDSs, with performance dependent on the maximal imaginary component of the eigenvalue of the transition matrix. In this case, our work may have applications to a much larger class of LDSs than we had initially anticipated, and provide the basis for general LDS training and understanding.

We appreciate your feedback and thank you for sharing your honest opinions. We hope we have largely addressed your concerns, and we would be happy for you to elaborate on any concerns that you believe still remain in the paper. In our revision, we will improve the discussion section to make the impact more visible, and we will include some of the points we have raised here.

Thank you for your time and feedback,

Authors

[1*] Voelker, A., Kajić, I., & Eliasmith, C. (2019). Legendre Memory Units: Continuous-Time Representation in Recurrent Neural Networks. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, & R. Garnett (Eds.), Advances in Neural Information Processing Systems (Vol. 32).

[2*] Gu, A., Dao, T., Ermon, S., Rudra, A., & Ré, C. (2020). HiPPO: Recurrent Memory with Optimal Polynomial Projections. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, & H. Lin (Eds.), Advances in Neural Information Processing Systems (Vol. 33, pp. 1474–1487).

[3*] Marsden, A., & Hazan, E. (2025). Universal Sequence Preconditioning. arXiv.

We appreciate the reviewer’s careful engagement and are grateful for the opportunity to clarify this point.

.

On the expressivity of symmetric LDSs:

It is correct that symmetric transition matrices yield real eigenvalues, and thus the impulse responses of such systems can be decomposed into linear combinations of exponential decays -- i.e., $(\rho^t)$ for $|\rho| < 1$. However, this space includes alternating sequences such as $f(t)=(−0.99)^t$, which are qualitatively different from smooth monotonic decays and can be difficult to capture with parameterized convolutional kernels.

These alternating sequences—common in modalities with an intrinsic sampling rate such as audio— are nontrivial to approximate and are challenging to express accurately with many kernel parameterizations. Notably, these eigenvalues are also the most challenging to learn with most deep learning methods but necessary to learn expressive symmetric (and asymmetric) LDSs.

While our theory applies to LDSs with symmetric transition matrices, we tested our method on filtered generated by LDSs with asymmetric transition matrices to demonstrate that the method generalizes well in practice and performs well on general LDSs.

We performed a series of experiments that tested the capacity of SpectraLDS to learn an LDS with asymmetric transition matrix and high effective memory compared with baselines of fitting an (asymmetric / general) LDS with the same state dimension and fitting an LDS with twice the state dimension. For both the baselines, to improve training stability, we set a learning rate of 1e-3, and we initialize the models with spectral norm of A bounded by 1. For the STU, we use learning rate 1.0. For all models, we use the Adagrad optimizer and gradient clipping.

In our experiments we found that, despite their stability adjustments, both baselines converged slower, attained worse fits, and remained unstable. In contrast, SpectraLDS attained better results substantially faster and with greater stability. It is worth noting that SpectraLDS almost always reaches near-convergence within 100 steps, and in the first 3 configurations, reached the same loss at the baselines more than 60x as quickly. We’ve included tables summarizing our results in a sub-comment.

.

On the surprisingness of the approximation via exponential sampling.

We respectfully disagree with the reviewer. We would like to offer some additional context that we hope will clarify why we view the outcome as more surprising and significant than it may initially seem.

While the final algorithm is relatively short, arriving at this approach was not straightforward. In practice, many alternative strategies that may appear promising fail to work. For example, in early attempts to distill the FlashSTU model [22], we trained both symmetric and asymmetric dynamical systems directly to fit the STU layers, a task analogous to learning a symmetric LDS. Despite using models with state dimensions exceeding 80,000, we found that these approaches only captured the signal accurately over the most recent 50 inputs (and incurred a substantial cost in efficiency). These difficulties occurred despite careful initialization of A and considerable effort to ensure stable training.

We used established methods for training such systems and found that, despite a substantial body of literature on fitting symmetric or near-symmetric LDSs, existing techniques struggled significantly in this setting and had not focused on sufficiently long-horizon tasks. This motivated the development of our approach.

We believe it surprising that the best way to fit an LDS involves first projecting to the spectral basis of Hankel eigenvectors, before then fitting each of the spectral filters with an LDS, In contrast, directly fitting the LDS without this transformation—even with significantly more parameters—was far less effective. Our experiments indicate that the spectral projection acts as a denoising step, which simplifies the learning problem and improves both stability and accuracy, but had not yet been fully understood.

In retrospect it seems simple, but to the extent of our testing, this is the only algorithm that was properly able to learn the LDS intrinsic to the FlashSTU. Moreover, the fact that a relatively short algorithm can be tuned to have great practical performance is part of what we believe will make this work more valuable.

With Algorithm 3, we have found a robust technique that is practically performant and well understood, and allows for the production of convolutional layers that are easily accelerated at inference.

Much of the surprise of our work comes from the result that our technique works well practically. In all tests we have performed, SpectraLDS has performed much better than the next best alternative, and we are excited to be able to bring such an algorithm to the broader community.

We apologize for the double comment, the experiment table expends many characters in the response.

---

Experiment Results: The configs are in order of difficulty and are:

To provide a fair comparison, average losses and time are reported only for runs that converged (final loss $\leq 100$). All tests were run on an H100-80GB Nvidia GPU.

---

Across these experiments, the STU achieved near convergence within 100 steps in almost all cases and outperformed baselines by a wide margin, both in accuracy and runtime. The mean error increase across STU-to-SpectraLDS conversions was 10% (min: 1.8%, max: 27%), thus preserving the considerable benefits of training speed and stability. In this experiment, we chose a SpectraLDS with hidden dimension 160, although note this does not affect STU training. A larger hidden dimension would reduce the conversion error.

We emphasize that SpectraLDS can be further refined into an asymmetric model for further improvement (at the cost of greater inference cost) if desired.

.

Summary

We are grateful for the thoughtful engagement of the reviewer and for offering us the opportunity to clarify our work. We agree that there are multiple places we can improve the exposition of our work and the discussion of its implications, and we will do so in the camera-ready copy.

While the restriction to symmetric LDSs is a fair concern, we hope that we have shown that even within this space, there are meaningful challenges and that our method offers a surprisingly effective solution. We are excited by how well it works in practice, for both symmetric matrices and asymmetric matrices, and we would be more than happy to run additional experiments to test other aspects. We also note that such results fall in line with prior work (e.g., S4 [10]), and imposing symmetric and near-symmetric structure on the transition matrix has not prevented other architectures from SoTA attention-free performance.

Thank you for a valuable conversation and helping strengthen our paper,

Authors

Dear Reviewer,

Thank you for your thorough and constructive feedback, and we appreciate the abundance of detail! Below, we will address each point.

.

[Major] As a reader unfamiliar with spectral attention methods, I was forced to read references [2] and [14] to get a good intuitive and technical idea of an STU.

We appreciate the extra time you spent on understanding our work, and we agree this certainly should not be necessary. Since submission, we have tried hard to improve accessibility and clarify core intuition, as well as reduce jargon in places we can. This has involved paper edits, but also the preparation of accompanying blog posts and content. For the submitted paper, we had added a section on notation in Appendix A.1, where we introduce the “impulse response” notation, among others, but in our revision we plan to add a formal glossary so the interested reader can accelerate through the introduction, and if they miss a definition, can easily glance to check.

We agree largely with the reviewer that more visuals would be helpful, and we plan to move Figure 2 to the introduction as you suggested, and we also plan to add Fig. 2 from [2], which is a convenient schematic for showing how the STU operates by decomposing an input filter into the spectral basis.

.

[Major] The core contribution of the paper, explained in Section 5, is restricted to a single page, leaving several of the most important details of the proposed methodology unclear.

Algorithm 1: Your assessment is correct, the random input vector is not necessary as an intermediate and the convolution suffices as well. We chose to introduce Algorithm 1 in this manner as we thought it was simpler to digest, and it has the same performance both theoretically and in practice, and both approaches are exceptionally quick.

Algorithm 2: For ease of reading, we have switched all notation to the first version of sampling $h$ random eigenvalues of $A$. However, both sections are accurate as the b and c coefficients can be incorporated into the matrix M, and training with them randomly sampled or with b = c = 1 are both very effective.

Algorithm 3 and appendix A.5: Thank you for the observation, we will be adding a table with all the parameters used for this part of the algorithm. In our forthcoming open-source release, we will provide code and all files needed for a complete reproduction, including all parameters for our results. We want to ensure that all aspects of this work are reproducible and publicly accessible to build on, and we thank the review for noticing areas we fell short on. We will additionally publish all model weights, infrastructure, and our set of 61,784 LDS-STU pairs used in Algorithm 3.

Appendix A.5.4: After the matrix inversion, we ran 50,000 steps of gradient descent to improve the fit of the coefficients, although this was a relatively minor improvement compared to the rest of the algorithm. We omitted this in the main text for clarity, although it will be included in our open-source filter generation code and in our revised section 5.2.

Generally, we agree the main section was not as clear as we would like, and we plan to address this by clarifying the above in the main paper, and by including the revised section 5.2, which we will discuss below.

.

[Minor] Thank you for catching the errors on lines 57, 150, 207, and in algorithm 2. We will fix them in the revision.

Thank you for the suggestion with the “impulse response.” We will add a clear definition and glossary reference early on to avoid any misconception.

The LDS filters refer to the convolutional kernel corresponding to the LDS, but we will define this more formally earlier.

Thank you for the tip on the LDS definition. We have found the definition remarkably inconsistent across SSM papers, and we will be more formal in our earlier definitions to make this point clear.

.

[Questions]

We certainly agree that it is important we improve the presentation and build greater STU intuition at the start. The additions of the new figures earlier on should hopefully help align the reader better with the STU and prepare them for our discussion.

Additionally, we plan to improve exposition of our main algorithm and clarify the additional steps taken in the practical algorithm. In earlier drafts of the paper, we had Algorithm 3 in the main paper. However, this made the exposition substantially more challenging and we felt the core ideas were less accessible. We plan to move section 5.2 to the appendix and instead replace it with a short text description of the improvements in Algorithm 3, leaving the full description of Algorithm 3, including all of our hyperparameters, to the appendix. We have included an example of the revised section 5.2 at the end of our response.

Generating the dataset of 1D-LDSs and corresponding filters is done offline and only needs to be done once as long as the STUs use the same set of spectral filters. We had fit 61,784 pairs offline, as described in section A.10, but the algorithm still has strong performance well below 10,000 pairs. In our forthcoming open-source release, we will provide the pair set we have computed. We will add full details for all the parameter choices in Algorithm 3 elsewhere in the appendix, and we will move a few of the core details, such as improving distribution election for $\alpha$, the gradient descent improvement, and the techniques to optimally select the eigenvalues of A to the main paper. Our hope is to strike a balance between being complete in the main paper while avoiding cluttering the main section and losing readability.

The final state dimension was chosen by examining filter plots, such as Figure 7, and reconstruction losses, such as Figure 9, and finding that the sharpest loss drop is in the first 60 state dimensions. The state dimension of 80 was validated in our testing, but it is possible larger models may need up to 160 state-dimension and smaller models can use much less.

.

[Limitations]

We agree that the restriction to symmetric LDSs is important to understand. This aspect was largely studied in earlier work on the STU [2], with the general results that although asymmetric LDSs have greater modeling capacity, they are practically bottlenecked on high-effective memory tasks by the challenges of training. In the S4 paper [10], which studied an expanded class of LDSs modeled by a normal plus low-rank matrix, the authors found that the LDSs they learned are nearly symmetric.

Recent work [1*] has extended the STU to a larger class of LDSs, with performance dependent on the maximal imaginary component of the eigenvalues of the transition matrix. In this case, our work may have applications to a much larger class of LDSs than we had initially anticipated.

.

[Closing]

We would be more than happy to elaborate on any lingering concerns or issues we have not yet suitably addressed. As a larger comment on the concerns of missing certain parameters, we care deeply about making this work something that people can replicate and build on, and we appreciate your feedback on cases where we fall slightly short. We have a full open-source release prepared, and we will be releasing all code, checkpoints, and data that could be helpful to end users.

Thank you for your time and feedback,

Authors

[1*] Marsden, A., & Hazan, E. (2025). Universal Sequence Preconditioning. arXiv.

.

[SAMPLE SECTION 5.2]

While Algorithm 2 provides the theoretical foundation for our distillation approach, we developed Algorithm 3 (detailed in Appendix A.4 and A.5) with several practical improvements that significantly enhance performance in applications. Rather than sampling eigenvalues $\alpha$ uniformly in $[0,1]$, Algorithm 3 samples in the range $[-1, 1]$ and deliberately oversamples eigenvalues near $\pm 1$ to better capture high effective memory (see Figure 6), which we use to assemble a large set of pairs $\mathcal{P} = \{(\alpha_i, m_i)\}_i$.

Instead of using a fixed random sample of $h$ eigenvalues, we employ a greedy selection strategy. We start with an initial subset of size $h_{\text{start}}$ chosen to minimize the reconstruction error $\| M^\dagger_{\text{sub}} \Psi_{\text{sub}} - \Phi_{1:k}\|^2_F$, where $\Psi_{\text{sub}}$ is a concatenation of the LDS impulse responses $\mu_{L}(\alpha_i)$ of that subset and $\Phi_{\text{sub}}$ contains the corresponding STU weights, where this subset is chosen through sampling over $\mathcal{P}$. We then incrementally add $h - h_{\text{start}}$ pairs $(\alpha_i, m_i) \in \mathcal{P}$ that most reduce the reconstruction error. After obtaining the initial transformation matrix $\tilde{M}$, Algorithm 3 refines it using gradient descent on the least-squares objective $\|\Phi_{1:k} - \tilde{M}\Psi_{\text{sub}}\|^2_F$, which typically yields approximately $1.4\times$ improvement in reconstruction accuracy.

Dear Reviewer,

Thank you for your time and for your feedback, and we appreciate your recognition of the significance of our work. We address each point below:

As Algorithm 3 has many moving parts, we felt that a full explanation of it in the main paper would be confusing to the reader, so we moved the algorithm to appendix A.4 and a full explanation to section A.5. Based on your feedback, we believe the best approach is somewhere in the middle, where we will instead move section 5.2 to the appendix and introduce a new section 5.2 where we explain how algorithm 3 differs from the main algorithm. We included a sample of what that section may look like below at the end of our response.

For Figure 1, we have a comprehensive description of the experiments performed in Appendix A.7, and in our forthcoming open-source release, we will include all the setup necessary to replicate this experiment. We will move more of the core details from Appendix A.7 to the figure caption and main text so the paper is easier to go through.

Although 340M is not large in scale, for custom model architectures, such as the FlashSTU, evaluating distillation techniques on models of this size is generally common practice (I.e., see [26], which evaluates performance after Laughing Hyena Distillery on models of size 153M and 355M). We tested our technique on all public pretrained FlashSTU [26] models, which included earlier checkpoints of a 550M parameter model as well. We achieved successful distillation without performance drop on that model and have included those results here:

However as this base model had worse evaluations, we only included our distillation on the 340M model in the final paper. The 550M model was published on 1/28/25, and the 340M model on 4/29/25, so we suspect the greater performance is due to better training choices.

Notably, our technique has shown effectiveness on a large range of evaluations, including modeling large linear dynamical systems with noise and with high effective memory, which we include in Appendix A.11. Although we agree with the reviewer that testing on larger FlashSTU language models would be ideal, it is not feasible for us to train larger FlashSTU language models, and we have tested distillation, and shown strong performance, to the maximal extent we are able to.

(Q1): Thank you for this observation. Line 66 is inaccurate and per-token generation costs are O(log L), and we will correct this in the revision.

(Q2): We agree this is confusing. The connection is that $[M_1, M_2, … M_K]$ from line 172 is equal to $m_i$ from algorithm 2, where the $M_k$’s were distilled from the LDS defined by the convolutional kernel $\mu_L(\alpha_i)$. In our revision, we will explicitly state that connection in Algorithm 1 and switch Algorithm 2 to refer to the matrix as Q.

(Q3): Yes, we can use the STU to learn a given LDS S and we can then distill the STU into another LDS S’. As the STU can learn S with error $\epsilon$ with $k = O(\log L \log \left( \frac{1}{\epsilon} \right))$ filters from line 190, where $L$ is the maximum sequence length, we thus have that we can achieve an $\epsilon$ approximation S’ of S with $O(d_{in} \cdot d_{out} \cdot \log L \cdot \log\frac{1}{\epsilon})$ parameters. Notably, S’ has size independent of the state-dimension of S, and thus for a high-dimensional LDS S, we can distill it into a relatively low-dimensional LDS S’ with bounded error. We agree that the term “high-dimensional” was not clear in this case, and we will clean up this exposition to make clear that our method for LDS-to-LDS distillation has $O(1)$ dependence on the state-dimension of the initial LDS.

(Q4): Yes precisely. For Algorithm 3, the matrix M is constructed in part by greedily choosing pairs $(\alpha_i, m_i)$ from a precomputed set so that $|| \Phi_{1:k} - M^{\dagger}\mu_L(\alpha_1, \dots, \alpha_i) ||$ is minimal. We will include this description in the revised section 5.2, where we discuss the differences between the improved but more complicated Algorithm 3, and the main algorithm we present.

(Q5): For the filters in figure 2, these are the same filters as in Appendix A.8 and A.9, and have reconstruction error of $1.23 \times 10^{-12}$. For a sense of scale, we show the relationship between LDS dimension and filter reconstruction in Appendix Figure 9. We will add the error metric to the figure caption.

(Q6): The proposed distillation algorithm does not require access to new data. In fact, once we have fit the spectral filters with Algorithm 2 or 3, we can immediately transfer the weights of any STU model that uses those filters to a SpectraLDS model. With the 340M FlashSTU model, this process was completed within 150ms. As all FlashSTU models are trained with the same set of 24 spectral filters, we can perform this immediate parameter transfer for any model in the FlashSTU series. The distillation algorithm provides an LDS of a given state dimension per set of spectral filters, and so generally will be rerun only when a different state-dimension is desired, either for more accurate reconstruction or for faster inference. However, it is also worth noting that after the initial step of collecting a set of pairs $(\alpha_i, m_i)$, the remainder of both Algorithm 2 and Algorithm 3 can be computed speedily. In our open-source release, we will provide the set of 61,784 pairs we have computed.

Thank you for your detailed feedback, and we will work on improving the clarity of the paper in the ways you suggested and fixing the minor mistakes. We hope that these fixes largely address your concerns, and we would be happy for you to elaborate on any concerns that you believe still remain in the paper. We also are happy to note that recent work [1*] has extended the STU to linear dynamical systems with asymmetric transition matrices, with bounds based on the maximum imaginary component of the eigenvalues, so we are enthusiastic about further applications of our distillation beyond symmetric LDSs.

Thank you for your time and feedback,

Authors

[1*] Marsden, A., & Hazan, E. (2025). Universal Sequence Preconditioning. arXiv.

SAMPLE SECTION 5.2

While Algorithm 2 provides the theoretical foundation for our distillation approach, we developed Algorithm 3 (detailed in Appendix A.4 and A.5) with several practical improvements that significantly enhance performance in applications. Rather than sampling eigenvalues $\alpha$ uniformly in $[0,1]$, Algorithm 3 samples in the range $[-1, 1]$ and deliberately oversamples eigenvalues near $\pm 1$ to better capture high effective memory (see Figure 6), which we use to assemble a large set of pairs $\mathcal{P} = \{(\alpha_i, m_i)\}_i$.

Instead of using a fixed random sample of $h$ eigenvalues, we employ a greedy selection strategy. We start with an initial subset of size $h_{\text{start}}$ chosen to minimize the reconstruction error $\| M^\dagger_{\text{sub}} \Psi_{\text{sub}} - \Phi_{1:k}\|^2_F$, where $\Psi_{\text{sub}}$ is a concatenation of the LDS impulse responses $\mu_{L}(\alpha_i)$ of that subset and $\Phi_{\text{sub}}$ contains the corresponding STU weights, where this subset is chosen through sampling over $\mathcal{P}$. We then incrementally add $h - h_{\text{start}}$ pairs $(\alpha_i, m_i) \in \mathcal{P}$ that most reduce the reconstruction error. After obtaining the initial transformation matrix $\tilde{M}$, Algorithm 3 refines it using gradient descent on the least-squares objective $\|\Phi_{1:k} - \tilde{M}\Psi_{\text{sub}}\|^2_F$, which typically yields approximately $1.4\times$ improvement in reconstruction accuracy.

Dear Reviewer,

Thank you for your time and constructive feedback on our paper. We appreciate your positive assessment of the theoretical contributions and novelty of our work. We address each point below:

[Pseudo-inverse Analysis]: We agree that more analysis of the pseudo-inverse conditions would strengthen the paper. In Theorem 1, we show that the error bound depends on $\lambda_{max}$, the largest eigenvalue of the Penrose-Moore pseudo inverse of matrix M. While for $h \ll k$, this can be exponentially large in k, we provide experimental evidence in Appendix A.2 showing that as h grows, $\lambda_{max}$ decays doubly exponentially.

[Figure 1 Readability]: We will improve Figure 1's accessibility by using more line styles (dashed, dotted, solid) and colors, increasing line thickness, and improving the legend font.

[Q1 - Epoched Future Fill Memory Limitation]: The Epoched Future Fill algorithm encounters a CUDA Out of Memory error on our H100-80GB setup at 262,144 tokens. While the theoretical memory complexity is reasonable, the implementation requires intermediate buffers for FFT operations and caches. The naive convolution approaches this limit but remains within bounds, whereas EFF's additional caching and overhead pushes it beyond the threshold. In contrast, SpectraLDS maintains constant memory requirements per token (O(h)) regardless of sequence length.

[Q2 - h vs. Accuracy Trade-off]: We provide detailed analysis of this trade-off in Figure 9 (Appendix A.10), which shows reconstruction error as a function of LDS state dimension. For our language modeling experiments, benchmark accuracy remains nearly identical for $h \geq 60$ but degrades rapidly below this threshold. We will expand Table 2 to include performance comparisons at various state dimensions to better illustrate this trade-off.

[Q3 - Beyond Symmetric Systems]: We are encouraged by recent developments in this direction. Recently, there has been promising work [1*] extending STUs to broader classes of transition matrices, with error bounds dependent on the maximum magnitude of the imaginary components of eigenvalues. We expect our distillation techniques to apply to these expanded filter sets. In our discussion section, we will add additional discussion in this direction.

We believe these revisions will address your concerns while improving the quality of the paper. We would be more than happy to elaborate on any additional concerns you have with the paper, and we appreciate your time and effort.

Thank you,

Authors

[1*] Marsden, A., & Hazan, E. (2025). Universal Sequence Preconditioning. arXiv.