SKOLR: Structured Koopman Operator Linear RNN for Time-Series Forecasting

Claim 1

We would kindly ask the reviewer for more supporting arguments for the assertions.

"connection is somewhat superficial"

We establish a clear connection between an EDMD-style approximation of the Koopman operator for a history-augmented state and a linear RNN. This motivates a highly effective but simple architecture, which turns out to be not superficial.

"theoretical justification is not fully developed"

We contend that we have fully developed the theoretical justification mathematically. Eq. 5 is the equation for a linear RNN with a finite history. Eq. 6-7 present the EDMD-style approximation to the Koopman operator. And Sec. 3.3 explains that if we consider an augmented state with historical observations then we can rewrite the approximate Koopman operator relation as Eq. 8. Obviously, Eq. 8 has the identical structural form as Eq. 5, implying that we can implement Eq. 8 using a linear RNN (with an MLP embedding).

"only intuition is provided"

The development is by no means "only intuition". It is a clear and concise mathematical development. However, we are happy to provide a more formal proposition-proof structure in an appendix if this is desired. Please flag this, and we will provide it in the final response.

"assumptions ... are not explicitly stated"

We kindly ask which assumptions are missing in this development? The key assumptions are expressed in the fact that we are applying EDMD-style approximation (so the EDMD assumptions apply); we specify the full rank assumption for a unique solution.

"a very similar equivalence was proved already"

App. E [a] contains NO equations connecting linear RNNs to Koopman theory — only a qualitative observation ("Sounds familiar? This is exactly what a wide MLP + Linear RNN can do.") without mathematical formulation or proof. While we acknowledge this prior insight, our contribution provides the explicit mathematical connection that was not previously established.

Connection to [a] and [b]

We would include the suggested works into our discussion.

However, we'd like to highlight key distinctions from [a]. As aforementioned, [a] only offers a qualitative discussion of Koopman operator theory relevance, listing equations for Koopman theory (Appx. E, Eq.s 17-22) followed by a conclusion without any mathematical development. More importantly, the architecture in [a] does not follow the qualitative observation in Appex. E: the stack of [linear recurrent units + MLPs] has strayed away from the Koopman operators. In contrast, our architecture is an exact match to our derivations in (5) and (8).

Furthermore, [a] does not address time-series forecasting with any experiments on this task. In fact, the proposed architecture in [a] performs poorly in the time-series forecasting setting (See Tab. 7).

Concerning [b], we will cite it and include more discussion about the intersection of machine learning methods and Koopman operators.

Statistical Significance

To address the reviewer's concern about statistical significance, we add Tab. 6 with standard deviation (std) across 3 independent runs for all datasets and forecast horizons. The low stds (<0.003 for most datasets) demonstrate the consistency and reliability of SKOLR's performance. Thus, we contend that SKOLR's improvements, even when small in magnitude, represent genuine performance differences rather than statistical fluctuations. Additionally, we will provide confidence intervals and statistical significance tests in our final response.

Efficiency and Parameter Reduction

We will add a dedicated "Theoretical Complexity Analysis" section to formally establish SKOLR's computational benefits. Due to the word limit, please refer to response to reviewer hUoy (point 1.2) for detailed information. This comprehensive theoretical analysis, combined with our expanded empirical results in Tab.4, provides thorough justification for SKOLR's efficiency claims.

Ablation study

As suggested by the reviewer, we have also conducted a more comprehensive ablation study on the design elements of SKOLR. As shown in Tab. 5, we compare our full SKOLR model with two ablated variants: (1) "w/o Structure": no structured decomposition, using a single branch with dimension (N×D); (2) "w/o Spectral Encoder": no learnable frequency decomposition, while maintaining the multi-branch structure.

The results show that both components contribute meaningfully. Removing the structured decomposition leads to performance degradation on 27/32 tasks, with notable declines on ETTh1 and ILI, while increasing computational overhead. Similarly, removing spectral encoder impacts performance on 23/32 tasks, though with a smaller overall effect.

Strengthen Claims with more Convincing Evidence

Branch Decomposition - frequency band specialization

We appreciate the reviewer’s insightful feedback and agree that "specialization" is too strong a claim. We consider that Fig. 3 provides evidence that different branches place more emphasis on some frequency bands (e.g., Branch 1 has a greater relative emphasis on the low-frequency component). But there is not evidence of specialization.

ILI performance

Regarding the performance of SKOLR on ILI, we recognize the concern about the phrasing of "remarkable forecasting capability." While SKOLR achieves the best MSE among the tested models, we agree that this is mischaracterization, given that the error is still high. We will change the text to more accurately state that SKOLR outperforms the baseline methods, with significant error reduction for the shorter horizons.

Ablation Study - learnable frequencies

The observation on the ablation study is insightful. We agree with the general observation that the learnable decomposition is not a critical component and does not lead to a major improvement. On the other hand, it does lead to a relatively consistent 2-5% improvement, and it is not many parameters to learn. We will modify the text to downplay the significance of the learnable frequency decomposition, and explain that while it does bring some benefit, SKOLR performs well without it. It suggests that the key aspect of the multiple branches is to allow for MLPs to learn different embeddings.

Theoretical Claims - linkage between Koopman operator and RNN

Our claim is not as strong as "the Koopman representation can be equivalent to a linear RNN". In the abstract and introduction, we were careful to write that we "establish a connection between Koopman operator approximation and linear RNNs". In Section 3.3, after equation 8, we write that "this structure can be implemented as a linear RNN". So, our conclusion/claim is more that a linear RNN (with MLP embeddings) provides an efficient and effective mechanism for implementing a specific structured Koopman operator approximation (involving an augmented state incorporating past observations). Our equivalence specifically refers to the scenario where we construct an extended state representation using lagged observations (Section 3.3), which effectively transforms the Koopman operator approximation into a form where Eq. 5 and Eq. 8 become structurally identical.

Since your review is thorough, indicating careful attention to the paper, it seems that we haven't made it clear enough that our theoretical claim (which is indeed more of an algorithmic development) is more limited. We will modify the language to make it very clear. This is perhaps also an issue we would raise with the qualitative connection in Orvieto et al. (2023) between Koopman operator analysis and an MLP + linear RNN; when you write equations, the connection is not quite so simple, nor as general, as suggested in that work.

References

Thank you very much for highlighting these essential references. We will incorporate them to properly situate SKOLR within the Koopman literature. We'll add Mezic (2005) alongside Koopman (1931) as foundational work on Koopman mode decomposition, and Rowley et al. (2009) when discussing DMD's relationship to Koopman theory. We'll discuss connections between our spectral decomposition approach and Kutz et al. (2015), whose frequency-based decomposition shares similarities with our branch structure. Importantly, we'll acknowledge Arbabi and Mezic (2017) when discussing time-delay embeddings in Section 3.3, as their Hankel DMD provides theoretical support for our extended state construction. We'll also correctly attribute EDMD to Williams et al. (2015) in Section 3.2, while maintaining the Li et al. (2017) reference for neural network extensions. These additions will provide proper attribution and better contextualize our contributions within the evolving landscape of Koopman-based methods.

Weaknesses

The three weaknesses are addressed in the comments above.

Suggestions

Thank you for these careful observations. We will standardize our usage of "time series" (without hyphen) throughout the paper for consistency.

We also appreciate you flagging the timeline in our literature review. We will revise this section to include pioneering works such as [1]-[3] and properly reflect the development of time series forecasting methods.

[1] Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735-1780. [2] Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. [3] Binkowski, M., Marti, G., & Donnat, P. (2018, July). Autoregressive convolutional neural networks for asynchronous time series.

1. High Efficiency

1.1 Computational efficiency:

We provide additional results for all datasets. Please see Tab.4. SKOLR achieves a compelling trade-off between memory, computation time, and accuracy.

1.2 Theoretical Complexity Analysis

SKOLR achieves computational efficiency through structured design and linear operations. For a time series (length L, patch length P, embedding dimension D, N branches):

• Time complexity: O(N × (L/P) × D²) from spectral decomposition, encoder/decoder MLPs, and linear RNN computation
• Memory complexity: O(N × D²) for parameters and O(N × (L/P) × D) for activations

Compared to a non-structured model with dimension D' = N×D:

• Non-structured approach: O((L/P) × N²D²) time and O(N²D²) memory
• SKOLR provides N-fold reduction in computational requirements

SKOLR avoids quadratic scaling with sequence length seen in transformers (O((L/P)² × D + (L/P) × D²) time, O((L/P)² + (L/P) × D) memory).

1.3 Parallel computing

The N separate branches are processed independently (in our code), reducing time complexity to O((L/P) × D²).

The linear RNN computation has no activation functions, so the hidden state evolution is: $h_k = g(y_k) + \sum_{s=1}^{L/P} W^s g(y_{k-s})$. This allows efficient matrix operations, reducing time complexity to O($D^3 \log(L/P)$ + $(L/P)^2 \times D$) per branch. For time series where $L/P \ll D$, this is a significant speedup.

2. Exceptional Performance Claim

Our claims are based on Tab.s 1 & 2 and Fig.s 2 & 4. SKOLR requires less-than-half the memory of any baseline and less-than-half of Koopa's training time (Fig.4). SKOLR has the (equal-)lowest MSE in 17 our of 32 tasks and ranks second in a further 7 (Tab.1). SKOLR significantly outperforms Koopa for synthetic non-linear systems (Tab.2). Given the very low memory footprint, the low training time, and the impressive accuracy, we consider that the claim of "exceptional" performance is supported, but we can use a less strong adjective.

We agree that the paper would be strengthened by more examples (showcases) demonstrating the capture of complex patterns. We do have one example in Fig. 3, but we will include more. Please see Fig. 2 as an example. SKOLR's predictions have much lower variance than Koopa's and track the oscillatory behaviour better.

3. Relevant Reference

We will cite it and add discussion. The paper evaluates the existing non-linear RNNs and Koopman methods for near-wall turbulence. It does not draw connections or develop a new method.

4. Clarity

4.1 Frequency Decomposition

We agree that the motivation for our learnable frequency decomposition could be better. Please see the response to Reviewer kXAS. Learnable frequency decomposition offers three key advantages. (1) Each branch can focus on specific frequency bands, decomposing complex dynamics. (2) Learnable components adaptively determine which frequencies are most informative for prediction. (3) This approach aligns with Koopman theory, as different frequency components often correspond to different Koopman modes.

4.2 Generalization capability

Constructing theoretical guarantees that the approach generalizes is challenging and would be a paper all on its own. We do provide experimental results for a large variety of time-series that exhibit very different characteristics, ranging from strongly seasonal temperature series to pendulums exhibiting highly non-linear dynamics. We consider that the experimental evidence in the paper is strongly supportive of a capability to generalize to a diverse range of time series.

5. Theoretical analysis: eigenvalues

This is an excellent suggestion. Our focus is forecasting, so our results and analysis concentrate on that task. However, the analysis of learned Koopman operator eigenvalues can indeed reveal important characteristics.

We analyzed eigenvalue plots for Traffic dataset (see Fig.3). We see that each branch learns complementary spectral properties, with all eigenvalues within the unit circle, indicating stable dynamics. Branch 1 shows concentration at magnitude 0.4, while Branch 2 exhibits a more uniform distribution. The presence of larger magnitudes (0.7-0.9) indicates capture of longer-term patterns.

6. Questions

6.1

It is a typo. We will revise Fig. 1 to show the proper time indexing across all components.

6.2 Long horizon prediction

To show SKOLR's capability for longer horizons, we included experiments with extended prediction horizons (T=336, 720) in App. B.2 (Tab. 9), SKOLR maintains its performance advantage at extended horizons, with lower error growth rates.

Please also see response to reviewer kXAS concerning test-time horizon extension and Tab. 1.

Longer test-time forecast horizon

We have now conducted experiments for increased test-time horizon. Please see Tab. 1. SKOLR has a recursive structure. Even if we train over a given horizon, we can recursively predict for a longer horizon. There is performance deterioration, but our results show it is not severe. SKOLR's performance compares favorably with Koopa's.

Non-overlapping patch tokenization

We apply patch processing before Linear RNN branches. This reduces complexity to O(L/P) from timestamps (O(L)). Our efficiency results do use this for baselines (PatchTST, iTransformer, Koopa). It slightly improves baseline performance.

Hyperparameter selection + sensitivity

We will clearly specify the hyperparameter selection protocol in App.A.2. We selected the optimal configuration based on MSE for the validation split. Our setup strictly separates training, validation, and test sets, ensuring no information leakage. Tab. 4 (paper) analyzes how branch number N and dimension D impact performance. Response Tab. 3 provides new results of sensitivity to patch length. SKOLR exhibits little sensitivity to patch length P. We do not tune this in our experiments; we used P = L/6 for all datasets.

Revise Tab. 8

We apologize for inconsistencies in Tab.8. We've verified all results against our original records and fixed the MASE metric issues for the Quarter and Others categories in the revised Tab.8.

Relationship to [1] Orvieto et al. 2023

Thank you for highlighting this paper. We will modify the introduction: "In this work, we consider time-series forecasting, and establish a connection between Koopman operator approximation and linear RNNs, building on the observation made by [1]. We make a more explicit connection and devise an architecture that is a more direct match."

In Related Work, we will add: "Orieto et al. made the observation, based on a qualitative discussion, that the Koopman operator representation of a dynamical system can be implemented by combining a wide MLP and a linear RNN. We provide a more explicit connection, providing equations to show a direct analogy between a structured approximation of a Koopman operator and an architecture comprised of an MLP encoder plus a linear RNN. Although [1] observe this connection, their studied architecture stacks linear recurrent units interspersed with non-linear activations or MLPs. While excellent for long-range reasoning tasks, this departs from the architecture in their App. E. By contrast, our developed architecture does consist of (multiple branches) of an MLP encoder, a single-layer linear RNN, and an MLP decoder. It thus adheres exactly to the demonstrated analogy between Eq. (5) and (8) of our paper. Whereas our focus is time series forecasting, [1] target long-range reasoning. Although it is possible to convert their architecture to address forecasting, performance suffers because it is not the design goal."

We recognize the importance of acknowledging [1], but we don't believe that its existence significantly diminishes our contribution. The connection observed by Orieto et al. is qualitative; there are no supporting equations. In contrast, we develop an explicit connection by expressing a linear RNN in (5) and a structured Koopman operator approximation in (8). This explicit connection adds value beyond the qualitative insights.

Presentation - Sec. 3

We apologize for notational confusion and will correct it. There are strong motivations for the frequency decomposition design choice. In classical time-frequency analysis, the value of adaptation to different frequencies has long been recognized. Wavelet analysis applies different filters at different frequency scales. In more recent forecasting literature, frequency decomposition has been shown to be highly effective in TimesNet (Wu 2023), Koopa (Liu 2023), and MTST (Zhang 2024). Low-frequency dynamics may be considerably different from high-frequency dynamics and are more easily learnt after disentanglement. We are motivated to allow for observable functions to be learned both in frequency and time, with explicit consideration of the frequency aspect.

We will modify Sec.3 to describe the whole pipeline for a single branch, and then describe the multiple-branch case.

Questions: Non-linear systems; Fig. label

Tab. 1 signals are real system measurements, e.g., Electricity Transformer Temperature. We do not know the exact dynamics. The 4 systems in Sec.4.2 are commonly-studied, synthetic non-linear systems. These allow us to study a setting where it is important to model non-linear dynamics. We will write "Synthetic Non-linear Systems" to stress the synthetic nature.

The left figure of Fig.3 corresponds to (a) and the right to (b). We will add clear labels.