Learning Robust Spectral Dynamics for Temporal Domain Generalization

We sincerely thank the reviewer for your valuable comments, which significantly strengthen our work. We are particularly grateful for your recognition of our contributions' key strengths: technical novelty, theoretical foundation, comprehensive experiments, and presentation quality. Below we address all concerns point-by-point.

W1: Explanation about $\mathbf{K}$

Learning $\mathbf{K}$: In our implementation, $\mathbf{K}$ is learned via a standard linear layer. It is not learned in isolation but is jointly optimized with the encoders and decoder via backpropagation as defined in our overall objective Eq. (11). Thank you your suggestion, and we will refine this part in the revised version.

W2: Clarification of K's spectral radius ρ(K) < 1.

We appreciate your suggestion on the learning mechanism for the transformation $\mathbf{K}$. As the reviewer correctly infers, ensuring the stability of $\mathbf{K}$ is crucial for preventing error explosion and achieving a tighter theoretical bound. To address this, we actively enforce a stability constraint during training in our code (please refer our offered code of the model implementation). Therefore, this condition is not merely a theoretical assumption for the tighter bound in Lemma 1; it is a practical constraint actively enforced in our implementation. This ensures that the learned operator is stable, which in turn helps to minimize error accumulation and leads to the more robust long-term forecasting performance demonstrated in our experiments.

W3: Explanation about the relationship between parameter evolution and data distribution

We thank the reviewer for this excellent question regarding the connection between parameter and data dynamics.

Our framework's end-to-end learning process directly establishes this relationship. As detailed in Algorithm 1, the parameter trajectory $\Theta$ is learned jointly by minimizing the task loss across all data domains. This ensures that the learned parameter evolution is an intrinsic reflection of the shifts in the data distribution $P(X,Y)$.

Therefore, a strong periodical component in $\Theta_{low}$ is indeed a direct consequence of a periodical shift in the data itself, as this is the optimal trajectory for minimizing loss on such data. We empirically validate this relationship in our Periodicity Modeling experiment (Section 4.4). The visualizations in Figure 3(b) provide clear evidence that our model successfully captures the induced periodical structure in the parameter space.

Thank you for the insightful question. We will emphasize this connection more explicitly in the revised manuscript.

(W4/W5) "Periodical" vs. "Periodic" and Missing reference information [11]:

Thank you for your suggestion. We will revise the manuscript to consistently use "periodical" for improved clarity and uniformity, and will thoroughly verify all content—including references—to ensure accuracy.

We appreciate your insightful feedbacks. To improve clarity, we have reorganized the comments from weaknesses/questions/limitations into a unified structure (order may differ from the original). This helps streamline the review process for reviewers/ACs/SACs.

1. Task and setting

1.1. Static vs. continuous settings (W4/W5/Q3): Our experimental design ensures fair comparison with SOTA TDG baselines [6,8,12]—the established standard in this field, i.e., discrete domains. Your mentioned concerns are talked in Limitations (Lines 376-377). Crucially, we considered the limitation arising from this discrete domain segmentation in Introduction (Lines 45-51). This represents a significant advance over conventional TDG approaches. In addition, we acknowledge that we did not consider the continuous TD. However, we have included the continuous TD methods (EvoS[nips2024]，Koodas[nips2024]) as the comparision, demonstrating our method's competitive performance. Our novel spectral analysis framework for parameter trajectory prediction remains the primary contribution, rigorously validated through theoretical analysis and experiments.

Abrupt distribution shifts: As discussed in Related Works (Lines 119-129), TDG fundamentally aims to leverage predictable evolutionary dynamics for proactive generalization. Abrupt shifts usually means unpredictable and thus violate the core assumption of TDG methods including ours, which is beyond this paper's scope.

1.2. (W4): While we did not use your suggested datasets ["A century of portraits: A visual historical record of american high school yearbooks"; "The clear benchmark: Continual learning on real-world imagery"], our evaluation was conducted on 5 real-world benchmarks which are popular in TDG and concept drift. They feature varying numbers of features, complex drift patterns and real-world uncertainties. The consistent SOTA performance across these diverse tasks demonstrates the practical utility and robustness of FreKoo in real-world scenarios.

We sincerely appreciate your valuable suggestions and references. These will be incorporated in the revised manuscript.

2. Assumptions (W1/W3/Q2)

2.1. Lipschitz assumption (W1): The Lipschitz assumption is a widely-used tool in the theoretical analysis of domain adaptation and generalization ["Continuous temporal domain generalization, nips2024"; "Fine-grained analysis of optimization and generalization for overparameterized two-layer neural networks, ICML2019"]. The primary purpose of this assumption is to provide a formal framework for analyzing how errors in the latent parameter space propagate to the final prediction risk. On the one hand, the high-frequency component is not forward-predicted but regularized via temporal smoothness, which naturally suppresses sharp transitions in the parameter space. This acts as an implicit regularizer that effectively controls local Lipschitz behavior in practice. On the other hand, while the Lipschitz constant of deep neural networks is theoretically unbounded, our implementation employs weight decay. This standard regularization technique penalizes large weights, which in turn empirically constrains the model's complexity and helps control its Lipschitz constant in practice.

Thank you so much for raising this point to help enhance the theoretical soundness of our work. We shall provide further clarification regarding these assumptions in the revised version

2.2 Smooth parameter evolution (W3/Q2:): We acknowledge that multiple optimal solutions may exist if we treat each domain's optimization independently. However, our method's core strength is its joint optimization (Eq.(11)) in an end-to-end way, which selects a solution sequence $[\theta_1, \theta_2, \dots, \theta_T]$ that explicitly enforces a coherent temporal structure. This objective function explicitly regularizes the learned parameter trajectory to exhibit desirable temporal properties. By minimizing this, our framework navigates the complex solution space to select a sequence of parameters that not only performs well on historical tasks but is also structurally predictable and smooth.

3. Long-range dependencies (W2/Q1)

3.1. Our method captures long-range dependencies from $0 \rightarrow T$:

• Spectral decomposition is applied to the entire historical parameter trajectory $\Theta = [\theta\_1, \theta\_2, ..., \theta_T]$. This transformation inherently encodes the full history of parameter evolution into the frequency domain. Long-range dependencies naturally manifest as concentrated energy in the low-frequency components ($\Theta\_{low}$).
• The Koopman operator learns the dynamics on the whole low-frequency latent representations via Eq.(6). Since $\Theta\_{low}$ already encapsulates information from the full history, the learned evolution $\hat{\theta}_{t+1} = \phi^{-1}(\mathbf{K} \phi\_{low}(\theta\_{t, low}) + \phi\_{high}(\theta\_{t, {high}}))$ is effectively propagating a state that is imbued with historical context.

3.2 Theorem 1: The primary goal of this theorem is to offer a qualitative decomposition of the generalization error. It provides a principled justification for our algorithm's dual-component design by showing how the total risk can be broken down into: the low-frequency dynamics ($\mathcal{E}\_{low}$) controlled by the stability of the Koopman operator (Lemma 1), and the high-frequency noise ($\mathcal{E}\_{high}$) managed by temporal smoothing regularization (Lemma 2). While the bound is presented for a single step, the logic applies inductively, and the state used for prediction already contains the information from all prior $T$ domains.

We also extended it. We focus on the propagation of the low-frequency latent error $e_{t,low}= \hat{z}\_{t,low}- z^{\*}\_{t,low}$. By recursively analyzing the single-step error recurrence $e_{t+1,low} = K e_{t,low} + e_{t,low}$, where $e_{t,low} = K z^{\*}\_{t,low} - z^{\*}\_{t+1,low}$ is the single-step model mismatch, we can express the error after an $h$-step predictions as:

$e_{t_0+h,low} = K^h e_{t_0,low} + \sum_{i=0}^{h-1} K^i e_{t_0+h-1-i, low}$

Taking the norm, we get the multi-step error bound:

$\|e_{t_0+h,low}\| \le \|K^h\| \cdot \|e_{t_0,low}\| + \sum_{i=0}^{h-1} \|K^i\| \cdot \|e_{t_0+h-1-i, low}\|$

This formulation reveals that the final error is not only dependent on the initial error ($e_{t_0,low}$), but also on the sum of all previous errors ($e_{t,low}$) accumulated along the entire prediction path.

This provides a stronger justification for our full-trajectory optimization approach: The stability of $K$ Lemma 1 bounds $\|K^i\|$ and controlls the propagation of intermediate errors; The objective function minimizes losses across all domains, directly works to reduce the magnitude of each single-step mismatch error $\|e_{t,low}\|$, thereby minimizing the second term in the bound. A more detailed discussion will be included in the revised appendix.

4. Computational and memory overhead (W6)

The computational cost of the additional components is modest. The Fourier Transform is efficient (O(T log T)) and applied only to the short sequence of $T$ domains. The encoders and Koopman operator are lightweight MLPs and a linear layer adding minimal overhead. To prove this, we compared the running time with the most popular DRAIN method.

As the results show, FreKoo's training time is competitive and comparable to that of DRAIN. We think the introduced overhead is manageable and highly justified by its superior performance. The added components are vaulable for modeling the complex dynamics that simpler methods fail to capture.

5. Analysis of $\tau$ (Q4)

We analyzed the sensitivity to the energy preservation ratio $\tau$ in Section 4.5. The optimal $\tau$ reflects a principled trade-off: higher $\tau$ is better for clean structured drifts (2-Moons), while a more moderate $\tau$ excels on noisy real-world data (Appliance) by filtering out disruptive high-frequency components. We explicitly highlight adaptive threshold $\tau$ as a future work in our Limitation (Lines 374-376). Our current work establishes the foundational effectiveness of using a fixed partition, paving the way for these more advanced adaptive methods.

6. Discuss about Koopman operator (Q5)

We thank the reviewer for this excellent and insightful question.

Rather than assuming linearity in the original system, Koopman theory finds a latent space (via a powerful nonlinear mapping $\phi$ learned by the encoder) that can be linearized [A data–driven approximation of the koopman operator: Extending dynamic mode decomposition, Journal of Nonlinear Science 2015]. Theoretically, $\phi$ can approximate many nonlinear systems (per the universal approximation theorem) given sufficient capacity. In this sense, Koopman theory provides a framework for modeling nonlinear dynamics. However, for fully chaotic systems with irregular or unpredictable dynamics, we believe modeling becomes inherently difficult, and the Koopman framework may also fail. This may go beyond the task assumptions of TDG, but it remains an important and thought-provoking direction for future exploration.

FreKoo vs. RNN-based methods: Actually, DRAIN is an RNN-based method. However, DRAIN attempts to fit noisy parameter trajectories directly with a single nonlinear model. In contrast, FreKoo simplifies the learning process by explicitly decomposing the trajectories into low-frequency (signal) and high-frequency (noise) components using the Fourier transform, before modeling the dynamics. Our experimental results (see Table 1) also show that FreKoo consistently outperforms DRAIN.

We sincerely thank the reviewer for your valuable feedbacks. We are also grateful for the reviewer's recognition of our work's strengths, including its technical novelty, theoretical arguments, and solid empirical results. We address your concerns and questions below.

W1：Writing

1. Writting logic: We apologize for making confusion, and we briefly outline the logical flow to hopefully provide a clearer step-by-step picture:

• Step 1: A crucial point we wish to clarify is that our method is an end-to-end joint optimization process. We do not first discretely learn an independent optimal $\theta_t$ for each domain $\mathcal{D}_t$. Instead, the entire parameter trajectory $\Theta = [\theta_1, \theta_2, ..., \theta_T]$ is learned jointly by minimizing our overall objective function (Eq. 11). We will claim this in the revised verson.

• Step 2: Spectral Decomposition (Section 3.2.1): Within this end-to-end learning loop, we apply the Discrete Fourier Transform (DFT) to decompose parameter trajectory into a low-frequency component $\Theta_{low}$ (capturing stable, long-term trends and periodicity) and a high-frequency component $\Theta_{high}$ (representing transient noise and uncertainties). This decomposition is the core of our frequency-aware perspective.

• Step 3: Dual-Strategy Dynamics Modeling (Section 3.2.2): We then apply a distinct strategy to each component: For $\Theta_{low}$, we learn a Koopman operator $\mathbf{K}$ to model its dominant dynamics in a latent space. This is guided by the $\mathcal{L}\_{{rec}}$ loss. For $\Theta_{high}$, we apply a temporal smoothing regularization ($\mathcal{R}\_{{high}}$). This prevents overfitting to domain-specific noise/uncertainties.

• Step 4: Finally, we give the prediction for the future parameter $\hat{\theta}_{t+1}$ and the joint optimzation objective, following by the Theoretical analtsis in Section 3.3

2. Position of Algorithm 1: We are sorry to place Algorithm 1 in the Appendix due to the strict page limitations for the initial submission. Thank you for pointing out its importance.

We are grateful for your suggestions, and we will carefully revise the manuscript to improve its clarity and flow.

W2：Vague claims

1. Explanation of challenges and current limitations. We apologize for the lack of mathematical formulations. To clarify:

• Challenge 1 (Periodicity): we refer recurring drift defined in [Learning under concept drift: A review. TKDE 2019] to formulate this challenge, i.e., $\exists L \in \mathbb{Z}^{+} : P_t(\mathbf{X},Y) \neq P_{t+1}(\mathbf{X},Y)$ and $P_t(\mathbf{X},Y) \approx P_{t+kL}(\mathbf{X},Y) \forall k \in \mathbb{Z}^{+}$, where $L$ is Period length. Current TDG methods assume monotonic/smooth drift [6,8,12] failing to model such periodicity, and Section 4.4 experiment provide a experimental evidnece.

• Challenge 2 (Uncertainties): Characterized by transient noise and domain variations that violate IID assumptions within temporal segments (Intro L45-48). While mathematically challenging to formalize, we provide intuitive visual evidence of these uncertainties in real-world scenarios through Appendix Figure 5. To our knowledge, no existing TDG method explicitly addresses this challenge.

We appreciate your suggestion again and will refine these points in the revision.

2: We apologize for making confusion regarding the averaging operation for computing spectral energy (Eq.3). If we were to select dominant frequencies based on each parameter dimension $d$ individually, we might select different sets of frequencies for different dimensions. This could lead to a fragmented and incoherent model of the overall dynamics. More critically, a single parameter dimension might exhibit a spurious, high-energy oscillation due to optimization noise or artifacts specific to that dimension. A model that latches onto this "localized noise" would not generalize well. By averaging the spectral magnitudes across all $D$ dimensions to get $M_f$, we are effectively performing a pooling operation. This computes a summary statistic that represents the overall importance of frequencyto the evolution of the entire parameter vector. We will revise the manuscript to incorporate these clarifications.

W3: The role of Koopman operator.

We sincerely thank the reviewer for this insightful comment and the thoughtful parallel drawn to Kernel SVMs. We would like to clarify that the Koopman framework is not just "jargon," but a fundamental concept that guides our entire model design, distinguishing it from a generic linear transformation. The core of our data-driven Koopman approach is to simultaneously learn both the nonlinear feature map $\phi$ (the encoder) and the linear transfermation $\mathbf{K}$. We aim to discover a nonlinear coordinate system via our encoder $\phi$ where the dynamics become globally linear. This is a significant departure from using a fixed feature map, which would be analogous to a "linear kernel". Our deep encoder learns a highly complex, data-driven feature map—a practice established in modern Koopman analysis [Deep learning for universal linear embeddings of nonlinear dynamics. Nature communications 2018]. Therefore, a more fitting analogy for our method would be learning the kernel function itself, rather than simply invoking a linear kernel. The "infinite-dimensional" nature of the Koopman operator is addressed by leveraging the universal approximation capabilities of a deep neural network $\phi$ to find an optimal finite-dimensional basis where a linear operator $\mathcal{K}$ suffices. In summary, the Koopman framework is not an unnecessary descriptor but the foundational blueprint for our model's architecture: it justifies the search for a nonlinear embedding $\phi$, dictates the linear structure of the latent dynamics $\mathcal{K}$, and provides the rationale for why this combination is effective for complex temporal dynamics.

We are grateful for this constructive feedback, as it will undoubtedly help us improve the accessibility and impact of our paper. We believe this revised presentation will strike a better balance, making the method easy to follow while still retaining the valuable theoretical context that Koopman theory provides.

W4: On the method

We thank the reviewer for this insightful comment on the elegance of our method and for raising the important point about the long-term value of simplicity. We appreciate the opportunity to discuss the design philosophy of FreKoo and the alternatives like Transformers.

On the method's principled design: While FreKoo involves several components, we think that this is not an arbitrary integration but rather a principled and structured design derived from a new frequency-aware perspective on TDG. We decompose the parameter trajectory into a stable low-frequency component (long-range dynamics) and a transient high-frequency component (uncertainties). Each part of our method is a dedicated tool for this decomposition: the Fourier Transform for disentanglement, the Koopman operator for stable extrapolation, and targeted regularization for noise suppression. This structured approach is supported by a rigorous theoretical analysis (Theorem 1) and, importantly, provides a novel interpretable solution to TDG by allowing for the analysis of distinct temporal drift components.

On a Transformer-based Alternative: Thank you for the excellent suggestion of a Transformer-based baseline. A Transformer is indeed a powerful sequence model. We have not yet conducted this comparison, but we believe it is a very valuable direction for future work. Investigating the trade-offs between our structured-bias approach and a powerful general-purpose model like a Transformer would be highly insightful and we are grateful for the suggestion.

We sincerely appreciate your valuable feedback and are grateful for the reviewer's recognition of our work's technical rigor, theoretical foundation, comprehensive experiments, and clear writing.

W1：Runing time

We thank the reviewer for this suggestion regarding experimental completeness. We agree that running time analysis is critical for assessing the practical feasibility of our method. As shown in the following table, we conducted running time tests on three classification datasets and one regression dataset, comparing against DRAIN, the most classic TDG method. We used the same batch size and number of training epochs for both methods to ensure a controlled comparison. As the results show, FreKoo's training time is competitive and comparable to that of DRAIN. This demonstrates that the additional components in our framework, such as the spectral decomposition and Koopman operator, do not introduce a significant computational overhead. Crucially, while maintaining similar efficiency, FreKoo consistently outperforms DRAIN in terms of generalization accuracy (as shown in Table 1). This highlights the key advantage of our approach: it provides superior performance without sacrificing practical feasibility.

Runing time (s)

W2：Comparison with TKNets.

Thank you for your suggestion. Actually, when we initially designed the experiments, we considered the TKNets method. However, since the datasets used in that paper differ from ours, we had no optimal parameter settings to reference, so we did not include it in our initial comparison. Nevertheless, we have already cited that paper in our work, i.e., Ref[11]. The baseline methods currently compared in our paper were all obtained directly from their original publications, making them more reliable. To address your concern, we have now attempted to re-train TKNet on two classification and one regression datasets. The results are presented in the following table.

While the TKNets uses of Koopman theory in TDG by modeling the evolution of the data distribution, our FreKoo framework offers a distinct and robust perspective. Instead of analyzing the data space, FreKoo introduces a novel frequency-aware view by directly modeling the dynamics of the model parameters themselves. Our approach is specifically designed to tackle two critical challenges not addressed by prior works: the robust modeling of long-range periodicity and the filtering of domain-specific uncertainties. By decomposing the parameter trajectory into low- and high-frequency components and applying a dual-modeling strategy — Koopman extrapolation for structure and temporal smoothing noise. FreKoo provides a new principled solution that excels in scenarios with complex temporal dynamics. Thank you so much for your suggestion and we will further discuss this point in the revised version.

Q1：Will the number of parameters affect the performance?

We thank the reviewer for this insightful question. Yes, the number of parameters will affect performance. As with most deep learning models, changes to the architecture—such as modifying the backbone or altering the structure of the encoder-decoder—would lead to a different number of parameters and a corresponding change in performance, which could be either an improvement or a decline. This is a general characteristic of deep models and has also been analyzed in prior works like DRAIN. To ensure a fair and controlled comparison, we maintained a consistent backbone architecture for all baselines, as detailed in our Appendix C.3. This approach mitigates the risk of performance differences arising from architectural inconsistencies rather than the novelty of the methods themselves.

Nonetheless, we believe this is an important question. The interplay between model complexity, the number of parameters, and generalization performance in the context of TDG is a valuable area for deeper investigation. We plan to explore this potential relationship in our future work. Thank you again for raising this thoughtful point.

Q2：Will overparameterization/overfitting of the model affect the selection of high and low frequency components?

We thank the reviewer for this very insightful question that probes the core of our method. Yes, overparameterization and the resulting overfitting will affect the selection of high and low-frequency components. This is a crucial challenge that we explicitly considered in our design. If a model overfits to the transient noise within each domain $D_t$, the learned parameter trajectory $\Theta$ will become erratic and dominated by high-frequency oscillations. This would "pollute" our spectral decomposition, causing energy to leak from the true low-frequency dynamics into the high-frequency bands, thus corrupting the signal we aim to model. To address this, our FreKoo framework is designed with built-in temporal regularization through its joint optimization objective (Eq. 11):

1. The Koopman loss ($\mathcal{L}_{{koop}}$) penalizes unstructured, unpredictable low-frequency trajectories. This forces the model to find a more regular and stable underlying dynamic, implicitly resisting overfitting.
2. The high-frequency regularization ($\mathcal{R}_{{high}}$) directly penalizes sharp, noisy fluctuations in the parameter trajectory.

These components act as a powerful regularizer against overfitting in the temporal dimension. They guide the model to learn a parameter trajectory that is not only accurate on the training data ($\mathcal{L}_{{total}}$) but is also temporally smooth and predictable. This ensures a cleaner and more meaningful separation between the low-frequency signal and high-frequency noise, even in the presence of overparameterization.

Q3：About hyper-parameter Analysis

Thank you for your question regarding parameter tuning. For hyperparameters $\alpha$, $\beta$, and $\gamma$, we used traditional grid search over $[0.01, 0.1, 1, 10, 100]$ while fixing other parameters, as described in Section 4.5. For the learning rate, we explored $[10^{-4}, 10^{-3}, 10^{-2}]$—a limited range that kept the tuning process manageable. These are standard ML tuning approaches, though admittedly not always adaptive. The spectral energy preservation ratio $\tau$ remains our most important parameter. Different real-world datasets exhibit varying drift types, frequencies, and uncertainties, necessitating dataset-specific tuning. Since $\tau$ is confined to [0,1], this adjustment remains relatively straightforward. Regarding the Appliance dataset, however, our method achieves SOTA (Table 1). Could you please let us know which part is your concern.

Thank you for raising this point. To be honest, parameter tuning remains a less-than-ideal aspect of our approach. We will focus on this area for further research in future work.