TimeEmb: A Lightweight Static-Dynamic Disentanglement Framework for Time Series Forecasting

Thanks for your constructive feedback!

W1: The clarification for the conceptual ambiguity about the "time-invariant" component.

To avoid ambiguity, we define the time-invariant component: The "time-invariant" component in TimeEmb refers to periodically recurrent, position-specific(decided by the time index) stability across the dataset, not absolute invariance across all timesteps. The embedding bank achieves "global recurrent features" by learning these stable patterns for each cycle position, ensuring they generalize across all instances of that position in the time series.

W2: How the optimal value of loss weight $\alpha$ is determined for each scenario?

We empirically select the loss weight $\alpha$ from the candidate set {0, 0.25, 0.5, 0.75, 1} and report the optimal results obtained. The detailed values of $\alpha$ are provided in the accompanying code, and a table listing these $\alpha$ values will be included in the main text in subsequent revisions.

W3: Analysis of the performance under a distribution-shifting setting.

As noted in related studies [1], certain datasets—ETTh2, for instance—exhibit marked irregular periodicity and distribution shifts. Our model demonstrates strong performance on such datasets, achieving substantial improvements over existing methods specifically designed to address distribution drift.

Q1: How would TimeEmb perform on time series characterized by highly irregular or chaotic patterns?

As noted in the response to W3, TimeEmb already demonstrates strong performance on datasets with irregular patterns such as ETTh2. Theoretically, even in scenarios of extreme irregularity, TimeEmb can still capture sufficient information: Static components, through learnable embedding banks, capture the underlying long-term stable patterns in data, thereby serving as a reference anchor for chaotic sequences. Even in the absence of traditional periodicity, latent regularities may still be uncovered. In contrast, dynamic components are specifically designed to capture irregular fluctuations and transient changes caused by rare events via frequency-domain filtering, enabling adaptation to abrupt disturbances from non-periodic events. Extreme irregular periodicity remains a valuable long-term research direction. In future work, we plan to expand our model to conduct in-depth analysis of this issue and develop targeted solutions.

Q2: A conceptual diagram or pseudocode illustrating the integration mechanism for TimeEmb.

In our model, sequences are processed in both the frequency domain and the time domain. Specifically, operations in the frequency domain can be interpreted as reshaping the sequence spectrum within the original feature space, while in the time domain, a projection layer is employed to generate the final prediction. Building on this framework, the time-domain projection process in our original model serves as a backbone component. When extending our approach to other models, integration is streamlined: only replace our backbone (i.e., the projection layer) with the alternative models. Notably, this modification yields significant performance improvements with minimal additional parameter overhead.

The pseudocode is as follows:

(Choosable) Instance Normalization($X$)

$\overline{X}$ = $rFFT(X)$

$X_d$ = $\overline{X}$ − $X_s$

$\dot{X}$ = $H_ω$ ($X_d$) + $X_s$

$Z$ = $irFFT(\dot{X})$

$Y$ = $Backbone(Z)$

(Choosable) Reverse Instance Normalization($Y$)

where Backbone is referred to the alternative model.

PS: Whether or not to perform Instance Normalization depends on the alternative model.

[1] Shao, Z., Wang, F., Xu, Y., Wei, W., Yu, C., Zhang, Z., ... & Cheng, X. (2024). Exploring progress in multivariate time series forecasting: Comprehensive benchmarking and heterogeneity analysis. IEEE Transactions on Knowledge and Data Engineering.

Dear Reviewer jvWT:

Thank you once again for your valuable suggestions on our paper – we truly appreciate the insights you’ve shared. As we’re now midway through the rebuttal period, we’d be incredibly grateful if you might share your thoughts on whether our response has adequately addressed the concerns you raised. Your feedback would mean a lot to us, as it will help us refine our work further. We’re also more than happy to engage in any further discussions and remain ready to clarify any additional questions you may have. Thank you again for your time and guidance.

Sincerely,

Paper 20677 Authors

Thanks for your constructive feedback!

W1: Absence of statistical significance reporting for main results.

The detailed results are as follows.

W2: How the learnable $\omega$ adapts across different datasets or non-stationary patterns?

Thanks for discussing the valuable point. From a design perspective, $\omega$ is a complex-valued vector that selectively weights time-varying frequency bands via element-wise multiplication. This design draws on the convolutional theorem in signal processing, equating frequency-domain filtering to time-domain convolution operations—enabling the model to approximate any linear transformation through end-to-end optimization and adapt flexibly to diverse data.

Cross-dataset adaptation:

• For low-frequency dominant data (e.g., power grids with diurnal patterns), $\omega$ amplifies low-frequency weights to capture stable fluctuations.
• For high-frequency rich data (e.g., weather with bursts), it prioritizes high-frequency bands to track transient changes.

Non-stationarity handling:

• In stable phases, $\omega$ weakens high-frequency noise for smoothness.
• During anomalies, it boosts high-frequency weights to capture abrupt shifts.

This adaptive weighting mechanism can be interpreted as learning the frequency response function of a linear time-invariant (LTI) system, enabling the model to robustly handle diverse spectral characteristics and dynamic shifts without the need for manual tuning.

W3: More discussion and comparison with time series papers.

Thanks for your suggestion, we will add two more categories of related discussions—LSTM-based models and LLM-based models to Related Work in the next release.

Q1: The impact of Instance Normalization in the context of non-stationary time series.

In time series forecasting, Instance Normalization (IN) differs from Batch Normalization (BN) and Layer Normalization (LN) in key ways. BN normalizes using batch-level statistics, which can blur temporal variations in non-stationary data (e.g., mixing peak and off-peak periods in a batch) . LN normalizes across features for a single sample, potentially smoothing critical local fluctuations. IN, by contrast, normalizes each instance independently at every timestep, preserving instance-specific temporal patterns. This suits non-stationary time series, as it avoids distorting dynamic shifts, providing a stable foundation for subsequent frequency decomposition and embedding learning in TimeEmb.

Thanks for your constructive feedback!

W & Q1: A fully adaptive, data-driven approach to learning multi-scale structures.

This is an excellent observation that merits further exploration. The embedding bank mechanism proposed in our paper is specifically designed to model regular periodic patterns—such as daily, weekly, or other recurring temporal structures—enabling it to effectively handle tasks characterized by such regular periodicity. Given the constrained search space of regular periodic patterns, manual parameter adjustment (e.g., setting M as discussed earlier) is more efficient in practical applications than fully adaptive learning mechanisms. Moreover, the regular periodic characteristics of most datasets can be pre-analyzed to guide this process. For datasets with irregular periods and mean drift, such as ETTh2 [1], our model still shows excellent performance. However, extreme irregular periodicity remains a valuable long-term research direction, and we plan to expand our model to conduct in-depth analysis of this issue and develop targeted solutions.

Q2: Specific types of insights for the disentangled components in real-world time series applications.

In this paper, we focus on one of the most critical tasks in long-term time series forecasting. Looking ahead, the time-invariant components modeled in our framework hold significant potential for extension to a broader range of related tasks. For instance, they can be leveraged to enhance performance in scenarios such as forecasting with missing values, data imputation, and other temporal data modeling tasks. From a practitioner’s perspective, consider the application of our framework in Energy Grid Management: Static (time-invariant) component: This component, learned via the embedding bank, captures consistent daily electricity usage patterns—such as recurring morning and evening demand peaks. For practitioners, this insight enables proactive grid resource allocation: for example, activating backup generators or pre-positioning energy reserves ahead of anticipated peak periods to ensure stable supply. Dynamic (time-varying) component: By contrast, this component identifies short-term, irregular fluctuations, such as sudden demand spikes triggered by extreme weather events (e.g., heatwaves driving increased air conditioning use). Armed with this real-time information, operators can make agile adjustments to supply—such as ramping up output from renewable energy sources or redistributing power across the grid—to mitigate risks of overloads or blackouts.

[1] Shao, Z., Wang, F., Xu, Y., Wei, W., Yu, C., Zhang, Z., ... & Cheng, X. (2024). Exploring progress in multivariate time series forecasting: Comprehensive benchmarking and heterogeneity analysis. IEEE Transactions on Knowledge and Data Engineering.

Dear Reviewer pNtD:

Thank you once again for your valuable suggestions on our paper – we truly appreciate the insights you’ve shared.
As we’re now midway through the rebuttal period, we’d be incredibly grateful if you might share your thoughts on whether our response has adequately addressed the concerns you raised. Your feedback would mean a lot to us, as it will help us refine our work further.
We’re also more than happy to engage in any further discussions and remain ready to clarify any additional questions you may have.
Thank you again for your time and guidance.

Sincerely,

Paper 20677 Authors

Thanks for your constructive feedback!

W1: The rationale for fusing the separated time-invariant and time-variant components in the feature space.

In our paper, we decompose complex time series into time-varying and time-invariant components, which are then modeled independently to enable each factor to learn a more precise representation. These two factors are recombined to integrate information to derive the final comprehensive forecasting representation.

W2: Explanation of TimeEmb's periodicity handling and differences from CycleNet

First, due to the patterns of human activities, time series data such as electricity consumption and traffic flow have stable periodic basic patterns. Second, while existing works have extensively focused on modeling such stable patterns [1], our work diverges by explicitly decoupling time-invariant static patterns (capturing long-term stable periodicities) and time-varying dynamic components (reflecting transient fluctuations). This design choice not only enhances interpretability but also enables more targeted modeling. Third, compared to CycleNet, we would like to kindly emphasize that it requires a strong expert knowledge through the RCF algorithm, that is, the length of a stable periodicity. By contrast, we simply model daily patterns, and have achieved optimal results in nearly all experimental settings without the limitation on domain expertise. Finally, we fully concur with your emphasis on the importance of modeling complex periodicity in time series analysis. We highlight that the core innovation of this paper lies in proposing an explicitly decoupled framework for complex time series, an efficient, interpretable, and compatible modeling scheme. To address more intricate periodic patterns, we plan to explore adaptive multi-period modeling methods in future work. Thank you again for this insightful feedback. For details on the selection of M, please refer to Q2.

W3: The clarification for the discrepancy between the model in the paper and the implementation in the code.

We kindly ask you to specify the inconsistency. In Section 3.2, we transform the original sequence into the frequency domain via rFFT, which is consistent with the implementation provided in the accompanying code.

Regarding the reviewers’ observation on the differential handling of real and imaginary parts, this likely refers to the subtraction of real components described in Section 3.3. To mitigate computational and storage overhead associated with the embedding bank, we leverage the properties of complex conjugation: specifically, we only use the real part of $\overline{X}$ for decoupling. Critically, the real and imaginary components are reintegrated before frequency-domain filtering, aligning with the procedures detailed in this paper(Line 180).

W4: The concrete explanation for the extensions of TimeEmb.

Thanks for pointing it out. In our model, sequences are processed in both the frequency domain and the time domain. Specifically, operations in the frequency domain can be interpreted as reshaping the sequence spectrum within the original feature space, while in the time domain, a projection layer is employed to generate the final prediction. Building on this framework, the time-domain projection process in our original model serves as a backbone component. When extending our approach to other models, integration is streamlined: only replace our backbone (i.e., the projection layer) with the alternative models. Notably, this modification yields significant performance improvements with minimal additional parameter overhead. Thanks again, and we will add this introduction in the next version.

Q1: The reason for performing fusion before transforming back into the time domain.

In contrast to existing methods such as DLinear, where each separated feature is projected into the prediction dimension before fusion, our approach prioritizes feature fusion. This design ensures that decoupling and fusion operations are performed within the original feature space, thereby mitigating the risk of information loss that may arise from premature dimensionality changes. Thank you for this valuable discussion. We have conducted experiments to validate this approach, and the results confirm that the "project-first-then-fuse" strategy leads to significant performance degradation and incurs computational overhead.

Q2: The choice of the embedding bank size M.

Thanks for highlighting this. We use the embedding bank to learn global time-invariant dataset information. The choice of M is based on common sense and requires no additional expert knowledge. Setting $M = 24$ captures daily patterns; $M = 7$ or $12$ gives weekly or half-daily patterns if needed. The setup of $M$ is straightforward, robust, and knowledge-free.

Q3: Additional experimental results for the compatibility analysis.

To verify compatibility, we select representative baselines within our study. As per your requirement, we conducted further experiments on the ETTh2 dataset. These consistently demonstrate superior performance when models are equipped with our TimeEmb. Furthermore, the additional trainable parameters introduced by integrating TimeEmb are negligible.

Q4: The explanation for the discrepancy between our definition and the observed clustering pattern.

The observed clustering pattern of time-variant components(${X_d}$) in Figure 4(b) and their day-of-week specificity can be reconciled with our definition by clarifying the granularity and nature of the disentanglement:

First, the time-invariant component(${X_s}$) captured by our embedding bank is designed to model long-term stable patterns across dataset (e.g., general daily rush hours in traffic or hourly electricity consumption baselines). In Figure 4, we explicitly configure the embedding bank with M=7 to capture week-level invariant patterns (i.e., shared characteristics across the same day of the week, such as typical Monday vs. Sunday behaviors). Thus, the core week-level stability is already encoded in ${X_s}$.

Second, the time-variant component reflects residual fluctuations that deviate from these stable patterns. While $X_d$ exhibits day-of-week clustering, this does not imply it is "stable" in the long term. Instead, these clusters represent subtle, context-dependent variations within each day type (e.g., additional electricity consumption on Monday due to factory start-up, and fluctuations in electricity consumption on Sunday due to increased household activities). These variations are not universal enough to be considered part of the invariant $X_s$, as they lack the persistent, dataset-wide consistency required for the time-invariant component.

In essence, the day-of-week clustering in $X_d$ arises from the fact that $X_s$ has already subtracted the most dominant week-level invariants, leaving behind structured but non-stationary residuals that retain contextual dependencies. This aligns with our definition: $X_d$ captures local dynamics that, while partially structured, are not stable enough to be included in the time-invariant component.

[1] Zeng, A., et al. (2023, June). Are transformers effective for time series forecasting?. AAAI, 2023.

Dear Reviewer 9Nmb:

Thank you once again for your valuable suggestions on our paper – we truly appreciate the insights you’ve shared.
As we’re now midway through the rebuttal period, we’d be incredibly grateful if you might share your thoughts on whether our response has adequately addressed the concerns you raised. Your feedback would mean a lot to us, as it will help us refine our work further.
We’re also more than happy to engage in any further discussions and remain ready to clarify any additional questions you may have.
Thank you again for your time and guidance.

Sincerely,

Paper 20677 Authors