TimeBridge: Non-Stationarity Matters for Long-term Time Series Forecasting

Thank you for your feedback and valuable comments. We greatly appreciate the time you spent reviewing our paper. Here are detailed responses to your concerns and questions.

Q1: The paper lacks originality, primarily building on existing methods without presenting a clear, novel contribution.

A1: We propose a novel approach by integrating the modeling of non-stationarity and dependency in long-term time series forecasting—an area where previous works typically address only one of these aspects. In particular, Integrated Attention is innovative in how it addresses non-stationarity while simultaneously capturing short-term dependencies within individual variables, achieved by manipulating the attention map directly. Cointegrated Attention is the first application of the concept of cointegration to long-term time series forecasting. Cointegration is a well-established statistical method in econometrics but has not been applied to forecasting, making its use in this context a novel contribution. We appreciate Reviewer 5kbC's recognition of this as an important advancement in time series forecasting.

---

Q2: The paper doesn’t explore varying input/output (I/O) ratios, which may impact model performance and fairness in comparisons.

A2: We have addressed this concern in our experiments. In Table 2 of the main paper, and the extended results in Table 11, we perform an exhaustive search for the optimal input lengths for all baselines. Specifically, we tested input lengths $I \in 96,192,336,512,720$, and the full search process is detailed in Appendix E.1 (lines 903-908).

---

Q3: The evaluation only considers MSE and MAE, which may not fully capture model effectiveness, especially for relative error.

A3: While MSE and MAE are the standard evaluation metrics used in the time series forecasting literature, including many influential papers such as TimeMixer, iTransformer, PatchTST, and TimesNet, we understand your concern. In response, we have added additional evaluation metrics for relative error: MAPE and MSPE, as shown in Table A in the following comment. These additional metrics further demonstrate the effectiveness of our approach across different evaluation criteria.

---

Q4: The paper does not clearly describe the hyperparameter tuning process for the baseline models, raising concerns about fairness in the comparisons.

A4: We apologize for any confusion. As mentioned in Answer 2, we have performed extensive hyperparameter tuning for all baseline models. The results reported in Table 2 (full results in Table 11) are based on the best-performing configurations for each model, after tuning key hyperparameters such as learning rates, encoder layers, model dimensions, and training epochs. These details are fully described in lines 906-907.

---

Q5: The experimental settings are not reasonable. Why is the input length set to 720, output length $O$ set to 96,192,336,720, and not something more practical, like 1 month of data?

A5: The input length $I$ was selected was selected through a search over $I \in 96,192,336,512,720$, with the best-performing length chosen. For output lengths, $O \in 96,192,336,720$ is commonly used in time series forecasting literature. For example, in Informer [1], for datasets like ETT, an output length of 96 corresponds to 1 month for ETT, where the time interval is 15 minutes (15 mins * 96 = 1 month). For Electricity and Traffic (1-hour interval), 720 also represents 1 month. This output configuration is both practical and widely adopted.

---

Q6: The paper does not assess the algorithm’s complexity, particularly in terms of computational cost, GPU usage, and runtime efficiency.

A6: We include a detailed comparison of the theoretical complexity of our method and other Transformer-based models. The table below summarizes the theoretical computational complexity, where $C$ is the number of channels, $I$ and $O$ are the input and output lengths, and $p$ is the patch length:

Moreover, theoretical complexity alone cannot fully capture real-world performance due to implementation differences. We tested on 2 NVIDIA 3090 GPUs, measuring training (1 epoch) and inference times for three datasets of increasing size, with results averaged over 5 runs in Table B in the following comment.

---

Overall, thank you for your valuable comments. We hope the detailed experiments and clarifications provided above address your concerns. If you have any further questions, please feel free to reach out, and we would be happy to provide additional clarifications. We kindly ask you to reconsider your evaluation in light of these explanations.

[1] Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting

Table A: Additional MAPE (Mean Absolute Percentage Error) and MSPE (Mean Squared Percentage Error) scores, both of which measure relative prediction error.

---

---

Table B: Running time of different Transformer-based methods.

Dear Reviewer nS1F,

Thank you for your valuable feedback during the first round of review. Your comments have provided us with significant insights, and we have conducted detailed experiments to address your concerns thoroughly. We believe these new results offer additional clarity and reinforce the contributions of our work.

We kindly hope you might reconsider your concerns in light of our response and the additional experiments. We would greatly appreciate any further feedback or suggestions you might have, as they would be invaluable in helping us improve our work. Looking forward to your reply.

Best regards,

Paper3735 Authors

Table A: Additional baselines (TSMixer) results for financial forecasting.

---

---

Table B: Additional Results of larger climate dataset.

Thank you for your feedback and insightful comments on our work. We appreciate the time you took to review our manuscript. Below, we address each of your concerns and questions:

Q1: The idea of Integrated Attention and Cointegration Attention is not new. The model, in general, lacks novelty.

A1: While it is true that traditional time series forecasting has explored related concepts, our work is the first to bridge the gap between non-stationarity and dependency modeling in the context of long-term forecasting using deep learning. We provide a comprehensive review of prior works, which primarily focus on either non-stationarity or dependency, but fail to address the interaction between these two aspects. The proposed Integrated Attention is innovative in its approach to removing non-stationarity while capturing short-term dependencies within variables by directly manipulating the attention map, which is a novel aspect of our model. Furthermore, the Cointegrated Attention introduces the concept of cointegration into long-term time series forecasting, marking the first time this technique has been applied in this context. We are encouraged by Reviewer 5kbC's recognition of this contribution as a novel step in the field.

---

Q2: The improvement by TimeBridge is not as much as claimed in the paper (over 10%). On most of the datasets, TimeBridge achieves similar results or marginally better results (Table 1 and 2). The popular benchmarking time series are relatively easier tasks.

A2: We apologize for any confusion caused. The over 10% improvement mentioned in the paper refers to the comparison with the baseline results reported in Table 1, where we compute improvements based on the top two methods (i.e., PDF and PatchTST) across all datasets. As for Table 2, the results include hyperparameter tuning for all baselines. In the paper, we specifically detail TimeBridge's performance compared to other state-of-the-art methods, such as Transformer-based PDF, CNN-based ModernTCN, and MLP-based TimeMixer. For example, TimeBridge achieves a 3.10%/1.64% reduction in MSE/MAE compared to PDF, 3.55%/0.81% compared to ModernTCN, and 6.92%/4.54% compared to TimeMixer. We hope this clarifies the magnitude of improvement. While the benchmark datasets may seem easier, they are widely adopted in time series forecasting research, and they encompass a variety of volatility and non-stationarity patterns. Moreover, we have conducted experiments on larger datasets, as discussed in Q4.

---

Q3: On finance applications, TimeBridge's performance is similar to TSMixer.

A3: We believe there might have been a slight misunderstanding. In Table 4, the model you referred to as TSMixer is intended to be TimeMixer, which was the correct baseline used in our study. To ensure we fully address your concern, we have now added the results for TSmixer [1] (which is a different baseline method) in Table A in following comment. When comparing TimeBridge with TSMixer and other deep learning-based methods, our results show that TimeBridge outperforms the other methods on most metrics, achieving the best performance in 7 out of 12 metrics. The second-best method, Crossformer, only outperforms TimeBridge in 3 out of 12 metrics. We kindly ask you to reconsider our model’s performance in finance applications based on this new information.

---

Q4: What is TimeBridge's performance on bigger time series datasets, such as New York Taxi or Climate Data?

A4: In this work, we have evaluated larger-scale datasets, including the Traffic dataset (862 channels) and PEMS 07 (883 channels), which are considered challenging in prior studies. For example, the Traffic dataset includes hourly road occupancy data from 862 detectors across the San Francisco Bay Area (2015-2016), totaling over 1,000,000 data points. Additionally, we conducted experiments on a larger climate dataset, which records regional precipitation data across 1,763 stations in Australia from [2]. This climate dataset is characterized by high-dimensionality (1,763 channels) and a long time span, with each channel containing 10828-time points. The results, where TimeBridge outperforms all other baselines, are presented in Table B in the following comment.

---

We sincerely appreciate your constructive comments. If our responses have addressed your concerns, we kindly ask if you could reconsider your evaluation. If you have any further questions or need additional clarification, please feel free to reach out, and we would be happy to assist.

[1] TSMixer: An All-MLP Architecture for Time Series Forecasting

[2] Monash Time Series Forecasting Archive

Dear Reviewer aC7S,

We deeply appreciate your valuable feedback during the first round of review and the thoughtful discussion that has significantly helped us refine our work. We hope that our responses have addressed your concerns.

We sincerely appreciate the opportunity to refine our work and kindly hope you might reconsider your concerns in light of our detailed responses and clarifications. Your insights are invaluable to us, and we would greatly welcome any additional feedback or suggestions you may have. Thank you once again for your time and thoughtful consideration. Looking forward to your reply.

Best Regards,

Paper3735 Authors

We sincerely appreciate your positive feedback and valuable insights into our work. Thank you for recognizing our effort in bringing the concept of cointegration into long-term time series forecasting, as well as for acknowledging the clarity of our notations, the novelty of our approach, and the comprehensiveness of our experiments. Below, we address your concerns and respond to your thoughtful questions in detail:

Q1: Figure 3 suggests that the proposed method's building blocks are streamlined and lack integration, leaving room for more systematic treatment of non-stationarity and dependency modeling.

A1: We would like to clarify that our method has been carefully designed with a simple and intuitive network structure, systematically addressing the challenges of non-stationarity and dependency modeling. Specifically:

• Integrated Attention is tailored to capture short-term dependencies within variables while mitigating spurious regression. It achieves this by modeling non-stationarity in a streamlined yet effective manner, utilizing a simplified and innovative technique (i.e., the normalized attention map) to eliminate and reconstruct non-stationary components.
• Cointegrated Attention is designed to preserve non-stationarity and model the long-term cointegration relationships across variables.
• Patch Downsampling serves as a bridge between the two blocks, enabling Integrated Attention Block to process aggregated short-term information that benefits the downstream Cointegrated Attention Block.

To validate our architectural decisions, we conducted an ablation study (Table 6 and Figure 6), which demonstrated the individual contributions of the Integrated and Cointegrated Attention Blocks. The choice of a serial architecture not only proves highly effective in capturing both short-term dependencies and long-term cointegrations but also makes the model easy to understand. Furthermore, the results highlight the necessity of modeling short-term dependencies before long-term ones. This ordering ensures that aggregated short-term information enhances subsequent long-term modeling, while reversing the order leads to information loss and performance degradation.

---

Q2: It would be helpful to revisit traditional cointegration techniques and benchmark stationary and non-stationary series before evaluating the attention-powered module.

A2: Thank you for this valuable suggestion. We will incorporate additional background on traditional cointegration techniques in operations research and other potential domains in the revised version of our manuscript. Specifically, we will expand the Normalization and Dependency Modeling sections in the Related Work to provide a more comprehensive overview of cointegration methods and their relevance. This will serve as a foundation for contextualizing the impact of our proposed attention-powered modules.

---

Once again, we are deeply grateful for your recognition of our work and your constructive feedback. If you have any further questions or suggestions, we would be more than happy to address them.

Dear Reviewer 5KbC,

Thank you for your positive feedback and high evaluation of our work. We are truly grateful for your recognition and would be delighted to hear any additional feedback or suggestions you may have to further improve the study.

Thank you again for your time and support. Looking forward to your reply.

Best regards,

Paper3735 Authors