Periodicity Decoupling Framework for Long-term Series Forecasting

Thanks for your hard work and valuable comments of this work. Here are responses for your concern and questions:

Question1: Unlike the frequency selection strategy in TimesNet, the reason for using a different strategy in the Multi-periodic Decoupling Block needs to be further explained.

Response1: Thanks for your suggestions. TimesNet strictly chooses $\text{top-}k$ amplitude values and obtain the most $k$ significant frequencies, which leads to increasing computational burden. Instead, we select frequencies not only focus on high amplitude but also incorporate those with significant values and amplitude. We assert that frequencies with high amplitude better represent the primary components, while those with larger values facilitate a more discernible distinction between long-term and short-term relationships. In this way, our method enhances the decoupling ability of long-term variations and reduces the computational complexity.

Question2: Were the experimental data in the article averaged over multiple runs with different random seeds? This can enhance the stability of the experimental results.

Response2: Following the settings in the previous work PatchTST, we selected a random seed of 2021 and conducted multiple experiments to ensure a fair comparison, as shown in Table 1 in the paper. Furthermore, as suggested, we have tested our method with two additional random seeds (2022 and 2023), where the results are reported in Table A below, from which we can see that our method performs stably with random seeds.

Table A: Results of MSE/MAE for PDF under different random seeds.

Question3: The experimental section mentions that the significant performance gain of TiDE in traffic largely stems from static covariates. It would be preferable to provide experiments to substantiate this claim.

Response3: Thanks for your suggestions. We removed the predefined static covariates in TiDE and reimplemented it on the traffic dataset, where the results are shown in Table B. It can be seen that the performance of TiDE without static covariates still performs worse than our method.

Table B: Results of MSE/MAE with PDF and TiDE (without static covariates) on the Traffic dataset.

Question4: What are the effects of different periods? Since the proposed method lies in periodic to decouple the original time series.

Response4: Long time series generally exhibit multi-periodicities. By decoupling different periods into 2D tensors with varying periods, we can theoretically transmute data into multiple resolutions. Table C below provides performance comparison between a single period ($k=1$) and five periods ($k=5$). The results show that the selection of multiple periods effectively enhances the predictive performance of the model.

Table C: Results of MSE/MAE with varied period counts ($k$) on the Electricity dataset.

Question5: Can the proposed method applied to other time series applications, like time series classification?

Response5: Yes! Although our method is designed for modeling long time sequences, it can also be applied to classification tasks by modifying the prediction head. We will explore the potential applications of our method in future works.

Thanks for your hard work and valuable comments of this work. Here are responses for your concerns and questions:

Question1: How to decouple the time series remains an open question. Although the authors propose a simple yet effective periodicity-based strategy to decouple the time series, how to evaluate the effectiveness of decouple strategy has been less explained.

Response1: Thank you for your valuable suggestions. In theory, there are various ways to evaluate the effectiveness of decoupling strategies. The most straightforward is to compare the performance (e.g. MSE and MAE) of models constructed using various decoupling strategies or assessing their computational complexity， which we adopts for evaluating our method (More details can be found in Appendix D). Another feasible approach may be to separately compute the similarity between the decoupled long-term or short-term variations and the original sequence or subsequence.

Question2: Experiments show the computational efficiency of the proposed methods over other transformer-based methods. However, the authors have not compared the running time of different methods.

Response2: Thanks for your suggestions. In fact, we have reported the Multiply-Accumulate Operations (MACs) among PDF and two other patch-based Transformer methods (PatchTST[1] and Crossformer[2]) in Table 3 in the paper. As suggested, we have compared the MACs results of two additional CNN-based methods (TimesNet[3] and MICN[4]) In Table A below.

Table A: Results of Multiply-Accumulate Operations (MACs) for TimesNet and MICN.

[1] Nie, Yuqi, et al. "A time series is worth 64 words: Long-term forecasting with transformers.", 2022, https://arxiv.org/pdf/2211.14730.

[2] Zhang, Yunhao, and Junchi Yan. "Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting.", 2022, https://openreview.net/pdf?id=vSVLM2j9eie.

[3] Wu, Haixu, et al. "TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis.", 2022, https://arxiv.org/abs/2210.02186.

[4] Wang, Huiqiang, et al. "MICN: Multi-scale local and global context modeling for long-term series forecasting.", 2022, https://openreview.net/pdf?id=zt53IDUR1U.

Thanks for your hard work and valuable comments of this work. Here are responses for your concerns and questions:

Question 1: The channel-independent strategy focuses on the dependencies of the time dimension adopted in this work and is inspired by PatchTST [1]. However, channel dependency is also helpful. In other works, such as Crossformer [2], modeling channels have also been beneficial to some extent in forecasting results. A more robust justification for this choice is needed.

Response 1: Thanks for your valuable comments and suggestions. As is shown in the previous work [4], the channel-independent strategy can better mitigate data drift. While channel dependencies theoretically have the potential to bring about information gain, in practice, data shifts between channels can pose also challenging modeling difficulties.

Questions2: The innovativeness of the patching method in this work needs to be articulated more specifically. What distinguishes this patching approach from other recent patch-based works (such as PatchTST [1] and PETformer [3])?

Response2: Our patching operation is designed to efficiently capture long-term dependencies in time series. We perform patching operations on identical time segments across all periods, while other approaches such as PatchTST and PETformer apply patching to adjacent time stamps. In comparison, our approach can effectively preserve more long-term information in each patch while reducing computational complexity, as shown in Table 2 and Table 6 in the paper.

Question3: What are the benefits of long-term variation and short-term variation extractors?

Response3: Long-term and short-term variation extractors are employed to independently model dependencies at different scales. In this way, it is beneficial for extractors to not only take into account the long-term and short-term characteristics of the input data but also to reduce the computational costs.

Question4: The periodicity obtained is adaptive to the time series of input?

Response4: The obtained periodicity is not adaptive for our method, since the adaptive nature of periodic scales may result in inconsistent 2D shape dimensions in the input. We will explore the adaptive nature of periodic scales in future works.

Question5: It is reasonable to model 2D temporal short-term and long-term variations. Is it possible to model 3D variations in time series?

Response5: Thanks for your suggestions. There may exist several ways of modeling 3D variations. 1) we can consider dependencies between channels, as is explored for our method; 2) we can further decompose temporal variations to model 3D variations. For example, we may use wavelet transformations to capture the dynamic periodicity of the data, which is worth exploring in future works.

[1] Nie, Yuqi, et al. "A time series is worth 64 words: Long-term forecasting with transformers.", 2022, https://arxiv.org/pdf/2211.14730.

[2] Zhang, Yunhao, and Junchi Yan. "Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting.", 2022, https://openreview.net/pdf?id=vSVLM2j9eie.

[3] Lin, Shengsheng, et al. "PETformer: Long-term time series forecasting via placeholder-enhanced transformer.", 2023, https://arxiv.org/pdf/2308.04791.

[4] Lu Han, et al. "The Capacity and Robustness Trade-off: Revisiting the Channel Independent Strategy for Multivariate Time Series Forecasting", 2023, https://arxiv.org/abs/2304.05206

Thanks for your hard work and valuable comments of this work. Here are responses for your concern and questions:

Question1: The motivation. This paper is not well-motivated. Why do we need 2D states to capture periodicity in time series forecasting? Do the authors show the necessity of the 2D modeling for periods?

Response1: Thanks for your comments. The necessity of the 2D modeling lies in the following reasons: 1) 2D tensors have more compact representations than 1D time series, which has been shown in TimesNet; 2) The decouple difficulty of time series lies in how to capture long-term variations. In fact, transforming the complex 1D series into simpler 2D tensors with various periodicity provides a proper solution. We would add more discussions to make our motivation more clear.

Question2: The novelty is limited. The biggest contribution of this paper, the formulation of 1D to 2D transformation seems to be the same as TimesNet, which makes the novelty largely limited.

Response2: Thanks for your comments. In fact, our core contribution lies in the proposed novel Periodicity Decoupling Framework (PDF) to decouple and model temporal variations, rather than the formulation 1D to 2D transformation. We are encouraged by the positive comments about novelty (Reviewer fZ8f) and interesting work (Reviewer gS6P), well-motivated and reasonable design (Reviewers gS6P, KqCU), and sufficient experiments (Reviewers fZ8f, gS6P) of our work.

As pointed out, we borrow the idea of utilizing periodicity in TimesNet to develop our Multi-periodic Decoupling Block to decouple into simpler short- and long-term series. However, there exist several major differences between our method and TimesNet as below:

1. The way of modeling variations is different. TimesNet heavily relies on convolutional neural networks (CNN) to extract temporal variations, which limits the receptive field size of TimesNet due to the use of CNNs. Instead, we introduced novel slicing and patching operations to the 2D tensors, contributing to subsequent CNN and Transformer to extract the decoupled short-term and long-term variations.
2. The frequency selection strategy of capturing periodicity is different. TimesNet strictly chooses top-k amplitude values and obtain the most k significant frequencies. Instead, we select frequencies not only focus on high amplitude but also incorporate those with significant values and amplitude. We assert that frequencies with high amplitude better represent the primary components, while those with larger values facilitate a more discernible distinction between long-term and short-term relationships (More details are in Section 3.2).
3. The time complexity varies much. For longer inputs, TimesNet will significantly increase computational cost, as it requires stacking multiple convolutional kernels to achieve an adaptable receptive field. In contrast, our method employs a computation-friendly patch-based approach for modeling long-term information, where the computational complexity is related only to the period length (see Table 6).

Lastly, we believe that our experimental results can show the effectiveness of our method. Table 1 highlights the success in long-term series forecasting, while Tables 2 and 4 show the efficacy of our decoupling approach and architectural design. Figure 1(b) and Table 3 illustrate the low complexity of our method. We would clarify the differences between our method and TimesNet in the revision.

Question3: The contribution is a little weak. Compared with existing works (such as TimesNet), it only revises several blocks for time series forecasting. The methods don't contribute well to the community.

Response3: Thanks for your comments. In fact, our method is not merely a modification of several blocks in time series forecasting, yet our core contribution lies in the proposed novel Periodicity Decoupling Framework to decouple and model temporal variations. Compared with TimesNet, there exist several differences: 1) The novel slicing and patching operations, and modeling variations are different; 2) The frequency selection strategy of capturing periodicity is different; 3) The time complexity varies much.

Question4: Missing of related work. More periodic modeling works should be discussed. Also, differences compared with TimesNet should be discussed.

Response4: Thank you for pointing it out. The previous two works [2][3] focus on addressing the inherent periodic properties, which may be helpful for capturing periodicity accurately for our method. We will further explore the potential applications of the inherent periodic properties in future works.

Compared with TimesNet, there exist several differences: 1) The novel slicing and patching operations, and modeling variations are different; 2) The frequency selection strategy of capturing periodicity is different; 3) The time complexity varies much. We would add such discussions in the revision.

We would like to thank the reviewer for the helpful discussion during the first round of the review. We hope our response has adequately addressed your concerns. We take this as a great opportunity to improve our work and shall be grateful for any additional feedback you could give to us.