Revisiting Long-term Time Series Forecasting: An Investigation on Affine Mapping

Thank you for your thoughtful review and for highlighting the strengths of our work. Your critiques are greatly appreciated and we believe they will significantly improve the quality of our paper. We would like to address your concerns as follows:

C1: The limited novelty of the study and theoretical results

We understand your perspective on the novelty of the study and theoretical results because they seems trivial and hardly novel. However, it is important to clarify that our primary objective in this study was not to introduce new algorithms, but rather to illuminate the intrinsic efficacy of affine mapping in LTSF tasks. We believe that our theoretical analysis and empirical studies do represent a substantial technical contribution. We not only elucidate why linear models perform well in LTSF tasks, but also remind future researchers to conduct more ablation experiments as we did in Section 2 to confirm the effectiveness of their methods. Additionally, we highlight the impact of the input horizon length and tricks like RevIN. This findings could potentially guide future research and enhance the development of more effective models.

C2: The rigour of the comparative study in Table 2 for the idea in Figure 8.

We acknowledge that a more rigorous ablation study would strengthen our paper. However, the primary aim of Table 2 is to show that simple baselines, such as RLinear and RMLP, can yield competitive performance compared to recent complicated methods on LTSF tasks, implying potential questions regarding the effectiveness of these methods and the suitability of the datasets used. On the other hand, Figure 8 illustrates the limitations of linear models in fitting trend terms, both with and without the help of disentanglement and normalization. The purposes served by the table and the figure are distinct and complementary, with each contributing unique insights to our analysis.

C3: Clearer explanation and clarification of some points

Thank you for your thoughtful suggestions to our presentation. We will revise Figure 1 and provide more detailed explanations of used baselines in Appendix for the readers. As for Footnote 4, we can write the full update rule of $W,b$ as $W=W-\eta\frac{\partial L}{\partial W}=W-\eta X^\top\frac{\partial L}{\partial Y}$ and $b=b-\eta\frac{\partial L}{\partial b}=b-\eta\frac{\partial L}{\partial Y}$ where $L$ denotes the loss function. Apparently the update of the weight and bias is coupled due to $\frac{\partial L}{\partial Y}$. We will provide additional explanations to clarify its importance in the context of our discussion. As for citing the DLinear paper formally, we thank you for pointing out the omission. We will update the citation with the accepted venue for the DLinear paper. As for grammar issues in the conclusion section, we will revise the conclusion section to correct these issues. As for discussing the TiDE paper, we are aware of the TiDE paper and agree that it tackles similar problems to those addressed in our paper. We will include a discussion on this paper in our revised manuscript, acknowledging its relevance to our work.

We hope that these responses address your concerns and that our revisions will make the paper clearer and more valuable. Thank you again for your thoughtful feedback.

Thank you for your valuable feedback and insights. We appreciate your recognition of the importance of our study and its contribution to the understanding of the fundamental mechanisms affecting the performance of recent LTSF models. We would like to address your concerns as follows.

C1: Limited technical contribution

We appreciate your feedback regarding the perceived absence of innovative solutions or algorithms in our paper. However, it is important to clarify that our primary objective in this study was not to introduce new algorithms, but rather to illuminate the intrinsic efficacy of affine mapping in LTSF tasks. We believe that our theoretical analysis and empirical studies do represent a substantial technical contribution. We not only elucidate why linear models perform well in LTSF tasks, but also remind future researchers to conduct more ablation experiments as we did in Section 2 to confirm the effectiveness of their methods. Additionally, we highlight the impact of the input horizon length and tricks like RevIN. This findings could potentially guide future research and enhance the development of more effective models. We are looking forward to see new algorithms that leverage these insights, such as how to cope with trend terms without RevIN, and how to process multi-channel datasets as mentioned in Section 4.

C2: Need for more real-world datasets

We agree with your suggestion that testing on more diverse real-world datasets could further strengthen our results. However, it is crucial to note that the datasets we currently utilize for evaluation are also widely used in other LTSF benchmarks such as Informer, FEDformer, DLinear, and PatchTST. Given the setting of LTSF tasks which covers longer input horizon and prediction length, these tasks inherently contain increased seasonality. Thus, we are prompted to question the rationale behind LTSF tasks, as they tend to heavily rely on inherent seasonality within datasets. This raises potential concerns about the datasets currently in widespread use for evaluation purposes. It is possible that these datasets, with their inherent biases, may not provide a comprehensive or fair assessment of different forecasting methodologies. As we stated in Section 3.1, this may explain why a single linear layer performs well in many real-world datasets, since they often exhibit seasonality in days, weeks, months, and years. We also list one "abnormal" dataset in C3 to further support our viewpoints.

C3: What about the efficacy of affine mapping when the models deal with noisy and low-quality time series data?

Indeed, our current datasets are noisy and of low quality. However, in LTSF tasks, the introduction of a longer observation window allows for the incorporation of more seasonal features, which in turn helps mitigate the impact of these noises. Here we introduce one "abnormal" dataset, Exchange, which has been evaluated in DLinear and PatchTST. This dataset comprises changes in exchange rates across eight countries over several years and exhibits minimal periodic characteristics. The table below presents the forecasting results from both DLinear and PatchTST.

Although the MSE and MAE evaluation metrics appear low, the actual visualized prediction results are disappointing. DLinear discovered that simply repeating the final value of historical time series outperforms all the baselines, while PatchTST deemed 'Exchange' as "chaotic" and consequently excluded it from their evaluation. We believe that this instance further supports the viewpoints we expressed in C2.

---

Once again, we appreciate your thoughtful comments and questions. They have provided us with valuable directions for improving our work and for future research.

We appreciate the time of the reviewer and thank the reviewer for acknowleding that our paper makes solid observations and gives a detailed analysis. We provide the detailed response to each question in the following.

Q1: However, there are no information about what affine mapping is or how affine mapping is effectively introduced in the LTSF models to improve performance.

The affine function is clearly defined in this paper, plese see the footnote of Page 1 (i.e., "In general, a linear layer is defined as an affine transformation $\mathbf{Y}=\mathbf{X}\mathbf{W}^\top+\mathbf{b}$") and Equation (1) of Section 3.1 (with a title ROLES OF AFFINE MAPPING IN FORECASTING).

W1: The findings of the paper in the abstract are not evidently proved in the experiments.

We are unclear to the logic why the reviewer draws this conclusion since the reviewer acknowledges that "this paper gives a detailed analysis of various use-cases of LTSF on public datasets and makes solid observations."

To make it clear, we relist our findings stated in the abstract here again: 1) affine mapping in some LTSF models dominates forecasting performance across commonly utilized benchmarks; 2) affine mapping can effectively capture periodic patterns but encounter challenges when predicting non-periodic signals or time series with different periods across channels; and 3) using reversible normalization and increasing input horizon can significantly enhance the robustness of models. It is not hard to find through this paper that Finding 1) is supported by Section 2, for example in Figures 2, 3 and Table 1; Finding 2) is supported by Section 3.1 and Section 4, in which we have provided extensive empirical and theoretical study of linear models on LTSF tasks; and Finding 3) is supported by Section 3.2 and Section 4, as shown in Table 3 and Figures 7, 8, 9, 12, 13.

Moreover our findings are supported by all the other three reviewers (SH4h, TkUX, NWqo). The evidences are as follows:

• Reviewer SH4h "This work provides theoretical and empirical evidence to support its findings."
• Reviewer TkUX: "This paper offers a timely and rigorous critique of the recent trend of using transformer-based models for long-term time series forecasting, and arrives at several surprising and important empirical conclusions."
• Reviewer NWqo: "This paper provides various types of visualization to support their claims and to present the results of experiments."

Therefore, we sincrely urge the reviewer to carefully read our paper and other reviews.

Q2: As an investigation paper, I would recommend to consider the pros and cons of the current research to effectively conclude the findings and future work that is required on the topic. For example, in this paper, X. M. Chen, Y. Li, R. Z. Wang; Performance study of affine transformation and the advanced clear-sky model to improve intra-day solar forecasts. J. Renewable Sustainable Energy 1 July 2020; 12 (4): 043703., the results about affine transformation is similar to the findings of our paper. This paper should consider investigating the use-case used in this paper and fetch out the results and challenges faced in this use-case.

We resectively disagree with the reviewer on "This paper should consider investigating the use-case used in this paper and fetch out the results and challenges faced in this use-case." The reasons are as follows.

The goal of this paper, as stated in the abstract and the introduction, is to provide an analysis on "previous studies have demonstrated that a single linear layer can achieve competitive forecasting performance compared to other complex architectures." As our research objects are other complicated architectures which are published in recent years, e.g., SCINet (NIPS 2022), TimesNet (ICLR 2023), DLinear (AAAI 2023), PatchTST (ICLR 2023), etc.

The paper by Chen et al. (2020) published in the Renewable Sustainable Energy journal does not provide an opportunity to investigate the pros and cons of recent Long-Term Traffic Speed Forecasting (LTSF) benchmarks. As a result, it is not relevant to our work. It is important to note that not investigating this paper does not impact the contribution of our paper to the LTSF community.

Moreover, in our paper, the affine functions are implemented using a linear layer as $\mathbf{Y}=\mathbf{X}\mathbf{W}^\top+\mathbf{b}$, as mentioned in the previous response. On the other hand, in the paper by Chen et al. (2020), the affine function is defined as $G_{\text{past}}=aG_{\text{est,past}}+b$, where $a$ and $b$ are scalar factors to control the bias. This definition is more similar to quantile mapping, as also mentioned in Chen et al. (2020). Besides, Chen et al. (2020) only provide experimental study on improving intra-day solar forecasts, while our study covers more fields and provides additional empirical and theoretical evidence.

Additionally, our paper has received positive feedback from Reviewer TkUX, who states, "This paper offers a timely and rigorous critique of the recent trend of using transformer-based models for long-term time series forecasting. " Reviewer SH4h also praises our paper that "The authors study the fundamental mechanisms that affect the performance of recent LTSF models, which is important for the community.

Therefore, we are concerned that there may be a misunderstanding between our study and yours. We hope that these responses address your concerns. Thank you again for your feedback.

Thank you for taking the time to review our paper and for your valuable comments. We would like to address your concerns as follows.

Q1: Are simple models are always better than complex models for LTSF, or is it only relative to the specific framework shown in Figure 1?

Thank you for your insightful question. We'd like to clarify that our primary claim is not that simple models are always better than complex models in forecasting tasks. Rather, we argue that in the context of LTSF benchmarks, which are widely used in current LTSF research, the simple models we've discussed typically demonstrate robust performance. It is worth noting that Transformer-based methods such as Informer, Autoformer, and FEDformer were once predominant in LTSF tasks. However, DLinear (Zeng et. al, 2022) showed that a simple linear layer can outperform these more complex approaches, which indicates the significance of directly modeling the relationship between input and output time series. Consequently, a trend has emerged to move away from the encoder-decoder structure towards designing modules that capture temporal and channel dependencies. These methods generally use a single projection layer to generate the final forecasting results, largely aligning with the framework we simplified in Figure 1. This is why we've chosen to analyze the efficacy of each component in these methods. We discovered the importance of RevIN and projection layer, which were not discussed in detail in the original paper.

Q2: How would the results change with a different, yet complex framework such as 1-dimensional CNNs?

We have provided one typical CNN-based model, SCINet, specifically designed for LTSF tasks. SCINet employs a tree-like structure of 1D-CNNs to handle interleaved subsequences, downsampled from the original time series. The experimental results obtained from SCINet align well with those from other models. Notably, due to the equivalence of a single 1x1 convolutional layer and a linear layer, substituting the linear layer with a 1x1 convolutional layer will obtain the same results.

Q3: How would the results change if traditional models such as AR or ARIMA were included?

Thank you for your suggestion to incorporate traditional models such as AR and ARIMA. However, as extensively discussed in previous works like DLinear (Zeng et al., 2022) and Informer (Zhou et al., 2020), these models tend to underperform in LTSF tasks due to the accumulation of errors caused by regressive generation. This is the primary reason we chose not to include these traditional models in our study. We only include recent methods that utilize non-autoregressive generation, as illustrated in Figure 1.

Q4: There is not enough description about the models and experimental setup, such as the definition of RevIN.

We appreciate your insightful comments and recognize the need for enhanced clarity in our methodology, particularly for readers who may not be as familiar with the topic. To address this, we will include comprehensive descriptions of the utilized baselines and modules such as RevIN in Appendix A. We apologize for any initial lack of detail, and we are grateful for your constructive feedback.

Q5: What is the main improvement of this work over DLinear (Zeng et al., 2022)?

In the original paper of DLinear, the authors questioned whether transformers effective for time series forecasting, and then proposed a family of linear models which outperform Transformer-based methods across multiple LTSF datasets. However, their demonstration of the robust performance of linear models was largely experimental, lacking in-depth theoretical substantiation for why linear models excel in LTSF tasks. Our research builds upon the foundation laid by DLinear, offering comprehensive empirical and theoretical results to elucidate the efficacy of linear models in LTSF tasks. Our work also highlights their limitations and propose potential remedies.

Besides, we have to emphasize that one crucial factor contributing to DLinear's success is the length of the input horizon. DLinear adopts a longer input horizon (336) compared to previous methods (96), without providing a comprehensive explanation — their reasoning was limited to 'underfitting' or 'overfitting' problems when varying the input horizon length. Subsequent works, such as PatchTST, have also adopted longer input horizons (512) to enhance their forecasting performance. Conversely, some approaches like TimesNet opt for shorter input horizons to undermine DLinear's performance in comparative studies. Our research underscores the importance of the input horizon length from both empirical and theoretical viewpoints, paving the way for more fair performance comparisons in the future.