TwinS: Revisiting Non-Stationarity in Multivariate Time Series Forecasting

1. The paper is not well-written, especially because some key concepts are not elaborated, such as periodic missing states, hysteresis, and non-stationary periodic distribution. Besides, some abbreviations are not informal and inappropriately used (dim, Conv, TS, TSformers), and the figures are not portable. Thus, this paper essentially needs further polishing.
2. The method part can be difficult to follow. In the last paragraph of Section 3.1, I still don't know how the input is embedded into the outputted representation because it is almost separated from the previous paragraph only introducing the relationship between transforms and convolutional layers. I hope the authors can elaborate more on it.
3. The proposed framework somewhat lacks novelty. Both wavelet analysis and patching by windows are previously used techniques for analyzing time series. The author does not compare other modular designs of related work like FEDformer. The overall architecture seems a refinement of Crossformer (What is the difference between Channel-Temporal Mixed MLP and FFN in Crossformer?) but endowed with a higher computational cost.
4. Unclear motivations. As non-stationarity is commonly reflected by changing distribution statistics, in particular, its mean, variance, or temporal covariance varies over time. This paper focuses more on the changing periodic distribution issues, which are not well-acknowledged to stem from non-stationarity but the inherent properties of the time series itself. Could the author give more examples of the periodic distribution in highly non-stationary series and relatively stationary series?
5. About the experiment parts. The performance improvement of non-stationary forecasting seems incremental. And the model can also perform worse than previous models on highly non-stationary data (Exchange). Also, the authors offer two variants of the model, but only solely one of the proposed models does not yield consistent promising results, indicating that model selection is inevitable in practical applications. Besides, The analysis of the model is not systematic enough. Some experiments (Tables 2,3,4) only use the results on partial datasets. It seems there is cherry-picking in particular configurations. Could the authors provide more complete results?
6. I find the code implementation is incorporated with re-normalization, Channel-Independence, and Patchfy strategies but are not shown in Fig.2 and the method part. Since each of them can work as a general performance booster, how much do these techniques help the model, and is it dominant?

1. While the motivations and introduction of the proposed method are quite clear, there are still areas that could be refined. For example, which component contributes to handling periodic distributions in non-stationary time series forecasting? I would appreciate more clarifications on the effects of each component of the proposed model.

2. The discovery of periodic distributions, including multiple nested and overlapping periods, in non-stationary time series is not a novel concept. From a theoretical standpoint, Kuznetsov, Vitaly, and Mehryar Mohri [1] have already established generalization bounds by leveraging periodic distributions. On a practical note, Zhang and Zhou [2] employed a two-step approach for prediction, selectively ensembling periodic patterns. These prior studies, along with their variations, should be discussed or even compared within the context of this paper.

Certainly, with the progression of time and technology, there's an intuitive inclination to employ large models to tackle complex problems. However, this doesn't imply that we should resort to a completely black-box or consequential approach. It's crucial to comprehend why the traditional transformer struggles with non-stationary data or what specific limitations arise when handling the non-stationary data outlined in this article. These points should be clearly delineated in Figure 1.

[1] Kuznetsov, Vitaly, and Mehryar Mohri "Learning theory and algorithms for forecasting non-stationary time series." NIPS'2015. [2] Zhang, Shao-Qun, and Zhi-Hua Zhou. "Harmonic recurrent process for time series forecasting." ECAI'2020.

1. The motivation is not clear. It is hard to find the period information in Figure 1. Specifically, there are no nested and overlapping periods as well as hysteresis in the example the authors provided. Moreover, it is suggested that the authors should introduce the meaning of the red frame in Figure 1.
2. In challenge (ii), the authors claim that the traditional attention mechanisms primarily rely on explicitly modeling period information through the values of each time step, but they do not mention any citation and provide any evidence, which is not persuasive.
3. The authors employ the Wave transform, which is one type of spectral transformation in time-series processing. There are several methods that address the time-series forecasting problem with the technique of spectral transformation, for example, fast Fourier Transform. What is the advantage of Wave transform in nonstationary forecasting compared with other spectral transformation methods?
4. As for the discussion of non-stationary time series methods in related works, the authors mentioned that some methods focus on evolving statistical features and overlook periodic distributions. Please justify the difference between these statistical features and periodic distributions. Can the author provide any concrete examples?
5. On page 5, the authors propose the Periods Detection Attention, which is used to model the nonstationary periodic. But this is heuristic. It is suggested that the authors should provide and visualization to prove that this mechanism can identify the periodic information.
6. It is suggested that the authors should consider more compared methods for nonstationary time-series forecasting like[1].

[1] Koopa: Learning Non-stationary Time Series Dynamics with Koopman Predictors

1. Many design details of network structure are not clearly described.
2. Although the motivation of each part of the network is described clear, many designs do not match with the motivations well.

See Questions.