FredNormer: Frequency Domain Normalization for Non-stationary Time Series Forecasting

The importance of the questions posed by the paper is not in doubt, but the execution and resulting conclusions may be less reliable due to experimental concerns. The technical claims of the paper are where the most significant concerns lie. The experimental methodology appears to have several flaws, which may compromise the validity of the results. Without a solid experimental foundation, the technical claims, no matter how innovative, cannot be adequately supported. The paper would benefit from a thorough revision of the experimental design and possibly additional experiments to validate the findings.

The initial discussion in the article pertains to the equivalence of linear combinations before and after applying the Fast Fourier Transform (FFT). However, this proof fails to address the first-order differences introduced in the practical algorithmic procedure. Specifically, Algorithm 1 computes a quantity $S$ based on the batch mean and variance of frequency domain points derived from the input sequence. Conversely, Algorithm 2 utilizes this $S$ to perform a linear transformation on the frequency domain representation of the first-order differences of the sequence. It is evident that the actual algorithmic workflow does not fully align with the theoretical proof provided.

When evaluating the experimental runtime, it is important to note that if the speed is not measured during the stable phase of training, factors such as initial data loading during training runs may obscure the speed reduction effect attributable to the FredNormer plugin.

The potential contribution of the paper to the field is significant if the method can be validated. However, as it stands, the experimental issues limit the trust that can be placed in the results.

My main concern is regarding the contribution of this work. What are the key differences between FredNormer and the two referenced papers [1][2]?

[1] Frequency Adaptive Normalization For Non-stationary Time Series Forecasting, in NeurIPS 2024

[2] Deep Frequency Derivative Learning for Non-stationary Time Series Forecasting, in IJCAI 2024

1. FAN is also a method based on adaptive normalization in the frequency domain, capable of handling both trend and seasonal non-stationary patterns in time series data. Compared with it, although this paper provides a theoretical guarantee, its contribution appears insufficient.

[1] Ye W, Deng S, Zou Q, et al. Frequency Adaptive Normalization For Non-stationary Time Series Forecasting[J]. Advances in Neural Information Processing Systems, 2024.

2. In Non-stationary Transformer, the authors argue that removing inherent non-stationarity from time series may reduce the model's ability to forecast real-world bursty events. Could suppressing non-stationary information in FredNormer similarly lead to over-stationarization, limiting its practical applicability?

[1] Liu Y, Wu H, Wang J, et al. Non-stationary transformers: Exploring the stationarity in time series forecasting[J]. Advances in Neural Information Processing Systems, 2022, 35: 9881-9893.

3. In Definition 2, you mention "M components with higher stability S(k)." How do you define "higher" stability? What is the threshold value used, and how is it selected?

4. The authors only compare against with Transformer-based and MLP-based methods. Why not include comparisons with more CNN-based baselines, such as TSLANet, ModernTCN, and TimesNet, to demonstrate the robustness and generalization of FredNormer?

[1] Eldele E, Ragab M, Chen Z, et al. TSLANet: Rethinking Transformers for Time Series Representation Learning[C]//Forty-first International Conference on Machine Learning, 2024.

[2] Luo D, Wang X. Moderntcn: A modern pure convolution structure for general time series analysis[C]//The Twelfth International Conference on Learning Representations. 2024.

[3] Wu H, Hu T, Liu Y, et al. TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis[C]//The Eleventh International Conference on Learning Representations, 2023.

5. I am curious about FredNormer’s impact on frequency-domain methods like FiTS and FreTS. Have you tested its performance on these models?

Minor Error:

1. Section 4.2 mentions running time, but Figure 4 seems to be missing.
2. In Section 4.2, you refer to a "fourth-shaped frequency component." Could you clarify what you mean by this?
3. In the notations section, the indicator function I{k=0} is said to equal 1 if k is not equal to 0. In Proof 1, you state that for k≠1,I{k=0}=0. Could you clarify which case is correct?

• The contribution of the paper seems limited. According to equation (10), the core approach mainly involves applying a linear transformation to frequency components based on the matrix $\mathrm{S}$. It is unclear how $\mathrm{S}$ functions as a constraint to realize the key motivation of enhancing the weight of stable frequency components.

• In Definition 2, the authors introduce the concept of a Stable Frequency Subset $\mathcal{O}$ but do not provide a clear criterion for determining what qualifies as a stable frequency. Some notations, such as Stable Frequency Subset $\mathcal{O}$, and Theorem 1 presented in Section 2 are not well linked to the design of the method proposed in Section 3.

• The experimental evaluation lacks thoroughness, as it includes only three baseline models (Dlinear, PatchTST, and iTransformer). Although the authors mention other models, such as CrossFormer, they do not include it in their comparisons. Additionally, the Nonstationary Transformer [1] would also be a relevant model for further comparison.

• The empirical analysis in Figure 3 is confusing to the reviewer. Could the authors provide additional clarification on how the adjustments to the amplitudes of the input series and forecasting target, based on the Frequency Stability score, affect model prediction accuracy? Additionally, could you explain why these adjustments are effective in enhancing the model's performance? Both Equations (9) and (10) have large spacing from the preceding text.

[1] Liu, Yong, et al. "Non-stationary transformers: Exploring the stationarity in time series forecasting." Advances in Neural Information Processing Systems 35 (2022): 9881-9893.

1. this paper is not very well-motivated. The paper mentioned that "Modeling solely in the time domain struggles to distinguish between different frequency components within superimposed time series" as their first motivation. Does it have any difference with the distribution shift or non-stationary forecasting? Also, it mentioned that "the z-score normalization applies uniform scaling across all frequency components, which leaves frequency-specific patterns unaltered". I believe only RevIN uses "learnable" z-score normalization but other methods like SIN, SAN, FAN do not use z-score normalization. What they do cannot be equal to z-score normalization. So how can you use z-score to summarize these works?

2. this paper is kind of confusing for the theoretical analysis. Based on the Theorem 1, the authors mentioned that "the normalization operation keeps the proportion unchanged". The proportion is unchanged does not mean it cannot normalize the time series. So the theoretical analysis cannot show the existing normalization would fail. Moreover, how can you say your defined stability is correlated with the non-sationarity? Why the frequency components are entangled and thus influence normalization? These parts don't have theoretical analysis I believe.

3. the experiments are somehow lack of comparisons with state-of-the-art normalization techniques, such as SIN [1].

[1] SIN: Selective and Interpretable Normalization for Long-Term Time Series Forecasting. In ICML.