We sincerely thank Reviewer Cccn for providing thorough and insightful comments. Here are our responses to your concerns and questions.

Weaknesses

Thank you for your insightful comments and for raising concerns about the clarity of our paper's core contributions.

We understand your concern that the work may appear to be a collection of components without a strong underlying problem statement.

We appreciate the opportunity to clarify our motivations and the significance of our approach, particularly focusing on the Recursive Residual Decomposition (RRD) concept.

Motivation and Solution

The motivation behind our work stems from the observation that existing time series forecasting models often struggle to disentangle the complex interplay between linear and nonlinear patterns within data.

This limitation hinders their ability to generalize and perform accurately across diverse real-world scenarios.

Our goal is to address this challenge by introducing a more nuanced decomposition strategy that can recursively separate and model these patterns more effectively.

Our RRD framework is designed to iteratively refine the decomposition process, allowing for a deeper extraction of both linear and nonlinear patterns.

This recursive approach is distinct from traditional methods that perform a single-level decomposition, and we believe it is a significant advancement in time series analysis.

Ablation Design

In fact, beyond conducting ablations on individual components of the proposed LiNo, our paper mainly focus on the capacity of our general LiNo framework.

Specifically, through Table 6(a) ("Ablation study of different No block choices"), we demonstrate that the proposed No block exhibits superior nonlinear pattern extraction capabilities compared to current state-of-the-art nonlinear models, such as iTransformer and TSMixer.

Additionally, Table 6(b) and Figure 3 provide evidence that our LiNo framework helps the baseline model to deliver more accurate and robust predictions.

That said, your suggestion is indeed insightful, and we have incorporated it into our work.

In Appendix F, we “compare LiNo with similar residual-based designs or variations to evaluate the overarching idea.”

Experimental results show that LiNo significantly outperforms N-BEATS and its successor, N-HiTS, across the ETTm2, ECL, and Weather datasets.

This highlights that, compared to leveraging Recursive Residual Decomposition (RRD) to refine final predictions based on residual errors from previous predictions (as in N-BEATS), employing RRD to capture more nuanced linear and nonlinear patterns, as proposed in this work, is a more effective approach.

Questions

Model design

The design of the LiNo framework aims to offer a deeper and more nuanced decomposition of time series signals to effectively handle real-world linear and nonlinear complexities by advancing the Recursive Residual Decomposition (RRD).

Specifically, the Li block extends classical techniques like moving average kernel or learnable convolution kernel, encompassing broader linear modes.

The No block offers better nonlinear pattern extraction by capturing diverse nonlinear features, including temporal variations, frequency information, and inter-series dependencies.

Take home message

Our recursive design ensures that each pattern—linear or nonlinear—is extracted iteratively and independently, minimizing interference and fully leveraging residual information.

This systematic separation not only enhances robustness but also boosts interpretability by isolating meaningful patterns at different scales.

Such a framework provides actionable insights into the intrinsic structure of time series, empowering precise and resilient forecasting.

Dear Reviewer Cccn,

We designed the following experiments to further validate the specific effectiveness of Recursive Residual Decomposition (RRD) in LiNo, which we hope will help provide a better understanding of our innovations and contributions.

Here are some essential definitions:

1. The linear feature extractor is denoted as $Li\\_Block$, the extracted pattern $L_i$, and linear prediction $P^{li}_i$.

2. The nonlinear feature extractor is denoted as $No\\_Block$, the extracted pattern $N_i$, and nonlinear prediction $P^{no}_i$.

3. The projection to get the prediction is denoted as $FC$

4. We assume the input to the current layer in deep neural network as $H_i$ and the next layer $H_{i+1}$, where $H_0= X\\_embed=XW+b$, $X$ is the input raw time series

5. Prediction of current layer $P_{i}$

6. Final prediction $\hat{Y}$

The learnable autoregressive model (AR) mentioned in the paper is employed as the linear feature extractor ($Li\\_Block$).

The Temporal + Frequency projection, combined with the Tanh activation function, is adopted as the nonlinear feature extractor ($No\\_Block$).

Following four schemes are designed :

'RAW': The input features pass through the Li Block and No Block sequentially, but no feature decomposition is performed. The model's final output features are directly used for prediction (Traditional Design).

• $L_i = Li\\_Block(H_i)$
• $N_i = No\\_Block(L_i)$
• $H_{i+1} = N_i$
• $\hat{Y}=FC(H_{N})$

'LN': The input features pass through the Li Block and No Block sequentially, with each block generating its own prediction based on the features extracted, but no residual decomposition is applied (Common Linear-Nonlinear Decomposition Design).

• $L_i = Li\\_Block(H_i)$
• $P^{li}_{i}=FC(L_i)$
• $N_i = No\\_Block(L_i)$
• $P^{no}_{i}=FC(N_i)$
• $H_{i+1} = N_i$
• $P^{i}=P^{li}_i+P^{no}_i$
• $\hat{Y}=\sum_{i=1}^{N} P^{i}$

'RRD': The input features pass through the Li Block and No Block sequentially, and the output from the No Block is used for residual decomposition and prediction. This does not perform explicit residual decomposition of the linear and nonlinear features separately, but instead uses the overall features for decomposition (Like did in N-BEATS).

• $L_i = Li\\_Block(H_i)$
• $N_i = No\\_Block(L_i)$
• $P_{i}=FC(N_i)$
• $H_{i+1} = H_i-N_i$
• $\hat{Y}=\sum_{i=1}^{N} P^{i}$

'LiNo': The input features pass through the Li Block, followed by prediction and residual decomposition. Then, the features go through the No Block for prediction and residual decomposition again (Ours).

• $L_i = Li\\_Block(H_i)$
• $P^{li}_{i}=FC(L_i)$
• $R^{L}_{i} = H_i - L_i$
• $N_i = No\\_Block(R^{L}_i)$
• $P^{no}_{i}=FC(N_i)$
• $R^{N}_{i} = R^{L}_i - N_i$
• $H_{i+1} =R^{N}_{i}$
• $P^{i}=P^{li}_i+P^{no}_i$
• $\hat{Y}=\sum_{i=1}^{N} P^{i}$

Below are the experimental results:

It is clear that, although both RRD and LN show slight improvements over RAW, our LiNo design significantly outperforms the other designs. This demonstrates:

1. The effectiveness of explicit linear and nonlinear modeling.
2. The effectiveness of RRD for representation decomposition.
3. The potential for further improvement by combining both approaches (LiNo).

We hope that these experimental results and analyses will help you gain a broad view of the contributions of this work.

BTW

Dear Reviewer Cccn,

We greatly appreciate your thoughtful reflections on the scientific and philosophical aspects of our work.

We understand that the combination of recursive, residual, frequency, and linear elements may seem pre-determined, but we would like to clarify that:

Our main contribution has never been the combination of complex models for extracting temporal, frequency, and inter-series dependencies.

Instead, it's the advanced Recursive Residual Decomposition (RRD) framework, a generic design, which enables a deeper and more nuanced extraction of both linear and nonlinear patterns.

As discussed and demonstrated in the paper, the Li Block (for extracting linear features) can be implemented using any linear model, such as moving average filters, exponential smoothing functions, or learnable 1D convolution kernels. Similarly, the No Block (for extracting nonlinear features) can adopt any nonlinear model, such as iTransformer or TSMixer, as shown in Table 6 (a).

This method is designed to capture complex, multi-level dynamics that are often overlooked in traditional approaches. It further allows us to obtain more robust and interpretable time series forecasting results, as shown in Table 6 (b) and Fig.3.

We agree that this work may not resonate with all perspectives. However, this advanced extraction of both linear and nonlinear features, lays the foundation for future exploration in the area of robust and interpretable forecasting models. This contribution, we hope, will encourage further developments in both theory and application, helping the community advance towards more effective, robust, and interpretable time series modeling techniques.

Finally, please allow us to quote your words: "Ultimately, few works stand the test of time for their merit."

This is absolutely true, especially in the field of deep learning.

Interestingly, in the community of Time Series Forecasting (TSF), the standout works that are ultimately remembered tend to because of their overall ideas, instead of their specific structures.

For instance, Autoformer, while introducing an Auto-Correlation mechanism to replace traditional self-attention, became prominent primarily due to its Seasonal-Trend Decomposition concept.

PatchTST gained recognition for its key ideas of Channel-Independent and Patching.

iTransformer is well-known for its Whole Series Embedding and Channel-Wise Attention mechanism.

So we believe the RRD presented in this paper can provide meaningful sight to the field.

Again, we are truly grateful for the reviewer’s recognition of our efforts and dedication!

Please reconsider our contributions and insights from our work. If you have any further questions, we are looking forward to discussing with you!

Dear Reviewer Cccn,

Thank you once again for your thoughtful feedback and for acknowledging our efforts.

We understand that the philosophical perspective of our work may not fully align with your preferences. However, I believe we both agree that diverse perspectives are crucial for fostering new ideas and advancing the field.

We hope the additional analyses and experiments we’ve conducted, particularly the validation of our Recursive Residual Decomposition (RRD) approach, clarify the significant impact of our framework. With over 200 new results and 7 new pages of content, we have strived to present a more robust and comprehensive argument for the effectiveness of our approach. We believe that these updates strengthen the work and demonstrate its potential to contribute meaningfully to the community.

Given that our current score of 5.5 places us at significant risk of rejection, we would greatly appreciate it if you could consider your assessment in light of these updates and insights. Your evaluation is crucial to our chances of acceptance, and we would deeply appreciate your reconsideration of the score.

We also wanted to highlight that, since the extended discussion period began, our score has remained unchanged. During these 6 additional days, many authors have engaged in fruitful discussions with their reviewers, receiving positive feedback and score increases. We sincerely hope that our case is no exception, and that the added clarifications and updates will also lead to a more comprehensive reassessment.

Thank you again for your time and thoughtful consideration.

Best regards,

Authors

We sincerely thank Reviewer dLTN for acknowledging our strength and providing thorough and insightful comments. Here are our responses to your concerns and questions.

Weaknesses

W.1 The components of the method, while effective, are not entirely novel....

The similarities of LiNo and N-BEATS has been discussed in the updated PDF, Sec.2, where we denoted the model designed in N-BEATS's fashion as Mu.

When check experimental results in Table.5(b) and Fig.4, counterintuitively, the usage of RRD in N-BEATS (Mu) does not significantly outperform classic designs in terms of either forecasting accuracy or robustness to noise.

In contrast, our LiNo framework excels in both aspects, which demonstrates that LiNo advanced RRD for more accurate and robust forecasting.

Actually, utility of frequency-domain MLPs in this work is more related to FITS, instead of that in FreMLP. But we are more than happy to include and discuss the similarity.

When we start this work, we do not notice that there is a work that so close to our channel mixing technique, thanks for mention it!!

I think our channel mixing technique and SOFTS both stems from the idea of learning channel-dependence, while maintain channel-independence information.

Here are some major differnce:

1. LiNo using weighted sum of all channels with weight generated by softmax function, while SOFTS used a stochastic pooling technique, which is more time consuming.
2. We directly perform softmax to input feature, while SOFTS first go through a Feedforward Network.

W.2 The authors argue that the primary distribution involves linear and nonlinear decomposition, with ablation studies primarily focused on multivariate time series forecasting...

Thank you for your suggestion!

We have redesigned an ablation experiment without the interference of inter-series dependencies.

Specifically, we designed a simplified No Block with the following variants: using only Temporal projection, only Frequency projection, Temporal + Frequency projection (TF), and introducing nonlinearity (Tanh activation function) on top of TF.

This simplified ablation study effectively demonstrates that extracting both temporal variations and frequency information, and the introducing of nonlinearity, is crucial for accurate predictions.

Regarding the ablation of the No Block which proved its superiority, please forgive us for having a different perspective.

Our argument lies in the fact that iTransformer and TSMixer primarily attribute their performance to the channel mixing part, which makes them less capable of extracting more complex nonlinear patterns in real-world time series.

In contrast, our No block is designed to extract nonlinear features other than inter-series dependencies, such as nonlinear temporal variations and frequency information, which leads to better performance.

W.3 The paper lacks an error bar analysis.

We conducted an error bar analysis (using five different random seeds), but to save space in the main text, we included it in Appendix C.1.

W.4 The paper contains a few minor typo errors...

Deep thanks for all the invaluable suggestions! We've revised the paper according to your invaluable comments.

Thank you for your response.

W1. Thank you for your clarification. I appreciate that the revised paper includes comparisons with N-BEATS and FITS, which are helpful and well-placed. However, while the rebuttal acknowledges the similarity between your method and SOFTS, this comparison has not been explicitly discussed or incorporated into the paper itself.

W2. Thank you for your effort and for redesigning the ablation experiments to provide additional insights. However, my concern remains that it is still unclear whether the performance gains primarily stem from the separation of channel-independent and channel-dependent components, or from the extraction of nonlinear features (beyond channel dependence). My question is less about the individual contributions of each part of the No-block and more about pinpointing the specific factor that makes your decomposition method genuinely effective.

W3. Thank you, this has been resolved.

W4. Thank you, this has been resolved.

Dear Reviewer dLTN,

We are glad that our response has helped clarify some of your concerns. Regarding your confusion about W.1&2, we will provide further clarification here.

W.1 While the rebuttal acknowledges the similarity between your method and SOFTS, this comparison has not been explicitly discussed or incorporated into the paper itself.

Due to time constraints, we had originally planned to include this discussion in the final camera-ready version, which is why we were unable to update our PDF in time.

Now, please refer to the latest version of the PDF, where we have presented this content. Kindly check Sec. 3.2, Lines 254-255, as well as Appendix G.

W.2 My question is more about pinpointing the specific factor that makes your decomposition method genuinely effective.

Thank you again for acknowledging our efforts and for articulating your questions and concerns so precisely and patiently.

We will further address your remain concerns, specifically regarding the overall effectiveness of the decomposition proposed in LiNo, beyond the validity of the individual components.

We designed the following experiments to investigate the specific effectiveness of Recursive Residual Decomposition (RRD) in LiNo under a Univariate Scenario, in order to exclude any interference from channel-independent or channel-dependent information.

Here are some essential definitions:

1. The linear feature extractor is denoted as $Li\\_Block$, the extracted pattern $L_i$, and linear prediction $P^{li}_i$.

2. The nonlinear feature extractor is denoted as $No\\_Block$, the extracted pattern $N_i$, and nonlinear prediction $P^{no}_i$.

3. The projection to get the prediction is denoted as $FC$

4. We assume the input to the current layer in deep neural network as $H_i$ and the next layer $H_{i+1}$, where $H_0= X\\_embed=XW+b$, $X$ is the input raw time series

5. Prediction of current layer $P_{i}$

6. Final prediction $\hat{Y}$

The learnable autoregressive model (AR) mentioned in the paper is employed as the linear feature extractor ($Li\\_Block$).

The Temporal + Frequency projection, combined with the Tanh activation function, is adopted as the nonlinear feature extractor ($No\\_Block$).

Following four schemes are designed :

'RAW': The input features pass through the Li Block and No Block sequentially, but no feature decomposition is performed. The model's final output features are directly used for prediction (Traditional Design).

• $L_i = Li\\_Block(H_i)$
• $N_i = No\\_Block(L_i)$
• $H_{i+1} = N_i$
• $\hat{Y}=FC(H_{N})$

'LN': The input features pass through the Li Block and No Block sequentially, with each block generating its own prediction based on the features extracted, but no residual decomposition is applied (Common Linear-Nonlinear Decomposition Design).

• $L_i = Li\\_Block(H_i)$
• $P^{li}_{i}=FC(L_i)$
• $N_i = No\\_Block(L_i)$
• $P^{no}_{i}=FC(N_i)$
• $H_{i+1} = N_i$
• $P^{i}=P^{li}_i+P^{no}_i$
• $\hat{Y}=\sum_{i=1}^{N} P^{i}$

'RRD': The input features pass through the Li Block and No Block sequentially, and the output from the No Block is used for residual decomposition and prediction. This does not perform explicit residual decomposition of the linear and nonlinear features separately, but instead uses the overall features for decomposition (Like did in N-BEATS).

• $L_i = Li\\_Block(H_i)$
• $N_i = No\\_Block(L_i)$
• $P_{i}=FC(N_i)$
• $H_{i+1} = H_i-N_i$
• $\hat{Y}=\sum_{i=1}^{N} P^{i}$

'LiNo': The input features pass through the Li Block, followed by prediction and residual decomposition. Then, the features go through the No Block for prediction and residual decomposition again (Ours).

• $L_i = Li\\_Block(H_i)$
• $P^{li}_{i}=FC(L_i)$
• $R^{L}_{i} = H_i - L_i$
• $N_i = No\\_Block(R^{L}_i)$
• $P^{no}_{i}=FC(N_i)$
• $R^{N}_{i} = R^{L}_i - N_i$
• $H_{i+1} =R^{N}_{i}$
• $P^{i}=P^{li}_i+P^{no}_i$
• $\hat{Y}=\sum_{i=1}^{N} P^{i}$

Here are the experimental results:

It is clear that, although both RRD and LN show slight improvements over RAW, our LiNo design significantly outperforms the other designs. This demonstrates:

1. The effectiveness of explicit linear and nonlinear modeling.
2. The effectiveness of RRD for representation decomposition.
3. The potential for further improvement by combining both approaches (LiNo).

This result and analysis will be included in the camera-ready version, too.

BTW

Dear reviewer dLTN,

With less than 3 hours remaining before the discussion period concludes, we are eager to receive your feedback. Please allow us to briefly summarize the key points of our rebuttal phase for your reference.

In the rebuttal, we emphasized the uniqueness of our Recursive Residual Decomposition (RRD) framework, which iteratively and independently extracts both linear and nonlinear patterns, addressing gaps in prior methods like N-BEATS.

We clarified that LiNo’s core innovation lies in its holistic decomposition approach, not just component designs.

Experiments, including ablation studies, error bar analysis, and visualizations, demonstrated LiNo’s superior forecasting performance, robustness, and interpretability.

Additionally, we incorporated new comparisons, such as with SOFTS, and improved the paper with clearer diagrams, fixed typos, and extended analyses (e.g., Tables 5, 6(b)).

Our efforts in the rebuttal period include more than 200 new results, 7 new pages. These updates solidify our contributions, and we hope them have addressed all your concerns comprehensively.

If our rebuttal has resolved your concerns or improved your understanding of the paper, we would greatly appreciate it if you could reconsider your assessment and update the score accordingly.

Best regards,

Authors

Dear reviewer dLTN,

We sincerely thank you for your insightful and detailed review. Your valuable suggestions have been instrumental in improving our paper.

With only a few hours remaining before the deadline for the reviewer-author discussion phase, we kindly ask if our responses have adequately addressed your concerns.

We look forward to your feedback and would happily answer any further questions. We also hope you will reconsider your original rating.

Best regards,

Authors

Dear Reviewer dLTN,

We would like to express our heartfelt gratitude for the time and thoughtful effort you dedicated to reviewing our paper. We are also thankful for your recognition of its strengths.

As the author/reviewer discussion phase has now concluded, we believe our rebuttal has fully addressed all of your comments and concerns. However, given the scoring standards of previous ICLR reviews, we notice that our current score of 5.5 is at the borderline level. We would greatly appreciate it if you could reconsider and potentially raise your score, allowing us the opportunity to present our work at the conference.

Throughout the rebuttal period, we have made significant efforts to strengthen our paper, including the addition of over 200 new experiments and 7 new pages of content. These updates provide a more comprehensive and robust presentation of our approach and its potential contributions.

Additionally, we would like to highlight that since the extended discussion period began, many authors have engaged in fruitful discussions with their reviewers, which has led to positive feedback and score increases. We sincerely hope that our case is no exception and that the improvements made during the rebuttal period will result in a more comprehensive reassessment of our work.

Thank you once again for your insightful review and constructive feedback.

Best regards,

Authors

We sincerely appreciate Reviewer Bbht's valuable suggestions regarding the representation, figures, and citation formatting in our manuscript. We also highly value your concerns about our proposed method and the weaknesses you pointed out.

Here are our responses to your concerns and questions.

We hope they will help address your confusion and provide clarity.

Minor Comments

We have addressed all minor comments, including formatting, typos, notation clarity, vector graphics for figures (Now, all figures are in PDF to ease readability), and improved references. We appreciate your thorough suggestions!!

Weakness

W1. The method presentation in (mainly) Sec. 3.2...

Thank you for the detailed feedback!!!!

We have revised Sec. 3.2 for clarity with simplified notation, addressed computational ambiguities. We also updated Figure 2 with consistent formatting and accurate representation of operations.

Please check Figure.2 and Sec.3.2 in the updated PDF.

W.2 It is unclear to me why the Li block is called Linear.

The linear pattern extraction in the Li block relies on an Autoregressive (AR) model. Therefore, the extraction of $\hat{L}_i$ is completely Linear with respect to the input $H_i$.

Dropout randomly setting some values in $\hat{L}_i$ to zero during $training$. During inference (after training), Dropout is typically turned off, and the model behaves deterministically. Thus, the application of Dropout does not change the linearity.

The Linear prediction is obtained by: $P^{li}_{i}=L_iW+b$

Thus, the overall process in the Li block — from feature extraction to the final prediction — is linear in nature.

And since we employ an Autoregressive model to capture linear patterns in raw time series, we can see in Figure.1 that the Autocorrelation Function (ACF) of the decomposed linear patterns (Linear 1 and Linear 2) shows clear pattern of AR(1) time series, which means current point is predominantly influenced by its previous point.

W.3 The learned representations are somewhat redundant.

Revisiting our assumptions, the composition of linear and nonlinear patterns in each time series is undoubtedly diverse.

We argue that such redundancy is both natural and reasonable, as it is likely that the time series contains only a single linear pattern.

Once this pattern is fully captured by one Li Block, the patterns extracted by other Li Blocks become insignificant. The same logic applies to No Blocks.

In Fig. 5, we observe a dominant linear pattern (LP1) on ETTh1, while the other two (LP2&3) appear less significant.

In contrast, the nonlinear patterns NP1&2&3 have almost identical weights.

In Table 4, we can also observe that the optimal LiNo blocks for each dataset are diverse, further highlighting the differences in the linear and nonlinear compositions of each dataset.

So, such prediction visualization of LiNo provides us with the possibility to explore the unknown linear and nonlinear components of time series data.

Q.5 If the addition in l. 238 is element-wise, isn't the overall $N_i^{TF}$ entirely linear up to the point of activation. How can it be beneficial over a simple linear layer? Why is Tanh chosen, given that ReLU is a more common choice today? I do not understand the motivation behind the

Separately using temporal and frequency linear projections allows the model to specialize in capturing distinct aspects of time series data. Combining these projections using $N_i^{TF}$ enhances expressiveness, improving the model's ability to learn complex patterns and leading to better performance than a single projection.

Tanh was chosen over ReLU to provide a smooth, symmetric activation function that better captures the complex dependencies in both time and frequency domains.

These are proved in the following experiment result.

1. Temporal means only extracting temporal variations (without introducing nonlinearity)

2. Frequency strands for frequency information extraction (without introducing nonlinearity)

3. TF means these two patterns are captured simultaneously (without introducing nonlinearity)

4. The nonlinearity of the No Block is introduced by adding ReLU or Tanh activation function (based on TF).

The combination of Temporal and Frequency domain information (TF) consistently outperform single domain, and Tanh consistently outperforms ReLU across all datasets.

Q.6 The iterative pattern extraction reminds me of N-BEATS (Oreshkin et al., 2019). Are they that closely related? If so, this might be relevant for Sec. 2.

Though LiNo and N-BEATS share some similarities, such as both employing the concept of Recursive Residual Decomposition (RRD), they are fundamentally different:

1. N-BEATS does not explicitly capture linear and nonlinear separately.
2. As a univariate model, N-BEATS cannot handle more complex multivariate time series.
3. The role of RRD differs in each model: while N-BEATS uses RRD to refine the final prediction based on residual errors from previous predictions, LiNo employs RRD to capture more nuanced linear and nonlinear patterns.

We added these comparisons in the updated PDF, Section 2. We included further analysis on N-BEATS and LiNo in Appendix F.

Q.7 What are the diagonal stripes in, for instance, Fig. 19 (bottom, left)?

Fig. 19-31 are visualizations of the trained weights of Li Blocks and No Blocks on each dataset.

We employed the visualization techniques provided in the An Analysis of Linear Time Series Forecasting Models to explore the patterns learned by each Li Block and No Block.

More details are included in Appendix D.2.

Questions

Q.1 What is meant by "multi-level characteristics of real-world time series" (Sec. 1)? How do the time series in Fig. 1 relate to each other? How are they combined (additively?)?

The multi-level characteristics of real-world time series suggests that real-world time series are the result of the additive combination of various linear and nonlinear components (as illustrated in Fig.1 and Sec.3.1).

Fig.1 is the illustration of this assumption, where a raw time series (red) is decomposed into a series of subsequences (2 linear and 2 nonlinear). In other words, the raw time series (red) is the sum of the four time series (additively).

Q.2 Why SegRNN and TimeMixer are not included as baselines?

We did not include SegRNN and TimeMixer mainly based on the following considerations:

1. SegRNN uses MAE as its loss function, which is widely regarded as an unfair comparison against baselines (including our LiNo, Informer, Autoformer, iTransformer, etc.) that use MSE as the loss function.
2. SegRNN uses an input length of 720, and TimeMixer uses 512, whereas in our paper, to follow more influential works like Informer, Autoformer, and iTransformer, the input length for all baselines is uniformly set to 96.
3. Both SegRNN and TimeMixer have an uncorrected critical bug, which first identified in FITS. Details of the bug can be found here. This bug causes the models to skip a significant portion of the test set, resulting in unfair comparisons.

Q.3 Why is padding performed? Is the linear prediction in l. 221 the same as the (not very linear) MLP in Fig. 2? Which of the model components are learned via gradient descent?

Padding before convolution ensures proper alignment for the proposed convolution operation (A concise and efficient design for the Autoregressive model). This makes sure the input $H_i\in \mathbb{R}^{C \times D}$ and output $\hat{L}_i\in \mathbb{R}^{C \times D}$ share the same feature dimension as in Autoformer.

The MLP (which, thanks to your suggestion, should be more appropriately called FC) of the Li Blcok in Fig.2 refers to the linear prediction in l. 221.

All parameters in Li Block, including the autoregressive coefficients, biases, and weights in FCs, are learned via gradient descent during training.

Q.4 Regarding Fig. 2: What is the "backbone"?

The "backbone" to refer to the LiNo Block, which includes a Li Block and a No Block. However, upon careful consideration, we realized that the original diagram might cause confusion.

Therefore, we decided to abandon this concept.

Please refer to the updated model diagram in Figure 2 (middle) for the structure of the proposed LiNo Block.

We sincerely apologize for any confusion caused by the initial version of the diagram!

Oddly, Figures 3 & 4 are still rasterized despite your comment. However, this is not relevant for assessing whether the work should be accepted and can also be fixed in a camera-ready version.

• W.1: The presentation (e.g., Figure 2) has indeed improved. Although generally, highlighting changes by color changes is helpful. It would make it easier to check this properly.
• W.2: Makes sense; thank you for clarifying.
• W.3: I'll assume that you meant Figure 4. I agree. This makes sense.
• Q.1: This could be clarified in the manuscript, too.
• Q.2: I strongly disagree with them being impossible to compare against for several reasons. Firstly, SegRNN could very likely be trained by MSE, too. Secondly, training with different optimization criteria does not diminish comparability as long as the evaluation is done the same way (see, e.g., language model benchmarks). Thirdly, the claim, at least about TimeMixer, is plainly wrong. They explicitly discuss that different input lengths are not really comparable (see paragraph "Unified experiment settings" in p. 6). Subsequently, they "fix the input length as 96 for all experiments" for their main results in Table 2 (see caption). Fourthly, even if so, this is just an implementation detail and could be made comparable by you. Fifthly and lastly, the error you mentioned is just an implementation detail and not a flaw in the methods. You could simply provide a fair comparison yourself by changing one line and re-running the models.
• Q.3: Thank you.
• Q.4: Thank you for the cleanup.
• Q.5: While I do not understand why exactly this is happening (I lack a theory), you provide very convincing results. This work does not require an even deeper analysis.
• Q.6: Thank you for elaborating. This makes sense.
• Q.7: Alright.

While I am willing to raise scores (rating from 3 to 5, presentation by one, confidence by one), I am still concerned by the incomplete evaluation as discussed in Q.2.

Thank you for expanding the paper in this regard. It is a very significant improvement, and resolves my remaining main concern beside the initially confusing presentation. I thus raise my score from 5 to 6 and the soundness by one. I will not change it further.

Closing, some remarks you may consider for the camera-ready version:

• Regarding the implementation of the test size, you are right. This, indeed, has the potential to skew results significantly and should receive a bit more attention than it already did.
• The results you reported do not quite match the ones in Table 13 of Wang et al. (2024), making TimeMixer appear slightly worse than in the original work. I expect you for the final paper to double-check that nothing went wrong.
• While different training objectives yield different results, I still do not see the problem of no comparability. From my point of view, this is merely another hyperparameter to tune.

Dear Reviewer Bbht,

We sincerely appreciate the time and effort you dedicated to reviewing our paper during this busy period, as well as your recognition of its strengths and our efforts in the rebuttal phase (with more than 200 new results and 7new pages) that eventually clarified all of your concerns.

With the author/reviewer discussion phase now concluded, and no further concerns raised, we believe our rebuttal has adequately addressed all of your comments. However, based on the scoring standards of previous ICLRs, we are concerned that our current score of 5.5 places us at risk of rejection. We would be sincerely grateful if you could reconsider your score and provide us with the opportunity to present our work at the conference.

Thank you once again for your thoughtful review and valuable feedback.

Best regards,

Authors

We sincerely thank Reviewer 236Efor providing thorough and insightful comments. Here are our responses to your concerns and questions.

Tips: All the valuable suggested clarifications you provided have been carefully considered and incorporated into our revisions. Thank you once again.

Q1. Advantage of frequency domain operations

Frequency domain operations capture periodic and high-frequency components, aiding predictions even with shifting or weak seasonality, as shown in FITS. They provide complementary insights alongside time-domain features, as shown in the experiment results below.

We denote the use of only the temporal projection as Temporal, the use of only the frequency projection as Frequency, and the combination of both as TF.

Q2. Line 239 to 243: Can you elaborate on how the softmax is applied and the weighted mean is computed? It would be helpful to include a figure showing this operation in detail.

Q.4 Where does H_i sit in figure 2?

We have redrawn the model diagram based on your invaluable suggestion, and additionally, we included an extra figure to provide a detailed explanation of the proposed channel mixing technique in detail.

$H_i$ is the input feature of each LiNo Block with $H_0=X_{embed}$ and $H_i=R^N_{i-1}$, so $H_i$ sit at the beginning of each LiNo block.

Please check Figure.2 (upper left) for channel mixing, and Figure.2 (left) for where $H_i$ sit.

Q.3 How is the FFT and IFFT calculated? In particular, is this operation differentiable is is back-propagated through?

Following FITS, the FFT/IFFT operations are implemented using PyTorch’s differentiable operations, ensuring seamless gradient flow for back-propagation and enabling the model to learn frequency-based features effectively.

Give an input $R^L_i\in \mathbb{R}^{C \times D}$, the FFT, FC linear projection, IFFT are sequentially applyed to the $D$ dimension.

Q.5 Why the MLP is'nt applied when computing the residual for the next block

Applying the MLP directly to the obtained residual risks altering critical features needed for subsequent blocks. Keeping residuals untouched ensures each block extracts complementary features effectively without redundant interference.

Besides, This helps ensure that all residuals can be reconstructed back to the original input features, as emphasized in the preliminary analysis of Sec.3.1, preserving information integrity.

Q.6 Figure 2: What does the backbone consist of?

Initially, we considered using "backbone" to refer to the LiNo Block, which includes a Li Block and a No Block. However, upon careful consideration, we realized that the original diagram might cause confusion.

Therefore, we decided to abandon this concept. Please refer to the updated model diagram in Figure 2 (middle) for the details of the proposed LiNo Block.

We sincerely apologize for any confusion caused by the initial version of the diagram!

Q.7 What is the purpose of dropout in Equation 2?

Dropout in Equation 2 is used for regularization, helping to prevent overfitting.

It doesn't directly affect coefficient distribution, as it will be turned off during inference. But it ensures the model generalizes better across the input window.

Dear Reviewer 236E,

Thanks again for your valuable and constructive review, which helps us revise our work to a clearer stage.

We kindly remind you that the reviewer-author discussion phase will end soon. After that, we may not have a chance to respond to your comments.

Besides, during the rebuttal period, following your suggestion, we have answered your concerns and improved the paper, which may be helpful to you in further justifying our contribution.

All points raised are discussed during the rebuttal phase, including experimental results and clarifications, will be incorporated into the camera-ready version to enhance the paper’s clarity and accuracy.

Thanks again for your dedication in reviewing our paper. It helps us a lot.

We are looking forward to your feedback.

Dear Reviewer 236E,

Thank you for your time and effort in providing a thoughtful review of our work. As the discussion period is drawing to a close, we hope that you have had the opportunity to review our rebuttal.

If our response has addressed your concerns or clarified any aspects of the paper, we would be grateful if you could reconsider your assessment and adjust the score accordingly. Your feedback has been invaluable, and we appreciate the chance to further refine our work based on your insights.

Once again, thank you for your constructive review and for considering our responses. If you have any further questions or require additional clarifications, please do not hesitate to reach out.

Best regards, Authors

Dear Reviewer 236E,

We sincerely appreciate the effort you have put into reviewing our paper during this busy period, as well as the recognition of the strengths of our work. We are truly grateful for your assessment.

We fully respect your current evaluation of our work. However, based on the scoring standards of past ICLRs, we find that our current score of 5.5 is at the borderline level. Therefore, we kindly ask if you could consider raising the score once again, so that we would have the opportunity to present our work at the conference. We would be deeply grateful.

During the rebuttal period, we added over 200 new experiments and 7 pages of content, strengthening the paper and providing a more robust presentation of our approach.

We also note that many authors have received positive feedback and score increases during the extended discussion period. We hope that the improvements made to our work will also lead to a more comprehensive reassessment.

Thank you again for taking the time to review and comment on our paper.

Best regards,

Authors