Enhancing the Maximum Effective Window for Long-Term Time Series Forecasting

Thank you very much for your valuable feedback and suggestions. Below, we address each point in turn.

---

Q1. The proposed MEW metric appears to be essentially equivalent to hyperparameter searching over input sequence lengths. The paper fails to clearly articulate how MEW differs from conventional parameter tuning approaches. If MEW is simply finding the optimal input length through grid search, then the performance improvements may be attributed to more extensive hyperparameter exploration rather than a fundamental methodological contribution.

Thank you for raising this important point. While MEW does involve evaluating model performance across different input sequence lengths—similar to hyperparameter tuning—we emphasize that MEW is not a tuning strategy, but rather a diagnostic metric designed to quantify a model’s effective ability to leverage long-term historical information.

Specifically:

• Objective: Traditional hyperparameter tuning aims to find the best lookback length to optimize performance. In contrast, MEW focuses on analyzing where performance saturates, revealing a model's limitation in capturing long-range dependencies, rather than optimizing for best-case performance.

• Analytical value: MEW provides a model-centric lens to assess historical information utilization capacity. For instance, a model with a larger MEW can leverage longer-term context effectively, which is crucial for long-horizon forecasting.

As shown in Figure 3(a) and Table 2, we apply our proposed HTM and IBF modules to multiple backbone models (PatchTST, Transformer, Autoformer, Informer), and consistently observe that both MEW increases and forecasting performance improves. This demonstrates that enhancing a model's MEW can lead to more accurate long-term predictions—highlighting MEW as an insightful metric beyond simple tuning.

---

Q2. While PIH development is presented as the main contribution, the performance comparisons lack evaluation against more recent state-of-the-art models. If the contribution is modular development, the paper should demonstrate that these modules can be applied to other contemporary models to achieve superior performance. If the contribution is model development, direct comparisons with recent state-of-the-art models are essential to establish the advancement.

We appreciate this suggestion. Our primary contribution lies in modular design: specifically, we introduce the IBF and HTM modules, which can be integrated into a wide range of forecasting models to improve their ability to utilize long-range dependencies.

To support this, we have already validated our modules across multiple backbones from different years and architectures—Transformer (2018), Informer (2020), Autoformer (2021), PatchTST (2023), and iTransformer (2024)—as shown in Table 2 and Table 7. In all cases, the integration of IBF and HTM improves both MEW and forecasting accuracy.

To further address the reviewer’s concern, we have conducted additional experiments on the recently proposed PowerFormer (2025). The table below summarizes the MEW and MSE results after integrating our modules ($L \in [72, 96, 120, 144, 168, 192, 273 228, 336, 512, 1024, 1440]$,$T=720$):

These new results further confirm that our modules generalize well and can enhance even the latest forecasting models. We will incorporate this additional evaluation into the final version.

---

Q3. The paper provides insufficient analysis of how the IBF functions within the model and contributes to performance improvements. While the theoretical foundation is provided, there is a lack of empirical analysis demonstrating what types of information IBF filters out, how it affects the learned representations, and why this leads to better forecasting performance.

We thank the reviewer for raising this important point. While the paper includes theoretical motivation for the Information Bottleneck Filter (IBF), we now elaborate on its empirical behavior and contribution, focusing on the following two aspects:

1. What IBF filters out
IBF selects input patches that are most informative for the forecasting target, based on a mutual information approximation. As shown in Eq. (5):

$\mathbf{c}_i = \operatorname{sigmoid}(\operatorname{MLP}(\mathbf{z}_i))$

each input token $\mathbf{z}_i$ is assigned a learned importance score $c_i$. A well-trained IBF learns to assign higher scores to informative tokens and suppress irrelevant or noisy ones.

In Figure 3(c), we visualize the top-20 patches selected by IBF (i.e., those with the highest $c_i$ values) for a sample from the Electricity dataset. These selected patches are primarily located around peaks and transition points, which are critical to the dynamics of the signal. We observe similar behavior across other datasets (e.g., Traffic, Weather, Exchange), where IBF consistently highlights high-gradient, or periodic regions, while suppressing redundant or low-variance segments.

These observations indicate that IBF functions as a saliency-aware filter, learning to emphasize structure-rich regions that are more predictive of future values.

2. Why this leads to better forecasting
By filtering out uninformative historical inputs and retaining only high-salience patches, IBF improves forecasting in the following ways:

• It reduces attention dilution, allowing attention mechanisms to focus on relevant content;

• It mitigates overfitting, especially in long-input regimes where noise can dominate;

• It improves temporal abstraction by ensuring that only informative components are passed to downstream layers.

This filtering process aligns well with the Information Bottleneck principle, enhancing the model’s ability to generalize across time and datasets.

Due to NeurIPS 2025 policy constraints, we regret that we could not include additional visualizations or case-specific analyses in the current submission. However, we will incorporate more case studies and dataset-level statistics (e.g., patch entropy, activation frequency) in the revised version to further illustrate the empirical behavior of IBF.

---

We hope that these detailed explanations address your concerns. Thank you again for your thoughtful and constructive review.

Thank you very much for your valuable feedback and suggestions. Below, we address each point in turn.

---

Q1. The difference between interval split and block split is not thoroughly clarified in the introduction. The methodology lacks specific guidance on how to choose the optimal block size.

Thank you for this insightful question. We agree that further clarification is warranted.

The key distinction between interval split and block split lies in the type of temporal semantics each method preserves:

• Interval split distributes patches across $K$ blocks in a round-robin fashion, as defined in Eq. (11), i.e., patch $j$ is assigned to block $b_i$ if $j \equiv i \pmod{K}$. This method emphasizes global temporal diversity within each block, enabling the model to access patterns from different time positions. It generalizes the even–odd downsampling strategy in SCINet^[1] to $K$-way interleaving and is especially suited for data with globally distributed seasonal structures.
• Block split, in contrast, divides the sequence into contiguous segments, as defined in Eq. (12), where each block $b_i$ spans a local window. This method is particularly effective for time series with local periodicity, where patterns repeat consistently within specific time intervals (e.g., daily cycles in electricity consumption).

Interval Split vs. Block Split: In essence, interval split captures global variation, while block split preserves local coherence. Both methods aim to retain the semantic meaning of the original long sequence after partitioning. We empirically compared the two splitting strategies across multiple datasets, as shown in Figure 4 (right). The results indicate that while both methods achieve comparable performance overall, their relative effectiveness varies across datasets, depending on the dominant temporal characteristics. For instance, datasets with strong periodicity tend to benefit more from block split. This validates our intuition and suggests that the choice of split strategy may depend on dataset-specific dynamics—an aspect we plan to investigate further in future work.

Block Size: Regarding the choice of block size, we provide further details in Appendix C.1, where we study the impact of the number of partitions $K$ as a tunable hyperparameter. Under the block split scheme, the block size is computed as $L/K$, where $L$ is the total sequence length (we set $L = 1024$ in our experiments). Our empirical results show that $K = 4$ typically yields the best performance, corresponding to a block size of 256. This setting offers a good trade-off between retaining meaningful local patterns and reducing Transformer input length.

---

Q2. Expanding the historical window in large-scale models could provide more meaningful insights. Although the authors touch upon this in the limitations section and acknowledge the computational challenges, further exploration in this direction would be valuable.

Thank you for this thoughtful suggestion. We fully agree that extending our method to large-scale models represents an important and promising direction for future research.

In this work, our focus is on analyzing and improving the MEW, and we conduct extensive experiments on widely-used compact models such as PatchTST, Transformer, Autoformer, and Informer. These choices allow us to systematically study how our proposed HTM and IBF modules contribute to enhancing MEW while keeping the computational cost tractable.

We acknowledge that scaling to state-of-the-art large models, such as Time-MoE^[2] or Timer-XL^[3], would offer additional insights—especially in extremely long sequence modeling scenarios involving billions of parameters and trillions of time steps. However, due to the significant training and resource demands, this was beyond the scope of the current study.

[1] Liu, Minhao, et al. "Scinet: Time series modeling and forecasting with sample convolution and interaction." Advances in Neural Information Processing Systems 35 (2022): 5816-5828.

[2] Shi, Xiaoming, et al. "Time-moe: Billion-scale time series foundation models with mixture of experts." arXiv preprint arXiv:2409.16040 (2024)

[3] Liu, Yong, et al. "Timer-xl: Long-context transformers for unified time series forecasting." arXiv preprint arXiv:2410.04803 (2024).

---

We hope that these detailed explanations address your concerns. Thank you again for your thoughtful and constructive review.

Thank you very much for your valuable feedback and suggestions. Below, we address each point in turn.

---

Q1. Referring to weakness 1: Is the proposed MEW metric related to the prediction horizon, data, and model? If so, what is its precise mathematical definition?

Yes. Given fixed settings of the dataset, model architecture, and prediction horizon, the Maximum Effective Window (MEW) is defined as the smallest look-back window length beyond which increasing the window no longer improves performance. In other words, any window longer than MEW provides no significant gain under the same conditions.

Thank you for this suggestion. While we included an intuitive explanation in the Introduction section, we provide a formal definition below: Let:

Then MEW is defined as:

$\\text{MEW} _{\\varepsilon}(M, D, H) = \\min\\left\\{ L _i \in \\Lambda : \\forall j > i, \; S(L _j) \ge S(L _i) -\\varepsilon \\right\\}$

In words: MEW is the smallest look-back length beyond which no further improvement larger than $\varepsilon$ is observed. Since MEW depends on $(M, D, H, E)$, it is setup-specific and must be re-estimated for different tasks.

---

Q2. Could you quantify the computational overhead of the IBF and HTM modules in more detail (e.g., in terms of FLOPs or additional parameters) to better clarify the trade-offs of the hybrid approach?

We appreciate the reviewer’s interest in this aspect.

IBF introduces negligible overhead, as it is implemented as a lightweight MLP consisting of two Linear layers with one ReLU activation in between. Both input and output dimensions match the Transformer’s patch embedding size (typically 16 or 128 in PatchTST). This structure results in only a small number of additional parameters and almost no increase in FLOPs, making IBF extremely efficient in practice.

HTM significantly reduces computational cost by replacing one Transformer layer with a Mamba layer, while keeping the overall model depth identical to PatchTST (e.g., three layers in total). In our implementation, all $K$ subsequences share the same Transformer, so the main overhead difference lies in the substitution of one Transformer layer with one Mamba layer. Below, we provide detailed comparisons under the typical setting where $L=1024$, patch size = 16, stride = 8 (thus 128 patches), and $d_{\text{model}} = 128$:

For datasets like Weather, Traffic, ETTm1/2, and Electricity (input shape: 1 × 128 × 128)

For ETTh1 and ETTh2 (input shape: 1 × 128 × 16):

From these numbers, we observe that Mamba yields significantly lower computational complexity than Transformer, especially in terms of FLOPs, while keeping parameter counts in the same range.

The theoretical complexity analysis further supports these observations. In PatchTST, the patching mechanism reduces the Transformer’s effective input length from $L$ to $L / (P \cdot K)$, where $P$ is the patch length and $K$ the number of subsequences. Combined with the full-sequence Mamba processing, the overall complexity becomes:

$O(L/P)+O((L/PK)^2)$

Although the latter term still exhibits quadratic complexity, appropriate choices of $P$ and $K$ can maintain $L/P$ within an acceptable constant range.

In summary, the proposed architecture offers clear advantages in both efficiency and performance. IBF adds almost no overhead, while HTM achieves a 2–5× reduction in FLOPs relative to a pure Transformer layer. As shown in Figure 4, this comes with consistent improvements in forecasting accuracy, confirming that combining Mamba and Transformer captures complementary temporal features without increasing computational burden.

We hope this detailed explanation clarifies the trade-offs and justifies the practicality of our hybrid design.

---

We hope that these detailed explanations address your concerns. Thank you again for your thoughtful and constructive review.

Thank you very much for your valuable feedback and suggestions. Below, we address each point in turn.

---

Q1. There are minor inconsistencies that could suggest a lack of careful proofreading. For instance, the caption for Figure 5 states the prediction length is T=720, while the plots are labeled T=96.

Thank you for catching this mistake. You are absolutely right: the caption should read T = 96 rather than T = 720.

---

Q2. Why Mamba + Transformer?

In Appendix E.2, we provide a detailed explanation of why we adopt this hybrid architecture:

• Computational trade-off: Although Mamba is theoretically less expensive than the Transformer for very long sequences (e.g., $L=1024$), the Transformer becomes more efficient for shorter windows (e.g., $L\le336$).
  – Our design uses Mamba to process the original long sequence and the Transformer to handle the resulting shorter subsequences, yielding a clear efficiency advantage over either model alone.

• Complementary strengths: Empirically, the hybrid (HTM) outperforms the pure-Mamba variant (see Figure 4, left), likely because Mamba and Transformer capture different but complementary features, resulting in a richer sequence representation. Similar findings appear in recent works such as Jamba^[1] and Mamba-2-Hybrid^[2].

---

Q3. While the components themselves (e.g., Mamba, IB theory) are drawn from prior work, the originality lies in their integration and application to LTSF.

We fully acknowledge this point. Our main contribution lies in the novel integration of the HTM module and the IBF within a Transformer-based framework—along with their demonstrated effectiveness in long-term time series forecasting and in improving the model’s MEW.

---

Q4. While the proposed method demonstrates strong empirical performance, its contribution should be evaluated in the context of its incremental nature and the associated overhead.

Thank you for raising this important point about overhead and incremental contribution.** Figures 3(b) and 4 (left) directly address this concern:

1. Efficiency gains:
   • Figure 3(b) reports both computation time and memory usage. Replacing the pure Transformer (PatchTST) with our HTM (Transformer + Mamba without IBF) reduces time and memory by 2–3×. Adding IBF to form PIH introduces negligible overhead, as it is implemented as a lightweight MLP.
2. Incremental performance:
   • Figure 4 (left) compares PatchTST (baseline) against three variants—+IB, +HTM, and PIH—plus a pure-Mamba variant (+HMM). Using $L=1024$ and $T\in{96,192,336,720}$ over seven datasets, we observe:
     1. Both IBF and HTM individually improve accuracy, and their combination (PIH) achieves the best results.
     2. HTM slightly outperforms HMM, confirming that Transformer and Mamba excel on different sequence lengths and benefit from hybridization. In summary:

• HTM delivers a 2–3× efficiency boost, and IBF adds virtually zero extra cost.
• Both components yield clear accuracy improvements, with their combination (PIH) providing the strongest results.

---

We hope these explanations and corrections satisfactorily address your concerns. Thank you again for your thoughtful review.

[1] Lieber, Opher, et al. “Jamba: A hybrid transformer-mamba language model.” arXiv preprint arXiv:2403.19887 (2024).
[2] Waleffe, Roger, et al. “An empirical study of mamba-based language models.” arXiv preprint arXiv:2406.07887 (2024).