TimeBase: The Power of Minimalism in Efficient Long-term Time Series Forecasting

Dear Reviewer Pc2P,

Thank you for taking the time to review our work and for your valuable feedback.

---

D1. Discussion on Other Related Works

Thank you and Reviewer 2K2o for suggesting a comparison with related work and clarifying our distinctions and contributions. We provide an overall comparison in the table below, followed by a brief analysis and discussion.

• FITS performs interpolation after low-pass filtering in frequency domain. FITS utilizes frequency domain feature, while TimeBase only operates in time domain, which is a fundamental difference between the two.

• SparseTSF performs sparse prediction after local feature extraction, focusing on using TCN to capture local features. In contrast, TimeBase leverages the low-rank nature of long-term time series and enhances the encoding of key patterns through basis extraction. In terms of data efficiency, SparseTSF makes predictions using all downsampled segments, whereas TimeBase improves efficiency by extracting only a small number of key basis components.

---

W1. Integration with Patch-Based Methods

TimeBase can serve as a lightweight complexity reducer for patch-based methods by replacing their feature extraction step with basis extraction and aggregation. Specifically for any patch-based models, after transforming the time series into N patches, TimeBase aggregates them into R core patches (R ≪ N). The model then performs forward computation based only on these R core patches, significantly reducing computational redundancy. For example, when integrated into PatchTST, TimeBase reduces the MACs from 14.17G to 1.58G (Table 7, Appendix C.2).

---

W2. Batch Size Selection (Section 4.3)

For efficiency analysis, we set the batch size to 12 for all models to ensure a fair comparison of efficiency metrics. This choice considers that an overly small batch size is impractical, while an excessively large batch size may cause some models to exceed memory limits. Following the comparison method used in FITS, we also set the batch size to 12 when testing the models' peak memory usage.

---

W3. Why Use Linear Layers

TimeBase is designed to avoid unnecessary complexity while maintaining strong predictive performance. CNNs and Transformers are also effective in capturing sequential dependencies, but they introduce higher computational overhead. Additionally, empirical results from methods like DLinear and TimeMixer have demonstrated the effectiveness of linear models in time series forecasting. Therefore, our method leverages linear basis extraction, which efficiently captures key temporal patterns while significantly reducing computational cost.

---

S1. Table 2 Formatting Improvement

Thank you for your suggestion. We will update Table 2 by using different colors to highlight the best and second-best results, improving readability.

---

Q1: Replacing BasisExtract() with an Attention Mechanism

Thank you for your question. Here, we compare implementing BasisExtract() with Attention() VS the previous Linear(). The Attention-based BasisExtract() first applies a multi-head attention mechanism for global segment feature extraction, followed by a fully connected layer to map the features into a small set of basis components.

The table below (720-input, 720-output on the Electricity dataset) shows their scale comparison, where the computational cost and parameter count of the Attention() version are 3-4 times higher than the previous implementation. The second table presents their prediction performance comparison (720-input, 720-output), indicating that on the Electricity and Traffic datasets, the Attention-based BasisExtract() achieves performance improvements. This suggests that increasing the parameter count enhances the expressiveness of the extracted basis components, leading to higher forecasting accuracy.

---

Q2: Selecting an Appropriate P Value Without Prior Knowledge

When no prior domain knowledge is available, data-driven heuristics such as Singular Spectrum Analysis (SSA) or Fourier Transform can help estimate dominant periods.

Dear Reviewer JfQc,

Thank you for your interest in our paper and the constructive feedback you provide. We will address your questions point by point.

---

W1. Real-world low-resource scenarios

Thank you for your suggestion! TimeBase can be applied in various resource-constrained environments, such as edge computing, mobile devices, and IoT-based forecasting. In these scenarios, the ability to handle large-scale time series data with minimal computational overhead makes TimeBase an ideal candidate.

---

W2. Comparison with more input length

Thank you and Reviewer 2K2o for suggesting an analysis of different look-back window sizes effect to improve the experimental quality of our paper. In our original experiment, we choose a fixed length of 720 time steps to maintain consistency with FITS [1] and ensure a fair comparison. Additionally, we provide experimental MAE comparisons with input lengths of [96, 336, 720], all with a 720-step output length, demonstrating that TimeBase maintains both its lightweight design and high prediction accuracy across different input lengths. Two interesting observations from this analysis are:

• Most datasets, i.e., Electricity, ETTm1, ETTm2, show improved prediction performance as the input length increases.
• For lightweight models such as TimeBase, SparseTSF, FITS, and DLinear, when the input length is only 96, none of them achieve good prediction performance on the Traffic dataset, but iTransformer and PatchTST could maintain prediction accuracy.

We appreciate your key suggestions for improving the quality of our paper presentation. We will include all results in the revised manuscript to provide more experimental analysis.

---

S1. Decimal Precision in Table 3

Thank you for pointing this out. We will standardize the decimal precision to three decimal places for consistency in the revised manuscript.

---

S2. TimeMixer Description Correction

We appreciate your attention to detail. We will correct the description of TimeMixer on page 5, lines 250-251, and clarify that it is not based on LLM. This will be updated in the revised manuscript.

---

Q1. Impact of the 'drop_last' correction

The correction to the 'drop_last' parameter ensures that incomplete batches at the end of a sequence are excluded during testing. In some previous baselines, setting 'drop_last=True' with a very high batch size led to the discarding of difficult-to-predict samples, resulting in a significant improvement in prediction accuracy. This issue was later pointed out by FITS [1] and TFB [2], helping the time series forecasting field achieve better evaluation of predictive models. TimeBase addresses this bug to provide a fairer experimental comparison.

---

[1] FITS: Modeling Time Series with 10k parameters. ICLR 2024.

[2] TFB: towards comprehensive and fair benchmarking of time series forecasting methods. VLDB 2024.

Dear Reviewer n697,

Thank you for taking the time to review our paper and for your valuable suggestions to improve its quality.

---

W1. Basis Orthogonal Restriction Analysis:

We appreciate your suggestion. From the perspective of the data space, the orthogonality of the basis vectors enhances its representation power, providing them ability to express as any vector in the data space through linear combination [1]. Therefore, the ts basis components should also be diverse and distinct, preventing the extraction of very single time-series patterns [2]. Based on this, we apply the Orthogonal Constraint. In the time-series space, each time series segment can be viewed as a composition of several orthogonal basis components. The goal of TimeBase is to transform the long-term forecasting task at the time-step level into a task of basis extraction and prediction at the period level, thus enabling an efficient time-series forecasting model (0.39K, 1000 times smaller than DLinear). In the revised version, we will add a detailed analysis discussing the potential advantages of "Basis Orthogonal Restriction."

---

W2. CPU Detail:

Thank you for pointing this out. The CPU used in our experiments is an Intel Xeon E5-2609 v4 CPU (16 cores, 2 sockets, 1.7 GHz base frequency), and we will update the manuscript accordingly.

---

W3. TimeBase for Multivariate Time Series:

Thanks for your question. TimeBase does not consider relationships between variables; instead, it transforms the multivariate time series forecasting task into multiple univariate forecasting tasks. We have explained this in the original manuscript on Page 3, lines 150-156, “ Most existing multivariate time series are homogeneous, meaning that each sequence within the dataset exhibits similar periodicity [3]. This characteristic allows them to be organized as a unified multivariate time series. Based on this property, we employ the Channel Independence [4] to simplify the forecasting of MTS data into separate univariate forecasting tasks.”

---

W4&Q1. Handling Multiple Time Series Patterns:

Most forecasting scenarios (e.g., traffic, electricity) exhibit a dominant period or overlapping periods (e.g., a weekly period encompassing a daily period). Therefore, we set the shortest period as the fixed P. For datasets with multiple periods, TimeBase can extract different basis components using various P. In Appendix C.6, we demonstrate that

$\text{MSTimeBase} = \sum_{i} \text{TimeBase}(X; P = p_i)$

could achieve a 0.006 MSE improvement on the traffic dataset. The marginal improvement suggests that the primary patterns could been effectively captured by single&primary basis length .

---

C1. Minor Typo

We acknowledge the typo regarding Figure 7 not being referenced in the Appendix. We will correct this and ensure that Figure 7 is properly referenced in the revised manuscript.

---

Question

Q2. TimeBase can be applied to MTS data by independently extracting basis components from each time series in the multivariate dataset. Each time series is processed to capture its key temporal patterns, and these components are then aggregated to form a unified forecast.

---

[1] Time series with periodic structure. Biometrika 1967.

[2] Decomposition principle for linear programs. Operations research 1960.

[3] TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis, ICLR 2023.

[4] A Time Series is Worth 64 Words: Long-term Forecasting with Transformers, ICLR 2023.

Dear Reviewer 2K2o,

We greatly appreciate your constructive feedback and have made our best to address your concerns.

---

C1. Sentence Explanation

This sentence (Page 6, Line 316-318) describes the plug-and-play integration of TimeBase into PatchTST. TimeBase enhances the extraction of compact 'Essential Patterns' and enables a more effective 'Patch Interactions' for PatchTST, which leads to improved accuracy and reduced model complexity (as shown in Table 3 of main text) . We will refine the description to make it clearer and more understandable.

---

M1. Fixed Basis Length

Most forecasting scenarios (e.g., traffic, electricity) feature a single dominant period or overlapping periods (e.g., a weekly period encompassing a daily period). Balancing data efficiency and accuracy, we set the shortest dominant period P to capture key temporal patterns.

Besides, TimeBase can extract various-length basis using different P. In Appendix C.6, we show that

$MSTimeBase = \sum_{i} TimeBase(X; P = p_i)$

yields a 0.006 MSE improvement. The minor improvement suggests that the only capturing primary pattern by TimeBase is sufficient for lightweight forecasting.

---

M2. Contribution of TimeBase

TimeBase simplifies LTSF by extracting basis components. In fact, it has distinct differences from FITS and SparseTSF (Analysis is in D.1 response to Reviewer PC2P). Overall, TimeBase’s Key Contributions are three-fold,

• Modeling Techniques: Propose basis extraction with orthogonal constraints to capture primary patterns and avoid redundant computation from surge of time-series.
• Lightweight, Efficient and Effective Forecaster: Achieve ultra lightweight but effective forecasting (0.39K Params, 2.77M Macs).
• Plug-and-Play Usability: Act as a complexity reducer for patch-based methods, cutting PatchTST’s MACs to 1/9 of its original one (e.g., 14.17G → 1.58G, Tab.7, Appendix C.2).

---

M3. Basis Extraction and Low-Rank Adaptation

The key distinction between them lies in how to handle long-term dependencies in LTSF.

• Basis extraction identifies a compact set of representative time patterns from historical data. Unlike low-rank approximation, TimeBase explicitly extracts key basis components and uses them as fundamental units for prediction.
• Low-rank adaptation, in contrast, often relies on SVD to approximate data with a reduced-rank structure but does not explicitly extract interpretable components.

---

T1. Theoretical Supplement.

TimeBase with O(T+L) scale is more efficient compared to SparseTSF/DLinear with O(T*L) in LTSF. This will be added in revised version.

---

E1. Baseline Settings

---

E2. Inference Time

We provide CPU inference time as it is more practical for edge deployment. Thanks for your advice! We now include GPU (A100-80GB) inference times (ms) for 720-output Elec.

---

E3. Function of Figure 4

Theorem 3.1 analyzes model scale theoretically. To intuitively verify TimeBase's efficiency, we visualize three computational metrics (Mem, Time, Paras) under ultra-long look-back windows in Figure 4.

---

E4. Impact of Input Length

Thanks for your advice! We provide comparisons and analysis for input lengths [96, 336, 720] in the W2 response to Reviewer JfQc.

---

D1. Discussion of FITS and SparseTSF

Thanks for your valuable advice. We have discussed the differences between TimeBase, FITS, and SparseTSF in the D.1 response to Reviewer PC2P, and we will add this discussion in the related work.

---

Question

Q1. We apply a sliding window to segment the dataset into (num, patch) samples of 720 input length, perform SVD on each sample, sort the singular values, and then average them to derive the dataset’s typical singular value distribution.

Q2. Figure 2 primarily highlights the comparison between TimeBase and DLinear. We have updated SparseTSF (1.0K Parameters, 125.2M Memory, 2.59ms CPU Inference Time, 12.71M MACs) in Figure 2, and you can check it in our original anonymous code link in Abstract. Thank you!

Q3-4. Thank you very much! We will update them.

Q5. P=4 (ETTm/Weather) | P=24 (Others)