Thank you for the thoughtful review and constructive feedback! With our best efforts, we conducted numerous additional analyses to address your concerns. The full results are at https://anonymous.4open.science/api/repo/ICML25Reb-034C/file/Results.pdf?v=b00206ad. Please understand that due to the strict 5000-character limit, we can only summarize the most critical results and insights in the rebuttal :)

Q1: Compare with AutoGluon/AutoSklearn2 ensemble and Chronos-Bolt

We implement and test the following approaches following your suggestions:

• Ensemble: forward selection (Caruana), portfolio, and zero-shot techniques from AutoGluon and AutoSklearn2.
• Foundation Model: Chronos-Bolt-Base, both zero-shot and finetuned.
• AutoML Ensemble: AutoGluon-timeseries with all 24 available base models (with high_quality presets).

They are trained and evaluated on all 16 datasets, which are categorized into 3 task types:

• Long-term Multivariate: ETTh1/h2/m1/m2, Weather, Electricity, Traffic
• Short-term Multivariate: PEMS03/04/07/08
• Short-term Univariate: EPF (NP/PJM/BE/FR/DE)

We report the averaged MAE for each task type below (with 1st*/2nd/3rd best highlighted), please see full results for each dataset and metrics in Table 1 in the link.

Key Findings

1. TimeFuse shows a clear advantage across various task types, and we summarize the reason below.
2. Caruana/portfolio/zero-shot ensemble search the optimal fusion weights at dataset-level, but still statistic at test time. In contrast, TimeFuse can adaptively predict the optimal weights for each test sample.
3. Chronos is limited to univariate forecasting only and is pretrained with a short predict length (64). In Chronos's paper, evaluations are also limited to univariate short-term (4-56) tasks. Our experiments reveal that Chronos suffers in long-term OR multivariate forecasting tasks. While fine-tuning offers improvements, its performance on such tasks is still far from optimal.
4. AutoGluon shares similar limitations with Chronos as it also only supports univariate prediction, and Chronos is one of its main base models. Additionally, AutoGluon inference is notably slow on long-multi tasks (e.g., needs ~20 hours for predict on traffic dataset with 862 variates), primarily due to (i) the need for independent inference per variate, and (ii) many AutoGluon base models lacking GPU acceleration and running solely on CPUs.

Q2: Clarification on the evaluation protocol

We appreciate your point on the necessity of testing meta-learning methods on unseen datasets, but we would like to clarify two aspects:

1. Table 5 presents valid zero-shot results: TimeFuse does not train on data from the target dataset in any predict length for zero-shot results. For instance, the results for ETTh1 are obtained with the fusor trained solely on 6 other long-term forecasting datasets. Similarly, results for PEMS03 involve training the fusor exclusively with data from PEMS04/07/08.
2. TimeFuse is not a "pure" meta-learning approach: We note that TimeFuse is a model fusion framework, it cannot directly predict new datasets without training base models on the new data first.

Therefore, we opted to compare TimeFuse with other methods (e.g., Caruana/portfolio/Chronos) that also access new data for fine-tuning/searching optimal weights to answer Q1, following your suggestion.

Q3: Discussion of related works

We greatly appreciate the suggested papers. We have read these articles and the references cited therein, gaining significant insights from them. We will revise our paper to more accurately reflect our position. Specifically, we believe that compared with existing works, TimeFuse's core novelty and contribution lie in introducing a methodology for instance-level dynamic ensembling based on the input sample's meta-features, and demonstrating its broad effectiveness in practical applications. We will discuss these additional related works in the paper to more accurately and comprehensively articulate TimeFuse's relationship with existing research and its position within the literature.

Finally, we want to thank you again for the thoughtful review, they have been very helpful in improving this paper. We’ve dedicated over 60 hours to implementing the new algorithms, training models, testing, and organizing results, and we will continue expanding those results to further enhance the paper’s quality. We hope these responses address your concerns and would be glad to continue the discussion if you have any further questions!

Thank you for your recognition and thoughtful review! We've conducted extensive analyses to address your concerns. Full results: https://anonymous.4open.science/api/repo/ICML25Reb-034C/file/Results.pdf?v=b00206ad. Please understand that due to the 5000-character limit, we can only summarize key findings here.

E1: Training objective of base models

Base models are trained using their standard prediction losses; the overall (fused) forecasting loss in Eq (1) and (2) is only for training the fusor. In other words, the base models’ training and inference remain entirely standard—we do not modify their input format, loss functions, or inference procedures. Please refer to Appendix A.3 Implementation Details – Base Forecasting Models for more details.

E2: Compare with advanced ensemble/MoE algorithms

Thank you for the great suggestion. We test 4 more advanced ensemble methods from AutoSklearn2 and AutoGluon:

1. Forward Selection: Builds an ensemble by iteratively adding the model that most improves current predictions on a validation set.
2. Portfolio: Optimizes the model weights to directly minimize validation loss, similar to risk minimization.
3. Zero-Shot: Computes the agreement/similarity among model predictions and assigns higher weights to models that have more agreement with others.
4. AutoGluon: (AutoML) learns the optimal ensemble of 24 base models (with high_quality presets).

They are trained and evaluated on all 16 datasets:

• Long-term Multivariate (LM): ETTh1/h2/m1/m2, Weather, Electricity, Traffic
• Short-term Multivariate (SM): PEMS03/04/07/08
• Short-term Univariate (SU): EPF (NP/PJM/BE/FR/DE)

We report the averaged MAE for each task type below (with 1st*/2nd best highlighted), please see full results for each dataset and metrics in Table 1 in the link.

Key Findings

1. TimeFuse consistently outperforms others across diverse task types.
2. Caruana, portfolio, and zero-shot ensembles optimize fusion weights at the dataset level, but still statistic at test time and thus underperform TimeFuse's sample-level adaptive ensemble.
3. AutoGluon is limited to univariate forecasting and is extremely slow for long-multivariate tasks, due to per-variate inference and lack of GPU support in many base models.

We note that we also evaluated the pretrained foundation time-series model Chronos. Due to space constraints, please kindly refer to our response to Reviewer oty5 Q1 for more details.

R1: Related work

Thank you for pointing out this related work. We have carefully reviewed the paper and its references and will include a detailed discussion in our revised version.

W1 & W2: Clarification on expanding model zoo and oversampling

We appreciate the weaknesses you pointed out and would like to clarify two points:

1. Retraining the fusor incurs minimal computational cost—usually just a few minutes. This allows easy integration of new models by simply adding their predictions to the meta-training data and retraining the fuser.
2. We use oversampling to balance the data distribution and prevent the pattern of minority datasets from being overwhelmed by larger ones. We understand concerns about its impact on data representation quality, a potential solution is to adopt more advanced data augmentation techniques to achieve distributional balance.

Q1: Input format of base models

Yes, all forecasting base models accept the same input format. As mentioned in our response to E1, the base models’ training and inference remain entirely standard, with no additional processing required.

Q2: Why not raw features?

Sorry for the confusion. We’d like to clarify that using raw features directly as meta features results in higher-dimensional meta features, rather than more samples. For example, consider a input time series sample $X$ with $D$ variates and length $L$. Using raw features means treating $X\in R^{(L,D)}$ as a single input sample (instead of $D$ samples) to the fusor. In contrast, meta features compress information from $X$ into a lower-dimensional space, reducing the risk of overfitting when learning from complex, high-dimensional raw features. We also ran additional experiments using raw features to validate this point, please see Table 6 in the full results. Results show that using meta features consistently outperforms raw features by a significant margin across all datasets.

Thank you again for your recognition :) We’ve devoted over 60 hours to get the new results and analysis, which we believe significantly strengthen the paper. We hope our responses have addressed your concerns and would be glad to discuss any further questions.

Great thanks for your thoughtful review and constructive feedback! With our best efforts, we conducted numerous additional analyses to address your concerns. Full results: https://anonymous.4open.science/api/repo/ICML25Reb-034C/file/Results.pdf?v=b00206ad. Please understand that due to the 5000-character limit, we can only summarize the most critical findings in the rebuttal :)

Q1: Experiment with long input length

Following your suggestion, we trained multiple base models and TimeFuse with longer input context lengths $L$ (both 336 and 512) on 7 long-term forecasting datasets. Given limited time, we trained PatchTST and models newer than it except for TimeMixer, which encounters OOM on V100-32GB GPU when training with L=336 or higher.

We report the averaged MSE for $L=96/336/512$ below (with 1st*/2nd best highlighted), please see full results for each dataset and metrics in Table 4 in the link.

Key findings

1. Across different $L$, TimeFuse consistently outperforms individual models by dynamically fusing their predictions.
2. Both PatchTST and PAttn benefit from longer input lengths. At L=336/512, PAttn replaces TimeXer as the best model, with PatchTST showing comparable performance.
3. TimeXer and iTransformer are insensitive to input length, while TimesNet degrades as input length increases. Despite these mixed trends, TimeFuse effectively learns each model’s strengths across samples to produce more accurate predictions.

Additionally, we want to highlight that we use TSLib for implementing all models with unified APIs and recommended hyperparameters. While there may be slight differences from the original PatchTST implementation, we confirm that our PatchTST results align with (often better than) those reported in recent papers (e.g., TimeMixer).

Q2 & M1: Results and clarification on TSFEL feature set

Please see Table 6 in the full results for more results with the TSFEL feature set.

We want to clarify that our goal in proposing a new meta-feature set was NOT to outperform TSFEL (nor do we claim this as a core contribution), but rather to (i) avoid TSFEL's engineering issues and (ii) demonstrate that TimeFuse can integrate with various meta-feature sets and still perform well.

More specifically, we propose the compact set alongside TSFEL for three main reasons:

1. Unexpected NaN values in TSFEL: We observed that TSFEL often outputs a large number of NaN values, e.g., 15,876 NaNs on the weather testset. Table 5 in the full results shows NaN counts across datasets. TSFEL’s documentation offers no clear explanation or fix. In our implementation, we impute NaNs with the feature-wise mean.
2. Lack of multivariate features in TSFEL: TSFEL computes features based on single variable. We believe that inter-variable relationships are also important. Therefore, we constructed a smaller meta-feature set that explicitly includes such information.
3. Demonstrating TimeFuse’s flexibility: We see TimeFuse’s ability to work with different meta-feature sets as an advantage: with a well-engineered implementation, TSFEL and other existing sets (e.g., catch22) can all be seemlessly intergrated into TimeFuse. Users can also design their own feature sets based on domain needs or interpretability considerations.

Q3 & Q4 & M2: Comparison with AutoGluon, stronger ensemble algorithms, and foundation time-series model.

Please see Table 1 in the full results for a comprehensive comparison with the suggested baselines. Due to space constraints, we kindly refer you to our response to Reviewer oty5 Q1 for detailed analysis.

In short, TimeFuse outperforms the ensemble baselines with its test-time adaptive ensembling capability. AutoGluon and Chronos are limited by their univariate-centric design and short pretrain predict length, leading to especially poor performance on long-term or multivariate forecasting tasks.

M3: Performance with raw features

Please see Table 6 in full results for the performance of using raw features. Using either ours or TSFEL feature sets consistently outperforms the result with raw features by a significant margin.

D1: Discuss on the evaluation datasets

We greatly appreciate you pointing out the related works on the limitations of long-term time series benchmarks. We have carefully read the literature and found them very insightful, relevant discussion will be included in the paper.

We want to thank you again for the thoughtful review. We’ve dedicated over 60 hours to get the new results and will include them to enhance the paper’s quality. We hope our responses address your concerns and are happy to discuss if you have any further questions!

Thank you for the thoughtful review and constructive feedback! With our best efforts, we conducted numerous additional analyses to address your concerns. Full results: https://anonymous.4open.science/api/repo/ICML25Reb-034C/file/Results.pdf?v=b00206ad. Please understand that due to the 5000-character limit, we can only summarize the most critical insights in the rebuttal :)

W1: Why not TSFEL

We propose a compact meta-feature set alongside TSFEL for three main reasons:

1. Unexpected NaN values in TSFEL: We observed that TSFEL often outputs a large number of NaN values, e.g., 15,876 NaNs on the weather testset. Table 5 in the full results shows NaN counts across datasets. These NaNs scattered across different samples and features. TSFEL’s documentation offers no clear explanation or fix. In our implementation, we impute NaNs with the feature-wise mean.
2. Lack of multivariate features in TSFEL: TSFEL computes features only on single variable. We believe that for multivariate time series, inter-variable relationships are important. Therefore, we constructed a smaller meta-feature set that explicitly includes such information.
3. Demonstrating TimeFuse’s flexibility: We see TimeFuse’s ability to work with different meta-feature sets as an advantage: with a well-engineered implementation, TSFEL and other existing sets (e.g., catch22) can all be seemlessly intergrated into TimeFuse. Users can also design their own feature sets based on domain needs or interpretability considerations.

W2: Training objective and other details

• Training objective: Explicitly computing optimal model weights for each sample requires solving a least squares problem for each sample, which is computationally expensive and unnecessary. In practice, our training objective in Eq (1) can directly optimizes the fusor via backpropagation.
• How to ensemble: Our goal is to learn the best weighted sum of base models. This has several advantages over selecting the best single model: (i) no need to break ties since we’re not picking just one model; (ii) more flexible integration: for example, if one model overestimates and another underestimates, their combination can outperform either model alone. In cases where all models over-/underestimate, our approach can fall back to single-model selection. In practice, such cases can be also mitigated by including more diverse models in the zoo.

We will carefully revise the paper to better clarify these details.

C1: Sample-level breakdown analysis

Great suggestion! Please see Table 3 in the full results, where we report the sample-level win rate and average rank for all 13 base models and TimeFuse, using the same settings as Fig 1.

Overall, TimeFuse achieves a 46.11% win rate with base models range 0.12%-9.89%. For average rank, TimeFuse scores 2.37, compared to 13.83–4.09 for base models, significantly outperforming any single model at the sample level. For cases where TimeFuse doesn’t outperform all base models, your point about “all models over/underestimating” is likely a cause. This might be addressed by adding a prefiltering step to select which models participate in the ensemble.

C2: Update Figure 4

Please see Figure 1 in the full results for an updated version, where we order the base models by performance. Generally, TimeFuse tends to assign higher weights to more accurate base models.

C3: Clarification on Figure 6

Thank you for the suggestion. We note that Fig. 6 shows the fusor weights jointly trained across all long-term forecasting datasets. Its goal is to reveal the relationship between model weights and input sample properties, so the characteristics of a single dataset should not affect the results in Fig. 6.

Q1: On large MTSF models

We believe ensemble methods can complement large MTSF models rather than compete with them. In fact, existing AutoML frameworks like AutoGluon already use pretrained MTSF models as base models in ensembles and achieve improved performance.

Additionally, current large MTSF models still have notable limitations: for example, we found that Chronos (a pretrained MTSF model) struggles with long-term or multivariate forecasting due to its univariate-centric design. Please kindly refer to our response to Reviewer oty5 Q1 for more details.

W0: Accuracy-time tradeoff

Since TimeFuse's fusor has a very simple architecture, its computational cost is minimal compared to the base model. As shown in the Table 2 of full results, the batch inference time of TimeFuse is nearly identical to that of the base model, indicating negligible overhead. We will include a detailed discussion in the paper.

We want to thank you again for the thoughtful review. We’ve dedicated over 60 hours to get the new results and will include them to enhance the paper’s quality. We hope our responses address your concerns and are happy to discuss if you have any further questions!