Masking the Gaps: An Imputation-Free Approach to Time Series Modeling with Missing Data

Comment 1: While the motivation is clear, the experimental results do not show significant improvement over the other baselines.

As stated in our global response, MissTSM demonstrates consistently competitive performance across various datasets and excels in challenging scenarios, including high masking fractions and extended horizons. It surpasses baselines on real-world datasets like PhysioNet, underscoring its practical utility. Furthermore, its imputation-free design eliminates computational overhead and avoids the propagation of imputation errors into forecasting errors.

Comment 2: Time series missingness is a well-studied and widely researched field, making it difficult to assess the significance of the contribution here.

There are two primary contributions of our work relative to other works in time-series modeling with missing values.

1. Conceptually, MissTSM is the first adaptation of the idea of Masked Autoencoders (MAE) for modeling multivariate time-series with missing values. While there exists other adaptations of MAE for time-series data such as [1, 2], previous adaptations expect both training and testing datasets to be clean without missing values. This makes them limited in their applicability to real-world applications with varying data irregularities. In contrast, our method provides the flexibility to handle missing values directly within the framework of MAE, bringing the power of representation learning and masked pre-training to the domain of time-series modeling.

2. Empirically, we show that MissTSM shows competitive performance consistently across an extensive set of experiments involving forecasting and classification datasets. This when coupled with the fact that our method is imputation-free, makes it an ideal choice for handling missing values in time-series forecasting problems, because (a) it does not suffer from the computational costs of training complex imputation models such as SAITS, and (b) it does not involve propagation of imputation errors to forecasting errors as is the case for other baselines, as described in Appendix C.

[1] Dong, Jiaxiang, Haixu Wu, Haoran Zhang, Li Zhang, Jianmin Wang, and Mingsheng Long. "SimMTM: A simple pre-training framework for masked time-series modeling." Advances in Neural Information Processing Systems 36 (NeurIPS) 2023.
[2] Li, Zhe, Zhongwen Rao, Lujia Pan, Pengyun Wang, and Zenglin Xu. "Ti-MAE: Self-supervised masked time series autoencoders." arXiv preprint arXiv:2301.08871 (2023).

Comment 3: This work primarily represents an empirical study rather than a theoretical advancement.

We agree that our work primarily represents an empirical study rather than making theoretical contributions. However, empirical evaluations play a crucial role in advancing the field of time-series modeling, as reflected in several recent influential works focused on empirical contributions such as SimMTM [1] and iTransformer [2], published at NeurIPS 2023 and ICLR 2024, respectively. Our work contributes by systematically exploring the challenges of missing values in multivariate time-series data and proposing a novel approach to address these challenges that is the first adaptation of MAEs for handling missing values in time-series problems.

[1] Dong, Jiaxiang, Haixu Wu, Haoran Zhang, Li Zhang, Jianmin Wang, and Mingsheng Long. "SimMTM: A simple pre-training framework for masked time-series modeling." Advances in Neural Information Processing Systems 36 (NeurIPS) 2023
[2] Liu, Yong, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. "iTransformer: Inverted transformers are effective for time series forecasting". International Conference on Learning Representations (ICLR) 2024

Comment 1: The introduction is not well-written, which seems more like the related work. It is suggested to further polish the manuscript, to improve the readability.

We apologize if our Introduction section was hard to read. We will reduce the discussions of related works and move them to the next section to improve the readability of this section.

Comment 2: There already exists single-stage frameworks such as RobustTSF that perform time series modeling with low quality data, including missing values and anomalies. Therefore, the motivation is unpersuasive, claiming that all the existing works are two-stage approaches. RobustTSF should be included in the experiments.

Thank you for pointing us to the impactful work of RobustTSF, which we are happy to include in our related works discussion. There are four key differences between RobustTSF and MissTSM.

1. RobustTSF is a filtering approach to ignore samples with anomalies or missing values (treated as anomalies after being imputed with zero values) during training to ensure robustness. In contrast, MissTSM focuses on representation learning to learn latent representations of observed variates at every time-point, without dropping time-points with missing values. MissTSM thus leverages all of the available information of observed features across time while only ignoring missing time-feature pairs.

2. RobustTSF makes several simplifying assumptions to identify anomalies, including trend filtering based on smoothness assumptions and anomaly score computation based on Dirac weighting functions, which may not be held in real-world applications. These assumptions additionally require user-input parameters such as anomaly score threshold. In contrast, MissTSM makes no such modeling assumption and completely relies on the supervision contained in observed data.
3. While RobustTSF is designed to handle missing values during training, it assumes that there are no missing values during testing. As a result, RobustTSF is unable to handle missing values in the inputs during testing beyond simply imputing it with zero values. In contrast, MissTSM can handle missing values both during training and testing.
4. RobustTSF is formulated for univariate time-series problems. Even though it is possible to apply RobustTSF in multivariate settings, it treats each variate independently of other variates and does not take into account interactions among variates. In contrast, MissTSM is designed for multivariate time-series modeling.

To compare RobustTSF with MissTSM, we consider a multivariate forecasting experiment over the ETTh2 dataset across 60% to 90% MCAR missing data fractions. We use the Autoformer model as the time-series model integrated with RobustTSF | Fraction | Horizon | RobustTSF + Autoformer | MissTSM | |----------|---------|--------------------------|------------------| | 60% | 96 | 1.5 ± 0.054 | 0.24 ± 0.007 | | | 192 | 1.45 ± 0.065 | 0.26 ± 0.002 | | | 336 | 1.34 ± 0.082 | 0.28 ± 0.009 | | | 720 | 1.33 ± 0.024 | 0.33 ± 0.012 | | 70% | 96 | 1.81 ± 0.117 | 0.25 ± 0.009 | | | 192 | 1.76 ± 0.056 | 0.27 ± 0.013 | | | 336 | 1.76 ± 0.049 | 0.30 ± 0.008 | | | 720 | 1.69 ± 0.032 | 0.35 ± 0.01 | | 80% | 96 | 2.2 ± 0.084 | 0.26 ± 0.025 | | | 192 | 2.23 ± 0.091 | 0.28 ± 0.021 | | | 336 | 2.2 ± 0.085 | 0.29 ± 0.029 | | | 720 | 2.11 ± 0.041 | 0.37 ± 0.056 | | 90% | 96 | 2.72 ± 0.077 | 0.32 ± 0.029 | | | 192 | 2.74 ± 0.036 | 0.35 ± 0.072 | | | 336 | 2.77 ± 0.024 | 0.32 ± 0.014 | | | 720 | 2.67 ± 0.034 | 0.35 ± 0.045 |

We observe that MissTSM clearly outperforms RobustTSF + Autoformer across all the missing fractions and prediction horizon, thus highlighting the impact of our contribution in relation to this prior work.

Since ``MissTSM can be easily extended to handle anomalies by coupling it with off-the-shelf algorithms'', can you show the specific solution with the experimental results, compared with RobustTSF? In addition, why only high missing rates 60%-90% are considered? How about the experimental results for lower missing rates?

Comment 1: The authors should consider a broader range of baseline models, including methods that incorporate missingness indicators such as GRU-D, or methods that handle irregular time intervals using ODE/SDE-based approaches such as Neural ODE, Latent ODE, and Neural Stochastic Differential Games for Time-series Analysis

We thank the reviewer for suggesting these prior works [1,2,3,4] that we are happy to include in our discussion of related works. While prior works such as GRU-D [4] embed time intervals between observations as auxiliary features to handle irregular time-sequences, our approach aims to bypass this step and learn representations based on the observed data along both time and feature dimensions. Also, while ODE-based prior works [1,2] offer an elegant framework for modeling irregularly sampled data in continuous time, they rely on solving differential equations that can be computationally demanding and difficult to scale, as experienced in our experiment runs. To include these prior works in our comparisons of results, we have performed new experiments with GRU-D and Latent ODE for both classification and forecasting tasks, as summarized below.

Comparison of MissTSM with GRU-D on Classification tasks: We use the available GRU-D re-implementation (as the official code is not available currently) and adapt it for the classification task. The model was run for 100 epochs. We consider an MCAR setup with a missing fraction of 80%. Datasets considered are Epilepsy, Gesture, and EMG. | Dataset | GRU-D (F1) | MissTSM (F1) |
|-----------|------------|--------------|
| Epilepsy | 6.52% | 64.9% |
| Gesture | 3.16% | 55.70% |
| EMG | 2.78% | 59.45% |

Comparison of MissTSM with LatentODE on ETTh2: We consider Latent ODE with ODE-RNN as the encoder. Since ODE-based methods are very computationally expensive, we consider a simple setup of 336 context length and 96 prediction length. For simplicity, we only consider MCAR masking with varying masking fractions. | Fraction | Latent ODE (MSE) | MissTSM (MSE) |
|----------|------------------------|--------------------|
| 60% | 4.25 | 0.243 |
| 70% | 3.181 | 0.250 |
| 80% | 2.543 | 0.264 |
| 90% | 2.624 | 0.316 |

We can see from both these Tables that MissTSM performs significantly better than GRU-D and Latent ODE, highlighting the impact of our contribution in relation to prior works in time-series modeling with missing values.

[1] Chen, Ricky TQ, Yulia Rubanova, Jesse Bettencourt, and David K. Duvenaud. "Neural ordinary differential equations." Advances in neural information processing systems 31 (2018)
[2] Rubanova, Yulia, Ricky TQ Chen, and David K. Duvenaud. "Latent ordinary differential equations for irregularly-sampled time series." Advances in neural information processing systems 32 (2019)
[3] Park, Sungwoo, Byoungwoo Park, Moontae Lee, and Changhee Lee. "Neural stochastic differential games for time-series analysis." International Conference on Machine Learning (2023)
[4] Che, Zhengping, Sanjay Purushotham, Kyunghyun Cho, David Sontag, and Yan Liu. "Recurrent neural networks for multivariate time series with missing values." Scientific reports 8, no. 1 (2018): 6085.

I appreciate the authors taking the time to address my concerns in their rebuttal. However, I still have reservations about the technical novelty of this work, particularly in light of the new experiments comparing it to LoCF. While the finding that MissTSM performs better only over longer time horizons is intriguing, I believe a more thorough discussion of its advantages and disadvantages compared to simpler imputation methods is necessary.

Additionally, I'm unfamiliar with the concept of biasing attention scores using mask indicators in BERT, as mentioned by the authors. To my knowledge, BERT utilizes [mask] tokens, and I'm not aware of any mechanism for directly biasing attention scores in this way. (I would be grateful if the authors could clarify this point and correct me if I'm mistaken.) Furthermore, I think the authors need to explain the reasoning behind this biasing approach more clearly. The impact of such a bias could vary significantly based on hyperparameter choices, and it seems like those choices would be heavily dependent on the specific temporal dynamics of the data. This aspect requires further elaboration.

While I appreciate the authors' efforts, I don't believe this paper currently meets the high standards of ICLR. Therefore, I will maintain my original score.

Comment 2: I believe a more thorough discussion of its advantages and disadvantages compared to simpler imputation methods is necessary.

Thank you for this suggestion. Here is a summary of the practical takeaways of using MissTSM compared to SOTA baselines with simpler imputation techniques:

1. MissTSM shows consistently competitive performance across most dataset variations with varying synthetic masking schemes and masking fractions. MissTSM is especially effective in more challenging scenarios with larger masking fractions and longer horizon windows resembling real-world settings (e.g., PhysioNet).

2. MissTSM is especially useful in resource-constrained applications where computational costs need to be kept low, since MissTSM is imputation-free and does not suffer from the complexity-accuracy tradeoffs of SOTA imputation techniques. While simpler imputation methods may have low computational overhead, they suffer from the complexity-accuracy trade-off, as seen from the performance on PhysioNet.
3. MissTSM does not currently beat simple imputation-based baselines in simpler scenarios like shorter horizon windows in terms of prediction performance (as seen on the ETTm2 dataset above). While simple imputations may appear more effective in such scenarios, we would like to emphasize that the performance gap is relatively small compared to the gain in improvement achieved over more complex scenarios. Moreover, MissTSM represents the first work in a new direction of methods for time-series modeling that are fast, imputation-free, and modeling assumption-free. We anticipate MissTSM to inspire future directions in the same conceptual framework as masked auto-encoders to address its current limitations, e.g., by extending MissTSM to jointly learn cross-channel correlations along with non-linear temporal dynamics.

Comment 3: I'm unfamiliar with the concept of biasing attention scores using mask indicators in BERT, as mentioned by the authors. To my knowledge, BERT utilizes [mask] tokens, and I'm not aware of any mechanism for directly biasing attention scores in this way. (I would be grateful if the authors could clarify this point and correct me if I'm mistaken.) Furthermore, I think the authors need to explain the reasoning behind this biasing approach more clearly. The impact of such a bias could vary significantly based on hyperparameter choices, and it seems like those choices would be heavily dependent on the specific temporal dynamics of the data. This aspect requires further elaboration

Apologies for the misunderstanding. The concept of masking in attention mechanisms was first introduced in the "Attention Is All You Need" paper [1], specifically for causal masking in auto-regressive tasks. The authors state: "We need to prevent leftward information flow in the decoder to preserve the auto-regressive property. We implement this inside of scaled dot-product attention by masking out (setting to −∞) all values in the input of the softmax which correspond to illegal connections." While this paper introduced masking for causal dependencies, the concept has since been extended to handle masked tokens in various contexts, such as masked language modeling in BERT [2]. Here is a more elaborate explanation of the masking approach.

$\text{Softmax}(\frac{Q.K^T}{\sqrt{d}} + \eta M)$

Here, $\eta$ is a large negative value tending to −∞ (implemented as float(“-inf”) in PyTorch). M is a hard binary mask with values either 0 (for unmasked positions) or 1 (for masked positions). The $\eta M$ thus converts the attention scores for the masked positions (M = 1) into $\eta$, while leaving the unmasked scores unchanged. Applying the softmax ensures $e^\eta = 0$, thus effectively removing the masked positions from the attention scores and preserving the relative probabilities of the unmasked tokens. Note that the bias term is not a hyper-parameter that requires to be fine-tuned, rather is fixed to a constant value of float(“-inf”).

An alternative to the above formulation could be applying M as a hard-constraint post-softmax as follows,

$\text{Softmax}(\frac{Q.K^T}{\sqrt{d}}) \cdot (1 - M)$

Here, we first compute the normalized attention scores and then apply the mask by multiplying 1 – M to zero-out the masked positions. A drawback of this method would be that it would break the sum of probability = 1 property across the final attention scores, leading to an invalid distribution.

We are happy to clarify if there are any further questions about our formulation.

[1] Vaswani, A. "Attention is all you need." Advances in Neural Information Processing Systems (2017).
[2] Devlin, Jacob, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding." arXiv preprint arXiv:1810.04805 (2019)

Thank you for the further intriguing questions. Here are our responses to these questions.

Comment 1: While it's true that imputation methods prioritize interpolation over predictive power, the significant performance loss with SAITS remains concerning. The reported values deviate considerably from simple baselines, which raises questions about whether SAITS was properly trained with well-chosen hyperparameters.

For our experiments, we used the same default hyperparameters as specified in the original implementation of SAITS. We would like to emphasize that the poor performance of SAITS is a reflection on SAITS’s usability on ETTm2 and not our proposed framework (MissTSM does not use SAITS or any other imputation method inside its framework).

Comment 2: The authors haven't adequately explained why the proposed method excels only at longer time horizons. This behavior likely depends on the underlying temporal dynamics of the dataset. A more in-depth and systematic analysis, potentially incorporating varying temporal characteristics, is crucial to fully understand the advantages of the proposed method

Thank you for this suggestion. To explain why MissTSM excels at longer time horizons, note that missing values tend to have more catastrophic effects on predictions made far out into the future, because of the compounding effects of missing information across time. Additionally, imputation methods primarily focus on capturing local trends to fill missing values and hence are less effective at longer horizons. Please note that the strengths of MissTSM are not limited to longer forecasting horizons alone; we find that MissTSM is useful in other challenging scenarios too. For example, on classification datasets, where the notion of forecasting horizons does not apply, our approach demonstrates competitive performance especially with large masking fractions. The real-world applicability of MissTSM is reflected in the experiments on the PhysioNet dataset, where we see a significant jump in MissTSM’s performance compared to imputation baselines.

Regarding the depth of our experimental analyses, we would like to note that we have conducted experiments over 183 variations of benchmark datasets with varying patterns of missing values and missing value fractions. This scale of experiments is unprecedented for any previous work in time-series modeling that handles missing values. Hence, while we agree that there is potential to further expand our analyses to more datasets, we believe that our current results comprehensively show the usefulness of MissTSM over a wide range of dataset variations. We will be happy to include additional experiments on any other time-series dataset that we have not included so far in the discussion period.

Comment 3: While I was familiar with BERT and masking strategies in general, I hadn't previously encountered the use of mask indicators (with −∞) within the softmax function for attention score calculation as introduced in this paper. A more intuitive approach might involve a hard constraint (as mentioned in the rebuttal) that only includes non-masked elements in the softmax calculation. This would ensure the attention scores sum to one, similar to the approach used in causal self-attention for decoder structures. This will provide a clearer explanation of how unmeasured observations are excluded during attention

Thank you for suggesting this alternate formulation. Based on our understanding of the reviewer’s comment, this alternate formulation that includes hard constraints on masked tokens but inside the softmax computation can be expressed as:
$\text{Softmax} (\frac{Q.K^T}{\sqrt{d}} \cdot (1 - M))$,
where the inputs to the softmax are set to 0 if the mask M is 1. While this would ensure that the outputs sum to one, a limitation of this framework would be that the masked tokens would receive non-zero attention scores. In particular, softmax would transform the zero inputs of masked tokens to output $\frac{1}{N + \sum e^{U_i}}$, where $U_i$ represents unmasked tokens and $N$ is the number of masked tokens. Additionally, the outputs of the unmasked tokens would also be influenced by the number of masked tokens $N$ as follows: $\frac{e^{U_i}}{N + \sum e^{U_j}}$. To avoid these limitations, we set the inputs to the softmax to $–\infty$ if the mask M is 1. Note that this is an implementation detail that is often not fully described in the main paper but is included in code implementations of seminal works in Transformers, e.g., attention implementation of transformer model in PyTorch - https://pytorch.org/docs/stable/generated/torch.nn.functional.scaled_dot_product_attention.html#torch.nn.functional.scaled_dot_product_attention, attention layer in BERT - lines 705-716 in https://github.com/google-research/bert/blob/master/modeling.py. Our formulation of masked attention using $-\infty$ is thus not a novel contribution but a standard practice in the community

Comment 1: Three datasets for each task are limited and not enough. Why both ETTh2 and ETTm2 are selected from ETT datasets for experiments? Generally, datasets from different application domains are more reasonable.

We chose ETT datasets because they are commonly used in most previous works in time-series forecasting, such as [1, 2, 3]. While we agree that using three datasets for forecasting and three for classification appear small, please note that for every dataset, we have considered extensive variations of the same dataset (40x for forecasting and 20x for classification) to study the effects of missing values on time-series models (namely, with different types of synthetic masking schemes, varying masking fractions, and multiple random runs). This is a unique feature of our experiments that differentiates us from prior works in the literature that only consider one setting of every dataset. Since every variation of a benchmark dataset can be viewed as a different sample dataset for the purpose of evaluation, we are able to add sufficient diversity in the nature and scale of dataset variations (183 in total) considered in our experiments.

To further increase the diversity of datasets used for forecasting, we have added results on a new dataset – Solar-Energy [4] with MCAR masking scheme, summarized in the Table below. Here, we compare the MSE performance of MissTSM with iTransformer coupled with the SAITS-imputation method. The standard deviations are reported based on 3 runs for iTransformer. | Masking (%) | Horizon | iTransformer | MissTSM | |-------------|---------|--------------------------|---------------| | 60% | 96 | 0.213 ± 0.02 | 0.222 | | | 192 | 0.234 ± 0.03 | 0.253 | | | 336 | 0.237 ± 0.03 | 0.237 | | | 720 | 0.233 ± 0.08 | 0.236 | | 70% | 96 | 0.232 ± 0.03 | 0.217 | | | 192 | 0.255 ± 0.04 | 0.218 | | | 336 | 0.268 ± 0.04 | 0.222 | | | 720 | 0.254 ± 0.04 | 0.252 | | 80% | 96 | 0.215 ± 0.07 | 0.251 | | | 192 | 0.237 ± 0.03 | 0.289 | | | 336 | 0.248 ± 0.02 | 0.265 | | | 720 | 0.240 ± 0.07 | 0.267 | | 90% | 96 | 0.264 ± 0.03 | 0.290 | | | 192 | 0.298 ± 0.08 | 0.370 | | | 336 | 0.305 ± 0.02 | 0.319 | | | 720 | 0.296 ± 0.01 | 0.321 |

We observe that MissTSM shows competitive performance as iTransformer across all masking fractions of Solar, similar to the trends observed on other forecasting datasets. This when coupled with the fact that our method is imputation-free makes it an ideal choice for handling missing values in time-series forecasting problems, because (a) it does not suffer from the computational costs of training complex imputation models such as SAITS, and (b) it does not suffer from the propagation of imputation errors to forecasting errors as is the case for other baselines, as described in Appendix C.

[1] Wu, Haixu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. "Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting." Advances in Neural Information Processing Systems 34 (NeurIPS) 2021 [2] Liu, Yong, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. "iTransformer: Inverted transformers are effective for time series forecasting.” International Conference on Learning Representations, (ICLR) 2024
[3] Dong, Jiaxiang, Haixu Wu, Haoran Zhang, Li Zhang, Jianmin Wang, and Mingsheng Long. "Simmtm: A simple pre-training framework for masked time-series modeling." Advances in Neural Information Processing Systems 36 (NeurIPS) 2023
[4] Lai, Guokun, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. "Modeling long-and short-term temporal patterns with deep neural networks." In The 41st international ACM SIGIR conference on research & development in information retrieval, pp. 95-104. 2018.

We sincerely thank the reviewers for their insightful comments and the constructive feedback, which has been immensely helpful in improving the quality of our work. As the discussion period draws to a close, we invite you to share any remaining questions or requests for clarifications. We take this opportunity to re-emphasize the main contributions of our work, and summarize the main concerns raised by the reviewers and our responses to those concerns below.

Main contributions:

(1) We introduce a time-series modeling framework that can handle missing values directly within the framework of Masked Autoencoders (MAE), thus bringing in the power of representation learning and masked pre-training to the domain of time-series modeling.

(2) Our proposed framework is simple, fast, lightweight and scalable compared to the complex two-stage imputation-based approaches and existing time-series models for handling missing values, making it an effective choice for practitioners especially in resource-constrained settings.

(3) We show that our proposed framework consistently achieves competitive performance as the SOTA models under diverse synthetically generated missing value scenarios on benchmark datasets for forecasting and classification, especially for larger masking fractions and longer horizon windows. Our framework also demonstrates superior performance on a real-world dataset for classification.

Main reviewer concerns and responses

Concern 1: The work presents a limited perspective on time-series methods that avoid imputation. More baselines need to be included in the experiments (UWVS, G7Mn)

Response: We added new experiments comparing our work with three more baselines as suggested by reviewers.

• We compared MissTSM with GRU-D on three classification datasets (see our response to Reviewer UWVS)
• We compared MissTSM with LatentODE on the ETTh2 forecasting dataset across 4 different missing value fractions (see our response to Reviewer UWVS)
• We compared MissTSM with RobustTSF on the ETTh2 forecasting dataset, across 4 different missing value fractions (see our response to Reviewer G7Mn)

In all the experiments, we see that MissTSM outperforms baselines by significant margins, further bolstering the contribution of this work in relation to previous works.

Concern 2: Improvement in MissTSM’s performance appears marginal (UWVS, 7Jfe)

Response: As discussed in our global response, MissTSM consistently shows competitive performance across diverse datasets and shows significant performance gains in challenging scenarios such as high masking fractions and large horizon windows. MissTSM also significantly outperforms baselines on a real-world dataset, Physionet, showcasing its practical applicability. Moreover, the imputation-free nature of our model prevents the computational overhead and propagation of imputation errors to forecasting errors.

Concern 3: Lack of variety in datasets (sZVS)

Response: While our study was limited to only 6 datasets, we have conducted experiments with extensive variations of each dataset (40x for forecasting and 20x for classification), contrary to prior works in the literature. Overall, we have conducted experiments on over 183 variations of benchmark datasets with varying patterns of missing values and missing value fractions. This scale of experiments is unprecedented for any previous work in time-series modeling that handles missing values. To further increase the diversity of datasets, we added results on a new forecasting dataset, Solar-Energy (see our response to Reviewer sZVS) and observe that MissTSM shows similar trends in performance as observed on other forecasting datasets.

Concern 4: The experiments consider only high missing rates 60-90% (G7Mn, 7Jfe)

Response: We considered moderate to extreme ranges of missing value fractions in our experiments to closely resemble real-world scenarios where the missing value levels are very high. To show results on smaller missing value fractions, we conducted additional experiments on ETTh2 dataset across 10-40% missing data fractions (see our response to Reviewer G7Mn). Through these results, we see that while MissTSM is very effective under challenging conditions it also gives good performance under simpler scenarios with low missing data.