MIRA: Medical Time Series Foundation Model for Real-World Health Data

We sincerely thank the reviewer for their thoughtful and constructive feedback. We are particularly grateful for your recognition of MIRA’s architectural innovations. In response to your valuable comments, we have taken the following steps to further strengthen our submission:

1. Provided detailed reasoning for each architectural component, clarifying why CT-RoPE, temporal MoE, and CD-Extrapolation lead to performance gains through ablation comparisons and theoretical justifications.
2. Offered deeper qualitative insights to illustrate how MIRA captures clinically meaningful temporal dynamics beyond what baseline models can achieve, especially under irregular sampling conditions.
3. Quantified MIRA’s computational efficiency.

We hope these additions clarify the rationale behind MIRA’s design and its practical utility. Please find our detailed responses below.

Q1: Lacks a critical analysis explaining why the proposed components yield the observed performance gains.

A: We provide comprehensive ablation studies (Table 5) demonstrating each component's contribution. Here we explain why each component yields performance gains and what alternatives were used in ablation studies:

CT-RoPE vs. Standard Positional Encoding: In our ablation, we replaced CT-RoPE with standard learned positional embeddings. Standard approaches treat irregular timestamps as discrete indices, losing crucial temporal distance information. For example, a 5-minute gap and 2-seconds gap between measurements would be treated identically. CT-RoPE's continuous encoding (θᵢ(t) = ωᵢ · t) preserves actual temporal relationships, enabling the attention mechanism to appropriately weight recent vs. distant observations based on real time differences rather than sequence positions.

MoE vs. Standard FFN: Medical signals exhibit vastly different temporal scales (e.g. ECG operates at milliseconds while lab values change over hours). A single FFN must compromise between these scales, diluting its effectiveness. MoE routing allows specialized experts: some optimized for high-frequency cardiac rhythms, others for slow-varying metabolic trends. This specialization prevents interference between temporal regimes.

CD-Extrapolation vs. Standard Autoregressive: Traditional approaches can only predict at the next sequence position (i.e. next-token prediction), limiting clinical utility where prediction times are irregular (e.g., predicting patient status at discharge time). Our Neural ODE approach evolves hidden states continuously, enabling predictions at clinically relevant timestamps rather than fixed intervals.

Q2: Beyond metric gains, can you provide qualitative examples or deeper analysis illustrating why MIRA succeeds where baselines fail?

A: Beyond metrics, our results reveal that MIRA learns fundamentally different representations from baselines:

MIRA learns "physiological continuity", understanding that biological processes evolve continuously even when measurements are discrete. Through expert activation patterns (Appendix L), we observe frequency-specific specialization: high-frequency datasets (MIT-BIH) activate different experts than low-frequency data (CDC-IHA), suggesting learned temporal hierarchy decomposition.

As a example, consider a cardiac patient with arrhythmia where we need to predict the next ECG values:

Input sequence (with missing values): [0.8, –, –, 1.2, –, –, –, 0.4, –] at timestamps [0, 1, 2, 3, 4, 5, 6, 7, 8]

Target prediction: Next 3 values should be [–, 1.0, –] at timestamps [9, 10, 11]

Baseline approach:

• First interpolates: [0.8, 0.9, 1.0, 1.2, 1.1, 0.9, 0.7, 0.4, 0.5]
• Then predicts: [0.6, 0.7, 0.8] (smooth progression, missing sharp jump)

MIRA prediction:

• CT-RoPE encodes irregular timestamps [0, 3, 7] directly, capturing temporal relationships without interpolation
• Neural ODE models continuous dynamics: dh(s)/ds = f(s-t₇, h(s); θ), where h(t₇) = h₀.₄ evolves from timestamp 7 to 10
• The ODE solver integrates: h(t₁₀) = h₀.₄ + ∫₇¹⁰ f(s-7, h(s); θ)ds, enabling extrapolation to arbitrary timestamps
• Predicts: [–, 0.98, –] (captures physiological jump pattern, 0.98≈1.0)

The baseline’s interpolation smooths out abrupt voltage drops, masking key signs of arrhythmia (Please also refer to Reviewer vk9d’s Q2 for different type of missing data’s imapct). In contrast, MIRA's continuous-time modeling preserves the underlying arrhythmic dynamics through its ODE formulation, enabling more accurate and clinically meaningful forecasting under irregular sampling

Q3: Please comment on training resources and inference efficiency compared to the baselines.

A: We appreciate this important question about computational efficiency. We provide detailed analysis in our response to Reviewer Lv9n Q1, and summarize the key findings here: MIRA achieves 2× faster inference than TimesFM with 2.5× fewer activated parameters (~200M vs 500M), demonstrating superior parameter efficiency through sparse MoE activation. While ContiFormer uses significantly less memory (1072.1MB vs 2575.65MB), its iterative extrapolation approach significantly increases inference time for multi-step forecasting.

We thank the reviewer for their insightful comments and for highlighting the clear motivation behind our work. In response to your insightful questions, we have made the following additions and clarifications:

1. Quantified potential dataset imbalance and domain dominance by analyzing cross-dataset attribution using influence score analysis.
2. Explain the standardization of time alignment, by outlining our model architecture and modelling mechanism.
3. Evaluated MIRA under realistic block-missingness patterns
4. Report context window configurations
5. Added statistical significance testing to validate the contributions of each architectural component.
6. Discuss model performance and interpretability with domain-specific models.

We appreciate your thoughtful feedback and hope our responses address your concerns clearly. Please see below for point-by-point replies.

Q1.1: How did you balance differences in class distributions and sampling rates across ICU, sleep, and pediatric datasets? Are certain domain-specific signals disproportionately influencing the learned representations?

A: Rather than manually resampling or reweighting datasets to correct for size or sampling-rate imbalance, we rely on the scale and heterogeneity of our corpus and the MoE architecture to learn domain-robust representations.

To quantify whether certain datasets dominate the learning process, we conducted an influence score analysis using the data attribution techniques, i.e. gradient dot product (Grad-Dot) method [1,2]. This method measures how much each training sample contributes to the test loss gradient, allowing us to estimate the relative importance of different training sources.

Results across five target test datasets (see table below) show that while MIMIC-III has the largest share, its average influence scores are not disproportionately dominant. In fact, other datasets also show consistent, non-negligible influence, indicating that MIRA internally balances its attention across domains rather than overfitting to dataset size.

Q1.2: How is time alignment standardized given the heterogeneity of these datasets?

A: Similar to other foundation models like Chronos [3] , which implicitly handles this by quantizing arbitrary-length sequences in a fixed number of bins, our method handles the heterogeneous sampling frequencies through a unified token-by-token input approach, where our MoE architecture automatically learns to process different temporal scales without explicit frequency standardization.

Each data point becomes a token with its associated timestamp, regardless of original sampling rate. Our time quantization procedure (Appendix A) standardizes absolute timestamps into relative time intervals while preserving temporal ordering. This enables CT-RoPE to directly encode continuous-time relationships across variable intervals. In addition, the MoE routing mechanism learns to specialize different experts for distinct temporal regimes. As evidenced in Appendix L Figure 3 (right), high-frequency datasets (MIT-BIH) activate different expert combinations compared to low-frequency datasets (COVID-19), demonstrating automatic adaptation to temporal characteristics.

Q2:Have you tested MIRA’s robustness to block‐missingness patterns, and how might such structured gaps impact its performance compared to your random‐masking experiments?

We evaluated MIRA under both random (scattered) and structured (block) missingness at 30% and 50% masking rates at illness dataset. As shown below, MIRA performs robustly across both settings, consistently outperforming Moirai. Specifically, MIRA shows minimal performance degradation between the two settings. In contrast, Moirai exhibits more pronounced deterioration under block missingness, which shows that block-missingness removes contiguous segments of input, making interpolation harder due to the lack of nearby temporal anchors, showing the meaning of developing an irregular TS foundation model.

Q3.1: What is the size of the context window used in your experiments?

A: During training, we used a context window of 512 time steps. This length balances computational efficiency with our need to process parallel value and timestamp sequences, and proved sufficient to capture the temporal dependencies in our tasks.

When it comes to inference, we follow each benchmark’s established protocol. For example, Illness dataset are {24, 36, 48, and 60} [4]. We’ll include a summary table of all testset context window sizes in the revised manuscript for full clarity.

Q3.2:How does the model account for underlying variability (e.g., children vs. elderly), particularly in the absence of demographic features?

A: Our model is designed to learn generalizable temporal patterns that can transfer across demographic groups without requiring explicit demographic features. The model implicitly learns to capture demographic-related temporal dynamics directly from the signals themselves. For instance, pediatric patients typically exhibit higher heart rates and different baseline ranges [5], which are naturally encoded in the time series data. Our MoE architecture enables the model to route different types of physiological patterns to specialized experts, potentially allowing adaptation to demographic-specific characteristics without explicit labels. Our model's performance well on pediatric data (WAVES) despite representing only 1% of training data, demonstrates that the model successfully generalizes across age groups and demographic variations.

Q4: Can you add confidence intervals or p-values for the MAE/RMSE gains in Table 5?

A: We conducted paired t-tests based on results from five independent runs for each ablation variant. The results show that all ablated variants exhibit statistically significant degradation compared to the full model, with p-values of 0.025 (w/o CT-RoPE), 0.022 (w/o MoE Block), and 0.004 (w/o Extrapolation Block), respectively (p < 0.05 in all cases). We will add this in next revision.

Q5: How does its performance compare to specialized, domain-specific models? Does MIRA trade off any accuracy, efficiency, or interpretability compared to domain-specific models?

A: Thank you for this insightful question. While there are limited specialized models specifically designed for irregular medical time series forecasting, we provide comprehensive comparisons against irregular time series models (ContiFormer, T-PatchGNN, Neural-CDE, ODE-RNN) trained in a full-shot manner on specific medical datasets (Table 2).

Accuracy: MIRA may achieve slightly lower accuracy than full-shot specialized models on some datasets, but significantly outperforms zero/few-shot alternatives while providing consistent cross-domain performance without requiring domain-specific training or fine-tuning. This competitive performance is achieved through our CT-RoPE and Neural ODE-based extrapolation block, which enable effective modeling of irregular temporal patterns across diverse medical domains.

Efficiency: While MIRA incurs higher upfront pretraining costs compared to individual domain-specific models, it provides substantially higher deployment efficiency in zero/few-shot scenarios (please refer to Reviewer LN9n's Q1). Once pretrained, MIRA applies immediately to new tasks without retraining. Additionally, during inference, our model only activates approximately half of the model parameters, significantly reducing computational overhead while maintaining performance. This efficiency gain is enabled by our frequency-specific mixture-of-experts architecture that learns generalizable temporal specializations during pretraining.

Interpretability: MIRA's MoE architecture provides superior cross-domain interpretability through expert activation patterns (Figure 3), offering insights into temporal frequency specialization that narrow specialized models cannot provide.

[1] Charpiat, G., Girard, N., Felardos, L., et al. (2019). "Input similarity from the neural network perspective," Advances in Neural Information Processing Systems, 32.

[2] Wei, D., Padhi, I., Ghosh, S., et al. (2024). "Final-Model-Only Data Attribution with a Unifying View of Gradient-Based Methods," arXiv preprint arXiv:2412.03906.

[3] Ansari, Abdul Fatir, et al. "Chronos: Learning the language of time series." arXiv preprint arXiv:2403.07815 (2024).

[4] TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis

[5] Fleming, Susannah, et al. "Normal ranges of heart rate and respiratory rate in children from birth to 18 years of age: a systematic review of observational studies." The Lancet 377.9770 (2011): 1011-1018.

We sincerely thank the reviewer for their thoughtful and constructive feedback on our submission. We are especially appreciative of your recognition of the motivation and technical contributions of MIRA in addressing the challenges of medical time series. In response to your suggestions, we have made the following updates:

1. Quantified MIRA’s computational efficiency.
2. Explore the potential for adapting MIRA to classification tasks, demonstrating its broader applicability beyond forecasting.
3. Investigated potential dataset imbalance and bias, using influence score analysis and analysis of model menchinism.
4. Demonstrated MIRA’s sample-efficient domain adaptation, by comparing few-shot fine-tuning with from-scratch training on an out-of-distribution dataset.

We hope these additions clarify the contributions and practical value of our work. Please see below for our point-by-point responses.

Q1: Could you provide a detailed analysis of MIRA’s computational footprint, including inference latency (seconds per forecast window) and memory requirements, directly compared to TimesFM and ContiFormer?

A: We measured average inference latency and GPU memory consumption over 10 batches (batch size = 32) with history length 512 and prediction length 96:

MIRA achieves 2× faster inference than TimesFM (500m) despite comparable model complexity, demonstrating superior parameter efficiency through sparse MoE activation. Although ContiFormer's memory usage is smaller, it requires iterative extrapolation for multi-step forecasting, significantly increasing inference time compared to the other two models.

Q2: How can MIRA’s architecture or learned embeddings be adapted to non-forecasting clinical tasks (e.g., mortality prediction on the CinC 2012 dataset or patient phenotyping), and can you share any preliminary downstream results?

A: We use HeartBeat (contains ECG-based labels for normal and abnormal rhythms) from CinC 2016 as classification dataset to evaluate whether our proposed model could be adopt to other task format. We keep the same data split and metrics reported in Time Series Library (TSLib) and baselines[1,2]. We extract the final hidden states from MIRA as sequence-level representations and pass them to a lightweight classifier (e.g., CNN variant) for training.

Our results show that MIRA achieves competitive accuracy compared to specialized models. These findings suggest that MIRA’s learned embeddings retain clinically useful information and can be effectively adapted for non-forecasting tasks. We will extend this line of work to additional downstream tasks and datasets.

Q3.1: Could MIMIC-III’s 88% share in the pre-training corpus introduce demographic or institutional biases into MIRA’s representations?

A: Thank you for this insightful question. While MIMIC-III play a major role in pre-training corpus, we believe the risk of demographic or institutional bias in MIRA is minimal, for the following reasons:

First, MIRA operates solely on physiological time series (e.g., vitals, labs, ECG) and is not provided with age, sex or hospital information. This prevents the model from conditioning on potential sources of bias and encourages learning representations based only on universal temporal patterns present in the data.

Second, MIRA aim to autonomously learn latent temporal patterns from physiological signals, rather than memorizing dataset-specific artifacts. By design, the model encourages generalization across diverse domains, and the MoE architecture facilitates this by dynamically routing inputs based on their signal properties, not dataset identity. This enables MIRA to develop domain-robust representations that transfer well beyond any single data source.

Our empirical results support this: For example, table 2 shows that the performance is still comparable to other datasets. If institutional bias were significant, we would expect substantial performance degradation on these datasets, which we do not observe. Additionally, please also refer to reviewer vkqd’s Q1.1, where we conduct the influence score of training dataset to quantify their impact. Result shows that although MIMIC-III constitutes a large portion of the pretraining corpus, it does not overwhelmingly influence the learned representations, suggesting that potential biases introduced by MIMIC-III are limited.

Q3.2: Can you provide few-shot fine-tuning results on an out-of-distribution dataset to compare MIRA’s sample efficiency against a model trained from scratch?

A: We constructed a training set with 20k samples based on MIT-BIH dataset, each having a sequence length of 256. During inference, the prediction lengths are set to 24, 36, 48, and 64, with the context length being twice the prediction length. Average RMSE performance reported.

Notably, Time-MoE required 80% of the data (~16k samples) to reach similar RMSE levels as MIRA with 128–256 samples. This underscores MIRA’s strong few-shot performance, as well as its potential to significantly reduce data and computational costs in new clinical settings.

[1] Wu, Haixu, et al. "Timesnet: Temporal 2d-variation modeling for general time series analysis." arXiv preprint arXiv:2210.02186 (2022).

[2] Wang, Yuxuan, et al. "Deep time series models: A comprehensive survey and benchmark." arXiv preprint arXiv:2407.13278 (2024).

[3] Haq, Kazi T., et al. "Demographic and methodological heterogeneity in electrocardiogram signals from guinea pigs." Frontiers in physiology 13 (2022): 925042.

Dear LN9n, it is mandatory you provide your feedback