From Token to Rhythm: A Multi-Scale Approach for ECG-Language Pretraining

We thank the reviewer for the thoughtful evaluation and for recognizing the strength of our comprehensive experiments and the novelty of our multi-level supervision design. We also updated the manuscript accordingly.

[D.1] Clarification of Token-level Pretraining (Q1)

Thanks for acknowledging our approach may offer novel insights! From our understanding, token-level embeddings may denote fine-grained ECG features, such as P wave shape, QRS duration, and ST segment changes, by modeling local temporal patterns. Compared with beat-level, it provides more low-level understanding of ECG signals. Although the cardiologists may not explicitly mention token-level features, we think incorporating this level of analysis still encourages deep learning models to better capture the fine-grained characteristics of ECG signals, ultimately improving the model's generalizability.

[D.2] Ablation on Isolated Variant (Q2)

Thanks for the valuable comment! As the global rhythm level plays a central role in zero-shot ECG classification, we did not initially report isolated module results. To address this, we conducted additional ablation studies evaluating each level independently. Results are shown in Table D.1. These results show that our full model consistently outperforms each isolated variant consistently, confirming the effectiveness of multi-level supervision.

Table D.1

[D.3] Analysis on Number of Beat Tokens

Thanks for your insightful suggestion! Our initial choice of 10 heartbeats was based on the assumption that most ECG recordings in MIMIC-IV-ECG, with a 10-second duration, would contain roughly one beat per second. However, after analyzing the dataset, we found that the median number of heartbeats per recording is approximately 12–13, as shown in Table D.2.

To investigate the impact of this hyperparameter, we conducted an ablation study with varying numbers of heartbeat tokens. The results, presented in Table D.3, show that our model is relatively robust to this variation. Notably, using 14 heartbeat embeddings yields slightly better performance than our initial setting.

Table D.2

Table D.3

[D.4] Further Discussion (Q3)

Thanks for your comment! From our perspective, the contrastive learning excels at learning global representations from ECG. However, its ability to capture fine-grained details is relatively limited. To address this, we have incorporated token-level pretraining based on generation to further enhance the model's detailed understanding for ECG representions. Its benefit can be witnessed in Tables 5 and 6 of our original manuscript, by showing better performance in various downstream datasets. While this added complexity may seem significant, we think it is manageable. Our entire model can be pretrained on 4 GTX 3090 GPUs, with a batch size of 64 per device. We have discussed this point in our revised manuscript.

We thank the reviewer for the thoughtful review and for acknowledging the novelty of our multi-level supervision and the strength of our experiments. We also appreciate your constructive feedback on the token-level design, which we address below. All responses are updated in the revised manuscript.

[B.1] Clarification of Token-level and Beat-level Supervision (Q1)

Thanks for your insightful feedback. While some ECG reports summarize rhythm-level findings (e.g., “sinus rhythm”), many contain detailed waveform-level observations. Below are examples from MIMIC-IV-ECG, with fine-grained observations in bold:

• Sinus rhythm. Poor R wave progression – probable normal variant. Anterolateral T wave changes may be due to myocardial ischemia. Abnormal ECG.
• Sinus tachycardia. Short PR interval. Borderline ECG.
• Atrial fibrillation. Extensive ST-T changes are nonspecific. Abnormal ECG.
• Probable accelerated junctional rhythm. Low QRS voltages in limb leads. Abnormal ECG.

High-level diagnoses, like "sinus rhythm", still depend on low-level indicators such as P wave consistency and PR interval regularity. Table B.1 illustrates more examples of how clinical interpretations often depend on both global and local ECG features.

Since our token-level pretraining uses a GPT-style objective, rather than masked token prediction, to generate full diagnostic reports from token-level ECG embeddings. By providing full waveform features to the decoder, we allow the model to learn these relationships and generate reports with varying levels of granularity. It may encourage the model to analyze local features and implicitly learn these indicators. Thus, our token-level pretraining design could enhance the model’s ability to learn more generalized, fine-grained ECG representations.

As for beat-level pretraining, it aims to align beat embeddings with corresponding sentences. While general descriptions in sentences may hinder this process, we think that detailed descriptions encourage the model to understand the ECG at a beat-level.

As shown in the ablation studies in Tables 5 and 6 of the original manuscript, both token-level and beat-level pretraining improve the learned ECG representations, which further supports our claims.

We also agree that including more detailed descriptions in the ECG report would further enhance token-level and beat-level pretraining. Please refer to Reply [B.3] for more details.

Table B.1

[B.2] Details of Positive and Negative Pairs (Q2)

Thank you for the helpful question. We train with a batch size of 64 per device across 4 GPUs. Each ECG and its paired report form a positive pair, while all other ECG-report combinations within the mini-batch (i.e., 255 pairs) are treated as negatives for contrastive learning.

[B.3] Further Discussion of Limitation (Q3)

Thank you for the suggestion. We further clarified motivation of token-level pretraining in Reply [B.1]. Additionally, we agree that incorporating more explicit fine-grained knowledge, such as detailed waveform descriptions for rhythm diagnosis, could provide stronger supervision and better alignment with clinical criteria. For instance, breaking down terms like "sinus rhythm" into their underlying features (e.g., P waves before each QRS, consistent PR intervals) may enhance learning. We have explicitly added this discussion to the revised Limitations section.

[B.4] Not Discussed References and Typos

Thanks for the careful feedback. We have corrected typos and not discussed references in the revised manuscript.

We sincerely thank the reviewer for the thoughtful evaluation and for recognizing the novelty of our multi-scale ECG-language pretraining approach. We also appreciate your positive remarks on the clarity of our writing and the thoroughness of our ablation studies. Below, we address your concerns regarding the experimental section in detail. For each point, we have updated our manuscript accordingly.

[C.1] Source of Baseline Results (Q1)

Thank you for your helpful question. The baseline results reported in Tables 2 and 3 are cited from MERL [1], which is a standardized benchmark by pretraining its model along with 10 existing self-supervised approaches on the MIMIC-IV-ECG dataset. To be fair, we strictly followed the same experimental setup to ensure a fair comparison. Specifically, we adopted the same pretraining dataset, dataset splits, preprocessing pipeline, and fine-tuning hyperparameters as provided in the official MERL GitHub repository. To verify the reported results, we reproduced the MERL model using their released code and pretrained weights (as seen in Table C.1). Our reproduced results were consistent with those in the original paper, supporting their reliability. In some cases, our reproduced performance was slightly lower, likely due to differences in hardware or software environments. To ensure fairness, we report the original (higher) results from the MERL paper in our comparisons. We will clarify this point in the revised manuscript and include the updated explanation accordingly.

Table C.1

[C.2] Release of Source Code and Weights (Q2)

Thanks for your great suggestion! To support reproducibility, we will release the full codebase and pretrained model weights to facilitate further research in the community.

[C.3] Curation of Cardiology-related Corpus (Q3)

Thank you for the comment. We followed the HeartBERT [2] procedure to collect cardiology-related data from PubMed and Wikipedia. For PubMed, we used cardiology journal names and glossaries from SJR, NIH, Aiken, and the Texas Heart Institute to query abstracts via the API. For Wikipedia, we extracted articles under the "Cardiology" category and its subcategories, supplemented with glossary-based queries. This resulted in a curated dataset of ~5.6 GB (912.5M corpus). We have added more details in the Appendix of revised manuscript .

[C.4] Not Discussed References and Typos

Thanks for pointing these out! We have modified typos and updated not discussed references in our revised manuscript according to your suggestions!

[C.5] Clarification about Methodology Details

Thank you for the positive feedback and helpful suggestions. We have revised the methodology section to improve clarity.

References

[1]. Liu et al. Zero-shot ecg classification with multimodal learning and test-time clinical knowledge enhancement. ICML, 2024.

[2]. Gwon et al. Medical language model specialized in extracting cardiac knowledge. Scientific Reports, 2024.

Thanks for the thoughtful and detailed review. All responses have been updated in our revised manuscript.

[A.1] Support for Multi-level Motivation

Thanks for your suggestion. From prior work, we have found several supports for diagnosing ECG by multi-level observations:

• Token-level: Atrial Fibrillation is identified by the absence of P waves and QRS duration <120ms [1].
• Beat-level: Sinus rhythm requires each QRS complex to be preceded by a normal P wave [2].
• Rhythm-level: Left Bundle Branch Block diagnosis requires Variable ventricular rate [2].

As such, we think our proposed multi-scale ECG pretraining is reasonable. We have also incorporated more support in our manuscript.

[A.2] Additional Compared Baselines (Q1)

Thanks for your comment. As you suggested, we have compared with Wave2Vec 2.0 + CMSC + RLM (Oh et al., 2022) and ECGFM (McKeen et al., 2024) in below Table A.1. As shown, MELP consistently outperforms these two approaches across all experimental settings.

Table A.1

[A.3] Justification of ECG Encoder Architecture (Q2)

Thanks for your suggestion. Our understanding is that CMSC is a patient-specific contrastive training approach, and Random Lead Masking is a data augmentation strategy. Neither serves as a network backbone. We have used Random Lead Masking by default in our implementation (see code). We also tested CMSC by pretraining the ECG encoder before multimodal training (as it cannot be directly applied into our framework). As shown in Table A.2, the CMSC variant consistently underperforms, with an average drop of -3.53%. Therefore, we did not adopt it in the final model.

Table A.2

[A.4] Additional Downstream Tasks (Q3)

To further show the generalizability of MELP, we evaluated it on two tasks: ECG report generation and patient identification.

We evaluated report generation using the ECGBench dataset (500 samples), comparing it to the 7B PULSE [3] model. As shown in Table A.3, MELP significantly outperforms PULSE, highlighting its strong fine-grained ECG understanding. Moreover, we evaluated MELP on patient idenfitication in PTB-XL (Table A.4), where it achieved the highest Top-K recall. These results further demonstrates MELP's superior generalizability.

We agree that ECG question answering is a promising direction, while this task requires text encoder finetuning and we will include in future work.

Table A.3

Table A.4

[A.5] Analysis of Number of Beat Tokens (Q4)

Thanks for your valuable question. We initially chose 10 heartbeats based on the assumption of one beat per second in 10-second ECGs. Please see Response [D.3] under Reviewer 5Gq6 for more analysis.

References

[1]. Mitchell et al. Canadian Cardiovascular Society atrial fibrillation guidelines 2010: Prevention and treatment of atrial fibrillation following cardiac surgery. Can J Cardiol, 2011.

[2]. Mattu et al. Electrocardiography in emergency, acute, and critical care. American College of Emergency Physicians, 2019.

[3]. Liu et al. Teach Multimodal LLMs to Comprehend Electrocardiographic Images. arXiv preprint 2024.