H-Tuning: Toward Low-Cost and Efficient ECG-based Cardiovascular Disease Detection with Pre-Trained Models

We are grateful for your insightful comments on our work. In this rebuttal round, we provide an external validation on a wearable ECG dataset, a more thorough ablation study to support the claims of our study. At the same time, experiments on different backbones strengthen the flexibility of the proposed method (Please refer to our responses to other reviewers).

We sincerely thank the reviewer for all your questions and suggestions.

• Methods And Evaluation Criteria: To assess the generalizability of our classifiers in mobile cardiac healthcare, an external validation set consisting of 7000 wearable 12-lead ECG signals, provided by [1], is used for testing. We first utilized the G12EC, PTB-XL, Ningbo, and Chapman datasets for downstream training, where only 10% of labeled ECG signals are used to fine-tune the backbone. We employ two fine-tuning methods (Full FT and LoRA) and H-Tuning to train three teacher models, followed by knowledge distillation to create three corresponding student models. The CVD detection performance of the six classifiers on the external dataset is shown in Table R1.

Table R1 External validation on a wearable ECG dataset

The results demonstrate that the teacher generated by H-Tuning achieves comparable performance to Full FT and LoRA, but with significantly less GPU memory consumption, as shown in our manuscript. Additionally, our student model performs better than the compared methods.

We cannot compare our classifiers with the winning challenge team on the Physionet 2020/2021 test datasets, which are not publicly available. However, we can report the performance of H-Tuning using the model designed by the winning team (See our response to Reviewer m5NF, Table R4, reference [2]).

[1] Lai, J., etal. Practical intelligent diagnostic algorithm for wearable 12-lead ECG via self-supervised learning on large-scale dataset. Nature Communications, 14(1), 3741.

• Experimental Designs Or Analyses: Please refer to the above section.

• Relation To Broader Scientific Literature: (1) We need to clarify that the proposed H-Tuning and the compared SOTA methods in our manuscript are all fine-tuning methods, which do not utilize unlabeled data for semi-supervised or self-supervised learning. In addition, the experiments presented in our manuscript are all conducted on the downstream datasets. External validation was performed on a wearable 12-lead ECG dataset (Table R1). (2) Comparisons between H-Tuning and CE-SSL. Please see Questions For Authors:.

• Essential References Not Discussed: We thank the reviewer for this comment. The downstream datasets used in our study were also included in the PhysioNet challenge. We will add this reference in next version of our manuscript.

• Other Comments Or Suggestions: We are grateful for your valuable suggestions. We will correct the issues in the next version of our manuscript.

• Questions For Authors: (1) LoHO applied first-order optimization and zeroth-order optimization on different trainable parameters. It did not explore how to utilize their advantages in a more flexible way to fully optimize all parameters. Addax and the proposed mix-order optimization both utilize first-order gradients to refine the direction of the zeroth-order gradients. However, our mix-order optimization proposed a gradient normalization technique to regulate the norm of the zeroth-order gradients, which has not been explored by Addax. Our ablation studies in section 3.4 demonstrated that the proposed technique has an obvious positive impact on fine-tuning performance. At the same time, Table 1 in our manuscript demonstrated the superior performance of H-Tuning compared with Addax and LoHO. (2) The differences between CE-SSL and H-Tuning can be listed as:

   a. CE-SSL is a semi-supervised method, which utilizes unlabeled data to achieve robust CVDs classification performance. In contrast, H-Tuning is a supervised method focusing on low-cost and efficient fine-tuning and inference.
   b. CE-SSL utilizes first-order optimization for model training, which needs extensive activation outputs for gradient backpropagation. H-Tuning avoids this drawback by designing a mix-order optimization method, which greatly reduces the GPU memory costs.
  c. In our study, we integrated H-Tuning with a knowledge distillation technique to reduce the model inference costs on wearable devices, which was not explored by CE-SSL.
  d. To ensure a fair performance comparison between CE-SSL and H-Tuning, we remove the semi-supervised learning module from the CE-SSL while preserving the other modules, which makes CE-SSL a supervised fine-tuning method. We report their average performance across four downstream datasets (following the data split method and the backbone used in our manuscript).
  

Table R2

The results show that H-Tuning performs similarly to CE-SSL with much less GPU memory costs.

We sincerely thank the reviewer for all your questions and suggestions.

• Claims:

The reasons for choosing the ECG setting: (1) The ECG community offers many open-access datasets for model training and evaluation. (2) Developing low-cost methods for accurate and mobile cardiac healthcare is an important topic in AI/ML for health. We apologize for not being able to extend the experiments to other signals within this timeframe, but we will include this as future work in the next version of the manuscript.

• Methods:

Q3: A bandpass filter (1-47 Hz) was used to filter out the noise, which is also adopted by [2].

[2] P. Nejedly et al., “Classification of ECG using ensemble of residual CNNs with attention mechanism,” in Proc. Comput. Cardiol., 2021, pp. 1–4.

Q4: An ideal joint framework should not only reduce the computational costs associated with fine-tuning and deploying pre-trained models but also maintain the performance of the fine-tuned models on downstream datasets. Therefore, under the constraint of achieving comparable fine-tuning performance to full fine-tuning, the joint framework proposed in our study includes two subtasks: (1) reducing the fine-tuning cost and (2) reducing the inference cost.

The unit for Memory is GB.

• Experimental Designs:

Q2: Yes, a backbone with 50 million parameters can be trained or fine-tuned with high-level GPUs (> 8GB memory). However, we want to point out that the costs for fine-tuning and inference become prohibitive on low-level devices (with 2-4 GB memory), which are affordable and typically deployed in clinics or home settings. Therefore, the motivation of our research is to maintain model performance under limited resources. We validated that H-Tuning achieves similar performance to Full FT while reducing GPU memory costs by 6.34 times, fulfilling our research objective. Additionally, we utilized a knowledge distillation technique to reduce the inference costs of the fine-tuned models, enabling them to be deployed on mobile devices (<20MB memory).

Q3: Motion artifacts often behave as baseline wander. Theoretically, a bandpass filter (1-47Hz) can filter out most of the baseline wander noises (generally below 1Hz). We will add a comparison between the raw and the filtered signals in the next version of the manuscript.

Q4: In the previous manuscript, no dataset was collected by wearable devices. To support our claims, we conducted an external validation of the final classifiers generated by H-Tuning on a wearable 12-lead ECG dataset. Please refer to our response to Reviewer ChMh (Table R1).

Q5: There is no 'standard layer' in the proposed model update scheme. All layers are trainable. Without this scheme, all layers can be optimized with first-order optimization, such as Full FT or LoRA. However, we agree that we should include an ablation study on the low-rank adaptation. The results are shown in Table R3.

Table R3

Q6: The success of MeZO-based methods in fine-tuning pre-trained models relies on prompts[3]. However, prompts are not feasible in ECG analysis because there are no text inputs, making the MeZO unable to find a stable optimization path. The MeZO paper also reported a performance collapse (30.3%-47.2% loss) in the absence of prompts.

[3]. Malladi, S.etal. (2023). Fine-tuning language models with just forward passes. Advances in Neural Information Processing Systems, 36, 53038-53075.

Q7: H-Tuning integrates low-rank adaptation to reduce the GPU memory consumption and decrease the variance for gradient estimation, which introduces extra time for the forward process. As shown in Table R3, training time is significantly reduced without this module at the expense of increased memory costs.

• Supplementary Material: We will release our code after the publication of our manuscript.

• Relation To Literature: The architecture of the backbone used in our study is provided by [4], which validated its effectiveness on various datasets. We also verified that the performance of H-Tuning is comparable to Full FT on another backbone provided by [2].

Table R4

• Other Comments: $\eta$ is the learning rate. $\mathbb{E}$ denotes the expectation of the zero-order gradients over vector $z$.

• Note: All evaluation metrics are averaged across 4 downstream datasets and 4 seeds.