Thank you for the valuable suggestions to improve our baseline. Following your recommendations, we evaluated hold-out test results using demographic features (age, sex) and PPG features (sVRI, IPA, SQI), for both regression (ridge) and classification (logistic regression) tasks (Appendix E, Table 15; Link to table: https://imgur.com/Rz4855N):

• Ablation Study: We compared PaPaGei with three baselines—demographics alone, PPG features alone, and demographics + PPG features. Our results show that while demographics is a stronger baseline than statistical features, PaPaGei outperforms the demographics + PPG baseline in 14 out of 18 tasks.

• Effect of Demographics: We trained a model combining PaPaGei-S with demographics. The results indicate that incorporating demographic features with PaPaGei-S creates a stronger model than using PaPaGei-S alone.

These findings underscore an important point: demographic features are not competing with PaPaGei but rather complement it, as previously established in studies including demographics with sensor data [1]. This highlights the complementary potential of combining PaPaGei's advanced feature extraction with demographic context for improved task performance.

Ablation study using Demographics, PPG features, Demographics + PPG. Comparing PaPaGei performance improvements with supervised baselines for each task: Classification Tasks (Positive is better): ICU (+0.13), Mortality (+0.01), Smoker (-0.02), Pregnancy Stage (+0.22), Hypertension (0.00), SDB (no demographics), Mood Disturbance (-0.08), Valence (-0.01), Arousal (+0.04). Regression Tasks (Negative is better): AHI > 3% (-1.43), AHI > 4% (-1.61), gestation age (-1.54), SBP-VV (-0.31), DBP-VV (-0.46), SBP (-0.11), DBP (-0.65), Avg. HR (-4.07), HR (-2.93). PaPaGei-S performs better for real-time sleep and cardiovascular outcomes such as sleep apnea, heart rate and blood pressure, respectively. In particular, we notice that outcomes such as heart rate benefit substantially from PPG rather than demographics. Demographics are useful in tasks without real-time dependence such as smoking, which is established to be associated with age and sex [2]. Importantly, demographics do not add much to already homogeneous populations. For example, consider the NuMoM2B dataset which has women within a specific age range. Here, we observe that PaPaGei obtains much higher AUROC and MAE than the supervised baselines.

Demographics complement PaPaGei-S embeddings. We observe that adding demographics to PaPaGei-S embeddings improves over PaPaGei in the following tasks: Mortality (+0.03), Hypertension (+0.03), AHI > 3% (-0.62), AHI > 4% (-0.09), gestation age (-0.03), SBP VV (-0.38), DBP VV (-0.03), SBP (0.40), DBP (0.10). Based on these results, PaPaGei-S + Demo is a stronger model in many cases. Importantly, these results indicate that PaPaGei-S embeddings learn features that are complementary to demographics and are not simply proxies for age or sex. However, it is important to note that while demographic features can be valuable for personalization, they may not always be readily available, and in reality, we cannot use them in isolation to predict real-time outcomes such as blood pressure or heart rate. Therefore, our PaPaGei models are designed to function effectively with real-time sensor data alone, ensuring their applicability in situations where complete demographic information is not accessible.

[1] Spathis, D., Perez-Pozuelo, I., Gonzales, T. I., Wu, Y., Brage, S., Wareham, N., & Mascolo, C. (2022). Longitudinal cardio-respiratory fitness prediction through wearables in free-living environments. NPJ Digital Medicine, 5(1), 176.

[2] Chung, W. S., Kung, P. T., Chang, H. Y., & Tsai, W. C. (2020). Demographics and medical disorders associated with smoking: a population-based study. BMC Public Health, 20, 1-8.

Thank you for the suggestion to improve the regression plots. To address Issue 3, we have made the plots more informative in the following ways:

• Scatterplot of True vs. Predicted Values: We now include a scatterplot of true versus predicted regression values, annotated with the slope of the regression line (m) and the R^2 value. These metrics provide a clear indication of the model's predictive performance.
• Kernel Density Plot: We overlay the true distribution with the predicted model distributions using kernel density plots. This visualization demonstrates how well PaPaGei captures the true distribution.

Updated figures: (1) AHI: https://imgur.com/IkR7XYd; (2) Avg. HR: https://imgur.com/ru8ZjhM

These additions make it easier to assess the alignment between predictions and true values. It is important to note that steeper slopes and higher $R^2$ values indicate better predictions, as the scatterplot illustrates the alignment between true and predicted values. As shown in the updated figures (Figure 8, Appendix F - Figure 24), PaPaGei-S exhibits steeper slopes and higher R-squared values for both AHI and HR prediction, indicating superior performance. In the AHI plot, PaPaGei-S demonstrates a closer match to the true distribution, particularly in the peak around 22 and the spread of the base. Similarly, for average HR, PaPaGei-P aligns more closely with the left tail of the true distribution (around the 42 to 50 range).

(Q1)

To summarize the regression results, we have provided the symmetric mean absolute percentage error (sMAPE) [1] in addition to the average MAE (Tables 3 & 4). We choose sMAPE instead of MAPE because sleep apnea has true values close to 0, resulting in extremely large MAPE values. However, sMAPE performs weighting and scales the values between 0 to 100%. Our results indicate that PaPaGei-S obtained an sMAPE of 13.34% whereas the next best-performing method is Moment with an error of 13.82%. It is important to note that sMAPE applies weighting and scales errors. When unweighted error percentages are considered, PaPaGei-S demonstrates larger performance gains over the baseline methods, highlighting its superior accuracy.

[1] https://permetrics.readthedocs.io/en/latest/pages/regression/SMAPE.html

(Q2)

Updated plots: https://imgur.com/pF4Kc4v

We performed the Wilcoxon Signed Rank test (non-parametric paired t-test) to evaluate statistical significance at 90% and 95% confidence. The significance values are now indicated in Figures 6 (a)-(b), and 5. In Figure 6, we notice that the Full model is statistically significant ($p<0.05$) compared to both sVRI + SQI and sVRI + IPA. We also notice that sVRI is the most important component with performance matching the Full model. However, we also notice that sVRI is not significant compared to sVRI + SQI, thus indicating the need for the Full model.

In Figure 5, we notice statistically significant performance gains as we increase data for MAE. We observe a similar trend for AUROC except that VitalDB + MIMIC-III is not significant compared to All the datasets.

(Q3)

This is a good idea to obtain insights about the link between PPG-specific losses and downstream tasks. We have added the results to Table 14.

(Q4)

Unfortunately, we did not evaluate models smaller than 5M. We hypothesised that scaling model size would result in some monotonic trend. However, we obtained a non-monotonic trend with the smaller and larger models performing better than the middle. To address this concern, we have clarified our claim about 5M being the optimal model size for PPG data in general.

“This suggests the 5M model is better suited for our pre-training datasets, aligning with prior findings on the proportionality between data and model size [1]” “Nevertheless, our scaling analysis shows a non-monotonic trend, indicating other factors strongly influence performance.”

[1] Narayanswamy, G., Liu, X., Ayush, K., Yang, Y., Xu, X., Liao, S., ... & McDuff, D. (2024). Scaling Wearable Foundation Models. arXiv preprint arXiv:2410.13638.

(Q5)

Yes, this is a visual artefact of smoothing and the way the KDE histogram is plotted. The smallest distance is 0.13.

(Q6)

Here, we refer to model-to-model differences. We notice clearly that PaPaGei-S performs better for light tones than other models for SBP and DBP. However, for darker skin tones it is difficult to identify one model that is the best. We have rephrased the sentence to better reflect our observation. “PaPaGei-S achieves the best BP estimation across light tones. Across dark tones, we notice that BYOL and REGLE obtain the lowest MAE for Systolic BP and Diastolic BP. However, identifying a single model that performs best across all skin tones remains challenging. While PaPaGei-S obtains the best overall performance, additional work is necessary to improve robustness on darker skin tones.

(Q7)

Thank you for the comment, we have added whiskers to Figure 9.

Hi R-d898,

We hope this message finds you well. This is just a follow up on our response to your review. We have carefully addressed your comments and made revisions to the manuscript accordingly. We would be grateful if you could acknowledge our rebuttal and consider raising your score if you are satisfied with our revisions.

Thank you for your time and consideration.

Dear Reviewer d898,

Thank you for taking the time to review our work. We sincerely appreciate your valuable feedback and have revised the paper to incorporate your suggestions. Given the approaching deadline, we would be grateful if you could acknowledge our revisions.

(Q1)

This is a good point. Yes, we use single-channel PPG (we will update the paper with this detail).

(Q2)

We have improved Figure 5 by performing the Wilcoxon Signed Rank test (non-parametric paired t-test) to evaluate statistical significance at 90% and 95% confidence. We hypothesize that adding additional data would increase the diversity of users and signals, thus improving performance consistently. Our findings in Figure 5 indicate that adding data does indeed improve performance consistently. In particular, we notice statistically significant performance gains as we increase data for MAE. We observe a similar trend for AUROC except for the VitalDB + MIMIC-III ablation. Furthermore, considering the aggregate improvements across all tasks in AUROC and MAE of 0.61 to 0.67 and 12.29 to 10.13, respectively, indicate potentially beneficial differences for the Full model.

(Q3)

You are correct to question the relevance of the Llama3 paper (Dubey et al., 2024) to our observation about the MESA dataset. We apologize for the potentially misleading comparison. We intended to simply highlight the general trend that more data often leads to better performance, which holds in various machine learning domains, including language modeling. While the Llama3 paper specifically focuses on the relationship between compute and performance, it also demonstrates that increased compute allows for the utilization of more data points (tokens), which in turn can lead to improved performance. This general principle aligns with our observation that the MESA dataset, despite having fewer participants, yielded the best performance among the single datasets, possibly due to its higher number of segments. To avoid any misinterpretation, we have revised the statement in the paper to more accurately reflect our intended meaning. We emphasize the general observation about the relationship between data size and performance without drawing a direct comparison to the specific findings of the Llama3 paper. Furthermore, recent work around scaling wearable signals hinted that increasing the signal segments (hours) results in better performance compared to increasing the number of users [1] (Section 5.2).

[1] Narayanswamy, G., Liu, X., Ayush, K., Yang, Y., Xu, X., Liao, S., ... & McDuff, D. (2024). Scaling Wearable Foundation Models. arXiv preprint arXiv:2410.13638.

(Q4)

Thank you for your thoughtful question. We believe that PaPaGei-S outperforms PaPaGei-P due to the following reasons: (1) Similarity in PPG Characteristics Across Users: PaPaGei-S defines positive pairs based on the sVRI. Our intuition here is grounded in the observation that different users often exhibit similar PPG characteristics. By defining positive pairs this way, we encourage the model to learn inter-user similarities and form clusters of users based on shared signal properties. In contrast, PaPaGei-P encourages the model to learn a different cluster for each user. (2) Explicit Learning of Signal Quality (SQI): PaPaGei-S incorporates signal quality explicitly into its training by predicting the signal quality index (SQI). This means the embeddings generated by the model inherently encode signal quality information, which is crucial for downstream tasks. In comparison, PaPaGei-P does not explicitly account for signal quality, potentially leaving the embeddings less robust to variations in signal conditions. (3) Physiologically-Informed Multi-Objective Loss: PaPaGei-S is trained with a multi-objective loss function that leverages domain knowledge about PPG signals. This SSL approach guides the model to learn representations that are both physiologically meaningful and robust. Contrastive learning methods that incorporate physiological insights, as highlighted in prior work ([1]), have shown significant success in improving model performance.

To further substantiate our assumptions, here we describe some empirical results to describe the utility of PaPaGei-S. In Figure 7, we evaluate the inter-participant distances on the out-of-domain SDB dataset. PaPaGei-S demonstrates appropriate participant separation, which contributes to its superior performance in downstream tasks. Figure 9 highlights that PaPaGei-S is notably more robust to variations in skin tone compared to PaPaGei-P. This robustness stems from its ability to encode signal quality and inter-user similarities effectively, leading to improved performance in diverse populations. We hope this clarifies the reasoning behind the stronger performance of PaPaGei-S.

[1] Gopal, B., Han, R., Raghupathi, G., Ng, A., Tison, G., & Rajpurkar, P. (2021, November). 3KG: Contrastive learning of 12-lead electrocardiograms using physiologically-inspired augmentations. In Machine Learning for Health (pp. 156-167). PMLR.

Thanks for pointing this out. We have included additional clarification in Appendix A: “Additionally, our SSL baselines use the same batch size, learning rate, input sampling frequency, and training steps as PaPaGei. We use the same augmentation types and intensity for BYOL, SimCLR, and PaPaGei-P. Furthermore, we investigated 0.07 and 0.5 temperatures as MoCo [1] and SimCLR [2], respectively, and selected the best-performing model.”

[1] He, Kaiming, et al. "Momentum contrast for unsupervised visual representation learning." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020.

[2] Chen, Ting, et al. "A simple framework for contrastive learning of visual representations." International conference on machine learning. PMLR, 2020.

(W 1.1)

Thank you for your suggestions to better represent prior work. Following your suggestions, we discuss [1] in terms of large-scale training, method, and evaluation. In particular, we discuss specifics about their contributions and also indicate how we differ from their work (Section 2): “[1] demonstrated that embeddings derived from PPG signals can predict over 45 diverse downstream health-related tasks using proprietary Apple Watch data. Their approach uses an SSL framework based on patient-level positive pair contrastive learning.” “In contrast to [1], our work exclusively uses public datasets for large-scale PPG training and extends the SSL framework to incorporate PPG morphology. While their evaluation is limited to a single proprietary dataset, we validate our approach on 10 diverse downstream datasets, showcasing greater generalizability and robustness across varied real-world scenarios.”

Importantly, we have added a new Table 17 to highlight the key similarities and differences between PPG studies with regards to: the number of users, number of devices (& type of devices), openness, and number of downstream tasks and datasets. We hope that these additions address your concerns.

Moreover, we have added model design comparisons to other studies throughout the paper: ”Segment the signal into 10-second windows (whereas larger studies use 30s [2] and 60s [1])” (Section 4.1) “We adopt a ResNet-style CNN encoder, following [2], whereas [1] utilize an EfficientNet-style.” “PaPaGei-P and PaPaGei-S having 5M and 5.7M parameters, respectively, other studies use model sizes of 3.3M [1]” (Section 4.1) “In contrast to a smaller embedding size of 256 adopted by [1], we project the learned representations to a 512-dimensional embedding after the convolutional block (we investigated larger embedding sizes of 768 and 1024 and found no significant performance changes).” (Appendix A)

We have comprehensively evaluated demographic characteristics in the newly added Appendix E. In particular, we have evaluated PaPaGei’s ability to predict demographic targets through age regression, age classification, and sex classification. We compare these results to the [1] in Table 16, in Appendix E. Furthermore, we comment on this in the main text in Section 6 as follows: “PaPaGei-S achieves 7.78 MAE in age regression, 0.85 accuracy in age classification, and 0.79 accuracy in sex classification, advancing open-source efforts despite trailing larger closed studies [1] by 2.18, 0.05, and 0.13, respectively.”

(W 1.2)

We directly acknowledge [2] in the methods section: “Prior studies have shown that incorporating PPG signal quality during training yields positive results.” Furthermore, we also discuss key similarities and differences between this study and in new Table 17.

(W 1.3)

We have added a discussion on foundation models for physiological signals in the extended related work (Appendix H): “Recently, there is growing interest in modality-specific FMs tailored to physiological signals [3] and human activity [4]. For instance,[3] trained a large-scale 12-lead ECG model for detecting 60 diagnostic terms, while [6] developed an open-source ECG FM using 1.6 million 12-lead signals. In brain signal analysis, [5] introduced Brant-2, an EEG and SEEG model supporting tasks like sleep staging and seizure detection. Building on this progress, we adopt a domain-specific approach focused on photoplethysmography (PPG) signals.”

We also acknowledge the link between time series methods and LLMs in Appendix I.

“An increasingly popular approach involves feeding timeseries data and prompts directly to Large Language Models (LLMs). However, despite promising results, LLMs struggle with high-dimensional signals due to their text-based processing [7]. A modality-specific encoder like PaPaGei addresses this limitation by providing representations of raw signals, which can be combined with text and fed into more powerful multimodal foundation models, such as AnyMAL. This approach offers several advantages: computational efficiency through a fixed LLM, flexibility due to the modular design of encoder, adapter, and LLM components, and interoperability with other high-performing models (e.g., a state-of-the-art IMU encoder [5]. Crucially, this encoder-LLM approach does not require paired data with other modalities to train a single multimodal model. However, it may introduce complexity by limiting end-to-end gradient propagation and reduce interpretability in encoder-LLM communication compared to natural language prompts. Despite these trade-offs, PaPaGei serves dual purposes: as a generic feature extractor for various PPG signals and applications, and as a modality encoder in next-generation frontier models. This versatility positions it as a valuable tool for advancing multimodal sensory AI systems.”

Thank you for highlighting this concern. We plan to discuss it in the following ways: We acknowledge the capability of general-purpose time-series models to capture information in certain downstream tasks. Indeed, models like Chronos perform comparably well for targets such as mortality and smoking. We have added the following to Section 5.1: “Chronos obtains good performance in predicting mortality, pregnancy stage, and smoking, likely due to their slower rate of change and reduced reliance on granular PPG-specific features. General-purpose models suffice for these broader characteristics. However, tasks requiring finer PPG-specific granularity, such as heart rate prediction, blood pressure estimation, or sleep apnea, benefit from PaPaGei's specialized feature extraction.”

To evaluate the significance across models on a per-task basis, we do the following. First, we performed the Friedmann Chi-Square test and identified statistically significant differences across PaPaGei and the baseline models at p < 0.05. Next, we created critical difference (CD) diagrams to rank the best-performing models, as suggested by the literature to compare models over multiple datasets [1, 2]. The CD diagrams (Figures 22 & 23) are available here: https://imgur.com/cPS1yBG and https://imgur.com/k4lvydU. The CDs indicate that PaPaGei performs the best across both classification and regression tasks. Furthermore, it has a statistically significant average rank as indicated by the lack of a horizontal line.

We performed the Wilcoxon Signed Rank test (non-parametric paired t-test) to evaluate statistical significance at 90% and 95% confidence. The significance values are now indicated in Figures 5 and 6 (https://imgur.com/pF4Kc4v). In Figure 6, we notice that the Full model is statistically significant ($p<0.05$) compared to both sVRI + SQI and sVRI + IPA. We also notice that sVRI is the most important component with performance matching the Full model. However, we also notice that sVRI is not significant compared to sVRI + SQI, thus indicating the need for the Full model. In Figure 5, we notice statistically significant performance gains as we increase data for MAE. We observe a similar trend for AUROC except for the case of VitalDB + MIMIC-III which is not significant compared to All the datasets.

[1] Demšar, J. (2006). Statistical comparisons of classifiers over multiple data sets. The Journal of Machine learning research, 7, 1-30. [2] https://scikit-posthocs.readthedocs.io/en/latest/tutorial.html#critical-difference-diagrams

I command the authors' efforts in revising their manuscript and addressing my concerns in the short window of revision period. I have carefully read the manuscript and most of my concerns have been addressed or at least have been discussed in the paper. However, I have 3 remaining (minor) comments:

1. Upon my review, I noticed the reported numbers in Appendix Table 16 and Discussion do not match with the prior work's Table 2 top row. I suggest the authors revise these numbers to their correct values, and there seems to be a large gap and the potential reasons need to be further discussed in the Discussion.

2. I recommend adding amount of waveform data (e.g., in units of time or any other alternative) for each row in Appendix Table 17.

3. It appears to me that the recently added analysis about comparisons with demographics (Appendix E) are not included in the main text or not even discussed. I highly recommend properly including and discussing these results in the main text to better contextualize the study results and findings.

Given that these comments are straightforward to address in the manuscript, authors efforts in the presentation / implementation / rebuttal, the openness of this work and that an open weight/code PPG foundation model can foster the health research, I have raised my score contingent upon my final review of the above changes.

While we acknowledge that SSL frameworks can use different encoder architectures, we decided to match the encoder architecture and parameters, rather than the total model parameters, to maintain consistency in encoder design. For example, SimCLR and PaPaGei-P have the same parameters/architectures (5M). In contrast, BYOL and TF-C use two parallel encoders, resulting in model sizes of 12M and 9.7M. Importantly, please note that each encoder used in BYOL and TF-C is the same as SimCLR, PaPaGei-P, and PaPaGei-S. Furthermore, the extra +0.7M in PaPaGei-S arises from the Linear layers in the multi-task heads. We believe this provides a fair basis for evaluating performance across tasks while accommodating the requirements of different SSL frameworks.

We appreciate this feedback, and we address this concern in two ways: (1) Evaluating the significance of the statistical features and (2) Comparing against a stronger supervised baseline.

Evaluating the significance of the statistical features.

We conducted a paired t-test between the statistical features and PaPaGei-S. For both classification and regression tasks, we observed a p-value of < 0.01 with t-statistic values of 7.75 and -6.45, respectively. These results strongly demonstrate that PaPaGei-S significantly outperforms the statistical baseline. Furthermore, the t-statistic values indicate that the proposed model's performance is not close to the statistical baseline. Among the 18 tasks evaluated, only four tasks—Mood Disturbance, Arousal, Smoker, and Diastolic BP (PPG-BP)—show results that can be considered “close” to the baseline. It is important to note that cross-dataset generalization in mental health using sensor data is an inherently challenging problem. Previous studies have reported AUROC values ranging between 0.55–0.60 for various mental health tasks in similar setups [1]. Therefore, we believe it is neither surprising nor controversial that all methods, including ours, exhibit similarly low performance for Mood Disturbance and Arousal tasks under these circumstances.

Improved supervised baseline using demographics and handcrafted features.

We evaluate supervised baselines with demographics and PPG features as requested by other reviewers (https://openreview.net/forum?id=kYwTmlq6Vn&noteId=0Ol4N2R521). We conducted two key experiments for our regression (ridge) and classification (logistic regression) tasks:

• Ablation Study: We compared PaPaGei with three baselines—demographics alone, PPG features alone, and demographics + PPG features. Our results show that while demographics is a stronger baseline than statistical features, PaPaGei outperforms the demographics + PPG baseline in 14 out of 18 tasks.
• Effect of Demographics: We trained a downstream model combining PaPaGei-S embeddings with demographics. The results indicate that incorporating demographic features with PaPaGei-S creates a stronger model than using PaPaGei-S alone.

These findings underscore an important point: demographic features are not competing with PaPaGei but rather complement it, as previously established in studies including demographics with sensor data [1]. This highlights the synergistic potential of combining PaPaGei's advanced feature extraction with demographic context for improved task performance. However, it is important to note that while demographic features can be valuable for personalization, they may not always be readily available, and in reality, we cannot use them in isolation to predict real-time outcomes such as blood pressure or heart rate. Therefore, our PaPaGei models are designed to function effectively with real-time sensor data alone, ensuring their applicability in situations where complete demographic information is not accessible.

[1] Xu, X., Zhang, H., Sefidgar, Y., Ren, Y., Liu, X., Seo, W., ... & Dey, A. (2022). GLOBEM dataset: multi-year datasets for longitudinal human behavior modeling generalization. Advances in Neural Information Processing Systems, 35, 24655-24692.

We address your concern by providing reasoning for our design choices and empirical evidence that showcases that our method generalizes across different scenarios. We believe that PaPaGei-S performs well due to the following design choices: (1) Similarity in PPG Characteristics Across Users: PaPaGei-S defines positive pairs based on the sVRI. Our intuition here is grounded in the observation that different users often exhibit similar PPG characteristics. By defining positive pairs this way, we encourage the model to learn inter-user similarities and form clusters of users based on shared signal properties. (2) Explicit Prediction of Signal Quality (SQI): PaPaGei-S incorporates signal quality explicitly into its training by predicting the signal quality index (SQI). This means the embeddings generated by the model inherently encode signal quality information, which is crucial for downstream tasks. (3) Physiologically-Informed Multi-Objective Loss: PaPaGei-S is trained with a multi-objective loss function that leverages domain knowledge about PPG signals. This SSL approach guides the model to learn representations that are both physiologically meaningful and robust. Contrastive learning methods that incorporate physiological insights, as highlighted in prior work ([1]), have shown significant success in improving model performance.

To further substantiate the effectiveness of PaPaGei-S, here we describe some empirical results. (1) In Figure 7, we evaluate the inter-participant distances on the out-of-domain SDB dataset. PaPaGei-S demonstrates appropriate participant separation, which contributes to its superior performance in downstream tasks. (2) In Figure 8 and other prediction plots, we notice that PaPaGei-S’s predictions better align with the true target distributions indicated by steeper slopes and larger R2 values. Furthermore, we notice that PaPaGei-S has less tendency to regress to the mean indicated by the lower peak in the Figure 8 distribution plot. This leads to better capture of the left tail. (3) In Figure 9, we demonstrate that PaPaGei-S is the best model for light skin tones and in the top 3 for darker skin tones. Overall, this showcases relatively better robustness to tone variations than other methods. (4) Finally, in comparison to other PPG studies (Table 17), we evaluate 10 datasets with 7 different kinds of devices. Our results showcase superior performance in the majority of the tasks, suggesting good robustness to noise and data shifts.

[1] Gopal, B., Han, R., Raghupathi, G., Ng, A., Tison, G., & Rajpurkar, P. (2021, November). 3KG: Contrastive learning of 12-lead electrocardiograms using physiologically-inspired augmentations. In Machine Learning for Health (pp. 156-167). PMLR.

We appreciate the reviewer’s feedback, but we respectfully disagree with the suggestion that “All the components … are not novel”. Particularly, we believe our approach introduces an important innovation: employing morphology targets, such as sVRI and IPA, within a contrastive and multi-task learning framework. To the best of our knowledge, such inductive biases have not been previously applied to training a foundation model for biosignals. We believe this is crucial and highlights the contributions of our work (as also recognized by other reviewers). Additionally, our work contributes towards openness by conducting an extensive study on integrating datasets from multiple institutions into a unified framework, which we believe offers substantial value to the community.

On PPG morphology, we clarify that our motivation is supported by the following evidence: (1) Skewness is a critical measure of signal quality (SQI), as demonstrated in [1]. As stated in Section 3.2, we use SQI to complement signals where IPA cannot be computed, addressing diverse and noisy PPG morphologies. (2) We performed a permutation test [6] to statistically evaluate PPG segments where IPA is unavailable. By splitting SQI values into no IPA and IPA groups and testing significance over 1000 permutations, we observed statistically significant differences (p < 0.05) in both mean (+0.18) and median (+0.32) SQI values, with the IPA group having larger SQI, consistent with results in [1]. These findings empirically validate SQI's ability to handle limited PPG morphology. This integration provides initial empirical insights into our framework. Additionally, IPA has demonstrated utility in estimating cardiovascular metrics like blood pressure (BP) [2], while sVRI, originally proposed for cognitive load measurement, also correlates with BP and heart rate. Our ablation study further supports this integration, as shown in Fig. 5(a) and (b), where the full model with all components outperforms individual approaches using SQI and IPA.

To better contextualize the significance of our contribution, we emphasize the following:

First, we fill an important gap in the research community: the absence of a large-scale open-source study focused on training models for PPG data. To train PaPaGei, we unify the largest openly available datasets (as highlighted in [3]): VitalDB, MIMIC-III, and MESA. In contrast, other comparable large-scale datasets, such as Apple’s AHMS and Google’s Fitbit dataset, are proprietary and not feasible for researchers to obtain. Our work addresses this critical need by offering a foundation for reproducibility in PPG analysis. We strongly believe this contribution will be useful to the research community in this domain.

Second, we conduct comprehensive generalization and out-of-domain evaluations. Unlike most existing studies, which either do not include cross-dataset experiments or perform limited evaluations [4, 5], we evaluate our model across a diverse range of datasets. Importantly, We do not conduct a selective study, nor do we claim that our model achieves the best performance across all datasets or settings. Instead, we have expanded our evaluation to as many datasets as possible. We position our work as a significant step forward in this field, though not the final one. While we acknowledge that our model does not achieve the best performance on the DaLiA dataset, it outperforms others in 9 out of 13 out-of-domain tasks. This demonstrates the model's strong generalization ability and robustness when applied to unseen, non-ICU datasets, underscoring its practical utility across diverse real-world scenarios. Considering the popularity of the DaLiA dataset in the community, we would not want to hide this evaluation, despite the fact that it does not favor our model.

[1] Elgendi, Mohamed. "Optimal signal quality index for photoplethysmogram signals." Bioengineering 3.4 (2016): 21.

[2] Robb, Richard A., et al. "Proceedings of the 15th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Part 3 (of 3)." Proceedings of the 15th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Part 3 (of 3). Publ by IEEE, 1993.

[3] https://peterhcharlton.github.io/post/ppg_datasets/

[4] Abbaspourazad, S., Elachqar, O., Miller, A. C., Emrani, S., Nallasamy, U., & Shapiro, I. (2023). Large-scale training of foundation models for wearable biosignals. arXiv preprint arXiv:2312.05409.

[5] Ding, C., Guo, Z., Chen, Z., Lee, R. J., Rudin, C., & Hu, X. (2024). SiamQuality: a ConvNet-based foundation model for photoplethysmography signals. Physiological Measurement, 45(8), 085004.

[6] https://faculty.washington.edu/kenrice/sisg/SISG-08-06.pdf

I appreciate the authors' detailed response and the effort put into the revision. However, after reviewing both the responses and the updated manuscript, I find that the revision raises more concerns than it resolves.

While the Friedman test and CD diagrams are reasonable tools for comparing models across tasks, the significant overlap in confidence intervals between the proposed method and the baselines reduce the robustness of the ranking differences. Rankings based solely on mean performance risk overstating the significance of small differences when confidence intervals suggest no clear separation. Even the approach that is used during revision potentially introduces bias into the evaluation. This issue is particularly problematic in this case, where the reported improvements are on the order of 0.01–0.03, accompanied by substantial overlap in confidence intervals.

Also, the compared baseline models do not have any expert blocks with additional FCNs. The comparison is even unfair. How can a reader can understand if the performance improvement comes from additional FCNs or extracted features? Why not excluding the heads during the evaluation of S model such that a fair comparison can be made?

What do the authors mean by BYOL use two parallel encoders? It uses only one encoder during inference. In the original paper [1], the authors managed to match the parameter number of models with SimCLR, also mentioned multiple times, at the end of training, we only keep the encoder, $f_{\theta}$. I also checked the code, I could not find BYOL implementation, authors emphasized multiple times the open-source of the work, however, most of the implementations are not given in the code.

[1] Bootstrap your own latent: A new approach to self-supervised Learning. NeurIPS 2020.

I do not think IPA, SVRI, and SQI features can be referred as morphology of PPG, they are the features that are affected from morphology of PPG signals. Also, how do the authors claim these features are more robust to Gaussian noise? Gaussian noise would change these features depending on the magnitude, the statement is wrong. Even, the Gaussian noise would change the location of the systolic peak so SVRI should change.

In the revisited script Table 14, adding SQI feature to the model decreases the performance in more than half of the tasks, but the authors claim (2) Explicit Prediction of Signal Quality (SQI): PaPaGei-S incorporates signal quality explicitly into its training by predicting the signal quality index (SQI). This means the embeddings generated by the model inherently encode signal quality information, which is crucial for downstream tasks. The experiments and claims by the authors even contradict with each other.

Given these contradictions with controversial/unclear baseline comparison with the marginal improvements, the discussion of novelty is unnecessary at this point for the current work. However, it is important to note that extracting (non-trivial) features (like SVRI and IPA for PPG) and forcing models to estimate them is a well-established method in the ML community [2]. More importantly, as mentioned before but could not see any answer from authors, extracting the presented features is not trivial for noisy PPG segments, especially the sVRI and IPA as they depend on the waveform. This is a significant drawback in terms of the generalization of the presented framework.

Considering the evaluation issues (additional FCNs and unclear baseline implementations), marginal improvements, limited novelty (extracting features and training models to estimate them), and the framework’s lack of scalability (requiring clean signals for feature extraction) all require significant attention.

While the work may hold value in more specialized contexts, its contribution in both findings and methodology is significantly limited for a general ML conference like ICLR. Therefore, I will maintain my original rating.

[2] Masked Feature Prediction for Self-Supervised Visual Pre-Training, CVPR 2022.

(Q1)

We have added the code to ‘DaLiA.ipynb’ in the repository. Please let us know if you have any questions.

(Q2)

We would like to clarify that we performed an initial investigation into different augmentations and intensities. However, we did not systematically evaluate them. [1] notes that cropping (cut out) and channel permute were the most important PPG augments. We use a strong cut-out intensity and do not use channel permute because we use single-channel PPG where we wanted to maintain temporal dependency for morphology calculation. Our initial experiments with Warping required significant time for the cubic spline relative to their performance contribution, so we did not adopt them. The augmentations that we used such as scaling, flipping, and negation are popular time series approaches. Additionally, we emphasize that the multi-task predictions of SQI and IPA in PaPaGei-S are similar to pretext tasks in SSL.

To the best of our knowledge, the field of PPG-specific data augmentation remains underdeveloped, and we could not identify any comprehensive studies systematically evaluating the most effective data augmentation techniques for PPG signals. While there are examples, such as a GAN-based approach [2], we emphasize that exploring various data augmentation strategies could enhance our methodology. However, this is not the primary focus or novelty of our work. Given the limited revision time and significant computational resources required, this lies beyond our current scope and time budget. We leave this for future investigations. We added a discussion to Appendix I and discussed this direction for future research.

[1] Abbaspourazad, S., Elachqar, O., Miller, A. C., Emrani, S., Nallasamy, U., & Shapiro, I. (2023). Large-scale training of foundation models for wearable biosignals. arXiv preprint arXiv:2312.05409.

[2] Kiyasseh, D., Tadesse, G. A., Thwaites, L., Zhu, T., & Clifton, D. (2020). PlethAugment: GAN-based PPG augmentation for medical diagnosis in low-resource settings. IEEE journal of biomedical and health informatics, 24(11), 3226-3235.

We appreciate your thoughtful feedback. You are correct that adding Gaussian noise can introduce small deviations in the PPG waveform, which may affect its morphology. In our context, we meant to refer to IPA, SVRI, and SQI features which are more robust to Gaussian noise. Considering that our Gaussian noise is zero-mean, it introduces random variations in the signal amplitude across the waveform without systematically shifting the entire signal. For metrics such as IPA and SVRI, the addition of zero-mean Gaussian noise does not significantly affect the final computed values. This is because the noise averages out over the signal, with its mean value across different points being zero, thereby minimally impacting these aggregate measures. However, we will revise the manuscript to clarify this. Furthermore, we have acknowledged other works faithfully in the revision including CLOCS: for example, “Prior work has shown that this strategy is effective for physiological signals [CLOCS, ICML 21]” (Section 3.1)

(C4) Significance to the ICLR community.

As novelty is subjective, we focus on the contribution and significance of our work to the ICLR community:

• Practical real-world impact: PPG is one of the most widely utilized biosignals for deriving critical medical metrics, including heart rate and blood pressure. Prior to our work, there was no foundation model available for researchers to extract embeddings or fine-tune for new tasks and scenarios, especially when working with limited data. Now, ML and healthcare researchers have a foundational tool that enables streamlined and effective exploration of PPG-based applications. We emphasize the practical implications of this as acknowledged by Reviewers DtmN (“The framework presented can have practical implications for cardiovascular monitoring and be used for several applications”), XN3M (“foster the use of PPG for health applications and for health research” & “the openness of this work and that an open weight/code PPG foundation model can foster the health research”), and d898 (“PPG foundation models, in particular open/reproducible foundation models, have the potential for high impact in both health research and practical applications.”)
• OOD Evaluation: A defining characteristic of foundation models is their adaptability to a wide range of downstream tasks under diverse conditions, such as varying devices, populations, and environments. Prior work in this domain primarily focuses on in-domain evaluations using datasets collected from the same device, with limited exploration of out-of-distribution (OOD) generalization. In contrast, our work evaluates the proposed model across 20 tasks spanning 10 datasets, encompassing diverse devices, environments, and clinical populations. To the best of our knowledge, this is the most comprehensive exploration of PPG foundation models for predicting downstream tasks across datasets, addressing a critical gap in the field. This contribution is a strength of our work acknowledged by Reviewers DtmN (“performs extensive testing on different tasks.”).
• Extensive analyses: Beyond overall performance metrics, our thorough analyses provides the community with useful findings about: (1) incorporating domain-specific features into SSL, (2) the effect of additional pre-training data, (3) downstream data-efficiency analysis, (4) the utility of demographics features, (5) impact of scaling, (6) analysis on underlying participant embeddings, (7) assessment regression models to capture distribution tails, and (8) robustness towards skin-tone. The importance of these analyses has been acknowledged by all reviewers. Reviewer MitC: “The authors perform the skin tone analysis, which can be important to evaluate fairness.”, Reviewer DtmN: “It also includes ablation studies that prove the performance gains of each component.”. Reviewer XN3M: “I really appreciate the level of details in the paper including the ablations studies; authors did a good job for providing good level of information and detailed ablations for a curious reader.”. Reviewer d898: “Analysis that was performed in addition to performance reporting on downstream tasks, such as dispersion of subject embeddings and sensitivity of some tasks to skin tone.”

(C1) Fair Comparisons

We appreciate your feedback. We believe we have identified the differences between our understanding. Specifically, we report “training parameters” in Table 4, whereas the BYOL paper reports “inference-time weights”, as clarified in Section 3.1 of [1]: “When comparing to other methods, we consider the number of inference-time weights only in the final representation $f_{\theta}$.” Below, we provide a detailed explanation based on “training” and “inference” parameters.

Training Parameters:

• PaPaGei-S: Our multi-task SSL framework comprises three components. The ResNet encoder block ($E$ and $P$ in Figure 2) includes 5M parameters. And two expert blocks, $M_1$ and $M_2$, each containing fully connected layers of 350K parameters, totaling 700K parameters. Thus, PaPaGei-S has a total of 5M + 0.7M = 5.7M parameters during training.
• BYOL: The training setup employs two identical encoder blocks: one for the online encoder and another for the target encoder, each comprising 5M parameters, resulting in 5M x 2 = 10M parameters. Additional projections for these encoders contribute ~1.7M parameters. The total training parameters for BYOL amount to approximately 12M parameters.
• SimCLR: SimCLR utilizes the same encoder block as PaPaGei-S, with a total of 5M parameters during training.

Inference Parameters:

• PaPaGei-S: Feature extraction during inference is performed using only the encoder block, which contains 5M parameters. Importantly, the outputs from the expert blocks ($M_1$ and $M_2$) are scalars for specific tasks ($y_{ipa}$ and $y_{sqi}$) and are not used for feature extraction.
• BYOL: As noted in [1], inference employs only the online encoder ($f_{\theta}$), which has 5M parameters, with other components discarded.
• SimCLR: As SimCLR does not involve discarding components, inference also uses 5M parameters for feature extraction.

Based on these details, we emphasize that our comparisons are fair since the downstream evaluation of BYOL, SimCLR, and PaPaGei-S consistently uses ~5M parameters during inference. We will provide explicit distinction between training and inference parameters with additional clarifications.

[1] Grill, J. B., Strub, F., Altché, F., Tallec, C., Richemond, P., Buchatskaya, E., ... & Valko, M. (2020). Bootstrap your own latent-a new approach to self-supervised learning. Advances in neural information processing systems, 33, 21271-21284.

(C3) Clarification on referred work

Our work focuses on developing an open foundation model for optical biosignals using a self-supervised learning (SSL) framework, incorporating PPG-specific metrics as inductive biases to predict health outcomes. In contrast, the aforementioned paper [1] is not directly related to our work, as it proposes an SSL framework for videos by predicting HOG features. We respectfully disagree with the notion that methods designed for video analysis can be directly extended to biosignal processing. Below, we outline key distinctions between these domains that highlight why such generalizations are not straightforward:

• Physiological Context: PPG is inherently physiological, capturing vital information from an individual's body. In contrast, the paper [1] focuses on videos and is unrelated to medical tasks or biosignal processing. This fundamental difference in application domains significantly impacts the design of the model and evaluation metrics.
• Sampling Characteristics: The sampling rates of PPG signals range from 64Hz to 1KHz, which is significantly more granular than the frame rates of videos (24–60 fps). This disparity necessitates the development of architectures and preprocessing techniques tailored to the high-frequency, continuous nature of biosignals.
• Data and Resource Constraints: While computer vision has benefited immensely from large, web-scale standardized datasets, biosignal processing datasets are relatively minuscule and domain-specific. In our paper, we compare and contrast the available PPG datasets, highlighting the unique data challenges faced in this domain. These constraints make biosignal research significantly different.
• Architectural Differences: Deep learning architectures for video and image analysis often rely on 2D or 3D convolutions, designed to capture spatial and temporal patterns. In contrast, biosignal processing typically employs 1D convolutions, which are optimized to extract features from sequential data. These architectural differences underscore the unique challenges in modeling biosignals.

In summary, we want to emphasize that developing open-source foundation models for biosignals to address real-world health problems is highly non-trivial and understudied. This has also been acknowledged as a strength in your review: “The paper focuses on applications of healthcare with monitoring which is relatively less studied in machine learning community.” The challenges posed by physiological, sampling, data, and architectural differences underscore the need for domain-specific approaches. Therefore our work presents an opportunity to explore these areas, and we acknowledge that additional future work addressing these challenges is critical (as discussed in our paper).

[1] Masked Feature Prediction for Self-Supervised Visual Pre-Training, CVPR 2022.

We appreciate your response. However, we respectfully disagree with the comments that misrepresent our work.

First, we are happy to address the ambiguities about “parameter size" by adding “training parameters are reported” in the Tables.

Second, in our code, the use_projection enables us to test different projection layers. The initial projection is always present as it is necessary for training as shown in Lines 238-241 in resnet.py: self.dense = nn.Linear(out_channels, n_classes).

Third, in the bootstrap ranking experiment, we observe that PaPaGei obtains significant gains in 30 out of 35 AUROC and 32 out of 35 MAE comparisons. Moreover, we have shown and acknowledged that Chronos and TF-C perform well across both classification and regression: “Among baselines, we notice that Chronos and TF-C are strong baselines in many tasks.”. We want to reiterate that we do not claim our model is the best across all cases. We have discussed the strengths of baselines throughout the paper. Regarding the reference to “In my experiments”, we cannot respond, as we do not have complete details or data on how the “simulated distribution” was estimated.

Fourth, regarding the referred work, we respectfully clarify that none of these are claims from our paper. R-MitC’s previous review refers to a video analysis paper and argues that the underlying principles are similar. In our response, we outlined the key distinctions between video and biosignal domains, emphasizing why such generalizations are not straightforward. This is stated in our reply: “Below, we outline key distinctions between these domains that highlight why such generalizations are not straightforward” & “Therefore our work presents an opportunity to explore these areas, and we acknowledge that additional future work addressing these challenges is critical (as discussed in our paper).” Importantly, we clarify that these are facts about the domain differences, not claims. For example, our work focuses on optical biosignals from 10 publicly available PPG datasets with sampling rates ranging from 60Hz to 1KHz, as shown in Table 8. In contrast, video frame rates are generally lower, ranging from 24–60 fps, and are further downsampled during training. Additionally, computer vision (CV) benefits from extensive resources like common crawl [1], which provides large datasets, such as the 400 million image-text pairs used to train CLIP [2]. Moreover, CV has a rich variety of open datasets, and combining datasets to enhance training for foundation models is a common practice [3 (Section 2.4), 4]. For instance, [5] combines the COCO and VG datasets. Finally, 1D convolutions are typically used for time-series data such as biosignals [6].

Unfortunately, the current response misunderstands these domain distinctions as claims, and the overall intent and scope of our work.

Lastly, about “Regarding the OOD”, we refer to data used to train the foundation model. Table 17 compares our work to previous large-scale PPG studies. It shows that we have compared against diverse devices, tasks, and datasets.

[1] https://commoncrawl.org/

[2] Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., ... & Sutskever, I. (2021, July). Learning transferable visual models from natural language supervision. In International conference on machine learning (pp. 8748-8763). PMLR.

[3] Awais, M., Naseer, M., Khan, S., Anwer, R. M., Cholakkal, H., Shah, M., ... & Khan, F. S. (2023). Foundational models defining a new era in vision: A survey and outlook. arXiv preprint arXiv:2307.13721

[4] Madan, N., Møgelmose, A., Modi, R., Rawat, Y. S., & Moeslund, T. B. (2024). Foundation Models for Video Understanding: A Survey. arXiv preprint arXiv:2405.03770

[5] Chen, Y. C., Li, L., Yu, L., El Kholy, A., Ahmed, F., Gan, Z., ... & Liu, J. (2020, August). Uniter: Universal image-text representation learning. In European conference on computer vision (pp. 104-120). Cham: Springer International Publishing.

[6] Abbaspourazad, S., Elachqar, O., Miller, A. C., Emrani, S., Nallasamy, U., & Shapiro, I. (2023). Large-scale training of foundation models for wearable biosignals. arXiv preprint arXiv:2312.05409.

Thank you for highlighting this. We have added a supervised baseline experiment with demographics and hand-crafted PPG signals. (https://openreview.net/forum?id=kYwTmlq6Vn&noteId=0Ol4N2R521)

Our new regression (ridge) and classification (logistic regression) tasks experiments (added in Appendix E, Table 15) present the following findings:

• Ablation Study: We compared PaPaGei with three baselines—(1) demographics alone, (2) hand-crafted PPG morphology features alone, and (3) demographics + hand-crafted PPG morphology features. Our results show that while demographic features create a stronger baseline than statistical features, PaPaGei outperforms the demographics + PPG baseline in 14 out of 18 tasks.
• Effect of Demographics: We trained a model combining PaPaGei-S with demographics. The results indicate that incorporating demographic features with PaPaGei-S creates a stronger model than using PaPaGei-S alone. These findings highlight an important point: demographic features are not competing with PaPaGei but rather complement it, as previously established in studies including demographics with sensor data [1]. This highlights the complementary potential of combining PaPaGei's extracted features with demographic context for improved task performance. However, it is important to note that while demographic features can be valuable for personalization, they may not always be readily available, and in reality, we cannot use them in isolation to predict real-time outcomes such as blood pressure or heart rate. Therefore, our PaPaGei models are designed to function effectively with real-time sensor data alone, ensuring their applicability even in situations where complete demographic information is not accessible.

[1] Spathis, D., Perez-Pozuelo, I., Gonzales, T. I., Wu, Y., Brage, S., Wareham, N., & Mascolo, C. (2022). Longitudinal cardio-respiratory fitness prediction through wearables in free-living environments. NPJ Digital Medicine, 5(1), 176.

Thank you for your feedback. We have included the $R^2$ metric in Figure 8. As shown in Figure 8, PaPaGei-S demonstrates steeper slopes and higher $R^2$ values for both the AHI prediction task ($R^2$=0.29 compared to 0.18 and 0.16). Additional regression plots are available in Figure 24. These results highlight the superior performance of PaPaGei-S in regression tasks. Moreover, the higher $R^2$ values indicate better alignment between the true and predicted values, further validating the quality of the model's predictions.

Furthermore, we have added the following to Section 6. To assess performance under class imbalance, we examined the F1-score, a robust metric in such cases. PaPaGei achieves the highest F1 in 6 out of 9 classification tasks, demonstrating its effectiveness in handling data imbalance. For regression, PaPaGei-S achieves the highest $R^2$ in 7 tasks (Appendix D), reflecting better alignment with the true distribution. These results highlight the robustness and versatility of PaPaGei-S across classification and regression tasks.

We believe the addition of this result and discussion provides further clarity.

(Q1)

Thank you for your insightful suggestion. Presently, we evaluate the downstream performance using a simple logistic regression model, following the linear probing paradigm. To address low performance or class imbalance, domain generalization (DG) and domain adaptation (DA) approaches could be quite useful. While DA requires access to the downstream dataset, DG methods use losses during training to make models more generalizable. The biosignal processing community can use our models to achieve this in future work. To this end, we have added a discussion in Appendix I providing an example:

For instance, let’s consider the nuMoM2B dataset which consists of pregnant women. PaPaGei-S obtains an AUROC of 0.78 in pregnancy stage classification and 6.05 in gestation age classification. Compared to the pre-training population with diverse ages and gender, the nuMoM2B consists of women generally aged between 20-35. Furthermore, the gestation age readings are collected approximately around the first and third trimesters. Given these factors, the target nuMoM2B dataset has many variables contributing toward the distribution shift. Therefore, PaPaGei-S can be fine-tuned to address the shift in the following ways: (1) We can align the pre-trained embeddings to the nuMoM2B embedding using unsupervised or semi-supervised domain adaptation. (2) Domain Generalization is also an option during the training phase to improve generalization robustness. (3) Newer methods such as LoRA can provide another way to quickly fine-tune. (4) Importantly, given that more women are present on the first visit compared to the third visit (Figure 13), we can optimize different metrics to improve accuracy under the imbalance. For example, AUPRC can be optimized instead of AUROC. Fairness of classification across genders can also be considered during training. Exploring these avenues to further enhance the performance and applicability of PaPaGei is a promising direction for future studies.

(Q2)

You're correct. Most of the out-of-domain (OOD) datasets utilize different devices for PPG signal collection compared to the devices used in our pre-training datasets. This difference in devices is a key factor contributing to the OOD nature of these datasets, as it introduces variations in signal characteristics, hardware calibration, and potential biases. There's one exception: the nuMoM2B dataset. This dataset, which focuses on pregnant women, uses a Nonin pulse oximeter device, similar to the one used in the MESA dataset, which was part of our pre-training data. A caveat is that MESA uses a sampling rate of 256Hz whereas nuMoM2B uses 75Hz. This overlap in devices presents an interesting opportunity to analyze how PaPaGei performs when the device type remains consistent while other factors, such as the study population and physiological conditions, vary. To highlight this aspect, we will add a note in the revised manuscript specifying the devices used in each OOD dataset and their relation to the pre-training datasets. This clarification will provide readers with a clearer understanding of the OOD evaluation and the potential influence of device variability on model performance. To further expand on devices, we have a new Table 17 comparing different PPG studies which includes no. of devices and types.

(Q3)

We acknowledge that potential biases may exist in the datasets beyond just skin tone, arising from factors like sex/gender imbalances, age skewness towards older populations, and inclusion of participants with specific health conditions. These factors may not be representative of the general population. This was the reason we included a detailed description of each dataset and its most important features in Appendix B. To mitigate these biases, we are actively working on diversifying our training data, evaluating PaPaGei on a wider range of OOD datasets, and exploring fairness metrics during training. Additionally, we have evaluated PaPaGei’s ability to predict demographic targets through age regression, age classification, and sex classification. We compare these results to the [1] in Table 16, in Appendix E. Furthermore, we comment on this in the main text in Section 6 as follows: “PaPaGei-S achieves 7.78 MAE in age regression, 0.85 accuracy in age classification, and 0.79 accuracy in sex classification, advancing open-source efforts despite trailing larger closed studies [1] by 2.18, 0.05, and 0.13, respectively.” We hope this answers some of your questions.

Hi R-DtmN,

We hope this message finds you well. This is just a follow up on our response to your review. We have carefully addressed your comments and made revisions to the manuscript accordingly. We would be grateful if you could acknowledge our rebuttal, and we are happy to answer any further questions or concerns you might have.

Thank you for your time and consideration.

Dear Reviewer DtmN,

Thank you for taking the time to review our work. We sincerely appreciate your valuable feedback and have revised the paper to incorporate your suggestions. Given the approaching deadline, we would be grateful if you could acknowledge our revisions.

(IV) Redesigned figures with additional information: As requested, to provide additional evidence that our predictions match the true distributions, we have improved the regression prediction results to include slope ($m$) and $R^2$. Furthermore, we also overlaid the baselines and PaPaGei prediction distributions with the true distribution (Figure 8).

(V) References to additional metrics: To address comments related to class imbalance and generalization, we have included a discussion in Section 6. Furthermore, as requested, to provide an alternative to aggregating overall regression performance across tasks, we have included the symmetric mean absolute percentage error (sMAPE) metric (Tables 3 & 4).

(VI) Improved discussion: To provide users with useful insights, we have extended the discussion to include suggestions for future research such as fine-tuning models for distribution shifts and systematic evaluation of PPG-specific augmentations (Appendix I).

(VII) Reorganized sections: We have reorganized the ablation study section to improve readability between text and figures (Section 5.2).

(VIII) Ablation study on demographic analysis: We have added a new section "Effect of Demographics" in Section 5.2 of the main text. This highlights our analysis and findings of the demographic analysis.

(IX) Additional statistical motivation of design choices: To address questions regarding inclusion of model components, we provide our statistical motivation analysis comparing SQI and IPA in Appendix D.5