Democratizing Clinical Risk Prediction with Cross-Cohort Cross-Modal Knowledge Transfer

Dear Reviewer BXf7,

Thank you for the review and comments. We appreciate your recognition of the "strong real-world motivation" and the "novel architectural components". We respond to your questions below.

1. Evaluation on Alzheimer’s and related dementias.

Our focus on Alzheimer’s disease and related dementias (ADRD), including Alzheimer’s disease, vascular dementia, dementia with Lewy bodies, frontotemporal dementia, and mixed dementia, is motivated by both public health significance and technical modeling considerations. ADRD represents one of the most pressing global health challenges, currently affecting over 55 million people worldwide, with projections to triple by 2050. It is the leading cause of disability among older adults and consistently ranks among the top causes of death across countries.

Unlike some chronic conditions that may present abruptly or lack well-defined preclinical stages, ADRD is characterized by a long prodromal phase, during which subtle cognitive, behavioral, and clinical changes gradually accumulate, yet often go unrecognized in routine care. This lack of definitive early biomarkers in structured EHR data further elevates the importance of computational prediction approaches.

In contrast to conditions such as diabetes or hypertension, where early diagnosis is routine and evidence-based treatment guidelines are well established, ADRD remains underdiagnosed in its early stages and lacks curative treatments. This makes early detection and risk stratification not only more urgent but also more impactful for guiding clinical trials, patient monitoring, and intervention planning. In addition, ADRD is comparatively well represented in large-scale EHR datasets, offering sufficient cases for the development and validation of machine learning models, more so than some other chronic diseases with lower prevalence or more ambiguous diagnostic labeling.

Lastly, due to the time-consuming nature of data preprocessing, such as defining disease cases and controls with clinical expertise, constructing appropriate cohorts, and harmonizing real-world EHRs across the four sites, we have focused our current efforts on ADRD. As noted in our discussion of future work, we plan to extend the proposed framework to other chronic neurodegenerative conditions, such as predicting Parkinson’s disease. To apply C3M to Parkinson’s disease, we will follow the same data preparation procedure described in Appendix 1.1, while incorporating domain expertise to define Parkinson’s disease cases and controls based on clinical criteria. Once the data is prepared, the C3M model can be readily applied to the Parkinson’s disease cohort without model modification.

2. Ablation study for genetic data

We presented ablation studies with randomly selected datasets due to space constraints in the submission. Here, we provide additional results on our model and its gene-free variant across two other target datasets: INSIGHT MS and INSIGHT Columbia in the Tables below. The results show notable differences between C3M and its gene-free variant. Besides, although the differences in AUROC and F1 metrics for OHSU are not particularly large, substantial improvements in Sensitivity@95 and PPV@95 indicate that the model performs considerably better on high-confidence predictions. This demonstrates its enhanced ability to accurately identify true positives while satisfying the stringent 95% specificity constraint, which is especially valuable in clinical decision-making contexts.

Dear Reviewer jjL6,

Thank you for the review and comments. We appreciate your recognition of the "well-motivated approach" and "clearly written and well-structured paper". And we respond to your questions below.

1. (W1) The concept of cross-modal knowledge transfer via distillation

We would like to clarify that knowledge transfer via distillation is a necessary component of our model. The teacher model, trained on the source cohort, requires both EHR and genetic data as input. Specifically, it takes phenotypical and genetic representations. However, the target cohorts contain only EHR data and lack access to genetic information, making it infeasible to directly apply the teacher model in target cohorts. To address this, we design a distillation strategy that transfers knowledge from the teacher model to a student model, which operates solely on the phenotypical representation derived from EHR data. By leveraging this distillation framework, we enable effective knowledge transfer from a gene-aware model (teacher) to a gene-free model (student), allowing the model to generalize to real-world target cohorts without genetic input. Therefore, distillation serves as a foundational component of our model; without it, the model cannot be effectively adapted to the target cohorts.

2. (W2) Details on the input features

We provide a brief introduction below, with full details in Appendix 1.1.

• EHR features include structured, timestamped records of patient healthcare data such as diagnoses, medications, as well as demographics, standardized across sites using SNOMED and RxNorm coding vocabularies.
• Genetic features are derived from the All of Us ACAF callset and filtered based on established SNPs recorded in the GWAS catalog associated with ADRD.
• We describe the cohort construction, case-control matching strategy, and index date assignment in detail following the literature across all four datasets (All of Us, OHSU, INSIGHT MS, and INSIGHT Columbia), and provide the statistics on the four patient cohorts in Table 1 of the Appendix.

3. (W3 and Q3) How the method handles potential temporal patterns in EHR data

In our paper, we adopt the EHR foundation model MOTOR to capture temporal patterns by taking EHR sequences as input and learn patient-level embeddings, analogous to how BERT-series models are used in natural language processing to learn contextualized representations.

4. (W4) Reproducibility

We would like to kindly clarify that the GitHub link is included in Section 5 of the attached checklist. As the rebuttal guidelines prohibit including links in the response text, we respectfully refer the reviewer to Section 5 for access.

5. (Q1) Provide more details on missing data patterns across datasets and how these are handled during training and evaluation

Thanks for the question. We focused on a practical scenario, where the target cohorts from local healthcare providers only collect routine EHR data, while the All of Us program, as the source cohort, has measured genetic data through nationwide efforts. Therefore, the genetic data is completely missing on target datasets, namely OHSU, INSIGHT MS, and INSIGHT Columbia. Meanwhile, random missingness of genetic data also occurs on All of Us at a rate of 26.7%. Our model addresses missingness in the All of Us through the teacher–student framework, where the teacher remains fixed and the student model learns solely from EHR data when genetic information is absent.

6. (Q2) Explanation for the number of aggregation tokens

As shown in Figure 3 and discussed in Section 5.2, we observe that increasing the number of aggregation tokens from 1 to 2–3 improves performance, because a single token tends to focus narrowly on a subset of genetic features, potentially missing important signals. The use of multiple tokens enables a mixture-of-aggregation strategy, capturing complementary biological patterns from different perspectives.

However, performance saturates or slightly declines with 4 tokens, which we attribute to the introduction of redundancy or noise. Multiple aggregation tokens may begin to extract overlapping or irrelevant information, thereby diluting the model’s discriminative capacity. This phenomenon is likely task-dependent, particularly influenced by the complexity and signal strength of the disease under study. For the task involving subtle or sparse genetic associations, increasing the number of aggregation tokens beyond three may introduce more noise than meaningful diversity, thereby limiting potential performance gains.

Dear Reviewer aRk9,

Thank you for the review and comments. We respond to your questions below.

W1. The practical relevance and general applicability of the setting are not sufficiently justified

We respectfully argue that our work utilizes a source cohort from the All of Us Program, a large-scale biomedical initiative launched in 2015 by the National Institutes of Health. As one of the most comprehensive and demographically diverse biobanks, it represents a nationwide effort and remains openly accessible to researchers worldwide, enabling studies on various health conditions [1]. Our problem setting reflects a growing trend in translational biomedical AI [2], and aligns with current priorities of the HHS Plan 2025 [3], which calls for democratizing AI to reduce the “digital divide” for underserved populations and under-resourced healthcare systems, where models must bridge the gap between rich-resource training environments and real-world clinical deployment.

However, compared with well-funded biobanks, most clinical institutions (e.g., community hospitals, regional health systems) only retain structured EHRs in routine practice, lacking genetic testing due to cost, infrastructure, or policy constraints. This results in a practical distribution shift between well-resourced source cohorts with both EHR and genetic data, and target scenarios with EHRs only.

Our goal is to capitalize on biomedical signals available in multimodal nationwide cohorts (e.g., All of Us) to improve clinical risk prediction in resource-limited settings. This transfer setting is particularly realistic in ongoing nationwide efforts to democratize precision medicine [3], where insights derived from well-characterized research cohorts should generalize to more typical clinical settings. The target real-world clinical repositories, e.g., OHSU and INSIGHT, only contain EHRs without accompanying genetic data. Our work explicitly addresses this critical yet underexplored setting, which we believe will grow increasingly relevant as genomics-informed models gain traction, but data gaps remain across clinical institutions.

[1] All of Us Research Program Investigators. The 'All of Us' research program. New England Journal of Medicine. 2019.

[2] Biobanking with genetics shapes precision medicine and global health, Nature Reviews Genetics, 2024.

[3] U.S. Department of Health and Human Services. Strategic Plan for the Use of Artificial Intelligence in Health, Human Services, and Public Health. 2025.

W2. Limited technical novelty

We respectfully clarify that our paper addresses a critical yet underexplored task, cross-cohort cross-modal knowledge transfer, and proposes a framework specifically designed for this setting, which, to our knowledge, has not been studied before. While our method incorporates concepts such as graph learning and knowledge distillation, these techniques are not simply and directly applied in our setting. Instead, we introduce novel designs specifically tailored to the unique challenges of this problem. We emphasize that our goal is not to combine existing concepts simply, but to design a cohesive system where each element is purposefully constructed to serve the overarching objective, rather than being used as standalone techniques.

In particular, our mixture-of-aggregation module for genetic data is fundamentally different from the standard mixture-of-experts approach, which routes input to relevant neural networks for training. In our design, we introduce the aggregation token to capture patient-level genetic information, accounting for the noise and context dependency in genetic expression; meanwhile, we leverage multiple tokens to model the cumulative effects across different genetic pathways, enabling fine-grained fusion on modality-specific signals. Likewise, the graph-guided fine-tuning strategy also diverges from conventional graph learning or traditional fine-tuning. Rather than updating large pretrained models, we adopt a lightweight, graph-based mechanism to adapt patient representations by leveraging cohort-specific healthcare patterns. This enhances adaptability to population, coding convention, and disease pattern shifts across sites, while preserving efficiency in low-resource settings. Importantly, knowledge transfer is a necessary part of our solution, serving the ultimate goal of transferring knowledge from rich-resourced cohorts to the target ones. To our best knowledge, both the mixture-of-aggregation and the graph-guided phenotypical representation fine-tuning components are novel contributions, thoughtfully developed to meet the particular demands of cross-cohort cross-modal clinical risk prediction.

We would also like to note that, rather than being purely motivated by methodological challenges, our work is grounded in a real-world problem: the limited availability of high-cost data modalities and limited computational resources in many regional healthcare systems. While issues such as noisy or imprecise multimodal data have been studied in domains like CV and NLP, they remain underexplored, particularly in the context of genetic and EHR data fusion. Canonical multimodal methods are not capable of handling our setting, as discussed in related work. Moreover, the challenge of limited computation at local cohorts is a practical constraint to our cross-cohort setting. This particular need and task setup remain unaddressed, underscoring the novelty and significance of our contribution.

W3. Difference in used datasets

We would like to clarify that the dataset for the ablation study was randomly chosen, considering space limit and figure readability, and not guided by any specific selection criteria. We provide additional experimental results on the OHSU cohort below to augment the analyses presented in Figures 3 and 4.

For Figures 3 and 4, the additional results from the OHSU dataset show a consistent pattern with those observed in the All of Us, INSIGHT-MS cohorts, and INSIGHT-Columbia. We will standardize the dataset choices in revision.

W4. Case studies or qualitative analyses

Thank you for the suggestion. We will include t-SNE visualizations in the revision as case studies to illustrate how gene representations are distributed under different aggregation tokens before being passed to the Transformer encoder. While t-SNE provides an intuitive view of representational differences, we are unable to include additional figures during the rebuttal phase. As a quantitative alternative, we compute the Earth Mover’s Distance (EMD) between the two clusters of gene embeddings when using two aggregation tokens. EMD measures the minimal effort required to transform one distribution into another, making it suitable for comparing embedding distributions. In our case, the EMD is approximately 5.6, versus 0 for identical distributions. This non-zero value indicates that the representations learned from the two aggregation tokens are indeed distinct, supporting the hypothesis that our design promotes diverse and potentially disentangled genetic representations.

W5. Medical validation

We respectfully note that our validation on the OHSU and INSIGHT cohorts can be viewed as a form of practical medical validation. These target EHR datasets are derived from real-world hospitals actively serving patients, containing routinely collected records from genuine clinical encounters in Oregon and New York City. Our cohort construction adheres closely to domain expertise and established medical research criteria. As such, our target cohort results demonstrate the model’s practical utility and robustness in realistic clinical environments. We also fully agree that direct medical validation is an important next step and plan to collaborate with our clinical partners to assess the alignment between model outputs and clinical decision-making processes.

W6. Dataset statistics

Though we utilized the “source cohort” term in the paper, it has no implication on the patient number but refers to a nationwide population and biological modality availability. The All of Us program recruits participants nationwide, forming one of the most diverse health research datasets to date. In contrast, both INSIGHT-MS and INSIGHT-Columbia represent patient populations across the NYC area, which enables larger patient coverage due to the region's high population density.

The patients in each cohort are identified through a strict procedure with clinically relevant definitions and filtering criteria; the final cohort size depends only on the covered population scale and the cohort construction procedure, for which we kindly refer the reviewer to the cohort descriptions in Appendix 1.1.

Importantly, the work focuses not on absolute cohort sizes, but rather on addressing real-world disparities between cohorts, from a national biobank with diverse, representative populations to regional populations and systems; from multimodal data (EHR + genetics) to unimodal settings (EHR only); and from high-resource computing environments to resource-constrained clinical sites. This setup simulates the practical challenges in translating risk prediction models into more typical real-world healthcare settings.

W7. Computational efficiency of baselines

We provide a comparison between our model and two representative baselines, SMIL and MoMKE. The fine-tuning time on the OHSU is approximately 300s for our method (Appendix 2.1 (Computation Analysis)). In contrast, SMIL and MoMKE require approximately 2× and 4.3× longer, respectively, under the same computing setup. Our method shows significantly lower computational overhead and better predictive performance.

Dear Reviewer GdmP,

Thank you for the review and comments. We appreciate your recognition of the "significant challenge in healthcare", "novel method", and "The framework is well-structured". And we respond to your questions below.

1. The framework is validated primarily on Alzheimer’s disease and related dementias (ADRD)

Our focus on Alzheimer’s disease and related dementias (ADRD), including Alzheimer’s disease, vascular dementia, Lewy-body dementia, frontotemporal dementia, and mixed dementia, is motivated by both public health significance and technical modeling considerations. ADRD represents one of the most pressing global health challenges, currently affecting over 55 million people worldwide, with projections to triple by 2050. It is the leading cause of disability among older adults and consistently ranks among the top causes of death across countries.

Unlike some chronic conditions that may present abruptly or lack well-defined preclinical stages, ADRD is characterized by a long prodromal phase, during which subtle cognitive, behavioral, and clinical changes gradually accumulate, yet often go unrecognized in routine care. This lack of definitive early biomarkers in structured EHR data further elevates the importance of computational prediction approaches.

In contrast to conditions such as diabetes or hypertension, where early diagnosis is routine and evidence-based treatment guidelines are well established, ADRD remains underdiagnosed in its early stages and lacks curative treatments. This makes early detection and risk stratification not only more urgent but also more impactful for guiding clinical trials, patient monitoring, and intervention planning. In addition, ADRD is comparatively well represented in large-scale EHR datasets, offering sufficient cases for the development and validation of machine learning models, more so than some other chronic diseases with lower prevalence or more ambiguous diagnostic labeling.

Lastly, due to the time-consuming nature of data preprocessing, such as defining disease cases and controls with clinical expertise, constructing appropriate cohorts, and harmonizing real-world EHRs across the four sites, we have focused our current efforts on ADRD. As noted in our discussion of future work, we plan to extend the proposed framework to other chronic neurodegenerative conditions, such as Parkinson’s disease. To apply C3M to Parkinson’s disease, we will follow the same data preparation procedure described in Appendix 1.1, while incorporating domain expertise to define Parkinson’s disease cases and controls based on clinical criteria. Once the data is prepared, the C3M model can be readily applied to the Parkinson’s disease cohort without model modification.

2. Potential societal impacts and broader implications

This work aims to democratize clinical risk prediction by enabling the transfer of predictive models trained on multimodal, nationwide datasets to local clinical settings with limited resources and unimodal EHR data. If widely adopted, the proposed C3M framework could improve early detection and risk stratification of chronic neurodegenerative diseases, such as Alzheimer’s Disease and Related Dementias, especially in underserved populations and healthcare systems with limited access to genetic testing and computational infrastructure. By reducing reliance on costly or inaccessible modalities, this approach promotes more equitable access to precision medicine. However, careful attention is needed as performance disparities may emerge across cohorts, which is not inherent flaws of the model but practical challenges due to variations in healthcare practices. This highlights the importance of local validation, continuous monitoring, and inclusive data practices.