Unified Insights: Harnessing Multi-modal Data for Phenotype Imputation via View Decoupling

Dear Reviewer dJb6

Thank you for the review and valuable comments. We respond to your questions below.

1. Time complexity.

The time complexity of data quantization for each patient is composed of three primary components. The first component is the encoder, which has a time complexity of $O(D \cdot F)$, where $D$ represents the input dimensionality and $F$ is a factor that depends on the number of layers and the operations performed within the encoder. The second component is vector quantization, with a time complexity of $O(K \cdot d)$, where $K$ denotes the number of entries in the codebooks and $d$ represents the dimensionality of latent embeddings. The third component is the decoder, which has a time complexity of $O(d \cdot G)$, where $G$ is a factor related to the number of layers and operations in the decoder. Consequently, the overall time complexity of data quantization can be expressed as $O(D \cdot F + K \cdot d + d \cdot G)$. Given that both the encoder and the decoder in this study are implemented as MLPs, we simplify the expression to $O(D \cdot d + K \cdot d + d^2)$ for ease of calculation.

Then the updating of GNNs in each iteration mainly involves the updating of node vectors and weight matrices, whose time complexity is $O(n_t\cdot d^2 + z\cdot d)$, where $n_t$ and $z$ are the total number of nodes and the total number of edges in graph $\mathcal{G}_f$ and $\mathcal{G}_p$, respectively. $d$ is the embedding dimensionality.

Lastly, the time complexity of cross-view contrastive knowledge distillation for each patient is $O(d \cdot N)$ where $N$ denotes the number of negative patients.

Therefore the time complexity of MPI in each iteration is $O(D \cdot d + K \cdot d + d^2 + n_t\cdot d^2 + z\cdot d + d \cdot N)$. Since $K \ll D$, $d \ll D$, $N \ll D$, and $D \ll z$, the time complexity simplifies to $O(n_t\cdot d^2 + z\cdot d)$ which is linear with $(n_t\cdot d^2 + z\cdot d)$, depending on the number of nodes and edges in the constructed graphs. It is well-known that canonical GCNs are not characterized by high time complexity, indicating the efficiency and scalability of our model.

2. Why Alzheimer’s disease and related dementias

Chronic diseases, especially neurodegenerative diseases, frequently exhibit missing phenotypes due to mild or nonspecific initial symptoms. Routine data collection processes might overlook these subtle signs until more pronounced symptoms emerge. This can be particularly challenging in the context of neurodegenerative diseases like Alzheimer’s Disease and Related Dementias (ADRD), where early detection is crucial for timely intervention and management. Furthermore, research on ADRD particularly emphasizes early detection and intervention, which aligns well with the research goals of identifying historical phenotypes.

Additionally, the choice of an ADRD cohort involves a relatively larger cohort compared to other neurodegenerative diseases. Alzheimer’s is one of the most common neurodegenerative diseases, which ensures that the data includes a wide range of phenotypic expressions and stages of disease. This diversity is crucial for studying the full spectrum of phenotype presentation and identifying underlying missing signs.

3. M3Care, Grape, and MUSE

These models can be applied to integrate biological data and EHR data. However, they are often inferior to the naive combination of clinical and biological data. This is due to the conflict between clinical and biological views. In detail, GRAPE uses each feature dimension as a node, which is not suitable for high-dimensional feature imputation. M3Care computes patient similarity for each modality separately, thereby failing to explore cross-modality correlations. MUSE connects patients to only two modality nodes, introducing noisy edges between patients. These methods are not specifically designed to integrate biological data with EHR data, highlighting the importance and unique capability of our model.

4. Figure 1 and pseudocode

Thanks for your suggestion. We will revise Figure 1 and add pseudocode in the Appendix.

We hope our explanation can solve your concerns. If you have any other questions or concerns, please feel free to let us know and we are more than happy to answer and make clarifications.

Dear Reviewer W1JA,

Thank you for the review and valuable comments. We respond to your questions below.

1. Other modalities.

Yes, our method can be certainly applied to other modalities if data is available. For each modality, a distinct encoder and decoder can be utilized to uncover the latent factors specific to that modality. Subsequently, a patient-factor graph can be constructed alongside the patient-phenotype graph. Predictions can then be made based on the representations learned by GNN models.

2. Codebook setting and its impact

The quality of the learned codebooks indeed has an impact on the model performance. For example, a smaller number of codebooks, such as one or two, may fail to capture sufficient fine-grained information from the biological data. Conversely, larger codebooks might introduce additional underlying factors due to finer granularity, which could reduce their discriminative power for patient profiling. In addition, smaller codebook sizes may fail to capture some underlying biological factors, resulting in insufficient information for patient profiling. Conversely, larger codebook sizes might lead to certain codes being underutilized, which can hinder the overall optimization of the codebook. Both the number of codebooks and the codebook size can affect the quality of the learned codebooks. The empirical results in Figure 3 demonstrate the effect of different codebook settings.

We hope our explanation can solve your concerns. If you have any other questions or concerns, please feel free to let us know and we are more than happy to answer and make clarifications.

Dear Reviewer 8HRK,

Thank you for the review and valuable comments. We respond to your questions below.

1. Residual quantization

We respectfully highlight that our proposed biological data quantization is not for reducing noise in biological data.

The motivation behind residual quantization as we discussed in the introduction is that the biological conditions of patients uncover major underlying factors that indicate health status. In other words, patients sharing similar underlying biological factors could have similar phenotypes. Identifying these latent factors would facilitate the effective characterization of patients and their phenotypes. Therefore, we propose quantizing the biological data and uncovering the corresponding factors using Residual Quantization. Then we are able to model the correlation between patients from a biological view and learn patient representations by modeling the relationship between patients and latent factors in a patient-factor graph.

Comparison with AE: Applying an Autoencoder (AE) to encode the biological data results in extracting a single low-dimensional biological feature for each patient. This feature serves as the initial embedding of patients in a GCN. By using GCN on the patient-phenotype graph, we can then obtain patient representations that enable the prediction of missing phenotypes based on these learned representations. The result comparison can be found in the table below. We can observe that roughly using the encoded biological data as the feature cannot achieve better performance compared with MPI.

2. The technical contribution

We would like to highlight that our main technical contributions lay in:

1. The view decoupling strategy, which separates the modeling of multi-modal biological data from phenotype data to address the conflicts arising from their joint modeling.

2. The latent biological factor uncovering in patients through data quantization, which Identifies these latent factors enables more effective characterization of patients and their phenotypes.

3. The cross-view contrastive knowledge distillation, which builds upon traditional contrastive learning by introducing a novel approach that incorporates cross-view information with intra-view and inter-view negative pairs.

Additionally, we want to clarify that the graph neural model is not the focus of our paper. We use GCNs as a basic backbone model, which can be substituted with any existing advanced graph neural network. Our emphasis is on demonstrating that the improvements in our model primarily stem from our designed components rather than the sophistication of the GNN itself. Therefore, we opted to use the basic GCN model.

3. Proteomics and metabolomics

We leveraged proteomics and metabolomics as biological studies have pointed out the associations and predictive value of plasma proteomics and metabolomics with ADRD. Since biological processes could begin years before the onset of clinical symptoms, proteomics and metabolomics which comprise the end-product of genes, transcripts, and protein regulations, offer insight into identifying alterations in multiple biochemical processes and the risk of ADRD among cognitively healthy adults [1,2]. The model could be certainly applied to other modalities with a corresponding pre-trained encoder.

[1] Plasma metabolomic profiles of dementia: a prospective study of 110,655 participants in the UK Biobank

[2] Proteome Network Analysis Identifies Potential Biomarkers for Brain Aging. Journal of Alzheimer's Disease, 2023.

4. Why Alzheimer’s disease and related dementias

Chronic diseases, especially neurodegenerative diseases, frequently exhibit missing phenotypes due to mild or nonspecific initial symptoms. Routine data collection processes might overlook these subtle signs until more pronounced symptoms emerge. This can be particularly challenging in the context of neurodegenerative diseases like Alzheimer’s Disease and Related Dementias (ADRD), where early detection is crucial for timely intervention and management. Furthermore, research on ADRD particularly emphasizes early detection and intervention, which aligns well with the research goals of identifying historical phenotypes.

Additionally, the choice of an ADRD cohort involves a relatively larger cohort compared to other neurodegenerative diseases. Alzheimer’s is one of the most common neurodegenerative diseases, which ensures that the data includes a wide range of phenotypic expressions and stages of disease. This diversity is crucial for studying the full spectrum of phenotype presentation and identifying underlying missing signs.

5. Downstream tasks

Thanks for your suggestion. In this paper, we concentrate on the algorithmic development for phenotype imputation. In future work, we will adapt and validate our model for various downstream clinical tasks.

6. Ranking loss

We included classification loss as a variant in our implementation, yet experimental results show that ranking loss outperforms classification loss (see below). This is likely because ranking loss compares the similarity between positive and negative edge pairs rather than showing class probabilities. Given the individual specificity in phenotype imputation, ranking loss better captures the hidden patterns in patients' phenotypes.

Dear Reviewer QunD,

Thank you for the review and valuable comments. We respond to your questions below.

1. Sparsity/missing rates

The biological modalities used in this study exhibit significant missingness at random, with approximately 90% missingness in proteomics and 50% in metabolomics. The proposed MPI demonstrates superior performance compared to baseline methods in handling extreme missingness. Given the high missing rates, it is impractical to conduct experiments with varied missing rates. However, we evaluate MPI through ablation studies that utilize each modality independently, representing scenarios with complete missingness in the respective modalities.

2. Modality data pre-processing

Proteomics data, including levels of around 3,000 proteins, are provided as Normalized Protein eXpression (NPX) values, obtained after UK Biobank preprocessing, which includes median centering normalization between plates and log2 transformation. We used these NPX values directly as the encoder input without further processing [1]. For metabolomics, we applied a natural logarithmic transformation (ln[x+1]) to all metabolite values, followed by Z-transformation [2].

[1] Chen, Lingyan, et al. "Systematic Mendelian randomization using the human plasma proteome to discover potential therapeutic targets for stroke." Nature Communications 13.1 (2022): 6143.

[2] Zhang, Xinyu, et al. "Plasma metabolomic profiles of dementia: a prospective study of 110,655 participants in the UK Biobank." BMC Medicine 20.1 (2022): 252.

We hope our explanation can solve your concerns. If you have any other questions or concerns, please feel free to let us know and we are more than happy to answer and make clarifications.

Dear Reviewer wLvH,

Thank you for the review and valuable comments. We respond to your questions below.

1. General graph neural models

We respectfully highlight that our experiments compared general graph neural models, specifically GraphSage and GIN. The models you referred to are designed for recommendation systems and path-based link prediction, and thus, are not general graph neural models. They are used as references for dataset partitioning to demonstrate the rationale behind the dataset division.

2. Cross-view contrastive optimization

We respectfully highlight that we provided experimental evidence in the ablation study to support this claim. Specifically, in Table 3, V4 is a variant model without cross-view contrastive optimization, whereas MPI incorporates this design. As observed in Table 3, MPI consistently outperforms V4, demonstrating that our cross-view contrastive optimization effectively enables robust patient representation learning.

3. Validity

Thanks for your question. In this paper, we concentrate on the algorithmic development for phenotype imputation and demonstrate the model’s superiority against baselines based on the selective metrics. In future work, we will adapt and validate our model for various downstream clinical tasks.

4. Supporting References

Thanks for pointing it out. We will add references in the revision. Current methods simply treat biological data as features and employ canonical machine learning techniques to encode them, failing to disentangle the heterogeneous factors.

• Imputation of label-free quantitative mass spectrometry-based proteomics data using self-supervised deep learning, Nature Communication 2024

• Toward an integrated machine learning model of a proteomics experiment, Journal of Proteome Research, 2023

• Recent advances in mass spectrometry-based computational metabolomics, Current Opinion in Chemical Biology, 2023

5. Determine the value of the margin hyperparameter

The margin hyperparameter 𝛾 is determined through a search in the set {1, 3, 5,10}. We will include this detail in the implementation supplements.

6. Only Alzheimer’s disease and related dementia dataset

Chronic diseases, especially neurodegenerative diseases, frequently exhibit missing phenotypes due to mild or nonspecific initial symptoms. Routine data collection processes might overlook these subtle signs until more pronounced symptoms emerge. This can be particularly challenging in the context of neurodegenerative diseases like Alzheimer’s Disease and Related Dementias (ADRD), where early detection is crucial for timely intervention and management. Furthermore, research on ADRD particularly emphasizes early detection and intervention, which aligns well with the research goals of identifying historical phenotypes.

Additionally, the choice of an ADRD cohort involves a relatively larger cohort compared to other neurodegenerative diseases. Alzheimer’s is one of the most common neurodegenerative diseases, which ensures that the data includes a wide range of phenotypic expressions and stages of disease. This diversity is crucial for studying the full spectrum of phenotype presentation and identifying underlying missing signs.

Diabetes typically involves more straightforward clinical measurements (e.g., blood glucose levels, HbA1c) and may not have the same challenges as ADRD. In the future, we will identify additional chronic diseases suitable for the task to evaluate our method.

7. Patients with the rare phenotypes and the most prevalent phenotypes were excluded from the dataset.

We filter out phenotypes with an occurrence of less than 20 while our cohort population reaches about 15000. The small occurrence (0.06%) reflects the less practical value in this work of imputing these phenotypes. Meanwhile, there are a few phenotypes with quite high frequency, e.g., more than 4000 individuals with hypertension. Since ADRD generally focuses on the elderly population, these phenotypes are typically with less specificity and regarded as possible confounders due to aging. Meanwhile, these phenotypes might dominate the dataset, obscuring other important associations, while focusing on moderately prevalent phenotypes could reveal more subtle associations.

8. Variance not in the ablation study results

We did not report the variance for the ablation study in Table 3 due to the table space constraints. However, we are happy to include the variance in the Appendix during the revision.

9. The structure of the V4 model

We employ the same GCN used in MPI as the V4 model. The key difference is that the V4 model operates on a single homogeneous graph that integrates biological factors, patients, and phenotypes, whereas MPI utilizes two separate graphs to model patients from both phenotype and biological perspectives. Note that both MPI and V4 models are based on biological factors derived from Biological Data Quantization.

10. Show a table like Table 2 for the results of the ablation study

We are happy to reframe Table 3 of the ablation study to match the format of Table 2 in the Appendix. Thanks for the suggestion.

We hope our explanation can solve your concerns. If you have any other questions or concerns, please feel free to let us know and we are more than happy to answer and make clarifications.