DHENN: A Deeper Hybrid End-to-end Neural Network for Highly Accurate Drug-Drug Interaction Events Prediction

We thank the reviewer for the thoughtful and detailed feedback. Below, we address the weaknesses and questions one by one.

+++ Regarding Weaknesses +++

W1: Limited novelty and incremental improvement over [1].

A1: We respectfully clarify that DHENN represents a significant departure from MKG-FENN [1]. While MKG-FENN employs shallow neural networks to avoid oversmoothing, our DHENN introduces a novel PECN architecture design, which allows for deeper neural networks to improve the learning efficacy without bottlenecked by oversmoothing. This allows DHENN to capture richer, higher-order relationships in the data, which are critical for DDI prediction. The results in Tables 1–3 clearly demonstrate that DHENN significantly outperforms MKG-FENN across all tasks, establishing its superiority in terms of predictive accuracy and robustness.

W2: Lack of convincing evidence for proposed motivations.

A2: We elaborate our three motivators. 1) For high-order interactions, we argue that our MKG is more comprehensive than the referenced methods [2–5] by unifying four types of drug-centered binary relationships: <drugs, chemical entities>, <drugs, substructures> (SMILES), <drugs, drugs> (DDIs), and <drugs, molecular fingerprints> (MACCS), all into a single MKG topology. This comprehensive integration allows DHENN to better explore complex relational semantics and topological structures, which are underrepresented in prior works.

2. For learning capacity, our PECN demonstrates a clear advantage by enabling deeper NNs without oversmoothing, as evidenced by Figure 3. Unlike [1–6], which rely on shallow (one-layer) network due to oversmoothing concerns, DHENN achieves improved performance through a deeper PECN, bridging the gap between learning capacity and predictive performance. 3) For end-to-end learning, while we acknowledge that [2–6] are also end-to-end methods, they often rely on less comprehensive knowledge graphs, limiting their ability to capture diverse drug-centered relationships. DHENN integrates a richer MKG with four different drug-centric information and leverages PECN to learn it, significantly outperforming these methods as demonstrated in our experiments.

W3: Missing comparisons with key KG-based methods [1–5].

A3: We appreciate the suggestion for additional comparisons. However, we would like to note that, in the original manuscript, we have already compared DHENN with MKG-FENN [1]. To complete the state-of-the-art, we have now included comparative experiments with KnowDDI (2024) [4] and EmerGNN (2023) [5] in the revised manuscript. We do not include [2–3] because they were published in 2021, and have been outperformed by the latest models, such as [1, 4, 5] published in 2023 or 2024. The new results confirm that DHENN consistently outperforms all thirteen baseline methods, including [1, 4, 5].

W4: A more representative SMILES sequence is recommended.

A4: The example “986” is a numerical attribute derived from the molecular structure of Lovastatin. Please refer to the A3 response to Reviewer nZSN for more details. For better clarity, we replace “986” with a more representative SMILES sequence and provide additional explanations in the revised manuscript.

W5: Descriptions of Tasks 2 and 3 are unclear.

A5: In the revised manuscript, we have clarified their differences. Task 2 evaluates DHENN’s ability to predict DDIs when partial drugs are excluded from the training set (partial generalization), while Task 3 assesses its performance when all drugs are excluded (complete generalization). These tasks reflect progressively more challenging scenarios, highlighting DHENN’s robustness in generalization.

+++ Regarding Questions +++

Q1: Does the model adaptively assign different weights to relationships in the MKG?

A1: Yes, the relationships in the MKG are adaptively fused through our end-to-end learning approach. The model learns to assign weights implicitly based on the contributions of different relationships to the prediction task, ensuring that critical relationships are prioritized.

Q2: Can the model handle incomplete data or missing modality information?

A2: Yes, DHENN can handle incomplete data or missing modalities. Its modular architecture ensures that predictions remain robust even when certain types of information are unavailable. This capability makes it particularly suitable for real-world applications where data completeness is often a challenge.

Q3: Has dynamic neighborhood sampling been considered?

A3: We investigated the impact of neighborhood sampling in Figure 6 (Appendix). Our analysis showed that a small size of neighbors hinder information propagation, while excessively large neighborhoods introduce noise. While we currently adopt a fixed neighborhood size to balance these factors, we acknowledge the potential benefits of dynamic sampling and will explore this direction in future work.

We thank the reviewer for the discussions and providing us the opportunity to further clarify your concerns. We address them point by point as follows.

Q1: The experimental results suggest that increasing the depth of PECN does not result in significant improvements. Additionally, oversmoothing does not appear to be a major concern for DDI prediction, as comparable performance can be achieved with shallow networks.

A1: Oversmoothing is one severe limitation of GNN [7-8], which makes the indistinguishable representations of nodes in different classes. Notably, as demonstrated in [1, 9], oversmoothing also causes the indistinguishable representations of drugs by GNNs, which limits the performance of DDI prediction. Thus, we argue that oversmoothing is a major concern for DDI prediction. To aid this issue, we design deeper PECN to mitigate the oversmoothing associated with the nodal feature extraction in DDI prediction. Table 7 (Appendix) clearly shows that our deeper PECN can significantly improve the performance, such as by 18.03% at most. Besides, Tables 1-3 (manuscript) clearly show that our model significantly outperforms nine state-of-the-art models.

Q2: Regarding previous comments that ”these works [2-6] are all end-to-end methods for DDI prediction”.

A2: We clarify that non-end-to-end is not the major drawback of previous studies [2-6]. Their main drawback is to consider drug-centered KGs partially or not unify multiple KGs into an end-to-end learning way in both embedding and prediction stages. For example, MUFFIN [2] adopts the pretrained-GNN to obtain the embeddings of molecular graphs of drugs, which is not included in the remaining end-to-end learning process. Another example, the classifier parameter “W_c” of KnowDDI [4] is not included in its loss function, which means that the prediction stage is not included in the end-to-end learning process.

[1] Wu, D., Sun, W., He, Y., Chen, Z., & Luo, X. (2024, March). MKG-FENN: A Multimodal Knowledge Graph Fused End-to-End Neural Network for Accurate Drug–Drug Interaction Prediction. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 38, No. 9, pp. 10216-10224).

[2] Chen, Y., Ma, T., Yang, X., Wang, J., Song, B., & Zeng, X. (2021). MUFFIN: multi-scale feature fusion for drug–drug interaction prediction. Bioinformatics, 37(17), 2651-2658.

[3] Yu, Y., Huang, K., Zhang, C., Glass, L. M., Sun, J., & Xiao, C. (2021). SumGNN: multi-typed drug interaction prediction via efficient knowledge graph summarization. Bioinformatics, 37(18), 2988-2995.

[4] Wang, Y., Yang, Z., & Yao, Q. (2024). Accurate and interpretable drug-drug interaction prediction enabled by knowledge subgraph learning. Communications Medicine, 4(1), 59.

[5] Zhang, Y., Yao, Q., Yue, L., Wu, X., Zhang, Z., Lin, Z., & Zheng, Y. (2023). Emerging drug interaction prediction enabled by a flow-based graph neural network with biomedical network. Nature Computational Science, 3(12), 1023-1033.

[6] Lin, X., Quan, Z., Wang, Z. J., Ma, T., & Zeng, X. (2020, July). KGNN: Knowledge Graph Neural Network for Drug-Drug Interaction Prediction. In IJCAI (Vol. 380, pp. 2739-2745).

[7] Chen et al., Measuring and relieving the oversmoothing problem for graph neural networks from the topological view. AAAI 2020.

[8] Liu et al., Towards deeper graph neural networks. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 338–348, 2020.

[9] Tengfei Lyu, Jianliang Gao, Ling Tian, Zhao Li, Peng Zhang, and Ji Zhang. Mdnn: A multimodal deep neural network for predicting drug-drug interaction events. In IJCAI 2021

We thank the reviewer for the encouraging and constructive feedback, where we address your concerns point by point below.

+++ Regarding Weaknesses +++

W1: Further strengthen the paper by analyzing the model’s computational performance.

A1: Thank you for the suggestion. We have analyzed the training time and memory/GPU usage across all models. The results are as follows:

We will add these results plus discussions in our camera ready.

W2: How different drugs perform based on the connectivity in the multimodal graph. Can this model be applied to new drugs that may not have connections within the multimodal graph?

A2: Our Multimodal Knowledge Graph (MKG) integrates four types of drug-centered relationships: <drugs, chemical entities>, <drugs, substructures> (coding by SMILES), <drugs, drugs> (i.e., DDI events), and <drugs, molecular fingerprints>(coding by MACCS). The MKG ensures connectivity through these diverse relationships. In our experiments, we create two learning setups, namely, Task 2 (partial drugs unknown) and Task 3 (all drugs unknown), with their results showing in Tables 2 and 3, respectively. Through evaluating the two setups where new drugs are introduced without prior connections in <drugs, drugs> interactions, we provide affirmative answer that our DHENN model can generalize to new drugs, even when they lack direct connections in the <drugs, drugs> interaction graph.

W3: Stricter evaluation of generalization capabilities?

A3: In our current work, we employed five-fold cross-validation to evaluate generalizability on a single dataset. We acknowledge that evaluating our model by training and testing on different datasets with minimal overlap (e.g., DrugBank vs. a smaller external dataset) would provide a more rigorous assessment of its domain adaptation capabilities. This evaluation is valuable for discovering and testing new drugs. However, due to time constraints and the scope of this study, we were unable to conduct such an extensive cross-dataset evaluation. We plan to explore this direction in future work, as it aligns closely with our long-term goals of enhancing model robustness and applicability in novel drug development.

+++ Regarding Questions +++

Q1: Regarding the comparison based on new drugs, are the authors reporting the 5th part, i.e., the one containing the new drugs? Are these parts equally sized?

A1: Yes, the results reported in Tables 2-3 are for the new drugs in two datasets. This is how we did: we divided the dataset into five equal parts. In each repeated experiment, 1/5 of the drugs were excluded from the training dataset and used exclusively for testing. Namely, 20% of the drugs were new to the model in each run. We implemented cross-validation and performed checks to prevent any potential data leakage between training and test.

Q2: How is the non-end-to-end approach implemented?

A2: Correct. We implemented it as a sequential two-phase training process with different optimization objectives. The GNN phase optimizes the reconstruction loss of graph structures, and the PECN phase minimizes the prediction loss for DDI events.

We thank the reviewer for the insightful and constructive feedback. Below, we address each weakness and question raised one by one.

+++ Regarding Weaknesses +++

W1: High-order interactions.

Q1: We acknowledge the concern about the single-layer GNN capturing only first-order relationships. However, the constructed Multimodal Knowledge Graph (MKG) in our framework inherently encodes high-order interactions by integrating four types of drug-centered binary relationships: <drugs, chemical entities>, <drugs, substructures> (coded by SMILES), <drugs, drugs> (representing drug-drug interactions), and <drugs, molecular fingerprints> (coded by MACCS). This topology enables DHENN to capture complex, high-order relationships without requiring multiple GNN layers. Note, while stacking GNN layers could propagate higher-order interactions, it often results in diminished model performance due to oversmoothing, which we address directly in our next point.

W2: Oversmoothing.

Q2: We have demonstrated this in Figure 3, which shows that increasing GNN layers decreases performance. In contrast, our PECN model component enhances performance as its depth increases, validating its ability to mitigate oversmoothing. Prior works like MKG-FENN (Wu et al., 2024) and MDNN (Lyu et al., 2021) also rely on single-layer GNNs for the same reason, as discussed in their original papers.

W3: Non end-to-end framework design.

A3: We acknowledge that some existing methods are end-to-end. However, note that these methods often rely on incomplete or less comprehensive knowledge graphs, limiting their predictive power. In contrast, DHENN constructs a multimodal and comprehensive MKG, which enhances its predictive capacity. The end-to-end integration of this enriched graph structure ensures that representation learning and prediction are jointly optimized, yielding better performance. We will clarify this and refine our discussion of this limitation in the camera-ready.

+++ Regarding Questions +++

Q1: What constitutes a "hybrid method" and how does it differ from KG-based methods?

A1: DHENN is termed "hybrid" because it combines two distinct neural networks: a GNN for learning KG representations, and the Prediction-Enhanced Cascading Network (PECN) for DDI prediction. In contrast, other KG-based methods often separate drug-centered binary relations, neglecting high-order DDI interactions. In addition, the existing KG-based methods tend to use shallow architectures to avoid oversmoothing, whereas our PECN network component allows for stacking deeper GCN layers to enhance learning efficacy.

Q2: What specific modalities are used in DHENN to establish a multimodal framework?

A2: Our MKG encodes high-order interactions by integrating four types of drug-centered binary relationships: <drugs, chemical entities>, <drugs, substructures> (coded by SMILES), <drugs, drugs> (representing drug-drug interactions), and <drugs, molecular fingerprints> (coded by MACCS).

Q3: How are substructures extracted and selected from drug SMILES?

A3: We followed prior studies such as Veber et al. (2002) and Baranwal et al. (2020) to select 13 MACCS key features and 7 additional molecular features, ensuring the relevance and robustness of the selected features.

Q4: Why is neighborhood sampling necessary in DHENN?

A4: As shown in Figure 6 (Appendix), neighborhood sampling balances efficiency and noise handling. Small neighborhood sizes hinder information propagation, while large sizes amplify noise sensitivity. Our approach ensures optimal performance by balancing these trade-offs.

Q5: What is the difference between Task 2 and Task 3?

A5: We elaborate Tasks 1, 2, and 3 for better comparisons. Task 1 assumes all drugs are known during training, Task 2 introduces settings where only partial drugs are known, and Task 3 evaluates the model’s ability to generalize to entirely new drugs unseen during training. These tasks reflect progressively more challenging scenarios, highlighting DHENN’s robustness in generalization.

Q6: What are the experimental settings and optimization objectives for the non-end-to-end version of DHENN?

A6: We implemented the sequential two-phase training with distinct optimization objectives, where the GNN phase optimized the reconstruction loss of graph structures, and the PECN phase optimized the prediction loss for DDI events. In contrast, the end-to-end version of DHENN unifies these objectives under a single framework (Equation 9), ensuring cohesive optimization across all components and ultimately improving performance. This unified optimization process is key to DHENN’s superior performance compared to the non-end-to-end version.

We thank the reviewer for the discussions and for providing us the opportunity to further clarify your concerns. Following your comments, we have revised the inappropriate claims and descriptions. We address the raised concerns point by point as follows.

+++ Regarding Weaknesses +++

W1: High-Order Interactions.

A1: Thank you for pointing this issue out. We agree that the drug-centered multiple binary relationships associated with chemical entities, substructures, and molecular fingerprints mostly represent the high-order “information” instead of “interactions” for DDI prediction. We coin it as “high-order” because, comparing with other DDI methods, our MKG design includes multiple different types of drug-centered relations. We agree that coining such diverse topological information captured by MKG as “high-order interaction” may impose comprehension challenges and confusions. We have revised this term as “high-order information” throughout the manuscript. Please refer to the updated manuscript.

W2: Oversmoothing.

A2: Oversmoothing of GNNs (Chen et al. 2020) in DDI events prediction is associated with two aspects, i.e., the message-passing (via edges) and the structural feature extraction (from nodes) on a drug-centered knowledge graph. We would like to kindly note that, in previous studies (e.g., Wu et al., 2024, Liu et al., 2021), their GNNs remain shallow not only in the message-passing layers but also in the representation (fully-connected) layers due to oversmoothing. Indeed, our DHENN does not address the oversmoothing issue associated with message-passing in DDI events prediction. However, by designing the deeper PECN, our design mitigates the oversmoothing associated with the nodal feature extraction in DDI events prediction. As shown in Figure 3, increasing the depth of PECN can boost performance. We agree that claiming that DHENN can address oversmoothing in GNNs in general is not fully accurate. We have clarified that our paper addresses the oversmoothing issue associated with feature extraction by designing PECN that allows for deeper layers in the updated manuscript. We acknowledge that the oversmoothing issue associated with message-passing remains an open problem that we will discuss in the camera ready.

[1] Chen et al., Measuring and relieving the oversmoothing problem for graph neural networks from the topological view. AAAI 2020.

[2] Wu et al., Mkg-fenn: A multimodal knowledge graph fused end-to-end neural network for accurate drug–drug interaction prediction. AAAI 2024.

[3] Tengfei Lyu, Jianliang Gao, Ling Tian, Zhao Li, Peng Zhang, and Ji Zhang. Mdnn: A multimodal deep neural network for predicting drug-drug interaction events. In IJCAI 2021

W3: Non end-to-end problem.

A3: We appreciate this suggestion. You are correct that non-end-to-end is not the major drawback of previous studies. The main challenge of current DDI prediction is how to comprehensively explore drug-centered KGs to learn a unified predictor. Previous studies either partially consider drug-centered KGs or do not unify multiple KGs into an end-to-end learning model. To this end, our DHENN constructs the unified MKG that includes drug-centered multiple KGs. Such MKG is learned by an end-to-end learning way. Table 4 (manuscript) and Table 5 (Appendix) have validated the effectiveness of multiple KGs. Figure 4 has verified the performance improvement by an end-to-end learning way. Following your suggestion, we have clarified and revised the corresponding introduced limitations and challenges of current DDI prediction in the updated manuscript.

+++ Regarding Questions +++

Q1: Modalities used.

A1: The considered four types of drug-centered binary relationships are: <drugs, chemical entities>, <drugs, substructures> (coding by SMILES), <drugs, drugs> (i.e., DDI events), and <drugs, molecular fingerprints>(coding by MACCS). They are different binary relationships. Table 4 (manuscript) and Table 5 (Appendix) have validated the effectiveness of each type of binary relationship.

Q2: MACCS features.

A2: Thanks for pointing this issue out. One important mechanism of DDIs prediction is to discover the similar characteristics of drugs. Although the metabolic pathway and drug-drug interaction predictions lack transferable knowledge, the MACCS features can still represent some characteristics of drugs, which is benificial for finding the similar characteristics of drugs. Table 4 (manuscript) and Table 5 (Appendix) have validated the effectiveness of MACCS features (i.e., P4 ). In the furture, we will investigate the selecting issue of MACCS features following your suggestion.

We thank the reviewer for the encouraging and constructive comments, where we address your concerns point by point below.

Q1: Icons used in Figure 1 were inappropriate.

A1: Thank you for pointing out these issues with Figure 1. We have uploaded the updated Figure 1, which now uses correct and standardized icons to represent targets, transporters, and enzymes as proteins.

Q2: SMILES representation, and why two molecule graphs in Figure 1.

A2: We have revised Figure 1 to represent SMILES using actual SMILES strings to provide a more accurate depiction, as you suggested. Regarding the two molecule graphs, they were intended to represent two different drugs, drug$_i$ and drug$_j$, involved in a drug-drug interaction. We acknowledge that this was not clearly conveyed in the original figure. In the revised version, we have explicitly labeled the molecules as drug$_i$ and drug$_j$ to clarify their roles within our framework. This should help readers better understand the context and purpose of the illustration.

Q3: What does the SMILES attribute “986” refer to? Why not mentioning SMILES?

A3: The number “986” represents a molecular fingerprint, which is a numerical descriptor used to encode chemical properties. For example, the triplet <Lovastatin, includes, 986> indicates that the drug Lovastatin contains the substructure corresponding to bit “986” in its molecular fingerprint, where the value "986" refers to a unique molecular fingerprint generated using the Morgan Fingerprint method (also known as ECFP) from the RDKit library [1]. To further clarify, this process include 1) generating the drug molecular information, including the SMILES string, drug ID, target, and enzyme data., 2) converting the SMILES string into a molecular object using the "Chem.MolFromSmiles" method from RDKit, and 3) using the "AllChem.GetMorganFingerprintAsBitVect" method to generate the molecular fingerprint from valid molecular objects.

To aid readers who do not have a background in molecular research, we have introduced the concept of SMILES in the revised manuscript to avoid ambiguity. Specifically, SMILES (Simplified Molecular Input Line Entry System) is a text-based notation system that represents chemical structures. It uses atomic symbols (e.g., C, O, H) and bond types (e.g., - for single bonds, = for double bonds) alongside parentheses for branching and numbers for ring structures. SMILES provides a compact way to describe molecules, making it easy to store and share chemical information. We will supplement this description in our camera ready.

[1] https://zenodo.org/records/13990314