Customized Subgraph Selection and Encoding for Drug-drug Interaction Prediction

Thank you for your time and efforts in reviewing our paper. Please find our responses below to your concerns.

W1. For the supernet training phase, in addition to the algorithm process, the authors are encouraged to provide an illustration to help the reader understand the corresponding steps more clearly.

R1. Thanks to your suggestion, we will add a graphical illustration of the search process in the revised version to show the supernet training process more clearly.

W2. A typo error in line 220: “Subgraph Repersentation“

R2. Thank you for your careful review! We have thoroughly checked the entire text and corrected all spelling and formatting issues.

Q1. Given the method itself is general, is there any reason why the authors would like to specifically focus on DDI prediction?

R3. DDI is a critical task in pharmacology and healthcare. Compared with the generalized multi-relational graph link prediction, DDI prediction has more explicit application scenarios and practical needs. Therefore, we design CSSE-DDI based on the DDI prediction task and dataset. However, our method can also be used for generalized multi-relational graph link prediction, i.e., knowledge graph relation prediction, after the modification of search space, And we will include some discussions about knowledge graph relation prediction in the revised version.

Thank you for your time and efforts in reviewing our paper. Please find our responses below to your concerns.

W1. About task-specific customization expected for DDI.

R1. Please refer to the general response GR1.

W2. Time efficiency and accuracy about implicit encoding sampling method.

R2. The time overhead of using explicit subgraphs is huge. When we use explicit subgraphs, we need to sample all candidate subgraphs of a query. For $\eta=3$ on Drugbank, the sampling time is 15 hours. In the process of searching, the memory overhead will be multiple times that of the subgraph-based method.

However we can prove the effectiveness of our method in the following way: We collect the corresponding subgraph sizes of different queries obtained from the search process, explicitly sample the subgraphs of the corresponding sizes and use the searched message functions to encode the subgraphs and predict the interactions between drug pairs， called CSSE-DDI(explicit). Table 4 shows the experimental results. It can be seen that the prediction performance of explicit sampling and implicit sampling is similar. However, in terms of single running time, the explicit sampling method needs to perform a separate message passing on each subgraph, whereas the implicit sampling method is more efficient as it can quickly obtain subgraph representations using the subgraph representation approximation strategy.

W3. About the advantage of the proposed subgraph sampling space.

R3. Please refer to the general response GR1.

W4. About time overhead between their NAS method and the baseline methods.

R4. Please refer to the general response GR2.

W5. Lack comparisons with the latest baselines.

R5. Thanks for the suggestion. We add the results of comparisons with multiple latest baselines, as shown in Table 6, including LaGAT[1] published in 2022, ACDGNN[2] published in 2023, and TransFOL[3] published in 2024. All codes are publicly available.

As can be seen, CSSE-DDI is still competitive against the latest baselines. For LaGAT[1], it still extracts the same range of subgraphs for different queries, and in addition R-GAT is still a suboptimal solution for handling diverse DDI interaction data. For ACDGNN, its overall framework is still based on a complete bio-heterogeneous network for reasoning, and its expressive power is still limited compared with the subgraph-based approach.For TransFOL, which utilizes cross-transformers and graph convolutional networks to deal with interactions in DDI datasets, it mainly focuses on complex logical query tasks in DDI data, and does not have any advantage in single drug-drug interaction prediction tasks.

Q1. About visual analysis.

R6. We visualize some of the sampled subgraphs in Section 4.5.1 and make some case studies to demonstrate the effectiveness and interpretability of our method. In the revised version, we will add more visual analysis.

Q2. Provide more baseline comparisons, inductive setings, as well as experiments on more datasets.

R7. For the experimental results of the latest baselines, please refer to R5.

In terms of inductive settings. we use the inductive setting and datasets in EmerGNN[4] method, to predict drug-drug interactions between emerging drugs and existing drugs. The experimental results are shown in Table 7. Emergnn, which is specifically designed for new drug prediction, achieves optimal performance. However, CSSE-DDI still achieves impressive performance, which is mainly due to the robust learning ability of NAS technology for unknown data.

For more datasets, most of the research work on DDI prediction only uses the DrugBank and TWOSIDES datasets, but we have been continually looking to see if there are other available datasets to validate the effectiveness of our approach.

[1] LaGAT: link-aware graph attention network for drug–drug interaction prediction, Bioinformatics' 22.

[2] Attention-based cross domain graph neural network for prediction of drug–drug interactions, Briefings in Bioinformatics' 23.

[3] TransFOL: A Logical Query Model for Complex Relational Reasoning in Drug-Drug Interaction, IEEE Journal of Biomedical and Health Informatics' 24.

[4] Emerging Drug Interaction Prediction Enabled by Flow-based Graph Neural Network with Biomedical Network. Nature Computational Science' 23.

Thank you for your time and efforts in reviewing our paper. Please find our responses below to your concerns.

W1. Examples of symmetric semantic patterns(headache, pain in throat) are not very convincing.(sematic property)

R1. In the field of multi-relational graphs, the modeling of different semantic types of interaction relationships has been a research hotspot. Many works[1-4] have tried to propose diverse hand-designed interaction functions to model different properties of semantic attributes. In the DDI dataset, the semantic nature of drug interactions is diverse, and our example for asymmetric type and symmetric type is to validate the semantic diversity in different datasets. Such diversity is what motivates us to use NAS to search for adaptive models.

Q1.1 CSSE-DDI-FF and KnowDDI both are fine-grained Subgraph Selection. What is the reason for the performance difference？

R2.1. It should be noted here that the query subgraph of KnowDDI, which is extracted by combining external biological knowledge graphs (HetioNet), contains richer external information, such as genes, diseases, proteins, etc. We label KnowDDI as a fine-grained subgraph selection method, which means that it utilizes a graph structure learning module to realize a fine-grained information supplementation and pruning of the extracted subgraphs. Whereas, although our variant (CSSE-DDI-FF) carries out fine-grained subgraph selection, the variant only employs a basic GNN backbone (CompGCN) to encode subgraphs without external knowledge, which makes it difficult to effectively capture the semantic information in the subgraphs, and thus lead to some performance differences.

And CSSE-DDI still outperforms KnowDDI without relying on external knowledge by fine-grained subgraph selection and data-specific encoding function, which directly demonstrates the effectiveness of our method in capturing complex interactions in DDI datasets.

Q1.2 Whether CSSE-DDI can be used to predict DDI for new drugs?

R2.2. As the reviewer pointed out, there is a strong need to predict DDI related to new drugs in real-world scenarios. Therefore, to further validate the effectiveness of our method, we use the inductive setting and datasets in the EmerGNN[5] method, to predict drug-drug interactions between emerging drugs and existing drugs. The experimental results are shown in Table 3. It can be seen that there is a significant performance drop from the transductive setting to the inductive setting, which shows that the DDI prediction for new drugs is more difficult. Emergnn, which is specifically designed for new drug prediction, achieves optimal performance. However, CSSE-DDI still achieves impressive performance, which is mainly due to the robust learning ability of NAS technology for unknown data.

References

[1] Translating Embeddings for Modeling Multi-relational Data, NeurIPS' 13.

[2] Embedding Entities and Relations for Learning and Inference in Knowledge Bases, ICLR' 15.

[3] Holographic Embeddings of Knowledge Graphs, AAAI' 16.

[4] RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space, ICLR' 19.

[5] Emerging Drug Interaction Prediction Enabled by Flow-based Graph Neural Network with Biomedical Network. Nature Computational Science' 23.

Thank you for your time and efforts in reviewing our paper. Please find our responses below to your concerns.

W1. The motivation for using NAS to search components is not clear.

R1. The use of NAS technology to customize components stems from our analysis and understanding of the DDI prediction problem and datasets in the following two ways:

1. Different DDI datasets have different semantic natures of interactions. (asymmetric patterns in DrugBank and symmetric in TWOSIDES)
2. Reasoning that different drug-pair queries should require different ranges of subgraph information, rather than a fixed range. (Our empirical analysis in the supplementary PDF(Figure A1))

Based on the above analysis, we believe that in the face of diverse data properties, the use of customized search techniques, i.e., NAS, can help improve the performance of DDI prediction task and the interpretability of the results, as evidenced by our extensive experiments.

W2. Despite the designed efficiency mechanisms, the inherent overhead of neural architecture search remains significant, especially for large-scale DDI prediction tasks.

R2. We would like to kindly argue that our design for the search algorithm with the implicit subgraph sampling strategy is precisely to cope with the large-scale graph data. Tab. I shows the running overhead of our method and other subgraph methods on the drugbank dataset, for the subgraph methods, before the "+" sign represents the explicit subgraph sampling overhead, and for our method, before the "+" sign represents the searching overhead, and after the "+" sign represents the model's single running time.

As can be seen, CSSE-DDI and the subgraph approach are comparable or even superior in terms of time overhead, which stems from the fact that our implicit subgraph sampling strategy saves a lot of time in the subgraph sampling and representation phases. It is conceivable that if the size of the dataset is further increased, the number of subgraph sampling will be further increased and the advantage of our implicit sampling strategy will be even more significant.

W3. While the paper compares the proposed method with existing ones, a more detailed comparative analysis, including discussions on computational costs and efficiency metrics, would provide a clearer picture of the trade-offs involved.

R3. For the discussion of the time overhead, please refer to R2. Overall, compared with subgraph methods, CSSE-DDI has lower time overheads, which is the advantage brought by our proposed implicit subgraph sampling strategy. Compared with GNN-based methods, CSSE-DDI, by analyzing the DDI prediction problem and datasets, designs a search space that can be adapted to different data by analyzing the DDI prediction problem. By efficiently searching for the components, it substantially improves the prediction performance under tolerable time overhead compared with the traditional GNN-based methods. In addition, CSSE-DDI can design robust and customized model structures to cope with unknown datasets, which is exactly the advantage of utilizing NAS technology to solve the DDI prediction problem.

Q1.1. Why is the performance of GAT in DrugBank so poor?

R4.1. A large number of literature[1-3] points to the fact that although GAT[4] considers the attention on different edges, it fails to consider more generalized and more complex multi-relational graph data, which contains diverse semantic information, as its performance is also not good. Therefore, the direct application of GAT on DDI graphs results in suboptimal solutions.

Q1.2. In [3], it seems to work well.

R4.2. Compared with GAT, LaGAT extends the graph encoding module of SumGNN, and utilizes multi-relational GAT to take into account the diverse types of relations and the attention weights of different edges in the DDI data. We re-run its code in the DrugBank and TWOSIDES data, and the comparison results are shown in Table 2. As shown below, our approach still achieves the best performance by customizing subgraph selection and encoding process. For the performance reported by LaGAT, we were unable to reproduce it. The official repository does not release the corresponding dataset, although it claims it uses the same DrugBank dataset as SumGNN.

Q2. Why does the NAS method proposed by the authors appear to perform much better than other NAS methods in the DrugBank dataset? What is the core reason for this?

R5. Compared with other NAS methods, our method has advantages in both search space and search algorithm.

• Search space: we design subgraph selection spaces and subgraph encoding spaces adapted to the DDI dataset (Sec 3.2), whereas other methods are only limited to generalized multi-relational graphs and do not have task-specific customization for the DDI prediction task.

• Search algorithm: we design a message-aware partition supernet training strategy for the operation coupling problem(Sec 3.3.4), which improves the consistency and accuracy of supernet, enabling the search algorithm more stable and robust, whereas the other methods only use the traditional one-shot search algorithm, which leads to sub-optimal performance.

References

[1] SumGNN: Multi-typed Drug Interaction Prediction via Efficient Knowledge Graph Summarization, Bioinformatics' 21.

[2] r-GAT: Relational Graph Attention Network for Multi-Relational Graphs. Arxiv' 21.

[3] LaGAT: link-aware graph attention network for drug–drug interaction prediction, Bioinformatics' 22.

[4] Graph Attention Networks, ICLR' 18.