FusionDTI: Fine-grained Binding Discovery with Token-level Fusion for Drug-Target Interaction

Weaknesses:

Weakness 1: Architectural Considerations - The fusion module's novelty could be better justified beyond combining existing approaches - There is a contradiction in the description of model flexibility: it claims that the encoder can be replaced but relies on specific SELFIES and SA representations.

Reply: Thank you for your comments. However, we would argue that the work done in this paper is novel. We sincerely hope that our general responses have addressed your concerns.

Fusion module's novelty:

We believe the fusion module's novelty is well-justified, as it goes beyond merely combining existing approaches by incorporating biomedical principles to address fine-grained limitations in existing models. Specifically, we integrate Fine-grained Representation and Innovation Strategy, ensuring token-level fusion to capture fine-grained interactions between atoms and amino acids. Moreover, we incorporate biological knowledge to perform Granular Interaction Validation, systematically comparing token, substructure, and molecular-level interactions. This approach demonstrates the effectiveness of token-level fusion in improving prediction accuracy, as evidenced in Figure 5.

Model Flexibility:

Our proposed model is flexible in that the encoder can be replaced with any pre-trained model capable of generating token-level representations, such as the recent SELFIES-BART (16 Oct 2024). While the current implementation leverages SELFIES and SA representations, other representations could also be explored, provided they capture fine-grained interactions effectively. For instance, alternatives like SMILES-BERT for drugs or amino acid sequence-based models for proteins could serve as replacements in scenarios where SELFIES or SA are unavailable. We will clarify this point more clearly in the revised paper.

Weakness 2: Methodological Aspects - The dataset selection and splitting strategy, while valid, follows previous work (DrugBAN) without significant adaptation - The evaluation metrics suite could be expanded to include F1-score and Matthews Correlation Coefficient.

Reply:

1. Most previous DTI tasks used in-domain splitting strategies, which often lack practical relevance. Our study goes well beyond that, adding for example a cross-domain setting. Note that cross-domain data segmentation strategies have only been used in the DTI literature for a short period of time, and adapting them, as we did in this paper, to realistic biomedical scenarios is still a great challenge.

2. Thank you for your suggestion of evaluation metrics. The following is the latest experiment results with F1-score and Matthews Correlation Coefficient as evaluation metrics. We will add these new results in the revised paper:

In-domain Performance

---

Out-domain Performance

Weakness 3: Experimental Validation - Case studies could be more innovative and differentiated from DrugBAN - The same evaluation metrics in DrugBAN should be shown.

Reply: Our proposed model allows us to directly visualise binding sites using attention maps without the aid of docking visualisation tools. The case study examines three drug-target pairs with ground truth from Protein Data Bank (PDB) for easy comparison with DrugBAN. Notably, all three pairs included binding sites and are validated by wet experiments. Therefore, we do actually compare the predicted binding sites with the ground truth, which shows that our model can predict more binding sites than DrugBAN. To further address the reviewer’s suggestion, we will also provide an additional ten pairs of comparisons between predicted binding sites and real data in the revised version.

Weaknesses:

Weaknesses 1:

Reply: Thank you for your comments. However, we would argue that the work done in this paper is novel. We sincerely hope that our general responses have addressed your concerns.

Weaknesses 2:

Weaknesses 2.1: The selected cases for demonstrating FusionDTI's interpretability are not representative. Readers would be interested not only in the very top predictions but also in moderate and poor predictions (like some bad case analysis) because there is no clear threshold or metric provided to assess whether a prediction is good enough for practical interpretative use.

Reply: Thank you for your suggestion. We examine the three drug-target pairs with ground truth from Protein Data Bank (PDB) to allow for comparison with DrugBAN. Some binding sites without evidence of support (poor predictions) are also highlighted by attention mapping. In the submitted code, we include a notebook file that enables users to visualise specific attention maps for both strong and weak predictions. Since there is no well-established threshold for determining prediction quality, we adopt a ranking-based approach. This allows the performance of the model to be explored, providing a way of assessing interpretability in the top, medium and poor cases.

Weaknesses 2.2: Some inconsistencies in the case study results need to be addressed. For instance, GLN92 is highlighted in Table 5 but does not appear in Figure 9. Please double check that.

Reply: Thanks for your comments. The highlighted amino acid (GLN92) validation is based on a published paper but is not present in the PDB database. In Figure 9, we only labelled the results predicted by the model that are validated by the PDB database. We will better explain the issue in Figure 9’s caption in the revised version.

Weaknesses 2.3: Incorporating a binding structure visualization analysis would greatly enhance the comparison between the predicted interactions and the experimentally validated interactions. It would be also helpful for determining which one (FusionDTI or DrugBAN) aligns best with the known interactions.

Reply: Our proposed model allows for direct visualisation of binding sites using attention maps without the aid of docking visualisation tools. Therefore, we compare the predicted binding sites with the ground truth and show that our model can predict more binding sites than DrugBAN (c.f. Table 5). Moreover, please note that the goal of our task is to identify whether a drug and a target will interact rather than predict the exact binding state of their docking.

Weaknesses 2.4: A better solution could involve quantifying the attention visualization results. For example, calculating how much of key residues or interactions are highlighted by attention weights on a larger scale dataset, such as PoseBusters or CASF, would help to verify the tool’s effectiveness in elucidating drug-protein binding modes.

Reply: Thank you for your suggestion. Quantifying attention visualisation results is extremely time-consuming since processing known binding sites requires manual manipulation of the data, so we will not be able to provide more visualisation results during the author-response process. However, to address the reviewer’s suggestion, we will be providing at least ten pairs of comparisons between predicted binding sites and real data in the final version.

Questions:

Question 1: What specific selection criteria or threshold for attention weights were used to determine the predicted interactions between ligand atoms and protein residues?

Reply: There is currently no well-known threshold for measuring interaction strength, so we identified binding sites of specific atoms and amino acids in the attention map by ranking. We will clarify the issue in the revised paper.

Question 2: The accuracy results for the Human dataset are missing.

Reply: Thank you for your suggestion. The absence of accuracy results for the Human dataset aligns with the standard practice in existing studies, including advanced models like DrugBAN and BioT5, which focus on AUC and AUPRC as primary metrics due to their ability to provide a more comprehensive evaluation of imbalanced datasets like DTI. These metrics are particularly relevant when AUC and AUPRC values approach 99%, making accuracy less informative for distinguishing performance. However, we appreciate the reviewer’s suggestion and the importance of a comprehensive evaluation. Hence, we will include the accuracy results for the Human dataset in the revised version. The results are also shown below.

In-domain Performance (Human)

---

Out-domain Performance (Human)

Questions 1: The authors leverage two existing backbone models (BAN and CAN) to achieve token-level interactions and ultimately search for binding sites using a dense linkage of all these tokens, which appears to be both simple and computationally intensive. Notably, DrugBAN has already employed BAN for a quite similar fusion objective, with the only difference being that the basic element is the substructure. Therefore, the novelty proposed by the authors is concerning.

Reply: Thank you for your comments. However, we would argue that the work done in this paper is novel. We sincerely hope that our general responses have addressed your concerns.

Questions 2: The paper lacks a theoretical contribution regarding the proposed method for the DTI task.

Reply: As we explain in our general response, the existing models in the literature are not sufficiently fine-grained. Despite remarkable advances in recent DTI models, significant challenges remain, particularly in aligning the model design with biomedical principles (token-level interaction). Our proposed solutions can be characterised as conceptual contributions advancing the existing literature through an innovative new framework for the DTI task. Below we list the research questions and how they can be addressed to advance DTI discovery.

1. How can fine-grained representations improve DTI predictions?

Existing models rely on SMILES and amino acid sequences, which lack atomic-level precision. We address this by using SELFIES and structure-aware protein sequences ​(FusionDTI).

2. What is the best way to capture sufficiently fine-grained interactions?

We propose a token-level fusion with pre-trained encoders, which captures token-level interactions overlooked by the existing models in the literature​.

3. Can token-level fusion improve both accuracy and explainability?

Our method demonstrates a superior predictive performance on both in-domain and cross-domain datasets and facilitates the highlighting of binding sites, as shown and validated through case studies​.

Responses to these address critical challenges in DTI modelling, offering a novel, explainable framework for fine-grained interaction prediction and advancing the field.

Questions 3: In the case study for searching for binding sites, FusionDTI-CAN is adopted for comparison with DrugBAN. It seems more reasonable to use FusionDTI-BAN for a fair comparison, which raises confusion. So why not choose BAN as backbone model?

Reply: Thank you for your suggestion. In the case study, we preferred to show whether our proposed model can predict more binding sites. However, we will also compare FusionDTI-BAN with DrugBAN for completeness and put the results in the appendix section of the revised version.

Questions 4: Although the TF module is useful, its computational complexity clearly indicates that it is quite time-consuming. What will happen if the model is faced with larger drug molecules or larger protein sequence datasets?

Reply: For larger drug molecules or larger protein sequence datasets, the inference time will be the same for the TF module, since the output dimensions of the protein encoder and molecular encoder are fixed. We will add a clarification in the revised paper.

Questions 5: It should be clear whether the improvements benefit from the pre-trained language models. The ablation results of w/o LLM pre-trained feature is needed.

Reply: Thank you for your comment. While we did not conduct a specific ablation study without pre-trained features, the comparison between FusionDTI-BAN and DrugBAN indirectly demonstrates the benefits of the pre-trained encoders. FusionDTI-BAN, which leverages pre-trained features, consistently outperforms DrugBAN, which does not. To address the reviewer’s comment, we will also add the explicit ablation results in the appendix section of the revised version.

Weaknesses

Reply: Thank you for your comments. However, we would argue that the work done in this paper is novel. We sincerely hope that our general responses have addressed your concerns.

Questions:

The case example appears overly simplistic. Is this prediction based on in-domain or out-of-domain data? I recommend conducting an analysis of out-of-domain cases. Additionally, the results would benefit from external validation. For instance, performing blind docking studies on the drug-protein pairs could confirm whether they truly interact as predicted, and visualizing the binding sites would provide further insights into the interaction.

Reply: Thank you for your suggestion. To clarify, the three DTI pairs in our case study are based on out-of-domain (cross-domain) data, which means they belong to neither the training dataset nor the validation dataset. They are derived from the Protein Data Bank (PDB), which contains binding sites that wet experiments have validated. Therefore, we do actually compare the predicted binding sites with the ground truth data without additional validation. Note also that as Shown in Figure 8, our proposed model allows us to directly visualise binding sites using attention maps without the aid of docking visualisation tools. We will make this clearer in the revised paper.

Dear reviewers,

We sincerely hope that our responses have addressed your concerns and hope you will consider increasing your score. If we have left any notable points of concern overlooked, we would greatly appreciate your feedback, and we will attend to these points. Additionally, we will incorporate all the suggestions and discussions mentioned in the latest manuscript. Thanks again for your thoughtful review and consideration.