DeltaDock: A Unified Framework for Accurate, Efficient, and Physically Reliable Molecular Docking

Dear Reviewer, thank you for your insightful comments on our dataset, figures, and methods. Below are our responses to your questions.

---

For Weaknesses 1

Thank you for raising this important question. This work utilizes the established approach of training on the PDBbind time-split training set and testing on both the PDBbind time-split testing set and the PoseBusters dataset.

While the time-split strategy of PDBbind, first employed by EquiBind, aims to reflect real-world application scenarios with varying data quality, a detailed sequence identity analysis for this approach was not provided by the authors.

To address this, we employed mmseqs2 to analyze the sequence identity between the PDBbind training and testing sets. Our findings, presented in the table below, indicate an average sequence identity exceeding 0.4 for both test sets. Notably, the "unseen test set," where protein UniProt IDs are absent from the training set, exhibits a lower average sequence identity compared to the complete testing set. This observation sheds light on the significant performance drop observed for all previous GDL docking methods on the unseen test set, highlighting the increased difficulty posed by novel proteins.

Turning to the PoseBusters dataset, a comprehensive sequence identity analysis was conducted by the dataset creators. They further categorized the dataset into three subsets based on sequence identity: 0-0.3, 0.3-0.95, and 0.95-1.0, containing 129, 111, and 188 data points, respectively.

The subsequent table presents a comparative analysis of docking performance across these subsets for various baseline methods. Notably, DeltaDock consistently outperforms other methods across all three subsets, demonstrating robustness against varying levels of sequence similarity.

For Weaknesses 2

Thank you for your valuable suggestions. Here, we update the baselines on the PoseBusters dataset(Fig2b) to be the same as the baselines on the PDBbind dataset (Fig2a). Below, we list the docking success rate (% of RMSD < 2.0 Å) of these baselines on both datasets. We have updated this figure in the PDF we submitted.

For Weaknesses 3

Thank you for your valuable question. In this work, we prioritized efficiency and therefore employed DSDP for pose initialization due to its speed. While other docking methods like VINA and SMINA can provide accurate results, they require significantly more computational time.

To address your point, we conducted additional experiments using VINA and SMINA for initialization. It's important to note that our model was trained on DSDP-generated structures. Even without specific training on VINA/SMINA output, DeltaDock demonstrates an ability to consistently improves performance across all initialization methods, highlighting its effectiveness in refining poses regardless of the initial docking method used.

For Weaknesses 4

Thank you for raising this important point. To assess the performance gains achieved by combining Vina with DeltaDock's pocket prediction, we conducted blind docking experiments using the PDBbind dataset. The results, summarized in the table below, clearly demonstrate that incorporating DeltaDock's CPLA pocket prediction module significantly enhances Vina's blind docking accuracy.

As for where the improvement comes from, we performed a series of in-depth analyses. This included ablation studies, discussed in Section 4.5.2, and additional experiments presented earlier. Our findings consistently indicate that both the pocket prediction module and the refinement module contribute significantly to the observed performance.

For Questions 1

Thank you for your valuable Question. For the ligand graph $\mathcal{G}^\mathcal{L}$ and protein atomic graph $\mathcal{G}^\mathcal{P}$, the distance threshold $cut^{\mathcal{L}}$ is setted as 5.0 Å. For the protein residue graph $\mathcal{G}^{\mathcal{P}\*}$, the distance threshold $cut^{\mathcal{P}*}$ is setted as 30.0 Å. When constructing the protein residue graph, we follow the EquiDock[1] to build the K-NN (k=10) graph.

[1]. Independent SE(3)-Equivariant Models for End-to-End Rigid Protein Docking, ICLR 2022.

---

Once again, thank you for reviewing our paper and providing valuable suggestions, please let us know if you have any further concerns, and we are willing to answer any further questions you have about our paper.

Best regards,

Paper Authors

Dear Reviwer,

Thank you for taking the time to read our response, and provide valuable feedback. Here’s our response to your further concerns:

The PoseBusters results on varying levels of sequence similarity are different from those in the PoseBuster paper.

This discrepancy likely stems from the different versions of the PoseBusters dataset used. Our initial results were based on PoseBusters v1 (428 data examples), while your analysis might be based on PoseBusters v2, a subset of v1 containing 308 data points, divided into three similarity subsets of 109, 76, and 123 data points.

All baseline performance metrics, except for VINA and SMINA, were directly extracted from the PoseBusters paper. We independently ran VINA and SMINA to obtain the generated poses for downstream refinement experiments with DeltaDock. We carefully followed the experimental settings outlined in the PoseBusters paper to ensure consistency and reproducibility.

A bounding box with a side length of 25 Å was created around the centroid of the crystal ligand. 40 poses were generated with an exhaustiveness setting of 32, and the top-ranked pose was selected. For VINA, which only accepts PDBQT files as input, we followed the PoseBusters methodology, preparing ligand PDBQT files with Meeko and protein PDBQT files with the ADFR prepare_receptor script.

For clarity and comparison, the following Table presents the experimental results on the PoseBusters v2 dataset. This table includes both our reproduced VINA results (denoted as VINA) and the VINA results reported in the PoseBusters paper (denoted as VINA*). As you can see, the performance of VINA, SMINA, and DSDP is comparable. While we aimed to replicate the VINA* results precisely, achieving identical performance can be challenging due to potential variations in computational environments. Importantly, DeltaDock consistently maintains a slight performance advantage over VINA*.

This work is actually about pocket prediction and refining the result of an existing docking tool (DSDP), which should be made clearer. It is not proper to conflate these with a new docking framework

We appreciate your point and understand the concern regarding the framing of our work. While we acknowledge it builds upon existing docking tools like DSDP, our primary contributions lie in reframing pocket prediction and developing an effective iterative refinement model that generalizes well across different docking tools. In our response, we have demonstrated its applicability to DSDP, VINA, and SMINA, showcasing its broader impact. For example, the CPLA pocket prediction model can combined with VINA to perform blind docking, and VINA/SMINA poses can be further refined by DeltaDock refinement model.

We believe the evaluation of a work's could be comprehensive. While we have not developed a novel end-to-end docking framework, our framework offers a new perspective on pocket prediction and poses refinement, ultimately enhancing the accuracy and efficiency of molecular docking.

---

Thank you again for your valuable feedback. We hope our revised explanation addresses your concerns and provides a better understanding of our contributions. We believe our work offers valuable advancements in the field and kindly hope you can reconsider its significance.

Best regards,

Paper Authors

Dear Reviewer，thank you for your insightful comments on our methods. Below are our responses to your questions.

For Weakness 1:

Thank you for your valuable comment. FABind is an effective method, particularly its inspiring framework that first predicts pockets and then performs docking. Recently, FABind+, an updated version of FABind, has also been proposed. We compare DeltaDock with both FABind and FABind+ to provide a thorough analysis.

Key Differences and Advantages of DeltaDock:

1. Pocket Prediction: While FABind/FABind+ predict pocket residues, DeltaDock redefines pocket prediction as a pocket-ligand alignment task. This contrastive approach leverages established pocket prediction methods, leading to improved accuracy. As shown below, DeltaDock achieves higher accuracy in predicting ligand binding sites:

   Method	% of DCC < 3.0 Å	% of DCC < 4.0 Å
   CPLA Top-1	54.8	70.0
   FABind/FABind+	42.7	56.5

2. Structural Detail: FABind/FABind+ prioritize speed by focusing on residue-level structure. In contrast, DeltaDock adopts a bi-level strategy, modeling both residue and atomic-level protein structure. This enables a more accurate representation of binding interactions.

3. Physical Constraints: DeltaDock incorporates physical constraints in its training loss and utilizes a fast pose correction step. This ensures the physical validity of predicted poses, a feature absent in FABind/FABind+.

Performance Comparison:

The aforementioned differences translate into superior performance for DeltaDock on both the PDBbind and PoseBusters datasets.

PDBbind: While FABind+ demonstrates strong performance on PDBbind, particularly in achieving a high percentage of predictions with RMSD < 5 Å, DeltaDock consistently achieves higher accuracy in predicting poses with RMSD < 2 Å, a more stringent and relevant metric for accurate binding mode prediction.

PoseBusters: The performance of FABind+ significantly declines on the PoseBusters dataset, which is specifically designed to challenge docking methods with poses that appear reasonable but are physically implausible. Notably, FABind+ achieves a 0% success rate when considering physical validity (PB-Valid). This suggests potential overfitting to the PDBbind dataset. DeltaDock, on the other hand, maintains high accuracy, demonstrating its robustness and ability to generalize to challenging cases.

Conclusion:

While FABind and FABind+ are valuable contributions to the field, DeltaDock's innovative approach to pocket prediction, its consideration of detailed structural information, and the incorporation of physical constraints result in a more accurate and robust method for protein-ligand docking.

For Weakness 2:

Thank you for raising this important point regarding the choice between end-to-end and hybrid frameworks for molecular docking. While end-to-end deep learning models have gained significant traction, their direct application to molecular docking, particularly for site-specific scenarios, presents unique challenges.

For instance, blind docking methods like FABind and FABind+, while innovative, often struggle with site-specific docking. Their reliance on pre-predicted pocket information for feature embedding limits their ability to accurately model the complex interactions within a defined binding site.

DeltaDock, in contrast, adopts a hybrid approach that leverages the strengths of both deep learning and established molecular docking tools. This strategic integration offers several advantages:

• Versatility: DeltaDock seamlessly handles both blind and site-specific docking scenarios, overcoming the limitations observed in purely end-to-end methods.
• Accuracy and Generalizability: By incorporating well-established tools and physics-based principles, DeltaDock achieves superior predictive accuracy and demonstrates robust generalization capabilities across diverse datasets.
• Balance between Speed and Accuracy: While not as rapid as some end-to-end methods, DeltaDock achieves a docking time of 2-3 seconds per ligand, striking a balance between computational efficiency and predictive power. This speed, coupled with its enhanced accuracy, makes DeltaDock a practical solution for high-throughput virtual screening campaigns.

In conclusion, while the pursuit of a fully end-to-end docking framework remains a worthwhile endeavor for the field, hybrid approaches like DeltaDock offer a currently more effective solution for molecular docking.

For Weakness 3:

Thank you for your valuable suggestion. Addressing the complexities of flexible docking is undoubtedly crucial. DeltaDock's framework possesses the flexibility to accommodate such scenarios. One viable approach is integrating sampling techniques specifically designed to account for flexibility, such as DSDP-flex[1]. Furthermore, enabling protein coordinate updates during the structure refinement phase allows for greater conformational exploration. Indeed, expanding DeltaDock to directly incorporate flexible docking is a high priority in our future research endeavors.

[1]. DSDPFlex: An Improved Flexible-Receptor Docking Method with GPU Acceleration, ChemRxiv, 2023.

---

Once again, thank you for reviewing our paper and providing valuable suggestions, please let us know if you have any further concerns, and we are willing to answer any further questions you have.

Best regards,

Paper Authors

Dear Reviewer, thank you for your insightful comments on our papers and methods. Below are our responses to your questions.

---

For Originality 1

Thank you for your insightful observation and suggestion. We acknowledge that Equations 7 and 8 in the manuscript may have been misleading regarding the weighting by interatomic distances. Taking $\hat{m}\_{i,j}$ as an example, this term can be further written as $\hat{m}\_{i,j}=\frac{(x\_i^{(l)} - x\_j^{(l)})}{d\_{i,j}^{(l)} + 1} \phi\_{\hat{m}}(m_{i,j})$. $\frac{(x\_i^{(l)} - x\_j^{(l)})}{d\_{i,j}^{(l)} + 1}$ is a unit vector, whose direction is defined by $(x\_i^{(l)} - x\_j^{(l)})$. Therefore,The actual weighting of $\hat{m}\_{i,j}$ is determined by $\phi_{\hat{m}}(m\_{i,j})$. We will revise the manuscript to explicitly clarify this point.

As for the skip connection (SK), it is an important component for DeltaDock's performance. Here, we present the ablation study of SK in the following table. It is obvious that the model without SK performs poorly.

For Originality 2

Thanks for your valuable suggestion. FAbind+ is an updated version of FABind, with improvements in larger model size, pocket radius prediction, and so on. Despite this improvement, FAbind+ still considers residue structure only. In our work, we emphasize the "difficulties in handling large binding pockets" for methods that take atomic structure into consideration. Here, we analyze the pocket radius predicted by FAbind+ and we find that the radius predicted is generally smaller than 20.0 Å (about 93% data). According to the paper, the radius less than 20 Å will be adjusted to 20.0 Å, which indicates that the radius prediction of FAbind+ only works for about 7% of data points.

Then, we perform experiments on the PDBbind and PoseBuster datasets. According to the result on the PDBbind dataset, although DeltaDock outperforms FAbind+ on % RMSD < 2 Å, we observe the promising result of FAbind+, especially for % RMSD < 5 Å.

However, when it comes to the PoseBusters dataset, the performance of FABind+ drops significantly, which may be caused by the ignoring of atomic structure and physical constraints.

In summary, DeltaDock shows significantly more robust and promising results than FAbind+.

For Quality 1

Thanks for your valuable suggestion. We acknowledge the space constraints and have opted to present a concise table with key metrics in the main manuscript. Detailed tables, encompassing all metrics, are provided in the submitted PDF. We plan to incorporate the full table into the appendix of future versions as well.

For Clarify 1

Thanks for your valuable suggestion. We agree that removing the model blocks would enhance the clarity of our framework figure, and we are actively working on this improvement. As for line 31, we want to express that these methods mainly focus on the blind docking setting, which can be used to find new drugable binding sites and explore unseen proteins. We will revise the sentence in the next iteration of our manuscript to ensure greater clarity and precision.

For Question 1

Thanks for your valuable suggestion. Previously, we directly selected the top-1 poses as the initial conformations. Here, we try to generate diverse poses by sampling different conformations. We sample top-n (n<=10) poses and select the best structures (minimum RMSD) predicted by DeltaDock-SC to calculate the docking success rate. Our findings indicate a significant improvement in the performance of the best-performing poses with an increase in the sampling number. This suggests that DeltaDock exhibits the capability to generate diverse poses.

For Question 2

Thanks for your insightful suggestion. Existing docking methods like DiffDock and FABind+ utilize pose confidence evaluation modules to identify optimal poses from a set of candidates. In our response to Question 1, DeltaDock exhibits the capability to generate diverse poses. At this time, we hypothesize that integrating a confidence model for pose selection after the generation of diverse poses by DeltaDock could further enhance docking accuracy. Inspired by FAbind+ and DiffDock, we propose training a confidence model using a classification loss which aims to predict whether the RMSD between a pose and ground truth pose less than 2.0 Å. The training data for this model would be generated using the trained DeltaDock. We are actively pursuing the implementation of this idea. While we anticipate promising results, please allow us a few more days to finalize our experiments.

---

Once again, thank you for reviewing our paper and providing valuable suggestions, please let us know if you have any further concerns, and we are willing to answer any further questions you have.

Best regards,

Paper Authors

Dear Reviewers,

Please kindly review the authors' response with care, engage in discussions to clarify any uncertainties, and, if necessary, revise your comments accordingly.

Thanks, AC