Group Ligands Docking to Protein Pockets

Thank you for your insightful and constructive comments as well as your appreciation of our work in improving data utilization and model quality by docking multiple molecules to the same protein. Below are some clarifications and answers to your questions. If our response does not fully address your concerns, please post additional questions; we will be happy to discuss further.

Q1: The authors should account for the effect of protein similarity on the results by performing redundancy removal on the test set proteins that are either duplicated or highly similar to those in the training or validation sets. Tools like MMseqs or other alternatives could be used for this process.

Following DiffDock's benchmark, we employed a time-based split for the training, validation, and testing sets. This approach inherently ensures redundancy removal by preventing test set proteins that are duplicated or highly similar to those in the training or validation sets from being included.

Additionally, in Appendix Secion F (Figure 8), we present results from new experiments conducted on the ComBind benchmark. In this setup, proteins are clustered to ensure minimal similarity between groups [1], resulting in 248 experimentally determined ligand-binding poses across 30 proteins representing all major families of drug targets. We compare the performance of DiffDock and GroupBind on these 30 proteins with minimal sequence or structural similarity. Our results show that GroupBind consistently outperforms DiffDock under these conditions, demonstrating the robustness of our model.

Q2: The authors need to explain how the similarity between ligands in the group ligand set impacts the results.

As shown in Figure 4(c), knowledge of dissimilar ligands that can dock to the same pocket provides informative signals to the model, generally resulting in lower docking RMSD (Spearman correlation = -0.39). This suggests that grouping dissimilar ligands benefits our model by enhancing its ability to identify correct protein-ligand interactions, ultimately leading to more accurate docking poses. Additionally, we have included a case in Appendix A to further illustrate the motivation behind our method.

Q3: How would the results of GROUPBIND change if tested on the latest PoseBuster?

In Appendix G, Figure 7, we present a comparison of our model with DiffDock under the PoseBuster benchmark [2]. Our model achieves a higher overall success rate than DiffDock, although it shows a lower PB-Valid success rate. Upon closer examination, we observe that our generated docking poses may contain more unreasonable protein-ligand distances compared to those produced by DiffDock. This could be a side effect of incorporating augmented ligands into our approach.

Q4: In Figure 4 on page 8, does the success rate refer to Top 1 or Top 40? What do “SG” and “AG” specifically mean? These details should be clarified in the figure caption. In Figure 6 on page 8, does the success rate refer to Top 1 or Top 40? The meanings of “NG”, “SG”, and “AG” should also be clarified in the figure caption.

In Figure 4 (b)(c)(d), the success rate corresponds to the Top 40 ligand poses. We have clarified this in the caption by stating that "the best docking pose among 40 candidates is used to calculate the success rate." Additionally, we have explicitly defined the abbreviations in the caption: "w/SG, w/AG, and w/o G indicate the subsets where augmented ligands from the test set, training set, and no augmented ligands are used, respectively. SG, AG, and NG represent the corresponding model performance under these different ligand augmentation settings."

Q5: What does Figure 4 on page 8 illustrate, and where is it referenced in the article?

Figure 3 (we reordered figures in the new version) illustrates an example of docking poses generated by GroupBind and DiffDock, demonstrating that "reference ligands tend to bind the target protein with similar poses, which is effectively captured by GroupBind, resulting in lower average RMSD compared to DiffDock." This figure is referenced in Section 4.2 as: "We also provide examples of docking poses generated by GroupBind compared to DiffDock in Figure 4."

Q6: On line 457 on page 9, the percentage 36.3% is mentioned twice, which could cause confusion and should be clarified.

We apologize for the minor typographical error. It has been corrected to: "When provided with the native reference pocket, the top-1 ligand RMSD is significantly improved to 36.6% and 36.3%, respectively."

References:

[1] Paggi, Joseph M., et al. "Leveraging nonstructural data to predict structures and affinities of protein–ligand complexes." Proceedings of the National Academy of Sciences 118.51 (2021): e2112621118.

[2] McNutt, Andrew T., and David R. Koes. "Open-ComBind: harnessing unlabeled data for improved binding pose prediction." Biophysical Journal 123.3 (2024): 186a.

[3] Corso, Gabriele, et al. "Deep confident steps to new pockets: Strategies for docking generalization." ArXiv (2024).

[4] Buttenschoen, Martin, Garrett M. Morris, and Charlotte M. Deane. "PoseBusters: AI-based docking methods fail to generate physically valid poses or generalise to novel sequences." Chemical Science 15.9 (2024): 3130-3139.

Thank you for your insightful and constructive comments as well as your appreciation of our work in leveraging the insight that similar ligands that bind to the same protein target are expected to binding similarly. Below are some clarifications and answers to your questions. If our response does not fully address your concerns, please post additional questions; we will be happy to discuss further.

Q1: The contributions are inappropriate as written. This is not the first time the concept that multiple similar molecules can be used to enhance molecular docking has been described. ComBind (Paggi et al, 2021; cited in the paper) does exactly that. There are other methods that use this insight in different ways (e.g. selecting poses from ensemble docking). The contributions should be qualified that this is the first end-to-end deep neural network approach to molecular docking that uses this insight.

Thank you for highlighting relevant work [1,2] that utilizes the concept of similar molecules to enhance docking. We have incorporated these references into the Related Work section for completeness. Additionally, we have clarified our contributions as follows: "This is the first time the concept that multiple molecules should yield similar docking poses has been integrated into an end-to-end deep neural network approach for the molecular docking task."

Q2: The results only compare to single-ligand docking. Comparing to ComBind (or OpenComBind if lacking a Schroedinger license) would be more relevant. Can the diffusion model approach make better use of the similarity bias than previous methods? This question goes unanswered. Replicating the ComBind evaluation would provide predefined groups of ligands while also making it possible directly compare to a conventional approach.

Thank you for your valuable suggestion! We have included the experiment on the ComBind benchmark in Appendix F, Figure 8, which includes 248 experimentally determined ligand-binding poses across 30 proteins representing all major families of drug targets. By replicating the the ComBind evaluation framework, as shown, our method consistently outperforms DiffDock across these 30 distinct proteins, benefiting from the insight that multiple ligands binding to the same protein pocket tend to exhibit similar docking poses. However, our model still shows performance gaps compared to ComBind. We attribute this to the more limited generalization ability of diffusion-based generative models (as discussed in [3,4]) compared to classic physics-based docking methods like GLIDE, which is used in ComBind. Combining our framework with a more robust backbone model that offers better generalization remains a promising direction for future work.

Q3:There are issues with using a time split to assess generalizability, but as this is the same split used with DiffDock it is appropriate to use for the comparisons performed here.

There are indeed challenges with using a time-based split to assess generalizability. However, since this split is consistent with the setup used in DiffDock, it is appropriate for the comparisons conducted in this work. By evaluating both our baseline methods and proposed methods under the same setting, we ensure a fair and reliable comparison.

Q4: The overloading of "k" to mean two different things in Fig 2 is confusing.

Thank you for pointing this out. In Figure 2, $i, j, k$ refer to atoms, $L_k$ denotes the k-th ligand, and $G_{L_k}$ represents the graph of the k-th ligand. We have carefully proofread the text to ensure that all notations are used consistently throughout the manuscript to avoid any confusion.

Q5: I found equations (4) and (5) confusing due to the overloading of z - the text says these two separate values (presumably equivalent to AF2 incoming and outgoing triangle attention edges) will be stacked, while the equations say they are summed.

They are stacked together -- the z has already been updated in the module defined by Eq(4), and then it serves as the input of the module defined by eq(5)

Q6: Table 2 is apparently showing Top-10 results, but this fact is only stated in the main text. Why not replicate the reporting in Table 1? "Med." isn't defined, but is presumably median RMSD.

Thank you for pointing this out. We have clarified in Table 2 that "The results of perfect selection under 10 samples are reported." Additionally, we have defined "Median." as Median RMSD and explicitly included the Median RMSD values in Table 2 for consistency with Table 1.

Q7: 4.4: "Section" is used when "Figure" is meant. "ifself"

We have carefully reviewed and updated all references to figures, tables, and sections in the manuscript. "Figure" is now consistently used for images, and "Table" is used for tabular data.

Thank you for your thoughtful feedback and for pointing out an important oversight in our previous response. We apologize for any confusion caused by the omission of key details.

To clarify, ComBind uses the center of mass (CoM) of the entire protein as the docking center. Our earlier interpretation of blind docking was based on the absence of prior binding site knowledge. Since ComBind uses the protein's CoM as the docking center, we initially assumed it does not rely on binding site information. However, upon further reflection, we recognize that the strict definition of blind docking includes an additional requirement: the docking region must encompass the entire protein to allow unbiased exploration of all potential binding sites. By this stricter definition, ComBind would only qualify as blind docking if the bounding box were large enough to cover the entire protein, rather than being restricted to a smaller region (e.g., 15–30 Å). We acknowledge this oversight and thank the reviewer for bringing it to our attention.

It remains notable that ComBind achieves such strong results without prior binding site information. We agree that aligning our method's ligand and pocket selection criteria to ComBind would strengthen this aspect of our study. If the paper is accepted, we will revise the manuscript to include these details, ensuring greater transparency and addressing any potential confusion.

We sincerely appreciate your constructive feedback and will incorporate the necessary revisions to enhance the clarity and rigor of our work.

Thank you for your insightful and constructive comments as well as your appreciation of our work in leveraging other ligands that bind to the same pocket to enhance the binding prediction. Below are some clarifications and answers to your questions. If our response does not fully address your concerns, please post additional questions; we will be happy to discuss further.

Q1: Why are the results of DIFFDOCK in Table 1 worse than those in the original paper, and it seems that the bold text annotations might be inaccurate?

We strictly adhered to the sampling procedure outlined in the DiffDock repository (https://github.com/gcorso/DiffDock) to generate the results presented in our manuscript. To facilitate further examination, we have provided the sampling results and a simple notebook for quick evaluation in this anonymous link: https://drive.google.com/file/d/1awPMogbZpXOBIEo1YpzWWLkZpP3lRIgI/view. One possible cause of discrepancies may stem from RDKit initialization issues, which could impact the accuracy of local structures. However, since both our model and DiffDock were evaluated under the same machine setup, we believe our comparison is fair and reliable. We are actively investigating the root cause of the performance discrepancy and will update both the DiffDock results and our own in Table 1 once we have further insights.

Regarding the bold text annotations, we refrained from comparing GroupBind-Ref to other baselines to maintain a fair comparison, as it leverages additional reference pocket information. The bold text in Table 1 highlights the best-performing model within the blind docking setting. We have updated the caption of Table 1 to clarify this distinction.

Q2: From Figure 1, we can see that molecules binding to the same pocket indeed have similar structures, but how many pockets in the dataset exhibit this situation? Is there any statistical data on the number of pockets and the corresponding similar ligands in the PDBBind dataset?

In Figure 6(a), we present a histogram showing the number of ligands per pocket. There are 7697 unique pockets for the 19k complexes in the PDBBind dataset. Among these, 3,681 pockets (47.8%) contain more than one ligand. Additionally, Figure 6(b) illustrates the Tanimoto similarity of ligands within each group (with two or more ligands). This analysis shows that while some pockets are indeed bonded with similar ligands, a larger portion of pockets contain ligands that are dissimilar. Despite this, our model effectively handles such cases. Specifically, as shown in Figure 4(c), the presence of dissimilar ligands binding to the same pocket provides valuable information to our model, leading to improved docking accuracy and generally lower RMSD values (Spearman correlation = -0.39). This suggests that grouping dissimilar ligands is advantageous for our model in discerning correct interactions with proteins, ultimately yielding more accurate docking poses.

For further illustration, we have added a case study in Appendix A Section A to clarify the motivation behind our method. Specifically, PDB entries 6g2c and 6g29 contain distinct ligands (Tanimoto similarity = 0.26) that bind to the same pocket. Despite their differences, several key interactions are conserved: for instance, a hydrogen bond is formed between N3 in 6g2c / N12 in 6g29 and an oxygen atom in the side chain of a pocket residue, while π-stacking interactions occur between the ring C7C8N9C10N11 in 6g2c / ring C8N7N6C4C3C2 in 6g29 and two benzene rings in the side chains of pocket residues. These conserved interactions ultimately result in similar binding poses for these distinct ligands.

Q3: During inference, when searching the database for ligands similar to the query ligand, how many entries in the test set can retrieve similar ligands? If similar ligands cannot be retrieved, does that mean the model becomes ineffective?

As described in Section 4.1 (Experimental Setup), the 363 test ligands are organized into 86 groups containing more than one ligand (255 ligands in total) and 108 groups with a single ligand. By expanding the augmented ligand database to include the training set, we can construct additional ligand groups for 32 out of the 108 single-ligand groups.

In cases where binding information is unavailable for a target protein, an alternative strategy is to use ligands that bind to homologous proteins, as discussed in Section 4.5, where we also observe performance improvement. If neither ligands binding to the same protein nor those binding to homologous proteins are accessible, our approach defaults to the backbone model (DiffDock in this study). This fallback ensures that the model remains applicable and that performance is minimally impacted.

Q4: The reported top-1 docking success rate for DiffDock (32.4% with 40 samples in Table 1) appears considerably lower than previously reported results (38.2% in [4] and 36.0% in [5]). This discrepancy requires clarification.

We strictly adhered to the sampling procedure outlined in the DiffDock repository to generate the results presented in our manuscript. To facilitate further examination, we have provided the sampling results and a simple notebook for quick evaluation in this anonymous link: https://drive.google.com/file/d/1awPMogbZpXOBIEo1YpzWWLkZpP3lRIgI/view. One possible cause of discrepancies may stem from RDKit initialization issues, which could impact the accuracy of local structures. However, since both our model and DiffDock were evaluated under the same machine setup, we believe our comparison is fair and reliable. We are actively investigating the root cause of the performance discrepancy and will update both the DiffDock results and our own in Table 1 once we have further insights.

Q5: The clarity of the writing could be improved. For instance, "beyound" on line 101 should be corrected to "beyond." A comprehensive review of the entire text is recommended.

We have conducted a thorough proofread of the manuscript to address spelling and grammatical errors, including correcting "beyound" to "beyond". All sections have been carefully reviewed to improve overall language quality. We hope this revised version is clearer and easier to follow.

Q6: The definition of "ligand ground graph" needs explicit formalization. The current lack of clarity makes understanding this crucial concept challenging. For example, when referring “noisy group ligands” in line 214, it is difficult to understand this concept.

We have clarified the term "ligand ground graph" to explicitly describe its meaning: "We construct a graph across ligands in the group by placing all group ligands in the same coordinate system and adding edges between different ligands."

We have replaced the term "Noisy group ligands" with "diffused group ligands" to more accurately reflect their role in the training process. Specifically, "diffused group ligands" refers to "the ligands after noise is added during the diffusion process."

Q7: The references to tables and figures are confusing. Avoid using "Section" to denote figures, tables, and section of manuscript at the same time. Use "Figure" when referring to images (e.g., Figure 1, Figure 2) and "Table" for tabular data (e.g., Table 1, Table 2).

We have carefully reviewed and updated all references to figures, tables, and sections in the manuscript. "Figure" is now consistently used for images, and "Table" is used for tabular data. This ensures clarity and avoids confusion for the reader.

Q8: Figure 3's inclusion of Figures 3, 4, 5, and 6. For better visual organization and clarity, these figures should be presented as subfigures within a single figure (e.g., Figure 3a, 3b, 3c, and 3d).

We have renumbered Figure 4 including 4a,4b,4c and 4d (as we reordered the figures in the revised version for better visual flow) to ensure consistent labeling throughout the manuscript. The numbering of subsequent figures has also been updated accordingly.

Q9: Lacking explanation of "NG" in Figure 3d. While the meaning of "NG" in Figure 6 might be discernible from the main text, its meaning should be explicitly stated. This ensures clarity and avoids requiring readers to search through the text for an explanation. A concise definition or clarification of "NG" within the caption is crucial.

We have added a new caption to clarify the meaning of "NG" directly in Figures 4d and 6. Specifically, "w/SG, w/AG, and w/o G" indicate subsets where augmented ligands from the test set, training set, and no augmented ligands are used, respectively. Similarly, "SG, AG, and NG" represent the corresponding model performance under these different augmentation settings: test set, training set and no augmented ligands. This addition ensures clarity and eliminates the need for readers to search the main text for definitions.

Thank you for your insightful and constructive comments as well as your appreciation of our work that leveraging similar binding poses among ligands targeting the same protein is intriguing and biologically relevant. Below are some clarifications and answers to your questions. If our response does not fully address your concerns, please post additional questions; we will be happy to discuss further.

Q1: MCS docking assumes that ligands with similar substructures exhibit similar docking poses, while GroupBind assumes all ligands share similar docking poses. Conversely, Figure 3 illustrates cases where structurally distinct ligands adopt distinct poses. A statistical comparison quantifying the difference between the MCS docking assumption and GroupBind's assumption is warranted.

This is a very good question! To clarify the motivation behind our method, we have added an example in Appendix A Section A. Specifically, PDB entries 6g2c and 6g29 contain distinct ligands (Tanimoto similarity = 0.26) that bind to the same pocket. Despite their differences, several key interactions are conserved: for instance, a hydrogen bond is formed between N3 in 6g2c / N12 in 6g29 and an oxygen atom in the side chain of a pocket residue, while π-stacking interactions occur between the ring C7C8N9C10N11 in 6g2c / ring C8N7N6C4C3C2 in 6g29 and two benzene rings in the side chains of pocket residues. These conserved interactions ultimately result in similar binding poses for these distinct ligands.

In Appendix B, Figure 6(b), we present the average Tanimoto similarity of ligands within each group, revealing that while some pockets do bind similar ligands, a larger number accommodate dissimilar ones. And as demonstrated in Figure 4(c), grouping dissimilar ligands helps our model better capture relevant protein-ligand interactions, leading to more accurate docking poses. In Section 4.5, we further highlights the improvement provided by our model even when augmented ligands are unavailable.

Overall, our approach relies on a more general assumption than the MCS docking assumption [1,2], and our model—leveraging the capabilities of deep neural networks—effectively handles these more diverse scenarios. DeltaDock's Figure 4 [5] illustrates cases where structurally distinct ligands adopt distinct poses, but the context is how to handle "large" binding pockets, which is a valuable finding and doesn't contradict with our assumption. Studying how to better handle large binding pockets in our framework would be an interesting future direction.

Q2: Comparing GroupBind-Ref against blind docking methods like DiffDock is unfair.

We did not compare GroupBind-Ref with DiffDock, as GroupBind-Ref utilizes additional reference pocket information. The bold text annotations in Table 1 indicate the best results achieved in the blind docking setting. We have updated the table caption to clarify this. GroupBind-Ref variants are not taken into consideration for a fair comparison.

Q3: The evaluation should include more baselines such as FABind and FABind+ , for which source code is available. Expanding the evaluation to include datasets like PoseBuster would assess GroupBind's ability to predict physically plausible structures.

Thank you for highlighting FABind [3] and FABind+ [4]! We appreciate the suggestion and have now included these works in the Related Work section to acknowledge their contributions. However, since our model is based on a diffusion-based generative docking framework, we chose not to include FABind and FABind+ in the main results table to maintain a fair comparison. Integrating our framework with other cutting-edge docking methods is an exciting direction for future work. Given that our model demonstrates significant improvements when combined with diffusion-based docking approaches, we believe our results showcase both the novelty and promise of our method.

Additionally, we have compared our model with DiffDock on the PoseBustesr benchmark [6] in Appendix Section F. Our model achieves a higher overall success rate than DiffDock, though with a lower PB-Valid success rate. Upon closer investigation, we found that our generated docking poses may contain more instances of unreasonable protein-ligand distances than those generated by DiffDock, which could be a side effect of incorporating augmented ligands.