Streamlining Generative Models for Structure-Based Drug Design

Thank you for your feedback and constructive questions. We address them below.

Firstly, it is important to note that the method did not actually achieve de-novo drug design, as it sampled the unlabeled ligand graph from an existing dataset. This raises questions about the novelty of the generated drug.

This is a good observation. We would like to point out that we compare with other baselines also in terms of chemical diversity (main results Table 1) and obtain comparable results (68% vs 72%), where diversity is defined as 1 -tanimoto similarity as in all other baselines [1, 2].

Novelty of the generated molecules. The novelty of generated molecules with our method depends on which variant is used. The FastSBDD-$\mathcal{DR}$ described in Section 3.3.1 does not generate novel molecules, it is included to showcase methods capability for drug repuprosing. The other variants, however, FastSBDD-$\mathcal{ND}$ and FastSBDD-$\mathcal{PO}$ achieve 100% novelty, meaning that none of the generated molecules are present in the training sets. This shows that these two variants do in fact perform de-novo drug design.

Furthermore, there is a lack of clarity regarding whether bond types are preserved in the unlabeled graph. Even if bond types are not preserved, it is not intuitively clear whether there is enough flexibility for atom type when the graph structure is fixed.

Bond types are indeed preserved in the unlabeled graph. We will make it more explicit in the paper. There is substantial flexibility for atom types with the fixed graph structure. We discuss in more detail below.

In light of these concerns, the focus on overall diversity is not as meaningful as it could be. It would be more informative to assess and showcase the diversity of generated ligands for each template unlabeled graph. This analysis can reveal the extent of chemical variation achieved by the method, which is a more relevant measure of its potential impact. If the changes between the generated ligands and the template ligands are minor or do not significantly affect the chemical properties, the overall impact of the method may be considerably diminished.

Chemical diversity of our generated molecules is significant. Thank you for pointing this out. There is in fact substantial variability in the chemical space even with fixed 2D unlabelled graph structures.

New experiment based on your feedback. In this experiment, we show that by only changing atom types, we can generate hundreds (and in 98% of cases at least a thousand) novel molecules, which are almost as diverse as the ZINC250k dataset itself. Please see the General Response for details.

Without more evidence and explanation regarding the mentioned issues, I think comparing the drug repurpose-like generating method with the de-novo generating method is not fair enough. It might be more appropriate to compare it with other drug repurposing methods

Our scoring model enables both repurposing and new drug generation. We note that the FastSBDD-$\mathcal{DR}$ variant of our method, designed for drug repurposing, was meant to demonstrate that our method can have wider applications. Due to our scoring model, we are able to perform both drug repurposing as well as novel drug generation with the latter being the primary concern. This is why we benchmark our method against the generative models for SBDD.

Thanks again for your review and excellent questions. In response to your concerns, we have provided additional empirical evidence which we hope has been convincing. We would appreciate it if you could reflect this in an updated score.

[1] Peng, X. et al., Pocket2Mol: Efficient Molecular Sampling Based on 3D Protein Pockets, ICML (2022)

[2] Schneuing A. et al., Structure-based Drug Design with Equivariant Diffusion Models, arXiv preprint arXiv:2210.13695 (2022)

We once again thank the Reviewer for the review and insightful feedback. We believe that our response and especially the new analysis demonstrating chemical variability of molecules with fixed 2D unlabelled structures have addressed the raised concerns and would therefore kindly request re-examination of our work. If any further inquiries or issues persist, we encourage you to bring them to our attention for swift resolution.

Thank you for your feedback and comments. We address them below.

The rationality of the two-stage method is suspectable. In general, especially from the perspective of structural biology, the protein-ligand interaction is strongly dependent on atom types, positions. However, in this paper, the only dependence on proteins is used to infer the unlabeled molecular graph where only topology information is used. And the second stage does not involve modelling the protein pockets. This contradicts with the fundamental knowledge of structural biology.

This is a great point. We agree that this model definition and results are counter-intuitive and we believe that therefore it is especially important to publish such findings. Please see our General Response where we re-emphasize points made in the paper and discuss this model definition.

The main concern about the experiments is about the fairness of comparison with baselines. First, FastSBDD leverages additional molecular dataset, such as ZINC and ChemBL, while other baselines does not use additional datasets

This is a good point. However, it is important to note two things. First, the additional datasets, such as ZINC and ChemBL only contain molecules, they do not contain molecule-protein pairs indicating strong binding of a given molecule to a given protein pocket as in the CrossDocked dataset used by SBDD methods.

Second, our method is conceptually different than other benchmarks. Our approach with utilizing additional molecular datasets is novel in the SBDD domain and a part of our contribution. It is unclear how other baselines could benefit from these additional data.

Second, optimization is condered in FastSBDD. But the baselines are all generative methods. No optimization methods, such as MARS [1] and RGA [2], are included. From my perspective, it is more proper to compare FastSBDD with molecular optimization methods. RGA is a molecular optimization method proposed for SBDD.

Thank you for pointing this out. Please see our general response, where we discuss comparison with optimization-based methods.

Besides, many important SBDD baselines are missing, such as DecmopDiff [3]

In the comparison, we have included a recent SBDD baseline: TargetDiff (ICLR 2023). DecompDiff was published in a peer reviewed venue in June 2023 (ICML 2023), making it too recent (after 28.05.2024) to be expected to include in the analysis as per ICLR 2024 guidelines. DecompDiff uses a different software for estimating binding affinity (as indicated e.g. by different results for Pocket2Mol than originally reported), so it is not straightforward to compare with our method without recomputing the numbers ourselves. We will try to do that before the end of the discussion phase.

Strong performance compared to DecompDiff. DecompDiff takes on average 6189s to generate 100 molecules per protein pocket, making it 1.8x slower than the slowest baseline TargetDiff. Furthermore, as we argue in the paper, we believe that the ultimate goal of SBDD should be to focus on both the binding affinity and other molecular properties. DecompDiff demonstrates subpar performance in terms of most commonly used molecular properties:

• QED: FastSBDD ∈ [0.61, 0.8]; DecompDiff = 0.45
• SA: FastSBDD ∈ [0.69, 0.78]; Decompdiff = 0.61 Other metrics like LogP or Lipinski are not reported.

The main questions is about how much protein-ligand interation information is included in unlabeled graphs. More analyses are needed.

Empirically, we have shown that using the unlabelled molecular graphs alone is enough to predict binding scores with accuracy high enough to outperform many SBDD baselines. This is an interesting and unexpected result, especially given that this behaviour was exhibited not only by the Vina Score, but also Gnina, a newer and reportedly, significantly more accurate docking software (that we mention in Appendix A.4).

We believe that this opens up further questions about the design of SBDD methods and development of docking software, but an extensive analysis from the structural biology point of view is not in the scope of this study.

Thanks again for your feedback. We hope that our response and clarifications helped clear your concerns and you would reconsider and amend your original score.

Thank you once again for your detailed review and constructive feedback. We feel that our response and especially the clarifications of the baselines comparisons have sufficiently addressed your concerns and would thus kindly request a re-evaluation of our submission. Should there be any other concerns, questions or requests for clarifications, we are eager to respond.

We thank the Reviewer for a careful review and raising important questions. We address them below.

The proposed disentanglement doesn’t make much sense to me. I wonder whether the authors redock molecules with AutoDock Vina in their preliminary experiments (Sec 3.1 and Appendix A). I suspect that changing initial 3D conformation can also achieve a high $R^2$ of Vina score is caused by redocking. Otherwise, the Vina score should have a large change.

Role of redocking. Yes, redocking is precisely the reason why coordinates have marginal effect on the Vina Score. The Vina Score that we and other baselines use [1, 2] for estimating docking scores does use redocking. We make it explicit in Eq (16) in Appendix A.1 where we provide a detailed definition of the Vina Score.

The assumption justification is needed about why the unlabelled graph sampler could be independent of protein

Justification for assumption about conditional independence. We assume the Reviewer meant ”why the atom sampler could be independent of protein”, because the unlabelled graph sampler is not independent of the protein. Eq (3) explicitly shows the dependence of the unlabelled graph sampler on the protein.

The conclusion that the atom sampler can be effectively taken to be independent of the protein is drawn from the experiments mentioned in Section 3.1 and detailed in Appendix A. This choice is motivated by empirical findings based on the software approximating the docking scores. Please refer to our General Response for more details.

The proposed method has many filtering operations (filter high-affinity molecules with binding affinity scoring model, filter high QED+SA molecules in the atom type assignment phase) during the generation process. It would be fairer to compare with optimization-based models or compare with generative models and apply a similar property screening operation as post-processing.

This is a great point. We view our method as bridging the gap between the generative optimization-based models. To the best of our knowledge, we are the first to introduce the explicit optimization of binding (and other molecular properties) to a generative model without having to rely on expensive computations. On the contrary, our method is lightning fast compared to baseline generative models for SBDD, which are themselves faster than optimization based models. Please see our General Response for more details.

Post-hoc screening would result in an even more significant relative speedup. Regarding screening predictions of generative models as post-processing, the scoring model that we use for filtering is proprietary and is part of our contribution. If we compared other baseline models and filtered their predictions using the Vina software this would come at a cost. Scoring a single protein-ligand pair takes on average 16 seconds. Assuming that we generated double the number of predictions, scored them all with Vina and filtered out bottom half, this would quadruple the runtime of the fastest baseline, which already is two orders of magnitude slower than the slowest variant of our method. It is also not obvious whether optimization for binding alone would not hurt other molecular properties. We demonstrate with our method that we can do both: outperform baselines in terms of both binding and other molecular properties.

The proposed method can not generate novel 2D graph structures. I’m not sure whether it’s a real limitation. If the authors can justify all 2D graph structures can be covered by the ligand database, it’s also fine.

Thank you for the opportunity to clarify this. As we mention in Section 3.2, it is possible to extend the method to allow for sampling entirely new 2D graph structures, but we decide not to do that. We note that there is substantial variability in the molecular space even with a fixed set of 2D unlabelled graph structures. This is supported by the fact that our method has comparable chemical diversity to other methods as shown in Table 1.

Diversity of the generated molecules. To demonstrate the diversity of the molecular space with fixed 2D unlabelled graph structures, we conducted an additional experiment. We showed that we can generate hundreds (and in 98% of the cases at least a thousand) of novel and chemically diverse molecule for a fixed 2D unlabelled structure. Please see our General Response for more details.

Regarding the novelty of generated molecules, it is true that we cannot generate unseel 2D graph structures, but the FastSBDD-$\mathcal{ND}$ and FastSBDD-$\mathcal{PO}$ variants of our model achieve 100% novelty, meaning that none of the generated molecules are present in the train set.

We would like to thank the reviewer again for the thorough review and insightful questions. We believe that our response together with the additional analysis conducted have addressed the concerns and would kindly request re-evaluation of our work. If there are any further concerns or questions, we are happy to address them.

Thank you for your feedback. We address your concerns below.

The writing of this paper is not very clear. For example, one of the assumption is the conditional independence in Eq (1), however this equation is not self-inclusive. It’s not clear what the model assumptions are and what the conclusions are.

Eq (1) emphasizes that our model assumes that atom labels $\mathbf{a}^M$ are conditionally independent from the protein graph $G^P$ given the unlabelled graph $U^M$ . As we mentioned in this section, the conclusion is that we can independently train the unlabelled graph sampler model, which is conditioned on the protein context and the atom sampler, which is only conditioned on the unlabelled graph. Empirically, we verify that this formulation allows for a significant speed-up in computation without sacrificing binding performance.

My major concerns is that I think the methodology lacks novelty. Most of section 3 is spent on describing what architectures are used to model certain conditional probabilities. It’s not clear what is this paper’s techinical contribution

Recap of our contributions. We list our key contributions below:

• (Novelty of our metric aware optimization.) We are the first to introduce drug generation with metric tuning in the Structure-based drug design domain (SBDD). Previously, generative models were trained to match the data distribution, but evaluated against metrics such as binding affinity, which are not taken into account during training.
• (Practical impact.) Our SBDD method is both very accurate while also up to 1000x faster.
• (Theoretical contribtutions.) We include novel theoretical analysis of representational capacity and generalization in the context of SBDD.

Section 5 describes theoritical analysis, but I don’t see how it is connected to the main claim of this paper. For example, section 5.2 talks about generalization bound, the author may want to show that the model has good generalization ability with respest to some terms (L, m, k) in Eq (14)

Relevance of our theoretical analysis with respect to rest of the paper. Our approach differs from others in the SBDD domain in that we show that with significantly reduced models we can outperform other methods whilst being much more computationally efficient. Apart from the empirical evidence that we provided, we include a theoretical analysis, which emphasizes the point that increasing model’s size

• does not solve some representational challenges, as there are graphs that cannot be modeled accurately regardless of the depth of the model (Section 5.1)
• comes at a cost of generalization. The larger the models are the less likely they are to generalize well to unseen data distributions. (Section 5.2)

To strengthen our generalization claim, we provide empirical evidence at the bottom of section 4 that our model generalizes well when we change the molecular dataset that we use for sampling.

Eq (1), I think the term ”disentengled” is inappropriate here. It’s just conditional independence, disentenglement usually refers to something else.

Thank you for your suggestion. We agree and will change the wording to conditional independence.

Thanks again for all your questions. We hope that our response and clarifications have sufficiently addressed your concerns and you will consider updating your score.

We agree with Reviewer aZds that these findings are very unexpected and counter-intuitive from the perspective of structural biology. We believe that therefore it is especially important to publish such findings as they have the potential to inspire progress both in the SBDD domain as well as the development of more accurate docking software. Wet lab experiments and the analysis from the structural biology point of view were out of scope of this study.

Comparison with optimization based methods

Our method for generating a molecule likely to bind to a protein begins with sampling proposal unlabelled 2D graph structures and filter out those unlikely to bind according to our scoring model. Reviewers zMwy and aZds suggested that it may be more appropriate to then compare with optimization based methods rather than generative models for SBDD. We address this in two parts.

Novelty. First, to the best of our knowledge, we are the first to introduce explicit optimization for protein-ligand binding in a generative model for SBDD. Further, we demonstrate how our model definition allows for simultaneous optimization of binding and other molecular properties. The scoring model is an integral part of our method.

Massive computational speedup. Second, the optimization based methods are very computationally expensive. The ones proposed by Reviewer aZds, MARS and RGA, require 3-4 hours of GPU computations to generate predictions for a single protein pocket compared to 2 minutes of CPU time for all three variants of our method combined. To even include both these benchmarks in the comparison, one needs to generate predictions for 100 protein pockets, requiring 1 month of GPU computations. A significant advantage of our method is its ability to very quickly generate molecules likely to bind to a given protein without even needing a GPU.