DecompOpt: Controllable and Decomposed Diffusion Models for Structure-based Molecular Optimization

Q6: "Criteria for success rate seem arbitrary. How were these thresholds chosen?"

A6: We want to clarify that the thresholds of QED, SA, and Vina as criteria for success rate are not arbitrary but instead traceable. The Vina threshold (< -8.18 kcal/mol) corresponds to 1uM binding affinity, and it is a common requirement for a potential drug candidate in practical drug discovery; the QED and SA thresholds are calculated as the 10th percentile of DrugCentral [1] (a database of up-to-date drugs and pharmaceuticals), to reflect the latest drug property distribution. These thresholds are also used by [2] and [3].

Q7: "Where does the difference for the DecompDiff results between Tabs 1 and 3 stem from?"

A7: Following DecompDiff, we use different decomposed priors for ablation studies. Opt Prior is used in Tab 1 and Ref Prior is used in Tab 3. Different kinds of decomposed priors are defined by DecompDiff.

Ref Prior is calculated using a Gaussian distribution based on the reference molecule, determined through maximum likelihood estimation. Pocket Prior employs AlphaSpace2 [4] for identifying subpockets, which is used for estimating the prior center. Additionally, it uses a neural classifier to estimate the number of ligand atoms and the prior's standard deviation. Opt Prior adopts Ref Prior when the reference ligand meets the required success criteria. Otherwise, it employs the Pocket Prior.

Q8: "Is DecompOpt applicable if no/only few reference ligands are available? How does the performance depend on the number / performance of initial reference ligands?"

Thank you for your question. As clarified in Q5, reference ligands are not used as initial conditions for DecompOpt. Sample results provided in Table 1 initialize the arm lists from 20 ligands generated by DecompDiff. We further run the experiments with initial the arm lists of 1 and 5 ligands generated by DecompDiff. The results are shown as follows.

The results indicate that the initial number of ligands has a modest impact on the optimization outcomes, with a higher number of ligands generally leading to improved performance. Notably, even when starting with a single ligand generated by DecompDiff, DecompOpt demonstrates a higher success rate compared to other baseline models.

References:

[1] Ursu, Oleg, et al. DrugCentral: online drug compendium. Nucleic acids research (2016).

[2] Long, Siyu, et al. Zero-Shot 3D Drug Design by Sketching and Generating. NeurIPS (2022).

[3] Guan, Jiaqi, et al. DecompDiff: Diffusion Models with Decomposed Priors for Structure-Based Drug Design. ICML (2023).

[4] Katigbak, Joseph, et al. AlphaSpace 2.0: representing concave biomolecular surfaces using β-clusters. Journal of chemical information and modeling (2020).

Thank you for your constructive comments. Please see below for our responses to the comments.

Q1: "Confidence intervals in evaluation are missing."

A1: We have the confidence intervals in the appendix in the revision. Please refer to Appendix C.1 for the boxplot of the evaluation metrics.

We have conducted paired sample t-test for success rate of DecompOpt and its counterpart, DecompDiff. The p-value of the success rate difference between DecompOpt and DecompDiff is $1.83\times 10^{-17}$, which shows that the optimization works as we expected and demonstrates the significant superiority of our method.

The results show that the superiority of our method is consistent.

Q2: "The overall clarity of the text should be improved (typos/grammar/…)."

A2: Thanks for pointing this out! We will carefully proofread our manuscript and fix all the typos.

Q3: The complex framework is not described very well and is hard to grasp, such as the model architecture and loss function. "An overview figure might help to better show the overall framework and identify novel contributions."

A3: We have added more explanation of the model design and a more detailed overview figure in Appendix A.

Q4: "Is this just guiding towards similar arms, or is the sub-pocket important here? An additional ablation using only the arms, A_k = Enc(A_k), would be informative."

A4: Thanks for your insightful questions. The results with only arms and without subpockets are as follows. Recall that we use SE(3)-invariant features of arms (and subpockets) as conditions. Without subpockets, this feature would be agnostic to the molecular interaction and spatial relation between the arms and subpockets. Such information is important to some of the properties (e.g., Vina scores). The SE(3)-invariant features from pairs of subpockets and arms contain the aforementioned information and are better aligned with the protein-ligand complex being generated. The results indicate that DecompOpt, when solely with arms as conditions, is capable of optimizing all metrics. However, its efficiency in this scenario is not as well as DecompOpt that utilizes both arms and pockets as conditions.

Q5: It is unclear / not evaluated how much the method relies on the availability of high-quality reference ligands vs the generative capabilities of the diffusion model.

A5: We believe there might be a misunderstanding here. We do not use the reference ligands in the test set as conditions. Instead, we initialize the conditional arm lists with ligands generated by DecompDiff. We also ablate on how the quality of the initial ligands impacts the performance of DecompOpt.

We conducted an ablation study focusing on Vina Min score optimization, using ligands with high and low Vina Min scores as initializations for the arm lists. Due to limited resources, we chose to conduct this study on the protein 2v3r, which is randomly chosen from our test set. We generated 100 ligands using DecompDiff and selected 20 ligands with the highest and lowest Vina Min scores. These ligands were then used as the initial conditions for the optimization process.

As shown in the experiment result, the optimization outcomes are slightly influenced by the quality of the initial ligands. However, regardless of the quality of the initial ligands, DecompOpt can consistently improve the quality of the generated ligands.

Thank you again for your positive feedback and recognition of our efforts!

Q4: Evaluation of rationality of the generated conformations. The energy difference before and after force field optimization.

A4: Thanks for your valuable feedback!

We have already evaluated the rationality of generated conformations in Appendix B. More specifically, we compute atom-pair distance, different bond distance, different angle distributions of generated molecules and compare them against the corresponding reference empirical distributions. In most cases, our model performs better than all of the baselines.

Following your suggestions, we also evaluate energy difference (kcal/mol). We optimize the generated structures with Merck Molecular Force Field (MMFF) [3] and calculate the energy difference between pre- and pos- MMFF-optimized coordinates for different rigid fragments (consisting of 3/4/5/6/7/8 atoms) that do not contain any rotatable bonds. The results are as follows:

As the results indicate, DecompOpt achieves low energy differences and outperforms baselines in most cases.

We also calculate the energy difference before and after force field optimization for the whole molecules with 1/2/3/4/5/6/7 rotatable bonds respectively. The results are as follows:

Notably, although we do not explicitly introduce optimization over conformation, DecompOpt outperforms the other diffusion-based models by a large margin.

We also show the corresponding box plot in Appendix B in the revision. These results show that the conformation of ligands generated by DECOMPOPT is high-quality and stable.

Q5: “The second case in Figure 3 seems questionable or may be difficult to interpret.”

A5: Thank you for your suggestion. In R-group optimization, we iteratively modify R-groups for higher Vina Min scores, leading to improved binding affinities. While the changes in atoms and bond types might appear subtle, their effects with positional adjustments significantly impact binding affinity, as demonstrated in Figure 3. For example, for ligands of protein 4F1M (see the bottom left and bottom right of Figure 3), the Carbon atom connecting two Phosphorus atoms in R-group of the reference ligand is replaced with the Oxygen atom in the optimized ligands. We redraw Figure 3 in the revision for clarity. For broader validation, we've also conducted additional experiments on protein 4G3D, which is provided in Appendix D.1.

Q6: Changes in molecular properties (vina score, for example) after the hopping process.

A6: Scaffold hopping aims at finding scaffold structures that can connect existing functional groups without disrupting their interactions with the target protein. The main purpose of this is to find novel scaffolds which are not protected by existing patents while maintaining comparable properties as the original molecule. Therefore, we did not implement property optimization mechanisms in scaffold hopping tasks and solely focused on designing scaffolds that can connect existing arms. We provide the property comparison before and after scaffold hopping in the table below.

The evaluation results demonstrate that the properties of the ligands remain relatively consistent before and after the process of scaffold hopping.

References:

[1] Long, Siyu, et al. Zero-Shot 3D Drug Design by Sketching and Generating. NeurIPS (2022).

[2] Guan, Jiaqi, et al. DecompDiff: Diffusion Models with Decomposed Priors for Structure-Based Drug Design. ICML (2023).

[3] Halgren, Thomas A. Merck molecular force field. I. Basis, form, scope, parameterization, and performance of MMFF94. Journal of computational chemistry (1996).

Thank you for your feedback. Please see below for our responses to the comments.

Q1: Misalignment between the motivation and the approach. The importance of property is not directly addressed. "Which part of the article is designed to address QED or easily synthesizable compounds?"

A1: We would like to clarify that our model, DecompOpt, while property-agnostic in its core design, is adeptly complemented by a property-oriented optimization process. This combination is key to the success of our method and facilitates the generation of compounds with specifically desired properties such as QED and SA.

The iterative optimization is achieved by high-quality arm lists, which are curated using a z-score metric. As described in Section 4.1 (Implementation Details), this metric comprehensively evaluates QED, SA, and Vina Min, ensuring that the optimization process is aligned with the desired molecular properties. So our motivation and our approach are actually quite aligned.

Q2: "The Vina score from Table 1 appears to be subpar." The motivation for using Vina scores for optimization and data analysis (Figure 1). Why are the results of Vina min emphasized but not the median or some other indicators?

A2: Thank you for bringing up this concern. Indeed, this brings up a crucial aspect that merits clarification. To properly articulate our approach, it is essential to first distinguish between Vina Score, Vina Min, and Vina Dock, all computed by AutoDock Vina software.

• Vina Score predicts the binding affinity of a ligand to a target protein directly.
• Vina Min is calculated by a process where, after an initial docking prediction, further subtle refinement is done using force field-based optimization around the binding site.
• Vina Dock is obtained after re-docking a ligand into a protein's binding site.

All the three metrics are commonly used in estimating binding affinity in recent SBDD studies. We chose Vina Min as our optimization objective due to its capacity to reflect potential binding affinity with local optimization, aligning with our method’s focus on arm optimization without incurring a high time cost. Other binding affinity indicators can also be easily adapted to our property-oriented optimization process.

Regarding the CrossDocked2020 dataset, it is constructed from the PDBBind database through cross-docking ligands and proteins. The Vina Score shown in Figure 1 reflects the binding affinity of docked poses in CrossDocked2020.

Regarding the subpar Vina Score for DecompOpt in Table 1, it is important to note that while combining TargetDiff with our optimization process may yield superior Vina scores, this comes at the cost of reduced molecular diversity. DecompOpt effectively balances optimization objectives with diversity, leveraging the flexibility provided by the scaffold structure and diffusion process. In contrast, TargetDiff w/ Opt tends to generate ligands similar to the initial ligands, which is not ideal in drug design due to the need for diverse molecular candidates. To comprehensively evaluate the quality of generated ligands, metrics such as success rate, which was also used in [1,2], are more important.

Q3: Inference efficiency of the diffusion model. Generation efficiency compared with other methods.

A3: For inference efficiency, we test the run time for generating 20 molecules on average. In detail, AR, Pocket2Mol, TargetDiff, DecompDiff, and DecompOpt use 826s, 269s, 367s, 506s, and 731s, respectively. Our model is on par with some of the baselines. The inference is efficient and totally acceptable in practice of drug discovery.

Dear Reviewer ewuH,

Thanks again for your valuable feedback! We sincerely appreciate the time and effort you have dedicated to reviewing our paper.

To respond to your comments, we have made more detailed explanations of our motivation and method, and also done extensive experiments and comprehensive evaluation.

As the discussion phase will end tomorrow, we kindly request, if possible, that you review our rebuttal at your earliest convenience. If you have any other concerns, we would like to further discuss. If we have addressed your concerns, we sincerely hope you can reconsider the evaluation of our paper.

Dear Reviewer ewuH,

We kindly remind you that the author-reviewer discussion phase will end soon.

We have revised our paper accordingly and made more detailed explanations of our motivation and method, as well as extensive experiments and comprehensive evaluation.

We look forward to your feedback about our response and revision so that we can further address your other concerns if there are any.

And we sincerely hope you can reconsider the evaluation of our submission if we have addressed your concerns.

Thank you for your encouraging feedback. Please see below for our responses to the comments.

Q1: The reference ligand is used as the input for optimization.

A1: Thank you for pointing out the potential confusion about the initialization of conditional arm lists in our method. To clarify, our method initializes the conditional arm lists using DecompDiff, and not directly from reference ligands. We have clarified this point in the revision.

Q2: "How similar is your generated ligand to the reference ligand?"

A2: We believe there might be a misunderstanding here. We never used reference ligand as the input to our model. In all experiments, for each pocket, we start with the 20 ligands generated by DecompDiff, and then apply DecompOpt for 30 iterations. We have clarified this in the revision.

We additionally test the Novelty and Similarity of generated ligands compared with the reference ligand. Novelty is defined as the ratio of generated ligands that are different from the reference ligands of the corresponding pockets in the test set. Similarity is defined as the Tanimoto Similarity between the generated ligands and the corresponding reference ligands. The results show that the generated ligands are not similar to reference ligands in the test set.

Besides, we also test Uniqueness and Diversity of generated ligands. Uniqueness is the percentage of unique molecules among all the generated molecules. Diversity is the same as that in Section 4.1.

These results show that DecompOpt can design novel ligands, which is an important ability for drug discovery.

Q3: The influence of the reference ligand's quality on performance.

A3: We appreciate your concern about the potential influence of the initial ligands' quality on the performance of our optimization algorithm. Recognizing the importance of this issue, we have conducted an ablation study focusing on Vina Min score optimization, using ligands with high and low Vina Min scores as initializations for the arm lists. Due to limited resources, we chose to conduct this study on the protein 2V3R, which is randomly chosen from our test set. We generated 100 ligands using DecompDiff and selected 20 ligands with the highest and lowest Vina Min scores. These ligands were then used as the initial conditions for the optimization process.

As shown in the experiment result, the optimization outcomes are slightly influenced by the quality of the initial ligands. However, regardless the quality of the initial ligands, DecompOpt can consistently improve the quality of the generated ligands.

Thank you for your positive feedback. Please see below for our responses to the comments.

Q1: Reproducibility.

A1: We are committed to releasing the codes, model checkpoints, and samples generated by our models upon acceptance. We hope these will contribute to the community. Besides, we have provided more implementation details in Appendix A in the revision.

Q2: The concept of controllable generation can be used not only in R-group optimization and scaffold hopping but also de novo design. Lack of rigor in using the word.

A2: Thanks for pointing this out! We use controllable generation here to refer to molecular generation with reference structures or substructures. In this sense, controllable generation is distinguished from de novo design. If the reviewer has any better suggestion on appropriate terminology, we would like to follow it.

Q3: Specific differences between DecompOpt and DecompDiff.

A3: Thanks for your question. DecompDiff incorporates decomposition into diffusion models for SBDD by introducing the decomposed prior, which only contains spatial information. In contrast, DecompOpt introduces reference arms as control signals to leverage both chemical and spatial semantics from discovered molecular substructures. To achieve this, based on DecompDiff, we introduce an SE(3)-invariant encoder to encode a reference arm and its surrounding subpocket as the condition. This makes DecompOpt fully utilize the advantages of decomposition. Moreover, thanks to controllability, we can effectively achieve molecular optimization and controllable molecular generation based on the model, while DecompDiff does not support.

Thank you for your constructive feedback. Please see below for our responses to the comments.

Q1: Novelty of the diffusion part compared with DecompDiff. Reproducibility.

A1: DecompDiff incorporates decomposition into diffusion models for SBDD by introducing the decomposed prior, which only contains spatial information. In contrast, DecompOpt introduces reference arms as control signals to leverage both chemical and spatial semantics from discovered molecular substructures. This makes DecompOpt fully utilize the advantages of decomposition. Moreover, thanks to controllability, we can effectively achieve molecular optimization and controllable molecular generation based on the model, while DecompDiff cannot.

As for reproducibility, we are committed to releasing the codes, model checkpoints, and samples generated by our models upon acceptance. Besides, we have provided more implementation details in Appendix A in the revision.

Q2: "Why can solely maximizing the likelihood of training data mislead SBDD models?"

A2: Thanks for your insightful question. As we observed, the training data (e.g., CrossDocked2020, a widely used dataset) is not satisfying in terms of the desired properties. As shown in Figure 1, more than half of the ligands in the dataset are under the desired threshold, which means they may not bind to the targets perfectly. Therefore, the performance of the generative models that solely maximize the likelihood cannot satisfy the criteria of drug discovery. This motivates us to introduce molecular optimization into generative models.

Q3: "How is diversity achieved in the generated ligand molecules without explicitly introducing regularization over the representation space of conditions?"

A3: The diversity comes from the randomness in diffusion sampling and the dedicated design of our method. Though we provide the model with conditions on arms to achieve controllability, we leave room for exploring scaffolds. Since the atoms of the arms and scaffold are correlated during sampling, the randomness is also introduced to the arms. That is also the reason why the generated arms are similar to rather than exactly the same as the arms provided as conditions. Even in the absence of explicitly introducing regularization such as VAE over the representation space, a significant level of diversity can still be attained.

Q4: "Have you tried multiple time experiments with different random seeds to avoid randomness?"

A4: Following prior works, all results reported in tables are mean or median values of properties of generated ligands over 100 pockets. Per your suggestion, we furthermore run the experiments under different random seeds. The results are shown as follows:

We can see that the results are very stable across different random seeds.

Dear Reviewer SQ7r,

Thank you for the time and effort you have put into evaluating our submission!

In our response, we have further clarified the motivation, novelty, and contribution of our method, and done additional experiments. Besides, notably, we have provided extensive implementation details in Appendix A in the revision.

We kindly remind you that the discussion phase is coming to the end. Please let us know if there are additional concerns we can address for you. If we have properly addressed your concerns, we sincerely hope you can reconsider the evaluation of our submission.