Enhancing Ligand Validity and Affinity in Structure-Based Drug Design with Multi-Reward Optimization

[R3] Reviewer rt1G

We sincerely appreciate your positive and constructive feedback. Below, we address the comments and questions raised. Due to the word limit, additional experimental results can be found in anonymous pdf link

---

Q1) What difficulties arise when using multi-reward optimization?

Multi-reward optimization can be viewed as a multi-objective optimization, which sometimes underperforms compared to single-objective optimization due to task conflicts. Therefore, we encountered difficulties in designing a reward signal that simultaneously satisfies multiple metrics required for SBDD task.

To provide more insight, we analyze the reward values of winning and losing samples using both all seven rewards and three rewards (ours) and report their average rewards. Rebuttal Table 4 shows the average difference between winning and losing samples with seven rewards is much closer ($\times 10$) than that of the three rewards. We conjecture that the small difference introduces conflicts in the fine-tuning process, making it difficult for the model to optimize all properties.

---

Q2) Why choose Vina, SE, and QED instead of other metrics (such as Vina, SE, SA; Vina, SE, Clash, QED)

We first rule out using all possible metrics (as mentioned in Q1) and run single-reward experiments (Table 3) to identify unique, non-overlapping signals. We observe that optimizing one Vina metric improves all Vina metrics, so only one metric for binding affinity is sufficient. We chose QED over SA because improving QED also boosts SA, and we exclude clash due to its lack of improvement when used alone. Additionally, Vina and SE exhibit conflicting objectives, i.e., improving binding affinity (Vina) tends to compromise structural stability. This tendency is also observed in our single-reward experiment, so we include both to balance these competing objectives. Through this manual search, we determine that Vina, SE, and QED best complement each other in multi-reward optimization, as supported by our experimental results.

For further information, we additionally conduct an experiment using the reward combination given in R3's question statement—Vina, SE, QED, and Clash—whose results are presented in Rebuttal Table 5. As shown in the table, this combination leads to performance improvements over the pretrained model across most metrics. However, it still underperforms compared to our final model choice.

---

W1) Details in temperature parameter to ensure the rewards make sense.

We sincerely appreciate the reviewer’s insightful question, which helped us clarify the formulation. In the revised manuscript, for clarity, we have revised the formula in line 227 as

$\hat{r}_{i}\^{(j)} = \frac{\exp\bigl(f_j(r_i^{(j)})\bigr)} {\sum\_{b=1}^B \exp\bigl(f_j(r_b^{(j)})\bigr)}$, where $f_j(\cdot)$ is -r, \quad \frac{10}{r}, \quad r \ if $j$-th reward is Vina Dock, SE, and QED, respectively.

This ensures a higher value corresponds to a better reward across different metrics.

---

W2) Limited Improvement in Figure 3 compared to MolCRAFT, which serves as the backbone

The results from Figure 3 are based solely on pass/fail criteria while presenting a broader range of physical plausibility (e.g., bond lengths, angles, etc). Therefore, we believe it is important to interpret Figure 3 in conjunction with Table 1 in the main paper. While the pass rate in Figure 3 shows a modest increase, the overall physical validity of our generated molecules has significantly improved, as shown in validity metrics, SE and clash. Additionally, we highlight that SOTA models such as IPDiff and AliDiff in binding affinity exhibit lower pass rates in Figure 3. This implies that these models sacrifice physical plausibility to achieve higher binding affinity while our approach improves both key properties.

---

W3) Lack of Intuitive 3D Visualization as a 3D method

We appreciate the reviewer’s valuable suggestion regarding the intuitive presentation of our contribution. We will incorporate additional visualizations in the revised manuscript, highlighting the docking poses of our generated ligands with improved binding affinity.

---

W4) $K$ in lines 103 and 242 is not specified and inconsistent.

We recognize the inconsistency in notation. As R3 mentioned, $K$ in line 103 represents the number of atom features, whereas $K$ in line 242 denotes the number of rewards. we have revised the manuscript to clarify and correct the error.

---

W5) Unnumbered formulas

To improve readability, we have numbered the equations accordingly in the revised manuscript.

---

W6) Unspecified value $\gamma$ in 242

We chose $\gamma = 0.4$. It has been included in the revised manuscript.

We deeply appreciate your thoughtful comments and positive feedback. Here, we address the comments and questions mentioned.

---

W1) The model does not outperform baselines in ligand-protein clash detection, which may restrict practical applications.

We acknowledge the reviewer’s concern regarding the clash detection performance. However, we would like to clarify the following points: The clash metric reflects the number of colliding atoms, which inherently depends on the molecular size. As shown in the average size of main table 1 in our paper, Pocket2Mol and AR tend to generate smaller molecules on average compared to other models, which naturally results in fewer clashes. To address this discrepancy, we conduct an additional analysis where we filter the generated ligands based on the number of atoms to obtain comparable molecular sizes and report the clash performance accordingly.

Rebuttal Table 2. Clash performance after filtering for similar molecule sizes.

In Rebuttal Table 2, when comparing models with comparable molecular sizes, our method demonstrates strong performance in clash metric. Notably, the only model exhibiting a better clash score, AR, performs poorly in binding affinity. Given that both low clash and high binding affinity are critical for practical applications, these results indicate that our model offers the best balance, making it the most suitable choice for real-world deployment.

---

W2) The innovation of this paper is somewhat limited.

We acknowledge that each methodological component may not be significantly novel. However, integrating them into a single system that simultaneously enhances key properties for Structure-Based Drug Design (SBDD) remains a meaningful contribution. In particular, we are the first to apply DPO to Bayesian Flow Networks (BFNs) and to leverage reward normalization for multi-reward optimization—improving binding affinity, validity, and drug-likeness at once. This approach, not previously explored in the literature, helps make SBDD more practical.

---

W3) Validation is limited to the CrossDocked dataset, raising concerns about generalizability to other protein targets or real-world drug discovery scenarios. However, this is primarily due to the inherent issues and limitations of the SBDD task at its current stage, rather than a problem with the paper itself.

We completely agree with this point. As research in SBDD continues to grow, we expect more robust and diverse benchmarks to emerge, allowing broader validation and ensuring greater generalizability to real-world scenarios.

---

Q1) I'm curious about what results would be obtained if the fine-tuned model were sampled again and the fine-tuning process were repeated. Will the model continue to improve, or will its performance deteriorate due to certain reasons (such as overfitting)?

We appreciate the insightful question, which encourages further discussion. To address it, we provide results for a model fine-tuned iteratively for two stages (i.e., multistage) with newly sampled molecules using DPO, compared to our model and the pretrained model.

Rebuttal Table 3. Performance comparison of the pretrained model, a multistage fine-tuned model, and our model,

Although the two-stage DPO achieves competitive performance, it does not surpass our model choice. Xu et al. also suggest that fine-tuning via DPO does not necessarily guarantee significant improvements [1]. Furthermore, iteratively measuring rewards using external tools at each training step is computationally expensive compared to the training process itself, which led us to adopt an offline strategy. Nevertheless, we believe that exploring an online, iterative approach with efficient sampling remains a promising direction for future research.

Reference

[1] Xu, Shusheng, et al. "Is dpo superior to ppo for llm alignment? a comprehensive study." arXiv preprint arXiv:2404.10719 (2024).

We deeply appreciate your efforts and positive feedback. Here, we address the comments and questions mentioned.

---

W1) Lack of computation complexity analysis against diffusion baselines. might be complex given the complexity of Bayesian Flow Networks (BFNs) and multi-reward optimization

To address your concern regarding time complexity, we report the time required for one epoch of pretraining and fine-tuning our model, comparing it to the AliDiff baseline (which also uses DPO). To measure the computation time, we use the official source code provided by the author and run the code on a single NVIDIA A6000 GPU.

Rebuttal Table 1. Time needed for training per one epoch (in seconds)

The Rebuttal Table 1 shows applying DPO to a pre-trained model takes 16,354 seconds with our method per one epoch, compared to 22,884 seconds for AliDiff, representing a 28.5% reduction. These results demonstrate that our model is more time-efficient than the diffusion-based baseline. Furthermore, coupled with its strong performance, this highlights the practicality of our model for Structure-Based Drug Design (SBDD) tasks. We have added this computation time in the appendix and appreciate R1's feedback in helping us improve the manuscript.

Our approach does not significantly increase overall training time; however, obtaining rewards from an external tool can be time-consuming. To address this, we preprocess and store rewards in advance, but this requires an offline DPO framework. A promising future direction for SBDD is reducing reliance on external tools, enabling a more iterative and efficient online training loop.

---

W2) Minor issue: figure 1&2 should provide axis indications

Thank you for the editorial comment. We have added the axis indications in our revised manuscript.