Trading-off Multiple Properties for Molecular Optimization

Thank you for taking the time to review our paper and providing valuable feedback. We would like to answer your questions below.

1. [W1] The multi-objective optimization problem is not trivial.

Firstly, we believe it is not appropriate to dismiss the research field of multi-objective optimization as trivial. General multi-objective optimization is currently an important research direction in machine learning, with numerous efforts dedicated to solving Pareto optimization problems. And Linear Scalarization is considered a weak baseline in multi-objective optimization. Moreover, the Pareto optimization problem remains unresolved in molecular optimization.

Secondly, from SOMO to MOMO is the research trend in drug design. Previous works usually focused on single properties. However, it is until this year that we have witnessed the emergence of tasks involving multi-objective molecular optimization in machine learning conferences (ICML 2023 [1] and NeurIPS 2023 [2]). Futhermore, they adopt Linear Scalarization or Bayesian optimization to tackel this problem, which is the baseline of our method.

Thirdly, Thirdly, thanks for reviewer xqpc and reviewer dF44 to recognize the contribution of our work. Reviewer xqpc finds our work novel and interesting, and Reviewer dF44 considers it intriguing and a practically valuable contribution.

2. [W2 & Q1] About Linear Scalarization Critique

It has been mathematically proved by [3] in Chapter 4.7.4 and explaining why linear scalarization fails to capture preference-conditioned solutions. Our experiments also confirm this phenomenon. We provide the code for you to reproduce our Figure 1 and Figure 3 within few minutes. To avoid any other factors, we did not employ any tricks or set random seeds. The drawbacks of Linear Scalarization are quite evident as we mentioned in our paper. Please refer to the provided code for further details: https://github.com/ICLR2024/PCI

Our code can also compute the hypervolume:

PCI: 0.33 I-LS: 0.065

You stated ‘’Contrarily, based on my experience, linear scalarization can navigate along the Pareto front toward both extremes, and with weight tuning and keeping all points visited during the processes, results comparable to PCI's can be achieved.‘’ Could you please provide some evidence to support your viewpoint? Or could you please point out how to improve our code to achieve 'Linear scalarization's results are comparable to PCI'.

3. [W3] Generalizability of the Method.

If the problem is non-differentiable, we can train a differentiable surrogate model and apply PCI.

4. [W4] About Top-K molecules.

Top-k is a commonly used metric in molecular optimization, and we are simply following the existing works[1][4][5], as we have already mentioned in our paper. Generally, in molecular optimization, people usually evaluate the top points, which is more normal in optimization (since one generally cares about the optimum reached, not how one gets there). For instance, if we need to add 5 substructures to achieve a high property score, why do we need to consider the molecules (3 substructures) generated during the intermediate steps as part of the final result and include them in the statistics? In addition, we set K to 100 instead of the commonly used values of 3 or 10 in existing works, which makes our result more convincing.

5. [Q2] Oracle calls for Graph GA.

We apology for the typo. We have already corrected it to 0+25k.

Reference:

[1] Moksh Jain, Sharath Chandra Raparthy, Alex Hernandez-Garcıa, Jarrid Rector-Brooks, Yoshua Bengio, Santiago Miret, and Emmanuel Bengio. Multi-objective gflownets. In International Conference on Machine Learning, pp. 14631–14653. PMLR, 2023.

[2] Zhu Y, Wu J, Hu C, et al. Sample-efficient Multi-objective Molecular Optimization with GFlowNets[J]. arXiv preprint arXiv:2302.04040, 2023.

[3] Boyd, S. and Vandenberghe, L. Vector optimization. In Convex Optimization, chapter 4.7, pp. 174–187. Cambridge University Press, 2004.

[4] Emmanuel Bengio, Moksh Jain, Maksym Korablyov, Doina Precup, and Yoshua Bengio. Flow network based generative models for non-iterative diverse candidate generation. Advances in Neural Information Processing Systems, 34:27381–27394, 2021.

[5] Tianfan Fu, Wenhao Gao, Cao Xiao, Jacob Yasonik, Connor W Coley, and Jimeng Sun. Differentiable scaffolding tree for molecule optimization. In International Conference on Learning Representations, 2022.

4. [Q5] Is the standpoint of this study to specialize insights from general multi-objective optimization? Or does it also provide new insights into general multi-objective optimization?

Yes, the disadvantage of linear scalarization was first argued in general multi-objective optimization and this problem is still unsolved in molecular optimization task. And PCI also provide new insights into general multi-objective optimization. Currently, general multi-objective optimization focuses on continuous spaces, especially the continuous parameters of deep models. It is still unclear how to perform multi-objective optimization in discrete spaces instead of using black-box search methods such as Genetic Algorithms and Bayesian Optimization. Technically, we have transformed the discrete problem into a locally differentiable problem and provided convergence guarantees. Furthermore, we believe that drawing insights from general multi-objective optimization and addressing an still unsolved and important problem in molecular optimization is both meaningful and novel.

Reference:

[1] Jonas Degrave, Ira Korshunova. Why machine learning algorithms are hard to tune. (https://www.engraved.blog/why-machine-learning-algorithms-are-hard-to-tune/)

[2] Tianfan Fu, Wenhao Gao, Cao Xiao, Jacob Yasonik, Connor W Coley, and Jimeng Sun. Differentiable scaffolding tree for molecule optimization. In International Conference on Learning Representations, 2022.

[3] Nathan Brown, Marco Fiscato, Marwin HS Segler, and Alain C Vaucher. Guacamol: benchmarking models for de novo molecular design. Journal of chemical information and modeling, 59(3): 1096–1108, 2019.

[4] Kexin Huang, Tianfan Fu, Wenhao Gao, Yue Zhao, Yusuf Roohani, Jure Leskovec, Connor W Coley, Cao Xiao, Jimeng Sun, and Marinka Zitnik. Therapeutics data commons: Machine learning datasets and tasks for drug discovery and development. arXiv preprint arXiv:2102.09548, 2021.

Thank you for taking the time to review our paper and providing valuable feedback. We would like to answer your questions below.

1. [Q1 & Q2] About Non-Uniformity.

Before we discuss the Non-Uniformity, I would like to share an example I provided to Reviewer xqpc, which effectively demonstrates the importance of Non-Uniformity in MOMO and the advantage of PCI.

For instance, let's consider two properties (A, B) with a solution space of {(1, 1), (1, 0.5), (1,0.33), (0, 0)}. We aim to optimize these attributes using SOMO+scalarization, starting from the initial point (0,0). Assuming that a very high value of property B has a negative impact, we need to maintain it at a moderate level. Thus we assign a preference ratio of 2:1 and the ideal optimal solution is (1,0.5), and define the unified attribute as P = A + 0.5B. However, for the solution (1,1), P = 1 + 0.5 = 1.5. For the solution (1,0.5), P = 1 + 0.5 * 0.5 = 1.25. Clearly, 1.5 > 1.25, and we end up with the solution (1,1) instead of the desired (1,0.5). Even if we change the weight to 3:1, we still obtain the same solution (1,1) instead of desired (1,0.33). However, PCI can find the desired solution (1, 0.5) and (1,0.33) since we take preference condition and decent direction in to consideration by solving Eq. 9. We have demonstrated this through both theoretical analysis and experimental validation. Due to the limitations of the response's length, we strongly recommend reviewer to refer to a blog [1] posted by DeepMind in 2021 which illustrates why linear scalarization often results in biased solutions and fails to adjust scalar weights in an easy-to-understand manner.

1.1 [Q2] Why is Non-Uniformity?

Non-uniformity is a metric used to measure the similarity between the ratio of properties and preference. Therefore, any metric used to measure the similarity between two vectors can be employed, such as cosine similarity. We use non-uniformity because we follow the general multi-objective optimization.

1.2 [Q1] Non-Uniformity is appropriate for MOMO.

In the aforementioned example, if the preference is 2:1, scalarization fails to find the desired solution (1,0.5) , while PCI can. It is evident that (1,0.5) outperforms (1,1) in terms of Non-Uniformity. In practical drug design, it is not always desirable for certain properties to be maximized or minimized. Instead, we may need to control them within a certain range with preference. However, scalarization fails to achieve this level of control. Such requirements are quite common in drug design and remain unresolved.

2. [Q3] About Discretization.

Our sampling process is consistent with the differentiable scaffolding tree (DST), as detailed in [1]. After discretization, like other molecular optimization methods, we employ oracle to test the discretized molecules and select those that meet the conditions (maximum relative objective value $\check{\lambda}^t$ decreased) while discarding the ones that do not. As a result, the final set of molecules will also satisfy the conditions. It is worth noting that in our theoretical analysis, we directly analyze the discretized molecules instead of the continuous form.

3. [Q4] Can the surrogate part (DST) be replaced with other methods?

The surrogate part can be replaced, but the surrogate part should satisfy some conditions. PCI requires the surrogate part to be differentiable, which is why current Combinatorial Optimization (CO) algorithms are not suitable. Another category, Constrained Generative Models (CGMs), conducts optimization in a continuous latent space that is differentiable. However, achieving an ideal smooth and discriminative latent space, typically required by CGMs, has proven to be challenging in practice and CGMs do not perform well in molecular optimization [3][4], which is also shown in our Table. 2. Therefore, currently the most suitable surrogate part is DST, as it allows us to directly modify the molecular structure and is differentiable. The current backbones are all developed based on SOMO, but developing a backbone that is more suitable for MOMO could be an interesting future work.

Dear Reviewer dF44,

Once again, we appreciate your commitment and valuable feedback during the review of our work. We have provided responses to all your concerns. In summary:

• We explained why non-uniformity is appropriate for comparison in MOMO, and illustrate why PCI is better than I-LS with an example;
• We provided details about procedure for the "sample the discrete scaffolding tree" part;
• We also clarified our convergence analysis is based on discrete molecules. We have take the discretization into consideration;
• We explained that the surrogate part should satisfy some conditions to replace DST;
• We provided analysis about the relationship between PCI and general multi-objective optimization, and the new insights to conduct Pareto optimization in discrete space.

If there are any more questions, please let us know. We hope that you will take into account our previous responses when making your final assessment of our work!

Thank you for the detailed feedback! The responses are informative.

Still, I felt that this work is an application of some existing ideas to a very specific situation where "the goal is multi-objective but a preference for the objectives is explicitly given". The answers to Q1-Q4 made me feel that the presented idea doesn't sound like a novel framework, but rather it is just presenting a combined use of several specific ideas in this specific situation, specifically, differentiable scaffolding tree (DST) by Fu et al, 2022 and MOMO ideas by De ́side ́ri et al (2012) and Mahapatra & Rajan (2020). Also, the given answers to Q5 was still unconvincing to me. (with respect to the concerns in "Weakness")

A question:

I didn't quite catch what you mean by the presented example of "two properties (A, B) with a solution space of {(1, 1), (1, 0.5), (1,0.33), (0, 0)}". So can I have more explanations on this?

Actually, this example was hard to interpret, and just raises many many questions: What do you mean by solution space? What do you assume as the Pareto front in this case? Is the tuple like (1,1) the loss or the input variable? What is the given preference in this case? This is also needed for linear scalarization, right? What does "starting from the initial point (0,0)" mean? What is "initial point" here for linear scalarization in this example? What do you mean by "Assuming that a very high value of property B has a negative impact, we need to maintain it at a moderate level." Why is only this part qualitative? Is this about a loss or the value of objective functions? What do you mean by "PCI can find the desired solution (1, 0.5) and (1,0.33) since we take preference condition"? What is the preference condition in this example? ...

We thank you for recognizing the novelty and contributions of our paper and also for the positive feedback. We would like to answer your questions below.

1. Why is Linear Scalarization suboptimal and How does PCI outperform It?

 1.1 [W4] An intuitive understanding of the disadvantage of Linear Scalarization.

We provide an example to illustrate the limitations of Linear Scalarization, which also serves as the inspiration behind our work. For instance, let's consider two properties (A, B) with a solution space of {(1, 1), (1, 0.5), (1,0.33), (0, 0)}. We aim to optimize these attributes using SOMO+scalarization, starting from the initial point (0,0). Assuming that a very high value of property B has a negative impact, we need to maintain it at a moderate level. Thus we assign a preference ratio of 2:1 and the ideal optimal solution is (1,0.5), and define the unified attribute as P = A + 0.5B. However, for the solution (1,1), P = 1 + 0.5 = 1.5. For the solution (1,0.5), P = 1 + 0.5 * 0.5 = 1.25. Clearly, 1.5 > 1.25, and we end up with the solution (1,1) instead of the desired (1,0.5). Even if we change the weight to 3:1, we still obtain the same solution (1,1) instead of the desired (1,0.33).

The example intuitively illustrates why SOMO+scalarization fails to achieve the desired preference-based solutions, as shown in Fig. 1(b). Additionally, even when we change the preference, the obtained solutions remain unchanged and located near the end of the preference regions, as shown in Fig. 3(b). Such requirements are quite common in drug design and remain unresolved in current scalarization-based molecular optimization methods. Due to the limitations of the response's length, we strongly recommend reviewer to refer to a blog [1] posted by DeepMind in 2021 which illustrates why linear scalarization often results in biased solutions and fails to adjust scalar weights in an easy-to-understand manner.

Mathematically, A well-known literature [2] on optimization offers a formal mathematical analysis in Chapter 4.7.4, explaining why linear scalarization fails to capture preference-conditioned solutions.

 1.2 [W1 & W2 & W3] In the questions you mentioned, PCI exhibits clear advantages over SOMO + scalarization.

 (1) [W1] A specific preference: If the preference is 2:1, SOMO+scalarization often result in biased solutions (1,1). On the other hand, PCI can identify (1,0.5) since we take preference condition and decent direction in to consideration by solving Eq. 9. We have demonstrated this through both theoretical analysis and experimental validation.

 (2) [W2] A "small" number of preference: For example, we use preference 2:1 and 3:1. PCI can obtain corresponding Pareto optimal solutions (1,0.5) and (1,0.33) based on different preferences. However, SOMO+scalarization tends to generate biased solutions, and using different preferences may still result in the same solution (1,1).

 (3) [W3] A variety of preferences: In the example above, despite using a variety of preferences, SOMO+scalarization consistently misses the points (1,0.5) and (1,0.33). This corresponds to Fig. 3(b), where there is still a certain unexplored space on the Pareto front. However, PCI, being capable of generating preference-conditioned solutions, can approximately cover the overall Pareto front.

2. [W5] The Details About I-LS. In Fig. 2, I-LS dose not compute the non-dominated gradient $d_{nd}$ but instead linearly weights the gradient using preferences, i.e., $d=G*\lambda$. We keep the remaining parts consistent with PCI. We have included the algorithm for I-LS in Section C.3 of the appendix, highlighted in blue.

3. [W6] About experimental results. To ensure fairness of comparison, both PCI and I-LS employ DST as the backbone network. Since DST itself performs exceptionally well, I-LS also outperforms many baselines. However, we should focus on the relative improvement of PCI and I-LS over DST, which indicates that PCI outperforms I-LS by a significant margin across all performance metrics. The relative improvement is shown in the table below.

   GSK3$\beta$+JNK3  GSK3$\beta$+JNK3+QED+SA

Reference:

[1] Jonas Degrave, Ira Korshunova. Why machine learning algorithms are hard to tune. (https://www.engraved.blog/why-machine-learning-algorithms-are-hard-to-tune/)

[1] Boyd, S. and Vandenberghe, L. Vector optimization. In Convex Optimization, chapter 4.7, pp. 174–187. Cambridge University Press, 2004.

Dear Reviewer xqpc,

We express our thanks once more for your valuable time and constructive suggestions in the review process of our work. We have responded to each of your queries in detail. In summary,

• We provide a detailed explanation of the LS optimization scheme from both theoretical and intuitive perspective;
• We explained why PCI outperforms SOMO+LS with an example when a specific preference, a "small" number of preferences and a variety of preferences are given;
• We clarified the details of our baseline method I-LS and updated our manuscript;
• We also clarified that PCI significantly outperforms I-LS，I-LS even harm the surrogate part (DST).

If there are any further questions, we are more than willing to provide further explanation. We sincerely hope that our previous responses will be considered in your final evaluation of our work!

Thank you for taking the time to review our paper and providing valuable feedback. We would like to answer your questions below.

1. [W1 & Q1] How to assemble the scaffolding tree into a molecule?

We follow JTVAE[1], a classic work that transforms molecular graphs into substructure-based tree structures, which has been widely used in drug design. The process is as follows: (a) Ring-atom connection. When connecting atom and ring in a molecule, an atom can be connected to any possible atoms in the ring. Ring-ring connection. (b) When connecting ring and ring, there are two general ways, (1) one is to use a bond (single, double, or triple) to connect the atoms in the two rings. (2) another is two rings share two atoms and one bond.

2. [Q2] Time Complexity.

We have provided an analysis of the time complexity in Section B of appendix. The computation of $C = G^T G$ has runtime $O(m^2 n)$, where $n$ is the dimension of the gradients. With the current best LP solver [2], our LP (9), that has $m$ variables and at most $2m+1$ constraints, has a runtime of $O^*(m^{2.38})$. Since in deep networks, usually $n \gg m$, PCI does not significantly increase the computational cost of backpropagation gradient calculation. IIn molecular optimization, the time spent on oracle calls typically dominates the algorithm's complexity. PCI has the comparable oracle calls to that of DST, it requires $O(TM)$ oracle calls, where $T$ is the number of iterations (Alg 1). $M$ is the number of generated molecules, we have $M \leq KJ$, $K$ is the number of nodes in the scaffolding tree, for small molecule, K is very small. $J$ is the number of enumerated candidates in each node. Furthermore, the change in wall clock time when Pareto optimization is incorporated to baseline method to a initial molecule: $28s \to 63s$.

3. [W2] About convergence analysis.

There may be some misunderstandings here. When we talk about convergence, the goal is to discuss the number of iterations required to ensure that the distance between the current point $x_{T}$ and the minimum value point $x^*$ is guaranteed to be smaller enough. In our work, we provide this guarantee as you state ‘ After T optimization rounds, the maximum distance between the molecule obtained and the optimal molecule is bounded’. This definition of convergence is widely used, including many works about optimization in machine learning. For more detailed information, please refer to https://en.wikipedia.org/wiki/Rate_of_convergence.

Reference:

[1] Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation. In International conference on machine learning, pp. 2323–2332. PMLR, 2018.

[2] Cohen, M. B., Lee, Y. T., and Song, Z. Solving linear programs in the current matrix multiplication time. In Proceedings of the 51st annual ACM SIGACT symposium on theory of computing (STOC), pp. 938–942, 2019.

Dear Reviewer xC4n,

We again thank you for your valuable time and constructive comments in reviewing our work. We have responded to each of your concerns. In summary,

• We provided details about how to assemble the scaffolding tree into a molecule;
• We conducted time complexity analysis, where demonstrate the PCI is comparable to baseline methods;
• We further explained the misunderstanding about our convergence guarantee;
• We also revised and updated our manuscript following your suggestions.

Please let us know if you have any further questions. We’d sincerely appreciate it if you could take our previous responses into consideration when making the final evaluation of our work!