MF-LAL: Drug Compound Generation Using Multi-Fidelity Latent Space Active Learning

We thank the reviewer for their useful feedback.

The soundness and novelty of the main feature of ML-LAL - " using separate latent spaces and decoders specialized for each fidelity improves surrogate modeling and inter-fidelity information passing"- is questionable … the method considered in this paper, and the method I provide above are conceptually similar, and the latter one means nothing but adding more neural network layers to $\mu_k$ and $\sigma_k$ as $k$ increases, rendering the novelty of the proposed method trivial, and the claimed ''better message passing'' questionable.

In the case where we only sample from each latent space $z_{1, \dots, K-1}$ to predict the mean and variance in the subsequent latent space, then the process can be simplified by using a single network to predict the mean and variance at the final latent space and then adding noise. However, the surrogate models $\hat{f_{1}}, \dots, \hat{f_{K}}$ are the key components of our architecture that make sampling from each latent space separately necessary. As the surrogate model for each fidelity takes points $z_k^{(i)}$ in the associated latent space as input, it is necessary that we define each of these latent spaces separately and can sample from each individually. Additionally, the information passing between latent spaces (using $h_{\xi_k}$) is possible because each surrogate model $\hat{f}_k$ organizes its associated latent space $z_k$ to best aid in prediction at that fidelity level. Thus, it is necessary to retain each individual latent space and have separate neural networks to pass information between the latent spaces, or the benefits of the separated surrogate models cannot be realized.

...if we follows the conceptually similar setup I described above, algorithm 2 just reduces to getting a sample batch $\{ \hat{\epsilon}^{(i)} \}$ from $\mathcal{N}(0,1)$, and only keeping the samples $\hat{\epsilon}$ such that $z_k(\hat{\epsilon})$ is of high scores for each $k$.

The key difference between the reviewer’s approach and our proposed approach is that we perform optimization of compound scores at each fidelity level. This means we use a gradient-based optimizer to find locations in each latent space that have high surrogate-predicted scores. In contrast, the reviewer’s approach consists of sampling points in latent space $z_k$ and checking each one to see if it has high scores at all other fidelities. This sampling approach will not generate compounds with as good scores as the optimization-based technique. This is because the optimization-based technique takes advantage of knowing the gradient of the surrogate model, as opposed to the sampling technique that randomly guesses until it finds a good compound.

we would like to see a training curve as done in the baseline method[1], where the $y$-axis represents the values of performance metrics, and $x$-axis represents the computation cost until the budget is exceeded … However, the author only provides Table 1 in this paper, which summarizes the final performance metric values. This alone is insufficient to support the claimed capability for the trade-off.

We would like to emphasize Appendix C.1, which contains a training curve similar to the reviewer’s description. The plot shows the oracle-predicted scores of the generated compounds during the active learning process, and as expected the scores become more favorable as active learning proceeds. We would be happy to provide additional data if the reviewer has specific requests.

While the author claims that workflow in [1] is separate (lines 56-58), we note that the integrated workflow of MF-LAL is similar to [1]. The integrated part, which I found in MF-LAL is that in equation (1), the first term (corresponding to generative part) and the second term (corresponding to surrogate part) are optimized jointly. However, since the two terms are sequential, where the output of generation part is the input of the surrogate part, there is no big difference whether we optimize it sequentially as described in [1] or jointly as in MF-LAL.

The generative model in [1] does not explicitly model the different fidelity levels. Instead, it is a standard generative model that optimizes the acquisition function of a separate multi-fidelity surrogate. The generative model only optimizes a single scalar acquisition value, and is thus not aware of the multi-fidelity nature of the generation problem. In contrast, our model is integrated in that it explicitly models each fidelity inside of the generative model. This means the generative model is aware of the differences between fidelities, and is able to generate compounds tailored to each specific fidelity level.

(response continues below)

We thank the reviewer for their useful feedback.

I thought the experiments had some significant correctness issues. First and foremost, the results in Table 1 are described as "significant" despite a very small sample size of 15 and considerable variance in the scores of the generated molecules. I think the authors need to apply a statistical test to their results so that the differences between methods can be evaluated more objectively.

We performed evaluations on 15 compounds due to the high computational cost of ABFE, as evaluating the compounds from a single method takes 9.33 hours * 15 = 5.8 days. Nonetheless, we have now evaluated an additional 15 compounds (30 in total) from MF-LAL and only the single most competitive baseline for each target. These new results have been added to Table 1 in the updated draft. We found that MF-LAL still generates compounds with better ABFE scores than baselines for both targets. To evaluate the statistical significance of our results, we measured the p-value between the mean ABFE score of generated compounds from MF-LAL and the most competitive baseline. For BRD4(2), we computed p=0.16, and for c-MET, p=0.001. At least for c-MET, MF-LAL generates compounds with a significantly better mean ABFE score than the most competitive baseline. For BRD4(2), we approach but do not reach significance, likely because the spread of ABFE results appear to be larger for that target.

A key motivation for the method is to "ensures compounds generated at higher fidelities also scored well at lower fidelities, improving the quality of generated samples." (lines 075-077). This hypothesis seems questionable: why would one want this? The fundamental challenge in multi-fidelity AL is that the observations between different fidelities are not very well-correlated. Is it not possible that most interesting molecules at high-fidelity score poorly at lower fidelities?

Our results show that even the lowest fidelity oracle is somewhat accurate at distinguishing between active and inactive compounds (see the results in Appendix B.1 in the updated draft). Therefore, compounds that score well at the lowest fidelity are more likely to show binding at the highest fidelity than randomly chosen molecules. By only searching in the space of compounds at higher fidelities than scored well at the lower fidelities, we focus our search and greatly reduce the computational cost needed to find good compounds. While it is possible that some interesting molecules at the higher fidelities score poorly at the lower fidelities, that is not necessarily a concern for our method. Since there are many potential scaffolds that can score well at the highest fidelity, it is a worthwhile tradeoff to discard some of these scaffolds early on with lower fidelity oracles in order to decrease the computational cost at the highest fidelity and make the search tractable. We have added discussion of this point to the Limitations section of the updated draft.

The method uses SVGPs in quite a few places, which raises concerns about robustness. SVGPs are very sensitive to the location of their inducing points, and tend to overestimate variance away from their inducing points

We thank the reviewer for raising this potential concern. We have added discussion about the variance of SVGPs to the Limitations section of the updated draft. We note that while the high variance of SVGPs away from the inducing points might bias the UCB acquisition function towards molecules far away from the training distribution, we also include an L2 regularizer in the acquisition function that encourages generated compounds to stay in-distribution in the latent space. Empirically, we see that the acquisition function generally chooses molecules that fall within the training distribution, suggesting that the high variance of SVGPs is not a major concern for MF-LAL.

The distribution of the initial dataset will likely determine the location of the inducing points, and with the UCB objective it seems plausible that nothing away from the initial distribution will therefore be chosen. At the very least, the behavior will depend enormously on the locations of the inducing points.

Instead of using the initial dataset to choose the inducing points, we randomly choose 1,000 inducing points by sampling from the Gaussian prior of the latent space. This means the inducing points are independent of the initial data.

(response continues below)

We thank the reviewer for their useful feedback.

I would be interested how the 1d/SELFIES-based MF-LAL framework compares against newer 3d generative models such as DecompDiff [1] or PILOT [2] without applying guidance or calling the oracles, since their methods might not be directly applicable within the multi-fidelity hierarchy.

Thank you for the suggestion! We have now run DecompDiff as a baseline and report the results in Table 1. Our results show that DecompDiff generally produces molecules with slightly worse ABFE scores than Pocket2Mol, and significantly worse scores than MF-LAL.

How is the reconstruction accuracy for the 4 seperate encoder-decoder networks? … Could the author provide reconstruction learning curves over epochs? As the fidelity-level gets higher, the sparser the latent spaces become, since first these spaces are trained with less data.

We found that the lower fidelity latent spaces generally have lower reconstruction accuracy than the higher fidelity latent spaces, likely because the higher fidelity latent spaces have a more limited compound space to decode (even if the training data is more sparse). However, the reconstruction accuracy is generally high across all fidelities, showing that MF-LAL successfully learned the mapping from latent space to molecule. We have added these new results measuring the reconstruction rate of each decoder during the active learning process to Appendix C.2.

We thank the reviewer for their useful feedback.

Maximizing the likelihood that the molecule would also be generated at fidelity k−1 with a high property score apparently limits exploration space, and the oracle at the lowest level is likely to misguide the searching spot due to its poor accuracy.

Limiting the exploration space is the goal of the likelihood term as it reduces the number of oracle calls that must be expended at the highest fidelity, and should not be seen as a weakness of our method. While it is possible that the lower fidelity oracles somewhat misguide the search for good compounds at the higher fidelities, that is also not a major concern for our method. First, even the lowest fidelity oracle is still somewhat accurate (see the results in Appendix B.1 in the updated draft), so molecules that scored well at this level are more likely to show binding at the highest fidelity than randomly chosen molecules. Therefore, it makes sense to restrict the search at the higher fidelities to these regions that are richer in active compounds, even if not all regions end up being fruitful. Second, in order to make the search for good molecules at the highest fidelity computationally tractable, we must use the lower fidelity oracles to discard some regions of chemical space and focus our search. Since there are many potential scaffolds that can score well at the highest fidelity, our method is still able to find molecules that have good scores, even if some good scaffolds were discarded by the low fidelity methods. We have added discussion of this point to the Limitations section of the updated draft.

…as shown in Table 2, the top 3 results w/o likelihood term are not significantly lower than with it. The authors need to elaborate on the potential effect of using the likelihood constraint (eq 3) on constraining the exploration.

The likelihood term significantly improves the mean ABFE score of the generated compounds (Table 2). As the generated compounds are also highly structurally diverse (Section 4.3), this means that the likelihood term allows for the generation of a large number of diverse drug candidates with good binding (low mean ABFE scores). This is relevant for medicinal chemists, who desire a large number of structurally diverse and strongly binding drug candidates to pick from for further optimization. In contrast, MF-LAL without the likelihood term generates compounds with a much worse mean ABFE score, meaning there are fewer strongly binding compounds for medicinal chemists to choose from (even if the top compounds have similar scores). Thus, using the likelihood term gives more desirable results than without the likelihood term.

Using only 15 generated compounds is too few to evaluate statistically meaningful performance, which casts doubt on the conclusions. In particular, the top 3 values strongly rely on sampling. The authors may evaluate with more samples or discuss the limitations.

We agree that only evaluating 15 generated compounds may seem limiting. However, this is mainly due to the high computational cost of ABFE, as evaluating the compounds from a single method takes 9.33 hours * 15 = 5.8 days. Nonetheless, we have now evaluated an additional 15 compounds (30 in total) from MF-LAL and only the single most competitive baseline for each target. These new results have been added to Table 1 in the updated draft. We found that MF-LAL still generates compounds with better ABFE scores than baselines for both targets. To evaluate the statistical significance of our results, we measured the p-value between the mean ABFE score of generated compounds from MF-LAL and the most competitive baseline. For BRD4(2), we computed p=0.16, and for c-MET, p=0.001. At least for c-MET, MF-LAL generates compounds with a significantly better mean ABFE score than the most competitive baseline. For BRD4(2), we approach but do not reach significance, likely because the spread of ABFE results appear to be larger for that target.

Training the proposed model is terminated with a total time limit, which causes an unsystematic convergence criterion… The authors need to justify the consequences of their termination criterion (7 days) on the model performance.

Terminating training after a fixed run time most accurately reflects the real-world use case of our method in a resource-constrained drug discovery context. Waiting until training error converges for baselines such as the single-fidelity ABFE method would involve many expensive oracle calls and thus take a long time, so it is instead more valuable to measure the accuracy after expending some fixed computational budget. Additionally, accuracy and run time are correlated, so terminating training after a fixed run time does not yield unsystematic results.

(response continued below)

Thank you for your response!

That is true, but the top 3 scores w/o the likelihood are close to those with it; top N molecules are more important than the mean value in practice.

We agree, but our choice of top 3 is somewhat arbitrary. Considering the top 5 or 10 molecules, for instance, would show a clearer advantage for MF-LAL with the likelihood term.

This reviewer also understands that the notoriously high computation cost of ABFE does not allow sufficient running time, but this is a practical limitation of the proposed method that should be admitted.

We still think using a fixed computational budget gives systematic results, because the performance of each method after a fixed run time should be (inversely) proportional to the final convergence time. Nonetheless, we do agree that our experiments may have high variance because of the limited computation time, so we have added some discussion of this issue to the Limitations section in the updated manuscript.

We thank the reviewer for their useful feedback.

While it's true that the degree of resolution of free binding energies estimation can be higher for some systems, this is taken as a pillar fact in this work while this might not be true. It is of utmost importance to check any physics-based protocols with compounds with known affinity and actually corroborate that the low to high-fidelity methods stand as such

the approach is lacking from benchmarking the systems on docking and ABFE with compounds known to bind to them. That would give a more solid comparison point for generated molecules.

We would like to emphasize Appendix B.1, which contains results about the accuracy of each oracle on known binders. In brief, we measure the binary classification ability (ROC-AUC) of each oracle to distinguish between known actives and presumed inactives for BRD4(2). The results confirm that the presumed higher fidelity oracles indeed have higher accuracy. In light of the reviewer’s comment, we have now conducted a similar experiment for the other protein target, c-MET, and have included the new results in Appendix B.1. These new results also show that the higher fidelity oracles are more accurate.

…authors select 15 compounds for each method and compute ABFE metrics on all of them. However, they base a great deal of their analyses on 3 top scoring compounds per system. When looking at the mean of the 15 compounds generated vs each of the values for the top 3 compounds it's clear to see that they are outliers, which might not be bad per see (it is exactly what a generative campaign is looking for) but must be always followed by a careful analysis of the motifs the molecules generated.

Our evaluation of the generated compounds is based on the real-world use case of our method, which would involve taking the top generated compounds and using them as starting points for further optimization. In this case, the most relevant factor is how strong the binding is for the top generated compounds, so we do not think focusing on them is a weakness of our work. Indeed, evaluating only the top generated compounds is common practice in the literature on generative modeling for molecules [1, 2, 3].

[1] Lee et al. “Exploring Chemical Space with Score-based Out-of-distribution Generation.” ICML 2023.

[2] Luo et al. “GraphDF: A Discrete Flow Model for Molecular Graph Generation.” ICML 2021.

[3] Fu et al. “Reinforced Genetic Algorithm for Structure-based Drug Design.” NeurIPS 2022.

Scoring functions of any type, docking or ABFE, are biased towards certain types of chemical structures and assign a high score as a result of highly reactive topologies.... While those are valid molecule they would be potentially discarded since they would be considered artifacts.

The highly reactive motifs mentioned by the reviewer are known to be overly rewarded by docking scoring functions. However, it is not clear that the same is true for the more accurate ABFE calculations. This exemplifies the benefits of the multi-fidelity approach, where inaccuracies in lower fidelity oracles can be corrected by higher-fidelity, more accurate methods. Additionally, while the presence of the mentioned motifs might be concerning for medicinal chemists, the generated compounds that contain them still show strong binding and would therefore be valuable as starting points for further optimization (e.g. improvement of toxicity).