MITIGATING OVER-EXPLORATION IN LATENT SPACE OPTIMIZATION USING LES

We thank the reviewer for their time and effort in reviewing our work.

Experimental Design and Analysis

We calculate Equation 13 for a given Z because we are interested in a score that can be attributed to a single latent vector, rather than to a distribution over the output space $p(x)$. Indeed, $p(x)$ is not a function of a single latent vector and cannot be used to guide the optimization. We will make sure to emphasize this point in the revised version of our manuscript, under Equation 13.

** likelihood vs LES **

The reviewer is making a great point, and we are glad to see that the reviewer agrees with us (lines 234–237) that:

“As the Likelihood score requires only a single forward pass of the decoder, it offers a more computationally efficient alternative, which comes with some performance trade-offs.”

It is important for us to provide the full picture so that practitioners can use LES in cases where they feel the juice is worth the squeeze, but can also opt for the likelihood score if they have a limited computational budget.

We thank the reviewer for their time and effort in reviewing our work.

Comments and Suggestions

We thank the reviewer for their comments and suggestions, which we will incorporate into the camera-ready version of our work.

Question

All code and trained VAEs will be released if the paper is accepted.

We believe the UC early stopping procedure is too conservative, and essentially gets stuck around the initial data point—especially if the landscape is not smooth in terms of valid and invalid regions. Figure 1 provides some evidence that this is the case for the expression dataset. We will add this explanation in our revised version (lines 400–405) for further clarity.

In practice, we believe that for the more interesting molecular tasks, $\lambda$ = 0.5 should generally perform better than what is currently used (e.g., Turbo). We provide this guideline in lines 370–372:

“Based on our findings, we suggest $\lambda$ = 0.5 as a reasonable default value, while leaving the exploration of an optimal choice for future work.”

We thank the reviewer for their time and effort in reviewing our work. We are glad to see that the reviewer finds our work to address an appropriate hole in the research landscape.

We would like to clarify that we did not claim that LES is faster than all alternative approaches, but rather that it is faster than Bayesian uncertainty methods (lines 231–236):

“LES is computed faster than the Uncertainty score, achieving reductions of up to 85% in some cases. As the Likelihood score requires only a single forward pass of the decoder, it offers a more computationally efficient alternative, which comes with some performance trade-offs.”

We believe that this claim is well-supported by Table 1.

Is Gaussian with std = 5 OOD?

We believe that a standard deviation of 5 is sufficient to qualify as out-of-distribution (OOD). To substantiate this claim, we provide the proportion of valid solutions for each VAE on a random sample of 500 data points generated with std = 5:

We thank the reviewer for their time and effort in reviewing our work.

Typos

Thank you for pointing this out—we will update the text to replace "quadratic formula" with "quadratic function."

Additional Visualizations

We thank the reviewer for their great suggestion to add more visualizations to our manuscript.
As Figure 1 provides a visualization of the dynamics within the latent space, we would be happy to include additional figures using different starting points in the revised version.

Empirical Rigor

Our main claim is that LES can be used as a robust method (across VAEs) to improve the number of valid solutions discovered through latent space optimization. We consider a rigorous empirical evaluation of this claim to be one that explores multiple VAE models, multiple notions of validity, and multiple latent space optimization tasks. This definition aligns with the goal of LSO, where we aim to use a VAE to find realistic solutions for a given objective.

By this definition, we believe that our empirical evaluation is rigorous for the following reasons:

• We study 22 VAEs, varying both the decoder architecture and the beta parameters that control the trade-off between the prior and the reconstruction. This goes beyond previous related work, which studied only 4 VAEs [1].
• We study three notions of validity, ensuring our empirical results are not specific to a single definition.
• We evaluate five different latent space optimization tasks, all considered standard benchmarks in this literature [1,2].

References
[1] Notin, P., Hernández-Lobato, J.M., & Gal, Y. (2021). Improving black-box optimization in VAE latent space using decoder uncertainty. Advances in Neural Information Processing Systems, 34, 802–814.
[2] Maus, N., Jones, H., Moore, J., Kusner, M.J., Bradshaw, J., & Gardner, J. (2022). Local latent space Bayesian optimization over structured inputs. Advances in Neural Information Processing Systems, 35, 34505–34518.

Methodological Validity

We believe that the validity of Theorem 3.1 in our setup is well justified through the approximation results we cite. In addition, our extensive empirical evaluation further supports the soundness of our methodology.

Failure Cases of Other Methods

The failure mode we aim to address is that existing baselines tend to generate unrealistic solutions [1]:

“It has been widely observed that optimizing outside of the feasible region tends to give poor results, yielding samples that are low-quality, or even invalid (e.g., invalid molecular strings, non-grammatical sentences).”

Table 13 systematically demonstrates this widely known failure mode by comparing the percentage of valid solutions generated by each optimization method. In short, LES produces on average 62% valid solutions more than any unregularized method—L-BFGS (26%), TurBO (38%), and GA (55%).

Reference
[1] Tripp, A., Daxberger, E., & Hernandez-Lobato, J.M.
Sample-efficient optimization in the latent space of deep generative models via weighted retraining.

Discussion Section

We thank the reviewer for the valid suggestion to revisit Examples 1.1–1.3 in the discussion and to explicitly articulate the challenges being addressed, which we will happily do. A revised version of the first paragraph is included below:

“We propose LES to address over-exploration in latent space optimization (LSO). Desirable properties of LSO include stability—across tasks and VAE architectures—and the ability to consistently discover high-quality solutions that optimize a given objective. However, existing approaches often suffer from instability and excessive exploration, which can lead to unrealistic or invalid outputs without careful regularization. LES mitigates these issues by encouraging optimization to remain near the data manifold, thereby stabilizing LSO and improving the realism of generated solutions. LES is differentiable and fully parallelizable. Extensive evaluations demonstrate that incorporating LES as a penalty in LSO consistently enhances solution quality and objective outcomes. Moreover, LES outperforms alternative regularization techniques, proving to be the most robust across diverse datasets and varying definitions of validity—including syntactically valid expressions, chemically valid SMILES, and molecules that pass domain-specific property filters. In addition, LES has only a single hyperparameter (the regularization strength), and we observe empirically that deploying LES can provide significant performance gains. We therefore believe LES offers a powerful approach for discovering more realistic solutions, especially when the criteria for realism are difficult to define or validate.”

We thank the reviewer for their time and effort in reviewing our work.

Deep Kernel

The deep kernel affects only the computation of the kernel in the GP surrogate model—it does not reduce the dimensionality of the latent space on which LES is applied. Based on our results with the SELFIES VAE, we see no indication that achieving improvement becomes more difficult in higher-dimensional settings.

TuRBO Method

The TuRBO method (similar to every other method in this work) uses the LogEI acquisition function, for the same reasons noted by the reviewer. We will add clarification in our revised version, in section 4 under LSO setup paragraph.

Ablation studies on $\lambda$

As requested by the reviewer, we provide an ablation study of the parameter $\lambda$ for both LES and the Likelihood score. Due to space constraints, we present the full results for the average of the top 20 solutions—since this metric tends to exhibit lower variance—and summarize the key findings for both the best and top-20 solutions below. For the average of the top 20 solutions, we find that using a lower $\lambda$ (0.1) for the likelihood score significantly improves performance on the RANO task when using the pre-trained SELFIES VAE, increasing the score from 0.31 to 0.41. We will include this setting in both the appendix and the main text of the revised version. For the best solution, we observe the following notable differences: For the expressions transformer with $\beta = 1$, increasing the regularization to $\lambda = 0.2$ improves both LES and the likelihood scores to -0.46 and -0.42, respectively.

For the SMILES transformer with $\beta = 0.1$, setting $\lambda = 0.8$ improves the likelihood from 3.16 to 3.25, surpassing the 3.23 achieved by LES at $\lambda = 0.5$.

Expressions

LES

Likelihood

SMILES

LES

Likelihood

SELFIES

LES

Likelihood

SELFIES (Maus et al)

LES

Likelihood

We thank the reviewer for their time and effort in reviewing our work, and we are particularly glad to see that the reviewer agrees that our claims regarding the effectiveness of LES are supported by our empirical evidence, which the reviewer described as “extensive”.

First, thank you for catching our typos, which have been corrected (ScaLES → LES, Theorem 5 → Theorem 3.1).

The reviewer is correct to point out that LES is closely related to decoder performance. As this is an important point, we made sure to include it under the limitations of Theorem 3.1 (lines 247–252):

“Lastly, it is crucial to highlight that the ability of LES to detect out-of-distribution data is closely tied to the decoder’s capacity to accurately model the data. Although Theorem 3.1 holds for poorly trained decoders under the given conditions, we advise against relying on LES in such cases.”

Indeed, we expect the decoder to struggle with OOD data points in general (this has also been widely observed in many previous works that we cite [1]):

“...therefore all LSO methods known to us employ some sort of measure to restrict the optimization to near or within the feasible region.”

Lastly, we would like to offer some clarification regarding hyperparameter tuning (an important clarification which will be added to our submission).

We acknowledge that some manual selection of the hyperparameter $\lambda$ was performed based on the level of regularization needed for each task, which could give our method a slight advantage over the baseline. To ensure our improvements are not merely the result of this tuning, we included an ablation study for TurBO and L-BFGS in our original submission, which demonstrates that LES would out-perform these methods on average even if the best hyperparameter was retroactively selected.

We also include an additional ablation and analysis of $\lambda$ selection in our response to Reviewer SqeL.

Given this, we believe our comparisons are fair and that our conclusions are well supported: LES offers a consistent and meaningful improvement over the baselines, and these gains are not artifacts of hyperparameter tuning.

Reference

[1] Tripp, A., Daxberger, E., & Hernandez-Lobato, J.M.
Sample-efficient optimization in the latent space of deep generative models via weighted retraining.

We thank the reviewer for their time and effort in reviewing our work.

We address the reviewer's questions and comments chronologically.

Thank you for suggesting the addition of the important work by Tripp et al. We would like to point out that we refer to this work in line 155:

“The frequent generation of invalid solutions during acquisition optimization, which implies that the estimated uncertainty can be problematic in this setting, underscores the need for additional regularization (Tripp et al., 2020), which we aim to address.”

We thank the reviewer for their suggestion to clarify the meaning of the parameter beta, which balances the alignment with the prior and the reconstruction loss. We will add a description of this parameter in the first paragraph of Section 3.2.

Due to margin constraints, Table 2 is a truncated version of the full Table 11, which is deferred to the appendix. In the revised version, we will make sure to clarify this point in the last paragraph of Section 3.2.

The intuition behind using the log-determinant terms is explained in lines 172–188. In short:

“Why use the sequence density function? We argue that for a well-trained decoder network, the density should be higher in areas of the sequence space close to the training data.”

As we mention in line 206:

“...as the contribution of the prior is negligible in magnitude in all the decoders we study,”

this is why we only use the log-determinant term.

We will make sure to be consistent in using the term score when referring to LES—thank you for the suggestion.

The results from Daubechies state that ReLU networks (which are CPA spline functions) can approximate refinable functions—a class of non-smooth functions—in addition to known results on other classes of functions (e.g., Weierstrass, polynomials, and analytic functions), as described in this work. This clarification will be added to the limitations of Theorem 3.1.

The EMPTY category allows generating sequences shorter than the maximal length. For example, if the maximal length is 3, then we can generate the expression "x" as "x<EMPTY><EMPTY>". This example will be added to the first paragraph of Section 2 to enhance its clarity.

Sequential optimization means that we generate each solution separately, rather than optimizing all the solutions jointly. This setup was used in the work by Notin et al., which we follow. This clarification will be added to the LSO setup paragraph in Section 4.