We thank Reviewer yPeb for their thoughtful comments and insights. We address their outstanding questions in our response below.

Research Scope

While we focus on the discrete, biological sequence design tasks from Design Bench, it is important to recognize that GABO and Source Critic Regularization (SCR) can be used for both continuous and discrete tasks (unlike BIB and BootGen). This is why we evaluate SCR and GABO on continuous optimization tasks in our experiments, such as the Branin and Warfarin tasks. In choosing which Design bench tasks to evaluate, we focus on the biological sequence design tasks because these tasks have been the most reproducible across recent work, as highlighted by Reviewer bMXC. The research scope is not limited to offline biological sequence design.

Nonetheless, we evaluate BootGen on the discrete biological sequence design tasks to compare its performance against GABO for these subset of tasks. Our results are shown here:

The main conclusions of our manuscript are not changed after the inclusion of BootGen as an additional baseline method. We are unable to compare GABO against BIB due to limitations in the BIB authors' publicly available code implementation.

Top-1 Experimental Evaluation

Thank you for this comment. As discussed in Section 1, of our manuscript, the primary motivation in evaluating the top-1 candidates is that real-world use cases for offline generative design do not have access to large oracle query budgets. This evaluation schema is actually studied in other related offline MBO work (e.g., Kim et al. Proc NeurIPS 2024). This evaluation schema is important because offline optimization is most helpful when evaluating newly proposed molecules requires expensive experimental laboratory setups, or when we want to optimize a patient's drug regiment without being able to test multiple doses on that patient. In these settings, the evaluation of up a large number of designs (sometimes as much 256) as in prior work is not feasible and not representative of how an algorithm would perform in practice. This change in evaluation setup is also why BONET and DDOM perform worse on the Branin task in this more realistic setting.

In fact, GABO actually performs better using the more standard top-64 metric (Supp. Tables B1 and B2). In particular, GABO achieves an average rank of 2.1, which is better than any other reported method. Thus, our motivation for showing top-1 results is only because we believe it is more practical and relevant for real-world applications of offline model-based optimization.

We also cite Reviewer bMXC's review as well for additional insights. In particular,

• "One thing I really like about the paper is its evaluation which truly mirrors the offline optimization setting. The paper compares the proposed approach and baselines on a single evaluation from the oracle which I believe is the right way to evaluate algorithms for offline model-based optimization."

Performance on the Warfarin Task

Thank you for raising this point, and we appreciate that the Reviewer recognizes the importance of introducing practical tasks such as the Warfarin task. Compared with the other tasks assessed, the landscape of true oracle function for the Warfarin task (a LASSO model reported previously by domain experts) is uniquely a smooth convex function, and the trained surrogate likely captures similar properties over the design space. As a result, we hypothesized that this task was more conducive to first-order offline optimization methods, which is exactly what is shown in our results.

Definition of Offline MBO in Eq (2)

Thank you for this comment; we used the definition of offline MBO as in Eq (2) because (1) it is the definition of offline MBO that best frames the problem setup for our proposed approach of SCR; and (2) it is the definition consistent with the original Design-bench publication from Trabucco et al. CoRR 2022. In related work that study forward approaches to offline MBO (e.g., Yu et al. Proc NeurIPS 2021; Trabucco et al. Proc ICML 2021; Chen et al. Proc NeurIPS 2022), the authors similarly first consider a motivating problem setup for offline MBO identical to Eq (2), and then extend on it to solve a related problem through their own methodological contributions as pointed out by the Reviewer. This is how we approached framing the problem formulation to motivate SCR as well.

However, we also agree with the Reviewer that a few recent related work have used alternative formulations of offline MBO to motivate their work (e.g., Kim et al. Proc NeurIPS 2023; Krishnamoorthy et al. Proc ICML 2023; Krishnamoorthy et al. Proc ICML 2023). In the final manuscript, we will be sure to clarify that (1) such other methods for offline MBO exist; (2) we consider one possible definition of offline MBO in Eq (2); and (3) how these additional problem formulations relate to Eq (2) considered in our work.

Thank you for your detailed response. Unfortunately, several questions remain on the paper. Here are some follow-up questions.

Research Scope)

While authors conduct continuous tasks, Branin and Warfarin, those tasks are close to toy experiment settings. Branin is a 2D function, and Warfarin is a smooth convex function. While the superconductor task in Design-Bench has a reproducibility issue, other tasks, such as Ant and Dkitty, do not suffer from that issue.

Furthermore, it seems the results of BootGen significantly deviate from the reported score. It is hard for me to accept the results that the maximum score of $k=64$ in TFBind8 is 0.401, which is even below the maximum of the dataset. As shown in the Figure 4.1 of BootGen, the performance of even K=1 seems higher than 0.8.

Top-1 Experimental Evaluation)

While I also agree with the motivation that we do not have large oracle query budgets, evaluating top-1 candidates may lead to high variance. Furthermore, using a proxy to select one candidate is also not convincing as a proxy is fragile to out-of-distribution errors.

I strongly recommend that authors compare the proposed method with a commonly used evaluation setting, sample 128 designs without filtering with proxy (I think this part is also a procedure of the proposed algorithm and should not be included in baselines), and report the maximum and median scores.

Performance on the Warfarin Task)

The authors say that the oracle function of Warfarin is a convex function. However, the motivation of offline MBO is optimizing high-dimensional and complex black-box functions. I think the Warfarin task may not be appropriate for evaluating MBO algorithms.

Furthermore, it may be appreciated if the proposed algorithm is compared with [1], which also deals with conditional MBO problems.

[1] Chen, Mingcheng, et al. "ROMO: Retrieval-enhanced Offline Model-based Optimization." Proceedings of the Fifth International Conference on Distributed Artificial Intelligence. 2023.

We respond to Reviewer AH6U's comments below:

Rationale of SCR

To summarize our work and as discussed in our manuscript, the primary rationale behind our proposed source critic regularization (SCR) method is to stop optimization algorithms from extrapolating against offline surrogate models in generative design. We accomplish this by showing how to optimally and tractably balance the tradeoff between exploring the design space for potentially better designs while ensuring we don't over-extrapolate against the learned surrogate model.

Experimental Details

We have described the details of our experiments in Section 5 and Appendix A of our manuscript, and also have made our code available for easy reproducibility and transparency. If the Reviewer has any remaining questions on experimental details, please feel free to let us know.

SCR is not a regression method.

To clarify, our work is not proposing a regression method; instead, the primary contribution of SCR is to show how to balance the tradeoff between (1) optimization against a regressor model and (2) staying in-distribution in a computationally tractable way in offline MBO.

To validate this primary contribution empirically, we make use of generic, task-agnostic surrogate regression models trained using standard machine learning techniques (see Section 5.2, L271-5 for details). We do not optimize the hyperparameters of our learned surrogate regression models in any way, and use the same generic parametrization of $f_{\theta}$ across all tasks (L271-2). There are no special training methods that were used in constructing the surrogate regressor models. We are happy to clarify further if the Reviewer has any additional questions regarding the primary motivation and contributions of our work.

How does a learned surrogate model provide additional information without querying the true oracle function? Source critic regularization doesn't add anything.

The learned surrogate model provides information on the true oracle function because it is trained on historical designs and their corresponding true oracle function scores (represented as the dataset $\mathcal{D}$ in our manuscript). In regions of the design space that contain many examples of these historical designs in $\mathcal{D}$, the surrogate model agrees with the true oracle function because it has been trained on past observations of the true oracle function. However, in regions of the design space that do not contain many examples of historical designs, the surrogate model and true oracle function likely disagree due to the problem of extrapolation. A good illustration of this is shown in Figure 1 of our manuscript.

The value added by source critic regularization is in preventing optimization algorithms from taking advantage of these extrapolated regions of the design space that result in falsely "optimal" designs. In offline optimization, we want to avoid such designs that "look promising" according to the surrogate but in reality score poorly on the true oracle due to extrapolation of the surrogate model.

When do you evaluate a true objective function?

For each of the optimization methods (including GABO), we query the true objective function once (for Table 1) as the final step after an offline optimization algorithm is finished running and proposes a single design to evaluate using the hidden oracle objective function. This evaluation strategy is aligned with the motivation for this work; after we perform offline optimization experiments to propose a candidate design(s), we need to evaluate these designs with the actual oracle function to see how good these designs actually are. This evaluation schema is consistent with that used in most (if not all) other related work in offline generative design.

Reparametrization of $\lambda

The rationale behind re-parameterizing $\lambda$ in terms of $\alpha$ is that the feasible search space for $\lambda$ is all non-negative real numbers, which cannot be tractably searched over in a clear way. Our approach to solve this problem is to instead re-parameterize $\lambda$ in terms of $\alpha$ so that searching over a finite $\alpha\in[0, 1]$ is "equivalent" to searching over $\lambda\in[0, \infty)$.

Interpretation of $\mathcal{D}$(best) in Tables 1 and 2

Recall that $\mathcal{D}$ refers to the dataset of historical designs and their corresponding true oracle function scores that was used to train the learn surrogate model. In Tables 1 and 2, $\mathcal{D}(\text{best})$ refers to the best oracle function score observed from this set of prior historical designs. For example, in Table 1, the value of $\mathcal{D}(\text{best})=11.3$ for the LogP task means that out of all the 79,564 unique molecules and their corresponding oracle LogP values in the base offline dataset associated with the LogP task, the maximum score achieved by any given molecule is 11.3.

Therefore, if an optimization algorithm like GABO proposes a design with an oracle score greater than the best in $\mathcal{D}$ (e.g., a molcule with an oracle LogP score greater than 11.3), this means that we have "discovered" a new design not previously seen in the historical dataset that is even better than the best design in the historical dataset. Finding designs better than $\mathcal{D}(\text{best})$ is the main goal of offline optimization for generative design.

Our choice of notation and in reporting $\mathcal{D}(\text{best})$ is consistent with that used in other related work (Krishnamoorthy et al. Proc ICML 2023, Krishnamoorthy et al. Proc ICML 2023, Trabucco et al. Proc ICML 2021, Chen et al. Proc NeurIPS 2022).

We thank Reviewer Yd2B for their thoughtful review of our work, and appreciate that they recognize our proposed source critic regularization (SCR) formulation and strategy as a key contribution of our work.

Framing of the Manuscript's Narrative

We agree that our main contribution is Source Critic Regularization (SCR) in Algorithm 1, which is optimizer-agnostic and can be used with any optimization method. Our experimental evaluation of SCR using Bayesian Optimization (BO) is because we find it to be the most effective optimizer to use in conjunction with SCR, likely due to BO's well-documented effectiveness as an offline optimizer as outlined in Section 3.2 (L110-7). Indeed, as acknowledged by the Reviewer, we have already included evidence of the effectiveness of BO compared to alternatives such as Gradient Ascent (GA)$\textemdash$in particular, our comparison to Generative Adversarial Gradient Ascent (GAGA) in Supplementary Table B5. In addition, GABO also outperforms vanilla BO in all our experiments, demonstrating that SCR has significant added value as a methodological contribution.

In alignment with the feedback from the Reviewer, we are happy to revise the title of our work to the following: Generative Adversarial Model-Based Optimization via Source Critic Regularization. This improvement will help better emphasize that our main contribution is SCR independent of the choice of baseline optimizer. We will also do our best to revise our paper to make it more clear that SCR is our main contribution, and that BO is simply the most effective instantiation of our method.

Future Work on Time-Varying Optimization

Thank you for this comment; indeed time-varying BO is an interesting and nascent field of research and a complex problem in general. We hypothesize that there may be opportunities to exploit existing methods in time-varying BO for SCR if the updates to the source critic function $c^*$ follow some sort of pattern over time. In practice, we were unable to identify any such patterns in preliminary studies largely due to the limited number of updates to $c^*$ made over the course of the optimization process. Furthermore, factoring in time-varying BO was not necessary to demonstrate that GABO and SCR are effective empirically. Given the complexity of this problem and the empirical success of GABO and SCR already demonstrated in our work, we leave this opportunity to explore time-varying BO for future work.

We thank Reviewer bMXC for their thoughtful comments and insights, and share their enthusiasm for the proposed significance and strengths of our work. We address their outstanding questions in our response below.

The Role of the Surrogate NN in MBO

We thank the reviewer for this comment. We use this setup for generality; in many applications, the surrogate objective is not under our control. For instance, it might be a physics simulator or other domain-specific function that has been optimized by domain experts to give more information about the hidden oracle function across the entire design space than what can be accomplished with GPs alone. (Note that task-specific optimization of the surrogate is not done in our work.) By fitting a GP to this surrogate instead of just the observations of the hidden oracle function, we can take advantage of the information encoded in a more complex (and hopefully accurate) surrogate.

To demonstrate this empirically, we explore this potential strategy of using the single GP from BO directly as the both the surrogate function and for BO sampling. We leverage Source Critic Regularization (i.e., Algorithm 1) using this framework. We refer to this as Generative Adversarial Bayesian Optimization with a GP Surrogate (GABO GP-Surrogate), and compare this method with GABO:

As we can see, using a more complex, neural-network surrogate function for GABO leads to better optimization results than directly using the GP as the surrogate function.

Our evaluation strategy of using an neural network surrogate for GABO is also identical to the one used in the existing offline MBO literature to make it easier to compare our work with pre-existing methods: for instance, Trabucco et al. (ICML 2021), Yu et al. (NeurIPS 2021), Chen et al. (ICML 2023), Krishnamoorthy et al. (ICML 2023), and Krishnamoorthy et al. (ICML 2023) in addition to our work.

Sensitivity to Sampling Budget and Batch Size

Thank you for this comment. Prior work has shown that BO methods are relatively robust to perturbations in the batch size and other hyperparameters (i.e., Figure 7 in Eriksson et al. (NeurIPS 2019)), and we have observed that this similarly applies to GABO. We will include a discussion on the sensitivity to the optimization hyperparameters in an updated version of our manuscript.

Additional Citation

Thank you for sharing this related work from Chemingui et al. with us. We will include a citation to this work and appropriate associated discussion in the final manuscript.

We thank Reviewer U2YG for their thoughtful feedback on our work, and address their outstanding questions and concerns below.

Performance of GABO on the 90th percentile metric

Unlike the other evaluation metrics assessed in our work, GABO indeed does not perform as well on the 90th percentile metric. However, this is not surprising since GABO is not designed to target this metric (and it is not our metric of interest). In particular, in our analysis, we have found that this is largely due to the nature of the underlying Bayesian optimization (BO) optimization algorithm. Because BO is not an iterative first-order algorithm, the designs proposed by any BO-based algorithm often have high variance in practice. This is what we observe across all of our experiments, including in Table 1 and Supplementary Tables B1 and B3.

We also note that in most offline optimization applications, the 90th percentile metric$\textemdash$or any metric that does not use for the best proposed design(s)$\textemdash$is not as useful as the other metrics assessed where GABO does perform well. This is because in offline optimization tasks with a restricted budget to query the hidden, expensive-to-evaluate oracle function, we are not interested in "wasting" this budget on subpar design candidates. While the 90th percentile and similar metrics can be helpful to understand the limitations of algorithms (as in this case), we believe that the 100th percentile metrics reported in the main text are more useful and practical in assessing each of the optimization algorithms.

For completeness, we also show the distributions of the proposed designs for across all of the optimization methods for the LogP task to allow interested readers to compare the performance of the different methods on a distributional level. The plot is available here: Link. We will include this result in the final manuscript.

Why run BO on the learned surrogate when it is already differentiable and can be numerically optimized using GA?

Thank you for this question. We choose to use BO for the basis of our work because it is a well established principle in recent Bayesian Optimization (BO) literature that BO is useful even for differentiable objectives, outperforming baselines such as GA Eriksson et al. (NeurIPS 2019), Maus et al. (NeurIPS 2022), Hvarfner et al. (2024), Eriksson et al. (UAI 2021), Astudillo et al. (ICML 2019). It is the dimensionality and non-convexity of the search space that makes the optimization problems in our benchmark challenging, leading first order methods like GA to struggle. Based on these results, we believe BO is a natural basis for our work. We discuss this in detail in Section 3.2 (L110-7).

Nevertheless, we emphasize that our proposed source critic regularization (SCR) algorithm can be used with other optimization methods as well, such as Gradient Ascent (GA) as suggested. Indeed, as already noted by the Reviewer, we also evaluated SCR with GA, which we call Generative Adversarial Gradient Ascent (GAGA), and include results in Supplementary Table B5. Comparing GABO with GAGA helps motivate our decision to use BO for experimental evaluation of our SCR algorithm. To better emphasize that our main contribution is SCR, we plan to revise the title of our work to the following: Generative Adversarial Model-Based Optimization via Source Critic Regularization

We thank Reviewer czFU for their thoughtful review of our work, and appreciate that they recognize the technical and empirical contributions of GABO compared with prior work. Please find our responses to their questions below.

Search for $\alpha$

Importantly, we confirm that no extra data is used to compute the value of $\alpha$ in our algorithm -- because only the source critic $c^*$ is retrained dynamically over the optimization process, we only need to query the easy-to-compute neural network $c^*$ in computing $\alpha$.

We also confirm that we compute $\alpha$ through a grid search. However, in our implementation used for our experiments (which will be made publicly available), the grid search to compute $\alpha$ is highly vectorized such that the computational time is non-limiting even using a single-GPU setup. To benchmark our implementation, we evaluate both BO and Gradient Ascent (GA) both with and without our Generative Adversarial (GA) source critic regularization algorithm on the Branin and Penalized LogP optimization tasks:

Compute time is reported as mean $\pm$ standard deviation over 10 random seeds. As a reminder, the Branin task is a standard benchmarking task for offline optimization, and the Penalized LogP task is subjectively the most challenging task assessed in our manuscript with the highest dimensional design space out of the seven assessed tasks.

While there are obviously additional compute costs associated with running our Source Critic Regularization method (i.e., Algorithm 1), as made evident by the above results, we note that in most applications of offline optimization, obtaining labeled data is the main bottleneck in many practical applications; thus, it is often worth spending this extra compute to ensure the best results for the given budget.

Selection of Model Hyperparameters

The exact same generic FCNN-based source critic as introduced by the initial WGAN authors Arjovsky et al. (2017) is used across all of the different tasks assessed in our work. Similarly, we perform no hyperparameter fine-tuning of VAEs and use standard task-agnostic VAE architectures across the different tasks in our work. By using generic model architectures that have not been optimally tuned to each task, we can focus on assessing our contributions that are primarily algorithmic in nature. That is, the VAEs and source critic models may perform well on certain tasks and may perform poorly on others that were assessed. By evaluating our source critic regularization algorithm across a wide variety of different benchmarking tasks, we are able to give a good picture of the "average" performance of GABO and other optimizers independent of how well VAEs and source critics are tuned to the specific optimization task at hand.

What is D(best)?

In our results, $\mathcal{D}$(best) refers to the best oracle score achieved by any previously observed design in the offline dataset. For example, in Table 1, the value of $\mathcal{D}_{\text{best}}=11.3$ for the LogP task means that out of all the 79,564 unique molecules and their corresponding oracle LogP values in the base offline dataset associated with the LogP task, the maximum score achieved by any given molecule is 11.3. Therefore, if an optimization method (such as GABO) proposes a molecule design that achieves an oracle LogP score greater than 11.3, then we have found a design better than any of the designs previously seen in the offline dataset. Our choice of notation is consistent with that used in related work (Krishnamoorthy et al. Proc ICML 2023, Krishnamoorthy et al. Proc ICML 2023, Trabucco et al. Proc ICML 2021, Chen et al. Proc NeurIPS 2022).