Offline Model-Based Optimization by Learning to Rank

Q4 Experimental results for BONET [5] and GTG [6]

Thank you for your comment. We double-checked the experimental setting of these two recent backward methods, and confirm the correctness of our results in Table 2. Detailed information of our settings and reproduction on these two methods are listed below:

• BONET. We use the open-source repository released by the authors of BONET (https://github.com/siddarthk97/bonet) for reproduction. Since the repository does not provide instructions to reproduce, we run BONET based on our understanding on its code. Specifically, we ran scripts/<task_name>/train.sh for model training and scripts/<task_name>/test.sh for design sampling. We set the hyper-parameters according to the original paper, except that for the evaluation budget, we changed it from 256 to 128 for a fair comparison (since other methods use a budget of 128). We observed the performance drop after changing the evaluation budget, which is also mentioned in Appendix C of the GTG paper [6]. We also observed that our BONET results in Table 2 are similar to the results in Table 1 of GTG paper. Thus, we believe that our results for BONET are reasonable.

• GTG. We use the open-source repository released by the authors of GTG (https://github.com/dbsxodud-11/GTG) for reproduction. We double-checked the hyper-parameter settings in the config/ folder and found that the settings are correct. Specifically, the hyper-parameters of trajectories construction in Table 8 of the original paper are set in line 99-122 of construct_trajectories.py. We set hyper-parameters, train the model, sample candidate, and evaluate GTG based on the recommended methods in the repository. Taking Ant task for an example, we run GTG by conducting these commands sequentially:

python construct_trajectories.py --task ant
python train.py --task ant --horizon 64 --k 20 --n_traj 4000 --eps 0.05 --seed <seed>
python evaluate.py --task ant --horizon 64 --k 20 --n_traj 4000 --eps 0.05 --ctx_len 32 --alpha 0.8 --seed <seed>

Detailed settings of horizon, k, eps, n_traj follow Table 8 in the GTG paper, and settings of ctx_len and alpha follow Section 4.4 in the paper. We checked the hyper-parameters again and found that our results are correct. The difference between our results and the original results might result from the difference of machines or hardware, because we use RTX 4090 GPU while they use RTX 3090 GPU.

We ran these two methods using their open-source code, following either their provided instructions or our understanding. Please let us know if we made any mistakes in the execution process.

We hope that our response has addressed your concerns, but if we missed anything please let us know.

References

[1] Brandon Trabucco, Aviral Kumar, Xinyang Geng, Sergey Levine. Conservative objective models for effective offline model-based optimization. In Proceedings of the 38th International Conference on Machine Learning (ICML'21), pp. 10358-10368, Virtual, 2021.

[2] Rama Kumar Pasumarthi, Honglei Zhuang, Xuanhui Wang, Michael Bendersky, Marc Najork. Permutation equivariant document interaction network for neural learning-to-rank. In Proceedings of the 2020 ACM SIGIR International Conference on the Theory of Information Retrieval (ICTIR'20), pp.145-148, Virtual, 2020.

[3] Przemyslaw Pobrotyn, Tomasz Bartczak, Mikolaj Synowiec, Radoslaw Bialobrzeski, Jaroslaw Bojar. Context-aware learning to rank with self-attention. In Workshop on eCommerce @ SIGIR'20, Virtual, 2020.

[4] Zhen Qin, Le Yan, Honglei Zhuang, Yi Tay, Rama Kumar Pasumarthi, Xuanhui Wang, Mike Bendersky, Marc Najork. Are neural rankers still outperformed by gradient boosted decision trees? In Proceedings of the 9th International Conference on Learning Representations (ICLR), Virtual, 2021.

[5] Siddarth Krishnamoorthy, Satvik Mehul Mashkaria, Aditya Grover. Generative pretraining for black-box optimization. In Proceedings of the 40th International Conference on Machine Learning (ICML'23), pp. 24173–24197, Honolulu, HI, 2023.

[6] Taeyoung Yun, Sujin Yun, Jaewoo Lee, Jinkyoo Park. Guided trajectory generation with diffusion models for offline model-based optimization. In Advances in Neural Information Processing Systems 37 (NeurIPS'24), Vancouver, Canada, 2024.

Thank you for your valuable and encouraging comments. Below please find our responses.

Q1 Minor typos

Thanks for pointing these out. We have revised them and proofread thoroughly according to your suggestions. Thank you very much.

Q2 How to address the OOD issue in the ranking setting and provide a detailed analysis?

There have been various efforts to mitigate the OOD issue in offline MBO, but those techniques are done on the regression-based framework, which are not specially designed for the ranking-based framework. However, some of the techniques can also be applied to our method. In Table 3, we examined this by simply replacing the MSE loss of prior methods with the ranking loss, i.e., ListNet, which resulted in better performance. This shows the possibility to apply prior methods that aim to mitigate OOD issue in offline MBO to our proposed ranking-based framework, and more tricks that target towards model's generalization ability to properly rank samples remain unexplored (e.g., similar to COMs [1], how to design conservative objectives in the ranking setting), which is an interesting future work.

Besides, another important direction to address the OOD issue in the ranking setting is to utilize more complex surrogate modeling. In the field of LTR, it is well recognized that a simple multilayer perceptron (MLP) is inadequate to capture the ranking information conveyed by the ranking loss [2-4]. However in this work, in order to fairly compare to the regression-based methods, we simply parameterize the surrogate model as an MLP. More complex surrogate structures, such as DASALC [4], can be considered for offline MBO.

Q3 Revisiting the definition or reconsidering the term ``recall" in Definition 1

Thank you for your valuable suggestions! We acknowledge that the definition of Precision and Recall in Definition 1 is confusing on $\operatorname{Recall}@k$. We apologize for that.

We define the metric AUPRC for offline MBO based on the ranking perspective and the need to select top-$k$ designs in offline MBO, where $\operatorname{Precision}@k$ reflects the model's ability to select the top-$k$ designs according to varying $k$, and $\operatorname{Recall}@k$ reflects the coverage of $k$. With abuse of notation, $\operatorname{Recall}@k$ is different from the standard form in information retrieval since we do not have a relevant set in the context of offline MBO.

Following your suggestions, we have renamed $\operatorname{Recall}@k$ to $\operatorname{Coverage}@k$ and changed the term of AUPRC for offline MBO to Area Under the Precision-Coverage Curve (AUPCC) for offline MBO in our paper to prevent misunderstanding. Thank you again for this suggestion.

Thank you for your valuable comments. Below please find our responses.

Q1 Probable failure to accurately estimate the objective function

Thank you for your comments. We want to make some clarifications about the goal of offline model-based optimization (MBO). Given a metric $f(\cdot)$ that need to be optimized (e.g., GPU latency you mentioned), offline MBO aims to find the optimal design $\mathbf{x}^* = \mathop{\arg\max}_\mathbf{x} f(\mathbf{x})$ using only the static dataset. This requires the offline MBO algorithm to recommend $k$ promising designs for a single batch of true evaluation.

The primary aim of the offline MBO algorithm is to recommend designs with the potential to achieve optimal metrics, which are then sent to true evaluations to ascertain specific performance metrics. Consequently, during the training phase, it is not essential to know the exact metric values.

The prevalent approach for offline MBO is the forward approach, which constructs a surrogate model with $\mathbf{x}$ as input and predicts output values. Traditional methods train this surrogate model using a regression framework based on mean squared error (MSE). However, this approach suffers from out-of-distribution (OOD) issues, which misaligns with the goal of offline MBO, as depicted in Figure 1. We have conducted a comprehensive analysis of this issue and proposed a ranking-based framework to mitigate the OOD issue.

In scenarios where both high-performing designs and their precise scores are critical, our methods may perform less effectively compared to the regression-based models. However, to the best of our knowledge, all offline MBO tasks primarily require recommending suitable candidates, focusing on whether a design is good enough rather than its exact metric value. Therefore, we believe our method is both effective and appropriate for offline MBO tasks.

Q2 Issue in definition of Precision and Recall in Definition 1

Thank you for your valuable comments! In the standard definition of AUPRC in information retrieval, a relevant set is needed to indicate the true samples. Since we cannot define a relevant set in the context of offline MBO, we use $\mathcal{D}_0$ as the relevant set for simplification in this work. This simplification distorts the meaning of $\operatorname{Recall}@k$. We sincerely apologize for the caused confusion.

We have revised this part in our paper and hope it is clear now. Here we also give a detailed explanation. In the original paper, we defined the metric AUPRC for offline MBO based on the ranking perspective and the need to select top-$k$ designs in offline MBO, where $\operatorname{Precision}@k$ reflects the model's ability to select the top-$k$ designs according to varying $k$, and $\operatorname{Recall}@k$ reflects the coverage of $k$. With abuse of notation, $\operatorname{Recall}@k$ is different from the standard form in information retrieval since we do not have a relevant set in the context of offline MBO. Also according to the suggestion of Reviewer wgm9, we rename $\operatorname{Recall}@k$ as $\operatorname{Coverage}@k$ and change the term of AUPRC for offline MBO to Area Under the Precision-Coverage Curve (AUPCC) for offline MBO to prevent misunderstanding, which can be found in Definition 1 of our new version of PDF.

We also add discussion on the relationship between AUPCC and ranking in Section 3.2, and that is why we directly utilize LTR techniques for surrogate training.

We sincerely hope that this change could address your concerns. Thank you very much.

Q3 Does not the proposed approach require tricks like adversarial weight perturbation?

Thanks for your comment. We want to make some clarifications. We do not claim that the proposed approach does not require such tricks like perturbation. We just claim that learning with ranking aligns better with offline MBO than learning with MSE. Generalization tricks in prior approaches can also be applied to our method.

As discussed in Section 3.1 and 3.2, we show that it is enough for offline MBO by solely focusing on model's ability to preserve order. Thus, we propose to learn the ranking directly. Note that ranking is a weaker condition since it only commands for correct orders, while minimizing MSE argues for both order preserving and value approximating. The results in Appendix F.5 and Table 11 show that our method can improve ranking performance even in OOD regions.

However, our proposed ranking-based framework also suffers from OOD issue, where the surrogate model might erroneously predicts the order relationship in OOD regions. Note that there have been various efforts to mitigate the OOD issue in offline MBO (e.g., adding sensitivity regularization terms [1]), but such tricks are done on the regression-based framework. Although our proposed framework is based on ranking, some of those tricks can also be applied. In Table 3, we examined this by simply replacing the MSE loss of prior methods with our best-performing ranking loss, ListNet, and incorporating output adaptation, which lead to better performance. This demonstrates the versatility of ranking loss and the possibility to apply prior methods in offline MBO to our proposed ranking framework.

COMBO [2], i.e. adversarial weight perturbation as you mentioned, is an approach for offline RL, which is different from offline MBO. Specifically, offline MBO can be viewed as a one-step decision-making procedure, where we propose only one batch of design $\mathbf{x}$ for evaluation, while offline RL is a multi-step decision-making procedure to maximize a long-term discounted reward of a policy $\pi$. This causes differences between offline RL and offline MBO from many perspectives, as discussed in Table 1 of [3]. Thus, COMBO cannot be directly applied to our framework. However, we have combined our framework with other applicable techniques in offline MBO in Table 3 of our paper, like ensembling weak learners (i.e., Tri-Mentoring [4]), learning from policies (i.e., PGS [5]), and matching the gradient (i.e., MATCH-OPT [6]), by simply replacing the core MSE term in their loss function by ranking loss and incorporating output adaptation, which demonstrates both the versatility of the ranking loss and the compatibility of these techniques with our ranking-based framework.

Q4 Unclear how using an LTR loss alone could eliminate the need for approaches

Thank you for your comments. As our reply to Q3, we do not claim that using an LTR loss alone or using the ranking framework could eliminate the need for such approaches. As mentioned above, we combined our ranking framework with some recent approaches in offline MBO (e.g., Tri-Mentoring [4], PGS [5], and MATCH-OPT [6]) shown in Table 3, and we also explained why not combining it with other works in Appendix E.5. The superior performance led by these straightforward combination has demonstrated the possibility to apply prior methods in offline MBO to our proposed ranking framework, and how to combine them more effectively is an interesting future work.

Q5 How to guarantee that the ranking will generalize to out-of-distribution data?

Thank you for your insightful question. In this work, we point out that the model's ability to preserve order in OOD regions is crucial for offline MBO, instead of focusing only on OOD-MSE as in previous works. The OOD-AUPCC results in Table 11 show that using LTR techniques is more effective to generalize well on OOD data than regression-based methods. However, due to the inherent nature of the limited coverage of the offline dataset (which usually covers a narrow manifold [7]), the OOD issue is inevitable, and also exists for the ranking-based framework. In the future, it is expected to improve the model's generalization ability to properly rank the samples in OOD regions, e.g., by utilizing previous techniques in regression-based offline MBO as mentioned before.

We hope that our response has addressed your concerns, but if we missed anything please let us know.

References

[1] Manh Cuong Dao, Phi Le Nguyen, Thao Nguyen Truong, Trong Nghia Hoang. Boosting offline optimizers with surrogate sensitivity. In Proceedings of the 41st International Conference on Machine Learning (ICML'24), pp. 10072-10090, Vienna, Austria, 2024.

[2] Tianhe Yu, Aviral Kumar, Rafael Rafailov, Aravind Rajeswaran, Sergey Levine, Chelsea Finn. COMBO: Conservative offline model-based policy optimization. In Advances in neural information processing systems 34 (NeurIPS'21), pp. 28954-28967, Virtual, 2021.

[3] Han Qi, Yi Su, Aviral Kumar, Sergey Levine. Data-driven offline decision-making via invariant representation learning. In Advances in neural information processing systems 35 (NeurIPS'22), pp. 13226-13237, New Orleans, LA, 2022.

[4] Can (Sam) Chen, Christopher Beckham, Zixuan Liu, Xue (Steve) Liu, Chris Pal. Parallel-mentoring for offline model-based optimization. In Advances in Neural Information Processing Systems 36 (NeurIPS'23), pp. 76619-76636, New Orleans, LA, 2023.

[5] Yassine Chemingui, Aryan Deshwal, Trong Nghia Hoang, Janardhan Rao Doppa. Offline model-based optimization via policy-guided gradient search. In Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI'24), pp. 11230–11239, Vancouver, Canada, 2024.

[6] Minh Hoang, Azza Fadhel, Aryan Deshwal, Jana Doppa, Trong Nghia Hoang. Learning surrogates for offline black-box optimization via gradient matching. In Proceedings of the 41st International Conference on Machine Learning (ICML'24), pp. 18374-18393, Vienna, Austria, 2024.

[7] Aviral Kumar, Sergey Levine. Model inversion networks for model-based optimization. In Advances in Neural Information Processing Systems 33 (NeurIPS'20), pp. 5126–5137, Virtual, 2020.

Thank you for your valuable and constructive comments. Below please find our responses.

Q1 The idea that MSE is not always optimal for offline opt was recently pointed out in the MATCH-OPT paper

Thanks for pointing out this important observation! MATCH-OPT [1] has mentioned that focusing on value matching alone is not suitable for offline MBO. In this paper, we conducted a thorough and comprehensive analysis, providing empirical evidence that OOD-MSE is not closely related to the final design quality in offline MBO (as shown in Figure 2 and Table 1) and demonstrating the significance of training an order-preserving surrogate model (in Theorem 1).

We have updated our paper to highlight this important observation of MATCH-OPT in our Introduction, Section 2.1, and Appendix A, with the changes colored in red. Thank you again for your valuable comments.

Q2 MSE is not worse than any other metric for optimization, but it could be harder to be estimated accurately for OOD samples

Good point! We agree that MSE is not bad for optimization, since a model that can predict well on OOD region is quite good to optimize inside it. However, prior works have pointed out that OOD-MSE is naturally difficult to learn [2-3]. Recent works like MATCH-OPT [1] and Tri-Mentoring [4] focus on including more information (e.g., gradient information and weak pairwise supervision signal), but these works are based on the regression-based framework trained with MSE. In this work, we thoroughly propose a novel framework based on ranking, which is a weaker condition since it only commands for correct orders, while minimizing MSE argues for both order preserving and value approximating. In Theorem 1, we have demonstrated that an order-preserving surrogate model is sufficient for offline MBO by stating the equivalence. We examined that ranking metric is more efficient than MSE for offline MBO from an empirical perspective in Figure 2 and Table 1. The experimental results also show the strengths of our framework.

Besides, in Table 3, we combined our framework with approaches in offline MBO by simply replacing the MSE term in their loss with ListNet and incorporating output adaptation, to further show the effectiveness of the ranking framework. Here we also provide new experimental results on MATCH-OPT + ListNet and original MATCH-OPT, consistent with our observation in Table 3.

We have revised to add these new experimental results to Table 3. Thank you.

Q3 It makes no sense to compare the two metrics MSE and AUPCC (i.e., AUPRC in the primitive version)

Thank you for your comment. As mentioned above, the ability to preserve order is sufficient for surrogate models in offline MBO, which is a weaker requirement than minimizing MSE. Given that AUPCC is a ranking-related metric and reflects how well the model identifies the top-$k$ designs, AUPCC can intuitively be better than MSE. As this is the first work to explicitly propose a ranking-based framework for offline MBO, we conducted a thorough and systematic analysis with empirical evidence and theoretical statement.

Q4 The top-$k$ info cannot be used during training, which is of little relevance to the method.

Thank you for your comment, but there may be misunderstandings. In this work, we do not just simply replace MSE loss with LTR loss (e.g., ListNet). Note that the ranking information (e.g., top-$k$ info) is limited in the primitive offline dataset with unordered $(\mathbf{x},y)$ pairs. For better ranking-based model learning, we apply data augmentation to construct a dataset to obtain and utilize the ranking information, where each entry is a list of items to be sorted. Specifically, as illustrated in line 3-7 in Algorithm 1, we first randomly sample $m$ design-score pairs from the offline dataset to form a list to be sorted, then repeat this for $n$ times to construct the augmented dataset with $n$ lists. With this technique, we can use ListNet loss to feed the ranking information (e.g., top-$k$ info) to the surrogate model training process. These details can be found in Section 3.3. Thus, we believe that our method is capable to leverage such information to train a better surrogate model.

Q5 How do prior techniques relate to the idea of ranking samples?

Thank you for your valuable comment. There are various approaches in the context of offline MBO, and here we introduce some that may relate to the idea of ranking samples from our comprehension:

• MATCH-OPT [1]. MATCH-OPT claims that value matching is not enough for offline MBO, and enforces the model to match the ground-truth gradient, which is approximated via interpolation on the constructed trajectories. The idea of matching gradient is related to ranking samples since a model with proper gradient could reflect the relationship in a small neighborhood.

• Tri-Mentoring [4]. Tri-Mentoring facilitates mentoring among three parallel proxies through a combination of a voting-based pairwise supervision module and an adaptive soft-labeling module, resulting in a more robust ensemble of proxies. Each proxy uses weak semi-supervised pairwise-ranking-based voting signals provided by other proxies to fix its predictions and finetune its weights.

• BONET [5]. BONET constructs monotonic trajectories to mimic the behavior of a black-box optimizer and uses them to train an autoregressive model, after which the final designs can be sampled under a regret-to-go heuristic. Here the trajectories are constructed by ranking the collected samples, from which the model may capture some ranking information.

In this work, we directly identify the idea of ranking samples, and conduct a systematic analysis on this view. After that, we reformulate the objective of the training process by replacing the core MSE loss with a ranking loss, and apply data augmentation and output adaptation for model training and solution search, respectively. Thanks to your suggestion, we have revised to add some discussion on the relationship between prior methods and ranking in Appendix A.

Besides, our proposed ranking-based framework and these approaches are not mutually exclusive; rather, our framework can be combined with such approaches to enhance model's generalization ability. In Table 3 and our experiment in the reply to Q2, we examine this by simply replacing the MSE loss with ListNet and incorporating output adaptation, which demonstrates the versatility of ranking loss and the possibility to apply prior methods in offline MBO to our proposed ranking framework.

Q6 The theory needs extensions

Thanks for your constructive suggestion. As mentioned in our reply to Q2, learning to rank samples correctly is a weaker condition than learning to minimize MSE, since minimizing MSE commands for both order preserving and value matching. The aim of proving the equivalence in Theorem 1 is to show that the weaker condition, order preserving, is sufficient, which motivates the proposal of directly learn the ranking information by leveraging LTR techniques.

As for the question you raised, ``in which scenario will LTR generalize better than MSE on OOD data?", we are also curious about the question when conducting this work. We tried to identify some theoretical support or evidence from prior works, but surprisingly we cannot find that even in the field of LTR.

We also attempted to analyze this point theoretically, but encountered several challenges. Below we briefly present the most promising approach we explored and the difficulties we face.

• Try to find a special function class $\mathcal{F}$, from which the ranking model $\hat{f}$ to be learned is, such that models learned with LTR techniques have an upper bound guarantee on some ranking measure while models trained with MSE do not. Formally, let $R$ be a ranking measure (which can be the expected risk of a specific ranking loss or a ranking metric, e.g., NDCG), and denote the empirical risk of model trained with LTR and that trained with MSE as $\hat{R} _ {LTR}$ and $\hat{R} _ {MSE}$, respectively. For ease of exposition, $R$ here refers to the expected risk in Theorem 2. From Theorem 2, $R$ can be upper bounded by $\hat{R} _ {LTR}$ as

$$\sup (R - \hat{R} _ {LTR}) \leq \frac{C _ {\mathcal{A}} (\phi) N(\phi) }{\sqrt{n}} + \sqrt{\frac{2\ln (2/\delta)}{n}}.$$

Then, if we could find that for $\mathcal{F}$, $R - \hat{R} _ {MSE}$ always has a slower convergence rate, i.e.,

$$R - \hat{R} _ {MSE} \geq \mathcal{O}(\frac{1}{\sqrt{n}}),$$

we can show that models learned with MSE are worse than that learned with LTR. However, such an analysis can be difficult because: 1) There is no theoretical evidence to show the generalization bound on ranking by optimizing MSE. 2) Most generalization bound analysis in LTR assume i.i.d (as Theorem 2 in our paper), while OOD analysis in LTR is quite limited.

• Identify a special case that supports this intuition. Assume that the function class $\mathcal{F}$ is a linear function class and the offline data is drawn from a ground-truth function $f$ with long-tailed noise on the objective value. Models trained with MSE are susceptible to heavy-tailed noise, as the mean of $y$ is heavily influenced in regions with such noise. In contrast, models trained with pairwise ranking loss demonstrate greater stability in such scenarios. An illustrative example could be as follows. Assume that the ground-truth function is $f(\mathbf{x}) = \mathbf{x}^2$ and the offline dataset $\mathcal{D}=$ { $(1,1), (1.9, 3.7), (2.1, 4.5), (2, -12)$ } where $(2, -12)$ suffers from the heavy-tailed noise. Then, the model trained with MSE would exhibit negative correlation, while that trained with LTR would demonstrate positive correlation, which shows that the model trained with LTR is more robust. However, such counterexamples are still based on strong assumptions. A well-constructed example with theoretical support remains unexplored.

Above are the possible approaches that we have attempted from the theoretical view. Thank you for pointing this out, and we have revised to discuss the probable approaches and difficulties for theoretical analysis in Appendix C. We also welcome the reviewers to discuss on the topic for this emerging field.

Q7 Hard to conclude from results on offline optimization papers that any method is more significant and robust

Thanks for your valuable comment. We also notice this issue. To make the results in our paper reproducible, we use the open-source code provided by each paper and set hyper-parameters carefully according to the description in their papers. We have mentioned this in Appendix E.4, and listed the links of all open-source repositories we used.

Besides, in order to help the community better reproduce the results, we will open-source not only all the codes but also the model weights in our paper, as we mentioned in the readme.md file of our supplemental files.

Q8 Lack of comparison to the DDOM method [6]

We compared DDOM in line 11 of Table 2 in our submitted version. We believe that we have done a comprehensive comparison to other 20 methods. These experimental results clearly demonstrate the superior performance of our ranking-based model over regression-based model.

We hope that our response has addressed your concerns, but if we missed anything please let us know.

References

[1] Minh Hoang, Azza Fadhel, Aryan Deshwal, Jana Doppa, Trong Nghia Hoang. Learning surrogates for offline black-box optimization via gradient matching. In Proceedings of the 41st International Conference on Machine Learning (ICML'24), pp. 18374-18393, Vienna, Austria, 2024.

[2] Brandon Trabucco, Xinyang Geng, Aviral Kumar, and Sergey Levine. Design-Bench: Benchmarks for data-driven offline model-based optimization. In Proceedings of the 39th International Conference on Machine Learning (ICML'22), pp. 21658–21676, Baltimore, MD, 2022.

[3] Brandon Trabucco, Aviral Kumar, Xinyang Geng, Sergey Levine. Conservative objective models for effective offline model-based optimization. In Proceedings of the 38th International Conference on Machine Learning (ICML'21), pp. 10358-10368, Virtual, 2021.

[4] Can (Sam) Chen, Christopher Beckham, Zixuan Liu, Xue (Steve) Liu, Chris Pal. Parallel-mentoring for offline model-based optimization. In Advances in Neural Information Processing Systems 36 (NeurIPS'23), pp. 76619-76636, New Orleans, LA, 2023.

[5] Siddarth Krishnamoorthy, Satvik Mehul Mashkaria, Aditya Grover. Generative pretraining for black-box optimization. In Proceedings of the 40th International Conference on Machine Learning (ICML'23), pp. 24173–24197, Honolulu, HI, 2023.

[6] Siddarth Krishnamoorthy, Satvik Mehul Mashkaria, Aditya Grover. Diffusion models for black-box optimization. In Proceedings of the 40th International Conference on Machine Learning (ICML'23), pp. 17842-17857, Honolulu, HI, 2023.

Dear Reviewer ChpN,

Thank you very much for taking the time to carefully review our work and provide your constructive comments. Your feedback has been invaluable in helping us improve our work.

In response to your comments, we have included the following content:

1. We have updated our paper to highlight the observation of MATCH-OPT. We also combined MATCH-OPT into our framework and conducted experiments in our response to Q2. The experimental results show further improvement in incorporating MATCH-OPT into our proposed ranking framework.
2. We have introduced some prior techniques that implicitly relate to the idea of ranking samples or utilize the ranking information in Appendix A.1 and our reply to Q5, while our work directly identifies this point and conducts a more systematic analysis.
3. We have discussed the approaches for further theoretical analysis and given a theoretical example in Appendix C.1 and our response to Q6, where LTR losses are superior to MSE in obtaining more robust performance.

Besides, as suggested by Reviewer 9jkq, to better understand the theoretical example, we additionally conducted numerical experiments in Appendix C.2 and our response to Reviewer 9jkq. These results further support our theoretical findings.

May we kindly ask if our response has addressed the issues you raised? If you still have any further questions or concerns, we would be more than happy to continue the discussion.

Thank you again for your time and effort.

Best regards,

Authors