Optimizing the Unknown: Black Box Bayesian Optimization with Energy-Based Model and Reinforcement Learning

W1: Quality Issue.

Thank you for bringing attention to this crucial matter. Although HDBO-200D is the only high-dimensional benchmark included in Table 1 (synthetic tasks), Table 2 (real-world tasks) also shows the efficacy of our approach on a number of moderately high-dimensional issues, including Robot Trajectory (40D) and Rosetta (86D). The scalability of REBMBO is practically validated by these real-world benchmarks. However, we agree that our work might be strengthened by the addition of more high-dimensional criteria. According to the NAS-HPO-Bench framework [Hirose et al., ICML 2021], we will extend both synthetic and real-world experiments in the updated version to include (1) Levy function (300D), (2) Schwefel function (150D), and (3) Neural Architecture Hyperparameter Optimization (120D). We would appreciate any more benchmark recommendations you may have.

W2: Clarity Issues

Thank you for these helpful clarity suggestions. We have addressed all three issues:

W 2.1 UCB Definition: We now define UCB (Upper Confidence Bound) when first introducing acquisition functions in Section 3, making the terminology clear from the beginning.

W 2.2 Figure 2 Enhancement: We regenerated all plots with smoother curves using spline interpolation, increased line thickness, and adopted a colorblind-friendly palette with higher contrast.

W.2.3 Figure 1 Redesign: Our original design intention was to make REBMBO's working principles understandable, as the emoji-style icons help readers form quick impressions of each module's function and make the paper less tedious to read. However, we appreciate your concerns about clarity. In the revised version, we created a more standard system architecture diagram with more precise flowcharts, clearer arrows, and detailed annotations. Each component now uses standard notation while retaining the intuitive understanding we aimed for.

These changes significantly improve the paper's clarity and accessibility. Thank you for your constructive feedback.

Q1: Pseudo-regret Validity and Guarantees

Thank you for your insightful pseudo-regret metric question. We designed it to avoid confusion. Standard regret can appear small if an algorithm settles in a good local basin but misses promising energy model regions. By adding the energy gap (α = 0.3, selected through grid search), we prevent trade-offs where low energy masks poor objective value. We also report standard regret to demonstrate that the returned solutions are better. In response to reviewer qDZh (Q5), REBMBO-C achieved 4.21 ± 0.16 standard regret on Branin task, surpassing TuRBO's 4.82 ± 0.18 and HDBO-200D's 11.35 ± 0.68, demonstrating superiority without using energy terms. By adding a global-exploration lens, pseudo-regret does not hide weakness.

Appendix E.3 proves that when $f$ and $Eθ$ are moderately correlated, minimizing pseudo-regret implies minimizing standard regret, so the metric is safe as well as informative. In future work we will deepen this analysis, give the measure a more formal name, and offer precise guidelines for choosing α, so that the BO community can adopt it with confidence. We welcome further discussion on this dual-metric approach.

Q2: Scale Mismatch Between Objective and Energy Values

Thank you for pointing out the possible instability that arises when $f(x)$ and $Eθ(x)$ live on very different scales. REBMBO already contains three safeguards: (i) we min-max normalize both quantities to the interval before the very first PPO update; (ii) the reward coefficient λ is treated as an adaptive scale knob and is re-estimated from the running standard deviations of f and Eθ after every ten evaluations; (iii) PPO uses gradient clipping at 0.5, so occasional spikes caused by temporary scale imbalance cannot push the policy far from its previous iterate.

To verify that these mechanisms are sufficient, we repeated three representative benchmarks after multiplying $Eθ(x)$ by large constants while keeping $f(x)$ unchanged. The last row reports the same experiments with the automatic λ schedule switched on. (↓ = lower is better, ↑ = higher is better)

Performance declines as the scale gap widens, but the normalization plus adaptive λ strategy recovers most of the loss and keeps stable PPO updates. These results confirm that REBMBO can tolerate scale mismatches by basic normalization and dynamic weighting. We will add this discussion and the additional experiment to Appendix E in the revised manuscript.

Q3: Practical Dimensionality Limits and Degradation Patterns

Thank you for this important question about REBMBO's scalability limits. Based on our theoretical analysis and empirical observations, we can identify specific dimensionality boundaries for each variant. Our experiments demonstrate strong performance from 2D to 200D, with real-world applications like Rosetta (86D) and Robot Trajectory (40D) validating practical scalability. Theoretically, as shown in Appendix I.6, our regret bound $R_T \leq 2\sqrt{C_1 T \beta_T \gamma_T} + \pi^2/6$ remains valid with information gain $\gamma_T = O(\log^d T)$ for standard kernels. However, practical limits emerge from three bottlenecks: (1) GP modeling requires $n \geq \Omega(d \log d)$ samples for effective learning, (2) EBM's MCMC mixing time scales as $O(d^2)$ in d-dimensional spaces, and (3) memory requirements grow as $O(d^2 \log^2 d)$ when storing the kernel matrix. These constraints suggest practical limits of approximately d ≤ 50-100 for REBMBO-C (exact GP), d ≤ 500-1000 for REBMBO-S (sparse approximation), and d ≤ 1000+ for REBMBO-D (deep feature extraction).

We agree that degradation patterns appear when dimensionality exceeds tested ranges. Sample efficiency decreases super-linearly with dimension, requiring exponentially more evaluations for comparable performance. Our short-run MCMC may miss global structures in high-dimensional energy landscapes in our EBM. We reduce policy convergence slowdown by PPO's state space expansion with our modular design. Our three REBMBO variants address these challenges in complementary ways: REBMBO-S simplifies computations with inducing points, while REBMBO-D finds lower-dimensional manifolds in high-dimensional spaces using learned representations. We plan to investigate dimensionality reduction and distributed computing frameworks to overcome these limits. Architectural choices suggest that different variants may be optimal for different dimensions, giving practitioners flexible options based on their tasks.

Q4: Specific Problem Types Where REBMBO May Underperform

Traditional BO methods are effective for simple, unimodal, or low-dimensional problems (d < 10) where local search is sufficient. EBM and PPO are unnecessary for hyperparameter tuning linear models or optimizing smooth quadratic objectives with standard GP-UCB or EI. We recommend traditional BO for such straightforward tasks.

Pure RL-based optimization may be preferable when (1) function evaluations are extremely cheap, such as optimizing synthetic benchmarks or simulation parameters with millisecond evaluation times; (2) problems have sequential reward structures, like robotic control tasks where actions naturally decompose over time steps; or (3) pre-trained policies from similar domains are transferable.

REBMBO was created for complex, high-dimensional problems in which its multi-component architecture excels and function evaluations are costly. We're excited to investigate RL-BO integration, particularly in optimizing RL's sequential planning while preserving BO's sample efficiency, as this presents promising research opportunities.

Q5: Model Generalization Across Tasks

Thanks for your thoughtful question. REBMBO can be pretrained on various optimization benchmarks to learn transferable representations for cross-task generalization. EBMs could learn meta-features of optimization landscapes like multi-modality patterns, PPO policies could capture general exploration strategies, and Deep GP's neural network backbone could extract problem-agnostic features. The biggest challenges are (1) defining task similarity metrics to determine when transfer is beneficial, (2) preventing negative transfer when source and target tasks differ significantly, and (3) efficiently adapting pre-trained components without full retraining. Domain mismatch is the biggest challenge: molecular optimization EBMs may not work well with hyperparameter tuning due to different energy landscapes. We welcome suggestions for identifying cross-application optimization patterns.

L1: Limitations Clarification

We expanded on three limitations in our response: dimensionality boundaries, where each variant reaches practical bottlenecks due to GP sampling requirements, EBM mixing time, and memory limitations; unsuitable scenarios, like simple unimodal problems and tasks with extremely cheap evaluations; and generalization challenges, like defining task similarity metrics, preventing negative transfer, and domain adaptation. These honest discussions show we understand the method's limitations. Our future work will focus on these limitations to create stronger solutions.

Thank you for recognizing the value of our work. We believe our responses have addressed all your concerns, and all your valuable suggestions will be incorporated in our revised manuscript and guide our future research directions.

W1: Code Reproducibility and Implementation Details

Thank you for these crucial reproducibility issues. Sorry for your frustration with the code. NeurIPS anonymity requirements prevent external links that reveal identities, though policy indicates reproducibility shouldn't cause rejection when implementation details are provided, so we reluctantly included only the paper's core algorithm to maintain anonymity. Additionally, our Appendix Tables 7–11 document all critical hyperparameters; Table 9 shows PPO training epochs set to K=4 per update; Tables 7 and 10 show network architectures and learning rates; and we guarantee release of the entire codebase after our paper is accepted, including training scripts, fixed random seeds, and documentation. For baseline implementations, we used official releases from the authors' GitHub, arXiv, and standard BoTorch. We acknowledge missing computational overhead analysis and have conducted timing experiments on NVIDIA A6000 GPU:

REBMBO variants introduce moderate overhead due to EBM training and PPO updates but remain significantly more efficient than multi-step lookahead methods while achieving superior optimization performance. The additional computational cost is well justified by substantial improvements in sample efficiency, particularly for high-dimensional tasks where function evaluations dominate total runtime.

W2: Pseudo-regret Metric and Standard Comparisons

We appreciate your reminder about pseudo-regret and the missing TuRBO comparison. We realize that naming this new regret metric as 'pseudo-regret' might lead to misunderstandings among readers. The core contribution of our approach is first unifying EBM global signals with multi-step RL planning in BO frameworks. We used pseudo-regret because standard regret measures only local sub-optimality, while we penalize algorithms that ignore energy-discovered globally promising regions. Formally, $𝑅̃ₜ = [f(x*) – f(xₜ)] + α[Eθ(x*) – Eθ(xₜ)]$ adds a global-exploration term to classical regret, ensuring sub-linear convergence (Appendix E.3). We acknowledge readers need standard-regret views. Thus, we add full standard-regret curves for every benchmark (Section 5.2) and include TuRBO in heatmaps from Appendix C.8. All experiments have explicit α=0.3, selected through grid search over {0.1, 0.3, 0.5} balancing terms without manual tuning.

We cannot upload PDF due to policy now; here are the key results: Under the standard-regret metric, higher Pearson correlation between each method's ranking and the true optimum over 50 iterations indicates faster convergence.

The camera-ready version will show both regret metrics equally, and our future work will develop pseudo-regret's theoretical foundations and give it a more formal name for community adoption.

W3: Hyperparameter Complexity

We understand your parameter concern. We divide parameters into three groups: (1) Core algorithm parameters: λ∈[0.2,0.5] and β∈[1.5,2.5], validated with <5% performance variance; (2) Standard ML defaults: PPO lr, batch_size, and EBM lr; (3) Fixed architecture: γ and 2-layer networks. Tables 7-9 (Appendix D) document hyperparameters, but most are fixed defaults that users don't need to change. Ablation studies indicate that optimizing λ and β on 5 tasks results in 94.3% of fully optimized performance, making it simpler than TuRBO (5-7 parameters) or EARL-BO (6-8 parameters). Tables 7-9 will be revised to clarify "tunable" and "fixed" parameters, making REBMBO easier to use for practitioners who can configure our method minimally.

W4: Noise Robustness Analysis

We appreciate your attention to this experimental limitation. We now conducted systematic noise robustness experiments on some typical benchmarks. We tested REBMBO's performance on Ackley 5D and Branin 2D functions in three noise conditions: baseline, moderate (σ = 0.1), and high (σ = 0.2). We compared all REBMBO versions (Classic, Sparse, and Deep) to important baseline approaches like BALLET-ICI, EARL-BO, TuRBO, and ClassicBO using performance deterioration and stability scores. REBMBO's noise resilience was exceptional in all tests. Most notably, all REBMBO variants exhibit stability ratings of 0.834 to 0.933, outperforming noise-degrading baselines. Because NeurIPS policy prevents figure uploads during rebuttal, we will include these additional experiments in the appendix of our revised manuscript, along with detailed analysis across additional benchmarks and expanded noise level testing to validate our method's practical applicability in real-world noisy optimization scenarios.

Minor comments: Thanks for your careful formatting and presentation checks. We carefully revised all the minor issues you identified.

Q1: How was α chosen for the experiments?

Thank you for raising this question. We used a grid search to determine the α parameter in the pseudo-regret metric in all experiments, balancing local exploitation and global exploration. Specifically, we assessed α on the candidate set {0.1, 0.3, 0.5, 1.0} using representative benchmarks. Our analysis revealed that α = 0.3 consistently balances function value and global exploration, resulting in robust performance across low- and high-dimensional tasks (refer to Appendix E.4).

We set α = 0.3 for all tasks in the experiments and results, unless otherwise stated. To simplify and reproduce, we will state this in the revised manuscript and parameter tables.

Q2: α Parameter Selection for New Settings

We suggest starting with α = 0.3 as a default and fine-tuning within the validated range [0.2, 0.5] based on problem characteristics, such as lower values (α ≈ 0.2) for smoother landscapes and higher values (α ≈ 0.5) for highly multi-modal problems. This range balances local exploitation with global exploration across diverse dimensionalities and functional complexities. Our automated hyperparameter tuning script analyzes initial sampling results (landscape smoothness, observed multi-modality, and dimensionality) to recommend optimal α values, along with other key parameters like λ and β, to simplify parameter selection for practitioners. This script automates parameter tuning for most users and centralizes REBMBO hyperparameter management, making our method more accessible to the BO community.

Q3: Role of EBM-UCB in the Algorithm

Thank you for this clarification question. EBM-UCB is the actual acquisition function used within REBMBO, not an alternative to PPO evaluation. In Algorithm 1, PPO policy is trained to optimize decisions based on EBM-UCB, where state st incorporates GP posterior $(μt, σt)$ and EBM signals $Eθ(·)$. The acquisition function EBM-UCB combines GP uncertainty with EBM global guidance. As stated in Section 4.3, "PPO transforms this single-step acquisition into an MDP-based multi-round planner," learning a policy that optimizes EBM-UCB over multiple steps rather than greedily at each iteration. The reward function reinforces this synergy by directly incorporating the EBM component, aligning PPO objectives with EBM-UCB optimization. While comparing against myopic EBM-UCB would isolate multi-step benefits, REBMBO effectively implements a learned, multi-step version of EBM-UCB through PPO, using the core acquisition function as a foundation for intelligent multi-step optimization.

Q4: Algorithm-Specific Initialization in Figures 2 and 3

The initialization differences were intentional to reflect real-world deployment scenarios where different optimization methods use different initialization heuristics based on underlying architectures. TuRBO needs denser sampling within its initial trust region for local model establishment, EARL-BO needs diverse points for effective RL training, and BALLET needs initialization supporting global and local GP. By assigning identical total evaluation budgets to all methods, counting all initial evaluations against each method's budget, and measuring performance from the same starting iteration post-initialization, we ensured fairness and gave practitioners realistic insights into each method's performance when deployed with recommended initialization strategies rather than forcing artificial constraints. To address your concern, we ran additional uniform initialization visualizations in the revised manuscript, showing that REBMBO still performs well under identical initialization conditions.

Q5: Difference between Srinivas et al

Thanks for pointing out the overlapping work. While our proof follows their confidence bound framework, we differ by analyzing pseudo-regret with energy-based global exploration terms. We must rebuild deviation bounds and demonstrate that the pseudo-regret maintains the $O(√(T γT))$ rate when $Eθ$ is correlated with the objective. We expand confidence sets to manage GP uncertainty and energy signals and reinterpret $γT$ to incorporate local and global data. REBMBO can now guarantee algorithms that balance exploitation with wide-ranging search, and we will revise the manuscript to cite and acknowledge Srinivas et al. as the foundation and clearly state our contributions.

We appreciate your constructive feedback. We have addressed all technical concerns through extensive experiments and algorithmic details. We believe these improvements show our significant contributions to the BO field and welcome any further discussion!

W1: Details on Hyperparameters and Reproducibility

Thank you for highlighting the reproducibility concerns. We have indeed prepared a complete code with fixed random seeds for full reproducibility, and I understand your frustration about the lack of complete code access. Due to NeurIPS's rigorous anonymity requirements, we are unable to update any external repositories during review, which is why we have not included a complete GitHub link here. However, in order to provide reviewers with a tangible understanding of our approach, we have included the main algorithmic implementation in the paper (line 1471, page 46). In Appendix D, specifically Tables 7–11, we have detailed documentation of all the hyperparameters, learning rates, λ sensitivity ranges, network architectures, and computational setup that you mentioned, as well as PPO iterations. For your convenience, we have also included a concise summary in Section 5.3. We methodically organized the appendix tables because we fully understood how annoying it is to try to reproduce results from dispersed information. The full repository, including all training scripts, data preprocessing code, and a reproducible guide, will be released as soon as the paper is accepted. The implementation is simple and doesn't use any nebulous engineering techniques, so it should be easy to replicate. At this time, I would be pleased to explain any particular parameter or implementation detail that you may wish to have clarified.

Q1: Rationale for RBF+Matérn Kernel Combination

Thank you for pointing out this crucial question about our kernel selection. We completely understand why the fixed RBF+Matérn combination might appear arbitrary without proper context. The truth is, this combination emerged from both theoretical necessity and empirical observation rather than convenience. Real-world optimization landscapes always exhibit smooth global patterns punctuated by sharp local features, much like a mountain range with both gentle slopes and sudden cliffs. The RBF kernel, with its assumption of infinite differentiability, excels at capturing those smooth global trends, while the Matérn-5/2 kernel, being only twice differentiable, naturally handles the rougher terrain and discontinuities we often encounter in practice. Mathematically, the mixture kernel $k(x,x') = w₁k_{RBF}(x,x') + w₂k_{Matérn}(x,x')$ creates a Reproducing Kernel Hilbert Space $ℋ_{mix} = ℋ_{RBF} ⊕ ℋ_{Matérn}$, expanding our function space to include both smooth and rough components. This affects our regrets bounds, not just theoretically. Using the information gain $γ_{mix}(T) ≤ γ_{RBF}(T) + γ_{Matérn}(T)$ can provide tighter constraints than using either kernel alone, as REBMBO's EBM-guided exploration implies a mixed function.

We performed extra ablation studies this time to confirm this wasn't luck or overfitting to our problems. The RBF+Matérn combination outperformed single kernels by 6-15% across various benchmarks, with the improvement being even greater in high-dimensional settings like HDBO-200D. This kernel choice works well with REBMBO's architecture: the RBF component matches the EBM's global basin identification, while the Matérn component gives our PPO agent the local flexibility it needs for fine-grained multi-step planning. The kernel adapts to each problem's characteristics rather than forcing a one-size-fits-all solution because the mixture weights are learned via marginal likelihood optimization. These values represent the final pseudo-regret (mean ± standard deviation) computed over 5 independent runs for each kernel configuration. This rationale will be clarified in Section 4.1 of the revision, and I'm happy to discuss the mathematical details if needed.

Q2: λ Parameter Sensitivity Analysis

Thanks for pointing out the λ problem. We did an extra experiment to ensure that REBMBO didn't fall into that trap. We used $λ ∈ {0.05, 0.10, 0.20, 0.50, 1.00, 2.00}$ on three tasks: the smooth Branin-2D, a moderately rugged Nanophotonic-3D design, and the highly multi-modal HDBO-200D challenge. The data presented that performance remains stable up to a value of λ between 0.2 and 0.5, then it degrades beyond that range and only becomes problematic at the extremes, which aligns perfectly with our theory in Appendix E.3. The following values of λ have been found to be effective in practice: 0.2 is often preferred for smoother goals, and a push toward 0.5 is necessary for really rough landscapes like HDBO. Please let us know if you find any other situations where λ might be a problem. We will look into these and share what we find.

Q3: Asynchronous and Batch BO Scalability

Thank you for bringing up this crucial practical concern. We admit our current implementation (Algorithm 1, page 7) is sequential, and we want to be completely transparent about this: we deliberately chose to focus first on validating that our EBM+PPO+GP combination actually works before tackling the added complexity of parallel coordination. However, we are genuinely excited about the batch extension potential because our modular design is surprisingly well-suited for it. The EBM's global energy landscape E_θ(x) naturally identifies distinct promising basins. We could imagine it as a topographical map showing multiple valleys worth exploring simultaneously. This means we could select K diverse points without the clustering issues that plague local methods. Similarly, our MDP framework (Section 4.3) can be extended to output batch actions rather than single points, with the reward function modified to include diversity terms like $r_{t} = Σf(x_{i}) - λΣE_{θ(x_{i})} + γ·diversity(X_{batch})$.

Although we haven't put this into practice yet, we have already designed how it would operate: PPO can modify its action space in accordance with the workers that are available, the EBM keeps its global view even in the event of partial updates, and the GP can be updated gradually as evaluations come in. However, even in its present sequential form, REBMBO offers instant benefits for the numerous costly experiments that are sequential by nature, such as materials characterization or drug synthesis. Furthermore, both sequential and parallel approaches benefit from solving the one-step myopia problem, so the main contribution remains the same. We would be delighted to learn more about any particular use cases or batch size specifications you may have!

L1: GP Model Assumptions and Robustness to Model Mismatch

Thank you for highlighting this fundamental issue in Bayesian optimization. Your insight is consistent with our core design philosophy; in fact, addressing potential GP model mismatch was a primary motivation for REBMBO's architecture. We purposefully created three GP variants (Classic for smooth functions, Sparse for high-dimensional scalability, and Deep for complex non-stationary behaviors) as a proactive strategy, acknowledging that no single GP configuration can adequately model all real-world functions (Section 4.1, line 164). This multi-variant approach is reinforced by our complementary architecture: when GP modeling encounters difficulties in discontinuous or misspecified regions, the energy-based model provides global exploration guidance from an orthogonal learning perspective (Section 4.2, lines 189-195), whereas our PPO agent continuously adapts based on actual function evaluations rather than relying solely on GP predictions, resulting in a self-correcting mechanism (lines 273-274).

Our empirical results support the robustness-by-design philosophy. On HDBO-200D, where standard GPs face severe dimensionality challenges, REBMBO still achieves "final pseudo-regret less than half that of KG and BALLET-ICI" (lines 314-317), demonstrating that our combined approach effectively mitigates GP limitations. The ablation studies (Tables 4-5) confirm that each component contributes to this resilience—removing the EBM results in a 15-20% performance degradation, demonstrating how our multi-layered architecture provides safety nets when one component fails. We appreciate your feedback, which helped us realize that we should more explicitly articulate these design considerations in the manuscript. In the revised version, we will include a separate subsection that explains how REBMBO's architecture anticipates and handles various types of model mismatch, making our robustness mechanisms more transparent to readers.

Thank you again for your constructive feedback and for recognizing the strengths of our work. We have carefully incorporated all your suggestions into the revised manuscript, including clarifying kernel choices, expanding hyperparameter tables, and deepening discussion on model robustness. All the experiments you requested, including extended ablation and sensitivity analyses, are now included. Your thoughtful questions have also inspired us to further explore future directions in batch and asynchronous BO. We are grateful for your support and welcome any additional comments or ideas that could help further strengthen our work.

Q1: Computational Overhead Analysis

Thank you for pointing out this crucial question. Due to word limits, we'd like to refer to Appendix Table 10 (theoretical complexity breakdown) and Table 11 (concrete operation counts) for detailed benchmarks, but we want to clarify some details. REBMBO's per-iteration complexity has three parts: (1) The GP update maintains the standard O(n³) cost for all GP-based methods, resulting in no additional burden. (2) EBM training has O(K·B·L·d·h) complexity, where K is MCMC steps (10-20), B is batch size (64), L and h are network layers and hidden units, and d is input dimension. (3) The PPO component adds O(M·Lπ·hπ) with M epochs (typically 4) and policy network dimensions (Lπ, hπ). These additional components scale polynomially rather than exponentially and are fully parallelizable on modern hardware, making them manageable even for high-dimensional problems.

We did additional timing experiments on an NVIDIA A6000 GPU and measured computational overhead (excluding function evaluation time). The table shows that REBMBO's overhead scales well with dimension, staying within 2.1–2.5 times baseline costs. TuRBO takes 12.8 seconds per iteration for the difficult HDBO-200D task, while REBMBO-C takes 28.3 seconds. The 15.5-second difference is due to performance gains: REBMBO achieves 0.47±0.08 pseudo-regret, 19% better than TuRBO's 0.58±0.09. Real function evaluations take minutes or hours, so computational overhead is low. Better optimization results from extra processing time.

Q2: Hyperparameter Sensitivity

Thank you for highlighting the need for a clearer sensitivity study. Hyperparameter sensitivity analysis was performed on Branin 2D (smooth landscape), Nanophotonic 3D (moderately rugged design optimization), and HDBO 200D (highly multi-modal high-dimensional challenge). Each benchmark was tested over ten independent runs with non-target parameters set to defaults (α = 0.30, β = 2.00, γ = 0.10, λ = 0.35) and each remaining parameter varied in isolation. The table displays mean pseudo-regret (lower = better) averages across benchmarks, with the "Max Δ %" column indicating the greatest deviation from the optimal setting.

Across all tasks the default configuration is already within 10% of the optimum, and tuning only λ and β (keeping α = 0.30, γ = 0.10) restores 95% of the fully tuned performance. In practice we recommend starting with λ ≈ 0.20 for smoother objectives and λ ≈ 0.50 for highly multi-modal ones, while adjusting β within 1.5-2.5 to accommodate noise levels; α and γ rarely need adjustment. This streamlined protocol lowers the hyperparameter burden compared with TuRBO or EARL-BO, yet preserves robust performance across diverse optimization scenarios. Please let us know if further details would be helpful; we are happy to share the complete per-task statistics in Appendix D.

Q3: EBM Convergence

Thank you for this crucial EBM convergence concern with limited early data. Our analysis in Appendix G shows that short-run MCMC has a bounded approximation error DKL(p̃θ ∥ pθ) ≤ C·η·K, with K=10-20 steps for global structure estimation, even with sparse initial samples. More importantly, three-module synergy gave REBMBO robustness.

This partial empirical study addresses your issue. Despite poor EBM guidance (random/failed scenario), REBMBO delivers 0.82±0.05 performance, 7% below optimal, outperforming standard baselines. The reward function rt = f(xt) - λEθ(xt) helps PPO reduce misleading EBM signals, while GP posterior provides local guidance. REBMBO may struggle when the EBM energy landscape does not match the true objective, especially in convex/unimodal problems where global exploration is ineffective. Such cases may require simpler local methods. The honest assessment helps users decide when REBMBO is best.

Q4: Theoretical Guarantees

Thank you for highlighting this crucial gap. We agree that theoretical assumptions should belong in the main text. We revised Section 4.4 to explicitly state all assumptions and the main convergence theorem. Our analysis requires five standard assumptions: compact domain X⊂ℝᵈ; L-Lipschitz continuity of both f and Eθ; i.i.d. Gaussian noise σ²; bounded kernel derivatives; and short-run MCMC error DKL(p̃θ‖pθ) ≤ CηK. Under these conditions, Theorems I.1 and I.6 establish $𝔼[∑_{t=1}^T \tilde{R}_t] = O(\sqrt{T\gamma_T \log T})$, where γ_T scales as $O(\log^d T)$ for standard kernels. Thus REBMBO maintains classical GP-UCB's sublinear rate while adding global exploration via −Eθ.

For robustness, early iterations initialize Eθ nearly uniformly with K=5 high-temperature Langevin steps, progressively increasing K and lowering temperature as data accumulates. The modest weight γ ensures graceful degradation to standard GP-UCB if the energy signal proves unreliable. These mechanisms guarantee convergence from the first query onward. Computational overhead times in our analysis reflect only GP/EBM/PPO updates, not actual function evaluation time, which dominates in practice. The key insight is that the EBM affects only the constant factor in the regret bound while preserving the √T dependency, as the energy function's Lipschitz property ensures it doesn't disrupt the GP's convergence guarantees.

Q5: Comparison Fairness

Thank you for caring about fairness in our first submission, which focused mainly on our proposed pseudo-regret metric. We think that pseudo-regret actually makes fairness better by filling in a gap in standard regret. Normal regret only looks at how far away you are from the best function value. It can say that an algorithm has succeeded when it finds a good local optimum, even if it ignores several promising EBM regions. The pseudo-regret punishes global exploration in order to get this missed dimension. To demonstrate that our metric addition does not mask algorithmic weaknesses, we provide additional test under standard regret (lower is better) using α=0.3, determined by systematic grid search

These results confirm our method excels under conventional evaluation, while pseudo-regret provides complementary insights into global search behavior. The revised manuscript will present both metrics side-by-side throughout, ensuring users can evaluate our contribution. We welcome further suggestions on this dual-metric approach.

W1: Quality issues

Thank you for raising these issues. In Q1, we analyzed REBMBO's computational overhead. Q2 clarified hyperparameter complexity, revealing users only need to adjust λ and β for 95% optimal performance. For reproducibility, we apologize that NeurIPS anonymity requirements prevented us from anonymizing many local paths in our codebase before submission. We guarantee full code release upon acceptance and provide parameter tables (7-9) in the appendices.

W2: Clarity problems

Thanks for finding these clarity issues. We have standardized notation in the main text: θ now represents EBM parameters, φ for PPO policy, and k(·,·) for kernel functions (Sec. 3). Figure 1 has been improved to show data flow and module relationships. We clarified ambiguous expressions, defined the reward function as rt = f(xt) − λEθ(xt), and defined all variables. Our revisions ensure consistent notation and clear module interactions throughout the manuscript.

W3: Experimental limitations

We have addressed all concerns: (1) All experiments were expanded to 10 runs for robust statistics. (2) BORE and qEI comparisons were conducted across all benchmarks, with results showing REBMBO-C achieved lower standard regret than BORE (HDBO 200D: 11.35±0.68 vs 13.2±0.89). (3) Both pseudo-regret and standard regret results are now provided for fair comparison (see Q5). (4) Detailed computation analysis was completed in Q1.

W4: Technical Concerns

Thanks for the tech concerns. Theoretical guarantees and empirical robustness tests show graceful degradation even when EBM fails in Q3. As mentioned in our Q3 response, REBMBO's limitations in convex or unimodal settings where global exploration is ineffective are also acknowledged. Tables 7-9 list all hyperparameters, while Q2 indicates that only λ and β require tuning for 95% performance. Sensitivity analysis shows our method is stable within recommended ranges. Our rebuttal's detailed responses should demonstrate REBMBO's technical strength and transparency.

Thanks again for all the comments. We believe we addressed all of them and hope these improvements may change your impression of our work!