Enhancing Zero-Shot Black-Box Optimization via Pretrained Models with Efficient Population Modeling, Interaction, and Stable Gradient Approximation

Weakness 1: The innovations ...

Our submission first summarizes the principle of strong generalization of POM and then enhances it. The details are as follows:

Their core idea is to learn the mapping between task features and optimization strategies, enabling the model to directly output high-quality optimization strategies without fine-tuning for new tasks.

We summary this process into three key elements: task feature modeling, optimization strategies expression, and a mapping mechanism from task features to optimization strategies. First, the task feature set, typically consisting of normalized fitness values and centralized rankings of individuals, characterizes the structural characteristics of the optimization task. Its expressive power directly determines the POM's ability to perceive and adapt to different tasks. Second, the optimization strategies, as the output of the POM, must possess sufficient expressive power to cover search behaviors across diverse tasks. The mapping from features to strategies, learned during the pre-training phase, is key to achieving zero-shot performance.

Although existing POM methods have achieved some success, significant bottlenecks remain in feature modeling, strategies expression, and training stability. To this end, this paper proposes systematic improvements around the three core elements mentioned above, building a POM system with enhanced generalization and training robustness:

Enhanced Task Feature Modeling: To improve the expressiveness of task features, we propose the LMM/LCM Tokenizer. This module utilizes cross-attention to construct representations of decision variables of arbitrary dimensions and concatenates them with the original features, resulting in a richer and more structure-aware problem representation.

Enhanced Optimization strategies Representation: We introduce deformable attention into the original LMM module, proposing the Efficient LMM, to enhance flexibility and information exchange during strategies generation, achieving significant performance improvements on multiple benchmark tasks.

Enhanced Training Stability: To overcome the gradient estimation inaccuracies caused by Gumbel-Softmax, we design a differentiable crossover mechanism, Differentiable LCM. This replaces the original non-differentiable crossover process with a weighted summation, effectively improving the stability and convergence quality of the training process.

In summary, by synergistically enhancing feature modeling, strategies representation, and mapping mechanisms, this paper promotes the development of pre-trained optimization models with stronger zero-shot optimization capabilities and wider adaptability.

W. 2: Experiments are limited ...

We evaluate the performance of EPOM and POM on the planner mechanic arm problem, a engineering problem that has been widely used to evaluate the performance of BBO algorithms. The optimization goal of this problem is to search for a set of lengths $\mathbf{L} = [l_1, l_2, \cdots, l_n]$ and a set of angles $\mathbf{\alpha} = [\alpha_1, \alpha_2, \cdots, \alpha_n]$ to minimize the distance from to the target position based on a given $r$, represents the distance from the target point to the origin of the mechanic arm . In our experiment, we let $n=30$, $r=100$, fix the length $l_i = 10$, and search for the optimal angles only.
We used POM and EPOM to optimize the problem for 100 generations each, and recorded the best fitness (minimum distance) of the last generation. The optimal value achieved by EPOM was 2.63(1.07), and that of POM was 8.09(2.98).

W. 3: No comparison ...

LLM-based optimizers lack zero-shot optimization capabilities, so comparison with them is meaningless. Related work describes this capability as being related to pre-trained models. This may not be our weakness.

Question 1: Can you provide ...

Please see our rebuttal to Weakness 3.

Question 2: Have you tested ...

Please see our rebuttal to Weakness 2.

Q. 3: What is the ...

We recorded the time consumption (in seconds) of EPOM, POM and CMAES when optimizing 24 BBOB functions in 30- and 100-dimensional settings respectively, as shown in the following table.

EPOM can generalize well across dimensions. We tested the performance of POM and EPOM on BBOB in a 300-dimensional setting and recorded their best fitness in the last generation on each function, part of the results are shown in the following table.

Q. 4: Does EPOM ...

In our submission, EPOM was trained on only five functions under the 10-dimensional setting (as shown in Table 1). Compared to the 24 BBOB functions in the 30- and 100-dimensional settings used for testing, the training set can be considered a low-data regime. Despite this, EPOM demonstrates strong performance in the experiments, indicating that it does not require a large-scale pretraining datasets.

Weakness 1: The related work ...

We appreciate the author's comment and will enrich our Related Work section according to the advice.

W. 2 The shapes of ...

We believe that the "some network weights" mentioned by the reviewer refer to the attention network used to identify the individuals to be focused on, as described in Section 3.2.3. In fact, for the following reasons, we set the output dimension $d_{out}$ of such networks to a fixed value independent of the population size (as stated in the submission: "... and $d_{out}$ is the fixed output dimension.", and we set $d_{out}$ to 100 in this submission): (1) To avoid additional computational overhead. When the population size $n > d_{out}$, we identify the individuals to be focused on only from the first $d_{out}$ candidates in the population, thereby avoiding any increase in the computational burden of deformable attention. When $n < d_{out}$, we clip the network output and retain only the first n dimensions. This reason should also help address the reviewer's concerns in Question 6 and Limitation 1. (2) To preserve generalization ability. As the reviewer pointed out, if $d_{out}$ were determined by the population size $n$, EPOM's generalization ability across different population sizes would be compromised. By setting it to a fixed value, we mitigate this issue to some extent.

W. 3: In the Tokenizer, ...

We also took this issue into consideration when designing the LMM/LCM Tokenizer. Therefore, we suggest that both $d_T$ and $d_{1V}$ should not be set too large. In this submission, we set $d_T = d_{1V} = 4$ , resulting in a 16-dimensional decision variable feature. When concatenated with the original 2-dimensional feature, its contribution would not be overly diluted.

Other Issues 1: The reference for ...

We thank the reviewer for pointing out this citation issue. We will correct it in the next revision.

O. I. 2: Section 3.1.1 2) ...

We sincerely thank the reviewer for carefully identifying this issue. We acknowledge that some equations related to variables such as $\mathbf{cv}^t _ i$ were regrettably omitted in the current version. We provide the missing equations below:

$$\mathbf{cr}^t \leftarrow LCM(\mathbf{Z}^t) \\ \mathbf{r}^t _ i = rand(d), \; \; \mathbf{cv} _ i^t = gumbel\_softmax(cat(\mathbf{r}^t _ i, tile(cr _ i^t, d))) \\ \mathbf{u}^t _ i = \mathbf{cv} _ {i, 0}^t \cdot \mathbf{x} _ i^t + \mathbf{cv} _ {i, 1}^t \cdot \mathbf{v} _ i^t$$

We thank the reviewer for pointing out these two writing errors. First, the dimension of $\\mathbf{W}_{2V}$ should be $(\hat{d}_t, d_{out})$ , not $(n, d_{out})$. Then, we indeed missed the transpose operation on $\mathbf{K}_{1M}$ in the attention computation in Equation (3).

We sincerely thank the reviewer once again for pointing out issues (a) and (b). We will make sure to correct them in the next revision.

O. I. 3: In Algorithm 1 ...

To clarify, $\mathbf{V}^t$ is not an input parameter of Differentiable LCM. Instead, it is processed by the LCM Tokenizer, which converts $\mathbf{V}^t$ into $\mathbf{F}^t$. The Differentiable LCM then operates on $\mathbf{F}^t$ and $\mathbf{E}^t$ to generate the offspring population $\mathbf{U}^t$. Thus, while $\mathbf{U}^t$ is indeed an input to the overall LCM pipeline, it is not directly used by Differentiable LCM.

Questions 1: what is ... In this submission, we define $\bar{d} = \lceil \frac{d}{2} \rceil$. The operations of shaping and reshaping a solution are straightforward and are detailed as follows:

1. Padding (if necessary): If the dimensionality of the solution vector $\mathbf{x}$ is odd, we append a scalar 0 to its end to obtain a new vector $\hat{\mathbf{x}}$ of length $d+1$; otherwise, we set $\hat{\mathbf{x}} = \mathbf{x}$
2. Shaping into a matrix: We then shape $\hat{\mathbf{x}}$, which is of length $2 \cdot \bar{d}$, into a $2 \times \bar{d}$ matrix $\hat{\mathbf{K}}_T$.

for example, consider a 5-dimensional solution vector: $\mathbf{x} = [0.3, 1, 4, 9, 7]$
Since $d=5$ is odd, we append 0 to obtain: $\hat{\mathbf{x}} =[0.3, 1, 4, 9, 7, 0]$ Then we apply the $shape$ operation:

\hat{\mathbf{K}}_T = shape(\hat{\mathbf{x}}) = \begin{eqnarray} \begin{bmatrix} 0.3 & 1 & 4 \\ 9 & 7 & 0 \\ \end{bmatrix} \end{eqnarray}

The $reshape$ operation is simply the inverse, i.e., flattening $\hat{\mathbf{K}}_T$ row-wise to recover $\hat{\mathbf{x}}$. Both $shape$ and $reshape$ operations can be efficiently implemented in Python using tensor manipulation methods, such as $view()$ in PyTorch.

Q. 2: In Section 3.2.3 ...

We clarify the meaning of $prob(\mathbf{x}^t_i)$ and the process of top-$p$ sampling as follows: For a given individual $\mathbf{x}^t_j$, we define $prob(\mathbf{x}^t_i)=\hat{\mathbf{N}}_{t, (j, i)}$ as the importance score (i.e., probability) of $\mathbf{x}^t_i$ being selected as a focused-on individual from the perspective of $\mathbf{x}^t_j$. The full matrix $\hat{\mathbf{N}}_t$ stores all such pairwise importance scores between individuals in the population. To obtain the final index matrix $\mathbf{N}_t$, we perform the following 3 steps for each individual:

1. Sorting: For each row of $\hat{\mathbf{N}}_t$, which corresponds to one individual, we sort all other individuals in descending order of importance scores.
2. Top-$p$ sampling: For the $j$-th individual, we progressively select top-ranked individuals until the cumulative score exceeds a predefined threshold $p$. Since the importance distributions differ across individuals, the number of selected individuals $m_j$ will also vary. $m_j$ is calculated by the equation below.

$$\mathbf{\alpha} _ j = sort(\hat{\mathbf{N}} _ {t, (j, :)}) \quad \text{(in descending order)} \\ m_j = \min \{ m \;|\; \sum_{i=1}^{m} \mathbf{\alpha} _ j[i] > p \}$$

3. obtaining $\mathbf{N} _ t$: Let $k = max\{m _ j\}^n _ {j=1}$. We collect the top-$k$ indexes in $\mathbf{\alpha} _ j$ for each individual, and stack them to form the final index matrix, $\mathbf{N} _ t \in \mathbb{R}^{n \times k}$.

We hope this explanation also addresses the reviewer’s concern raised in Question 3.

Q. 3: It seems that ... Please see our rebuttal to Question 2.

Q. 4:

We should clarify that $\mathbf{N} _ t$ is used to calculate $\mathbf{V}^t$, instead of $\mathbf{S}^t_i$. We would like to explain this process in individual-wise. Given the index vector $\mathbf{N} _ {t, (j, :)}$, corresponding to the top-$k$ focused individuals of $\mathbf{x}^t _ j$, as well as the full population $\mathbf{X}^t$, and their features $\mathbf{E}^t$. Firstly, we collect the focused-on features $\mathbf{E}^t _ j \in \mathbb{R}^{k \times \hat{d}_t}$ according to indexes in $\mathbf{N} _ {t, (j, :)}$. Then, in a similar way with LMM in POM, we obtain the query vector $\mathbf{q} _ j$, key matrix $\mathbf{K} _ j$ by $\mathbf{e} _ j$ and $\mathbf{E}^t _ j$, respectively, and calculate $\mathbf{S}^t _ i \in \mathbb{R}^{1 \times k}$. Finally, we collect $\mathbf{X}^t _ j \in \mathbb{R}^{k \times d}$ in the same way as $\mathbf{E}^t _ j$, and calculate $\mathbf{v}^t _ j =\mathbf{S}^t _ i \times \mathbf{X}^t _ j$.

Q. 5

As showed in lines 261–265 of our submission, EPOM does demonstrate a certain degree of generalization capability across numerical search spaces. However, its ability to generalize across problems involving different data types remains an open question. We believe that the LMM/LCM Tokenizer has the potential to support such generalization, and this is indeed a promising direction for future research. We thank the reviewer for highlighting this important point and would be glad to further discuss and explore it in future work.

Q. 6

Please see our rebuttal to Weakness 2.

Q. 7

We recorded the time consumption (in seconds) of EPOM, POM and CMAES when optimizing 24 BBOB functions in 30- and 100-dimensional settings respectively, as shown in the following table.

Limitation 1

Please see our rebuttal to Weakness 2.

Lim. 2

Please see our reply to Question 7.

Weakness 1 Clarity and Terminology ...

We rewrote the Abstract and Introduction to highlight the motivation, novelty, and coherence of the paper.

Abstract Zero-shot optimization aims to achieve both generalization and performance gains on solving previously unseen black-box optimization problems over SOTA methods without task-specific tuning. Pre-trained optimization models (POMs) address this challenge by learning a general mapping from task features to optimization strategies, enabling direct deployment on new tasks.

In this paper, we identify three essential components that determine the effectiveness of POMs: (1) task feature modeling, which captures structural properties of optimization problems; (2) optimization strategy representation, which defines how new candidate solutions are generated; and (3) the feature-to-strategy mapping mechanism learned during pre-training. However, existing POMs often suffer from weak feature representations, rigid strategy modeling, and unstable training.

To address these limitations, we propose EPOM, an enhanced framework for pre-trained optimization. EPOM enriches task representations using a cross-attentive tokenizer, improves strategy diversity through deformable attention, and stabilizes training by replacing non-differentiable operations with a differentiable crossover mechanism. Together, these enhancements yield better generalization, faster convergence, and more reliable performance in zero-shot black-box optimization.

Introduction

... Traditionally, solving BBO tasks requires expert-designed or heavily tuned optimization algorithms for each new problem, making the process inefficient and non-scalable. Thus, the paradigm of solving new optimization problems without any task-specific tuning is extremely valuable.

Meta-learning, or learning to learn [1], particularly in the context of meta-black-box optimization [2] or learning to optimize [3], covers a broad range of scenarios. These approaches often leverage information from the target task to enhance the optimizer’s performance on that specific task. However, they typically exhibit poor generalization to new, unseen tasks. Even though some methods—such as LES \cite{lange2023discovering1} and LGA \cite{lange2023discovering}—can generalize to novel settings, their performance still falls significantly short compared to state-of-the-art optimizers specifically designed for those settings.

To better highlight our contribution, we introduce the concept of zero-shot optimization, which aims to simultaneously achieve strong generalization and high performance on new optimization problems—without requiring task-specific training. A particularly promising direction in this area is the development of pre-trained optimization models (POMs). These models are designed to learn general-purpose optimization behaviors across a wide variety of training tasks and effectively transfer that knowledge to previously unseen problems.

We view a POM as a function that maps task-specific features to optimization strategies. This process involves three key components:
(1) Task feature modeling, which captures characteristics of the problem (e.g., normalized fitness values and centralized rankings);
(2) Optimization strategy representation, which determines how candidate solutions are generated and refined;
(3) Mapping mechanism, which learns the transformation from features to strategies during pre-training.

Although recent POMs have made progress, they remain limited in three areas:

• The expressiveness of task features is insufficient to generalize across diverse problem structures;
• The strategy generation process lacks flexibility and diversity;
• The training process suffers from gradient instability due to non-differentiable operations.

To overcome these limitations, we propose EPOM (Enhanced Pretrained Optimization Model), which systematically enhances all three components:

• Feature modeling is improved by a cross-attentive tokenizer that captures decision-variable-level information and encodes it into fixed-length task representations;
• Strategy representation is enhanced through deformable attention, enabling dynamic and diversity-aware interactions among population members;
• Training stability is ensured by replacing sampling-based crossover operations with a differentiable, weighted-sum mechanism, leading to smoother gradient flow.

By jointly improving feature modeling, strategy generation, and training robustness, EPOM achieves superior generalization and performance in zero-shot black-box optimization. This work contributes to the growing body of meta-learning literature on learning-to-optimize methods, and offers a scalable pathway toward universal optimization agents.

W. 2: Misleading Acronyms ...

Thank you for your suggestion. We have revised the two terms to: Learned Mutation Module (LMuM) and Learned Crossover Module (LCrM) to distinguish them from the existing concepts.

W. 3: Conceptual Positioning ...

We thank the reviewer for raising this important point. While our work is indeed related to meta-learning, we use the term zero-shot optimization to emphasize a more specific setting: directly generating optimization strategies for unseen tasks without task-specific adaptation, based solely on task features.

Meta-learning, particularly meta-black-box optimization [1], encompasses broader scenarios such as algorithm selection, configuration, and solution manipulation. These settings do not align exactly with our goal of optimization strategy generation with strong generalization and superior performance. Similarly, terms like learning to optimize often imply inner-loop adaptation, which our method avoids.

Thus, introducing the term zero-shot optimization helps clarify our contribution: achieving both generalization and performance gains over SOTA methods without per-task training. We explicitly define this term in the introduction (Lines 25–27) and compare with meta-learning baselines in both related work and experiments, where our method demonstrates superior effectiveness.

Therefore, the use of this terminology is intentional and necessary, rather than a conceptual flaw.

[1] Hospedales, T., Antoniou, A., Micaelli, P., & Storkey, A. (2021). Meta-learning in neural networks: A survey. IEEE transactions on pattern analysis and machine intelligence, 44(9), 5149-5169.

[2] Z. Ma, H. Guo, Y. -J. Gong, J. Zhang and K. C. Tan, "Toward Automated Algorithm Design: A Survey and Practical Guide to Meta-Black-Box-Optimization," in IEEE Transactions on Evolutionary Computation, doi: 10.1109/TEVC.2025.3568053.

[3] Chen, T., Chen, X., Chen, W., Heaton, H., Liu, J., Wang, Z., & Yin, W. (2022). Learning to optimize: A primer and a benchmark. Journal of Machine Learning Research, 23(189), 1-59.

Question 1: Rewrite for ...

Based on your suggestions, we will simplify complex sentences.

Q. 2: Explicitly Position ...

In the introduction, we discussed the differences and connections between pre-trained optimization base models and meta-learning. For details, see the second and third paragraphs of the Introduction section.

Q. 3: Clarify Architectural Contributions ...

We thank the reviewer for their suggestions. We will enrich the ablation study in the revision and conduct in-depth analysis of each module.

Weakness 1: Figure 4 is a bit confusing ...

We hope the following explanation will help the reviewer better understand Figure 4. First, Figure 4 is a critical difference diagram, which visualizes the performance differences among the given algorithms on specified datasets based on their average rankings. The results of our ablation study are summarized in this figure.
Specifically, we remove each proposed module one by one and evaluate the corresponding variant of EPOM on 24 BBOB functions in 30 dimensions. For each function, we compute the ranking of all variants and then calculate the average rank, which is shown as the number above each polyline in Figure 4.
We suspect that what the reviewer referred to as “overlapping lines” mainly concerns the lines representing the “No Differentiable LCM” and “No LMM Tokenizer” settings. This overlap is due to their similar average rankings. To address this issue, we will increase the figure’s size and resolution in the next revision, which should alleviate the visual ambiguity.

Additionally, following the reviewer’s suggestion, we have made the following table based on the average rankings in Figure 4, which we hope will further clarify the results and assist in interpreting the figure.

W. 2: The clarity seems fair ...

(a) We sincerely apologize for our oversight and the confusion it may have caused the reviewer. The discrepancies between Table 2 and Figures 6–9 are mainly caused by their different calculation methods. For each entry $m(n)$ in Table 2, $m$ and $n$ represent the mean and standard deviation of the fitness of the full population, for a given function, method, and dimension setting. Fig. 6-9 represent the optimization trajectory curves consisting of the best fitness in each generation. Once again, we sincerely apologize for not describing these two experimental setups clearly in our submission, and for any confusion this may have caused.

(b) Following (a), our conclusion are drawn based on the best fitness of the last generation of each method, because the core goal of black-box optimization algorithms is to find the optimal solution as much as possible, rather than to make all solutions close to the optimal. The best fitness of the last generation can better reflect both this goal and the best performance of a given algorithm.

(c) We tested EPOM with constant crossover weight(0.5) and varying crossover weight on the BBOB in a 30-dimensional setting. On some functions, their performance is not much different, but on some functions, their performance is quite different, as shown in the following table (the best fitness of the last generation is recorded).

W. 3: however it adds much more complexity ...

We recorded the time consumption (in seconds) of EPOM, POM and CMAES when optimizing 24 BBOB functions in 30- and 100-dimensional settings respectively, as shown in the following table.

In summary, POM, thanks to its streamlined model structure, achieved the lowest time consumption in both settings. CMAES, however, lacks GPU acceleration, and while it consumes less time than EPOM in the low-dimensional setting, it consumes approximately 10.89 seconds more than EPOM in the 100-dimensional setting. For EPOM, the increased time and computational effort required for performance improvement, driven by its increased model complexity, is almost an inevitable price to pay.

Question 1: Could you clarify ...

Please see our rebuttal to Weakness 2 (a) and (b).

Q. 2: How much deviation ...

Please see our rebuttal to Weakness 2 (c).

Q. 3: How does the runtime ...

Please see our rebuttal to Weakness 3.

Paper Formatting Concern 1: There might be ...

We sincerely thank the reviewer for carefully identifying this issue. We acknowledge that some equations related to variables such as $\mathbf{cv}^t_i$ were regrettably omitted in the current version. We provide the missing equations below:

$$\mathbf{cr}^t \leftarrow LCM(\mathbf{Z}^t) \\ \mathbf{r}^t _ i = rand(d), \; \; \mathbf{cv} _ i^t = gumbel\_softmax(cat(\mathbf{r}^t _ i, tile(cr _ i^t, d))) \\ \mathbf{u}^t _ i = \mathbf{cv} _ {i, 0}^t \cdot \mathbf{x} _ i^t + \mathbf{cv} _ {i, 1}^t \cdot \mathbf{v} _ i^t$$

We will make sure to correct them in the next revision.

P. F. C. 2: I think figure 4 ...

Please see our rebuttal to Weakness 1.

Weakness 1: Unclear Problem Definition...

This article mainly addresses the zero-shot optimization problem for black-box optimization, focusing on its continuous optimization scenario. Based on the reviewer's suggestions, we have revised the second to fourth paragraphs of the Introduction to:

Traditionally, solving BBO problems requires carefully designed or heavily tuned optimization algorithms for each individual task. This manual design process is costly, requires expert knowledge, and lacks scalability across diverse domains. To overcome these challenges, the paradigm of zero-shot optimization has emerged.

Zero-shot optimization refers to the ability to solve a previously unseen optimization task directly—without any per-task tuning or adaptation. That is, given an unknown objective function $f$, the optimizer is expected to perform well without trial-and-error adjustments. This generalization ability is crucial for real-world applications where human-in-the-loop tuning is impractical.

A promising approach to zero-shot optimization is the use of Pre-trained Optimization Models (POMs) \cite{li2024pretrained}, which learn general optimization strategies across a diverse training set of functions. POMs aim to transfer this knowledge to solve new BBO tasks efficiently and automatically.

W. 2: The motivation is unclear...

Pre-trained optimization models (POMs) are a universal optimization framework designed to achieve zero-shot optimization. Their core idea is to learn the mapping between task features and optimization strategies, enabling the model to directly output high-quality optimization strategies without fine-tuning for new tasks.

We summary this process into three key elements: task feature modeling, optimization strategies expression, and a mapping mechanism from task features to optimization strategies. First, the task feature set, typically consisting of normalized fitness values and centralized rankings of individuals, characterizes the structural characteristics of the optimization task. Its expressive power directly determines the POM's ability to perceive and adapt to different tasks. Second, the optimization strategies, as the output of the POM, must possess sufficient expressive power to cover search behaviors across diverse tasks. The mapping from features to strategies, learned during the pre-training phase, is key to achieving zero-shot performance.

Although existing POM methods have achieved some success, significant bottlenecks remain in feature modeling, strategies expression, and training stability. To this end, this paper proposes systematic improvements around the three core elements mentioned above, building a POM system with enhanced generalization and training robustness:

Enhanced Task Feature Modeling: To improve the expressiveness of task features, we propose the LMM/LCM Tokenizer. This module utilizes cross-attention to construct representations of decision variables of arbitrary dimensions and concatenates them with the original features, resulting in a richer and more structure-aware problem representation.

Enhanced Optimization strategies Representation: We introduce deformable attention into the original LMM module, proposing the Efficient LMM, to enhance flexibility and information exchange during strategies generation, achieving significant performance improvements on multiple benchmark tasks.

Enhanced Training Stability: To overcome the gradient estimation inaccuracies caused by Gumbel-Softmax, we design a differentiable crossover mechanism, Differentiable LCM. This replaces the original non-differentiable crossover process with a weighted summation, effectively improving the stability and convergence quality of the training process.

In summary, by synergistically enhancing feature modeling, strategies representation, and mapping mechanisms, this paper promotes the development of pre-trained optimization models with stronger zero-shot optimization capabilities and wider adaptability.

W. 3: Limited Novelty...

Please see our rebuttal to Weakness 2.

W. 4: The experiments are ...

We hope that the following explanation of Table 2 can increase its readability.

• Header $d$ means the problem dimension (30, 100), F means test problems (F1, F2, ..., F24), and EPOM, POM, ..., LGA are the evaluated methods.
• Bottom $+/=/-: x/y/z$ shows that among the $24 \times 2$ tasks (24 BBOB functions in 2 dimension settings), EPOM is better than the comparison algorithm on $x$ tasks, comparable to the comparison algorithm on $y$ problems, and weaker than the comparison algorithm on $z$ problems.
• Data For the data $a(b)$ in the table, $a$ represents the mean fitness of the last generation of population in the three average tests, and $b$ represents its standard deviation. The bold data represents the best among all methods, and the underlined data represents the second best or equally best.

If there are any other issues regarding our Experiments section, we would be happy to discuss them.