Amortized Active Generation of Pareto Sets

Thank you to reviewer yJhX for the challenging questions. We will attempt to answer them below.

Q1: I don't think the training of alignment vectors is well justified, indeed intuitively if $x_1$ and $x_2$ are close in space, the alignment should reflect stronger than if they were apart ... the fact that pointing a preference vector ever so slightly outside the Pareto front results in poor suggestions from the algorithm is concerning, and is probably related to this fact.

Thank you for this feedback. One main motivation for using the permutation method for training the alignment CPE was its simplicity --- A-GPS is already complex, and we do not wish to add more hyperparameters if they are not essential. To this end, we have found that increasing the amount of permuted pairs, $(u_{\rho(n)}, x_n)$, to the aligned pairs $(u_n, x_n)$ by about 5:1 drastically improves the performance of the alignment component of A-GPS, and appears to improve the issue seen with the BraninCurrin test function. This procedure is similar to noise contrastive estimation (particularly InfoNCE, that uses negative example pairs), and MaxEnt estimation (where only positive labels exist, and the negative pairs are background points). This procedure rewards an estimator for discriminating relevant examples (aligned pairs) from irrelevant unaligned background/noisy samples.

Furthermore, we were reluctant to make additional smoothness assumptions about the $\mathcal{X} \times \mathcal{U}$ space, since in many applications these may not be valid. For example, one amino acid change in the active site of a protein can drastically impact the protein's properties (e.g. if a hydrophilic amino acid is swapped for a hydrophobic one both structural and functional properties are effected), changing the location of the protein in $\mathcal{Y}$ and hence also $\mathcal{U}$. We can include this point in the paper, and flag this as a direction for future investigation/improvement.

Q2: It appears that some of the methods have been implemented poorly, e.g. why is CbAS-TFM always outperformed by Random search

We use the same implementation as [Steinberg, 2025] used in comparison to VSD --- though the authors do note that it is originally designed as an offline method, and it has been adapted to the online setting. However, we did notice the performance of the method improve when we changed the labelling strategy to Pareto rank thresholding, as discussed with reviewer VCzN. This is also more inline with how CbAS is used in both the VSD paper, and in the original implementation --- increasing thresholds. We also found an issue with the LaMBO2 baseline (as implemented in poli-baselines), where the code was not using the correct acquisition function for MOO settings. We have fixed this, and have seen a marked improvement in the performance of the method. However, We still find A-GPS and VSD generally outperform these baselines.

Q3: How to set the preference vectors remains a little unclear. I think a clearer example of how a practitioner would go about choosing the vectors and how this would affect the Pareto front would improve the paper

Thank you for the feedback. We will happily clarify this in the paper. Assuming we start with a trained model, $q(x|u)$, perhaps the easiest way for a user to choose a preference vector, $u_\star$, for generation of designs is to start with a desired outcome, $y_\star$. We can then use Equation 9 to normalise this to a preference direction vector, $u_\star$, to input to $x_\star \sim q(x|u=u_\star)$. To control Pareto front exploration, we could parameterise a $q(u)$ directly, rather than learning similarly to that in [25]. We have opted to learn a $q(u)$ in our presentation as we are aiming for maximising the discovered hyper-volume in the experiments.

Q4: I find it confusing about what the paper means by "amortised", since indeed there is re-training of the generative model at each iteration of the optimisation loop which results in a large expensive procedure between iterations -- the only amortisation that I understood related to changing preference vectors however I think this makes the paper premise and title misleading.

Amortization in the context of variational inference typically refers to using a parameterized global inference function for the variational distribution, $q$, in contrast to the factored mean-field (independent) approach for each latent variable/parameter. This term came into being to describe variational models like Kingma and Welling's VAE, which can learn conditional distributions, $q(z|x)$, that can be queried with new $x$ without retraining. We can view our variational distribution, $q(x|u)$ in the same way --- we can query $q$ for various $u$ without the need for re-training, as factored mean field (and other MOBO) approaches would require. We can clarify this usage in the text, and perhaps we can re-word line 205 "amortization->retraining" if this was the source of confusion.

Q5: In Figure 3, we see the lines change for Random between the first plot and the rest, making it difficult to read

We found keeping the order of the colouring consistent with algorithm performance was clearer for these sorts of plots, but we can re-style if this was confusing, and keep the style aligned with the methods.

Q6: In Eq. (17) should the $D_{KL}$ lie outside the inner expectation?

The $D_{KL}$ term has to remain inside the expectation to be consistent with our objective in Equation 10. We can see this in the expansion in Equation 13, and then noting $q(u) \approx p(u|z)$.

Q7: Is there a way a user can set a preference over the spread of the generation in the model?

Probably the easiest way to do this post-learning would be to query $q(x|u)$ with multiple $u$, or draw them from a specific distribution --- e.g. bounded uniform over a subset of $\mathcal{U}$. Our framework also supports specifying a $q(u)$ rather than learning one during optimization if that is sensible for a particular application.

Q8: Why do you think that your method which does not calculate hyper-volume improvement, beats methods that do go through the explicit calculations and try to maximise it?

This is a very good question, and we believe it is because in quite general circumstances the Pareto non-dominance label is equivalent to a hyper-volume improvement indicator if a box-decomposition method is used. We sketch this now; if we define hyper-volume improvement,

$\text{HVI}(x) = \text{HV}(\mathcal{F} \cup \\{y(x)\\}) - \text{HV}(\mathcal{F})$,

where $\text{HV}$ is hyper-volume, and $\mathcal{F}$ is the Pareto front. We define hyper-volume as a union of the hyper-volume of hyper-rectangles defined by the vertices,

$\text{HV}(\mathcal{F}) = \lambda_L (\bigcup_{y \in \mathcal{F}}[r, y])$,

for a reference point $r \in \mathbb{R}^L$ and a measure, $\lambda_L$. Then the hyper-volume improvement indicator is, $1[\text{HVI}(x) > 0]$. We make two observations,

1. If $y(x)$ is dominated, then $z = 1[x \in \mathcal{S}] = 0$, but also the hypervolume is unchanged since $\lambda_L([r, y(x)]) \subseteq \text{HV}(\mathcal{F})$, and so $1[\text{HVI}(x) > 0] = 0$.
2. If $y(x)$ is not dominated, then $z = 1[x \in \mathcal{S}] = 1$ and also $\lambda_L([r, y(x)]) \not\subseteq \text{HV}(\mathcal{F})$ which means it must contribute positive volume, and so $1[\text{HVI}(x) > 0] = 1$.

This requires the reference point to be dominated by every feasible point, and also that there are no ties. The latter is not true for objectives that are discrete, and so in these circumstances the dominance indicator is not the same as HVI.

Therefore, in the continuous case, with a proper loss (e.g. logistic), training a class probability estimator on the dominance indicator should also estimate the probability of hyper-volume acquisition. This nicely parallels the probability of improvement setting in VSD, and we can extend our paper to present this.

Thank you for your concerns, and recognition that we can make this paper "much stronger".

We actually think that the requested changes, from all reviewers, are not major. Most requests are about providing additional experiments or ablations, which we have done, or textual clarifications of our method and its limitations, which exist in these discussions and are easily included in the main text and/or extended discussions in the appendix. The work is, for all intents and purposes, done, and visible here publicly (once this session concludes).

We have not had any requests for major changes to our methodology or approach (and some of the minor requests were actually anticipated by us and already in the code), our novelty has not be challenged by any reviewer, and our core conclusions have remained the same -- or have been actually strengthened, e.g. we can now simply prove that a Pareto CPE is a cheap approximation of PHVI.

We also wish to thank you again and sincerely for your commitment and continued engagement, especially as serving as a reviewer through this discussion phase is a lot of work (particularly if you are defending your own work(s) simultaneously).

We thank reviewer VCzN for the thoughtful feedback, and will attempt to address their concerns and questions now.

Q1: Key concern on $q(u|z)$ and random sampling step in Algorithm 1 (lines 11–12) relies on previously observed Pareto-optimal points. In settings where user preferences are not specified and the initial data has poor Pareto Frontier coverage, this empirical distribution will not support sufficient exploration of the Pareto front. This should be either addressed explicitly or the scope of the method should be reframed as targeting preference-aware MOO, rather than general MOO.

Thank you for this observation. Just as point of clarification, we use a constrained mixture of Gaussians over preferences, rather than an empirical distribution, for all experiments. We also study the consequence of an empirical distribution in Appendix E.2. Even so, after the submission of this version of the paper we were also concerned that a small Pareto set could lead to overly exploitative behaviour. We have subsequently made two small adjustments to A-GPS which we believe rectify this issue.

1. For the first round, we fit an unconditional $q(u)$ using all available $y$.
2. Rather than just using Pareto dominance labels, $z = 1[x \in \mathcal{S}]$, we can use the Pareto ranking method (non-dominance sorting) described in [Deb, 2002], and then specify labels based on a thresholded rank ($s$), $z = 1[x \in \\{\bigcup \mathcal{S}_s : \forall s \leq \tau_t \\}]$. Here $s=1$ indicates the Pareto set, $s=2$ the next non-dominated set (shell) once $\mathcal{S}_1$ is removed, etc. We are then able to choose a threshold, $\tau_t$, per round $t$ using the same mechanisms as the original VSD paper --- and we find the annealed quantile method, Equation 20 of [Steinberg, 2025] (on negative ranks, $-s$), directly applicable. We typically set $\tau_0$ to the 75th-percentile of (negative) ranks, and $\tau_T$ so only the Pareto set is positively labelled.

We do not find much difference in the experimental results with these modifications, with the exception of the ZDT3 synthetic test function, in which we see an improvement for A-GPS. We report updated round-10 results to reviewer uzmJ.

Q2: Time and computational cost profiling is missing. A breakdown of time per component (CPE training, generative sampling, ELBO optimization) would help assess scalability.

We have reported overall run-times for the methods in the supplementary material, Appendix E.1. We will expand upon these though for all seqeunce experiments. For example, here is the run-time summary (in minutes) of the additional methods for the Ehrlich vs. Naturalness experiments, for M = 64, as well as VSD and A-GPS (which are in the Appendix)

The runtime of the CPEs depends on many factors, such as their architecture, training epochs etc, but typically we find them to be less than 10% of the total runtime --- which is dominated by the A-ELBO computation. These timings above, and those in Appendix E.1 should hopefully provide insight as to the contribution of the generative model sampling in the runtime. We can include this discussion in the paper. The main determinant of scalability is the expense of sampling the generative model. For a transformer this is quadratic in sequence length.

Q3: While the paper claims to avoid costly hypervolume (HV) computation, in bi-objective settings HV is computationally cheap. An empirical demonstration in higher-dimensional MOO (e.g., 4–5 objectives) would strengthen the claim of improved efficiency over HV-based methods.

This is an excellent point, and we will include additional results summarised below for round 10 on two higher dimensional problems,

Evaluating these problems with EHVI-based is at or beyond their limits (it is claimed qNEHVI can scale to $L=5$, but we could not run the method in time for this rebuttal). We hope this helps to highlight the efficacy of our method.

Q4: On the benefit of a generative model: In low-dimensional continuous input spaces (e.g., Sec. 6.1), what is the practical advantage of using a generative model over directly optimizing $p(z=1 | x, u)p(u|z=1)$ via gradient-based or heuristic methods? A controlled comparison against direct optimization would clarify the end-to-end utility of the generative model.

This is an interesting question, however, note that

$\text{argmax}_x p(z=1 | x, u)p(u|z=1)$

Will optimize $x$ independently of $p(u|z=1)$, potentially ignoring any preferences. Perhaps a better alternative would be,

$\text{argmax}\_x \mathbb{E}_{p(u|z=1)}[p(z=1 | x, u)p(a=1 | x, u)]$,

where the expectation could be approximated with samples (corresponding to batch size). Or for a particular preference $u_\star$,

$\text{argmax}\_x p(z=1 | x, u_\star) p(a=1 | x, u_\star)$.

This will not yield an amortized model for preferences, and each new preference, $u_\star$, will require a separate optimization. In low dimensional spaces, this may not be costly, in which case this is probably less direct and effective than scalarization methods like MBORE [10] or qParEGO [8]. We expect the benefit of A-GPS in the, low dimensional, continuous case comes from its stochastic nature, perhaps analogously to stochastic vs. deterministic optimization algorithms providing some robustness to local optima. If the reviewer still feels such a study is of benefit, this (modified) direct optimization is simple to implement as an ablation.

Our focus is really on high-dimensional and discrete optimization problems, where direct optimization is impossible without e.g. latent space optimization or reparameterisation via a generative model and the utility of our method is clear --- we will clarify this in the text.

Q5: The scalability of the method on the number of objectives remains unclear. In particular, amortizing over preference vectors becomes increasingly difficult as the number of objectives grows, a clear elaboration of its scalability boundary, or a dicussion of future work or a limitation claim should be done on this.

As can be seen be the DTLZ7 ($L=6$) and DTLZ2 ($L=5$) experiments the method scales more effectively than some of its EHVI-based competitors --- the main cost being the non-dominance checking, which is worst case $\mathcal{O}(LN^2)$, or more typically $\mathcal{O}(LN \log N)$. We also visualised a (PCA) projection of the preference vectors and sampled points, like that in Figure 2, and found nice separated clustering of samples for each input vector. Sadly we are unable to share the figures, but these will be in the updated version of the manuscript, along with the hyper-volume improvement results.

---

[Deb, 2002] K. Deb, A. Pratap, S. Agarwal and T. Meyarivan, "A fast and elitist multiobjective genetic algorithm: NSGA-II," in IEEE Transactions on Evolutionary Computation, vol. 6, no. 2, pp. 182-197, April 2002, doi: 10.1109/4235.996017.

We thank reviewer uzmJ for the thoughtful and considered review, and we are encouraged they enjoyed reading the paper! We will attempt to address the reviewer's concerns succinctly.

Q1: Please can you provide a more detailed explanation for Fig 1(a).

Thank you for this feedback. We will consider moving Fig 1(b) to the appendix, or shortening its description since it is a visual representation of Alg. 1.

We will change the description of Fig 1(a) to something like the following: (a) A visualisation of a 2-dimensional Pareto front, $\mathcal{F}\_\text{Pareto}$, and the random variables used with A-GPS. $y$ are the noisy realisations of the objectives, $f$, when $z=1$ these observations lie on the Pareto front. Preference direction vectors, $u$, are simply unit vectors pointing to a region of the Pareto front from a reference point, $r$ --- Equation 9. We derive aligned ($a=1$) training preference direction vectors, $u_n$ from observation pairs $(y_n, x_n)$, and mis-aligned preference direction vectors ($a=0$) from permuting these pairs, $(y_{\rho(n)}, x_n)$ --- Equation 16. The aim is to learn the distribution of the Pareto set $q_\phi(x|u) \approx p(x|u, z=1, a=1)$, from which a user can input a preference encoded as $u$, for generating optimal samples. We show the image of this distribution on the Pareto front.

Q2: Please can you provide detailed info about the baseline methods you ran in BOTORCH? Please can you comment on why you used a sigmoid transform for the domains of the synthetic test problems? Did you then use Botorch in the unbounded space? This feels like it will impair the standard MOBO approaches ...

Many of these details are in the Appendix D included in the supplementary material along with the original submission, including, e.g. GP surrogate model, reference points for the problems, and details on the sigmoid transform (which is treated as part of the black box). We will expand on these for the final version.

The reference points used are supplied by the BoTorch library implementations of the black boxes where possible --- though in one case we found the supplied value erroneous. We have actually found that using the BoTorch botorch.utils.multi_objective.hypervolume.infer_reference_point function is a more reliable way of setting the reference point automatically, and it is used by other methods, e.g. LaMBO-2. So all of our experiments, methods and baselines have been adapted to use this method now.

All of the original synthetic test functions are only defined on $[0, 1]^D$, and so composing these black-boxes with a sigmoid function is an easy way of making them unbounded, freeing us to use more expressive generative models. The GP-based methods also optimize within this unbounded space --- though we do give them generous bounds of $[-15, 15]^D$, since the BoTorch interface requires bounds to be specified. We believe this is a fair comparison.

We will happily expand upon these details where the reviewer sees fit. We will add these details for the GP methods:

Are there any other settings in particular we should add?

Q3: I would have liked to see an ablation that justifies the use of the preference alignment component described in lines 148–153

We would like to clarify to the reviewer that the preference alignment indicators are necesary as one of the contributions of our approach is to support user preferences at test time. In order to train such a model, we need a way to tell whether a sample $x_i$ is indeed aligned with preference vector $u_i$. For this purpose, we use an alignment indicator $a_i$. Both preference direction vectors and alignment indicators are constructed internally by our framework (as described in the paper) during training so that our conditional generative model $q(x | u)$ learns to generate Pareto-optimal samples that are aligned with preferences. At test time, a user can specify a new preference vector $u_\star$ and our model will generate samples $\{x_\star\}$ that are aligned as much as possible with such preferences. We can add this clarification to the text.

Q4: Additional ablations (e.g., on the number of samples used to approximate Equation 18) would also strengthen the paper.

We will happily include these. Here are round ten, $t=10$, results for different sample sizes, $S$, on the Ehrlich vs. Naturalness experiment,

Interestingly, for the LSTM and Transformer we do not find significant variability -- we will expand this to other experiments.

Q5: I also found it concerning that Figure 2 is based on only five repetitions --- this raises questions about the statistical robustness of the results.

We will increase this to 10 replicates for the final version. We have implemented this already, and found little difference in the conclusions drawn, apart from improving qParEGO on DTLZ2 --- here are round 10 results presented as relative hyper-volume improvement:

Q6: Since the method lacks strong theoretical justification, a rigorous empirical evaluation is critical for a venue like NeurIPS ... That said, Sections 6.2 and 6.3 were compelling, and I appreciated the inclusion of more challenging, real-world examples to demonstrate the method’s applicability.

Subsequent to the paper's submission we have realised that under certain assumptions the CPE trained on non-dominance indicators is estimating probability of hyper-volume improvement (PHVI). We have sketched the proof for reviewer yJhX. Hopefully including this will improve the method's theoretical justification.

Futhermore, we have also implemented the Solvabity vs. SASA experiment from [31] which uses the Foldx simulator as a black box. This experiment is run for 64 rounds, with batches of 16, and only permits one mutation per sequence per round. This required a masked Transformer generative model (mTFM), $q$, that learns to mutate an input sequence, in a similar fashion to the generative back-bones of LaMBO and LaMBO-2. A-GPS and VSD perform similarly to LaMBO-2 on this experiment (and outperform a greedy random baseline and CbAS), and we feel it adds yet another compelling example of this method. Round 64 results summarised below,

We have managed to re-run some of the synthetic function experiments with the GPs in $[0, 1]^d$. Unfortunately, this meant re-defining the initial data, $\mathbf{x}_0$, as being sampled from Latin hypercube sampling in $[0, 1]^d$, and not in the sigmoid space, as it was originally (with bounds of $[-1, 1]^d$). In our haste, we forgot to re-fit AGP's prior to this new data, and so it has been run with suboptimal settings (prior being too concentrated). You can see this when comparing these results to those reported previously (though BraninCurrin uses a different scale, and so is not comparable). We don't have time to re-run these experiments, so we will just report what we have as a gesture of good faith:

Unfortunately DTLZ2 with $L=5$ did not finish in time.

We sincerely thank reviewer wcJY for the constructive feedback, and we will endeavour to respond to all of their concerns.

Q1: The experiments shown are relatively limited and would be more convincing with more diversity and quantity of test functions. For example, the DTLZ and ZDT test suites contain further problems with varying pareto frontiers

Reviewer VCzN also requested additional synthetic function experiment, but with higher dimensional objectives. As such we have tested two more, higher dimensional, synthetic functions, along with another low dimensional GMM test function from BoTorch:

Unfortunately the EHVI-based methods were not able to be used in the higher-dimensional settings (qNEVHI is claimed to scale to $L=5$, but we found difficulty running it in time for this rebuttal), which hopefully reinforces the efficacy of our method.

Q2: Additionally, the paper would be improved with clearer discussion of its limitations.

We apologise for not elaborating on the methods limitations. We did slightly expand on the limitations in the supplementary material (Appendix A), but we will further expand the discussion to include:

• A-GPS does not perform well on the GMM test function, we will include this result and attempt to explain the reason (still under investigation).
• Miss-specification of prior, including overfitting, or poor generative model choice. This is something we have noticed in high dimensional settings, especially with small initial training datasets.
• Extreme class imbalance when only a few Pareto-optimal solutions exist can lead to overly exploitative behaviour. We have subsequently addressed this --- see the discussion with reviewer VCzN.
• Incoporating constraints on the design space is less straight-forward with generative models compared to direct design optimization.
• A possible future work direction is to enhance the precision of the conditional generation as noted by reviewer yJhX.

Q3: Each evaluation appears to have detailed setups for both the baselines and proposed method, which differ across test functions. For example, the model sometimes uses an LSTM and sometimes uses a transformer. However, the performance varies widely across test functions.

We should clarify --- in both the Ehrlich vs. Naturalness and RaSP vs. Naturaless experiments we always use both Transformer and LSTM generative models (we do not select between them depending on the experiment).

We do notice in the original experiments that the performance can differ greatly between these generative models. After some digging, we realise that our prior fitting method was too naive, and could cause the Transformers to overfit, especially with such little training data. Subsequently, we have changed the procedure to include 20% dropout (deactivated for the A-ELBO phase), and use 10% held out data to assess convergence. This has lead to much more consistent behaviour, where the transformers and LSTM models perform similarly for shorter sequences ($M \leq 32$), and Transformers generally pull ahead of LSTMs for longer sequences ($M>64$). You can see evidence of this improved consistency in the sample ablation results requested by reviewer uzmJ. Our recommendation to practitioners would be to use Transformers for sequence design tasks so long as they are well validated.

Q4: How would you use A-GPS to approach a new problem apriori, including deciding which model to use?

This is something we will be happy to include. We have used the same configuration for all synthetic test functions, and it seems to generalise decently. For high dimensional sequences, we do have a few recommendations, e.g.:

• Generally transformers are better generative models (and more numerically stable) than LSTMs, when properly initialised.
• We find the sequence length specific configurations given by the VSD authors work well for these problems too.
• Our CNN CPE architecture seems to perform well with minimal reconfiguration for all sequence problems we have attempted. For longer sequences we suggest larger kernels for the CNN. Ensembles generally to not appear to make much of a performance difference.
• Make sure the individual components are well validated on the training data where possible --- if anything, slight over-fitting the CPEs leads to preferable outcomes compared to under-fitting, as the prior provides additional regularisation.
• Using round-varying Pareto-rank based labelling (see discussion with reviewer VCzN) makes the method more robust to cases where there are only a few Pareto-optimal solutions. We typically start with a threshold of the 75th-percentile of ranks, and by t=T, end with the Pareto set.

We will add this to the paper, or at least to the appendix and reference it in the main text.

Esteemed reviewer wcJY,

We referred to additional results to other reviewers in our response to you. To aid your assessment of our work, we have now copied them here for your convenience.

Q3: Each evaluation appears to have detailed setups for both the baselines and proposed method, which differ across test functions. For example, the model sometimes uses an LSTM and sometimes uses a transformer. However, the performance varies widely across test functions

You can see evidence of this improved consistency in the sample ablation results requested by reviewer uzmJ

We replicate these below (round t=10 relative hyper-volume improvement mean and std. dev.):

Q1: The experiments shown are relatively limited and would be more convincing with more diversity and quantity of test functions.

In addition to the extended synthetic test functions we presented, we also presented the following extended sequence optimisation results to reviewer uzmJ.

Futhermore, we have also implemented the Solvabity vs. SASA experiment from [31] which uses the Foldx simulator as a black box. This experiment is run for 64 rounds, with batches of 16, and only permits one mutation per sequence per round. This required a masked Transformer generative model (mTFM), $q$, that learns to mutate an input sequence, in a similar fashion to the generative back-bones of LaMBO and LaMBO-2. A-GPS and VSD perform similarly to LaMBO-2 on this experiment (and outperform a greedy random baseline and CbAS), and we feel it adds yet another compelling example of this method. Round 64 results summarised below,

Thank you for your appraisal of our work.

Esteemed reviewer -- we have managed to produce results for qNEHVI for $L=5$ below:

We did have to decrease the number of random restarts from 10 to 5, and reduce the number of samples to approximate EHVI from 128 to 64, and so this should viewed as a lower bound result. Furthermore, reviewer uzmJ has requested we change the experiment for the GP-based methods are optimising in the original, $[0, 1]^d$, space, and not the sigmoid transformed space. We are currently working on reproducing these synthetic test function results to honour this request.