ParetoFlow: Guided Flows in Multi-Objective Optimization

Dear Reviewer,

Thank you for your invaluable feedback, which has greatly guided our revisions. We have incorporated your suggestions into the manuscript, with all changes clearly marked in red for easier reference.

Weaknesses

The proposed method relies on the accuracy of the objective predictors, which might be a bottleneck on problems with complex landscapes and limited offline dataset.

We acknowledge that the accuracy of the objective predictors is indeed a critical factor, and it can be a limitation, especially for complex problem landscapes and limited offline data. This challenge is not unique to our method but is a common aspect of all offline optimization frameworks. Nevertheless, our evaluation results in the paper show that our ParetoFlow framework can achieve strong performance in the tested scenarios.

To mitigate this limitation in more complex tasks, one potential future direction is to pre-train a feature extraction model that provides robust representations as input for the predictor. For example, in the protein domain, pre-trained models such as protein BERT have demonstrated strong downstream prediction performance even with limited labeled data. Integrating such techniques could enhance the robustness of the predictors and improve overall performance. We plan to explore this approach and other enhancements in future work.

It seems that the required weight vectors will grow exponentially (i.e., a problem with 10 objectives neesds at least 1024 vectors to ensure each objective has at least 2 weights). It may be time consuming to generate and sample using these weights, while less weights may not fill the PF.

Generating these weights is straightforward, and we can efficiently generate them using the Das-Dennis method.

While optimizing numerous weighted objectives can indeed be challenging, our method is designed to scale effectively. For instance, if we have 1024 weight vectors, our parallelized code processes a batch of 1024 samples, which can be handled efficiently and in parallel using GPU resources. It is quite common to use batch sizes of 1024 or even up to 10240 in modern computational settings.

Additionally, we would like to note that most real-world problems typically involve only 2 or 3 objectives. Therefore, our framework remains efficient and effective for these more common cases.

Questions

In Section 3.1 authors consider the case of non convex PF. I wonder whether such cases show up in the experiemnts? If so, does the designed filtering successfully process such cases?

Yes, we have already discussed the case of nonconvex Pareto fronts (PFs) in our "Effectiveness of Local Filtering" Appendix A.9. We conduct experiments on the convex-PF task ZDT$1$ and the nonconvex-PF task ZDT$2$. When we remove the local filtering, the performance of ZDT$1$ nearly does not change: from $4.88 \pm 0.02$ to $4.81 \pm 0.08$. In contrast, the performance on ZDT$2$ drops obviously: from $6.79 \pm 0.16$ to $5.78 \pm 0.15$. This demonstrates the effectiveness of our local filtering scheme in handling nonconvex PFs and verifies its underlying motivation.

In the Neighboring Update, the next state is the best one in the K nearest weight vectors, will it make a large ratio of vectors generate the same solution and lead to early convergence?

No, early convergence is not an issue as different weighted objectives typically favor different samples. In our analysis, we observed that only 12% of the points in C-10/MOP1 and 1% in MO-Hopper were duplicates. The higher duplication rate in C-10/MOP1 is primarily due to the decoding of continuous logits back to the same discrete values. We have detailed this phenomenon and its implications in Section 4.5, Ablation Studies.

Overall

We are grateful for your careful review and valuable feedback. Does our response address your concerns? We look forward to further interaction during the rebuttal phase.

Dear Reviewer,

Thank you for your constructive feedback and positive comments on our title, 'ParetoFlow.' Your insights have greatly contributed to improving our paper. We have carefully revised the manuscript to incorporate your suggestions, with all changes highlighted in red for your convenience.

Strengths:

Section 2.2 is clearly explained. However, could you elaborate on how using a normalizing flow would alter or improve the results?

We have now included two additional high-impact papers [r1, r2] in our revised submission to provide stronger evidence that flow matching outperforms diffusion models. Specifically, Section 5.5 (Ablation on generative modeling approaches) in [r1] and Section 3.6.2 (Ablations) in [r2] address this topic in detail.

Furthermore, to empirically substantiate our method's efficacy, we implemented a variant of our ParetoFlow framework using a diffusion model and conducted comparative experiments on two tasks, MO-Hopper and C-10/MOP1, using five random seeds. The results, detailed in the table below and added to our Appendix A.10 titled "Further Ablations", clearly demonstrate the superior performance of flow matching:

These empirical findings corroborate our choice of flow matching over diffusion models, supporting our claim with both qualitative and quantitative evidence.

[r1] Le M, Vyas A, Shi B, et al. Voicebox: Text-guided multilingual universal speech generation at scale[J]. Advances in neural information processing systems, 2024, 36.
[r2] Polyak A, Zohar A, Brown A, et al. Movie gen: A cast of media foundation models[J]. arXiv preprint arXiv:2410.13720, 2024.

Weaknesses

Insufficient Mathematical Rigor: The paper lacks adequate mathematical depth and rigor. Incorporating more detailed mathematical formulations, proofs, and theoretical underpinnings would strengthen the credibility and scholarly value of the work.

We have provided a formal definition of multi-objective optimization (MOO), detailed the function and significance of Equation (9), and described how evolutionary algorithms (EA) are integrated with flow matching to enhance our method's robustness.

Incorrect Formalization in Line 105: The statement on line 105 is inaccurate. A Multi-Objective Optimization (MOO) problem cannot be "formally" formulated merely as a vector optimization problem. Such a formulation is more of an informal notational convenience rather than a formal definition. Clarification and correct formalism are needed here.

We have revised the manuscript to provide a more formal and rigorous definition of the multi-objective optimization (MOO) problem, addressing the concern about the original informal notation.

Unclear Purpose of Equation (9): The role and application of Equation (9) are not clearly explained. It is difficult to understand its significance and how it fits into the overall argument of the paper. Providing a thorough explanation or context for this equation would enhance comprehension.

Equation (9) follows standard ODE sampling, with the exception of the added noise term. In our approach, each sample is associated with a weight vector, and we aim to find suitable samples along its sampling path to optimize the weighted objective. Without the noise term, the sampling trajectory becomes deterministic after the initial random noise is set, restricting exploration to a single path. Introducing the noise term through the Euler–Maruyama method enhances diversity and exploration, allowing the sampling process to generate a wider range of samples from each trajectory and achieve better coverage of the objective space. When there are $K$ neighboring samples, this method facilitates exploration of $K$ times the space in a single pass, eliminating the need to run the sampling path $K$ separate times, thus improving efficiency. We have investigated the impact of K in Figure 4, and a moderate increase of K does improve performance.

We are glad to have addressed your concerns. Thank you for taking the time to review our response and for your support in revisiting the evaluation. We truly appreciate your thoughtful consideration and positive feedback.

Dear Reviewer,

Thank you for your constructive feedback. Your observations have been invaluable for refining our paper. We have made revisions accordingly, with changes marked in red for ease of review.

Weaknesses

The paper mentions the computational cost of ParetoFlow but does not provide a detailed comparison with other generative modeling methods.

Thank you for highlighting the need for a detailed computational cost comparison. We have now expanded our analysis to include both generative and traditional methods and incorporated these into Appendix A.5 "Computational Cost". The table below summarizes the computational times (in minutes) for ParetoFlow and other methods across two tasks:

Considering the minimal overhead and significant performance gains of our method compared to baselines, we believe it provides a practical solution for researchers and practitioners seeking effective results without sacrificing speed.

Providing some visualization results for the generated samples would be beneficial.

We appreciate the suggestion and have included visualization results for C-10/MOP1 and MO-Hopper in the Appendix A.8 titled "Visualization and Case Study Details." The appendix features comparisons between offline samples and samples generated by ParetoFlow, clearly demonstrating the superior quality of the latter.

Questions.

Can the authors offer a case study for the generated architectures in terms of structure and performance on MO-NAS?

We have incorporated a detailed case study on C-10/MOP5 in our Appendix A.8 titled "Visualization and Case Study Details." This case study focuses on optimizing prediction error, model complexity, and hardware efficiency, as outlined in [r1].

The case study analyzes three sets of solutions where the weight vector for unconsidered objectives is set to zero:

1. Prediction Error and Model Complexity: This set emphasizes minimizing prediction error and model complexity.
2. Prediction Error and Hardware Efficiency: This set prioritizes reducing prediction error with improved hardware efficiency.
3. Balanced set: This set aims to balance prediction error, model complexity, and hardware efficiency.

We discuss the frequency of operators used in these sets, noting that the 3x3 convolution is consistently preferred for its effectiveness in reducing prediction error, while the 3x3 average pooling is less favored. Additionally, there is a notable shift from the use of 1x1 convolutions to 'none' operators in moving from the first to the second set, suggesting a trade-off for better GPU latency.

This analysis provides insights into the structural preferences and performance trade-offs in the generated architectures.

 [r1] Lu Z, Cheng R, Jin Y, et al. Neural architecture search as multiobjective optimization benchmarks: Problem formulation and performance assessment[J]. IEEE transactions on evolutionary computation, 2023, 28(2): 323-337.

How to guarantee the validity of the generated samples? For instance, an architecture with the valid computational flow in the NAS task.

To ensure that our generated samples are always valid, we adopt a strategy of encoding discrete architectures into continuous logits, as suggested by [r2, r3]. This approach allows us to train our models effectively in a continuous space. During the sampling process, we generate samples in the continuous (logit) space and then decode them back into the discrete architecture space. The encoding and decoding processes are straightforward and designed to preserve the integrity of the structures, ensuring that the decoded samples are valid architectures.

[r2] Trabucco B, Geng X, Kumar A, et al. Design-bench: Benchmarks for data-driven offline model-based optimization[C]//International Conference on Machine Learning. PMLR, 2022: 21658-21676.
[r3] Xue K, Tan R X, Huang X, et al. Offline Multi-Objective Optimization[J]. arXiv preprint arXiv:2406.03722, 2024.

Overall

We appreciate your thorough review and valuable feedback. Thank you for your insights. Does this reply address your main concerns? We look forward to more discussion in the rebuttal.

Dear Reviewer,

We greatly appreciate your insightful feedback, which has played a crucial role in improving our manuscript. We have thoughtfully addressed your suggestions and highlighted all revisions in red to facilitate your review.

Weaknesses

You only cited [1] to state that Flow matching outperforms Diffusion models, which I find insufficient. This is especially true in the context of multi-objective optimization, where specific problems require specialized analysis.

We have now included two additional high-impact papers [r1, r2] in our revised submission to provide stronger evidence that flow matching outperforms diffusion models. Specifically, Section 5.5 (Ablation on generative modeling approaches) in [r1] and Section 3.6.2 (Ablations) in [r2] address this topic in detail.

Furthermore, to empirically substantiate our method's efficacy, we implemented a variant of our ParetoFlow framework using a diffusion model and conducted comparative experiments on two tasks, MO-Hopper and C-10/MOP1, using five random seeds. The results, detailed in the table below and added to our Appendix A.10 titled "Further Ablations", clearly demonstrate the superior performance of flow matching:

These empirical findings corroborate our choice of flow matching over diffusion models, supporting our claim with both qualitative and quantitative evidence.

 [r1] Le M, Vyas A, Shi B, et al. Voicebox: Text-guided multilingual universal speech generation at scale[J]. Advances in neural information processing systems, 2024, 36.
 [r2] Polyak A, Zohar A, Brown A, et al. Movie gen: A cast of media foundation models[J]. arXiv preprint arXiv:2410.13720, 2024.

detailed insights into how well the method scales

To provide more detailed insights, we have conducted a thorough analysis across a diverse set of MO-NAS tasks from the NAS-Bench series, including C-10/MOP1, MOP2, MOP5, MOP6, and MOP7.

Our findings indicate that despite variations in the number of objectives and design dimensions, the computational cost of our method remains consistent. Notably, for tasks with higher dimensional objectives such as MO-Swimmer and MO-Hopper, our computational efficiency is comparable to tasks with lower dimensions. This consistency underscores our method's scalability across a range of computational demands. Additionally, our method consistently achieves strong performance, further attesting to its robustness. We have detailed these results in the table provided above, which compares key metrics across tasks, and included more in-depth discussion in the Appendix A.5 titled "Computational Cost".

In terms of solution quality, we adopt a strategy of encoding discrete architectures into continuous logits, as suggested by [r3, r4]. This approach allows us to train our models effectively in a continuous space. During the sampling process, we generate samples in the continuous (logit) space and then decode them back into the discrete architecture space. This encoding and decoding process is straightforward and ensures that the structural integrity of the architectures is maintained, resulting in valid and high-quality decoded samples.

[r3] Trabucco B, Geng X, Kumar A, et al. Design-bench: Benchmarks for data-driven offline model-based optimization[C]/ ICML 2022
[r4] Xue K, Tan R X, Huang X, et al. Offline Multi-Objective Optimization[J]. ICML 2024