Many-Objective Multi-Solution Transport

Dear Reviewer ew6B,

Thank you for your detailed review and for recognizing the strengths of our work. We highly value your constructive feedback and provide responses below to address the issues you raised.

---

Q1. Differentiating MosT from works like [1] "Personalized Federated Learning under Mixture of Distributions" and [2] "Harmony Multi-Task Decision Transformer for Offline Reinforcement Learning"

MosT focuses on topics fundamentally different from both works, though the applications are related.

MosT focuses on optimizing many objectives by finding a few diverse and complementary solutions on the Pareto front, emphasizing objective coverage and solution trade-offs for $n \gg m$ cases. In contrast, [1] tackles personalized federated learning, modeling heterogeneity among clients, with no focus on the diversity of solutions or Pareto optimization. We acknowledge and cite [1] in the related work section.

Similarly, [2] is designed specifically for offline reinforcement learning, aiming to optimize a single unified policy across tasks using task-specific masks. MosT, however, provides a general framework that prioritizes multiple diverse solutions for many-objective optimization rather than unifying tasks under one policy.

Q2. Request for more detailed analysis of diversity-encouragement regularization in Section 6.5

Thank you for your question. A detailed analysis of how diversity-encouragement regularization enhances both performance and fairness is provided in Appendix A.5.3, as referenced in the first paragraph of Section 6.5.

Q3. Concerns about the inclusion of recent references

In the revision, we have incorporated recent advancements in many-objective optimization to ensure a comprehensive discussion of the field. We also clarified how MosT builds upon and differs from these works, highlighting its novel contributions, such as its focus on coverage of many objectives and solution diversity in the $n \gg m$ setting. This ensures MosT's position within the broader research landscape is clearly articulated.

---

Thank you once again for your valuable insights and suggestions.

Best regards,
The Authors

Dear Reviewer,

Given that the discussion period is coming to end soon, we sincerely look forward to your reply to our response, and we are open to any discussion to improve our paper.

Best wishes,
The authors.

Dear Reviewer ew6B,

We sincerely appreciate the detailed feedback you provided during the review phase and your time in reviewing our rebuttal. As the discussion phase is nearing its conclusion, we kindly wanted to check if there are any remaining concerns or comments from your side regarding our responses. Your insights are invaluable to us, and we would be happy to clarify any additional points if needed.

Thank you for your attention and support.

Best regards,
The Authors

Dear Reviewer GmMP,

Thank you for your detailed review and for highlighting the strengths of our work. We are pleased to provide responses and clarifications to address the points you raised.

---

Q1. Indicating choice of solutions in Figure 1 ... how you choose a particular solution

The method for selecting solutions is detailed in the experimental sections for each application. For example, in federated learning, each device selects the best model from $m$ solutions based on the validation set. Additionally, the selected solutions are visualized separately in Figure 8 for further clarity.

Q2. Clarification regarding the usefulness of MosT in cases where $N \leq M$

MosT is particularly effective for the $N \leq M$ case because it provides flexibility in addressing different trade-offs among objectives while avoiding redundancy among solutions. If each objective is assigned its own solution ($M = N$), the result is often trivial and inefficient, as it creates isolated, non-cooperative solutions that fail to leverage shared structure or features among objectives (tasks, domains, etc). This redundancy increases computational and storage costs without necessarily improving overall performance.

By contrast, MosT optimizes $M$ solutions in a coordinated manner using bi-level optimization, allowing solutions to specialize in subsets of objectives while maintaining diversity. This approach captures varying trade-offs and ensures balanced coverage of objectives, achieving greater adaptability and resource efficiency.

Q3. Rationale behind using only final layer bias weights for MGDA weight computation

We use the final layer bias weights for MGDA weight computation as they effectively capture objective-specific influences while simplifying computational complexity. This choice is based on empirical observations that final layer biases often hold sufficient representational power to approximate gradient alignment across objectives. This aligns with the primary goal of MGDA to minimize conflicting gradients.

Q4. Client diversity in FL and MosT’s lack of consensus

Client diversity is essential in non-i.i.d. federated learning, where heterogeneous data distributions limit the generalizability of a single global model. MosT explicitly promotes diversity by specializing in a few solutions each focusing on a different subset of objectives (clients). Unlike traditional FL, which enforces periodic aggregation for a consensus model, MosT’s design preserves client diversity throughout training to ensure robust specialization and adaptability across varied client distributions.

Q5. Qualitative/quantitative comparison with prior work ([1] "Few for Many: Tchebycheff Set Scalarization for Many-Objective Optimization")

We appreciate the reviewer highlighting [1] as a relevant study. However, we note that [1] is a concurrent work and, as of now, does not provide open-source code for reproduction. Additionally, [1] focuses on convex optimization problems training simple nonlinear models on synthetic datasets only, which are not directly comparable to the more complex real-world scenarios addressed in our work. As such, we do not include a comparison with [1].

---

Thank you once again for your valuable feedback and insights. Your comments are instrumental in refining our paper.

Best regards,
The Authors

Dear Reviewer,

Given that the discussion period is coming to end soon, we sincerely look forward to your reply to our response, and we are open to any discussion to improve our paper.

Best wishes,
The authors.

Dear Reviewer oKXp,

Thank you for your detailed feedback. We appreciate the chance to address your concerns and clarify our work.

---

Q1. How does diversity relate to its complementary subsets? Can "complementary subsets" be clearly defined? Does MosT's solution set optimize specific diversity metrics?

The solutions of MosT are diverse since they focus on different subsets of objectives or different subregions on the Pareto front (PF), capturing different user preferences. On the other hand, the constraint in OT enforces all solutions together to cover all objectives or the whole PF, so they are complementary subsets. More details:

• Diversity refers to the optimization of each solution focus on a different subset of objectives. We define diversity formally in Section 4 based on cosine similarity between solutions in terms of their matching weights to objectives in $\Gamma$. So it measures the distinctiveness of solutions' specialization on the objectives.
• Complementary subsets of objectives: The OT constraints in Eq. 3 enforce a balanced assignment of $n$ objectives to the $m$ solutions. By further encouraging sparsity in the matching matrix $\Gamma$, each solution in MosT optimizes a subset of objectives and all the solutions together cover all objectives. So the subsets covered by different solutions are complementary.
• The theoretical guarantees provided (Theorems 4.1 and 4.2) show that MosT encourages diverse solutions while ensuring convergence to Pareto-optimal points in both convex and non-convex settings.

Q2. Why prefer a balanced weight matrix in solution-objective matching

A balanced weight matrix is essential for capturing diverse trade-off points on the Pareto front and addressing varied user preferences over objectives. While a single solution may suffice for highly similar objectives, MosT is designed for optimizing many diverse and often conflicting objectives. By balancing the weight matrix, MosT ensures that (1) no objective is ignored or downweighed; and (2) no solution is dominating others or over-specialized, providing broad and well-distributed coverage of the objective space. This approach allows MosT to meet a wide range of user preferences and promotes robustness by ensuring no single solution dominates.

Q3. How is MosT theoretically distinct from MGDA

You are correct that for a single solution with fixed weights for objectives, MosT’s theoretical guarantee aligns with that of standard MGDA. However, MosT is designed to find a finite diverse set of multiple solutions, not just a single one. Through its bi-level optimization framework, MosT jointly optimizes the matching weights and solutions, leveraging optimal transport to dispatch objectives to solutions. This ensures each solution specializes in a complementary subset of objectives, promoting diversity and achieving a well-distributed Pareto set.

Q4. Comparison with sum-of-minimum optimization method [1]

[1] is clustering-based, which has inherent limitations: a notorious one is solution collapse, i.e., only one solution gets fully trained and dominates others on all objectives, as shown in our analysis (Appendix A.5.1, Figure 7). Clustering approaches are also rigid, often failing in practical scenarios where cluster structure assumption does not hold, e.g., when task relationships are dynamic and cannot be neatly grouped. In Table 2, experiments with TAG (clustering-based) method on multi-task learning datasets show poor alignment between task groupings and model performance, achieving only 31.05% average accuracy on DomainNet, failing to outperform other baselines or MosT. These results emphasize that MosT’s general framework, which dynamically identifies task relationships and ensures Pareto optimality, is better suited for addressing practical challenges beyond the clustering setting.

For your reference, we've provided the learnt task grouping results by TAG on DomainNet ($n = 6$, $m=4$):

• Group 1: [1,3]
• Group 2: [1,4,6]
• Group 3: [2,3]
• Group 4: [5,6]

Furthermore, in the table below, we've included the model performance per task on the DomainNet dataset. It's evident that the model performance does not align with TAG's task grouping, possibly due to task conflicts leading to oversight of some objectives. This highlights the difficulty of identifying task groupings.

Q5. Runtime analysis for linear scalarization vs MGDA

When the number of objectives is small, the computational cost of finding common descent directions in MGDA is negligible, so the observed runtime difference is likely due to other factors such as IO overheads or system latency.

---

Thank you once again for your valuable insights and questions.

Best,
The Authors

Dear Reviewer,

Given that the discussion period is coming to end soon, we sincerely look forward to your reply to our response, and we are open to any discussion to improve our paper.

Best wishes,
The authors.

Dear Reviewer RKfr,

Thank you for your detailed review and for recognizing the novelty and adaptability of our proposed method. We are pleased to address the points you raised and provide additional clarifications.

---

Q1. ... to compare methods based on their diversity and the coverage of Pareto Front, nontrivial benchmarks like [1] (and benchmarks therein) must be also taken into consideration.

We appreciate the relevance of WFR-GF and will cite it. However, a direct comparison is not appropriate due to fundamental differences in the focused problems and evaluation. MosT specifically targets many-objective optimization ($n \gg m$) in practical challenges, while WFR-GF is designed for traditional multi-objective problems with few objectives. Furthermore, MosT is evaluated on diverse, real-world benchmark tasks—including federated learning and multi-task learning—covering highly heterogeneous and scalable objectives. This broader benchmarking demonstrates MosT’s applicability to practical, high-dimensional scenarios, which are beyond the scope of WFR-GF's synthetic datasets and focus on low-dimensional Pareto front exploration.

Q2. Clarification on not using hypervolume for evaluation

The computational complexity of hypervolume calculations becomes prohibitive in high-dimensional many-objective scenarios, limiting its feasibility for our experiments. Specifically, calculating hypervolume for $m$ solutions over $n$ objectives has exponential time complexity in $n$, making it unsuitable for our large-scale many-objective tasks. Instead, we use average accuracy as a more important metric to practitioners, which reflects a balance between computational efficiency and the ability to capture overall performance across objectives.

Q3. Explanation of the chosen evaluation metric (average accuracy)

Average accuracy is calculated by averaging the accuracy across all objectives, ensuring a balanced evaluation across both high- and low-performing objectives.

Q4. Goal for the case where $M > N$ (Section 3.2). Is our goal to capture M different trade-offs among N objectives even though M >> N?

Yes, exactly. When $M > N$, our goal is to capture diverse trade-offs among the $N$ objectives rather than simply assigning each solution to a single objective. An example is shown in Figure 4. By maintaining $M$ solutions, we ensure a richer representation of the Pareto frontier, offering greater flexibility to accommodate various trade-offs and adapt to different contexts. This is especially valuable in practical applications where capturing fine-grained, nuanced trade-offs is critical for robustness and adaptability.

---

Thank you once again for your insightful feedback. Your comments have provided us with valuable directions for strengthening our paper.

Best regards,
The Authors

[1] Ren, Yinuo, et al. "Multi-objective optimization via Wasserstein-Fisher-Rao gradient flow." International Conference on Artificial Intelligence and Statistics. PMLR, 2024.

Dear Reviewer z8sS,

Thank you for your positive feedback and for recognizing the strengths and contributions of our work. We are pleased to provide clarification regarding your question.

---

Q. Convergence results regarding Pareto stationary solutions and potential arguments for Pareto optimality in convex and strongly-convex settings

Thank you for raising this point. As shown in Proposition 1 of [1], when each loss function $\mathcal{L}_i(\theta)$ is convex and continuous in $\theta$, and the solution space $\Theta$ is convex, the Pareto frontier of the multi-objective optimization problem is also convex. This property allows us to guarantee that any Pareto stationary solution found by MosT is indeed Pareto optimal in these settings.

Furthermore, in strongly-convex cases, the convergence guarantees provided in Theorem 2 (Strongly-Convex) in our paper ensure that the solutions converge to Pareto stationary points, and the convexity assumption ensures that these points are globally Pareto optimal.

---

Thank you for your encouraging feedback and for highlighting this opportunity to improve the clarity of our theoretical results. We have incorporated these enhancements in the revised version.

Best regards,
The Authors

[1] Wang, Yuyan, et al. "Small towers make big differences." arXiv preprint arXiv:2008.05808 (2020).

Dear Reviewer o2xh,

Thank you for your detailed review and constructive feedback. We have addressed each of your comments below and clarified specific aspects of our work.

---

Q1. problem not new ... Clarification on bi-level optimization and the dependency between $\Gamma$ and $\theta$

MosT addresses a novel problem: optimizing a small, diverse set of solutions for many-objective scenarios ($n \gg m$). Unlike existing works that focus on finding a single Pareto solution, MosT aims at the diversity of multiple solutions and their coverage over all objectives, ensuring each solution specializes in complementary subsets. This approach is particularly relevant for practical cases with large numbers of objectives, where balancing trade-offs with limited solutions is crucial.

We chose a simple connection between $\Gamma$ and $\theta$ in the upper-level optimization to avoid high-order gradients, which dominate computation and hinder efficiency. Estimating hypergradients $\frac{d\Gamma^*(\theta)}{d\theta}$ is computationally impractical and might not be reliable given the non-smooth nature of the optimal transport objective. Our simple design allows MosT to remain computationally efficient while effectively optimizing diverse solutions.

Q2. Limitations of MGDA-based methods when the number of objectives is large

It is challenging to apply vanilla MGDA to many objectives. But a very aim of MosT is to address this limitation of MGDA by assigning objectives to different solutions to be optimized, by leveraging optimal transport (OT).

As demonstrated in Proposition 1, Figure 2, and Section 5, OT induces sparsity in the solution-objective assignment matrix $\Gamma$, effectively reducing the number of competing objectives when optimizing each solution. This sparse grouping of objectives alleviates and avoids the well-known gradient conflicts in MGDA, especially on many objectives. By dynamically balancing objectives optimized for different solutions, MosT ensures more efficient optimization and better scalability, compared to standard MGDA-based approaches.

Q3. MGDA as a baseline and missing related work. Need for discussing solution diversity and the Pareto front with respect to [4]

While [4] provides an efficient scalarization approach for finding single solutions on the Pareto front, MosT tackles a different and broader challenge: optimizing many objectives with a limited budget of diverse solutions to be optimized. Our framework explicitly addresses the need for coverage and diversity in the solution space, leveraging optimal transport to allocate objectives dynamically.

Q4. Clarification on the TAG failure in MTL settings

TAG fails in MTL settings primarily due to the negative transfer within each group of the static clustering. This approach often forces tasks into rigid groups, resulting in shared parameters that degrade individual task performance. In DomainNet, for example, tasks come from diverse styles or domains (e.g., real images, sketches, paintings), making static clustering unsuitable. Task 1, involving real images, is fundamental and broadly relevant to other tasks, but TAG’s rigid grouping fails to capture these nuanced relationships. In contrast, MosT dynamically adapts to task relationships, effectively balancing trade-offs and avoiding the limitations of static clustering.

For your reference, we've provided the learnt task grouping results by TAG on both datasets:

• Office-10 Dataset ($n = 4$, $m=3$):

  • Group 1: Tasks [1,2]
  • Group 2: Tasks [1,3]
  • Group 3: Tasks [1,4]

• DomainNet Dataset ($n = 6$, $m=4$):

  • Group 1: Tasks [1,3]
  • Group 2: Tasks [1,4,6]
  • Group 3: Tasks [2,3]
  • Group 4: Tasks [5,6]

Furthermore, in the table below, we've included the model performance per task on the DomainNet dataset. It's evident that the model performance does not align with TAG's task grouping, possibly due to task conflicts leading to oversight of some objectives. This highlights the difficulty of identifying task groupings.

Q5. Eq. (45-46)->Eq. (47) seems need to be bounded functions. The authors do not make this assumption.

It actually requires the difference between loss at the initial point and the loss at the $T+1$'s around to be bounded. The dependence on this gap is standard in optimization, e.g., Theorem 1 in [1].

---

Thank you once again for your valuable feedback and for identifying specific areas for improvement! We have carefully addressed your comments and incorporated them into the revised manuscript.

Best regards,
The Authors

[1] "Convergence guarantees for a class of non-convex and non-smooth optimization problems." PMLR, 2018.

Dear Reviewer,

Given that the discussion period is coming to end soon, we sincerely look forward to your reply to our response, and we are open to any discussion to improve our paper.

Best wishes,
The authors.

Thanks for the author's reply. But these replies did not completely solve my problem.

---

R1: I understand that the proposed method tries to obtain a diverse set of solutions for many objective scenarios. It is different from standard gradient-based MOO, which only finds one solution.

However, my question is when we only focus on the optimization problem and optimization algorithm. This bi-level optimization (The author mentions it multiple times) in the paper is not correctly introduced and fully analyzed. Eqs. (1-3) is actually ill-posed. The vector $\Gamma$ is in the set related to $\theta$, so it is a function w.r.t $\theta$. This is why I ask the author to provide some related work to support their optimization problem's formulation. Can we just directly consider $\Gamma$ as a constant in the upper level? Imho, the original upper-level should be $\Gamma(\theta)$, then we can replace this function to the optimal value $\Gamma^*$, where $\Gamma^*$ is solved by OT solver. (we can consider the OT solver's solution to be accurate). Then we just ignore the high order gradient so the proposed method is reliable. So, removing a high-order gradient can affect the optimization, and the author needs to analyze whether this error can be bounded.

The author does not provide any evidence to support their alternating optimization algorithm because they consider $\Gamma$ as a constant in the upper level at the first step, which I think needs many words to explain. I understand the author's approach, but I think these descriptions are not rigorous enough.

---

R2-R3: I understand that the proposed method tries to solve a different problem. However, MGDA is still an old baseline. In the multi-task learning area, many new baselines only need $\mathcal{O}(1)$ computational cost and can achieve better performance compared with MGDA. The proposed method still needs to calculate the gradient for each task in each iteration. Considering that the improvement of the proposed method on the Office-Caltech10 and DomainNet is not large compared with MGDA, I still suggest the authors add more baselines. Also, in the FL setting, two baselines are old.

---

R5: I think the authors' reply is not correct.

I checked Theorem 1 in the provided paper ("Convergence guarantees for a class of non-convex and non-smooth optimization problems"). The Theorem 1 only requires that $F_1-F^*$ is bounded. This is reasonable because the initial point can be considered as a constant, and the global optimal is also a constant if the function has a minimizer.

However, what the authors used in Eq. (45-46)->Eq. (47) is the difference between the loss at the initial point and the loss at the $T+1$'s is bounded. These two things are different. We can not guarantee the generated sequence of $F_T$ is bounded. I suggest the author read the paper "Mitigating Gradient Bias in Multi-objective Learning: A Provably Convergent Approach, ICLR 2023". This paper uses bounded assumption to bound equation 78.

In fact, the assumption of bounded function is very important in gradient-based MOO. Many papers try to design an algorithm to remove this assumption, such as "Mitigating Gradient Bias in Multi-objective Learning: A Provably Convergent Approach, ICLR 2023". Therefore, I feel surprised that the author uses this property without assuming it.

---

Overall, I am not satisfied with the mathematical writing of this article, and the theoretical assumption in this article is also incorrect. Although the problem solved is novel, there are relatively few and older baseline methods compared. I will keep my score.

Thank you for your detailed feedback. We respectfully address your concerns below:

---

R1 Bi-level Optimization and Theoretical Rigor: Our problem formulation is rigorous and your proposed formulation is consistent with Eq. (1)-(3).

• "Can we just directly consider $\Gamma$ as a constant in the upper level?" - $\Gamma$ is a constant in the upper level optimization as shown in Eq. (1).
• "then we can replace this function to the optimal value $\Gamma^*$, where $\Gamma^*$ is solved by OT solver" - The $\Gamma^*$ you described is exactly what we defined in Eq. (2), which is the optimal solution achieved by OT.
• "Then we just ignore the high order gradient so the proposed method is reliable" - We did ignore the high order gradient in Eq. (6) so our method is reliable.

---

R2-R3 Baselines: we proposed MosT as a general multi-objective multi-solution optimization that can address rich and diverse learning problems. We examined it on three real applications of three different machine learning tasks. Because MosT is NOT specifically designed and optimized for one task (e.g., FL), we chose the most principal and representative approaches for each task as the baselines. MosT outperforms these baselines in every task on several benchmarks, validating its advantages and generalizability.

---

R5 Boundedness Assumptions: Eq. (45-46) -> Eq. (47) simply holds true since we have $L_i(\theta_j^1) - L_i(\theta_j^{T+1}) \leq L_i(\theta_j^1) - L_i(\theta_j^*)$ by the definition of the minimizer $\theta_j^*$. So we do not need to assume the generated sequence is bounded. We understand your intuition but it might add an unexpected layer of complexity to the problem.

---

We appreciate your thoughtful suggestions and will further clarify the above points in the next version to avoid any misunderstanding as yours. We hope we have clarified your confusion. Thanks!

Thanks for the author's reply.

---

R1: I think I have made my viewpoint on this problem very clear. I'm not saying that the author's approach is wrong. What I'm saying is that this approach is not standard and requires evidence.

The challenge of bi-level optimization is the nested dependency between upper-level variables and lower-level variables. Most studies of bi-level optimization focus on addressing this challenge because this nested dependency leads to hypergradients. However, the author simply uses one sentence to solve this challenge: "At a high level, the bi-level optimization problem described above can be decoupled into two sub-problems (over Γ and θ1:m)when fixing one variable and optimizing the other." (Line 178). No citation and no explanation. After this sentence, the bi-level structure and hypergradient simply disappeared. The problem becomes a single-level MOO. My point is this is not a standard way to solve bi-level optimization problems. The Primary Area of this article includes "meta-learning". I think most representative methods in learn-to-learn (which can be considered as bi-level optimization), such as MAML and DARTS, do not totally ignore hypergradient terms similar to the approach in this paper.

If the authors think their approach is standard and the hypergradients can be removed in bi-level optimization without much impact, list several references and say we just follow their approach. Problem solved. If the authors think their approach is not standard and is new, analyze the effect of removing the hypergradient information. Problem solved. I think my question is not hard to answer, and both ways can enhance the optimization method.

In addition, I just want to correct one sentence in the authors' reply: "$\Gamma$ is a constant in the upper level optimization as shown in Eq. (1).". Once we write this nested optimization formulation (Eqs. (1-3)), $\Gamma$ naturally becomes a function w.r.t. $\theta$. This is determined by this optimized structure because changing $\theta$ will change the value of $\Gamma$. No matter how the author writes about $\Gamma$ in Eq. (1), it does not change this dependency. Only when we use a gradient-based method to solve this problem and focus on one iteration can we consider it as a constant (Like the authors claimed in Lines 198-201). Let's say for $t$-th iteration, we want to get $\theta_{t+1}$ with $\theta_{t}$ and calculate $\Gamma_t^*(\theta_t)$; in this case, we may decide whether to consider it as a constant. But, this is different from the original problem formulation (Eqs. (1-3)).

---

R5: Thanks for your clarification. I find I misunderstood the authors' first reply, and the proof is correct. The problem here is different from my provided ref.

---

I apologize to the author for my factual error and my incorrect judgment towards Q10. For this reason, I raised my score from 3 to 5. The author's reply does not change my concern about the rigor of the optimization algorithm and insufficient baseline methods.