UniCBE: An Uniformity-driven Comparing Based Evaluation Framework with Unified Multi-Objective Optimization

Dear Reviewer 8sRK, we sincerely thank you for your valuable feedback on our submission. Below is our responses to the concerns you raised. We have incorporated these contents into the updated version of our paper, which we believe will help enhance the quality of our submission.

We have also provided a PDF version via an anonymous link for a clearer and more complete presentation of our response: https://anonymous.4open.science/r/Respones-to-Reviewer-8sRK-8A44/response%20to%20reviewer%208sRK.pdf. This can also be find in the Supplementary Material.

The presentation can be improved, especially lack in explaining why the method is better than others in an intuitive and easy to follow way. The authors argue that to avoid bias, the budget should be allocated uniformly, if so how could this method be more sample efficient than random? I guess the reason is if model A is much better than B and model B is much better than C, then it's not necessary to compare A and C a lot. But if thats the reason why this method is more sample-efficient, it would be contradictory to the uniform assumption. Could the authors provide more insight into this?

Your question raises important points for discussion, and we will respond from two perspectives:

First, you mentioned leveraging the transitivity of model performance to reduce the preference budget. This approach is feasible when the optimization goal is to determine the relative ranking of models or to identify the best model (as demonstrated by the UCB algorithm [1]). However, our goal is to precisely evaluate the true capability values of each model rather than their order or the selection of the optimal model. For example, for models $A$, $B$, and $C$, we aim to determine whether their capability values are [0.9, 0.41, 0.4] or [0.9, 0.89, 0.4], rather than just concluding that $A > B > C$. This distinction is crucial in practical applications because model deployment decisions often involve balancing performance and cost. If $A$’s API price is significantly higher than $B$’s and we know with precision that the performance gap between $A$ and $B$ is minimal (e.g., 0.9 vs. 0.89), we might prefer model $B$. Without precise capability values, such decisions become difficult when only ranking information is available.

Therefore, when our optimization goal is to determine the exact capability values of models, uniform sampling becomes intuitive. First, since tasks vary in difficulty, the same model may perform differently across tasks. As mentioned on line 213 in the original manuscript, the transitivity of model performance may not hold in some cases ($A> B, B>C, C>A$). Uniform sampling across candidates helps mitigate these biases and improves accuracy. Second, since we need the capability values for all models, uniformly sampling across them ensures balanced data collection, reducing the uncertainty in the estimated capability values for any particular model.

References

[1] Jin Peng Zhou, Christian K. Belardi, Ruihan Wu, Travis Zhang, Carla P. Gomes, Wen Sun, and Kilian Q. Weinberger. On speeding up language model evaluation.

We apologies for our earlier focus on explaining why the budget should be allocated uniformly. Now, let us address why UniCBE outperforms Random. (We have also provided a PDF version via an anonymous link for a clearer and more complete presentation of our response: https://anonymous.4open.science/r/Respones-to-Reviewer-8sRK-8A44/FURTHER%20DISCUSSION%20WITH%20REVIEWER%208SRK.pdf)

At the expectation level, random sampling ensures a relatively uniform distribution across different tuple combinations, thereby guaranteeing the accuracy of evaluation results (lines 108-110). However:

• Argument 1: On the one hand, randomness prevents the optimization objective from being fully achieved, meaning uniformity cannot be attained optimally.
• Argument 2: On the other hand, UniCBE’s optimization objectives go beyond improving accuracy to include enhanced scalability and convergence, which random sampling cannot achieve.

Experimental Validation:

As shown in Table below, the $\beta_{\text{acc}}$ achieved by RANDOM is lower than that of UniCBE, which validates that randomness results in suboptimal uniformity (Argument 1). Additionally, the $\beta_{\text{sca}}$ and $\beta_{\text{con}}$ achieved by RANDOM are also lower than those of UniCBE, confirming that UniCBE explicitly optimizes for scalability and convergence objectives, unlike Random (Argument 2).

Furthermore, we experimentally validated this from another perspective involving temperature. We reformulate sampling matrix $P^l$ to:

$f^{ts}_T(i,j,k)$

$=\frac{(P^l_{i,j,k})^{-T}}{\sum (P^{l})^{-T}}$

By adjusting the temperature $T$, we control the extent of uniformity constraints. As $T$ increases, the uniformity constraints are progressively relaxed. When $T=0$, it corresponds to UniCBE (greedy sampling), which enforces the strictest uniformity constraints. When $T = +\infty$, it corresponds to random sampling, where no uniformity constraints are applied. As illustrated in Figure 10, as $T$ increases from $0$ to $+\infty$, the evaluation results progressively deteriorate. This demonstrates that adopting greedy sampling to enforce the strictest uniformity constraints yields the best evaluation performance, further validating Argument 1.

We sincerely apologize once again for the oversight in our previous response! If you have any questions, we would be more than happy to engage in further communication with you. At the same time, we would like to express our heartfelt gratitude for your support and suggestions regarding our work.

The given choice of greedy sampling over probabilistic sampling and Bradley-Terry model over Elo rating system appears significant to the framework’s success. Could the authors conduct a small experiment to demonstrate that UniCBE maintains its effectiveness across different sampling and aggregation settings?

We appreciate your valuable suggestion. In fact, as shown in Figure 5, we explore the combination of UniCBE with probability sampling strategy, Elo rating and average win rate aggregation strategies, comparing these with the default configuration. Our findings are as follows:

• UniCBE consistently outperforms the baselines when combined with various settings.
• Under the default configuration (greedy sampling and BT model), UniCBE achieves optimal performance. We believe this is because greedy sampling maximizes sampling uniformity, and the BT model better alleviates sampling bias in cases of misaligned samples.

As the UniCBE is based on three matrix, each targeting different goal of accuracy, convergence, scalability, can user steer between those by adding hyperparameter for each matrix? Would it be also possible to quantify it through experiment?

We agree with your suggestion to add hyperparameters for each matrix to achieve controllability for different optimization objectives. In the original manuscript, we integrate sampling matrices targeting different optimization objectives with equal weights: $P^{l} = \frac{P^{acc\text{-}l} \circ P^{con\text{-}l} \circ P^{sca\text{-}l}}{\sum (P^{acc\text{-}l} \circ P^{con\text{-}l} \circ P^{sca\text{-}l})}$ In practice, when faced with varying requirements, it is straightforward to prioritize a specific objective by adjusting the weights $\theta_{acc}$, $\theta_{con}$, and $\theta_{sca}$ for these matrices, as shown below: $P^{l} = \frac{(P^{acc\text{-}l})^{\theta_{acc}} \circ (P^{con\text{-}l})^{\theta_{con}} \circ (P^{sca\text{-}l})^{\theta_{sca}} }{\sum ((P^{acc\text{-}l})^{\theta_{acc}} \circ (P^{con\text{-}l})^{\theta_{con}} \circ (P^{sca\text{-}l})^{\theta_{sca}})}$ As demonstrated in Table below, we set different settings and calculate the degree of achievement level for each optimization objective $\beta$ following the calculation procedure described in Appendix-E. Compared to equal-weight integration, users can easily increase the corresponding $\beta$ (e.g., $\beta_{acc}$) by assigning a larger weight to a specific optimization objective ($\theta_{acc}$), thereby better meeting their practical needs (accuracy). We also observe that enhancing a specific optimization objective often comes with a slight decrease in the achievement of other objectives. In Figure 11, we illustrate an example of improving accuracy, where $\theta_{acc}$ is increased from 1 to 2. We find that the increased focus on accuracy objective slightly slows down the convergence speed. As a result, when $T$ is relatively small, the performance of $\theta_{acc} = 2$ lags behind that of $\theta_{acc} = 1$. However, in the later stages, after convergence, the enhanced accuracy objective enables $\theta_{acc} = 2$ to outperform $\theta_{acc} = 1$, resulting in greater savings in the preference budget.

While scalability is addressed by sequentially adding models, the paper could enhance this section by incorporating real-world scenarios, where models enter and exit dynamically, further proving UniCBE’s robustness in evolving benchmarks.

We agree with your suggestion that testing UniCBE in a highly dynamic, real-time evaluation setting can help us more comprehensively assess its performance. To this end, we conduct the following experiments: Starting with a sample size of $N=600$ and model number of $M=12$, we execute a random operation at each time step. The operations included: adding one model to be evaluated with a probability of 0.01, removing one model with a probability of 0.01, adding one potential sample with a probability of 0.01, randomly deleting one sample with a probability of 0.01, and taking no action with a probability of 0.96. Based on the experimental results shown in Figure 9, we have the following observations:

• The convergence speed of all baseline methods significantly slowed down. None of the baseline methods achieve a Spearman correlation coefficient of 0.96 or a Pearson correlation coefficient of 0.97 by $T=2000$, highlighting the difficulty of model evaluation in this setting. In contrast, UniCBE achieve rapid convergence, reaching a Spearman coefficient of approximately 0.97 and a Pearson coefficient exceeding 0.98 by $T=2000$.
• Over the long term, as $T$ increases, UniCBE consistently demonstrates over 10% savings in preference budget across all metrics, even under this challenging setting, showcasing its strong practicality.
• An interesting observation is that AlpacaEval exhibits better convergence in the early stages compared to Random and Arena, supporting our previous conclusions in Table 1. However, as $T$ increases, AlpacaEval's lack of accuracy optimization objective leads to its performance being surpassed by Random and Arena.

Dear Reviewer 8LCX, we sincerely thank you for your valuable feedback on our submission. Below is our responses to the concerns you raised. We have incorporated these contents into the updated version of our paper, which we believe will help enhance the quality of our submission.

Since this page cannot display images, we have provided a PDF version via an anonymous link for a clearer and more complete presentation of our response: https://anonymous.4open.science/r/Respones-to-Reviewer-8LCX-809F/response%20to%20reviewer%208LCX.pdf. This can also be find in the Supplementary Material. We recommend viewing the PDF in the provided link for easier reading.

The novelty of balancing accuracy, convergence, and scalability needs further justification, as similar uniform sampling strategies have been discussed in prior works that highlight the uniformity, such as Vabalas et al. (2019) for sampling biases, which could diminish its uniqueness.

We agree with your suggestion to discuss related work concerning sampling uniformity in connection with our work. Here is our discussion:

Previous studies have discussed the risks of introducing sampling bias in incomplete sampling scenarios. Specifically, [1] demonstrated through simulation experiments that K-fold cross-validation (K-fold CV) can produce significant performance estimation bias when dealing with small sample sizes. This bias persists even when the sample size reaches 1000. In contrast, methods like nested cross-validation (Nested CV) and train/test split have been shown to provide robust and unbiased performance estimates regardless of sample size. [2] introduced a weighting scheme, as described in [3], to mitigate sampling bias in active testing scenarios. [4] proposed leveraging information obtained from source models to select representative samples from the test set, thereby reducing sampling bias. Additionally, [5] employed Item Response Theory [6] to correct sample bias in addressing this issue.

These studies inspired us to investigate the bias problem in the CBE scenario. Unlike the aforementioned studies, we found that in CBE scenario, not only does sample bias exist, but model bias also plays a role, and the two are coupled. This coupling poses greater challenges for analyzing and mitigating these biases. To address this, based on the analyses outlined in Section 3, we propose the UniCBE method, which effectively alleviates biases in this scenario.

Although the experiment of MT-Bench is based on human evaluator, larger portion of the evaluation is relied on AlpacaEval, as larger number of models and samples are used for the evaluation with AlpacaEval. The reliance on GPT-4 and GPT-3.5-turbo as evaluators, while useful, could benefit from validation against human judgments or additional LLMs, such as Claude, to establish greater reliability and generalizability across evaluator types.

Yes, in our experiments, we test the performance of UniCBE with humans, GPT-4o, and GPT-3.5-turbo as judges. We recognize that involving a broader range of evaluators helps enhance the robustness and generalizability of our experimental conclusions. Therefore, below we additionally test the compared methods using Qwen-Plus[7] as the judge. As shown in Figure 14, when Qwen-Plus is used as the judge, the experimental results are similar to those obtained with other types of judges. UniCBE achieves significant preference budget savings, exceeding 20%. This experimental result further validates the generalizability of UniCBE to different sources of preference signals.

Minor details, but the readability of all figures could be enhanced by widening the lines in each plot, which would improve clarity and interpretation for readers.

Thank you for your valuable suggestions. We have increased the width of the lines in the figure in the revised version to enhance readability. You can refer to the latest version.

References

[1] Sebastian Farquhar, Yarin Gal, and Tom Rainforth. On statistical bias in active learning: How and when to fix it.

[2] Jannik Kossen, Sebastian Farquhar, Yarin Gal, et al. Active testing: Sample-efficient model evaluation.

[3] Frederic M Lord and Melvin R Novick. Statistical theories of mental test scores.

[4] Felipe Maia Polo, Lucas Weber, Leshem Choshen, et al. tinybenchmarks: evaluating llms with fewer examples.

[5] Andrius Vabalas, Emma Gowen, Ellen Poliakoff, et al. Machine learning algorithm validation with a limited sample size.

[6] Rajan Vivek, Kawin Ethayarajh, Diyi Yang, et al. Anchor points: Benchmarking models with much fewer examples.

[7] An Yang, Baosong Yang, Binyuan Hui, et al. Qwen2 technical report.

Dear Reviewer 8LCX,

We sincerely appreciate your recognition of our work, UniCBE, as well as the time and effort you have dedicated to improving its quality. We have carefully addressed the questions and suggestions you raised, and have incorporated these valuable insights into the revised version, which we believe will further enhance the completeness of the manuscript and the generalizability of its conclusions. Noting that the discussion phase is approaching its end (11/26, AOE), we would be more than happy to engage in further discussion should you have any additional suggestions or questions.

Best regards.

Dear Reviewer PBe8, we sincerely thank you for your valuable feedback on our submission. Below is our responses to the concerns you raised. We have incorporated these contents into the updated version of our paper, which we believe will help enhance the quality of our submission.

Since this page cannot display images, we have provided a PDF version via an anonymous link for a clearer and more complete presentation of our response: https://anonymous.4open.science/r/Respones-to-Reviewer-PBe8-1DB1/response%20to%20reviewer%20PBe8.pdf. This can also be find in the Supplementary Material. We recommend viewing the PDF in the provided link for easier reading.

The paper can be improved with more performance comparison with existing work and state-of-the-art performance.

We agree with your suggestion that comparing UniCBE with state-of-the-art methods can help fully validate its effectiveness. In our experiments, we select the most widely used state-of-the-art methods, Arena and AlpacaEval, for comparison to evaluate the performance of UniCBE. In fact, although comparing-based evaluation holds significant research value in the era of large language models, the existing research in this area remains limited. Arena and AlpacaEval represent the most relevant and practical works we could identify. We encourage future researchers to explore this field further and investigate more efficient ways to utilize valuable preference signals.

Expect more statistical and experimental conclusions with the proposed CBE method for the scenario of large-scale preference learning.

We agree with your suggestion and therefore attempt to conduct experiments in a large-scale, highly dynamic CBE scenario that is closer to real-world conditions. Specifically: Starting with a sample size of $N=600$ and model number of $M=12$, we execute a random operation at each time step. The operations included: adding one model to be evaluated with a probability of 0.01, removing one model with a probability of 0.01, adding one potential sample with a probability of 0.01, randomly deleting one sample with a probability of 0.01, and taking no action with a probability of 0.96. Based on the experimental results shown in Figure 9, we have the following observations:

• The convergence speed of all baseline methods significantly slowed down. None of the baseline methods achieve a Spearman correlation coefficient of 0.96 or a Pearson correlation coefficient of 0.97 by $T=2000$, highlighting the difficulty of model evaluation in this setting. In contrast, UniCBE achieve rapid convergence, reaching a Spearman coefficient of approximately 0.97 and a Pearson coefficient exceeding 0.98 by $T=2000$.
• Over the long term, as $T$ increases, UniCBE consistently demonstrates over 10% savings in preference budget across all metrics, even under this challenging setting, showcasing its strong practicality.
• An interesting observation is that AlpacaEval exhibits better convergence in the early stages compared to Random and Arena, supporting our previous conclusions in Table 1. However, as $T$ increases, AlpacaEval's lack of accuracy optimization objective leads to its performance being surpassed by Random and Arena.

Dear Reviewer PBe8,

We sincerely appreciate your recognition of our work, UniCBE, as well as the time and effort you have dedicated to improving its quality. We have carefully addressed the questions and suggestions you raised, and have incorporated these valuable insights into the revised version, which we believe will further enhance the completeness of the manuscript and the generalizability of its conclusions. Noting that the discussion phase is approaching its end (11/26, AOE), we would be more than happy to engage in further discussion should you have any additional suggestions or questions.

Best regards.

Dear Reviewer k9DA, we sincerely thank you for your valuable feedback on our submission. Below is our responses to the concerns you raised. We have incorporated these contents into the updated version of our paper, which we believe will help enhance the quality of our submission.

Since this page cannot display images, we have provided a PDF version via an anonymous link for a clearer and more complete presentation of our response: https://anonymous.4open.science/r/Respones-to-Reviewer-k9DA-4842/response%20to%20reviewer%20k9DA.pdf. This can also be find in the Supplementary Material. We recommend viewing the PDF in the provided link for easier reading.

The motivation is not enough clear. Why the previous method cannot perform well in both accuracy, convergence, and scalability?

Previous methods often focus on a single optimization objective (lines 64–65, Table 1) while neglecting the optimization of other attributes. In the related work section (lines 107–118), we specifically explain that the Random, Arena, and AlpacaEval methods each focus solely on optimizing accuracy, convergence, or scalability, respectively. Furthermore, we demonstrate in Table 2 through experiments that these three methods result in unbalanced sampling across multiple dimensions due to their failure to simultaneously consider all three optimization objectives. This ultimately leads to their inability to perform well across all three attributes simultaneously.

The runtime of the previous CBE method ($O(NMM)$) is one of the major limitations, and the author starts from this limitation as one of the motivations for the proposed method. However, they lack the runtime analysis for the UniCBE but only an approximate number for saving time when compared to the previous method.

We fully agree with your suggestion to include an analysis of UniCBE's runtime. To address this, we provide the following statistics: when a win rate error $\Delta$ of less than 0.02 is required, UniCBE needs a preference budget of $T = 2800$ (compared to $T = 3200$ for Random). Similarly, when $\Delta$ is less than 0.01, UniCBE requires $T = 9400$ (compared to $T = 11300$ for Random). Additionally, the computational complexity of $O(NMM)$ is $N \cdot M \cdot (M - 1) / 2 = 84525$. Therefore, under specific evaluation accuracy requirements, UniCBE significantly reduces the preference budget needed.

While UniCBE shows promising results for scenarios with periodically introduced new models, it may be less efficient in highly dynamic, real-time evaluation settings where new models or samples are constantly introduced at high frequencies.

We believe that testing UniCBE in a highly dynamic, real-time evaluation setting can help us more comprehensively assess its performance. To this end, we conduct the following experiments: Starting with a sample size of $N=600$ and model number of $M=12$, we execute a random operation at each time step. The operations included: adding one model to be evaluated with a probability of 0.01, removing one model with a probability of 0.01, adding one potential sample with a probability of 0.01, randomly deleting one sample with a probability of 0.01, and taking no action with a probability of 0.96. Based on the experimental results shown in Figure 9, we have the following observations:

• The convergence speed of all baseline methods significantly slowed down. None of the baseline methods achieve a Spearman correlation coefficient of 0.96 or a Pearson correlation coefficient of 0.97 by $T=2000$, highlighting the difficulty of model evaluation in this setting. In contrast, UniCBE achieve rapid convergence, reaching a Spearman coefficient of approximately 0.97 and a Pearson coefficient exceeding 0.98 by $T=2000$.
• Over the long term, as $T$ increases, UniCBE consistently demonstrates over 10% savings in preference budget across all metrics, even under this challenging setting, showcasing its strong practicality.
• An interesting observation is that AlpacaEval exhibits better convergence in the early stages compared to Random and Arena, supporting our previous conclusions in Table 1. However, as $T$ increases, AlpacaEval's lack of accuracy optimization objective leads to its performance being surpassed by Random and Arena.

How does UniCBE perform when preference signals are less reliable, as is often the case with models lower than GPT-4 or inconsistent human annotations?

As shown in Figure 6, in addition to GPT-4o and humans, we also conduct experiments using GPT-3.5-turbo as the judge, whose preference signals are less reliable. The experiments (lines 463–466) demonstrate a noticeable decline in the performance of all methods (particularly, the Arena method performs almost on par with random sampling), which is likely due to the increased noise in the preferences provided by GPT-3.5-turbo, leading to slower convergence. In comparison, UniCBE still achieves over a 15% preference budget savings relative to random sampling.

To what extent could the uniformity constraints in the sampling matrices be relaxed while maintaining cost-effectiveness?

Based on our analyses in Section 3, the degree to which uniformity is achieved is positively correlated with performance in terms of accuracy, convergence, and scalability. To explore the empirical relationship between the degree of uniformity constraints and the final outcomes, we draw inspiration from the concept of temperature-based control in random sampling. By adjusting the temperature $T$ in the following formula for sampling $f^{ts}_T$, we regulate the extent of uniformity constraints:

$f^{ts}_T(i,j,k)$

$=\frac{(P^l_{i,j,k})^{-T}}{\sum (P^{l})^{-T}}$

As $T$ increases, the uniformity constraints become more relaxed. When $T=0$, it corresponds to greedy sampling $f^{ts}_g$, which imposes the strictest uniformity constraints. When $T=1$, it corresponds to probabilistic sampling $f^{ts}_p$, which imposes general uniformity constraints. When $T=+\infty$, it corresponds to random sampling, where no uniformity constraints are applied. Our experimental results are shown in Figure 10. As $T$ increases from 0 to $+\infty$, the evaluation results progressively deteriorate. This indicates that adopting greedy sampling to impose the strictest uniformity constraints yields the optimal evaluation performance. This observation also validates the correctness of our conclusions in Section 3.

Dear Reviewer D5Wd, we sincerely thank you for your valuable feedback on our submission. Below is our responses to the concerns you raised. We have incorporated these contents into the updated version of our paper, which we believe will help enhance the quality of our submission.

Since this page cannot display images, we have provided a PDF version via an anonymous link for a clearer and more complete presentation of our response: https://anonymous.4open.science/r/Respones-to-Reviewer-D5Wd-2AC1/response%20to%20reviewer%20D5Wd.pdf. This can also be find in the Supplementary Material. We recommend viewing the PDF in the provided link for easier reading.

The framework assumes preference signals (particularly from automated judges like GPT-4) are consistent with human judgment, a potentially risky simplification given known limitations in automated preference evaluations.

We understand your concern about the existence of bias between preference signals from models and humans. In fact, we don't think the preference signals from LLMs are consistent with human judgment . We experiment with both LLMs and human as judges for the following two reasons:

• First, high-quality benchmarks that align human supervisory signals at both the sample and model levels in CBE scenario are relatively limited, so we focus on evaluating the effectiveness of UniCBE under human preference signals using MT-Bench.
• Second, as an increasing number of studies (e.g., AlpacaEval) begin to adopt LLMs as judges, it is also important to validate the effectiveness of UniCBE under model preference signals.

Experimental results show that UniCBE demonstrates superior performance when using GPT-4o, GPT-3.5-turbo, and humans as judges, confirming the robustness of UniCBE to different sources of preference signals.

The formulation of multi-dimensional sampling matrices and their interaction in optimizing accuracy, convergence, and scalability may be overly complex for practical implementations and difficult to interpret for further tuning or adjustment. Could the authors elaborate on how UniCBE would handle scenarios with dynamic preference priorities, where, for example, accuracy is weighted more heavily than convergence?

In the original manuscript, we integrate sampling matrices targeting different optimization objectives with equal weights: $P^{l} = \frac{P^{acc\text{-}l} \circ P^{con\text{-}l} \circ P^{sca\text{-}l}}{\sum (P^{acc\text{-}l} \circ P^{con\text{-}l} \circ P^{sca\text{-}l})}$ In practice, when faced with varying requirements, it is straightforward to prioritize a specific objective by adjusting the weights $\theta_{acc}$, $\theta_{con}$, and $\theta_{sca}$ for these matrices, as shown below: $P^{l} = \frac{(P^{acc\text{-}l})^{\theta_{acc}} \circ (P^{con\text{-}l})^{\theta_{con}} \circ (P^{sca\text{-}l})^{\theta_{sca}} }{\sum ((P^{acc\text{-}l})^{\theta_{acc}} \circ (P^{con\text{-}l})^{\theta_{con}} \circ (P^{sca\text{-}l})^{\theta_{sca}})}$ As demonstrated in Table below, we set different settings and calculate the degree of achievement level for each optimization objective $\beta$ following the calculation procedure described in Appendix-E. Compared to equal-weight integration, users can easily increase the corresponding $\beta$ (e.g., $\beta_{acc}$) by assigning a larger weight to a specific optimization objective ($\theta_{acc}$), thereby better meeting their practical needs (accuracy). We also observe that enhancing a specific optimization objective often comes with a slight decrease in the achievement of other objectives. In Figure 11, we illustrate an example of improving accuracy, where $\theta_{acc}$ is increased from 1 to 2. We find that the increased focus on accuracy objective slightly slows down the convergence speed. As a result, when $T$ is relatively small, the performance of $\theta_{acc} = 2$ lags behind that of $\theta_{acc} = 1$. However, in the later stages, after convergence, the enhanced accuracy objective enables $\theta_{acc} = 2$ to outperform $\theta_{acc} = 1$, resulting in greater savings in the preference budget.