Active Evaluation Acquisition for Efficient LLM Benchmarking

Thank you for the review and insightful feedback. We appreciate the recognition of the well-written and structured nature of our paper, as well as the appropriate selection of baselines and the effort put into addressing the cold start problem. We will address the weaknesses and questions raised in this rebuttal.

Let us clarify the core problem and approach upfront:

The primary goal of our work is to improve the efficiency and reduce the substantial computational cost associated with evaluating LLMs on comprehensive benchmarks comprising thousands or millions of prompts.

Our approach tackles this challenge through two key components: 1) Subset selection, where we actively select the most informative subset of prompts for evaluation, and 2) Performance prediction, where we leverage dependencies across prompts to predict the LLM's performance on the remaining, unselected prompts without running actual inference on those prompts.

To give a concrete example, let's say a benchmark has $N$ prompts. The subset selection policies (including random, clustering-based, uncertainty-based, and RL-based policies) will select $K$ prompts based on their corresponding strategies. Then we will run LLM inference on these $K$ prompts to acquire the actual evaluation scores. Afterwards, the evaluation scores on the remaining $N-K$ prompts will be predicted by the neural process model.

By combining these two components, our method aims to provide reliable estimates of an LLM's overall benchmark performance while minimizing the number of prompts that require expensive LLM inference. This enables significant computational savings and reduces the associated costs.

Here are detailed responses to the questions you raised:

I am having some trouble understanding why choosing samples with less confidence in evaluation will result in a rigorous enough evaluation

The high uncertainty of the prediction model means the performance prediction on these prompts is not accurate. By selecting these prompts to acquire their actual evaluation scores, the overall estimation error will be lower. This is the same principle as in active learning literature, where the uncertain examples based on the current model are selected to further train the model.

clarify the objective of your RL-based method

The objective of the RL-based policy is the same as other selection policies, that is, to select informative examples. The RL agent does so by maximizing the accumulated reward, which is defined as how accurately we can recover the benchmark performance using the selected prompts (see line 242). We will clarify the RL objective in the camera-ready version.

include more analysis on why RL achieves near 0% absolute error on AlpacaEval. Also, only AlpacaEval achieves such results - the other datasets have an error of ~2%. Including a comparative analysis on the performance of AlpacaEval versus the other datasets can help readers understand the discrepancy in performance.

One potential reason for the exceptional performance on AlpacaEval could be the relatively smaller size of the benchmark compared to others. With fewer prompts, the dependencies across prompts may be easier to capture, leading to more accurate predictions.

Additionally, the scores on AlpacaEval 2.0 are computed by a logistic regression model that predicts the win rate from a GPT4 evaluator. These scores are essentially smoother than the typical accuracy scores in other leaderboards.

We will conduct a detailed analysis to investigate the factors that contribute to the varying performance across different benchmarks, such as the benchmark size, prompt diversity, and the complexity of the underlying tasks. This analysis will provide valuable insights and help us better understand the strengths and limitations of our approach in different evaluation contexts.

Could you motivate the use of an RL agent to estimate the value of a data point? Why are you using an RL agent to calculate the model uncertainty? While the formulation makes sense, a few more baselines would strengthen the motivation. Baselines can include: (1) self-consistency (sample the model multiple times for a response, if the response variety is large, classify the data sample as "not confident") or (2) model feedback (use the distribution over the tokens or perplexity to determine confidence).

The RL agent is used to select K prompts out of the total N prompts to acquire their actual evaluation scores. The evaluation scores on the remaining N-K prompts will be predicted by the neural process model. For the RL-based policy, uncertainty on the unselected prompts is implicitly handled by the policy. That is, the RL policy implicitly estimate the uncertainty of the available prompts and choose the one that maximize the long-term accumulated reward. This is in contrast to a myopic greedy policy that always select the most uncertain one.

Thank you again for the review and feedback. We hope that with the clarifications and improvements outlined above, you would consider increasing the rating for our paper.

Thank you for the clarifications in the first comment - all those are resolved on my end.

Regarding your second comment:

1. "AlpacaEval could be the relatively smaller size", "scores on AlpacaEval 2.0 are computed by a logistic regression" - understood, resolved.

2. "We will conduct a detailed analysis to investigate" - could you please show some analysis for the review? It can be preliminary, but it will be hard to be convinced without some analysis.

3. For my question "Why are you using an RL agent to calculate the model uncertainty?", I moreso meant why are you using an RL agent versus other common techniques to measure uncertainty? With your paper and above clarifications, I understand that using an RL agent saves inference computation and is cheaper. My main doubt is: does it perform better than inference-dependent methods? Hence, I suggested two baselines (self-consistency, and model feedback) that can help solidify your motivation for using an RL agent.

4. If you could also clarify whether there are any recent "Active Testing" works with language models (Question 1 in my review), that would be helpful too.

Uncertainty estimation baselines

Regarding your question about comparison with typical LLM uncertainty estimation approaches. You raise a good point that comparing with these inference-dependent uncertainties can further strengthen our paper.

As a preliminary comparison, we compared with two baselines: 1) directly use the generation log-likelihood as uncertainty estimate, and 2) semantic entropy (Kuhn et al., 2023), where multiple generations are clustered based on a pretrained NLI model, and afterwards, the entropy is calculated to incorporate the semantic meanings of the clusters.

Table R.4 presents the results on three models for estimating their performance on AlpacaEval benchmark. For each model, we use either our RL policy or these uncertainty estimations to acquire the evaluation scores for a subset of prompts. The evaluation scores on the remaining prompts are estimated by the same neural process model. From the results, we can see that our RL-based acquisition policy consistently performs better than these two uncertainty estimates.

Table R.4: Comparison of our RL-based acquisition policy and uncertainty-based sampling (using generation perplexity and semantic entropy) in estimating AlpacaEval benchmark performance. We compare the absolute error with selection budget of 10, 20, 50 and 100.

Our hypothesis for why the RL approach outperforms these inference-dependent methods is that the RL policy is trained jointly with the neural process model to align its uncertainty estimates. This allows it to acquire prompts where the neural process predictions are most uncertain or inaccurate. In contrast, while the inference-dependent uncertainties may capture the model's own uncertainty well, they can be misaligned with the uncertainties of the neural process model used for prediction, resulting in overall higher benchmark estimation errors.

Furthermore, we want to emphasize that these inference-dependent uncertainty methods are not practical to deploy in reality, as they require running inference on all prompts and sometimes generating multiple responses to estimate uncertainty. This essentially increases the overall evaluation cost, defeating the purpose of efficient benchmarking.

We will add these comparison in the appendix for the camera-ready revision.

To further investigate the impact of metric types, we analyzed the prediction errors for different metric types in Tables R.3 and R.4. The prediction errors for binary evaluation scores are significantly higher than those for real-valued scores like BLEU_4 and F1_score, aligning with our hypothesis.

Table R.3: Prediction error for different metric types.

Table R.4: Prediction error for different metrics.

This preliminary analysis provides valuable insights into the factors that can influence the accuracy of predicting evaluation scores. We plan to conduct a more comprehensive investigation across multiple benchmarks and develop strategies to improve prediction accuracy, especially for challenging metric types and datasets with high variability or informativeness.

Thank you for the additional comments and questions. We appreciate the opportunity to further clarify our work and address your remaining concerns.

Preliminary analysis on performance prediction accuracy

To investigate potential factors that can impact the accuracy of predicting evaluation scores, we conducted a preliminary analysis on the HELM-Lite benchmark due to its diverse set of datasets covering different types of tasks and metrics. To eliminate any potential impact from the prompt selection policy, this analysis was performed using randomly selected 50 prompts as condition to predict the evaluation scores.

It's worth noting that prompts that are inherently difficult for language models, resulting in consistently low scores, do not necessarily translate to high prediction error for the evaluation scores. For example, if no model can solve a particular prompt, its evaluation score will always be zero, making it relatively easy to predict.

For a dataset with $N$ prompt evaluated on $M$ models from training set, we denote the evaluation scores as $S \in \mathbb{R}^{N \times M}$. We consider the following factors:

• Metric Types: Our hypothesis is that discrete metrics, such as exact match accuracy, are harder to predict compared to smooth continuous metrics like BLEU.
• Prompt Diversity: A dataset with diverse prompts is potentially harder for LLM. We estimate the diversity using the pairwise similarity of the prompt embeddings.
• Task Difficulty: We estimate the difficulty of a task as 1 minus the average evaluation scores across all models, i.e., $(1-S).mean()$. A higher value indicates a more difficult task on average.
• Score Diversity: A dataset where the score distribution has a high variance can potentially lead to higher prediction error. We calculate score diversity as $S.var(dim=0).mean()$
• Task Informativeness: We estimate the informativeness of each prompt as the variance of its evaluation scores on all models, then the task informativeness is averaged over prompts. $S.var(dim=1).mean()$.
• Evaluation Variability: Calculating the variance of the mean scores across prompts is another way to quantify the variability or diversity in the task. $S.mean(dim=0).var()$

Please see Table R.1 for detailed results on 28 subsets from HELM-Lite. In Table R.2, we calculate the correlation between each factor and the prediction error.

Table R.2: Correlation between dataset characteristics and prediction error.

We can observe that:

1. Metric Type has a high correlation with prediction error, verifying our hypothesis. We further analyzed the differences across metric types in Table R.3 and Table R.4.
2. Prompt Diversity has a high negative correlation with the prediction error. Although counterintuitive at first, this aligns with the analysis that harder task for LLM does not necessarily mean harder evaluation score prediction. This is further supported by the negative correlation between Task Difficulty and prediction error.
3. Score Diversity only has a moderate correlation to prediction error.
4. Task Informativeness has a high correlation with prediction error, as expected, since prompts with high variance in evaluation scores are inherently harder to predict accurately.
5. Evaluation Variability is actually an estimation of the prediction error on test set using the scores from training set, so its high correlation with prediction error is unsurprising.

As far as I understand, you need to refit your Neural Process model (or at least the posterior of $Z_m$) for every LLM m you want to evaluate. How expensive is this step? One of the main applications of efficient evaluation of LLMs is quick evaluation during pre-training and it seems that your approach would not be friendly in this case;

We do not need to refit the neural process model for a new test model. As long as we acquire a subset of evaluation scores $Y^o$ following the subset selection policy, the neural process can predict the unobserved scores following $p(Y^u | Y^o, X)$.

Why do you compare against the Cluster-IRT approach and not IRT++ (from tinyBenchmarks)? IRT++ seems a better method;

For notation clarity, we used the name Clustering-IRT, but we indeed used the strongest model recommend by tinyBenchmarks, which is IRT++. We will clarify this in the camera ready version.

It seems that some curves are missing in Fig 1, for example. Why? If they are not included on purpose, the legend should not include them.

The legend is shared by all 5 plots to avoid cluttering, but for some benchmarks, such as HELM-Lite and OpenLLM, where the number of total prompts is large, Combinatorial Optimization and Information Gain policies are too expensive to finish in a reasonable time frame. Therefore, we left them out for those benchmarks. We will clarify this in the figure caption or legend.

We hope that by addressing these weaknesses and questions, we have strengthened the clarity, rigor, and positioning of our work. Please let us know if you have any further comments or suggestions.

Regarding your specific points:

Section 2.2 can be greatly improved; it is not even clear how fitting is possible without observing labels $Y^u$. The authors cite VAE, but in VAEs the outputs are known, which makes the decoder training possible. This suggests that some assumptions about the model are not included (or clear) in the text. A more detailed explanation in the text would be appreciated (possibly including an algorithm box for model training);

During training, the evaluation scores $Y_m^u$ are available for the models in train split. We will include a training algorithm, as described above, for the camera ready version to clarify these confusions.

I am not sure including Chatbot Arena makes sense here because it is a pairwise comparison dataset and will depend on other models you're comparing against. Your NP model does not seem to accommodate that.

It’s conceptually similar to evaluation scores for a single model. Instead, now we have evaluation scores for a pair of model. For Chatbot Arena, the train split contains a set of model pairs and the win rate scores on each prompt.

The paper does not comment on IRT adaptive testing on related work (see https://arxiv.org/pdf/2404.00712, for example). This class of methods is directly related to the paper and could be considered a competitor to the RL approach. It is important to mention why you do not compare against it;

You're absolutely right. We will include a discussion of IRT adaptive testing methods in the related work section and provide a comparison to our RL-based approach, highlighting the potential advantages and limitations of both approaches.

The paper does not comment on limitations. One major limitation is the need to fit a neural net for every LLM we want to make inferences about. Moreover, as noted by tinyBenchmarks, adaptive testing can reduce the scope of applications as it can make efficient evaluation more costly;

We appreciate you highlighting these potential limitations. We will include a dedicated section in the paper to discuss them, including the potential increase in evaluation cost for certain applications, as noted by tinyBenchmarks. We will also discuss the trade-offs and scenarios where the overhead of our approach could outweigh the benefits.

However, we want to emphasize that the bottleneck for LLM evaluation is on LLM inference, as it typically requires high-end hardware or calls to expensive APIs. Compared to the LLM itself, our neural process model and the subset selection policy are much smaller in size and computation (typically 2-3 linear layers in our experiments). Therefore, the overhead introduced by our method is relatively small compared to the computational savings achieved by reducing the number of prompts that require LLM inference.

While our approach does involve additional components and training steps, the primary computational benefit stems from the significant reduction in the number of prompts that require running the resource-intensive LLM inference. In our experiments, we observed that our method could achieve accurate performance estimation while acquiring evaluation scores for only a small subset (e.g., <1% for MMLU and OpenLLM, 1.5% for HELM-Lite) of the total prompts.

Does your model have any components trained using a training set? From the experiments section, it seems so, but this is not clear from the methods section.

Yes, our method does have components trained on a training set. As described above, the neural process model and the RL-based acquisition policy are trained on the train split of the leaderboard data.

In your equation 3 RHS, shouldn't we take the expectation wrt $Y^u$ the unobserved portion of (given that we do not observe it)?

Also from eq 3, it is not clear what are the assumptions on $p(Y^u|...)$ and how it can be modeled because without observing $Y^u$;

The evaluation scores $Y^u$ are assumed available during training for the models in the train split. They will be predicted during testing for the models in the test split. We will clarify the assumptions and provide more detailed explanations in the revised manuscript.

Thank you for the review and insightful feedback. We appreciate the recognition of the novelty and potential of our approach, as well as the strong empirical results. We will address the weaknesses and questions raised in this rebuttal.

Let us clarify the approach and training procedure upfront:

Our approach consists of two key components: 1) Subset selection policy, where we actively select the most informative subset of prompts for evaluation, and 2) Performance prediction, where we leverage dependencies across prompts to predict the LLM's performance on the remaining, unselected prompts without running actual LLM inference on those prompts.

The training procedure first divide the leaderboard evaluation scores into train and test splits. The train split contains a set of models and their corresponding evaluation scores on the benchmark prompts. Specifically, for each model m in the train split, we have access to the full set of evaluation scores Y_m = {y_{mn}}, where n indexes over all N prompts in the benchmark. The test split contains another set of (non-overlapping) models and evaluation scores.

During training, we utilize this train split to learn the neural process model and the subset selection policies:

For the neural process model:

• For each model m, we randomly sample two non-overlapping subsets from Y_m, treating one as the observed set Y_m^o and the other as the unobserved set Y_m^u.
• The neural process is trained to optimize the ELBO objective, learning to predict the unobserved scores Y_m^u based on the observed scores Y_m^o, i.e. p(Y_m^u \mid Y_m^o, X). The ELBO is feasible to compute since train split have all scores available.
• After training, the neural process model can predict the unobserved scores given an arbitrary subset of observed scores. It can generalize even to unseen models. Therefore, it does not require retraining on the test split.

For the subset selection policies:

• The random policy does not have trainable parameters, as prompts are selected at random
• The Clustering-Embed policy does not have trainable parameters either. We simply use the prompt embedding to group the prompts.
• The Clustering-Score policy directly use the evaluation scores for a prompt from the train split as its feature representation.
• The Clustering-IRT policy trains an IRT model on train split, then we cluster the prompts based on the learned prompt embeddings from IRT models.
• The Combinatorial Optimization policy, Uncertainty Sampling policy and Information Gain policy do not have trainable parameters either. They rely on the neural process model to predict the evaluation scores and their uncertainties.
• The RL-based policy trains a selection policy on train split by interactively select which prompt to acquire its evaluation score.

During testing, the subset selection policy selects a subset of prompts, whose actual evaluation scores will be acquired. Therefore, the actual evaluation scores are used for those prompts. While the evaluation scores for other prompts will be predicted by the neural process model. Since we have the gold evaluation scores for all prompts, we can compute the metrics by comparing the predicted scores to the gold scores.

When the system is actually deployed, for a new model to be evaluated, the subset selection policy will select a subset of prompts, and we run the LLM inference on these prompts to get the real evaluation scores. The evaluation scores on the remaining prompts will be predicted by the neural process model.

Thank you for all of your comments. You have a good paper but that needs more polishing, especially in the writing. I will increase my score, but please implement all the promised changes before the paper is published.

3. Apply the Method at the Design Stage

You're correct that it is unclear how a benchmark designer would implement our method without already having a large amount of benchmark results to draw on. However, this is not the primary setting we target with our approach.

Our method is designed to enable efficient evaluation of large language models on existing, well-established benchmarks that have already accumulated a substantial amount of evaluation data from various models. In such scenarios, our approach can leverage this prior evaluation data to learn the dependencies across prompts and develop effective acquisition policies, thereby reducing the computational cost of evaluating new models on the benchmark.

That being said, if one were to apply our method to a completely new benchmark with no prior evaluation data, we acknowledge that it would be challenging to get the method "off the ground." In such cases, we propose a two-stage approach:

Stage 1 (Cold Start): During the initial design phase, when no evaluation data is available, we can rely on simpler acquisition policies like random sampling or clustering based on prompt embeddings. These policies do not require any prior evaluation data and can be used to acquire an initial set of evaluation scores for a small number of seed models. As for the benchmark performance estimation, one can directly estimate it based on the scores on the selected prompts (corresponds to the "w/o pred" setting in Table 2). This will result in higher estimation error, but depending on the budget one can allocate, it provides a reasonable starting point.

Stage 2 (Warm Start): Once we have acquired some initial evaluation data from Stage 1, we can then train our neural process model and RL-based acquisition policy on this seed data. From this point onwards, our method can be used to actively acquire evaluation scores for new models more efficiently, leveraging the learned dependencies and acquisition policies. Note that both our neural process model and acquisition policy can be trained with partial available scores, thus the collected scores in Stage 1 can be effectively used.

We will include a detailed discussion of this two-stage approach in the paper. While our method may not be immediately applicable to brand-new benchmarks without any prior data, the two-stage approach offers a practical solution to bootstrap the process and subsequently benefit from the computational efficiency of our method as more evaluation data becomes available.

4. Handling New Models in an Existing Benchmark

As our results indicate, our approach can reasonably generalize to new models, even those that may exhibit different behavior compared to the models used for training. This generalization capability is demonstrated in our experiments where we split the models into older and newer groups (based on submission dates), train our method on the older group, and evaluate its performance on the newer, unseen group of models. We also tested generalization capability by dividing the models based on organizations (See Fig 2).

However, we recognize that if the new models introduced to the benchmark exhibit behavior that is significantly different from all previously seen models, it is possible that our method may give higher estimation errors. Dealing with such out-of-distribution scenarios is an interesting and important direction for future work, not only for our method but also for the broader machine learning community.

If significant distribution shifts are observed as new models are added to the benchmark, one potential approach could be to periodically retrain our method, incorporating the evaluation data from the new models. This would help our method adapt and capture the evolving dependencies across prompts and model behaviors.

Alternatively, we could explore techniques from continual learning or lifelong learning, which aim to enable models to adapt to new data while preserving the knowledge acquired from previous data. By incorporating such techniques, our method may be able to update its understanding of prompt dependencies and acquisition policies in an incremental manner, without the need for complete retraining from scratch.

While addressing out-of-distribution scenarios and distribution shifts is an important area for future research, we believe that our current work provides a solid foundation and demonstrates the potential of our approach in enabling efficient LLM evaluation within the scope of the existing benchmark data. We will include a discussion of these limitations and future directions in the paper, highlighting the need for further research in this area.

We deeply appreciate you taking the time to provide such detailed and insightful feedback. We believe that by addressing these concerns, we can significantly strengthen the contributions and practical impact of our work. We hope that with these improvements, you would reconsider your rating and advocate for the publication of our paper.

1. Evaluation on New Models (Out-of-Distribution)

You are correct that evaluating the interpolation error on existing benchmark results may not fully capture the performance on new models that could potentially be out-of-distribution compared to the models used for training. This is a valid concern, and we have already considered it in our evaluation methodology.

Note that our data splitting strategy (lines 360-371) is based on models, not prompts. Our train and test sets contain non-overlapping sets of models to assess how our methods generalize to new models. Due to space limitations, the data split details are moved to Appendix E. We will move them to the main text for the camera-ready version.

For the OpenLLM Leaderboard and MMLU benchmarks, we leveraged the natural split based on the submission dates of the models, dividing them into older and newer groups accordingly. We evaluate our method's performance in accurately estimating the benchmark scores for the newer group of models after training on the older group. This evaluation setup directly assesses our approach's ability to generalize to unseen, potentially out-of-distribution models.

For benchmarks like HELM-Lite and AlpacaEval, where submission dates were not available, we conduct ablation studies by dividing the models based on their organizations. For HELM-Lite, we use proprietary models, such as GPT-4 and Claude, for training and test on open-source models, such as LLaMA and Mistral. For AlpacaEval, we do the opposite, using open-source models for training and proprietary models for testing. Please see details in lines 447-464 and results in Figure 2.

Overall, our RL-based acquisition policy generalizes better to the out-of-distribution (OOD) models compared to other baselines. We appreciate you highlighting this important aspect of our work, and we will ensure that our evaluation methodology and results are clearly described in the paper to address your concern.

2. Ablation Studies on Moving Pieces

You're correct that our method involves several components, including prompt embeddings, subset selection policies, and neural process approximation, which could potentially introduce confounding sources of error. To address the potential confounding sources of error, we have performed the following ablation studies:

1. Prompt Embeddings: We have evaluated the impact of using different prompt embedding models, ranging from large pre-trained models like SFR and E5 to smaller models like BGE-large and BGE-small. Our results, presented in Table 3, demonstrate that our method's performance generally improves with more powerful embedding models that can better distinguish text inputs. Based on these findings, we recommend using the most powerful embedding model available for the best performance.

2. Subset Selection Policies: We have extensively studied and compared various subset selection policies, including random policies (uniform sampling and stratified random sampling), static policies (clustering-based on embeddings, scores, and IRT features), and our proposed dynamic RL-based policy. Our empirical results, shown in Figure 1, clearly indicate that the RL-based policy outperforms other baselines across multiple benchmarks, demonstrating its effectiveness in selecting informative prompts.

3. Neural Process Approximation: To assess the impact of the neural process approximation component, we have compared our proposed method, which leverages the neural process to predict evaluation scores for unselected prompts, with a baseline approach that directly aggregates the acquired evaluation scores without any prediction. The results, presented in Table 2, show that the neural process generally provides better benchmark performance estimation, highlighting the importance of this component in our method. Furthermore, In Table 1, we compare our neural process model and the IRT model proposed in tinybenchmarks, using the prompt selections from our RL-based policy with those from tinybenchmarks. The results demonstrate that for both prompt selections, using our neural process produces better benchmark performance estimates, indicating that our model better captures the dependencies and predicts the missing evaluation scores more accurately.

4. Technical novelty and comparison with active learning:

While our work shares similarities with active learning in the sense that we aim to select the most informative examples, there are key distinctions in terms of the objectives and settings.

In active learning, the goal is to select a subset of examples to label and then use them to train a better model. The objective is to improve the model's performance on the task at hand by acquiring useful training examples. In contrast, our work focuses on selecting a subset of prompts for efficient evaluation of an already-trained LLM, without any further fine-tuning or adaptation. The objective is to accurately estimate the model's overall performance on a comprehensive benchmark while minimizing the number of prompts required for evaluation.

Additionally, in active learning, the labeled examples are directly used for training, and the model's performance is evaluated on a separate test set. In our setting, the acquired evaluation scores are used to predict the scores for the remaining prompts, and the final performance estimate is derived from both the acquired and predicted scores.

While we do leverage some techniques from active learning, such as uncertainty sampling and information gain, our primary contribution lies in adapting these methods to the specific context of LLM evaluation and developing novel acquisition policies tailored to this setting. For instance, our RL-based acquisition policy (see equation 7) incorporates auxiliary information from the generative model and intermediate rewards to guide the exploration of the prompt space more effectively. Furthermore, we introduce the cold-start problem, where new prompts are added to the benchmark without any prior evaluation scores, and propose techniques to handle this scenario.

We will clarify the distinctions between our work and active learning in the paper and provide a more detailed comparison to highlight our specific technical contributions in the context of efficient LLM evaluation.

5. Others:

The writing needs more improvement. For example in lines 86-87, the subset $o^*$ has cardinality $K$ but is indexed from $1, \ldots, N$. Is $K < N$ Do you mean to say that there are some repetitive prompts?

The optimal subset $o^*$ is a subset of $\{1, . . . , N\}$ with cardinality $K$, where $K \leq N$. There are no repetitive prompts; instead, we select $K$ unique prompts from the set of $N$ prompts in the benchmark.

Thank you once again for the insightful review and constructive feedback. We appreciate the opportunity to clarify and address the raised issues, which will undoubtedly strengthen the paper's clarity and positioning. If you have any further suggestions or comments, please let us know.

Thank you for the detailed review and insightful questions. We will do our best to address the issues and questions raised. We believe this is an important setting for the LLM evaluation community, as it can significantly reduce the evaluation cost, which is a critical challenge given the rapidly increasing scale and complexity of LLM benchmarks. We would greatly appreciate if you could advocate for publication if we have adequately addressed your questions and concerns.

1. Comparison with few-shot learning:

You raise a valid point about the connection between our work and few-shot learning with LLMs. While there are similarities in the sense that both involve selecting a subset of examples, the key difference lies in the objective. Few-shot learning aims to select examples that can effectively adapt a pre-trained LLM to a new task or domain, with the goal of improving the model's performance on that specific task. In contrast, our work focuses on selecting a subset of prompts for efficiently evaluating an already-trained LLM across a diverse set of tasks or capabilities, without any further fine-tuning or adaptation. The objective is to accurately estimate the model's overall performance on the entire benchmark while minimizing the number of prompts required for evaluation.

Despite this difference in objectives, we acknowledge that there could be potential synergies between the two lines of work, and a more detailed discussion comparing and contrasting them would be beneficial. We will incorporate a dedicated section in the related work to establish this connection and highlight the key distinctions.

2. Significance of the motivation:

The motivation for our work stems from the increasing complexity and comprehensiveness of LLM evaluation benchmarks, which often comprise thousands or even millions of prompts spanning diverse tasks and capabilities. Evaluating an LLM on such large-scale benchmarks can incur substantial computational costs, both in terms of resources and time. For instance, as mentioned in the paper, evaluating on the HELM benchmark requires 4,200 GPU hours for a 176B model and over $9,000 in API costs. These substantial costs can hinder frequent evaluations during model development and limit extensive hyperparameter tuning during inference.

Our work aims to alleviate these challenges by reducing the number of prompts required for accurate LLM evaluation, thereby lowering the computational burden and associated costs. This is particularly important in scenarios where frequent evaluations are necessary, such as during model development or when exploring various decoding and prompting strategies.

We will strengthen the motivation section in the paper to better highlight the practical significance and potential impact of our work in enabling more efficient and cost-effective LLM evaluation.

3. Capturing task hardness in the acquisition function:

You make a good point. The current LLM evaluation benchmarks primarily focus on assessing the final answer or output quality, rather than explicitly evaluating the underlying reasoning capabilities or the solution process followed by the model.

While the reasoning requirements for different prompts may vary in difficulty, such as those involving mathematical problem-solving or multi-step logical reasoning, these complexities are effectively integrated into the final evaluation score assigned to each prompt. In other words, a model with stronger reasoning capabilities is likely to perform better and achieve higher scores on prompts that demand complex reasoning skills, as compared to models with more limited reasoning abilities. Consequently, the evaluation scores themselves implicitly capture the model's proficiency in handling prompts with varying levels of reasoning difficulty.

Our acquisition functions, by leveraging the dependencies among these evaluation scores, indirectly account for the differences in reasoning requirements across prompts. Models that excel at complex reasoning will have higher scores on the corresponding prompts, and our methods will prioritize the selection of such informative prompts during the acquisition process.

While our current approach does not explicitly model or quantify the specific reasoning types or difficulty levels, it effectively captures these nuances through the evaluation scores, which serve as a holistic measure of the model's performance on each prompt, integrating all aspects of the required reasoning capabilities. However, we acknowledge the value of your suggestion and the potential benefits of explicitly modeling and considering reasoning complexity during the acquisition process. As future work, we will explore techniques to incorporate more fine-grained representations of reasoning requirements. This could enable a more nuanced analysis and targeted evaluation of specific reasoning skills, in addition to the overall performance assessment provided by the existing benchmarks.