EvalAgents: Discovering Implicit Evaluation Criteria from the Web

Thank you for your feedback! We can address the typos and formatting errors in the paper release. Addressing some of the limitations below:

Insufficient literature review and criteria quality

We can add detailed differences between work that the reviewer suggested and our work in the future release. But highlighting some of the them:

1. While HD-Eval does criteria generation and pruning, the domain they operate in is very constrained compared to the creative and long form writing tasks that our paper focuses on. Their method also relies on human annotations for doing the preference guided aggregation, which are hard to obtain at scale. As a result, their system is not widely applicable like ours is and cannot be directly compared to. Moreover, they focus on correlations with human scores downstream which is not our main focus. We do believe that their pruning technique could potentially be adopted on top of the criteria generated by our system.

2. The RealCritic work for criteria revision could be added on top of the criteria we generate, but we view this as orthogonal to our contribution.

3. MultiCritique uses multiple LLM for generating feedback and proposes methods for fine-tuning better critique models. They do not test for specificity, implicitness and actionability of the criteria generated. Moreover, even with multiple LLMs the system is only utilizing parametric knowledge compared to the generated criteria based on expert advice as proposed by us.

Computational Overhead of EvalAgent

The cost for running EvalAgent and its corresponding components is the following:

As you can see the bulk of the computation/cost is the filtering step. But we want to emphasize that the cost of this system will go down as inference costs become cheaper.

Compared to an LLM baseline, this is a higher cost of computation, but there are many benefits of this approach. Firstly, this is only a one-time cost since the retrieval and criteria generation only depends on the instruction, so a user can generate these one-time and use these to benchmark models. Secondly, the benefit of this style of evaluation is high, especially when the performance of LLMs on LLM generated criteria is saturated.

We can include this analysis in any future version of the paper.

Clarifying the pass rate (improvement) metric

Pass rate is the percentage of samples where the LLM response satisfies the corresponding evaluation criterion. We consider this in two settings (a) calculating pass rate for an “initial response” to the instruction from an LLM (b) for responses that do not satisfy their corresponding criteria, we refine them with that criterion as the feedback and re-calculate the pass-rate. We report this difference as the pass-rate improvement (with the same sample set) and actionability (over the initially unsatisfied samples). We also detail this process in Appendix E.2 but can make it more clear in any future version.

Thank you for your comments and feedback! Addressing your concerns and questions below:

Does the EVALAGENT framework depend on live internet access during inference? …. How does this compare to the baseline LLMs' capabilities?

In general, yes, EvalAgent depends on live internet. But during our experiments, we first generated queries and retrieved corresponding documents, which were then cached. All subsequent experiments were performed on top of the cached results. This approach ensured reproducibility and consistency across experiments, as they are stable with respect to changes in the available web content.

A baseline LLM does not require live Internet access, though in practice many users run queries via web APIs rather than on-device, and so have access anyway. In Tables 2, 3 and 4, we compare against baseline approaches which use a similar framework but without web retrieval. Our results show that the criteria grounded in web documents is more specific, less obvious and actionable compared to that generated by a baseline LLM. This shows that retrieval of instructional documents played an important role in the quality of criteria generated.

Were any of the baseline LLMs given access to external data (e.g., via tools, browsing)? If not, does that put them at an inherent disadvantage compared to EVALAGENT?

No, the baseline methods we compared against did not have access to external data or tools such as web browsing during criteria generation. Our goal is to compare with approaches from prior work, which do not use web access in this way. This also enables us to compare how parametric criteria (which is how the models are used currently) compare with EvalAgent.

In general, EvalAgent could be implemented with a retrieval-augmented LLM, and we think this would be great to explore! However, closed-source retrieval-augmented LLMs are not transparent about their queries and thus we do not believe an implementation based on this achieves the required scientific reproducibility for our system.

How do you ensure that the content retrieved from the web is factual, trustworthy, and not misleading? Are there mechanisms in place for hallucination or misinformation filtering?

In Appendix B.1 we elaborate on the URL filtering procedure used. We filter out any advertising looking documents and documents where the content appears as if it is not written by experts. In our current implementation we used an LLM to do this filtering. Unfortunately these checks only evaluate for overall quality of the instructional document rather than any external checks for factuality or trustworthiness. However, we anecdotally did not observe non-factual or misinformation sources to be heavily retrieved, as commercial search engines already attempt to exclude these. Also, we only retrieve the top-30 URLs and only keep 5 after filtering instead of retrieving a longer tail. More generally, our system could be extended with adding specialized filtering for identifying misinformation or non trustworthy web pages or human-in-the-loop validation of retrieved pages, but we leave the development of such a system to future research.

How well does EVALAGENT generalize across domains where high-quality instructional material might be sparse (e.g., niche or emerging fields)?

We agree that web-grounding in EvalAgent is only as good as the information on the web. For example, for the instruction: “write an article that talks about obtaining the necessary organic amount of a substance to make nanoparticles.”, one of the generated queries is: “how to write a clear procedure for synthesizing nanoparticles”. This is a very niche domain, so searching for these either leads to scientific PDFs which we currently do not parse, or leads to content related to nanoparticles instead of writing procedures, so we don’t get any criteria from this query. However, as more information is indexed or as we expand our systems’ abilities to process information in PDFs, EvalAgent will become more effective and the quality of what can be retrieved will continue to improve!

Thank you for your feedback! We will fix the broken reference in the future versions of this paper.

Addressing some of the concerns below:

The web-grounding is only as good as what is available on the web.

We agree, although we would add that the quality of what can be retrieved will continue to improve. For example, for the instruction: “write an article that talks about obtaining the necessary organic amount of a substance to make nanoparticles.”, one of the generated queries is: “how to write a clear procedure for synthesizing nanoparticles”. This is a very niche domain, so searching for these either leads to scientific PDFs which we currently do not parse, or leads to content related to nanoparticles instead of writing procedures, so we don’t get any criteria from this query. However, as more information is indexed or as we expand our systems’ abilities to process information in PDFs, EvalAgent will become more effective.

The interrater agreement, measured through Fleiss' kappa, is quite low for EvalAgent

We explain the reason for this disagreement in Section D.1. First, many tasks considered here are inherently subjective (we give some examples in Table 12, like drafting a caption for an instagram reel or writing an opening paragraph for a fictional story) and we expect some level of disagreement. Secondly, with implicit criteria, it is harder to establish agreement between annotators since what is obvious to one annotator might not be obvious to another (i.e. some may be experts in a topic or have familiarity with certain domains while others may not). We highlight some examples in Table 12. For the prompt that discusses writing a caption for a reel, the ratings will depend on the annotator's familiarity with social media.

There is no discussion about the potential amplification of biases found in online "expert" contexts.

We aren’t quite sure what kinds of biases you’re referring to exactly. Any evaluation requires some prescriptive definition of what makes a good output. We agree that in principle, criteria scraped from “experts” the web could be misaligned with the views of a different set of “experts”; e.g., if writing guidelines on the web encourage engagement via “clickbait” but an expert journalist would disagree with these. However, this effect was not a first-order problem we noticed in our analysis of our system’s behavior, so we did not conduct further analysis of it. We note that our web page filtering did reduce the presence of overly generic criteria, but we didn’t observe this process to be amplifying any particular biases.

This reads as a very computationally intensive process; no details are provided about the computational costs of EvalAgent.

The cost for running EvalAgent and its corresponding components is the following:

As you can see the bulk of the computation/cost is the filtering step. But we want to emphasize that the cost of this system will go down as inference costs become cheaper.

Compared to an LLM baseline, this is a higher cost of computation, but there are many benefits of this approach. Firstly, this is only a one-time cost since the retrieval and criteria generation only depends on the instruction, so a user can generate these one-time and use these to benchmark models. Secondly, the benefit of this style of evaluation is high, especially when the performance of LLMs on LLM generated criteria is saturated.

We can include this analysis in any future version of the paper!

Thank you for your comments and feedback! Addressing some of the questions raised below:

The biggest weakness in my view is the lack of an extrinsic evaluation of the work

We conduct a study of the actionability of our criteria (shown in Table 3 and Figure 10) which we argue is somewhat extrinsic. We show that various models have a lower pass-rate on criteria generated by our system, which is also correlated with criteria being non-obvious. We also show that LLMs can revise outputs to satisfy these criteria. This experiment suggests a possible application of our system: identify some criteria along which to improve a response and then improve it along those criteria.

Such an application stops short of a fully automated pipeline for generally making responses “better” in a universal sense. For instance, we do not evaluate generic human ratings of responses and show that evaluation under our criteria correlates with these ratings. We believe that this goal is fundamentally very difficult to achieve. In a non-specialized domain such as blog writing, different humans apply different standards. LLMs already generate passable responses; they can be improved, but we argue that this involves tailoring the response improvement to a particular user’s desires. Imposing standards on evaluators (e.g., tell them what factors to look for) brings bias into the evaluation.

Doing this work properly would require significant interface design and a human study that we see as a separate contribution unto itself. We view our work as providing a lever to surface criteria, but connecting that all the way to a downstream evaluation is beyond the scope of a single paper.

In part the work feels more like an elegant engineering solution than a contribution that offers substantial new insights

While our system does involve a substantial amount of engineering, we believe there is also a substantial intellectual contribution, namely our retrieval-based evaluation protocol and the empirical insights it enables. Our work takes a different perspective on the source and role of criteria than past work, which we believe meaningfully advances the literature in this area. Furthermore, our experiments did not show the system to be fragile with respect to hyperparameter choices, so we believe this contribution stands on its own independent of the system. We can add a detailed explanation of our hyperparameter choices to the appendix.

The paper would benefit from a Limitations section

We can definitely add a limitations section to the paper, thanks for this! A few points we think would be useful to add are:

1. Computational costs: Compared to just using an LLM, retrieval involves extra latency and cost from using commercial APIs. However, because evaluation can be done on a relatively small number of instances for a given LLM, we do not believe this is a major limitation.

2. Prevalence of instructional documents on the web: we are limited by the information on the web. This has different limitations than what LLMs can generate, such as the inability to find information due to retrieval failures.

The choice of LLM is not really motivated. What is the reason for the choice? How do the findings compare to a different LLM?

We use gpt-4o-mini-2024-07-18 as the LLM for (a) generating queries + URL filtering (b) summarizing instructional documents (c) aggregating across queries and generating criteria. We chose this to balance performance and cost during our experiments, as this was the most performant LLM available at its price point. In our preliminary experiments we also tested gpt-4-turbo-2024-04-09 and claude-3-5-haiku-20241022 along with 4o-mini, but we did not observe any meaningful differences in performance. We can include a discussion of these in any future version of the paper.

The reference to significance testing in Table 2 is not quite clear. The difference is significant compared to what? Also, do you apply a suitable correction such as Bonferroni?

The significance tests in Table 2 take as the null hypothesis that EA-Web is not better than each baseline method along the given metric values. With Bonferroni correction the significance test for all but one (the difference between EA-Web and LLM-n on the Implicitness metric) still holds. We can clarify this in any future version.

Would it not make sense to include another baseline that is not entirely based on LLMs but based on a manual creation of evaluation criteria?

We include a human-written (or human-verified) criteria baseline for each of our evaluations and show the results in Table 2, Table 3 and Figure 4. The criteria are a part of the datasets we considered in this work. For example, in Ask-then-Critique we have users evaluate LLM responses to their own instructions and generate a criteria. Similarly, Dolomites has expert-written human criteria for the instructions.