NoveltyBench: Evaluating Language Models for Humanlike Diversity

We thank Reviewer KaJt for the detailed review and thoughtful engagement with our work. We appreciate the recognition that diversity in LLM content is "intriguing" and "important," that our benchmark development is "well-thought-out," and that our comprehensive evaluation "adds substantial value."

1. Should models be inherently diverse?

The reviewer raises a fundamental question: "whether these models should inherently possess such diversity" given that an LLM "functions more like a single person at a given point in time." However, we argue that there are requests users regularly make to LLMs for which users expect different answers to be produced every time. User expectations of a language model and a KBQA system are meaningfully different:

• When users ask "Tell me a joke" or "Suggest a vacation destination," they often want variety across interactions. Unlike KBQA systems that should always return factually correct entities, users making these creative/subjective queries benefit from multiple valid perspectives—chatbots would not be fulfilling user demands if they tell the same joke or recommend the same vacation spots repeatedly.

• While a single human asked the same question repeatedly might give consistent answers, LLMs serve millions of users simultaneously—they should be capable of reflecting the diversity of human responses across populations. In this work, we are interested in the scenario of multiple people each asking the same question to the LLM system, rather than one person repeatedly asking the same question multiple times.

Building on recent work on pluralistic alignment [1], we believe there is merit in building language models that are distributionally diverse. Nevertheless, as follow-up research, we plan to design surveys that explore user expectations of LLM diversity and answer whether distributional diversity is actually what users want.

[1] A Roadmap to Pluralistic Alignment

2. Separate reporting across question categories

We have additionally reported results across the four categories (Randomness, Factual Knowledge, Creativity, and Subjectivity). Overall, models generally perform worse at providing underspecified factual information and slightly better at other categories like randomness. However, in none of the four categories do models provide reasonable levels of diversity and utility. We will incorporate these results in our revision.

Diversity results | Utility results

3. Metric reliability

You are correct that evaluation of the metrics relies heavily on model performance, and this is a limitation of our work. In an ideal setting, both diversity and quality of model responses would be evaluated with human annotators, but it is near-impossible to build a standardized benchmark that relies entirely on human evaluation. We see our benchmark as a first step that model trainers should take to assess their models' diversity capabilities. When possible, automatic evaluation of diversity should be followed up with user studies and other human evaluation.

To address the 79% classifier accuracy concern: while not perfect, this substantially outperforms existing similarity metrics (BLEU: 66% AUC, BERTScore: 68% AUC vs. our DeBERTa: 81% AUC, as shown in Appendix A.3). We acknowledge this limitation and suggest that future work could explore more sophisticated equivalence detection methods.

4. Statistical confidence

We have computed 95% confidence intervals for our main evaluations. Overall, the error margins are small because we evaluate over a large number of prompts (1,100 total). Here are the updated results with confidence intervals:

We will present appropriate statistical confidence measures in our revision to help readers better interpret the findings.

The authors' comments have partially solved the problems, and I increase my rating from 5 to 6.

Q1: LLaMA model variants

Answer: We evaluated the instruction-tuned variants for all LLaMA models. We will clarify this explicitly in Table 1 in our revision.

Q2: Lexical similarity metrics comparison

Answer: Before developing our classifier, we extensively experimented with off-the-shelf similarity metrics, but found they performed poorly at capturing functional equivalence. For example, two stories with identical plots but different wording would be considered "diverse" by n-gram metrics but functionally equivalent to users.

Appendix A.3 provides detailed comparisons. Our results show:

• BLEU: AUC = 0.66
• ROUGE-1: AUC = 0.68
• BERTScore: AUC = 0.68
• Our DeBERTa classifier: AUC = 0.81

While our trained classifier is far from perfect, it does outperform existing metrics (e.g., lexical similarity-based ones).

Q3: Prompt complexity/ambiguity as alternative explanation

This is an insightful observation that deserves further investigation. In Section 4.3, we partially address this through experiments with prompt paraphrasing and diversity-seeking system prompts, neither of which significantly increased diversity. However, we did not systematically study how prompt complexity affects generation diversity.

We agree this would be valuable for future work. That said, we argue that models with sufficient entropy should exhibit diversity across all reasonable prompts, regardless of specific wording - this is the type of robust distributional diversity that model developers should strive for.

We thank Reviewer SHky for the thoughtful and detailed review. We are glad that the reviewer agrees our work addresses an important problem and provides a "critical addition to the evaluation landscape." We appreciate the constructive feedback and address the concerns below:

1. Fine-grained analysis of post-training stages

This is an excellent suggestion that would provide valuable mechanistic insights into diversity degradation due to post-training. Following the reviewer's suggestion, we have added evaluations of OLMo 2 models (built upon Tulu 3) after three stages of post-training (SFT, DPO and RLVR), and analyzed diversity loss after each stage.

Diversity figure

Utility figure

Our analysis shows that each alignment stage progressively reduces model diversity, with the most significant drop occurring during supervised fine-tuning. This finding suggests that our hypothesis that current alignment procedures inadvertently suppress distributional diversity. Interestingly, in some cases, the improvement in generation quality makes up for the loss in diversity (utility gain from SFT -> DPO). Future work should investigate how to strike a balance in alignment algorithms to get the best of both worlds.

2. Expansion to reasoning/coding tasks

We appreciate this suggestion and acknowledge that our current benchmark focuses primarily on open-ended, creative tasks rather than reasoning or coding scenarios. We deliberately focused on tasks where diversity is expected and desirable - areas where multiple valid answers exist and where users would reasonably benefit from variety.

However, we agree that understanding the impact of explicit reasoning on model diversity is valuable. We have added preliminary evaluations of reasoning models (o3, o3-mini, Gemini-2.5-flash) and found interesting patterns where additional reasoning effort can sometimes reduce diversity. We plan to explore this further in future work.

3. Third-party reward model bias

This is a valid methodological concern. We chose Skywork-Reward-Gemma-2-27B because it achieved top performance on RewardBench, outperforming other reward models including NVIDIA's Nemotron-70B model. While we acknowledge that any specific choice of reward model may introduce biases, the use of a standardized reward model is necessary for benchmark reproducibility and removes dependence on expensive manual annotations.

In preliminary experiments, we also tested LLM-as-a-Judge (specifically GPT-4o), but found it produced skewed score distributions (most scores between 6-10), unlike a scalar reward distribution that is easier to calibrate as we did in our approach.

4. Dataset filtering and representation

We appreciate this concern about potential underrepresentation in NB-WILDCHAT. Our filtering process was designed to balance comprehensiveness with quality: we used Llama Guard 3 for safety filtering, IP-based deduplication to reduce topic bias, and GPT-4o classification to select prompts likely to elicit diverse responses (85% agreement with human judges).

While this process may underrepresent certain domains (technical queries, multilingual prompts), we believe it provides a reasonable approximation of diversity-seeking user intents. We acknowledge this limitation and suggest that future work could develop domain-specific diversity benchmarks to complement our general-purpose evaluation.

We thank Reviewer 3g5a for the thoughtful review and positive assessment of our work. We are pleased that the reviewer recognizes the importance of our benchmark in addressing the critical issue of mode collapse in language models. Below, we address the specific concerns raised:

1. Lack of reasoning models as baselines

We chose to focus on the 20 most representative (non-reasoning) language models from major providers to establish a comprehensive baseline across model families and sizes, since reasoning capabilities weren't generally available among model families at the time of writing. That said, we agree that as more models carry out an explicit reasoning process before responding to user queries, it would be interesting to see how such reasoning affects diversity.

We have evaluated OpenAI and Google's latest reasoning models (o3, o3-mini, Gemini-2.5-flash) for which inference-time reasoning effort can be explicitly set, and explored how they impact diversity and utility on NoveltyBench (diversity figure and utility figure). For the Gemini model, setting the reasoning budget has minimal effect on our evaluation. On the other hand, there appears to be a trend with OpenAI's models where additional reasoning leads to less diverse (and useful) outputs. Future research should further investigate the effect of reasoning on generation diversity.

2. Temperature setting influence on diversity

This is an excellent point that deserves clarification. We deliberately chose temperature=1.0 to represent the best-case scenario for diversity as stated in Section 4.1 (L209). This choice was intentionally conservative—by using the most "diverse" setting, our results represent an upper bound on the actual diversity users experience at deployment time. A single choice is also somewhat inevitable to achieve consistent comparison among models.

We note that our results suggest today's frontier language models are not sufficiently diverse even at the "ceiling" temperature of 1, and alternative, lower temperature settings (e.g., 0.3 or 0.7) would result in even less diversity among the generations.

3. Analysis of dataset diversity and distribution to avoid potential bias

We would be happy to provide additional analysis of dataset diversity and distribution. Could you elaborate on what specific kind of analysis you would like to see?