Improving Uncertainty Estimation through Semantically Diverse Language Generation

We appreciate your recognition of SDLG’s innovative use of the NLI model for mapping output sequences to semantic clusters and computing token-level contributions to semantics. We are pleased that you found our method effective, efficient, and advantageous in eliminating the need for hyperparameter tuning for sampling temperature. Below, we address your comments and questions to further clarify and improve the paper:

The experimental evaluations are not overall enough. This paper mainly focuses on several QA tasks, but other tasks where hallucinations exist can be further explored.

While it is true that any evaluation could be extended to increase the scope, we decided to align the evaluation with current work focusing on QA tasks [1,2,3,4] to provide a fair comparison of the different uncertainty estimation methods within the scope of our computational budget. Noteworthy, the recent Nature publication on semantic entropy [2] additionally investigates paragraph-length generations of biographies, which they again break down into individual question-answering tasks. This implies that performance on QA tasks is expected to correlate with performance on longer generations as well.

This paper uses OPT model family for experiments, while more common models like LLaMA, GPT can be investigated. Related Question: Could you provide experiments on LLaMA, GPT model family?

For best comparability, we decided to focus on the same model family used in Kuhn et al. [1], which our method is building upon. Instead of adding another model family, we prioritized evaluating the generalizability across different datasets with different answer lengths (from short phrases to whole sentences). The rationale behind this is that related work suggests that performance trends generalise across model families (see line 377). For instance, figure 4 in Duan et al. [4] clearly shows that the uncertainty method that performs best on the OPT family also performs best on the LLaMA model family. However, to address your question, we are currently running the experiments with the LLaMA model family which will take several weeks and will include the results in the camera ready version of the paper.

Question: SDLR uses an extra NLI model to compute the contribution of every token to the final semantics. Does this can be replaced by the attention score distribution?

Using the attention scores instead of the NLI model would not be appropriate in our setting, since our objective is to identify sequences that diverge from a given sequence in semantic space. At present, we do not see a feasible way to achieve this using attention scores.

In general, we hope that our clarifications elucidate the empirical validity of our work. Should you have any further inquiries, we look forward to addressing them. Otherwise, we hope for a positive reassessment of our work.

[1] L. Kuhn, Y. Gal, and S. Farquhar, "Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation," arXiv preprint arXiv:2302.09664, 2023.

[2] S. Farquhar, J. Kossen, L. Kuhn, and Y. Gal, "Detecting hallucinations in large language models using semantic entropy," Nature, 2024.

[3] A. Nikitin, J. Kossen, Y. Gal, and P. Marttinen, "Kernel language entropy: Fine-grained uncertainty quantification for LLMs from semantic similarities," arXiv preprint arXiv:2405.20003, 2024.

[4] J. Duan, H. Cheng, S. Wang, A. Zavalny, C. Wang, R. Xu, B. Kailkhura, and K. Xu, "Shifting attention to relevance: Towards the predictive uncertainty quantification of free-form large language models," arXiv preprint arXiv:2307.01379, 2023.

We appreciate you taking the time to read through all our clarifications and the responses from other reviewers. We are pleased to hear that you no longer have any concerns or questions about our work.

Given this, we kindly ask you to consider revisiting your confidence and rating scores. If anything remains unclear, we would be happy to provide further details.

Thank you again for your positive feedback!

Thank you for the response! Since you no longer have any concerns or questions, we would appreciate it if you could share your reasoning for still considering the paper "marginally below the acceptance threshold".

We thank you for this extensive and thoughtful feedback. We are pleased that you found the paper clear and easy to follow, that you recognized the simplicity and computational efficiency of our method, and that you appreciated the alignment of our evaluation setup with prior work to ensure comparability. Below, we address your comments and questions to further clarify and improve the paper:

The new method relies on three scores, but their importance remains untested.

Thank you for pointing this out, we appreciate the opportunity to clarify the importance of each score and how they contribute to the effectiveness of our method. These three scores work together to identify the appropriate token to substitute in the initial output sequence to effectively alter its semantics. The Attribution score identifies which present tokens in the initial output sequence should be changed to modify the semantic meaning (see line 249 onward), while the Substitution score identifies potential alternative tokens for replacing the present tokens (see line 258 onward). Importantly, these two scores operate on different sets of tokens: the Attribution score considers the present tokens, whereas the Substitution score considers the alternative tokens. Additionally, the Importance score is crucial as it is the only score that considers the likelihood of the alternative tokens so that we avoid selecting very unlikely tokens that undermine the coherence of the overall output sequence (see line 287 onward).

While Section 4 dedicates an entire paragraph to each score to explain its theoretical motivation, we have further extended these explanations to clarify the distinct role of each score. Additionally, we conducted an ablation study to evaluate their impact, confirming our theory that the highest performance is achieved when all three scores are considered (see Figure 6 in the appendix). It can also be observed that the Substitution score has a greater positive impact than the Attribution score, which aligns with the fact that the output sequences consist of a relatively small number of present tokens. However, the Attribution score becomes increasingly important for longer output sequences, where many present tokens in the initial output sequence could be candidates for substitution. Thus, we recommend including all three scores even for shorter sequences, as each score positively contributes to performance and the computational cost of computing the scores is negligible compared to sampling output sequences.

We have revised the paper to make these clarifications more explicit in both the methodology section and the experimental results.

NLI models are trained on the sentence level, the method relies on the prefix and current token to score for entailment/contradiction, but the generation so far is only partial. It is surprising that NLI works so well for this - I therefore wonder whether the strength really comes from the NLI model (and loss), or a token-based similarity method (e.g. something as simple as word2vec distances) would be sufficient here. The paper does not empirically show that NLI is needed. The authors could conduct a comparison to a simpler word2vec-based method and report that as a comparison to their NLI-based scorer.

This seems to be a slight misunderstanding. The input to the NLI model is not a partial output sequence but the fully generated initial output sequence. The NLI-based scores are then used to identify which token in this initial sequence should be substituted to alter its semantic meaning (see explanation above). Once the token with the highest score is substituted, the remaining output sequence is generated using the usual sampling method (see line 300 onward). In general, the NLI-based scores provide more context-sensitive semantic information, going beyond surface-level similarity. The NLI model captures nuanced relationships (e.g., polysemy and homonymy) that simpler token-based methods like word2vec distances cannot fully account for. Furthermore, the NLI model is already needed to compute the semantic entropy [1, 2], thus SDLG efficiently leverages existing components instead of introducing additional ones.

In general, the NLI-based scores provide more context-sensitive semantic information, going beyond surface-level similarity. The NLI model captures nuanced relationships (e.g., polysemy and homonymy) that simpler token-based methods like word2vec distances cannot fully account for. Furthermore, the NLI model is already needed to compute the semantic entropy [1, 2], thus SDLG efficiently leverages existing components instead of introducing additional ones.

SE DBS is competitive with the proposed SDLG on TriviaQA (Table 1). This similar result is not sufficiently discussed in the paper. The examples in the appendix (Table 3) include only comparisons between SDLG and SE, but not DBS - To promote both baselines to the same level, adding examples for DBS and providing a detailed analysis of where and why SDLG outperforms DBS would strengthen the paper.

TriviaQA has the shortest output sequences among the datasets, where enforcing token-level diversity alone is often sufficient. In such cases, the advantages of our method are less pronounced because there are fewer options that need to be explored. We included TriviaQA primarily to demonstrate that even with minimal present tokens, our method outperforms the baselines. However, the true strengths of SDLG emerge in scenarios with longer output sequences, such as those in TruthfulQA, where simple token-level diversity is insufficient, and our method's ability to explore semantic diversity becomes critical (see line 435 onward). We updated the section “Analysis of results” to highlight this fact.

Discrepancy in motivation and experimental evaluation. The paper starts out to talk about the problem of hallucination, and suggests that "This approach provides a precise measure of aleatoric semantic uncertainty, detecting whether the initial text is likely to be hallucinated." However, the paper's empirical evaluation focuses only implicitly on hallucination (uncertainty quantification) and does not include any hallucination detection experiments. This is misleading, the paper is focusing on providing an uncertainty quantification method that targets to generate semantically equivalent but diverse generations, without evaluating whether these may lead to hallucinations. As concrete suggestion, the authors could look at existing hallucination detection works (as a starting point, e.g. [HADES dataset](https://aclanthology.org/2022.acl-long.464.pdf, code and data available).

At the beginning of our introduction, we define what we mean by hallucinations: “At the time of writing, there is no consensus on the exact nature of all causes of hallucination. We consider generated text to be hallucinated if it stems from contradictory or non-existent facts in the training data or input prompt. Such hallucinations are conjectured to be mainly caused by the predictive uncertainty inherent to probabilistic models. This type of hallucination is also referred to as confabulation.” Thus, we aligned our argumentation with the Nature paper by Farquhar et al. [2], which in turn aligned the experiments with uncertainty estimation papers.

We agree that incorporating suitable hallucination benchmarks could be an interesting avenue. However, hallucination detection experiments usually follow a different setting. For instance, the proposed benchmark HADES is not directly applicable for benchmarking our method. We focus on improving the estimation of semantic entropy by our improved sampling scheme SDLG, tackling the problem of sampling likely yet semantically diverse output sequences. Semantic entropy operates on a sentence (or paragraph) level, while the HADES benchmark considers hallucinations on a token level. It is true that uncertainty estimation and thus hallucination detection can also be done on a token level. Yet, semantic entropy, as well as SDLG, are explicitly designed for sequence-level uncertainty estimation. In our extensive literature review, we found that question-answering tasks are the absolute standard to evaluate those methods son a sequence level.

The results presented in the main text fall short and are poorly written. Results are mentioned without linking to result tables or figures. The reader has to infer where evidence is given.

Thank you for pointing out where the presentation of the results could further be improved. In line 463, we noted that “experimental details and insights into the two ablation studies on semantic clusters and computational expenses can be found in Sec. D in the appendix”. To further improve clarity, we have now revised the text to explicitly reference the relevant tables and figures, which we now also added to the main paper. These updates ensure that readers can easily locate the evidence supporting our claims.

We also thank you for the helpful suggestions, which we all have incorporated into the updated version of the paper. In general, your detailed and thoughtful review has been instrumental in further improving the clarity and quality of our work. We hope that the revisions address all your concerns. Should you have any further questions or suggestions, please feel free to reach out again. Otherwise, we hope for a positive reassessment of our work.

[1] L. Kuhn, Y. Gal, and S. Farquhar, "Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation," arXiv preprint arXiv:2302.09664, 2023.

[2] S. Farquhar, J. Kossen, L. Kuhn, and Y. Gal, "Detecting hallucinations in large language models using semantic entropy," Nature, 2024.

[3] J. Duan, H. Cheng, S. Wang, A. Zavalny, C. Wang, R. Xu, B. Kailkhura, and K. Xu, "Shifting attention to relevance: Towards the predictive uncertainty quantification of free-form large language models," arXiv preprint arXiv:2307.01379, 2023.

Thank you for acknowledging the strong methodological contributions of our work and for highlighting the clarity of the theoretical aspects. Below, we address your comments and questions to further enhance the overall presentation of our paper and provide additional context regarding the scope of our work:

Experimental Clarity: The experimental section lacks clarity, particularly in explaining how ROUGE and BLEURT metrics were applied. The reference in L.409 (“in general...”) requires citation if it’s a general principle, and L.411-414 discussing AUROC are somewhat confusing, particularly the sentence “AUROC is used as a metric for classifying.”

Thank you for this feedback, we have revised the sections for improved clarity.

As mentioned in line 406, the correctness of the initial output sequence is evaluated using ROUGE-L, ROUGE-1, and BLEURT. While we referred to the respective papers for detailed explanations, a summary is as follows: ROUGE-L measures the longest common subsequence, and ROUGE-1 measures the overlap of unigrams between the initial output sequence and the ground truth answer. BLEURT uses a learned evaluation to evaluate how well the initial output sequence conveys the meaning of the ground truth answer.

Regarding the phrase “in general,” it refers to a formula that can be applied across all datasets we consider, each with varying ground truth answers (e.g., only true reference answers or both true and false references). Using this formula, each metric produces a score for the initial output sequence. To determine whether each initial output sequence is classified as “correct” or “incorrect,” a threshold has to be applied (sequences with scores above the threshold are considered “correct”). For each of the three metrics and ten thresholds, the AUROC measures how well the uncertainty estimates align with this correctness classification, using the uncertainty estimate as a scoring function. A higher AUROC indicates a higher utility of the uncertainty measure.

We hope that the revised sections in the paper improve clarity on this comprehensive evaluation setup, which demonstrates the effectiveness of our method in reliably quantifying uncertainty in NLG.

Baselines and Related Work: It is unclear why some relevant works, such as [1], are not included as baselines. Additionally, a recent paper [2] appears to be highly relevant (and also provides a clearer explanation of the evaluation setup).

Thank you for highlighting these papers, they are indeed valuable contributions. While Lin et al. [1] focus on computing the similarity between output sequences, our work proposes an approach for generating these output sequences. This makes their method complementary to our method rather than directly comparable as a baseline, as it could be integrated with our approach. Similarly, Chen et al. [2] focus on computing the eigenscore of sequence embeddings of output sequences sampled with multinomial sampling. As such it competes with logit-based measures such as semantic entropy and our proposed sampling method SDLG is complementary to their approach. Although agree that investigating the effectiveness of SDLG for estimating the Eigenscore would be interesting, we deem it out of the scope of this work, as this would need more careful mathematical treatment regarding the validity of such an estimator, which we provided for using SDLG to estimate semantic entropy. For these reasons, we chose not to include these methods as baselines but rather ensure they are properly discussed in the related work section to contextualize our contributions within the broader field of uncertainty estimation and hallucination detection.

Model Choice: Restricting the analysis to a single model family (which is not among the most recent) slightly impacts the work’s soundness. Expanding the analysis outside OPT's scope would strengthen the findings and make them more generalizable.

We decided to focus on the same model family used in Kuhn et al. [3], which our method is building upon. Given a limited computational budget, we prioritized evaluating the generalizability across different datasets with different answer lengths (from short phrases to whole sentences), instead of adding another model family. The rationale behind this is that related work suggests that performance trends generalize across model families (see line 377). For instance, figure 4 in Duan et al. [4] clearly shows that the uncertainty method that performs best on the OPT family also performs best on the Llama family. However, to address your point, we are currently running experiments with the LLama model family. We will include these additional results in the camera-ready version of the paper, as the experiments, unfortunately, take too long to be available within the rebuttal period.

Question: L.441: I found the paragraph about semantic cluster unclear. What does this mean? "Our method results in a 19% increase of semantic clusters after generating the second output sequence, as well as a 74% increase of semantic clusters after the tenth output sequence".

We appreciate your attention to detail. This paragraph refers to Figure 7(b) in the appendix, which provides an ablation study to verify whether SDLG achieves its intended purpose, namely, sampling more semantically diverse output sequences. The results show that multinomial sampling generates output sequences that, on average, span approximately 1.4 semantic clusters, while SDLG generates sequences that span about 2.4 semantic clusters. This represents a 74% increase when considering ten output sequences per question, as reported in the main paper. We updated the paper to also include the relevant figures, ensuring that the connection between the ablation study and the main claims of the paper is clearly established.

With this, we hope to have addressed all your questions and concerns. Should you have any further questions or suggestions, please feel free to reach out again. Otherwise, we hope for a positive reassessment of our work.

[1] Z. Lin, S. Trivedi, and J. Sun, "Generating with confidence: Uncertainty quantification for black-box large language models," arXiv preprint arXiv:2305.19187, 2023.

[2] C. Chen, K. Liu, Z. Chen, Y. Gu, Y. Wu, M. Tao, Z. Fu, and J. Ye, "INSIDE: LLMs' internal states retain the power of hallucination detection," arXiv preprint arXiv:2402.03744, 2023.

[3] L. Kuhn, Y. Gal, and S. Farquhar, "Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation," arXiv preprint arXiv:2302.09664, 2023.

[4] J. Duan, H. Cheng, S. Wang, A. Zavalny, C. Wang, R. Xu, B. Kailkhura, and K. Xu, "Shifting attention to relevance: Towards the predictive uncertainty quantification of free-form large language models," arXiv preprint arXiv:2307.01379, 2023.

We thank you for the thoughtful feedback and recognition of the strengths of our work. We are pleased that you find our method innovative, straightforward, and computationally efficient in addressing uncertainty quantification in NLG.

Below, we address each of your comments and questions in detail to further enhance the clarity of our work:

1. Question: How does the method handle polysemy and homonymy within language models? Given that words may have multiple meanings based on context, how does SDLG differentiate and handle these variations in semantic assessments?

Thank you for bringing up this important point. Both the NLI model and the language model in our method are trained on large corpora of natural text, allowing them to interpret word meanings within their linguistic context and thus disambiguate polysemy and homonymy. Specifically, our method leverages the NLI model to identify words that change the semantics within the given context (via attribution and substitution scores) and the language model to evaluate the overall word likelihood given the context (via the importance score). Our context-aware method is thus capable of handling polysemy and homonymy.

2. Question: In SDLG, when a sentence (sentence B) is generated during the uncertainty measurement of a sentence (sentence A) after substitution, is it possible to reuse the semantic and uncertainty-related information obtained to directly measure the uncertainty of sentence B, or must a new sampling and evaluation process be initiated?

In general, predictive uncertainty estimation is concerned with quantifying a model's uncertainty for a given input, rather than for specific outputs it generates. In our work, we adhere to this principle by measuring the uncertainty of the language model when prompted with a particular input sequence (e.g. a trivia question), rather than evaluating the uncertainty of individually generated output sequences (e.g., sentences A and B).

3. Question: How would different NLI models, such as smaller versions of DeBERTa, influence the results? Can the robustness of SDLG be maintained across different NLI models? (simultaneously answering the 1. point of your stated weaknesses)

We agree that biases inherent in the NLI model could influence uncertainty estimates. However, it is important to emphasize that any potential biases in the NLI model would affect all methods that rely on it, including the baselines we compare against [3,4]. Thus, further investigations into NLI models would be an interesting research direction on its own but would go beyond the scope of this work. To ensure a fair comparison, we isolate the impact of our improved sampling strategy by using the same NLI model as Kuhn et al. [3]. The results clearly demonstrate that our method achieves superior performance under these controlled conditions, indicating that the improvement stems from the sampling procedure itself and not the choice of a specific NLI model. However, to address your recommendation, we are currently running the experiments with a smaller NLI model. We will include this ablation study in the camera-ready version of the paper, as the experiments, unfortunately, take too long for results to be available within the rebuttal period.

4. Question: In cases where there are no existing NLI models with an identical tokenizer, can you fine-tune a small NLI model from scratch for a novel LLM with any vocabulary? (simultaneously answering the 2. point of your stated weaknesses)

This is a thoughtful observation, and we appreciate your attention to detail. As briefly described in lines 934 to 939 in the Appendix, our method can be applied even when the NLI model does not share the same tokenizer as the language model. Artetxe et al. [5] demonstrate that a differentiable bilingual mapping can align token embeddings into a shared space, enabling our method to be used without any modifications. Alternatively, a dedicated NLI model with the same tokenizer as the language model could of course also be created, either by fine-tuning an existing model or training one from scratch. However, as mentioned above, we have not explored these approaches further, since investigations into the NLI model fall outside the scope of this work and our primary focus is on isolating the impact of our improved sampling strategy within the current setting.

Thanks for the clarifications, which have addressed most of my concerns. For Q3, do you have any preliminary experimental results regarding smaller NLI models? For W3, I recommend relocating a condensed version of Figure 11 to the main paper, ideally accompanied by an example, to illustrate the scoring mechanism more clearly.