Efficient semantic uncertainty quantification in language models via diversity-steered sampling

Thank you for your encouraging comments on our paper's novelty, applicability, and comprehensiveness. Please find below our point-by-point responses and new qualititative case studies motivated by your suggestions.

less scalable in large-generation scenarios compared to parallelizable approaches

We acknowledge that the sequential sampling cost prevents fully parallel generation. The scenario we consider is not all-purpose large generation but targeted generation with the specific goal of estimating a downstream uncertainty metric. As long as the target uncertainty metric is defined on top of semantic clusters, like SE or MI, steering sampling toward covering more clusters can help estimate it with fewer generations. Baseline iid sampling methods, in contrast, can "waste" sampling resources with semantically redundant generations. Figure 3 shows that our method helps cover more semantic clusters relative to baselines, and Appendix C presents case studies illustrating the improvement in semantic diversity. Moreover, Appendix B.3 presents an optional online stopping method that enables termination of sampling once uncertainty estimates stabilize, potentially reducing the required sample size below the >20 samples typically required by standard iid baselines.

As derived in Appendix B.5, the extra per-token cost relative to standard iid sampling is $O(V P M_{\rm NLI})$, where $P$ is the size of the running sample set, $V$ is the vocabulary size, and $M_{\rm NLI}$ is the cost of computing NLI entailment. In practice, we seek speed gains by restricting similarity computation to the top-$k$ tokens at each decoding step (Appendix B.5, L931–933) with $k=32$, as most tokens are not relevant to the query at hand. This reduces the extra cost to $O(k P M_{\rm NLI})$ per token. Please note that diversity penalty can still be computed in parallel over $k$ tokens at each step.

An interesting avenue for future work would be to combine batch iid sampling with diversity steering in a hybrid approach. Our method is still parallelizable over beams or batches given a running sample set, so we could allow the model to sample $M$ generations in parallel before applying the diversity penalty for the next generation if the semantic redundancy is high. We could also consider alternating block tokens of iid sampling with diversity-steered sampling.

We will revise the text to clarify which aspects of our added computational cost are parallelizable and elaborate on hybrid extensions.

accuracy of uncertainty estimation heavily depends on the quality of semantic clustering derived from the fine-tuned NLI model

We would like to clarify that we use the finetuned NLI for sampling (our method contribution) but the original NLI without finetuning for final uncertainty estimation, as done in prior work [1-3, 5]. We will clarify this distinction in the Methods section. That said, our finetuned NLI model can be used in place of the original NLI model as well; both the TRUNC and MASK finetuned models scored similar validation accuracies as the original model (~91%), as reported in Appendix A.

No matter what sampling scheme is used, be it ours or any of the baselines, incorrectly clustering the final samples can distort the SE/MI estimates, as pointed out by the reviewer. Semantic clustering is an interesting and challenging problem in its own right, for which bidirectional entailment has been shown to outperform embedding-based similarity or lexical similarity methods [1, 2]. While we agree that improving semantic clustering can improve the accuracy of semantic uncertainty estimates, it lies beyond the scope of this work. We instead focus on how to generate semantically diverse samples assuming our goal was to estimate SE/MI defined on top of bidirection entailment clusters.

If the reviewer was referring instead to the accuracy of the diversity-steering term used in sampling, we now provide additional results showing that our finetuned NLI model is not biased toward high or low entailment when the partial sequences do not carry sufficient information. Overall, we find that models tend to output entailment probabilities of ~33% when provided input with almost no information -- the examples in Figures 2 (for input 1-2 tokens long) and 4 (for near 100% masking) support this. The specific value of 33% can be understood as the maximum-uncertainty prediction in the three-way (entailment/contradiction/neutral) NLI classification task. We will add to Appendix A the accuracy of the entailment model at every level of truncation/masking in addition to the average across all levels (73-74%) reported previously. Accuracy goes up steeply from 34% to 72% as corruption decreases from 90% to 50% and reaches 91% at no corruption. We will also provide a distribution of predicted P(entailment) for single-token inputs and 95% masking, to demonstrate that the model is correctly uncertain in settings with very little information about entailment. In this no-information regime, the desired behavior is to default to iid sampling, which is why we initialize the steering strength hyperparameter at $\lambda_0 = 0$ and only dynamically adjust it upwards when sample redundancy is significantly above 0.3 in terms of bidirectional entailment. Under our adaptive scheme for setting $\lambda_t$, we find that $\lambda_t$ tends to monotonically increase over sampling time as we acquire more information about the text being generated (Figure 5).

qualitative examples or failure cases to illustrate how the method handles edge cases, hallucinations, or ambiguous prompts

Thank you for this suggestion. We have added case studies to the appendix demonstrating that our method can surface meaningful SE/MI values by uncovering relevant semantic clusters in three settings: (1) when the LM hallucinates, (2) when the LM is legitimately uncertain due to ambiguity in the prompt, and (3) a combination of the two. We summarize them briefly below.

(1) Hallucination. If the prompt asks about a domain-specific, long-tail fact, models tend to hallucinate. We expect to see one cluster containing the correct answer and a few hallucination clusters. MI should be high, as the model easily "changes its mind" when conditioned on its own response. SE should depend on the hallucination pattern -- low if there are a few dominant modes of hallucination and high if hallucinations are varied and diffuse. We manually analyze responses to a well-defined biomedical question (with a single correct answer) for this setting.

(2) Ambiguous prompt. If the prompt is vaguely worded or asks to name one item among many, models can output several valid semantic clusters. We expect SE to be high and MI to be moderate or low, as the model would not be internally conflicted about a given answer. We manually analyze responses to a precisely worded and easy question with multiple correct answers, taken from the AmbigQA dataset.

(3) A mix of both. In Appendix C, we provided sample responses for the ambiguous, domain-specific question with multiple possible correct answers: "Name one way to measure the Hubble constant." Given 8 sample generations, our method is able to surface more valid clusters corresponding to different methods of measuring the Hubble constant as well as hallucination clusters with made-up physics facts.

[1] Farquhar, S. et al. (2024). Detecting hallucinations in large language models using semantic entropy. Nature, 630(8017), 625-630.

[2] Kuhn, L. et al. (2023). Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation. ICLR.

[3] Lin, Z. et al. (2023). Generating with confidence: Uncertainty quantification for black-box large language models. TMLR.

[4] Manakul, P., et al. (2023). Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models.

[5] Aichberger, L., et al. (2025). Semantically diverse language generation for uncertainty estimation in language models. ICLR.

We thank the reviewer for the positive feedback and are happy that you found the paper theoretically grounded, flexible, and clearly written. We appreciate the comment that our finetuned NLI is interesting in itself. Please find below our point-by-point responses.

compatibility with DRAGIN (Su et al. 2024)

Thank you for this interesting question that has helped us consider the implications of semantic diversity steering in the dynamic RAG context, beyond single-turn QA settings we focused on in our work. One possible entry point into DRAGIN is the RIND module that decides when to retrieve. RIND uses a product of attention weights, token entropy, and stopword indicators to detect the position when the model begins to hallucinate. This type of scoring can be prone to false positives, which would waste retrieval augmentation compute. In particular, token entropy is susceptible to over-estimating uncertainties if softmax weights are shared across multiple tokens with the same meaning. To improve on this, when RIND classifies a position as requiring retrieval, we can verify this prediction by generating a few partial phrases starting from that position with diversity steering and quantifying a measure of semantic uncertainty. The finetuned NLI model can be used for both steering and estimation to accommodate partial sequences. While using the finetuned NLI model would be new, this approach would naturally extend prior work using NLI for hallucination detection (e.g., [1]).

compatibility with claim-conditioned probability (Fadeeva et al. 2024)

Thank you for bringing the CCP metric to our attention. While we have used SE and MI as illustrative examples, our method is compatible with any downstream uncertainty metric defined on semantic clusters. CCP quantifies uncertainties only on the factual "claim," after marginalizing over semantic clusters (thus removing uncertainties on "surface form", or style, as SE does) and conditioning on claim type/order. Instead of drawing multiple generations, however, it processes a single complete generation post hoc by substituting the token at each position with top-$k$ alternatives and computing the CCP of each alternative token based on NLI predictions: (1) if the text up to position $j$ with the $j$-th token swapped is predicted to "entail" the original text, then they are assumed to have the same meaning and claim type, (2) if "contradict", then they are assumed to have a different meaning but same claim type, and (3) if "neutral," they belong to different claim types. The CCP of the entire claim is then 1 minus the simple product of the (non-stopword) words constituting the claim.

We can think of plugging into CCP in a couple of ways:

• For a given generation $x$, the authors take a pretrained DeBERTa-large model as is to predict NLI relationships between prefixes $x_{1:j}$ and $[x_{<j}, \textrm{substituted token}]$ for each position $j$. This can be risky, as DeBERTa-large is trained on full premise-hypothesis pairs and the CCP scoring relies heavily on its predictions to determine membership to meaning/claim types. We can use the finetuned NLI model instead to better deal with prefixes.
• Instead of post-processing a single generation $x$ by swapping each token $x_j$, we can consider drawing multiple full generations with diversity steering (with the finetuned NLI model) and computing the claim-level CCP with the original NLI model, as done in our paper. Importance reweighting and optional variance reduction proceed as usual by reweighting the token probabilities. This sampling-based approach also eliminates the need to separately handle sub-word token substitutions.

lambda ... needs recalibration for each new model and query

We acknowledge that $\lambda$ is an important hyperparameter that must be tuned. In Section B.3, we describe an adaptive schedule for $\lambda$ that empirically transferred across models and across queries, and yielded ${\rm ESS}/N$ > 0.4 for all models and datasets in Table 1. The schedule goes as follows: $\lambda$, initialized at zero (no steering), increases only when the maximum entailment exceeds $E_{\rm target} = 0.3$ ($\approx 1/3$), corresponding to the maximum-uncertainty prediction in three-way NLI classification. Empirically, we find that $\lambda_t$ tends to monotonically increase over sampling time as we acquire more information about the text being generated (Figure 5). We will justify our adjustment scheme in more detail in the text.

Do authors assume full entailment holds between premise and hypothesis examples at all levels of corruption when training the NLI model?

No. During finetuning we retain the original MNLI gold label for each corrupted pair, so contradiction and neutral examples remain even when corrupted. Specifically, for every premise–hypothesis pair we (1) randomly choose one side, (2) truncate or mask it at a random position, without flipping the label, and (3) finetune only the new [TRUNC]/[MASK] embedding and the classifier head. This teaches the model that an incomplete sequence can entail, contradict, or be neutral wrt the other side, depending on the already-revealed content, as illustrated in Figures 2 and 4. Details of the NLI training can be found in Appendix B.1.

[1] Manakul, P., et al. (2023). Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models.

We are grateful for your positive evaluation of our work regarding the motivation, practicality, design of experiments, and clarity. Your constructive suggestions have helped improve our paper. Please find our point-by-point responses below and summaries of new results motivated by your comments.

limited evaluation of epistemic uncertainty

We have added new evaluations for MI since the manuscript submission. We now report results for both LLaMA 3 8B-Instruct and LLaDA 8B-Instruct (to represent MDMs) on CoQA, TriviaQA, AmbigQA, and TruthfulQA. Previously in the literature, MI has only been evaluated on TriviaQA and AmbigQA for the Gemini 1.0 Pro and Nano-1 models [1], so we believe our experiments using new models and datasets will serve as useful benchmarking results for the community. To summarize briefly, our sampling method shows comparable or slightly better mean AUROC (0.57 ~ 0.64) across TriviaQA, AmbigQA, and TruthfulQA relative to vanilla iid sampling with $\tau=1$ and $\tau=2$, but performs similarly to DBS and SDLG on AmbigQA. DBS results carry high variance, about 30-40% higher than ours. LLaDA shows similar patterns but slightly lower AUROC by 1-2 overall.

To complement the quantitative results, we also added case studies demonstrating that diversity steering uncovers relevant semantic clusters, leading to meaningful SE/MI values. We consider three settings briefly summarized below.

(1) When the LM hallucinates. If the prompt asks about a domain-specific, long-tail fact, models tend to hallucinate. We expect to see one cluster containing the correct answer and a few hallucination clusters. MI should be high, as the model easily "changes its mind" when conditioned on its own response. SE should depend on the hallucination pattern -- low if there are a few dominant modes of hallucination and high if hallucinations are varied and diffuse. We manually analyze responses to a well-defined biomedical question (with a single correct answer) for this setting.

(2) When the LM is legitimately uncertain due to ambiguity in the prompt. If the prompt is vaguely worded or asks to name one item among many, models can output several valid semantic clusters. We expect SE to be high and MI to be moderate or low, as the model would not be internally conflicted about a given answer. We manually analyze responses to a precisely worded and easy question with multiple correct answers, taken from the AmbigQA dataset.

(3) A mix of both. In Appendix C, we provided sample responses for the ambiguous, domain-specific question with multiple possible correct answers: "Name one way to measure the Hubble constant." Given 8 sample generations, our method is able to surface more valid clusters corresponding to different methods of measuring the Hubble constant as well as hallucination clusters with made-up physics facts.

ablation studies on ... the impact of variance reduction

We have added an appendix section with the following items, briefly summarized:

1. A brief primer on control variates (CVs). We present the concept in general terms and notation that apply clearly to the SE and MI cases.
2. Justifications for why our proposed CVs are reasonable choices: We define our CVs as the mean-centered log probabilities of each generation. In natural language generation, high-likelihood samples tend to be mapped to dominant clusters, so our CVs correlate well with the cluster probabilities, as desired. Moreover, we can reduce variance "for free," at no additional inference cost -- that is, they do not require extra forward passes through the base LM, as the log probability of each generation is already needed to evaluate the importance weights $w_i$. We will revise the text of Section 3.4 to make this benefit clearer as well.
3. Average percent variance reduction achieved (ablation results): We can express the percent reduction in variance in terms of the correlation between the original semantic entropy functional $\tilde w Y$ and our proposed CV $\tilde w X'$, where $Y = -\log {\hat p}(c(s^{(i)}))$ and $X'$ is the mean-centered version of $X = -\log p(s^{(i)})$. More precisely, if $\sigma_X^2, \sigma_Y^2$ are their respective variances and $\sigma_{XY}$ is their covariance, then the percent reduction in variance is a direct function of the correlation $\rho = \sigma_{XY}/(\sigma_X \sigma_Y)$. We find that $\rho \approx 0.3 - 0.4$, translating to 10-15% variance reduction.

the performance of the NLI model on truncated sequences

We now provide additional results in Appendix A showing the performance of our finetuned NLI model at every level of corruption (trunc/mask) in addition to the average across all levels (73-74% accuracy) reported previously. Accuracy goes up steeply from 34% to 72% as corruption level decreases from 90% to 50%, and continues going up to 91% at no corruption. This is consistent with the intuition that the meaning of the full text can be gathered as soon as few semantically important tokens are revealed.

We also show that our model is not biased toward high or low entailment when the partial sequences do not carry sufficient information:

• Models tend to output entailment probabilities of ~33% when provided input with almost no information (new histogram figures in the appendix). The existing examples in Figures 2 (for input 1-2 tokens long, leftmost) and 4 (for near 100% masking, rightmost) support this. The specific value of 33% can be understood as the maximum-uncertainty prediction in the three-way (entailment/contradiction/neutral) classification.
• We will also provide distributions of predicted P(entailment) for single-token inputs and 90% masking. They are both sharply peaked at 33%, demonstrating that the model is correctly uncertain in settings with very little information about entailment.

influence of tuning the diversity parameter $\lambda$

To avoid biasing the sampling unnecessarily, it is important for $\lambda$ to default to iid sampling when the partial sequence carries no information about semantic redundancy with the sample set. This is why we initialize the steering strength hyperparameter at $\lambda_0 = 0$ and only dynamically adjust it upwards when sample redundancy is significantly above 0.3 in terms of bidirectional entailment. Under our adaptive scheme for setting $\lambda_t$, we find that $\lambda_t$ tends to monotonically increase over sampling time as we acquire more information about the text being generated (Figure 5). We will justify our adjustment scheme more clearly in the text.

accompanying intuition or explanatory text [for Section 3]

Thank you for this suggestion. While our applications of importance sampling and control variates are standard, we agree that the definitions of SE and MI have quite a bit of detail that can benefit from explanations to guide the intuition. To supplement Section 3.3, we will include figures illustrating SE and MI for the running "Hubble constant" example in the appendix, with each generation $s^{(i)}$ illustrated as dots in the "semantic" space and the clusters shown as different colors. As Section 3.4 is more difficult to capture visually, we will include a step-by-step worked example of MI estimation for AR and MDM (Algorithms 2-3) for a simple QA pair.

[1] Yadkori, Y. A., Kuzborskij, I., György, A., & Szepesvári, C. (2024). To believe or not to believe your llm. NeurIPS.

Thank you for your positive evaluation of our method's novelty, theoretical grounding, and clarity. We appreciate the constructive suggestions that have helped improve our paper. Please find below our point-by-point responses and new results motivated by your suggestions.

... use of entailment models for semantic similarity is not well justified

The use of entailment models is well established in the literature of analyzing natural language generation, particularly in hallucination detection (please see [1-5] for some well-cited examples). It has been shown to outperform embedding-based or lexical similarity methods for clustering semantically equivalent generations [1, 2]. More generally, the detection of entailment has been extensively studied in linguistics [6]. We will make this positioning clearer in the Introduction and Related Work sections.

measuring the semantic similarity of partial sequence ... can be significantly challenging, given that the presence of a single word (or token, e.g., not) can change the semantics significantly

We agree that predicting semantic entailment between a complete sequence and a partial sequence, rather than between two complete sequences, can be subtle. In fact, the right panel of Figure 2 illustrates exactly the flipping of the polarity upon revealing the token "not." So far as we are using bidirectional entailment for steering generation, this behavior (sensitivity to a few important tokens) is desired. Taking the Figure 2 example, if the fully generated sample were "You and your friends are not welcome here, said Severn," and the partial generation so far were "Severn said the people were," then we would be discouraging the choice of "not" as the next token, as this would result in semantically redundant generations, and instead steer the sampling toward something like the token "welcome," for which the model would predict a low entailment score.

To ensure that the model generalizes well to partial-sequence input, we finetune it to treat [TRUNC] or [MASK] as unknown context. While only 0.3% of the model parameters are updated, the model is capable of predicting the correct label long before the entire text is revealed. In Appendix A, we report a validation accuracy of 73-74% averaged over all levels of truncation/masking. To complement this, we will add plots of accuracy at every level of truncation/masking. Accuracy goes up steeply from 34% to 72% as corruption level decreases from 90% to 50%, and continues going up to 91% at no corruption. The intuition is exactly as the reviewer described: the meaning of the full text can be gathered as soon as few semantically important tokens are revealed.

The reviewer raises an important point that we do not want to be biased toward high or low entailment in the absence of information. This motivates our choice to use P(entailment) scores directly rather than the class predictions. Overall, we find that models tend to output P(entailment) of ~33% when provided input with almost no information, as supported by Figures 2 (leftmost) and 4 (rightmost). The specific value of 33% can be understood as the maximum-uncertainty prediction in the three-way (entail/contradict/neutral) NLI classification task. In Appendix A, we will add plots of the distribution of predicted P(entailment) sharply peaked at 33% for single-token inputs and 90% masking, to demonstrate that the model is correctly uncertain in settings with very little information about entailment. In this no-information regime, the desired behavior is to default to iid sampling, which is why we initialize the steering strength hyperparameter at $\lambda_0 = 0$ and only dynamically adjust it upwards when sample redundancy is significantly above 0.3 in terms of bidirectional entailment. Empirically, we find that $\lambda_t$ tends to monotonically increase over sampling time as we acquire more information about the text being generated (Figure 5).

Lastly, to further justify our use of predicted entailment probabilities, we'd like to present a perspective of our framework as model-guided online decision making, similar to delayed feedback scenarios in reinforcement learning or Bayesian optimization. The entailment model serves as a cheap "surrogate" that helps guide our "decisions" (which token to choose next) toward maximizing "utility" (full generation that is semantically different from existing samples). There is signal in partial sequences for us to construct our belief of whether the one-token-ahead partial sequence will be semantically equivalent to a running sample set.

Computational efficiency

We acknowledge that the semantic repulsion term adds computational overhead due to look-ahead NLI model scoring. However,

• In practice, we restrict similarity computation to the top-$k$ tokens at each decoding step (Appendix B.5) with $k=32$, as most tokens are not relevant to the query at hand. The extra per-token cost relative to standard iid sampling is $O(k P M_{\rm NLI})$, where $P$ is the size of the running sample set and $M_{\rm NLI}$ is the cost of computing NLI entailment.
• Our experiments run on a single A100 GPU with <32 GB of VRAM and remain tractable for typical N=16 samples (Appendix B.6).
• Importantly, the user can optionally implement online stopping (Appendix B.3), which allows termination of sampling once uncertainty estimates stabilize, potentially reducing the required sample size below the >20 samples typically required by standard iid baselines.

That said, we agree that hybrid approaches using batch sampling are promising for further speed gains.

evaluation [focuses on correctness] ... does not focus on the uncertainty quantification aspects

We would like to clarify a possible misunderstanding. Our evaluation focuses on (frequentist) calibration quality of our uncertainties, as is standard in the uncertainty quantification literature [1-3, 5]. The AUROC metric evaluates rank calibration; it gives the probability that we assign lower uncertainty to a randomly selected correct answer than to a randomly selected incorrect answer; good calibration would yield AUROC scores ~1 while poor calibration would score ~0.5. That is, while we report AUROC against correctness, the SE and MI are the ranking variables. The ROC curve is obtained by sorting the uncertainties from highest confidence to lowest (call the negated uncertainties $z_i$), sweeping a threshold $t$ through them, and computing for each $t$, ${\rm TPR}(t)=\frac{\text{num positives with }z_i \ge t}{\text{num positives}}$ ${\rm FPR}(t)=\frac{\text{num negatives with }z_i \ge t}{\text{num negatives}}.$ AUROC is then $\int_0^1 {\rm TPR}({\rm FPR}) d({\rm FPR})$. We will add more details about the evaluation metric in the appendix.

The AUROC metric requires us to threshold the ROUGE-L score to determine correctness, so we have also reported Spearman $\rho$ between ROUGE-L scores and SE/MI as a complementary metric and find that ours outperform DBS and SDLG by 5% and 6%, respectively, as well.

reasoning behind specific weights in equation (8)

Equation (8) defines the standard importance ratio $w_i = p(s_i) / q(s_i)$ used to correct for the fact that we sample from the biased (diversity-steered) proposal distribution $q$ when we seek to analyze $p$, and its self-normalized form $\tilde w_i$ satisfying $\sum_i \tilde w_i = 1$.

equality in (9)

Equation (9) is a consistent estimator (equality holds in the limit $N \to \infty$) for the cluster probability under usual regularity assumptions for self-normalized importance sampling -- finite second moment of $w$ being sufficient. Rewriting as $\frac{\sum_{i=1}^N w_i 1[s^{(i)} \in c]}{\sum_{i=1}^N w_i}$, applying the strong law of large numbers to the numerator and denominator separately guarantees almost-sure convergence to $\frac{\mathbb{E}_{s \sim q}[w(s) 1[s \in c]]}{1} = p(c|x, \theta)$. The estimator has bias $O(1/N)$ and the usual $1/\sqrt{N}$ precision. We had omitted the proofs of consistency, finite-sample bias, and asymptotic variance, because they are standard results in self-normalized importance sampling (e.g., Theorem 9.2 of [7]), but they may be helpful to include given our particular definition of $q$ (which justifies the finite second moment assumption) and treatment of the indicator function $1[s \in c]$. We will expand on the main text around Equation (9) and add the proofs to the appendix.

Sections 3.3 and 3.4 could be better with interpretation, examples, or simulations

Thank you for this suggestion. While our applications of importance sampling and control variates are standard, we agree that the definitions of SE and MI have quite a bit of detail that can benefit from explanations to guide the intuition. To supplement Section 3.3, we will include figures illustrating SE and MI for the running "Hubble constant" example in the appendix, with each generation $s^{(i)}$ illustrated as dots in the "semantic" space and the clusters shown as different colors. As Section 3.4 is more difficult to capture visually, we will include a step-by-step worked example of MI estimation for AR and MDM (Algorithms 2-3) for a simple QA pair.

[1] Farquhar, S. et al. (2024). Detecting hallucinations in large language models using semantic entropy. Nature.

[2] Kuhn, L. et al. (2023). Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation. ICLR.

[3] Lin, Z. et al. (2023). Generating with confidence: Uncertainty quantification for black-box large language models. TMLR.

[4] Manakul, P., et al. (2023). Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. EMNLP.

[5] Aichberger, L., et al. (2025). Semantically diverse language generation for uncertainty estimation in language models. ICLR.

[6] MacCartney, B. Natural Language Inference (2009).

[7] Owen, A. B. (2013). Monte Carlo theory, methods and examples.