Robust Hallucination Detection in LLMs via Adaptive Token Selection

Thank you very much for the constructive comments. We are grateful for the positive comments on our innovation and empirical justification. Please see our response to your comments one by one below.

Question #1 & Weakness #1: Clarification on the top-k tokens based on hallucination scores. If it’s a hard selection, how is this really learning to select the right tokens?

We confirm that our current implementation employs a hard top-k selection strategy during training, without using a differentiable approximation. Although the selection operation itself is non-differentiable, the learning process remains effective due to the design of our Multiple Instance Learning (MIL) framework, where instance selection is treated as a latent and optimization is performed over the selected subset. Specifically, while the top-k selection is discrete, the hallucination scores for each token are generated by a fully differentiable detector network. Gradients from the MIL loss are backpropagated through this scoring function, allowing the model to learn how to assign more accurate scores with better supervision over time.

To validate the effectiveness of our approach, we manually annotated hallucinated tokens in 100 randomly selected samples and conducted a token-level recall analysis on the TriviaQA dataset. Our adaptive token selection achieved a recall rate of 0.84, indicating that the model is indeed learning to select tokens highly aligned with the exact hallucinations. This recall performance is remarkable considering that we do not have token-level hallucination labels.

Please also kindly refer to our responses to Reviewer NwzN's Question #1, Weakness #4, and Reviewer dBF6's Question #2 & Weakness #3 for more discussion on addressing this concern.

Question #2: Clarification on "For each dataset, we report the average AUROC scores of detectors trained on one of the other three datasets." (lines 275–276)

• Does this mean you trained three detectors, each on a different dataset and then averaged their performance on the target dataset?
• Or was only one source dataset chosen (randomly or fixed) for training, and results reported per target? If so, how did you choose it? Clarifying this setup is important to interpret generalization performance.

As correctly noted in your first point, given the four datasets, our cross-dataset evaluation follows the setup where detectors are trained separately on each of the three source datasets (leaving the target dataset out, one detector per source dataset), and we then evaluate the detectors on the target dataset and report the AUROC score averaged over the results of the three detectors. This setting is designed to assess the model's ability to generalize from various source datasets to the target dataset. We appreciate your attention to this part of experimental design, and we will clarify this point in the paper.

Question #3: In Figure 3(a), cross-dataset generalization is only meaningful if it outperforms non-learnable baselines, such as those based on token probabilities or logits. To strengthen their results, I suggest the authors provide a justification for why such probability/logit-based baselines would perform worse than their method. Ideally, and if time permits, an additional experiment could be included to compare against a simple heuristic—such as averaging the token probabilities or logits across a sentence—as a baseline for hallucination detection in the cross-dataset setup.

Thank you very much for the question. The predictive logits of the tokens reflect how likely a token is to be generated by an LLM, which is related to training distribution of the model. As such, probability- or logit-based measurements would be influenced by fluency and grammaticality while hallucinations often occur in fluent but factually incorrect text. This is often one main cause to their degraded performance in hallucination detection. In contrast, internal representations extracted from LLMs can encode deeper semantic and contextual information, including how the model encodes, processes and grounds knowledge, which allows for a more nuanced assessment of whether the model’s output is correct or not. These observations align with findings in previous studies [Ref1], which suggest that “LLMs know more than they express”.

For the comparison of cross-dataset generalization capability, we selected CCS, SAPLMA, and HaloScope because these methods access the internal states of LLMs, which is also applied in HaMI. We agree that including non-learnable/training-free baselines is also important for a comprehensive assessment of a method’s generalization ability. For the non-learnable methods, including p(True), Perplexity, SE, and MARS, their results in Table 1 reflect their cross-dataset performance since they involve no training on source datasets. Comparing these results to the results of HaMI in Figure 3 (a), it is clear that HaMI consistently outperforms these state-of-the-art non-learnable baselines. Specifically, the cross-dataset performance of HaMI on four datasets are 0.858, 0.799, 0.796, 0.797, which is higher than the best results on each individual dataset, 0.824, 0.787, 0.777, 0.757, that can be obtained by these non-learnable baselines (these results are based on single running of HaMI; the same conclusion can be drawn for the averaged performance of HaMI over three independent runs in Table A3 in our response to Reviewer NwzN's Question #3 & Weakness #2). Furthermore, the suggested simple logit baseline that averages token log-probabilities across the sentence achieves AUROC scores of only 0.732, 0.649, 0.659, 0.709 on the four datasets, which are significantly lower than the corresponding AUROC scores from HaMI.

We will add this additional justification and comparison in the revised version of the paper to address this concern.

Question #4: In Table 2 (left), the performance gap between “ours” and probing the last token appears relatively small. While this might be acceptable, it’s important to note that “ours” includes additional information from predictive uncertainty. At this point, it’s unclear whether the observed improvement stems from the use of predictive uncertainty or from the top-k token selection itself. Clarification on this would be helpful. If feasible, I would also recommend adding a new row to Table 2 (left) showing results for a variant of HaMI without predictive uncertainty (e.g., HaMI*), to isolate the contribution of top-k selection.

We would like to clarify that all the four variants of the method reported in Table 2 (left), including the “Last” and "Ours", are built upon the features enhanced with the uncertainty measure $P^{c}_{\text{uncertainty}}$, i.e., the four methods are all variants of HaMI rather than HaMI* in Table 1 in our paper. Therefore, the performance improvements observed can be solely attributed to the effectiveness of our token selection strategy rather than the uncertainty modelling.

Additionally, please also refer to the results of Table 1, where we compare HaMI with HaMI* and SAPLMA using three different LLM models. HaMI is the model using both uncertainty modelling and adaptive token selection, HaMI* is trained directly on the raw features extracted from LLMs without using uncertainty modelling but using adaptive token selection, while SAPLMA relies solely on the last-token representations and does not use both uncertainty modelling and adaptive token selection. The differences between HaMI* and SAPLMA solely highlight the contribution of our adaptive token selection, while the differences between HaMI and HaMI* solely highlight the contribution of the uncertainty modelling.

Weakness #2: Minor concern: The proposed method appears to ignore the sequential nature of the tokens. If I understand correctly, it simply averages over the top-k candidates without considering their order or relative positions—information that could be valuable for detection tasks.

Thank you for your insightful question. Our method indeed directly averages over the top‑k token candidates, without explicitly modelling their sequential order or relative positions. This design choice is based on the primary motivation of HaMI, aiming to learn which tokens are most informative for hallucination detection, regardless of their position in the generated sequence. By doing so, HaMI can robustly detect hallucination signals even when they appear sparsely or in varied locations across long generations. Therefore, the lack of explicit positional encoding in our MIL learning framework does not compromise performance, yet it helps our model adaptively select hallucination tokens in varying positions of the generated response. We agree that further incorporation of the sequence order may provide additional value; we will explore this addition under the MIL framework in our future work.

Reference:

• [Ref1] Azaria, Amos, and Tom Mitchell. The internal state of an LLM knows when it's lying. Findings of the Association for Computational Linguistics: EMNLP. 2023.

Table A1: Cross-dataset generalization capability compared with non-learnable/training-free baselines.

Thank you for your thoughtful suggestions. We sincerely appreciate your positive remarks on our contributions. Please see our response to your comments one by one below.

Question #1 & Weakness #2: How accurate are GPT-4.1 labels in practice? Could you report inter-agreement between the two judgment passes? Have you considered incorporating human annotations or knowledge-based QA datasets (e.g., where gold answer is guaranteed factual)?

To evaluate the correctness of free-form generations in QA tasks, GPT-4 labels have been widely adopted in recent hallucination detection studies, such as Semantic Entropy [Ref1] and TSV [Ref2], establishing a popular evaluation protocol with GPT-4 used as annotator in this field. In our work, we utilize updated GPT-4.1 as the labelling model. We also observed that, when provided with gold answers, GPT-4.1 would give highly reliable annotation results for LLM-generated answers. The inter-agreements between the two judgment passes are 96.1%, 82.3%, 91.2% and 92.1%, respectively for the datasets Trivia QA, SQuAD, NQ Open and BioASQ based on LLaMA-3.1-8B model, indicating a high level of agreement across datasets. In our experiments, we exclude samples where the two GPT-4.1 judgments are inconsistent, ensuring our evaluation is based on confident, aligned labels.

We also agree with the value of incorporating human annotations or leveraging knowledge-grounded QA datasets with gold factual references. To validate HaMI’s performance on such datasets, we conducted an additional experiment on TruthfulQA dataset [Ref3], which provides ground-truth true and false answers. HaMI achieves an AUROC score of 0.927 on this dataset, which is higher than SAPLMA (0.903) and HaloScope (0.685), as we tested. While human annotation provides high labelling fidelity, its high cost limits feasibility at scale. Due to time limitations, we haven't managed to obtain the results of all competing methods; we will include the full results on this dataset to provide empirical justification for scenarios with human-annotated labels.

Weakness #1: Limited baseline comparison: Important recent hallucination detection methods such as LLM-Check and CED (Comparing Embedding Differences) are not included. These would provide stronger empirical context and help position the contribution more rigorously.

We appreciate your suggestion and include the suggested two additional baselines LLM-Check and CED for a more comprehensive comparison. The comparison results across all four datasets (i.e., TriviaQA, SQuAD, NQ, and BioASQ) with LLaMA-3.1-8B are summarized in Table A1 below. Observed from the original paper on LLM-Check, we choose hidden state and attention-based variations, which claim stronger performance. As presented in Table A1, our method consistently outperforms these added baselines by a significant margin.

In addition, as suggested by Reviewer NwzN, we have also added another recent method G-NLL [Ref4] as our baseline, and our method also obtains substantially better performance than G-NLL. Please refer to our response to Reviewer NwzN's Question 4 & Weakness #3 for more details.

Table A1. Results for LLM-Check, CED and HaMI

Weakness #3 & Weakness #4:

• Implicit token selection: The MIL approach does not guarantee that selected high-scoring tokens are truly the ones responsible for hallucination. There is no interpretability or causal evidence to support their significance.
• Lack of statistical/token-level validation: There is no human evaluation or statistical analysis demonstrating that the adaptively selected tokens indeed localize hallucinated content better than baseline tokens.

Thank you for raising this important point regarding the validation of our token selection module. HaMI relies on coarse supervision signals, i.e., sequence-level labels rather than token-level labels. As a result, the MIL-based approach cannot guarantee that the tokens with the highest scores always perfectly align with the exact hallucinated words. However, we would like to clarify that although the identified token may not always correspond precisely to the hallucinated token, it often appears close, for example, immediately before or after if such inaccurate cases happen. Moreover, hallucinated tokens themselves tend to receive scores very close to 1.0, indicating strong relevance even in the absence of fine-grained labels. This property allows HaMI to effectively narrow down the region where hallucinated content is likely to occur, which is practically useful for interpretability.

To further support the validity of the token selection process, we conducted an additional study, a manual evaluation using 100 randomly sampled examples from the Trivia QA dataset. In this analysis, we manually annotated hallucinated tokens and measured the token detection recall rate of our method throughout training. Specifically, HaMI can achieve a recall rate at 0.84 as training progresses. This suggests that our approach enables the model to identify salient tokens even without token-level supervision.

Please kindly refer to our responses to Reviewer NwzN's Question #1, Weakness #4, and Reviewer dBF6's Question #2 & Weakness #3 for detailed discussion and empirical evidence on the quality of the selected high-scoring tokens.

References:

• [Ref1] Farquhar, Sebastian, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. "Detecting hallucinations in large language models using semantic entropy." Nature 630, no. 8017 (2024): 625-630. 2024
• [Ref2] Park, Seongheon, Xuefeng Du, Min-Hsuan Yeh, Haobo Wang, and Yixuan Li. "Steer LLM Latents for Hallucination Detection." Forty-Second International Conference on Machine Learning. 2025.
• [Ref3] Lin, Stephanie, Jacob Hilton, and Owain Evans. "Truthfulqa: Measuring how models mimic human falsehoods." 60th Annual Meeting of the Association for Computational Linguistics. 2022.
• [Ref4] Lukas Aichberger, Kajetan Schweighofer, and Sepp Hochreiter. "Rethinking uncertainty estimation in natural language generation." arXiv preprint arXiv:2412.15176, 2024.

Thank you very much again for your thoughtful feedback. Below, we provide additional results and clarifications in response to your suggestion.

Following your suggestion, we conducted an additional experiment, using the BLEURT-threshold method to generate labels and comparing HaMI with most baselines (due to time constraints). Results are reported in Table A2 below, which shows that our methods HaMI and HaMI* can also consistently outperform the baselines using the BLEURT-based labelling datasets. Comparing to the GPT labels-based AUROC results in Table 1 in our paper, the internal state-based methods, including CCS, SAPLMA, HaloScope, and our method HaMI*, generally obtain largely improved AUROC performance based on the BLEURT labels, whereas uncertainty-based methods Perplexity and SE perform not that stably, i.e., performing better on some of the datasets while being less effective on the other datasets. Although HaMI (the last row in Table A2) includes uncertainty semantic-based feature enhancement, our proposed MIL hallucination score learning helps well balance the internal states and the uncertainty semantic, achieving consistently better AUROC performance than HaMI* in Table A2. These results provide empirical evidence for the robustness of HaMI from another perspective (a data labelling perspective).

Table A2: AUROC results with labels generated by the BLEURT method. The results are based on LLaMA-3.1-8B.

Nevertheless, we would like to note that, as we manually checked the annotation results, the BLEURT-threshold method is found better suited for phrase-level LLM responses, which often yield unexpected incorrect labels for sentence-level responses. For example, for the question: “In April, which sportsman married his childhood sweetheart Kim Sears?” The given ground truth answer is "Andy Murray" in the dataset. BLEURT labelled “Andy Murray.” as correct, but incorrectly labelled the sentence “In April, tennis player Andy Murray married his childhood sweetheart Kim Sears.” as incorrect. This type of failure is likely due to its sensitivity to answer length and format.

Besides, we noticed that the referenced Park et al. (2025) also report GPT-based evaluation results in Appendix C.2. Based on your suggestion and the above observations, we will include the results of all baselines under the BLEURT setting in our final version to complement the GPT-based evaluation results, and discuss the potential strengths and weaknesses under the GPT/BLEURT-based evaluation.

Thank you again for the insightful suggestion. We hope these replies help address your remaining concern. We are more than happy to engage with you to discuss any other questions you may have.

We are grateful for your insightful comments and encouraging evaluation of our work. Below, we provide detailed responses to each of your comments.

Question #1 & Weakness #2: Can the authors provide more empirical analysis of when the smoothness assumption holds? Are there cases where abrupt changes in hallucination scores between adjacent tokens are actually beneficial (or would not degrade the performance much)?

Thank you very much for the question. The smoothness loss is designed under the assumption of continuity in hallucination scores between adjacent tokens. As shown by the added ablation study results in Tables A1 and A2 below, this assumption holds well in practice since adding the smoothness loss consistently leads to improved or comparable performance across multiple benchmarks; notably, the improvement is more pronounced in the cross-dataset setting, indicating its positive role in serving as a regularization term for better generalization across the domains. Please refer to our response to Reviewer NwzN's Question #2 for more detailed discussion of these ablation study results.

Table A1. Ablation study on the two proposed losses (AUROC).

Table A2. Ablation study on the two proposed losses under the cross-dataset setting.

The smooth hallucination scores can enhance the optimization due to the following two main reasons. i) For non-hallucination sequences, all tokens are non-hallucination ones, and enforcing the smoothness over them results in consistently small hallucination scores for them (see Figure 4 for an example); for hallucination sequences, the smoothness loss does not stop the hallucination tokens from gaining large hallucination scores, but it can increase the scores of the neighborhood tokens of the hallucination tokens, i.e., to have a gradual rise and down at around the hallucination tokens while having consistently small scores for the non-hallucination tokens. This is typically observed in the experimental datasets. Additionally, since we pick the top-k tokens based on the hallucination scores of the tokens, the hallucinated tokens would be chosen as long as the margin between the scores of hallucinated and non-hallucinated tokens is sufficiently large; the token selection results would be similar to the cases that we have very large margins under the abrupt changes in hallucination scores. ii) The smoothness loss also effectively regularizes the model to avoid excessive attention to suspicious tokens, helping avoid overfitting the training data, and thereby enabling better generalization.

Question #2 & Weakness #3: What are the main failure modes of HaMI? When does adaptive token selection fail to identify the most relevant tokens?

One primary limitation of HaMI lies in its coarse supervision signals, i.e., the token selection in HaMI relies on sequence-level class labels. Since we do not have fine-grained, token-level hallucination labels, HaMI may sometimes fail to attribute the exact hallucinated tokens with the highest scores. Nonetheless, while the hallucination scores to the true hallucinated tokens may not be the absolute highest value, these tokens often still receive scores close to 1.0, indicating strong relevance. Moreover, although the highest score is not with the exact hallucinated token, it typically appears before or after the exact token. As such, HaMI effectively narrows the region where hallucinated content is likely to occur. This localization ability is valuable in practical settings, as it supports interpretability and downstream correction mechanisms, even in the absence of perfect token-level attribution.

We also notice that the effectiveness of the token selection module may be limited if the generated sequence is short, such as phrase-level generations. In these instances, the entire sequence is typically related to the exact hallucination, and the pre-defined final token may encapsulate the concentrated semantic better.

Thank you for your thoughtful question and we will include the discussion in our paper.

Question #3: Do different types of hallucinations (factual errors, nonsensical responses, contradictions) show different patterns in token selection? How does HaMI perform on fine-grained hallucination taxonomies?

Thank you very much for the question. In the current work, we did not explicitly evaluate HaMI on fine-grained hallucination taxonomies such as factual errors, nonsensical responses, or contradictions. Our primary evaluation focused on general hallucination detection. However, we conducted an empirical analysis using the TruthfulQA dataset [Ref1], which includes class labels for different hallucination types. Due to time limitations, we did not separate the training set by hallucination type; HaMI is optimized on the training data with all hallucination types. At inference time, we included all response types and observed that HaMI's performance was relatively consistent across different hallucination categories. Hope this helps address your concern.

Weakness #1: The best-performing variant (HaMI with semantic consistency) significantly increases computational cost.

We include semantic entropy as uncertainty enhancement feature into HaMI because it provides a strong signal for hallucination detection. However, we agree that integrating the sementic entropy will introduce additional overhead due to the generation of multiple outputs and entailment evaluation. To eliminate the influence on computational efficiency, we also evaluated HaMI*, which relies on only a single generation.

As shown in Table 1 in our paper, HaMI* achieves consistently superior performance on all four datasets, compared to all other single-generation-based baselines, including strong methods like SAPLMA and MARS. Even when compared with multiple-generation-based approaches such as SE, HaMI* outperforms or matches the best results across datasets, demonstrating that our method is not only effective but also adaptable to resource-constrained settings.

References:

• [Ref1] Lin, Stephanie, Jacob Hilton, and Owain Evans. "Truthfulqa: Measuring how models mimic human falsehoods." 60th Annual Meeting of the Association for Computational Linguistics. 2022.

Thank you very much for your valuable and detailed feedback. We are grateful for your positive assessment of our technical contributions and empirical results. Please see our response to your comments one by one below.

Question #1 How does randomly choosing tokens early in training affect the process?

Thank you very much for the question. The optimization in HaMI is analogous to an alternating-optimization approach, in which one of the two sub-problems, token selection and hallucination detection, is alternatively solved, with the parameters in the other one fixed. Please note that, to simplify the optimization, there are no learnable parameters in token selection; we select the tokens with the largest hallucination scores.

It is true that at the beginning of training, the token hallucination scores predicted by the detector are not properly optimized, and therefore, the selected tokens based on these scores may be not that relevant to hallucination detection. However, as more optimization epochs are done, the detector becomes more and more optimal, resulting in better token-level hallucination scores and thus better token selection, and vice versa. Specifically, as training progresses on the datasets used in our experiments, the hallucination scores for tokens in positive bags increase gradually from below 0.5 (observed from the first 10 training steps) to values approaching 1.0, indicating stronger confidence in detecting hallucinated tokens. This phenomenon is observed on all our datasets and is consistently stable with different random parameter initialization seeds.

To further examine the effectiveness of token selection, we randomly selected 100 samples from the Trivia QA dataset and manually annotated the hallucinated tokens. We then tracked the recall rate in the token selection module at different stages of training. We observed that the recall rate of detecting the hallucinated tokens initially was relatively quite high (0.72 after just two training steps), then dropped to around 0.66 for several steps, and steadily improved and became stable thereafter, flattening at a recall rate of about 0.84. This recall rate is remarkable since the full training process is done without using token-level annotations.

These observations suggest that although the top-k token selection strategy may be random in the beginning, the MIL-based joint optimization framework enables HaMI to progressively refine its token selection and effectively learn to identify accurate hallucinated tokens. We will add these results and clarification to the final version to address this concern.

Question #2 & Weekness #1: How does the sequential smoothness loss impact the training and performance of the method?

Thank you very much for raising this question. We added ablation study on four benchmarks with LLaMA-3.1-8B model for the proposed loss functions, including both MIL loss and smoothness loss. Main results on each individual dataset and cross-dataset results are respectively presented in Tables A1 and A2 below. The results are averaged over three independent runs. As evidenced in the tables, incorporating the smoothness loss consistently yields improved performance across all the cases. On average, the smoothness loss consistently enhances HaMI, increasing the performance by 0.7% and 1.1% on the individual-dataset and cross-dataset scenarios, respectively. This demonstrates that the contribution of the smoothness loss is more pronounced in the more challenging cross-dataset setting. These consistent improvements highlight that the smoothness loss plays a rather positive role in helping achieve a more robust and generalizable HaMI, which is particularly valuable in the cross-domain scenarios. We will add these results and this discussion in the final version to clarify the role of this loss.

Table A1. Ablation study on the two loss functions (AUROC).

Table A2. Ablation study on the two loss functions (AUROC) under the cross-dataset setting.

Question #3 & Weakness #2: Would HaMI still achieve the best performance in Table 1 if a single hallucination detector were trained per model, rather than per dataset?... it is not clear whether the method generalizes beyond the question answering domain.

Thank you very much for the question. And yes, as shown in Figure 3(a) in our paper, HaMI maintains superior performance when it is trained per model rather than per dataset, i.e., the detector is trained once on one single dataset and then evaluated on all the other three datasets. This setting is referred to as cross-dataset generalization in the paper. Here we present the experimental results with different settings on four datasets based with the LLaMA-3.1-8B model in Table A3. For comparison, we also present the best results on each dataset from all non-learnable/training-free baselines: p(True), Perplexity, SE, MARS, MARS-SE (extracted from Table 1 of our paper), all of which consistently underperform HaMI across the datasets.

Table A3. Cross-dataset generalization capability compared with non-learnable/training-free baselines.

For your concern on HaMI's capability beyond QA domain in Weakness #2, we would like to clarify that HaMI can be directly employed in summarization tasks, where the target text and task instruction serve as the context prompt and question, respectively, and the summary is treated as the answer. For other tasks, such as consistency checking in storytelling, we can also reformulate them into a QA format for evaluation.

Question #4 & Weakness #3: How does the method compare to single-sequence uncertainty method both in performance and computational effectiveness?

Following your suggestion, we include the single-sequence uncertainty method G-NLL [Ref1] as a baseline method. Since official implementation code is not available, we follow the methodology described in the original paper to reproduce their method. All experiments were conducted using the LLaMA-3.1-8B model across four benchmark datasets.

As shown in Table A4 below, HaMI significantly outperforms G-NLL across all four datasets, with the improvement ranging from 20% to 30%. Notably, even HaMI*, a variant of HaMI that uses only a single generation, can substantially outperform G-NLL, further validating the superior performance of our formulation over G-NLL. The other baseline Perplexity, with its performance reported in Table 1 in our paper, also relies on sentence-level aggregation of token-level negative log-likelihoods. Its performance shows similar trends as G-NLL.

In terms of computational efficiency, HaMI requires additional overhead due to the need to process high-dimensional representations and the training of the detector. In contrast, the single-sequence methods like G-NLL only requires predictive logits and does not involve additional training, making it computationally more efficient than HaMI. However, HaMI remains practical since the inference process operates at the token level without requiring large computational costs. We believe this trade-off between computational cost and substantial accuracy gains justifies the approach.

Table A4. Results for G-NLL and HaMI

Weakness #4: Concerns regarding the ablation study results on token identification.

Thank you very much for the question. As mentioned in our paper, HaMI operates without access to token-level supervision. Consequently, while the model aims to identify tokens associated with hallucinated content, it relies on weak supervision signals derived from sequence/bag-level labels. This inevitably introduces some "noise". It means that the highlighted tokens may not always align precisely with the hallucinated words, but may appear before or after them, which can be meaningful for locating where hallucination appears. We have discussed this phenomenon in Section 5.3 (Lines 306-313). Nevertheless, as we discussed in our response to Question #1, given that the optimization of HaMI does not involve any token-level label information, its recall rate of detecting the true hallucinated tokens (~0.85) is rather high, offering strong supervisory signals for building an accurate hallucination detector.

References:

• [Ref1] Lukas Aichberger, Kajetan Schweighofer, and Sepp Hochreiter. "Rethinking uncertainty estimation in natural language generation." arXiv preprint arXiv:2412.15176, 2024.