Steer LLM Latents for Hallucination Detection

We greatly appreciate your thoughtful and valuable comments. Below, we provide detailed responses to each of your questions and comments.

A1. Optimal separability of the representation

Thank you for your insightful question! We acknowledge that there can indeed be failure cases where the representations of truthful and hallucinated contents overlap. For instance, in Table 1, the performance on the NQ Open dataset (76.1%) is relatively lower compared to other datasets, suggesting weaker separability. This is likely due to the inherently challenging QA task, which can make perfect separation harder.

We'd like to emphasize that our goal is not to guarantee perfect separability, but rather to improve the separability relative to the pre-trained model. This improvement holds consistently across all datasets, even if the final representation space is not perfectly disentangled. As a result, our method enables more reliable deployment compared to prior approaches that rely on fixed pre-trained embeddings.

A2. $\epsilon$ in Eqauation (7)

The value of $\epsilon$ is defined in Appendix L602 where it is set to 0.05, following [1] that demonstrated its effectiveness. For improved clarity, we will move this detail to the main paper. Thank you for pointing this out!

[1] Caron et al., "Unsupervised Learning of Visual Features by Contrasting Cluster Assignments," NeurIPS 2020

A3. Different pseudo-label assignment strategies

We appreciate the reviewer for highlighting this important point.

Our motivation for introducing the optimal transport (OT) is that we aim to generate pseudo-labels that align with the class distribution of unlabeled LLM generations, which is inherently imbalanced. Traditional pseudo-labeling methods [2,3] can suffer from class imbalance, leading to biased predictions—either toward majority classes or, in some cases, unstable predictions favoring minority classes. This leads to a mismatch between the class distribution of pseudo-labels and the true underlying distribution, which can degrade both pseudo-label quality and overall detection performance.

In contrast, our OT-based clustering approach introduces cluster-wise regularization through a constrained optimization framework, ensuring that cluster sizes align with the underlying or estimated class distribution.

In comparison, we evaluate two pseudo-labeling baselines—nearest centroid [2] and confidence thresholding [3] on LLaMA-3.1-8b using AUROC. We will include these comparisons and discussions in the main paper.

[2] Rebuffi et al., "iCaRL: Incremental Classifier and Representation Learning," CVPR 2017

[3] Sohn et al., "FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence," NeurIPS 2020

A4. References on LVLM latent space steering

Thank you for highlighting these interesting works in the LVLM field! We agree that comprehensively studying latent feature steering in LVLMs is also crucial, offering valuable insights that can inform both communities and help shape future directions. We are happy to include the following relevant works from the LVLM literature. We will also ensure a more thorough literature review to identify any additional related studies.

[4] Liu et al., "Reducing Hallucinations in Vision-Language Models via Latent Space Steering," ICLR 2025

[5] Yang et al., "Nullu: Mitigating Object Hallucinations in Large Vision-Language Models via HalluSpace Projection," CVPR 2025

[6] Chen et al., "ICT: Image-Object Cross-Level Trusted Intervention for Mitigating Object Hallucination in Large Vision-Language Models," CVPR 2025

[7] Duan et al., "TruthPrInt: Mitigating LVLM Object Hallucination Via Latent Truthful-Guided Pre-Intervention," ArXiv 2025

A5. Can TSV mitigate hallucination?

Thank you for the interesting question. TSV is designed to learn a representation space specifically for detection (i.e., classification), which is fundamentally different from mitigation (i.e., generation). As a result, it is not straightforward to directly apply a detection-trained TSV to a mitigation task.

However, TSV offers a plug-and-play mechanism for pre-trained LLMs. In practice, one can first apply TSV to classify truthful vs. hallucinated outputs from unlabeled generations in the wild, and then leverage these predictions to learn a steering vector for the mitigation task [8]. This perspective opens up a promising direction for hallucination mitigation under unsupervised/limited-label settings, where extensive supervision is often impractical.

[8] Li et al., "Inference-Time Intervention: Eliciting Truthful Answers from a Language Model," NeurIPS 2023

We sincerely appreciate your valuable feedback and insightful questions. Below, we provide detailed responses to each of your points. All the added experiments are performed with LLaMA-3.1-8b.

A1. Longer generations

Great point. In this work, we focus on short-form QA (phrase and sentence-level) which remains challenging, as evidenced by suboptimal performance reported in prior literature. We adopt this setting to ensure fair comparison with existing benchmark studies. That said, we fully agree that extending to long-form generation is a critical direction for real-world applications. We view our current work as a necessary step toward this goal.

One natural solution is to decompose the long-form generation into multiple QA pairs and verify each pair individually. This reframes the problem as hallucination detection over a set of QA pairs, where TSV can be applied. We believe this warrants a deeper investigation on its own, and we appreciate you highlighting it!

A2. Deployment in practice

TSV is a lightweight, plug-and-play hallucination detection module that can be integrated with any production LLMs. We first perform a standard forward pass using the original model parameters (without TSV) to generate a response. Then, during a second pass, we apply TSV at an intermediate layer and compute a truthfulness score using the steered representation. It can be flexibly toggled on/off, preserving the original model behavior when hallucination detection is not required. We will clarify these deployment steps more explicitly in the revision.

A3. Class distribution mismatch

Thank you for bringing up this point! We manually construct exemplar sets under three different scenarios: (1) distribution aligned with the unlabeled generations, (2) uniform distribution, and (3) distribution reversed from the unlabeled generations. We observe a slight performance decline in scenarios with distribution mismatches, but the overall performance remains competitive.

A4. Extension to the span-level hallucination detection

Great question! We agree this is an exciting direction, which requires localizing exactly which tokens or phrases are hallucinated. This is particularly challenging, as it requires adapting TSV to separate hidden states at different token positions.

One promising direction is to identify salient entities—often the primary source of hallucinations—and apply TSV before and after each entity span, then measure the resulting shift in hidden states or detection scores to infer potential hallucinations at the corresponding span. These challenges require deeper investigation. But again, we agree this could significantly increase practical utility and will consider it in future work.

A5. Ablation on LoRA

TSV is more effective for hallucination detection as it directly shapes the representation space while being more parameter-efficient. Additionally, we applied selective LoRA updates to specific model components within the same layer as TSV, and found these targeted adjustments still underperformed TSV, as shown below.

A6. Applying TSV to multiple locations

We appreciate your insights. We applied TSV to multiple layers or MHA components of LLaMA-3.1-8b on TruthfulQA and observed slight performance gains. However, a single-layer TSV already performs strongly, suggesting it is sufficient for effective hallucination detection and highlights the efficiency of our approach.

A7. Performance on the quantized model

Thank you for the great question. We experiment with an 8-bit quantized LLaMA-3.1-8b model$^1$ and observe a slight drop in performance. However, the overall results remain strong, indicating that our method is robust to quantization.

$^1$ https://huggingface.co/docs/transformers/main/en/quantization/bitsandbytes

A8. Transferability for different models

Thank you for the suggestion! While we demonstrate that TSV performs well across various models and scales, we believe full transferability is unlikely. This is because differences in architecture, training framework, and pretraining data across models (e.g., LLaMA vs. Qwen) lead to fundamentally different representation spaces, making direct application of TSV challenging. However, a promising future direction could be to train an adapter that aligns the representations between models of different architectures or scales.

A1. Steering vector for LLMs

Thank you for your insights! While it is true that the steering vectors have been explored in LLMs, our key contribution lies in how we specifically design them for hallucination detection, as recognized by you and Reviewers iT9L and tv5M. Sections 4.2 and 4.3 illustrate the novel algorithm that leverages both a few labeled and unlabeled LLM generations with an optimal transport-based pseudo-labeling framework for training. To our knowledge, this is the first work to formulate hallucination detection in this way.

Moreover, we emphasize that our contribution lies not only in the specific method but also in the perspective shift we offer to the hallucination detection community, which has largely relied on fixed pre-trained embeddings—despite their limitations. Our work challenges this status quo by demonstrating that actively steering the latent space—even with a lightweight vector—can substantially improve detection performance. We believe this perspective shift is fundamentally significant, as it opens up a new design space and encourages the community to look beyond pre-trained embeddings toward representation shaping as a core principle.

A2. Evaluation with other metrics

Thank you for the suggestion! In addition to AUROC, we report AUPRC, F1 score, precision, and recall at the generation level, consistent with prior works. We include results on TruthfulQA with LLaMA-3.1-8b, where we found that our method maintains a consistent advantage across all metrics.

A3-1. Robustness to pseudo-label noise

Thank you for bringing this up! As requested, we include the robustness analysis of pseudo-label noise for the selected unlabeled data ($K=128$). Specifically, with the same selected dataset on TruthfulQA, we gradually add more noise to their pseudo labels by manually flipping some of the correct labels. The resulting relationship between the pseudo-label noise ratio and the hallucination detection AUROC on LLaMA-3.1-8b is shown as follows, where our method remains relatively robust under increasing noise conditions.

A3-2. Quality of exemplar set, and how does the content of exemplars impact the results?

Another insightful point! To understand the quality, we randomly sample the exemplar set using different seeds and annotate them w.r.t. factuality. Manual inspection confirms that the exemplars are high-quality and cover diverse content. The TruthfulQA dataset contains 817 questions across 38 categories with an imbalanced distribution, making it a meaningful testbed for examining how content affects performance. To further verify, we construct exemplar sets on LLaMA-3.1-8b using three strategies, with imbalance ratio $\gamma$ defined as the ratio of the most to least frequent category counts.

Even in cases where the selected exemplars are biased toward a particular content, the test performance remains consistent. This reinforces the robustness of our approach.

We emphasize that our approach achieves strong performance with a very small labeled exemplar set (e.g., 32), which makes manual curation and quality control highly feasible in practice. Since the cost of labeling such a small set is minimal, our work offers a reliable and efficient solution for hallucination detection in real-world applications.

A4. Hyperparameters

We emphasize that TSV requires minimal hyperparameter tuning, making it practical and easy to deploy. Key hyperparameters (e.g., steering strength) are lightweight to tune and we provide default settings (see Table 6 in Appendix) that generalize well across all models and datasets.

In fact, all experiments in our paper use hyperparameters selected using the validation set of TruthfulQA with LLaMA-3.1-8b, and these settings are applied uniformly without re-tuning to all experiments, while still achieving state-of-the-art performance.

Moreover, our ablation in Figure 3b demonstrates that TSV’s performance is not overly sensitive to the choice of hyperparameters. The robustness of TSV is further supported by our generalization results in Figure 4, which show consistent performance across diverse data distributions, even with fixed hyperparameters. This significantly reduces the burden of hyperparameter tuning in real-world use.