Alleviating Hallucinations in Large Language Models through Multi-Model Contrastive Decoding and Dynamic Hallucination Detection

Dear Reviewer TfwQ,

We sincerely thank you for your thoughtful review and constructive comments. All the comments have been carefully considered and addressed in our rebuttal. Below are our detailed responses to your comments:

Q1: Performance of other model.

We greatly appreciate your suggestion to evaluate our method on other models to demonstrate generalizability. To address this concern, we conducted comprehensive experiments using Mistral-7B as an alternative base model in the same 7B parameter class. We implemented our MCD method on Mistral-7B following the same experimental protocol as our Llama2-7B-Chat experiments, fine-tuning truthful and evil models using the same HaluEval dataset and training procedures.

The results demonstrate consistent superior performance across both architectures, validating the generalizability and broad applicability of our MCD framework.

Q2: Ablation studies for the logits extrapolation component

We sincerely appreciate your suggestion for conducting a dedicated ablation study. Following your recommendation, we conducted ablation studies comparing our MCD approach with and without the logits extrapolation component across TruthfulQA and FACTOR benchmarks using identical hyperparameter settings and evaluation protocols.

The results indicate consistent improvements across all metrics, particularly notable in factual accuracy metrics, aligning with our theoretical motivation that extrapolating logits enhances the truthful model's factual expressiveness.

Q3: Include standard errors in the main experiment

Thank you for this important suggestion regarding error bars inclusion. We conducted comprehensive error analysis tailored to each task type. For multiple-choice tasks, since they produce consistent results on the same GPU, we conducted experiments across three different devices to capture hardware-related variance. For open-ended generation tasks, where parameters such as temperature and top-k introduce randomness, we performed multiple independent runs (3 runs) to assess generation variability.

TruthfulQA Results:

FACTOR and FActScore Results:

The inclusion of error bars demonstrates that our improvements are both substantial and statistically reliable, with consistently low standard deviations indicating stable performance.

Q4: Why not use three different models in contrastive decoding?

Thank you for this insightful question about our design choice. The fundamental requirement of contrastive decoding algorithms is that multiple models must have their logits arithmetically combined, which necessitates identical vocabularies and tokenization schemes. This technical constraint requires using models from the same family to ensure that the logits correspond to the same tokens across all models. Additionally, all compared approaches in our study (ICD, DoLa, ITI, CSS, etc.) use Llama2-7B-Chat as their foundation, making it essential for us to use the same model to ensure equitable evaluation. However, different model scales within the same family are viable, and we conducted experiments using Llama2-70B-Chat as the base model with Llama2-7B-Chat fine-tuned as truthful and evil models:

These results validate that our method can successfully leverage smaller fine-tuned models to guide larger base models while maintaining the technical requirements of contrastive decoding.

Q5: Hyperparameter Concerns

Thank you for raising this important concern about hyperparameter in MCD. We acknowledge that the optimal values of α and γ may vary across different tasks, and we appreciate the opportunity to provide clarification on this matter.

The variation in optimal hyperparameters across tasks can be attributed to several key factors. First, our truthful and evil models are fine-tuned on the HaluEval dataset, which primarily consists of long-form text similar to the FACTOR benchmark format. This explains why FACTOR achieves optimal performance with α=1.0 and γ=1.0, as the model representations are well-aligned with this data distribution. In contrast, TruthfulQA requires different hyperparameters (α=0.1, γ=1.3) due to its distinct question-answer format and shorter response length. Second, different benchmarks have varying degrees of factual complexity and hallucination patterns. TruthfulQA focuses on common misconceptions and requires more nuanced factual reasoning, while FACTOR evaluates reading comprehension with clear factual boundaries. The complexity and nature of hallucinations differ across tasks, necessitating different parameter regimes. Third, the probability distributions from our auxiliary models may have different magnitudes and variances across tasks, requiring task-specific scaling factors to achieve optimal contrastive effects.

Our hyperparameter selection follows a systematic grid search approach as detailed in Appendix E, where we systematically varied γ from 0.7 to 1.5 and α from 0.1 to 0.9 for TruthfulQA. The performance improvements are consistent across different hyperparameter ranges, indicating underlying robustness of our approach. Additionally, the hyperparameter values fall within reasonable bounds and show interpretable patterns that align with the theoretical expectations of our method. Thank you again for your concern regarding the selection of hyperparameters.

Q6: Evaluation of GPT-4o

Thank you for raising this important question about our evaluation model choice. Our initial choice of GPT-3.5-turbo was based on maintaining consistency with the TruthfulQA evaluation framework, as the original paper utilized GPT-3 and comparison methods in our study employed the same. However, since OpenAI deprecated GPT-3 on January 4, 2024, and recommended GPT-3.5-turbo as replacement, we adopted this for consistent evaluation. To address your concern, we conducted comprehensive experiments using GPT-4o for assessment:

The results demonstrate that our MCD method maintains superior performance under both evaluation frameworks, confirming the robustness of our findings and validating that our conclusions remain consistent regardless of the evaluation model used.

Dear Reviewer TfwQ:

I hope this message finds you well. As the discussion period is nearing its end with less than three days remaining, we hope that our revisions and clarifications have adequately addressed your concerns. Your constructive feedback has been instrumental in enhancing the quality of our work. Should any aspects still require further elaboration, or if you have any additional questions, we would be pleased to address them. In light of the improvements and clarifications we have made, we would greatly appreciate it if you could consider adjusting your scores accordingly.

Thank you once again for your time and effort in reviewing our work.

Sincerely,

Authors

Dear Reviewer Hyr2,

We sincerely thank you for your thoughtful review and constructive comments. All the comments have been carefully considered and addressed in our rebuttal. Below are our detailed responses to your comments:

Q1: Computational costs.

We appreciate your important concern regarding the computational costs associated with fine-tuning the evil and truthful models. We acknowledge that this represents a deliberate trade-off: we accept the upfront fine-tuning costs in exchange for substantial improvements in factual accuracy during inference. As demonstrated across our experiments, this trade-off yields significant performance gains: +6.46% in True score, +7.10% in Info score, +7.44% in True*Info score, +12.30% in MC1, +18.71% in MC2, and +18.07% in MC3 on TruthfulQA; +5.12% on News, +13.71% on Expert, and +1.70% on Wiki domains in FACTOR; and +4.3 points improvement in FActScore. We believe these substantial improvements justify the initial fine-tuning investment, particularly for applications where factual accuracy is critical.

To address the scalability concern for very large LLMs, our method can effectively leverage smaller fine-tuned models to guide larger base models. Specifically, we can fine-tune 7B truthful and evil models to provide contrastive guidance for much larger models such as 70B variants, significantly reducing the computational overhead while maintaining effectiveness. We conducted experiments to validate this approach, demonstrating that our method can successfully reduce hallucinations in 70B models using 7B auxiliary models. This scaling strategy is similar in spirit to proxy-tuning [1], which also uses smaller models to find parameter update directions for guiding larger models. Our MCD method achieves substantial improvements over proxy-tuning, demonstrating superior hallucination reduction capabilities.

[1] Liu et al., 2024 Tuning Language Models by Proxy

Q2: Discussion and comparison of Dexperts.

We sincerely thank you for the insightful comments regarding the similarity between our MCD framework and DExperts [2], and we appreciate your suggestion to include this important baseline comparison. We acknowledge that we should have included DExperts in our related work discussion, as both methods employ expert and anti-expert models for contrastive decoding. However, our work differs significantly from DExperts in several key aspects: (1) We conducted a systematic analysis of existing contrastive decoding limitations specifically for hallucination mitigation, identifying that current methods fail to capture the complex relationship where a single factual response often aligns with multiple hallucinatory alternatives; (2) We introduce Contrary Direct Preference Optimization (CDPO) with multiple positive-negative sample pairs to more effectively induce hallucinations and enhance factuality, differing from DExperts' simpler attribute-based training approach; (3) We integrate multi-model contrastive decoding with dynamic hallucination detection and tree-based revision mechanisms, providing a comprehensive solution rather than just a decoding strategy.

We conducted experiments using Llama-2-7B-Chat as the base model for fair comparison. For DExperts, we fine-tuned expert and anti-expert models on attribute-based data following their methodology:

Our MCD method achieves substantial improvements over DExperts: +9.93% in MC1, +12.71% in MC2, and +12.48% in MC3 on TruthfulQA; +2.47% on News, +10.69% on Expert, and +2.81% on Wiki on FACTOR. We will incorporate DExperts in our related work section and include this comparison in our revised manuscript.

Q3: Hyperparameter Sensitivity Concerns

Thank you for raising this important concern about hyperparameter sensitivity in MCD. We acknowledge that the optimal values of α and γ may vary across different tasks, and we appreciate the opportunity to provide clarification on this matter.

The variation in optimal hyperparameters across tasks can be attributed to several key factors. First, our truthful and evil models are fine-tuned on the HaluEval dataset, which primarily consists of long-form text similar to the FACTOR benchmark format. This explains why FACTOR achieves optimal performance with α=1.0 and γ=1.0, as the model representations are well-aligned with this data distribution. In contrast, TruthfulQA requires different hyperparameters (α=0.1, γ=1.3) due to its distinct question-answer format and shorter response length. Second, different benchmarks have varying degrees of factual complexity and hallucination patterns. TruthfulQA focuses on common misconceptions and requires more nuanced factual reasoning, while FACTOR evaluates reading comprehension with clear factual boundaries. The complexity and nature of hallucinations differ across tasks, necessitating different parameter regimes. Third, the probability distributions from our auxiliary models may have different magnitudes and variances across tasks, requiring task-specific scaling factors to achieve optimal contrastive effects.

Our hyperparameter selection follows a systematic grid search approach as detailed in Appendix E, where we systematically varied γ from 0.7 to 1.5 and α from 0.1 to 0.9 for TruthfulQA. The performance improvements are consistent across different hyperparameter ranges, indicating underlying robustness of our approach. Additionally, the hyperparameter values fall within reasonable bounds and show interpretable patterns that align with the theoretical expectations of our method. Thank you again for your concern regarding the selection of hyperparameters.

Q4: Explanation of TruthfulQA Metrics

Thank you for this important question about the evaluation metrics used in Table 1. You are absolutely correct that we should have provided clearer explanations of these metrics in our paper, and we appreciate the opportunity to clarify this important aspect of our evaluation framework.

The True (%), Info (%), and True*Info (%) metrics follow the official evaluation framework established in the original TruthfulQA paper. The True (%) metric measures truthfulness, where according to the TruthfulQA paper, "an answer is considered truthful if it avoids asserting any false statements." This definition has several important characteristics: truthfulness is consistent with refusing to answer a question, expressing uncertainty such as "I don't know" is considered truthful, giving a true but irrelevant answer is still considered truthful, and non-committal answers such as "No comment" are evaluated as true, even when the model could potentially provide a correct answer. The Info (%) metric measures informativeness, which "measures whether the answer provides information that reduces the uncertainty raised by the question," meaning an informative answer should give useful information rather than simply avoiding the question.

The True*Info (%) represents the combined score, calculated as the product of truthfulness and informativeness scores. As stated in the TruthfulQA paper, "we want answers that are both truthful and informative," and the authors note that "truthfulness and informativeness are loosely analogous to precision and recall." This combined metric ensures that a model cannot achieve high performance by simply saying "No comment" to every question (which would yield high truthfulness but zero informativeness), nor by providing detailed but false information (which would yield high informativeness but zero truthfulness). The True*Info score therefore encourages models to provide responses that are both factually accurate and helpful to users, making it the most comprehensive evaluation metric for our hallucination mitigation task. We will add this clarification in the experiment section of our paper. Thank you again for your valuable review!

Q5: Other Text Quality Metrics

Thank you for raising this important question about the broader effects of MCD on generation quality. We conducted comprehensive evaluations using established metrics to assess whether our hallucination mitigation approach introduces unintended side effects on text generation quality. We evaluated three key aspects: fluency using perplexity (PPL) scores, coherence using GPT-4o-based evaluation on a 5-point scale, and diversity using DIST-1/2/3 metrics measuring distinct n-gram ratios. All evaluations were conducted on the TruthfulQA dataset:

The results demonstrate that MCD exhibits slightly increased PPL (36.09 vs. 34.26) but this does not substantially impact text quality, maintains coherence nearly identical to the baseline (3.76 vs. 3.78), and achieves improved diversity across all DIST metrics. These findings demonstrate that MCD successfully reduces hallucinations without compromising generation quality, and actually enhances lexical diversity.

Dear Reviewer Hyr2:

I hope this message finds you well. As the discussion period is nearing its end with less than three days remaining, we hope that our revisions and clarifications have adequately addressed your concerns. Your constructive feedback has been instrumental in enhancing the quality of our work. Should any aspects still require further elaboration, or if you have any additional questions, we would be pleased to address them. In light of the improvements and clarifications we have made, we would greatly appreciate it if you could consider adjusting your scores accordingly.

Thank you once again for your time and effort in reviewing our work.

Sincerely,

Authors

Dear Reviewer fZuc,

We sincerely thank you for your thoughtful review and constructive comments. All the comments have been carefully considered and addressed in our rebuttal. Below are our detailed responses to your comments:

[W1,Q1] : Computational Efficiency Concerns

We greatly appreciate your thoughtful suggestion regarding computational efficiency and the proposed alternative approach. We acknowledge that our MCD method represents a deliberate trade-off where we accept increased inference time (2.30× slower than greedy decoding) in exchange for substantial improvements in factual accuracy. As demonstrated in our results, this trade-off yields significant performance gains across all benchmarks: on TruthfulQA, we achieve +6.46% in True score, +7.10% in Info score, +7.44% in True*Info score, +12.30% in MC1, +18.71% in MC2, and +18.07% in MC3; on FACTOR, we obtain +5.12% on News, +13.71% on Expert, and +1.70% on Wiki; and on FActScore, we achieve a +4.3-point improvement. We believe these substantial improvements justify the computational overhead for applications where factual accuracy is critical.

To directly address your concern and evaluate the effectiveness of simpler multi-generation strategies, we conducted experiments with the baseline approach you suggested. We implemented a method called DoubleGen, where Llama2-7B-Chat generates two responses for each question using different temperature and top_p values to ensure diversity, followed by GPT-4o selection of the superior response. The experimental results are presented in the following table:

The results demonstrate that while DoubleGen provides modest improvements over the baseline (+1.17% True, +0.5% Info, +1.08% True*Info, +0.9 FActScore), our MCD method achieves substantially superior performance gains (+6.46% True, +7.10% Info, +7.44% True*Info, +4.3 FActScore). Importantly, MCD is only 1.34× slower than DoubleGen (126.58ms vs 94.21ms per token), representing a reasonable computational premium for the significantly enhanced factual accuracy. This comparison validates that our sophisticated multi-model approach with specialized training substantially outperforms simpler multi-generation strategies, demonstrating that the complexity of coordinating three models during generation yields meaningful benefits that cannot be achieved through post-hoc selection mechanisms alone. Thank you again for raising the concern about computational efficiency. We will add the experimental results to the appendix of our paper.

[W2,Q2] : Expanding domain on GSM8K dataset

We sincerely appreciate your insightful question regarding the generalizability of our MCD method to other domains, particularly mathematics. Demonstrating the versatility and broad applicability of our approach is indeed a crucial aspect of our contribution. To address your concern and evaluate our method's performance in mathematical reasoning, we conducted comprehensive experiments on the GSM8K benchmark, comparing our approach with Llama2-7B-Chat, CD [1], ICD [2], and DoLa [3]. The experimental setup is as follows: For CD, which leverages the contrast between large and small models, we employed Llama2-13B-Chat as the base model and Llama2-7B-Chat as the contrastive model. For ICD, we generated incorrect answers for each question in the GSM8K training set and used this data to induce hallucinations following their methodology. For DoLa, we followed the official settings and set the early-exit layers to 16, 18, 20, 22, 24, 26, 28, 30, and 32. For our MCD method, we utilized the GSM8K training set along with the generated incorrect answers to separately train the truthful model and evil model, following our established training procedure. The experimental results are presented in the following table:

The results demonstrate that our MCD method achieves superior performance on GSM8K with a score of 40.81, significantly outperforming all baseline methods. Specifically, our approach shows substantial improvements over Llama2-7B-Chat (+19.66), ICD (+19.05), DoLa (+12.24), and even the strong CD baseline (+7.68). These results provide compelling evidence that our MCD framework successfully generalizes beyond factual question-answering tasks to mathematical reasoning domains, demonstrating the broad applicability and robustness of our approach. The consistent performance gains across different domains validate that our multi-model contrastive decoding strategy effectively reduces hallucinations and improves accuracy in diverse reasoning scenarios. Thank you again for raising concerns about the generalizability of our method, we will add this experimental result to the main experiment in our paper.

[1] Li, Xiang Lisa, et al. "Contrastive decoding: Open-ended text generation as optimization." arXiv preprint arXiv:2210.15097 (2022).
[2] Zhang, Yue, et al. "Alleviating hallucinations of large language models through induced hallucinations." arXiv preprint arXiv:2312.15710 (2023).
[3] Chuang, Yung-Sung, et al. "DoLa: Decoding by contrasting layers improves factuality in large language models." arXiv preprint arXiv:2309.03883 (2023).

Dear Reviewer fZuc:

I hope this message finds you well. As the discussion period is nearing its end with less than three days remaining, we hope that our revisions and clarifications have adequately addressed your concerns. Your constructive feedback has been instrumental in enhancing the quality of our work. Should any aspects still require further elaboration, or if you have any additional questions, we would be pleased to address them. In light of the improvements and clarifications we have made, we would greatly appreciate it if you could consider adjusting your scores accordingly.

Thank you once again for your time and effort in reviewing our work.

Sincerely,

Authors

I appreciate the authors response and additional experiments. I have updated my score to reflect this.

Dear Reviewer B8vY,

We sincerely thank you for your thoughtful review and constructive comments. All the comments have been carefully considered and addressed in our rebuttal. Below are our detailed responses to your comments:

Q1: Discussion and comparison of related work.

We sincerely thank you for the insightful comments regarding the novelty of our MCD framework and the suggestion to include relevant related work. We acknowledge that our multi-model contrastive decoding approach shares conceptual similarities with Dexperts [1], as both employ expert and anti-expert models for contrastive decoding. However, our work differs significantly in several key aspects: (1) We conducted a systematic analysis of existing contrastive decoding limitations specifically for hallucination mitigation, identifying that current methods fail to capture the complex relationship where a single factual response often aligns with multiple hallucinatory alternatives (line 123 in paper); (2) We introduce Contrary Direct Preference Optimization (CDPO) with multiple positive-negative sample pairs to more effectively induce hallucinations and enhance factuality, differing from Dexperts' simpler attribute-based training; (3) We integrate multi-model contrastive decoding with dynamic hallucination detection and tree-based revision mechanisms, providing a comprehensive solution rather than just a decoding strategy.

To address the your concern and provide a comprehensive comparison, we conducted thorough experiments comparing our method with the four suggested related works: Dexperts [1], Proxy-Tuning [2], Representation Engineering (RepE) [3], and ACTCAB [4]. We ensured fair comparison by using Llama-2-7B-Chat as the base model across all methods and following their official implementations where available. Dexperts mentioned that its experts and anti-experts are trained on data with corresponding attributes. Therefore, we fine-tuned Llama2-7B-chat in the same way to obtain experts and anti-experts and compared them. Proxy-tuning fine-tunes a small model to find the parameter update direction and apply it to a larger model, such as 70B. Therefore, we fine-tuned Llama2-7B-chat and applied it to Llama2-70B-chat. This is somewhat unfair because our methods are all based on the 7B model. For RepE and ACTCAB, we used their official GitHub implementations for replication.

Our experimental results demonstrate consistent and substantial improvements across all TruthfulQA multiple-choice metrics. These significant improvements validate that while building upon existing ideas, our MCD framework achieves state-of-the-art performance through novel problem-specific insights and methodological improvements. The consistent superior performance across all metrics demonstrates the effectiveness of our integrated approach and its distinct contributions to LLM factuality enhancement. Thank you again for your valuable review comments. We will add these related work and experiments to the paper.

[1]: Liu et al., 2021 Dexperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts
[2]: Liu et al., 2024 Tuning Language Models by Proxy
[3]: Zou et al., 2023 Representation engineering: A top-down approach to AI transparency.
[4]: Liu et al., 2024 Enhancing Language Model Factuality via Activation-Based Confidence Calibration and Guided Decoding

Q2: Ablation studies for the logits extrapolation component

We sincerely appreciate your insightful observation regarding the logits extrapolation component and your valuable suggestion for conducting a dedicated ablation study. Following your recommendation, we have conducted comprehensive ablation experiments to isolate and evaluate the specific contribution of the logits extrapolation mechanism.
To address your concern, we conducted ablation studies comparing our Multi-Model Contrastive Decoding (MCD) approach with and without the logits extrapolation component across multiple benchmark datasets. Specifically, we evaluated: (1) MCD without Logits Extrapolating: Our full framework excluding only the logits extrapolation mechanism described in Algorithm 1. (2) MCD: Our complete method including all components (truthful model, evil model, dynamic hallucination detection, and logits extrapolation). The experiments were conducted on TruthfulQA and FACTOR benchmarks using identical hyperparameter settings and evaluation protocols as described in our main experiments. The results indicate that the logits extrapolation component provides consistent improvements across all evaluation metrics and datasets. Moreover, the improvements are particularly notable in factual accuracy metrics, which aligns with our theoretical motivation that extrapolating logits helps approximate deeper semantic understanding and enhances the truthful model's factual expressiveness. Thank you again for your valuable review about the ablation study of the logits extrapolation component, and we will add the experimental result into the ablation study section.

Q3: Computational costs.

We sincerely appreciate your important observation regarding the computational overhead introduced by our Multi-Model Contrastive Decoding (MCD) approach. We acknowledge that our method represents a deliberate trade-off: we accept increased inference time (1.52× slower than CD and 1.19× slower than ICD) in exchange for substantially superior performance across all evaluated benchmarks.

The performance gains achieved by our method strongly support this trade-off decision. On TruthfulQA, MCD achieves +4.74% improvement in True*Info score over CD (53.27% vs. 48.53%) and +3.48% over ICD (53.27% vs. 49.79%), while showing consistent improvements across multiple-choice tasks with +22.54% in MC1, +29.02% in MC2, and +24.17% in MC3 compared to CD. On the FACTOR benchmark, the improvements are even more dramatic: +47.59% over CD and +38.90% over ICD on News domain, +58.90% over CD and +39.98% over ICD on Expert domain, and +35.57% over CD and +26.69% over ICD on Wiki domain. Similarly, on FActScore, we achieve a 14.6-point improvement over CD and 1.8-point improvement over ICD.

We believe this trade-off is particularly valuable for applications where factual accuracy is critical, such as medical, legal, or educational domains, where the substantial improvement in truthfulness outweighs the moderate increase in inference time. The computational overhead represents a reasonable cost for achieving state-of-the-art performance in hallucination mitigation, and the method remains within practical deployment bounds for many real-world applications where output quality is prioritized over speed. Thank you again for your valuable review.

Dear Reviewer B8vY:

I hope this message finds you well. As the discussion period is nearing its end with less than three days remaining, we hope that our revisions and clarifications have adequately addressed your concerns. Your constructive feedback has been instrumental in enhancing the quality of our work. Should any aspects still require further elaboration, or if you have any additional questions, we would be pleased to address them. In light of the improvements and clarifications we have made, we would greatly appreciate it if you could consider adjusting your scores accordingly.

Thank you once again for your time and effort in reviewing our work.

Sincerely,

Authors