To Trust or Not to Trust? Enhancing Large Language Models' Situated Faithfulness to External Contexts

Weakness 2: Sensitivity to Confidence Thresholds

• Sensitivity to RCR: We agree that the RCR methods are sensitive to thresholds and no universally optimal threshold applies across all tasks for a single method. This aligns with our discussion in Section 5.3, where we highlight that the rule for RCR can be flawed (e.g. threshold is suboptimal), presenting a limitation of these methods.

• Mitigation through Threshold Tuning: This limitation can be partially mitigated by tuning the threshold on a development dataset for each task, as demonstrated in Table 3. However, even with threshold tuning, RCR still underperforms compared to the SCR method (see Table 6) for GPT-4o, despite the advantage of leveraging additional data. This could be due to the noise in confidence estimations and the new bias of updated rules. SCR could avoid this pitfall by performing confidence reasoning internally in the text space.

• Additional Analysis Provided: To provide a more comprehensive analysis as you suggest, we include plots illustrating how performance varies as thresholds range from 0 to 1 for each task, offering deeper insights into the behavior of RCR across tasks. As shown in Figure 3 in Appendix D.3, no universally optimal threshold applies to all tasks; instead, each task requires a tailored threshold for optimal performance. In contrast, SCR methods inherently avoid these threshold-tuning challenges, offering a more robust and consistent alternative.

We appreciate the reviewer bringing up this point, as it reaffirms the value of our discussion on the limitations of RCR and our comparison with SCR as a more effective alternative.

Thank you for your time and effort in reviewing our work and for providing such insightful feedback. Regarding

Weakness 1: Limited Evaluation of Baselines, missing ActiveRAG

Thank you for bringing up ActiveRAG. We acknowledge that ActiveRAG can be considered an instance within the broader scope of the RCR approaches. However, the presence of similar concepts does not undermine the novelty of our work.

The core novelty of our contribution lies in systematically formalizing the SCR and RCR solution spaces under a newly defined and important problem, going beyond the scope of any individual method. Our work introduces a comprehensive framework that not only categorizes different approaches into SCR and RCR, but also critically analyzes their strengths and limitations, offering new perspectives and novel improvements. This broader contribution, including formalization and framework-driven evaluation, extends beyond individual methods. Consequently, the conceptual overlap of one single method within our framework with approaches like ActiveRAG does not detract from the significance of our work.

We are also happy to provide a detailed analysis of how ActiveRAG fits within our framework and acknowledge its relevance more explicitly in the paper. Below, we discuss its connections and differences from our work from two perspectives

• Methodologies: Their methodology could be categorized under RCR approaches. Under our setup, it will resemble the similar idea of InternalConf. The difference is in ways of confidence estimation – ActiveRAG utilizes external context when any one token’s probability is below a threshold, while the InternalConf utilizes external context when the probability of the whole answer is below a threshold
• Motivations: The problem setup and focus of ActiveRAG are fundamentally different from ours. ActiveRAG iteratively retrieves documents to refine model answers, aiming to enhance performance through external retrieval loops. It primarily focuses on correct or irrelevant contexts, neither accounting for incorrect contexts nor providing insights into the model’s true ability to resolve knowledge conflicts. In contrast, our work evaluates the model’s ability to resolve knowledge conflicts between its internal knowledge and external contexts in a clean, controlled environment, focusing on model reliability and decision-making under these conditions.

Results on GPT-4o and Llama-3 are shown below. Overall, ActiveRAG’s performance is close to its conceptual counterpart, InternalConf, and does not impact our key findings:

1. LLM performs SCR better than RCR on strong models like GPT-4o
2. LLMs perform RCR better than SCR on smaller models like Llama-3-8B, while CR-DPO could enhance SCR.

Thank you for this great suggestion and we have added the discussion of ActiveRAG into the part of InternalConf.

Thank you again for your thoughtful comments and feedback. We sincerely hope that our response and revisions have addressed your concerns. If possible, could you please consider adjusting your scores in light of these updates? If you still have any remaining concerns or questions, please don’t hesitate to share them. We would be more than happy to discuss them further to ensure clarity and address any remaining issues.

Thank you for your effort in reviewing our work and for the positive feedback—it is truly encouraging, and we deeply appreciate it.

We also appreciate you pointing out the typo and citation issues, which we have now corrected.

Regarding the quality of training data, we employ two key strategies to ensure robustness:

1. Adding In-Context Examples: We incorporate in-context examples when sampling COTreasoning paths from the model. These examples help guide the model to produce more coherent reasoning paths compared to using instructions alone.
2. Post-Filtering: To further enhance quality, we apply a post-filtering step to verify whether the ground-truth answer appears at the end of the selected CoT reasoning path. Specifically, we calculate the token-level recall score of the ground-truth answer within the last 3 $\times$ the length of the ground-truth answer tokens. If the recall score exceeds a threshold (0.5 in practice), we consider the reasoning path valid; otherwise, we discard the data point.

We generated 5,000 data points each for TriviaQA, NaturalQA, and RedditQA, and 2,895 data points for PopQA. In experiments where we sampled one CoT reasoning path per data point, this yielded a total of 17,895 CoT reasoning paths, of which only 375 (2.1%) were deemed invalid and discarded.

Thank you again for your suggestion. We have now included these details in Appendix A.2.

Thank you for your time and effort invested in reviewing our paper. We are grateful for your constructive comments and positive reception to our work, which has been really encouraging.

Regarding

Weakness: Additional computational

We agree that these methods may introduce additional computational overhead during inference. For RCR methods, generating both the internal answer and the context-based answer requires running the model twice. The computational overhead of the ExplicitSCR comes from the COT reasoning. Future research can potentially explore ways to distill the confidence reasoning process into a unified model. We have included a future work section, currently in the appendix to discuss this aspect.

Q1: calibration scores.

Thanks for the great suggestion! Below, we present the ECE scores and AUC-ROC scores for Llama-3 on TriviaQA (complete results for all models and tasks are included in Appendix D.2). The columns represent ECE, AUC-ROC, and overall situated faithfulness, while the rows correspond to the InternalConf, InternalConf+Calibration, and ContextConf methods. As shown in the table, the model demonstrates significantly better confidence estimation for its internal knowledge compared to external contexts. Additionally, while calibration improves the ECE score, this improvement does not necessarily translate to better situated faithfulness.

Results are with %

Q2: Typo.

Thanks for pointing that out and we have modified that.

Thanks for your insightful comments and positive review of our work, and we really appreciate that.

Regarding the two weaknesses:

We acknowledge that the performance of both ImplicitSCR and ExplicitSCR is close and the choice can be dataset-dependent. In this response, we aim to clarify the key takeaways, address concerns about dataset dependence, and provide practical guidance for real-world usage.

• Takeaways: we want to emphasize our major high-level takeaways are: 1) LLM have the ability to resolve knowledge conflict, evidenced by large improvements of both RCR and SCR methods to direct input augmentation. 2) SCR outperforms RCR for strong models 3) CR-DPO could further enhance RCR

• Dataset-dependence: We acknowledge that the best-performing method may vary depending on the dataset. However, we believe that averaging situated faithfulness across all datasets provides a reasonable indicator of a method’s general performance. As the diversity of datasets increases, it becomes less likely for a single method to excel universally. This approach aligns with standard practices in other benchmarks, such as MMLU, where overall results are compared despite domain-specific variations.

• Practical Usage Instructions: To address real-world applicability, we have included practical guidelines for selecting a method:

1. If a development set is available, users can test ExplicitSCR and ImplicitSCR on the dev set to identify the best-performing method.
2. If efficiency is a priority, ImplicitSCR is recommended due to its lower computational cost.
3. If explainability is critical, ExplicitSCR is a better choice as it provides interpretable chain-of-thought reasoning.

Thank you again for your thoughtful feedback. We hope the addition of usage instructions in Appendix F.1 further supports practitioners in applying these methods effectively.