FacLens: Transferable Probe for Foreseeing Non-Factuality in Large Language Models

Dear Reviewer aR8Q,

We sincerely appreciate your effort in reviewing our paper. We are glad that you found it clear and recognized its excellent efficiency and transferability. Here we will carefully address your concerns and questions in detail.

1. Performance Improvement

- Clarification on Figure 3: There may have been a misunderstanding regarding the meaning conveyed by Figure 3. The similar performance of $f_m$ and $f_{mix}$ is, in fact, a positive outcome. Both $f_m$ and $f_{mix}$ represent our FacLens model, differing only in training data. Our goal in comparing $f_m$ and $f_{mix}$ is to demonstrate FacLens’s transferability across LLMs. The similarity in their performance supports FacLens’s transferability across LLMs.

- Clarification on Table 1: Table 1 shows that FacLens achieves clear performance gains over most baselines. While the performance gains over LoRA and Self-Evaluation are slightly smaller, FacLens significantly outperforms both of them in terms of training efficiency (see Table 2), which is crucial for practical applications. Moreover, as shown in Figure 2, we compared FacLens with post-hoc methods. Despite post-hoc methods having access to additional information (i.e., the generated answers), FacLens still performs better.

2. Experimental Datasets

We adopted widely recognized datasets frequently used in the factual QA research, including the Natural Questions dataset, which is sourced from real user queries. Furthermore, we conducted human evaluation to collect real queries to further provide a practical assessment.

Thank you very much for the valuable suggestion on incorporating more QA datasets! We supplemented main experiments with TriviaQA and HotpotQA, which are more complex QA datasets involving reasoning. The results demonstrate that our FacLens continues to perform well, and the domain adaptation is still effective.

Comparison between FacLens and NFP baselines on TriviaQA and HotpotQA

Note: "-" indicates that the method is not applicable to the dataset. Compared to the most competitive baselines (i.e., LoRA and Self-Evaluation), FacLens demonstrates a significant improvement in training efficiency (see Table 2 in our paper).

Evaluation of domain adaptation for cross-LLM FacLens on TriviaQA and HotpotQA

Through supplementing the suggested experiments, we found another interesting observation. We observed that for the more complex factual QA tasks (TriviaQA and HotpotQA), FacLens prefers hidden question representations from the later layers. This may be because LLMs tend to engage in deeper thinking when facing complex factual questions. We will incorporate this interesting observation to the next version of our paper.

---

We sincerely thank you again for the constructive comments and the recognition of the strengths of our work. We would be delighted to engage in further discussion and address any additional questions you may have.

Sincerely,

All authors

Dear Reviewer aR8Q,

Thank you very much for taking the time to review our paper. We sincerely appreciate your valuable comments and have provided detailed responses to address your concerns.

We would greatly appreciate it if you could let us know whether our responses have addressed your concerns. If there are any remaining questions, we would be more than happy to provide further clarification and work towards resolving them.

Thank you once again for your time, effort, and consideration, and we look forward to your valuable feedback.

Best regards,

All Authors

Dear Reviewer aR8Q,

With the discussion period nearing its end, we notice that we have not yet received your feedback on our responses.

We tried our best to address your concerns, providing detailed explanations and additional experimental results to clarify your concerns. We kindly request you to check our responses and reconsider your assessment of our paper.

Thank you once again for your time and effort, and we look forward to hearing from you.

Best regards,

All authors

Dear reviewer yYWm,

We sincerely appreciate your constructive review and are delighted that you found our work interesting. Below, we will provide detailed responses to the questions raised.

1. Explanation About the Tansferability of FacLens

- How to Explain Observations in Section 4.2

Current LLMs commonly share the similar architecture (i.e., Transformer), training scheme (i.e., pre-training, SFT, and RLHF), and overlapping knowledge resource (i.e., training corpora), e.g., LLMs such as Qwen and LLaMA, use a mix of data from publicly available sources. This suggests that LLMs are likely to have similar cognition styles.

In human cognition, individuals with similar cognitive styles show similar brain activity when performing the same task. This inspires us to explain why hidden question representations from different LLMs (i.e., LLMs' brain activation during question-thinking) exhibit similar NFP patterns.

- Why Unsupervised Domain Adaptation (DA) is Effective for Cross-LLM FacLens

The reason was explained in Section 4.2. In specific, we outlined the premise of unsupervised DA. Then we designed experiments (see Figures 3, 4, and 5) to demonstrate that cross-LLM FacLens satisfies this premise.

- Generalization of the Observations in Section 4.2

In Section 4.2, we conducted experiments on various LLMs, including LLaMA2, LLaMA3, Mistral, and Qwen2. The results across different LLMs lead to the same observation (see Figures 3, 4, and 5), demonstrating the generalizability of our observations. For example, in each subfigure of Figure 5, samples from different LLMs form clusters and share the same classification boundary for NFP.

- Correctness of the Transferability

The transferability of FacLens is validated by comparing results without and with DA, as shown in the upper and lower of Figure 6. Without DA (upper), FacLens trained on one LLM performs poorly on another, especially Qwen2, which differs from the other three LLMs in hidden dimensions and scale. With DA (lower), the performance of FacLens on the target LLM, including Qwen2, improves significantly.

Note that Qwen2 used in this paper differs in scale from the other LLMs. As mentioned on page 10, while FacLens shows transferability between any two LLMs, better transferability is observed among LLMs of similar scales. This does not affect the validity of the demonstrated transferability. Future work will explore methods to enhance FacLens’s transferability across LLMs of different scales.

2. QA Datasets with Long Answers

The factual QA scenario naturally favors short answers, as it focuses on querying specific real-world facts, such as names, numbers, birthplaces, dates, occupations, and so on. Additionally, most mainstream factual QA datasets adopt short answers. For these reasons, like previous works [1, 2, 3], this paper focuses on common factual QA, which typically involves short answers.

While general QA with long-form answers (e.g., text summarization or literary creation) may include factual errors in generated responses, addressing factual errors in this context is another problem requiring a separate methodology, which presents a promising direction for future research.

3. Selection of LLM for Constructing Labeled Training Data

The choice of LLM for constructing labeled training data is flexible. Our goal is to explore whether a FacLens trained for one LLM can quickly adapt to another. The unsupervised DA of cross-LLM FacLens has shown to be effective across any LLM pair (see Figure 6). When a specific LLM can be selected, based on our experiments, we recommend transferring between LLMs of similar scales or from larger to smaller-scale LLMs.

4. Including Baselines in Human Evaluation

Thank you very much for the helpful suggestion! We will certainly integrate the NFP baselines into our demo to facilitate more comprehensive human evaluation.

Q: Clarification on the End of Section 2

In Section 3.1, we defined that if an LLM-generated answer fails to convey the queried fact, it is a non-factual response. Therefore, the output statement “I apologize, but I don’t have information on ...” should be considered as non-factual. Our intent is not to criticize this statement but to clarify that they do not convey the queried fact. We will better claim this in the next version.

[1] Alex Mallen, et al. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. In ACL, pp. 9802–9822, 2023.

[2] Mert Yüksekgönül, et al. Attention satisfies: A constraint-satisfaction lens on factual errors of language models. In ICLR, 2024.

[3] Saurav Kadavath, et al. Language models (mostly) know what they know. CoRR, abs/2207.05221, 2022.

---

We sincerely thank you again for your detailed review and valuable suggestions. We would be happy to engage in further discussion and address any additional questions you may have.

Sincerely,

All authors

Dear Reviewer yYWm,

Thank you very much for taking the time to review our paper. We sincerely appreciate your valuable comments and have provided detailed responses to address your concerns.

We would greatly appreciate it if you could let us know whether our responses have addressed your concerns. If there are any remaining questions, we would be more than happy to provide further clarification and work towards resolving them.

Thank you once again for your time, effort, and consideration, and we look forward to your valuable feedback.

Best regards,

All Authors

Dear Reviewer yYWm,

As the discussion period nearing its end, we notice that we have not yet received your feedback on our responses.

We tried our best to address your concerns with detailed explanations. We kindly request you to check our responses and consider reassessing our paper.

Thank you once again for your time and effort, and we look forward to hearing from you.

Best regards,

All authors

Dear Reviewer GNy3,

We really appreciate your recognition of our paper as novel, important, and well-written. We are delighted that you consider our work an important contribution to the field of NLP and encourage its broader dissemination within the community.

Your constructive suggestions will help us improve this paper. Here we will carefully address your questions in detail.

1. Potential Use of LLMs with Larger Scales

Thank you very much for this suggestion! We acknowledge the potential of using LLMs with larger scales (e.g., over 50B) and plan to explore this direction as the future work. In practice, for many practitioners, training and deploying much larger LLMs presents substantial resource challenges, so they may still opt for LLMs under 10B. Our current work offers practical insights for them.

2. Experimental Setup for the Domain Adaptation (DA)

Table 1 does not include the DA results. The reason is that existing baselines do not investigate and implement DA for cross-LLM Non-Factuality Prediction (NFP).

In the DA setting, we have no labeled data for the target LLM. For a fair comparison, we compare the domain-adapted FacLens with unsupervised NFP baselines, including PPL, Prompting, Entity-Popularity, and Self-Familiarity. We summarize the results in the following table, where the domain-adapted FacLens demonstrates promising performance compared to the unsupervised NFP baselines. We appreciate your suggestion and will rephrase the related experimental setup.

Comparison Between Domain-Adapted FacLens with Unsupervised NFP Baselines on LLaMA3 (the same trend can be found on other LLMs)

---

We sincerely thank you again for your thoughtful review and positive feedback. Your recognition is a great encouragement to us. We would be happy to engage in further discussion and address any additional questions you may have.

Sincerely,

All authors

Dear Reviewer GNy3,

Thank you very much for taking the time to review our paper. We sincerely appreciate your valuable comments and have provided detailed responses to address your concerns.

We would greatly appreciate it if you could let us know whether our responses have addressed your concerns. If there are any remaining questions, we would be more than happy to provide further clarification and work towards resolving them.

Thank you once again for your time, effort, and consideration, and we look forward to your valuable feedback.

Best regards,

All Authors

Dear reviewer 3fBW,

We sincerely appreciate your effort in reviewing our paper. Here we will carefully provide responses to your concerns.

1. Motivation Behind FacLens

- Why not use the MoE architecture: Non-MoE dense LLMs remain widely studied and applied. Following previous works (i.e., the baselines), we conducted experiments on non-MoE dense LLMs. Actually, there is no fundamental difference between MoE-based and dense LLMs, as both use the Transformer architecture, share training schemes (pre-training, SFT, RLHF), and overlap in training corpora. Thus, our work has the potential to be applied to MoE-based LLMs, though exploring this is beyond the scope of this paper and will be addressed in future work.

- Proof of the hypothesis that LLMs share similar cognitions: The rationality of our work is based on the transferability of FacLens across different LLMs, instead of proving the hypothesis that LLMs share similar cognitions. This hypothesis only serves as a motivating inspiration. In terms of our focus, we have conducted comprehensive experiments in Section 4.2 to validate the transferability of FacLens.

2. Novelty

Our work is the first to explore an efficient and transferable approach for ante-hoc non-factuality prediction (NFP). Reviewer GNy3 explicitly acknowledged the novelty of our work, emphasizing that this kind of transferability is importance in the NLP community. Unlike previous works, our work simultaneously offers several key features: 1) it is not limited to specific question templates, 2) it achieves promising results with a lightweight architecture, and 3) it supports unsupervised domain adaptation across LLMs for efficient development.

3. Performance Improvement

Our work has clear improvements over existing works in practical applications (efficiency beyond performance) due to the following reasons.

- Superiority of the ante-hoc (NFP) method FacLens over post-hoc (NFD) methods: In Figure 2, we compare the ante-hoc method (FacLens) with post-hoc methods (SAPLMA and INSIDE). Unlike post-hoc methods, which rely on costly answer generation, the ante-hoc method avoids inference costs and controls risks in advance. As shown in Figure 2, despite post-hoc methods having more information (i.e., generated answers), FacLens still performs better.

- Improvement compared to NFP baselines: Table 1 shows that FacLens achieves clear performance gains over most baselines. While the performance gains over LoRA and Self-Evaluation are slightly smaller, FacLens significantly outperforms both baselines in terms of training efficiency (see Table 2), which is a crucial factor for practical application.

4. Baselines

NFD models (post-hoc) are fundamentally different from NFP models (ante-hoc) in their settings (see the 3rd response). Therefore, comparing FacLens with NFD models in Table 1 is not necessary. To highlight the advantages and potential of the ante-hoc approach, we have compared FacLens with typical NFD methods in Figure 2.

For the NFP baselines, we tried to systematically investigate the related work and compared FacLens with existing NFP baselines. We are open to include more works if you have any suggestions.

5. Experimental Settings

- Form of answers: The factual QA scenario naturally favors short answers, as it focuses on querying specific real-world facts, such as names, numbers, birthplaces, dates, and so on. Additionally, most mainstream factual QA datasets adopt short answers. For these reasons, and to ensure a fair comparison, this paper follows our baselines to focus on common factual QA with typically short answers.

While general QA with long-form answers (e.g., text summarization or literary creation) may include factual errors in generated responses, addressing factual errors in this context is another problem requiring a separate methodology, which presents a promising direction for future research.

- NFP baselines: We have done our best to investigate related work and compared with existing NFP baselines. They are not all naive methods, including advanced baselines using model internals such as SAT Probe [1] published on ICLR 2024.

[1] Mert Yüksekgönül, et al. Attention satisfies: A constraint-satisfaction lens on factual errors of language models. In ICLR, 2024.

6. New Findings

While existing studies have shown that hidden representations in middle layers have good properties for other tasks, our work is the first to uncover that hidden question representations in middle layers contribute to the NFP task.

Besides, this paper has several useful findings. For example, (1) hidden question representations from different LLMs share transferable NFP patterns (approved by other reviewers), and (2) ante-hoc method potentially perform better than the post-hoc method.

Q1: Typos

We will proofread the paper again and fix the typos in the paper.

We sincerely thank you again for your detailed review. We hope our responses will address your concerns. We are pleased to engage in further discussion and address any additional questions you may have.

Sincerely,

All authors

Dear Reviewer 3fBW,

Thank you very much for taking the time to review our paper. We sincerely appreciate your valuable comments and have provided detailed responses to address your concerns.

We would greatly appreciate it if you could let us know whether our responses have addressed your concerns. If there are any remaining questions, we would be more than happy to provide further clarification and work towards resolving them.

Thank you once again for your time, effort, and consideration, and we look forward to your valuable feedback.

Best regards,

All Authors

Dear Reviewer 3fBW,

As the discussion period nearing its end, we notice that we have not yet received your feedback on our responses.

We tried our best to address your concerns with detailed explanations. We kindly request you to check our responses and consider reassessing our paper.

Thank you once again for your time and effort, and we look forward to hearing from you.

Best regards,

All authors