Truth is Universal: Robust Detection of Lies in LLMs

Thank you for the thoughtful review. We are glad that you found our identification of the 2D truth subspace to be a significant contribution and appreciate your constructive feedback.

Regarding theoretical background: We agree that a theoretical explanation for why these truth directions emerge inside LLMs would be highly valuable. However, this is significantly beyond the scope of our empirical work. Such an explanation might require a detailed analysis of the training dynamics of the LLM and/or significant theoretical breakthroughs.

“Comparison with similar classifier-based like ITI [1] in order to mitigate hallucinations by intervention.” We agree that making models more honest through causal interventions based on the 2D truth subspace is an interesting question. However, we believe that it is beyond the scope of this paper. We will mention in the revision that this is an exciting direction for future research.

Regarding the examples of failures of the classifier: This is a good point. In reply, we analysed the LR classifier failures for several datasets. We observed two main failure modes for misclassified statements: In the first failure mode, almost all misclassified statements in a specific dataset had the same truth label. While the learned truth direction is still able to separate true vs. false statements, the reason for these errors is that the bias (learned from other datasets) did not generalize sufficiently well. In the second failure mode, the learned truth direction was not able to accurately separate true vs. false statements. We conjecture that a possible reason is that for these statements the LLM may not have been certain if the statement is true or false. This failure mode occurred in the inventors and in the scientific facts datasets. We will add a discussion of this in the revision, along with a few example sentences which were misclassified.

“How can it be that adding cities_conj spoils facts_disj in Fig. 5?” This is a good question. We currently do not have a satisfactory explanation for this behaviour. In the revision, we will add the cross-domain generalization accuracy matrix, and make a more detailed study of this issue.

“Could you please explain the experiment with scenarios?” Thanks for pointing out that this part was not clear enough. In the revision we will provide a more detailed and clear explanation. After generating the LLM response, we pass the text with the context to the LLM (Context + Buyer: 'text' + Agent 'text') and record the activations over the final token. The truth or falsity of the LLM’s response is then predicted by the classifier probes based on these activations.

Regarding the small comments and typos: Thanks for catching these! We will correct/update the paper.

“I find the claim that the truth is universal in the beginning (and in paper name) is a bit misleading.” Thank you for raising this issue. The claim of universality made in our paper is that all considered LLMs (including also the added Gemma-2-27B-Instruct), represent truth in a similar manner, for affirmative sentences, negated sentences and conjunctions thereof. We agree that the truth representations might differ in other yet unknown truth dimensions and will make this clear in the revision.

“Authors provide the investigation of only one type of embeddings” Thanks for raising this important point which was not sufficiently clearly explained in the original submission. We choose the last token and a specific layer for a good reason: Marks and Tegmark [2023] showed empirically that truth information is encoded in the intermediate layers of the LLM after processing this last token, which encodes the end of the input statement. We will clarify this in the revision. The choice of layer is justified by the results shown in Figure 2.

Thank you for your positive review! We appreciate the criticism regarding clarity. In the revision, we will improve the writing throughout the manuscript.

Regarding code release: We fully agree. In the spirit of reproducible research, with the revision, we will make our code and scripts public, so other researchers can reproduce the results of our manuscript.

Regarding the extraction of the activation vectors: Thanks for raising this issue which was not sufficiently clear in our original submission. You are correct that we extracted the activation vector that corresponds to the “.” token at the end of the input statement. This final token contains the most enriched representation of the information present in this sentence. This scheme is identical to that of Marks and Tegmark. We will explain this in the revision.

Regarding the names of the layers from which we extract the activations: Thank you for raising this issue. In the revision we will explain more clearly from which layer we extracted the activations and what the residual stream is. The residual stream layers have no specific names beyond their layer number.

“How similar were tG and tP between the different LLMs?” Note that $t_G$ and $t_P$ for different LLMs come from layers that may have different dimensions and as such may not be directly comparable. The universality phenomena we uncover is that the various LLM models (including from Llama and Gemma families) all have a similar 2D truth representation. We will clarify this in the revision.

Note that Google DeepMind and Meta AI do not publish their training datasets. Hence, it may be plausible that the training datasets are very similar, but we do not know this for sure.

Regarding asking the LLM whether it just lied: This is an excellent question. However, it has already been explored in previous work by Pacchiardi et al. [2023] , so we did not include it again in our paper. We will clarify this important point in the revision. Pacchiardi et al. [2023] asked GPT-3.5 "Are you lying?" after it generated a lie, and found that GPT-3.5 lied again more than 80% of the time (see Appendix B.2.2 of their paper). In general, our approach is designed for scenarios where the LLM knows it is lying, but is doing so in pursuit of some goal or reward, see for example Scheurer et al. [2023]. If it is a sufficiently capable LLM, it will not reveal that it has just lied, but will hide it. But from the internal model activations we might be able to detect the lie! We will add a discussion of this in the revision.

“Also, could something like this prompting scheme also be used with the intermediate layers method proposed in this work for possible performance improvements?” This is an excellent suggestion for future research. We will mention this in the revision.

Thanks a lot for your review! We are glad you found the 2D subspace explanation convincing and liked the presentation of the results.

Regarding Novelty: We agree that our work was directly motivated by the empirical findings of Marks and Tegmark and Levinstein & Herrmann. The novelty and importance of our work is two-fold. (i) We explain their empirical findings via our 2D truth representation subspace analysis; and (ii) our analysis clarifies why linear classifiers may still be highly accurate for the task of truth/falsity classification. We will clarify this in the revision.

“this insight does not lead to any novel method of training a more generalizable truth probe” As mentioned above, the insight that there is a 2D subspace explains why linear classifiers may still be highly accurate in classifying truth/false statements. In direct reply to the above concern, in recent follow up work (after our original submission), based on this insight, we constructed a new classifier, which is still linear, but achieves an even higher accuracy than LR and CCS. We will gladly discuss this in the revision.

Regarding generalizability and testing on more challenging and out of distribution statements: Please note that Section 5 presents such settings. Specifically, to the best of our knowledge, we are the first to quantify the generalisation accuracy of lie detectors based on internal probe activation, when tested on a variety of challenging real-life scenarios. We would like to emphasise that Pacchiardi et al., the creators of the real-life scenarios, used the output of the LLM to follow up questions (after lying or telling the truth) and not the internal activations of the LLM to classify the LLM responses as truthful reply or lie. We will clarify this important point in the revision.

“A more comprehensive investigation of generalizability would strengthen the paper”: In general, we agree that further investigations are of interest and we explicitly mentioned this in the original submission. That said, please note that the evaluation of generalizability in our submission is more comprehensive than most prior works, both in terms of type of statements, number of diverse datasets considered, etc. We will clarify this in the revision.

Regarding Universality: We agree that this claim can be strengthened by considering larger models and other model families. In reply to your concern, we extended our analysis to Gemma-2-27B-Instruct, a model twice the size of the largest model considered in our original submission, and to Mistral-7B-Instruct-v0.3, a LLM from a different model family. The results are qualitatively the same as for the other LLMs with a 2D truth subspace in the activation space of the models. We will include these results in the appendix of the revision.

Regarding causal experiments: We agree that causal intervention based on the 2D truth representation is an exciting research direction. However, it is beyond the scope of this paper, as our focus lies on providing rigorous evidence for the existence of the 2D truth subspace and using this knowledge for robust lie detection. We will mention in the revised conclusion section that causal intervention based on our insights is an exciting direction for future research.

Regarding the Mass Mean Probing technique of Marks and Tegmark: We thank the referee for this suggestion. In response, we did a comparison with MM probing on the Llama3-8B-Instruct model. For a fair comparison we in fact extended the MM probe to include a learned bias term. The MM probe generalised a bit better than our original LR based lie detector classifier. However, a new classifier we constructed after our original submission (mentioned above) achieved even higher generalisation accuracies. We included a table comparing the four methods in the global response. We will include all of these results in the revision.

Regarding the normal LR probe in Section 5: This is an excellent question and we agree that the motivation for this was not sufficiently well explained in the original submission. Our analysis of the 2D truth subspace showed that there is a general truth direction $t_G$ which we can disentangle from $t_P$ by training a linear classifier on a balanced number of affirmative and negated statements. Logistic Regression was simply our choice for the linear classifier and the direction it learned is indeed similar to $t_G$. We will clarify this in the revision.

Is there a reason to expect one method to be superior to the other? Good question! Empirically, we have found (after our original submission) that truth directions which are learned separately from the bias (as in Section 3) generalise better to the unseen statement types and real-world scenarios than truth directions which are learned together with the bias (as in LR). However, we are not aware of any fundamental reason why one method would be superior to the other. We will discuss this in the revision.

Thank you very much for your review! We appreciate your comments and hope that our response below can convince you that the contribution of our work goes beyond just showing that training on more diverse and balanced data improves generalization.

“The analysis of section 3 is qualitative”. Please note that the accuracies of the classifiers based on the directions $t_G$ and $t_P$ appear in Figure 3. This figure shows the clear advantage of classification based on $t_G$. The accuracy of a classifier trained only on affirmative statements ($t_A$), is 81%, whereas using $t_G$ it is 95%. We will add these numbers to figure 1 and to the main text. In addition, following the suggestion of the referee, we will add to Figure 3 another column with the accuracy of a logistic regression classifier trained on affirmative and negated statements.

Regarding the importance of the two directions, the decomposition, etc: We agree with the referee that in general training on a larger dataset, more representative of future test instances, is often beneficial. However, it is a-priori unclear that a linear classifier would be suitable. In the following, we explain why in our opinion, our discovery of the 2D truth subspace spanned by $t_G$ and $t_P$ is a significant contribution. Prior to our work it was unclear how LLMs internally represent the truth or falsity of factual statements. For example, it was unclear whether there is a single "general truth direction" or multiple "narrow truth directions”, each for a different type of statement. However, this knowledge is essential in order to construct a robust LLM Lie Detector based on its internal model activations. In particular, insights from our 2D truth subspace representation can be directly used to construct robust truth classifiers, which are still linear. Furthermore, these insights might allow other researchers to construct even more accurate non-linear classifiers which leverage both dimensions of the truth subspace.

Regarding the analysis of Section 4, and the first 2 PCs accounting for most of the variance: We thank the referee for raising this issue, which was not sufficiently clearly explained in the original submission. A key point is that we did not perform PCA on the raw activations. Instead, we first preprocess the data to isolate truth-related variance. As described in Section 4, we have a two-step process: (1) we center the activations in each dataset D_i, see L200-202. (2) We average the resulting activations for true and false statements in each dataset, see Eq. 7. The first two PCs after this preprocessing capture 60% of the variance in the centered and averaged activations, but not in the raw activations. On the raw activations, these two vectors capture only ~10% of the variance. Therefore, we are not removing most of the information contained in the activation functions. We will clarify this important point in the revision.

Regarding comparison to CCS, and the statement “I take the main results to be that a simple classifier trained on diverse and balanced data outperforms a more complex and specialized learning algorithm”: Here we respectfully disagree with the referee. First, let us point out that we trained both our method, as well as CCS on the same diverse and balanced training set. Second, the high accuracy of our method is supported by our 2D analysis of the truth representation in the activation space, hence showcasing the tight connections between the various sections of our manuscript.

To conclude, given the clarifications above, as well as our replies to the concerns raised by the other reviewers, in our opinion, the contributions of our manuscript are of interest to the community and of sufficient significance to warrant publication at NeurIPS.

Thank you for your response, and apologies for the delay in responding.

However, it is a-priori unclear that a linear classifier would be suitable.

What is the significance of the classifier being linear? Do you assume that linear implies more robust, at little to no cost in accuracy? This might not hold beyond the toy tasks considered in this work (e.g., the method already fails to generalize to logical conjunctions well, let alone actually detecting “lies” in deployed systems).

Insights from our 2D truth subspace representation can be directly used to construct robust truth classifiers, which are still linear. Furthermore, these insights might allow other researchers to construct even more accurate non-linear classifiers which leverage both dimensions of the truth subspace.

I fail to see what insights you used to construct the final linear classifier. Is it not just standard linear regression over the activations? What insights could be used to construct more accurate non-linear classifiers?

On the raw activations, these two vectors capture only ~10% of the variance.

This addresses my concern regarding most of the variance being projected out.

First, let us point out that we trained both our method, as well as CCS on the same diverse and balanced training set. Second, the high accuracy of our method is supported by our 2D analysis of the truth representation in the activation space, hence showcasing the tight connections between the various sections of our manuscript.

Yes, I was commenting under the assumption that both CCS and your method are trained on the same data. The fact that linear regression outperforms CCS is to me more indicative of CCS being a poor method for this particular task rather than of the merits of linear regression. Linear regression (linear probing) is typically the baseline for classification tasks. For the more challenging tasks (Figure 6b, real-world scenarios), the error bars are so large that it is not even clear that the performance of LR and CCS are significantly different.

My general assessment remains “I believe that the main contribution of the paper is to show that, for the task of true/false discrimination from internal activation functions, training on more diverse and balanced data improves generalization” I’ll add the following “The work shows that a linear classifier is sufficient to obtain high accuracy in simple true/false discrimination tasks”. I still do not think that these two contributions are of sufficient significance to warrant publication.