Interpreting and Editing Vision-Language Representations to Mitigate Hallucinations

We thank the reviewer for the valuable comments and feedback on our paper and address the concerns below.

Applicability of the methods to other elements of visual scenes (ex. people, attributes, actions)

We focus on object hallucinations because they have standard evaluation suits, and it is difficult to get precise quantitative results for attribute hallucinations (mostly due to various possible phrasings). However, in Appendix A.7, we’ve added qualitative examples for attribute hallucinations (color, object number) based on images and questions from the VQA 2.0 challenge [1]. We find that our model confidence scores (Section 3) can identify when responses are inaccurate, and our editing technique can correct them appropriately.

Potential impact of editing on accuracy in non-hallucination tasks

Our analysis examines overall caption quality (not just tied with hallucinations) by measuring changes in correctly detected (CD) objects – the non-hallucinated objects that appear in the scene. As Table 1 shows, our editing technique does not produce a substantial change in CD objects, indicating that the new captions convey a similar degree of specificity for the objects contained within the scene. Appendix A.7 also shows the potential for using our editing technique to improve VQA performance by reducing attribute hallucinations. Outside of editing, we further show promising results for using logit lens to perform zero-shot classification, another non-hallucination task, in Appendix A.8.

Other state-of-the-art MLLMs

As Reviewer bJVo03 mentions, “InstructBLIP and LLaVA are representative LVLMs,” but we agree that the landscape of LVLMs is constantly evolving and want to ensure our analysis is thorough. We conducted additional evaluations on the same 500-image validation subset from Section 5.2 (“Hallucination reduction”) using more recent models – LLaVA-NEXT 7B [2] and Cambrian-1 8B [3] with Llama 3. The results demonstrate consistency with our original findings, suggesting that our conclusions generalize across model architectures. Our method results in a 27.73% reduction in hallucinations with LLaVA-NEXT and 28.26% with Cambrian-1. For simplicity, we use the same hyperparameters for LLaVA, though optimizing this selection would likely result in further improvements. While our original baselines are LLaVA and InstructBLIP, which are thoroughly evaluated in related hallucination reduction papers [4], this supplementary evaluation strengthens our claims. We include these new results in Appendix A.5.

[1] https://visualqa.org/index.html

[2] https://llava-vl.github.io/blog/2024-01-30-llava-next/

[3] https://arxiv.org/abs/2406.16860

[4] https://arxiv.org/pdf/2311.17911

We thank the reviewer for giving feedback on the paper and providing valuable comments. We address the concerns and questions below.

The current structure performs its main analysis of the editing technique under idealized conditions and potentially overestimates its effectiveness as a result.

We see the reviewer’s point about highlighting the practical approach upfront. Our intention in Section 4 (“Erasing Knowledge from VLMs”) is to study the editing technique’s effects independent of model confidences. This section highlights the surprising result that linear orthogonalization can effectively remove knowledge of objects from image captions, whether they are hallucinated or not. In our intro and abstract, we only highlight the hallucination reduction results from Section 5.2 (“Hallucination reduction”) to avoid suggesting that the idealized findings from Section 4 represent expected outcomes in practical applications. Nevertheless, we would be happy to hear suggestions for a better structure from the reviewers.

“Have you conducted layerwise probing or training of separate unembedding matrices?”

We use the model’s unembedding matrix to interpret intermediate layer representations and provide a training-free method for interpreting the internal representations. The logit lens [1] method on text-only models shows that the model’s unembedding matrix effectively interprets LLMs, and we demonstrate that, surprisingly, it is true for LVLMs as well. We intend to show that it is possible to interpret the internals of these models without requiring additional training and present a novel interpretability method that can be widely used across VLMs without repeated training.

Justifying Last Tokens for Multi-Token Object Representations

Our approach of using the last tokens is motivated by past work that find that information about multi-token entities is moved to the last token position. For example, [2] finds that the last subject token encodes crucial factual associations, and [3] demonstrates that information is carried over to the last token position through relation propagation and attribute extraction. Thus, extracting a residual hidden representation of the last token, which is conditioned on the previous tokens of the class, is the most likely to contain the concept of the whole class (ex. “traffic light”) and not merely a single part.

[1] https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru

[2] https://rome.baulab.info/

[3] https://arxiv.org/abs/2304.14767

We thank the reviewer for the detailed questions and feedback and aim to address them below.

“Is the unembedding matrix trained by the authors?”

We use the unembedding matrix from the LVLM and do not train, as we intend to provide a training-free method for interpreting the internal representations of LVLMs. The logit lens method, applied to text-only models, showed that the model’s unembedding matrix effectively interprets language models. We show that this capability can surprisingly be extended to LVLMs.

Vision-language modality gap

We agree with the important point about the modality gap and token distributions. We include classification results in Table 8 in the Appendix, performing patch-level evaluation on 500 images in the COCO dataset. We find that the classification accuracy varies highly between different COCO classes, with strong top-3 classification accuracy with classes such as “toothbrush” (78.9%), “toilet” (92%), and “banana” (77.2%) and much lower accuracy with some classes such as “person” (0.5%) and cup (9.4%). We hypothesize that the variation stems from how consistently objects are represented linguistically - classes that map to specific, consistent tokens perform better than those that can be described with many specific terms (e.g., "person" → "doctor", "skier", "girl"). The LVLM sometimes captions the image with more specific terms (such as in the case of class “person”) so we can interpret the image representations with these more specific terms well, e.g. since there is only one way to describe “banana”. More importantly, our quantitative results demonstrate that the logit lens effectively captures the model’s learned semantic alignments in practice. Section 5.1 shows that our method can distinguish objects present vs. not present, achieving significant improvements in hallucination detection, despite this modality gap. Additionally, in our zero-shot segmentation results (Section 5.3), we demonstrate that these projections accurately localize classes spatially. These quantitative results across multiple tasks suggest that the logit lens captures the model’s internal understanding of visual semantics, specifically with practical applications in hallucination intervention, despite modality differences.

Does random sampling choose some objects “obviously” not present in the image?

We sample only from the set of 80 COCO classes, where many objects commonly co-occur, and believe that the strong performance across applications validates that distributions of objects present and not present are reliably separable. In Section 5.1 (“Hallucination detection”), we narrow down the scope of randomly selected objects to only hallucinated objects, where hallucinations are often objects that are not present in the image but less obviously so. Through this specific application, we intend to show here that the logit lens can classify these objects as present or not present with strong results (mAP improvement by 22.45% in LLaVA and 47.17% in InstructBLIP), even while these objects not being present may be less obvious.

Last Tokens for Multi-Token Object Representations

Our approach of using the last tokens is motivated by past work that find that information about multi-token entities is moved to the last token position. For example, [1] finds that the last subject token encodes crucial factual associations, and [2] demonstrates that information is carried over to the last token position through relation propagation and attribute extraction. Thus, extracting a residual hidden representation of the last token, which is conditioned on the previous tokens of the class, is the most likely to contain the concept of the whole class (ex. “traffic light”) and not merely a single part.

“Is the generated class name from LLaVA or the ground truth label used?

Similar to the reviewer’s findings, we found that LLaVA tends to generate some very general class when classifying an image. We use the generated class name from LLaVA in zero-shot segmentation for two reasons. (1) We generate the segmentation without knowing the ground truth label, and only the LLaVA object prediction, to have true end-to-end zero-shot segmentation. (2) If LLaVA predicts a “dog” rather than a “husky” in the image, we find that it maps the image representations closer to “dog” tokens than to “husky” tokens. This is likely because this is how it internally processes the objects in the image representations (as “dog,” not “husky,” resulting in higher internal confidence for “dog”), which we interpret with text.

[1] https://rome.baulab.info/

[2] https://arxiv.org/abs/2304.14767

We thank the reviewer for the valuable comments on the paper and address the concerns below.

Hyperparameters are determined through a cumbersome ablation process

Our ablations in Figure 6 show that for LLaVA, there is significant (>15%) hallucination reduction across the vast majority of hyperparameter selections. When testing our hallucination reduction method with two different VLMs, Cambrian and LLaVA-Next, we took similar hyperparameters and achieved strong results despite the lack of ablation studies. Moreover, all of the ablations can be done automatically as a one-time cost for a given model using a simple grid search.

Applying method to other types of hallucinations and other tasks like VQA

We focus on object hallucinations because they have standard evaluation suits, and it is difficult to get precise quantitative results for attribute hallucinations (mostly due to various possible phrasings). However, in Appendix A.7, we’ve added qualitative examples for attribute hallucinations (color, object number) based on images and questions from the VQA 2.0 challenge [1]. We find that our model confidence scores (Section 3) can identify when responses are inaccurate, and our editing technique can correct them appropriately.

Overall caption quality was not evaluated quantitatively

We quantitatively evaluate caption quality by measuring changes in correctly detected (CD) objects, the non-hallucinated objects that appear in the scene. As Table 1 shows, editing hallucinations out does not lead to a substantial change in CD objects. We are not familiar with strong, comprehensive evaluation criteria that can automatically measure the quality of non-object attributes (ex. Color, shape, relation) in captions and are not prone to breaking under small phrase changes.

The authors only seem to test their model on COCO2014.

In Appendix A.9, we add more qualitative examples of hallucination reduction on images from LLaVA-Bench [2] and find that they align with the strong results seen with COCO2014. We primarily use COCO2014 because the object hallucination metric CHAIR is tied to the dataset and drives our quantitative results. The images contained within COCO2014 are diverse, and we use separate image data to select hyperparameters (ex. Section 4.2 - “Ablations”) and to test the model (Section 5.1 and 5.2).

[1] https://visualqa.org/index.html

[2] https://huggingface.co/datasets/liuhaotian/llava-bench-in-the-wild