VISCON: Identifying and Benchmarking Vision Hallucination for Large Vision-Language Model

We sincerely thank Reviewer kDMt for their thorough evaluation of our work and appreciate the opportunity to further clarify important aspects of our research. We apologize for the delay in our response and hope that our revisions address your concerns. Please feel free to reach out for further clarification or justification if needed.

W1. EMD calculation is complex to apply.

Briefly speaking, EMD-based evaluation is more complex than using LLMs end-to-end or asking concept-wise questions. However, it evaluates LVLMs better in real-world application scenarios in a more reliable way.

In previou methods, there are several straightforward method to evaluate hallucinations based on visual concept annotations. While using LLMs/VLMs to directly evaluate model response (e.g., FaithScore) against reference visual concepts might seem straightforward and simple for evaluating hallucinations, they have notable limitations. LLMs/VLMs themselves can introduce subjective errors in such complex task handling long caption and multiple visual concepts together, and often lack consistency across different samples, making it hard to distinguish between major and minor hallucinations (e.g., confusing "cat" with "dog" versus "computer monitor" with "laptop"). As a result, FaithScore only achieved a Pearson correlation of 0.48 while VISON achieved a much higher value of 0.95. Additionally, discriminative (i.e., QA-based) metrics (e.g., POPE, AMBER) that simply query LVLMs about the existence of specific visual concepts are too simplistic for real-world applications, where models need to handle multiple concepts simultaneously, such as in detailed captions or multi-turn questions. In contrast, despite more complex, our EMD-based approach offers several key advantages to address these deficiencies of current benchmarks: 1) we only abstract visual concepts with LLM, reducing ambiguity and simplifying the task for LLMs, which only require LLMs to repeat specific words from the input. 2) we use specialized sentence embedding models that are better suited for discriminating text pairs, providing nuanced, soft similarity scores rather than binary 0/1 values. 3) we evaluate LVLM's hallucinations through their long response when required to describe image in details, better discriminating different models with this more complex task. By delegating the evaluation to two simpler tasks and specialized models—concept extraction and similarity discrimination—we enhance reliability and ensure intra-sample consistency, which end-to-end LLM-based scores often lack. By evaluating through detailed captions, our metric better discriminate different LVLMs. Our benchmark's effectiveness is further validated by human preference alignment experiments, as shown in Table 5, demonstrating its superiority over previous methods and simpler QA-based evaluations.

W2. If model tends to respond shorter, it may lead to less hallucination evaluation result.

We appreciate the reviewer's observation that the tendency of models to produce shorter captions, may lead to lower hallucination evaluation scores. However, it is important to clarify that our metrics are specifically designed to assess hallucination in actual model outputs during captioning tasks. While shorter captions may yield less hallucinations, this phenomenon reflects a fundamental trade-off between providing informative descriptions (requiring longer captions) and maintaining factual accuracy (easier with shorter captions), rather than a limitation of our evaluation method.

Moreover, to mitigate the impact of caption length on hallucination measurement results in VISCON, we have implemented two key strategies: - Output Length Control: We observe that most current LVLMs naturally produce outputs within a moderate length range (100 ± 50 words), with the exception of GPT-4V, which tends to generate longer responses. To address this, we control GPT-4V's output length by including specific instructions in the prompt, such as "in 80 words," as detailed in Appendix B.2. This approach helps standardize output lengths and ensures consistent evaluation across models. For future models with more variable output lengths, similar prompt engineering techniques can be applied to maintain consistency. - Normalization in Edit-Distance Metric: The Edit-Distance metric further accounts for caption length by normalizing the total edit distance by the caption length. This normalization ensures that the metric remains robust to variations in caption length, allowing for fair comparisons across models, even when caption lengths vary slightly.

These measures ensure that our evaluation framework remains fair and effective against different detailedness level tendency of models.

We sincerely thank Reviewer 58J2 for their thorough evaluation of our work and appreciate the opportunity to further clarify important aspects of our research. We apologize for the delay in our response and hope that our revisions address your concerns. Please feel free to reach out for further clarification or justification if needed.

W1, W3. Need discussion on the differences of previous work (e.g., AMBER, PhD, FaithScore, Reefknot). How VISCON's annotated visual concept more comprehensive than previous work also using VisualGenome (e.g., CCEval)?

Quantitatively comparing our method to previous approaches is challenging due to the diversity of datasets and benchmarks, many of which are still in preprint form and not yet published (e.g., AMBER, PhD, FaithScore, Reefknot, CCEval, as reviewers indicate). According to the ICLR 2025 Reviewer Guide, it is not mandatory to discuss and compare with unpublished or very recent works. However, we can highlight key differences between our approach and several representative existing methods:

• FaithScore: This method relies on visual entailment models, which are themselves are VLMs and prone to hallucinations, compromising their reliability. In contrast, our method uses dense human annotations to directly evaluate hallucinations, providing a more reliable and comprehensive assessment.
• CCEval: This method extends reference object annotations using VisualGenome but still focuses primarily on objects and real-world photos like POPE, limiting its ability to capture the full range of visual concepts, such as attributes and relationships. Our method evaluates hallucinations across three types of visual concepts and multiple image domains, offering a more comprehensive evaluation of LVLM performance and investigation on the causes of hallucinations.
• AMBER, PhD and Reefknot: These methods extend hallucination benchmarks to include attributes and relationships but face limitations. They either struggle with real-world scenarios involving multiple visual concepts due to their discriminative evaluation methods (only involving one concept per question) or lack dense annotations, leading to false negatives. Our method addresses these deficiencies by evaluating detailed caption responses, better discriminating LVLMs in complex tasks involving handling multiple concepts together, and enriching visual concept annotations with dense data from VisualGenome and oracle annotations from a 3D simulator, significantly reducing false negatives and enhancing reliability.

Beyond the extensive visual concept types and annotation density that VISCON provides, our method uniquely explores the impact of image domains on LVLM hallucinations, a factor not addressed in previous research. Moreover, our experiments show that our method aligns more closely with human judgments, as demonstrated in the updated Table 5, highlighting its effectiveness and reliability in real-world scenarios.

Regarding the visual concept annotation comprehensiveness and density, our method surpass previous method like CCEval by 1) considering more visual concept types, rather than only focusing objects in CCEval, 2) considering another series of image domain from 3D simulator, that also enrich the comprehensiveness of VISCON.

W2. Rendered images are limited indoor scene.

While our current dataset includes a significant number of outdoor images sourced from VisualGenome, we focused on 3D indoor scenes primarily because they are readily available from existing 3D simulators. This choice allows us to leverage detailed scene-graph annotations, which are crucial for our evaluation framework. However, we recognize the importance of expanding the diversity of image domains. In the future, we plan to incorporate more outdoor images, potentially from realistic environments in games like GTA V, to enhance the robustness and applicability of our benchmark across different settings.

We sincerely thank Reviewer erAn for their thorough evaluation of our work and appreciate the opportunity to further clarify important aspects of our research. We apologize for the delay in our response and hope that our revisions address your concerns. Please feel free to reach out for further clarification or justification if needed.

W1, Q1, W3, Q3. Need to evaluate two VISCON evaluation pipelines on more dense annotated datasets and more image data. Need to compare with more benchmarks.

• Densely annotated datasets are less common, which presents a challenge for adding more such evaluations. To address this, we have utilized existing datasets such as VisualGenome and supplemented them with a new dataset created using oracle annotations from a 3D simulator. Therefore, we argue that we have evaluated two metrics on a decent broad range of densely annotated datasets. Moreover, our evaluation framework is built upon the mutual combination of dataset and metrics (evaluation pipelines).

• Regarding the dataset size, our experiments have demonstrated that the current dataset is sufficient to effectively discriminate between different models (Appendix Figure 7). We observe that the evaluation metrics stabilize as the ratio of test data used increases, indicating that the dataset size is adequate for reliable and consistent model evaluation. Additionally, since our dataset is constructed without human annotators, scaling up is straightforward and cost-effective, but might be of diminishing returns as the dataset size increases.

Quantitatively comparing our method to previous approaches is challenging due to the diversity of datasets and benchmarks, many of which are still in preprint form and not yet published (e.g., AMBER, PhD, FaithScore, Reefknot, CCEval, as reviewers indicate). According to the ICLR 2025 Reviewer Guide, it is not mandatory to discuss and compare with unpublished or very recent works. However, we can highlight key differences between our approach and several representative existing methods:

• FaithScore: This method relies on visual entailment models, which are themselves are VLMs and prone to hallucinations, compromising their reliability. In contrast, our method uses dense human annotations to directly evaluate hallucinations, providing a more reliable and comprehensive assessment.
• CCEval: This method extends reference object annotations using VisualGenome but still focuses primarily on objects and real-world photos like POPE, limiting its ability to capture the full range of visual concepts, such as attributes and relationships. Our method evaluates hallucinations across three types of visual concepts and multiple image domains, offering a more comprehensive evaluation of LVLM performance and investigation on the causes of hallucinations.
• AMBER, PhD and Reefknot: These methods extend hallucination benchmarks to include attributes and relationships but face limitations. They either struggle with real-world scenarios involving multiple visual concepts due to their discriminative evaluation methods (only involving one concept per question) or lack dense annotations, leading to false negatives. Our method addresses these deficiencies by evaluating detailed caption responses, better discriminating LVLMs in complex tasks involving handling multiple concepts together, and enriching visual concept annotations with dense data from VisualGenome and oracle annotations from a 3D simulator, significantly reducing false negatives and enhancing reliability.

Beyond the extensive visual concept types and annotation density that VISCON provides, our method uniquely explores the impact of image domains on LVLM hallucinations, a factor not addressed in previous research. Moreover, our experiments show that our method aligns more closely with human judgments, as demonstrated in the updated Table 5, highlighting its effectiveness and reliability in real-world scenarios.

W2, Q2. Attributes might change after some stylizations. How VISCON handle these changes?

We acknowledge that initially, our annotations did not fully account for changes in attributes after stylization. We have updated our annotations to address this oversight, manually removing color and material attributes from line painting and sketch-stylized images. After implementing these changes, we observed that the overall trend in model performance remained consistent, demonstrating the robustness of our evaluation framework.

We sincerely thank Reviewer 8UyP for their thorough evaluation of our work and appreciate the opportunity to further clarify important aspects of our research. We apologize for the delay in our response and hope that our revisions address your concerns. Please feel free to reach out for further clarification or justification if needed.

W1. Need to clarify the he motivation for creating the hallucination dataset, particularly regarding its specific focus on hallucination evaluation and the targeted sources of hallucination.

The motivation for our dataset is driven by several key factors:

• Complexity of Visual Concepts: Our dataset captures the complexity of visual concepts, particularly attributes and relations, where LVLMs are prone to hallucinate. Unlike previous methods such as POPE, AMBER, and CCEval, which often overlook these complex concepts, our dataset includes detailed annotations that require LVLMs to handle multiple components, such as object-subject relations and attribute-object pairs. This allows for a more accurate assessment of the true capabilities of different LVLMs.
• Dense Annotations with Minimal False Negatives: We provide denser reference visual concepts by utilizing comprehensive annotations from VisualGenome and oracle annotations from a 3D simulator. Previous methods, like POPE and AMBER, often sample negative objects from non-annotated sets, leading to potential false negatives due to insufficient annotation density. As shown in the updated Table 1, our dataset achieves the highest density across all three types of visual concepts over previous annotation-based metrics, ensuring more reliable evaluations.
• Rigorous and Realistic Evaluation: Our dataset offers a more rigorous and realistic evaluation of model performance by focusing on detailed captions that require LVLMs to handle multiple concepts simultaneously. This approach contrasts with simpler question-based benchmarks, providing a more comprehensive test of model capabilities in real-world scenarios.

W2, W3. Need more discussions on domain shifts and length impact on hallucination. Small differences in EMD on different domain shifts.

We have discussed the impact of domain shifts on hallucination in our paper in Section 4.2 of our paper. While the numerical range of EMD might appear relatively small, this does not diminish its ability to capture meaningful differences across domains, especially when comparing with human performances as shown in Figure 4(a). When comparing with human, a clear trend can be seen that LVLMs tends to generate more hallucinations as images deviate from real-world photos, particularly in more abstract domains like line paintings. This trend is consistent across models. Quantitatively, the EMD values increase as FID scores increase (Figure 4(b)), indicating that models struggle more with hallucinations as the visual domain becomes less conventional. This highlights the importance of evaluating models across diverse image types, which is a key contribution of our work.

Regard on the impact of length, we found that the EMD values are consistent across different caption lengths (Figure 5(a)), indicating that the length of the caption does not significantly impact the hallucination levels. In Figure 5(b), we observe that edit distance linearly increase with caption length with a slope close to 1, indicating that the per-word hallucination rate level is not significantly impacted by the length of the caption, which is consistent with our observations on EMD.

W3, Q1, Q2. The impact of image style to hallucination seems to be because of insufficient training. POPE attributes hallucination to textual coexistence in training corpus, which may not be solved by extending data. Adding new tasks and image styles in evaluations seems to be of limited contribution.

It's important to note that while extending training data might reduce hallucinations over altered image styles, this does not eliminate the inherent deficiency in LVLMs. Unlike humans, who can well recognize visual concepts in artistic works despite seeing artistic works less frequently, LVLMs still struggle with response authenticity. Our findings highlight a gap in LVLMs that is not present in human cognition.

Regard on the contribution of our dataset, we, and POPE both focused on uncommon data distributions, that leads to hallucinations for LVLMs. Ours focus on image domains (where some images may be less seen), and POPE on visual concepts' occurrences and co-occurrences (where some concepts or concept combinations are less seen). In principle, both of them could be solved by extending data. Moreover, our dataset adopt much denser annotations to minimize the presence of false negatives, which POPE tends to include more since it uses sampled negative visual concepts (in their work, object only) that are non-existent in annotations, which is much scarce and might sample objects exists in image, but un-annotated.

We sincerely thank Reviewer QEkM for their thorough evaluation of our work and appreciate the opportunity to further clarify important aspects of our research. We apologize for the delay in our response and hope that our revisions address your concerns. Please feel free to reach out for further clarification or justification if needed.

W1. Attributes might change after some stylizations. How VISCON handle these changes?

We acknowledge that initially, our annotations did not fully account for changes in attributes after stylization. We have updated our annotations to address this oversight, manually removing color and material attributes from line painting and sketch-stylized images. After implementing these changes, we observed that the overall trend in model performance remained consistent, demonstrating the robustness of our evaluation framework.

W2. EMD differs little. GPT-4V even surpass human on real-world photos.

While the EMD differences might seem small on subsets of some image domains, they are quite significant when considering the entire dataset (Column 2 of Table 2). Moreover, these differences effectively distinguish between models and align well with human preferences (Table 5), underscoring their relevance and utility.

Regarding GPT-4V's performance, it's possible that models tend to perform better on real-world images (those unaltered, natural photos). However, the real challenge—and where our research truly wants to reveal—is in evaluating performance on style-altered or abstract images. In these scenarios, models often struggle, especially on abstract images, similar to findings reported in HallusionBench. This finding also demonstrates the necessity of VISCON, which evaluates models on a diverse set of image domains like abstract and style-altered images.

W3. Need to demonstrate the robustness of edit procedure.

To address the need for demonstrating the robustness of the edit distance in our "Evaluate-By-Edit" pipeline, we have included qualitative results in Figure 9 of the appendix. This figure provides a detailed visualization of the edit process used to calculate the edit distance between the original model output and its revised version. In this visualization, we highlight text modifications with different colors to clearly show which parts of the text were removed, substituted, or inserted.

As shown in Figure 9, the majority of the edits effectively correct vision hallucinations. These corrections specifically target references to entities that do not exist in the image, such as "a cat," "potted plant," and "a vase." Inconclusive contents from annotations, such as "some paintings appear to be smaller" is remained as-is. This visualization demonstrates how our method accurately identifies and corrects hallucinations, reinforcing the robustness of the edit distance as a measure in our evaluation pipeline.