Automatically Generating Visual Hallucination Test Cases for Multimodal Large Language Models

1, The paper mainly addresses the MLLM hallucination by constructing more data and then fine-tuning on them, which is not very exciting. 2, In Table.8, we can see the proposed method causes the MLLM comprehensive ability to decrease.

(1) As an automated method for generating VH test cases, this approach addresses the issue of insufficient data volume in previous benchmarks like MMHAL-BENCH and POPE.

(2) The entire paper seems like an application of transferring an attack method to VLMs, lacking novelty, as the attack approach itself has been previously used.

(3) The overall paradigm of automatically constructing benchmarks is also very similar to many works, such as: 1. Autobencher: Creating salient, novel, difficult datasets for language models 2. Multimodal Self-Instruct: Synthetic Abstract Image and Visual Reasoning Instruction Using Language Model 3. How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMs For example, in the third paper, some test cases were also constructed using a similar attack method.

• While the paper discusses common and adversarial perturbations, it might not fully explore the spectrum of possible perturbations that could trigger visual hallucinations in MLLMs, such as resize/affine transform/crop/generation-based image editting, etc. There could be more sophisticated or subtle perturbations not covered.
• The method relies on an LLM to automate negation of questions, which might introduce its own errors or biases. The makes the method still need correction by human and thus the method is still semi-automatic instead of the stated fully-automatic.
• In my opinion, Table 8 shows that fine-tuning on expanded VH test cases degrades the model’s performance on other general-purpose VQA datasets (MME in this case) since the scores are harmed. However, authors tend to interpret the results as " the model’s performance are maintained" in line 565.

1. The proposed method's focus on binary questions limits its generalizability, as both the VHExpansion framework and the symmetric accuracy metric may not directly apply to other types of VH datasets with non-binary answer formats. It would be valuable to explore whether the performance improvements from fine-tuning on expanded VH test cases can transfer to other VH datasets with different answer structures, such as multiple-choice questions. Although the symmetric accuracy metric is tailored to binary questions, the image perturbation techniques could potentially be applied to other VH datasets.
2. As indicated in line 288, the VH datasets used in this study primarily focus on low-level understanding, such as object properties (e.g., color, counting, and position) and object presence. It would be interesting to evaluate whether the proposed method could be effective for VH scenarios related to more advanced comprehension, like sentiment or context-based analysis.
3. The paper would benefit from a more detailed analysis of question types. For example, while the authors note that three of four common perturbations even increase InstructBLIP’s symmetric accuracy on VHTest, they do not delve into the underlying reasons. Analyzing how the proposed method performs across different question types and the specific capabilities it tests would provide more insight. For instance, a question involving counting (as shown in Fig. 1) might become more challenging with defocus blur, whereas the question in Fig. 2 might benefit from the same blur by reducing visual distractions.
4. The experimental setup in the "MITIGATING VH VIA FINE-TUNING" section (5.4), claimed as a major contribution, has some issues. The comparison between "fine-tuning on the original VH test cases (baseline)" and "fine-tuning on expanded VH test cases using VHExpansion (expanded)" is not entirely fair. In the baseline condition, the fine-tuning dataset includes 200 VH samples and 4,000 general VQA data points, while the expanded condition features 4,000 VH samples alongside 4,000 general VQA data points. This imbalance skews the comparison, as the expanded condition naturally emphasizes VH cases more. A more balanced comparison would involve scaling (repeating) the VH questions in baseline up to match the 4,000 samples in the expanded condition, thus equalizing the training set size and ensuring a fair evaluation.
5. Additionally, the use of symmetric accuracy in the evaluation poses a challenge. The baseline condition does not include negated questions, yet the evaluation includes them. A fairer approach would be to assess performance using standard accuracy or conduct an ablation study that excludes negation questions in the expanded condition. This would clarify whether the observed improvements in VH performance are indeed due to the proposed method.

Minor Points:

• In Tables 3-5, the metric # New successful VH test cases is used. It would be beneficial to also provide the ratio of successful cases to the total number of generated questions for easier comparisons.