TUBench: Benchmarking Large Vision-Language Models on Trustworthiness with Unanswerable Questions

Weakness1: Can the authors provide suggestions for future LVLM training to solve the "unanswerable" questions? From both the model perspective and training data perspective.

Thank you for your question. The following methods could enhance LVLMs’ performance on unanswerable questions from both model and training data perspectives:

1. Model Perspective: LVLMs may benefit from mechanisms to explicitly model uncertainty and determine when they lack the information needed to answer a question. For instance, training models with objectives focused on uncertainty estimation, such as confidence calibration or abstention techniques, could enable them to recognize when a question is unanswerable.

2. Training Data Perspective: A likely reason for LVLMs’ lower performance on TUBench’s unanswerable questions is their limited exposure to unanswerable scenarios during instruction tuning, which primarily features answerable examples. To address this, (a) Expanding Unanswerable Training Samples: Developing a large, diverse set of unanswerable training questions would expose models to various unanswerable conditions, allowing them to better identify missing information across different contexts. (b) Targeted Data Augmentation: Crafting datasets where answerable and unanswerable questions are generated from similar images could help the model learn subtle cues that distinguish answerable from unanswerable questions.

Weakness2: Although human-annotated questions ensure high quality, manual creation of unanswerable questions may introduce subjective biases or inconsistencies, which could affect reproducibility or lead to unintentional patterns that models might exploit.

We appreciate the insightful comment regarding the potential for subjective biases or inconsistencies in manually crafted unanswerable questions. However, to mitigate these concerns, we have considered the following aspects during the construction of the TUBench dataset:

1. Diverse Question Creation Strategies: Unanswerable questions in TUBench were generated using ten distinct strategies tailored to different contexts (e.g., occlusion, unclear details, and uncertain spatial relationships). These strategies ensured a systematic approach that minimized reliance on individual annotator subjectivity (see Section 3.1 of the paper for details).
2. Diverse Visual Contexts: TUBench spans multiple domains (e.g., code reasoning, natural images, geometric diagrams, statistical tables), further reducing the risk of domain-specific biases or reproducibility issues.
3. Quality Control: Each manually created question was reviewed by three annotators to assess its relevance, accuracy, and adherence to the guidelines. Questions flagged by more than half of the reviewers were excluded from the final dataset. The inter-annotator agreement, measured by Fleiss’ kappa, is 0.71, indicating substantial agreement among annotators (greater than 0.6) (more details in Section B on page 18). This rigorous validation process reduces the likelihood of subjective bias influencing the quality and reproducibility of the dataset.

As demonstrated in Table 2 on page7, even the best-performing models struggled to identify unanswerable questions, indicating that TUBench does not contain patterns easily exploitable by models. While no dataset is entirely free from bias, our systematic approach and rigorous quality control procedures aim to ensure that TUBench provides a robust and reproducible evaluation framework. We welcome suggestions for further enhancing its quality.

Dear reviewer 653o,

We would like to kindly follow up on our previous response to your comments and hope that the clarifications provided have addressed your concerns. As the discussion period is nearing its end, if there are any remaining questions or issues with our response, we would appreciate it if you could let us know as soon as possible. We are committed to resolving any remaining concerns to your satisfaction. In addition, in response to the reviewers' valuable feedback, we have updated the manuscript and added new experiments. In light of these efforts and the additional findings we have presented, we would kindly ask you to reconsider your evaluation.

Thank you again for your time and thoughtful consideration.

Q3: Does the 'hallucination' mentioned in the paper mean 'object hallucination'? How is the error in the UCR dataset classified in The result in Fig 5 (b)?

Thank you for your question. As stated in Line 426, the term 'hallucination' in this paper refers to content that is inconsistent with the image, and it is not limited to `object hallucination'. We provide detailed examples in Figures 13–16 in Section D.2 (page 26) of the updated manuscript. Specifically:

• Figures 13 and 14 illustrate cases of object hallucination, where the generated content includes objects that are not present in the image.
• Figures 15 and 16 depict instances where the generated content is inconsistent with the information in the image, which extends beyond just object hallucination.

Regarding the errors in the UCR dataset presented in Figure 5(b), these errors are further analyzed in Figures 15, 17, and 19 in the updated manuscript: Figure 15 showcases examples of hallucinations in the generated content. Figure 17 provides an example of errors caused by wrong calculations in the explanations generated by VLMs. Figure 19 demonstrates errors that fall outside of hallucinations or wrong calculation. For more detailed examples and analysis, please refer to Section D.2 (page 26), Section D.3 (page 30), and Section D.4 (page 32) in the updated manuscript.

Q4: Can the author provide the equation.

The equation for OACC is:

$\text{OACC} = \frac{C_u + C_a}{Q_u + Q_a}$

Where:

• $Q_u$ is the total number of questions in the unanswerable set.
• $Q_a$ is the total number of questions in the answerable set.
• $C_u$ is the number of unanswerable questions correctly identified as unanswerable.
• $C_a$ is the number of answerable questions for which the model correctly determines that the questions are answerable and provides the correct answers.

Weakness1: Some examples may be a bit not rigorous.

Thank you for your thoughtful comments. We appreciate your attention to detail regarding the example in the bottom left subfigure of Figure 1. Here is our clarification:

• Scope of Code Context: During our evaluation, we require models to base their answers strictly on the code visible in the provided image. The assumption of additional, unseen code beyond the displayed snippet exceeds the scope of the information given. Notably, the code in the figure does not imply further content, as there are no ellipses (...) or comments indicating continuation. Assuming the existence of hidden code introduces unsupported conjecture.

• Reasonable Inference from Shown Code: For the existing portion of the code (e.g., where variable x is uninitialized), it is a logical and contextually valid inference to deduce the outputs depend on the value of x. While the complete output is uncertain, certain outcomes (e.g., "hello") can be reasonably ruled out based on the visible information. Thus, our methodology reflects logical inference based on provided information rather than speculation.

• Purpose of the Design: The goal of such examples is to test whether models can accurately determine when a question is unanswerable based on visible information, without making assumptions about non-existent elements. By carefully curating the code snippet and its context, we aim to assess the model's ability to discern the sufficiency of available data to answer questions.

We hope this explanation addresses your concerns, and we welcome any further discussion or suggestions!

Weakness2: About this point, can the authors show the best results of MLLMs on the answerable subset of the dataset?

Thank you for your insightful comment. We have addressed this concern by presenting the evaluation results for both the answerable and unanswerable subsets of TuBench in Table 11 of the updated manuscript. Please refer to Table 11 and the accompanying analysis on Page 22 for detailed insights.

As shown in Table 11, Gemini-1.5-Pro achieved the highest OACC score of 65.7% on the answerable subset, which is significantly better than the random guess baseline of 20%. However, it performed poorly on the unanswerable subset, achieving an accuracy of only 26.7%, which is far below the random guess baseline of 50%. In contrast, Gemini-1.5-Flash achieved an OACC score of 36.3% on the answerable subset but demonstrated superior performance on the unanswerable subset with a 2ACC of 52.2%, substantially outperforming Gemini-1.5-Pro in this aspect.

These results highlight an important observation: the ability of VLMs to solve answerable questions does not always align with their ability to determine whether a question is answerable. We believe this emphasizes the need for further exploration and improvement of VLMs in handling unanswerable questions effectively.

Q1: Line 414 refers to Figure 13, but Figure 13 does not show that GPT4o is wrong in question answerability; it just got the wrong answer. This example seems marginal.

Thank you for your suggestion. We have carefully reviewed the example referred to in Figure 13 and agree that it does not clearly demonstrate that GPT-4o was wrong in determining question answerability. To address this, we have replaced the example with two new examples where hallucinated content led the models to incorrectly assess the answerability of the questions. Please refer to Line 414, Figure 14, and Figure 15 in the updated manuscript for these new examples.

Q2: What bad outcomes in which situations would the weakness of MLLMs on this problem lead?

Thank you for the insightful comment. The significance of the proposed problem lies in addressing the critical risks posed by hallucinations of VLMs when encountering unanswerable questions. Prior research has shown that these models may generate confident yet incorrect responses even when the necessary information is absent (Yin et al., 2023b; Sun et al., 2024). This behavior is particularly concerning in domains requiring high reliability, such as medical diagnosis and autonomous driving. For example, in medical contexts, a hallucinated diagnosis or treatment recommendation could lead to severe health consequences or even loss of life. Similarly, in autonomous driving, an incorrect response to a question about environmental conditions or vehicle status could result in unsafe decisions, potentially causing accidents.

The expectation for models in these situations is clear: they should refrain from generating fabricated responses and instead explicitly acknowledge their inability to answer. Addressing this issue is essential to ensure the trustworthy deployment of MLLMs in critical applications, highlighting the importance of systematically evaluating and mitigating hallucination risks.

We highlight the research significance of our research in Lines 57-64 of the paper.

3 Response to ''Second, on the above two datasets, there are cases when the overall 2ACC between models is similar, but the 2ACC on individual subsets is very different. I feel like these (and potentially more) observations need more explanations to make the behaviors of the models clearer.''

Upon comparing Table 2 and Table 11, we observe that while the overall 2ACC scores of some models are similar, the 2ACC scores on the unanswerable and answerable sets, show noticeable differences. This suggests that the models exhibit different inherent tendencies. For instance, Qwen-VL models tend to classify questions as answerable, even when they are actually unanswerable. In contrast, Gemini and GPT-4 models have lower inherent tendencies to classify questions as answerable, indicated by the lower average 2ACC scores on the answerable set, but higher 2ACC scores on the unanswerable set.

We hope our explanation addresses your concerns, and we welcome any further discussion or suggestions!

Thank you for your follow-up comments.

1 Response to " It is still unclear to me whether there is a coherent standard for answerability in UCR. If the assumption of no hidden/unseen code is not allowed, then the answers to the bottom right example in Figure 1 should be changed because the code has a bug, and no output will be generated for sure."

Our methodology adheres to a consistent standard: models are required to base their reasoning strictly on the information provided in the images, avoiding unsupported assumptions or excessive reasoning. For the code reasoning dataset, models should derive their conclusions from the code snippets provided in the screenshots. The reviewer’s concern about "other parts of the code may change the result" represents an unsupported and excessive inference.

First, there is no evidence in the code snippets (e.g., the bottom-left and bottom-right of Figure 1) to suggest that additional code segments exist in the original code that could alter the output. Thus, this reasoning lacks a foundation and constitutes excessive speculation. Adopting such a perspective would render all questions related to the code snippet in top-left of Figure 1 unanswerable. Moreover, other widely used code reasoning datasets, such as MME [1], would face analogous challenges if we accepted this excessive reasoning standard.

When constructing the code reasoning dataset, all of our reasoning is based strictly on the code provided in the screenshots. For example, in the bottom-left snippet of Figure 1, the output depends on the value of the variable x. Since the image does not provide a value for x, the output of the code depends entirely on its value. However, there is no evidence in the image suggesting the presence of additional code lines that could modify the result. Thus, we can conclude that the output of the code can only be “Hello” or “World” and cannot be “hello” or “world.”

Similarly, for the bottom-right code snippet in Figure 1, we can infer that the output involves an incomplete segment related to two print statements. While the specific output cannot be determined due to the incompleted code lines, there is no evidence in the snippet to support the assumption that other code segments outside the image might modify the output. Thus, the reasonable conclusion is that the original output begins with “Hello,” based on the visible information.

[1] MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models (https://arxiv.org/abs/2306.13394)

Q3: UGeoQA contains questions in Chinese and most VLMs are predominantly English. Therefore, it is unclear if the ineffectiveness of the VLMs derive from inability to process Chinese or actual reasoning.

Thank you for your insightful question. It is widely recognized that Chinese is not a low-resource language, and all seven proprietary models evaluated in our study support multiple languages, including Chinese. Therefore, the poor performance of these models on UGeoQA is primarily due to their challenges in handling unanswerable questions rather than an inability to process the Chinese language.

To further clarify, we present a detailed comparison of model performance on answerable and unanswerable subsets of TUBench, including UGeoQA, in Table 11 on page 22. We invite you to refer to the updated manuscript. The results demonstrate that proprietary VLMs generally perform significantly better on answerable subsets than on unanswerable subsets. For example, Gemini-1.5-Pro achieved a 2ACC score of 95.9% on the answerable subset. However, Gemini-1.5-Pro performed poorly on the unanswerable subset, achieving a 2ACC score of only 26.7%, which is substantially below the random guess baseline of 50%. Moreover, Gemini-1.5-Pro achieved an OACC score of 65.7% on the answerable subset, far exceeding the random prediction baseline of 20%. This indicates that it possesses a certain level of competence in addressing answerable questions of UGeoQA. This stark contrast underscores that the ineffectiveness of these models on UGeoQA stems primarily from their inability to handle unanswerable questions effectively, and is not due to the models' limitations in processing the Chinese language.

Q4: What the 'Other' category of explanation evaluation categories consists?

The 'Other' category in the explanation evaluation refers to errors that do not fall under the categories of hallucinations or wrong calculations. We provide two illustrative examples of other errors in Section D.4 on pages 32-33 of the updated manuscript. In the first example, the error occurs because the VLMs overlooks the case-sensitive nature of the output required by the question. In the second example, the error arises from the model's wrong conclusion. For more details, we invite you to refer to Section D.4 on pages 32-33 of the revised manuscript.

Weaknesses1: The generation of approach for UVQA and UTabMWP domains are less diverse; specifically in UVQA, perturbations in relations/attributes using scene graphs to generate unanswerable questions have been already addressed in current literature [a].

Thank you for your insightful comment! The primary goal of our work is to evaluate the performance of VLMs when dealing with unanswerable questions. Literature [a] primarily generates unanswerable questions using two strategies: (1) objects mentioned in the question are absent in the image, and (2) the objects exist, but those with specific attributes are missing. In contrast, our proposed TUBench adopts ten diverse strategies to construct unanswerable questions.

For the UVQA dataset, we use strategies such as occlusion of objects in the image (S.4), unclear object details (S.5), missing overall object information (S.6), and uncertain spatial relationships (S.7) to construct unanswerable questions. These strategies provide a broader diversity of unanswerable scenarios compared to literature [a]. Furthermore, while literature [a] uses natural images exclusively, TUBench integrates visual contexts from four distinct domains: natural images, code snippets, geometry diagrams, and statistical tables, adding significant diversity to the data.

We also compared our unanswerable question construction approach with the image replacement method (detailed on page 10, ''Comparison to Image Replacement in UVQA''). Image replacement typically generates unanswerable questions by substituting original images with other ones, where unanswerability arises because the objects mentioned in the questions are absent from the images. Our results, supported by examples in Figures 24, 25, and 26 (page 36), indicate that questions constructed with unclear details, and spatial uncertainties pose a greater challenge to models. Specifically, Figure 24 highlights unanswerable questions caused by uncertain spatial relationships and unclear details. In contrast, Figures 25 and 26 feature images where the objects mentioned in the questions are entirely absent. Figure 25 is semantically similar to Figure 24, whereas Figure 26 is semantically distant from it. We observed that VLMs found it relatively easy to recognize the unanswerability of questions in Figures 25 and 26 but struggled significantly more with those in Figure 24.

In summary, TUBench presents a significantly more challenging and diverse benchmark for evaluating VLMs' trustworthiness compared to prior works. For further details and comparisons, please refer to the examples and experimental results in Sections 4.4 (page 10) and D.7 (page 36) of our updated paper.

Weaknesses2 & Q1: UCR, UVQA are posed in Y, N, U option order. Has there been analysis with changing the option order?

Thank you for your insightful comment! We evaluated the impact of option order on VLMs including GPT-4o on the UCR and UVQA datasets. Our analysis shows that while adjusting the option order has a slight impact on the performance of VLMs, it does not improve their ability to determine whether a question is answerable. This highlights the significant challenges posed by the proposed datasets to current VLMs. For further details, please refer to Section D.8 on page 39 of our paper.

Q2: In UTabMWP, table being incomplete derives in Unanswerable situation which is very easy to embed into the VLM with few-shot examples.

Thank you for your insightful comment! We conducted experiments with VLMs including GPT-4o under 3-shot and 6-shot settings. However, the ability of GPT-4o to determine whether a question is answerable did not show significant improvement in the few-shot setting. This indicates that simply providing demonstration examples is insufficient to address the issue effectively. For further details, we kindly refer you to Section D.9 on page 39 of our updated paper.

Dear reviewer m7Wd,

We would like to kindly follow up on our previous response to your comments and hope that the clarifications provided have addressed your concerns. As the discussion period is nearing its end, if there are any remaining questions or issues with our response, we would appreciate it if you could let us know as soon as possible. We are committed to resolving any remaining concerns to your satisfaction. In addition, in response to the reviewers' valuable feedback, we have updated the manuscript and added new experiments. In light of these efforts and the additional findings we have presented, we would kindly ask you to reconsider your evaluation.

Thank you again for your time and thoughtful consideration.

Weakness1: Present the performance of the LVLM models with answerable questions and unanswerable questions.

Thank you for your suggestion. We present the evaluation results for the unanswerable and answerable subsets of TUBench in Table 11 of the updated manuscript. For more details, we invite you to refer to Table 11 and the corresponding analysis provided on Page 22 of the revised manuscript.

Weakness2: Does this indicate correctly answering the answerable questions is also challenging?

Figure 5(a) presents the human evaluation results for 50 answerable and 50 unanswerable questions—not just answerable questions. Figures 15(a) and 16(a) show the results for the answerable and unanswerable subsets, respectively. We speculate that the reviewer is asking whether, based on Figure 15(a), it can be inferred that correctly answering the answerable questions is also challenging.

It is important to note that although the results for answerable questions are shown in Figure 15(a), the models did not know in advance whether the questions are answerable. To correctly answer these questions, the models must first determine that the questions are indeed answerable and then provide the correct answers (as measured by the OACC metric).

From Figure 15(a), we can infer that it is challenging for the models to both determine the answerability of questions in the answerable subset and correctly answer them. However, this does not imply that correctly answering questions in the answerable subset is challenging, since this conclusion assumes that the models already know the questions are answerable and only need to provide the correct answers.

The reviewer may also refer to our response to #Q1# below for a clearer understanding of the distinction between these two aspects.

Q1: Why is OACC introduced to measure the combination of both accuracy of answerability classification and the accuracy for answerable questions? Is the redundant?

Thank you for your thoughtful question. The primary goal of our paper is to evaluate how models perform when faced with unanswerable questions. Therefore, our fundamental assumption is that the answerability of a question is unknown beforehand.

Introducing a metric that focuses solely on the accuracy of answerable questions would require informing the models in advance whether the questions are answerable. This assumption contradicts those of the first two evaluation metrics, F1 and 2ACC, which evaluate performance without prior knowledge of answerability. Moreover, due to this inconsistency in assumptions, we would need to re-evaluate the models under the new assumption for the same set of questions. For the GPT-4 series models, this would incur additional evaluation costs.

In contrast, the OACC metric provides a holistic view by simultaneously considering the accuracy of answerability classification and the accuracy for answerable questions. Importantly, it adheres to the same assumption as the F1 and 2ACC metrics, enabling us to evaluate all three metrics in a single evaluation run. This consistency streamlines the evaluation process and avoids additional computational and financial overhead.

We hope this clarification demonstrates why OACC is not redundant but instead complements the other metrics by maintaining consistency while offering a broader perspective. For the prompts used in our evaluation, please refer to Table 7 on page 20 of the revised paper.