Can Video LLMs Refuse to Answer? Alignment for Answerability in Video Large Language Models

[Response to Weaknesses & Questions]

Weakness 1: Evaluation on fully human-annotated set with videos from a different distribution.

Thank you for this suggestion. We want to note that the current UVQA dataset has no video overlap between the training and evaluation sets. That being said, we agree that the evaluation of different video sources will be a valuable addition to our experiments. Thus, we are currently in the process of annotating unanswerable question-and-answer pairs on videos from MSR-VTT. We will update the results once they are available.

---

Weakness 2: Trade-offs between answerability performance and standard VideoQA performance.

As is often the case in multi-objective optimization, there is a trade-off in our setup as well. This trade-off is an inherent limitation, which we have explicitly acknowledged and addressed in our work. The metrics we propose, such as $S\_{\text{ex-ref}}$ (Eq. 5), are designed to evaluate models while considering such trade-offs. For example, $S\_{\text{ex-ref}}$ quantifies the extent to which models, after alignment, decline to answer answerable questions that it was previously capable of answering correctly. A lower $S_{\text{ex-ref}}$ score is better, and unaligned models naturally achieve a score of 0. However, after alignment through SFT or DPO, models exhibit $S\_{\text{ex-ref}} > 0$, indicating some trade-off in performance on previously answerable questions.

To further address this, we added a section in [Appendix A.10: Pareto Front Analysis] to comprehensively illustrate the trade-off. Specifically, we observe that alignment for answerability improves performance on the unanswerable UVQA dataset but slightly decreases accuracy on the answerable QA dataset. Notably, the Pareto front analysis demonstrates that the aligned models (SFT, DPO) achieve a better balance, positioning them at an optimal point where performance on unanswerable questions improves without excessively compromising performance on answerable questions.

---

Question 1: Extension to images

Our framework can easily be applied to image-based vision-language models (VLMs) since images can be treated as a single frame of a video. However, we would like to note that while some prior research [1, 2] has addressed unanswerability in image-based models, to the best of our knowledge, there has been little to no investigation into this issue within open-ended QA tasks for Video-LLMs. This gap highlights the importance of our contribution. According to [1, 2], image-based VLMs demonstrate approximately 12% accuracy in unanswerability detection under zero-shot settings. In contrast, as shown in Figure 1-(a) and (b) of our paper, open-source Video-LLMs exhibit near-zero capability in detecting unanswerability. This indicates that the challenge is significantly more pronounced in the video domain, making it a critical issue to address.

[1] Whitehead S, Petryk S, Shakib V, et al. Reliable visual question answering: Abstain rather than answer incorrectly, ECCV 2022
[2] Guo Y, Jiao F, Shen Z, et al. UNK-VQA: A Dataset and a Probe into the Abstention Ability of Multi-modal Large Models. TPAMI

Thank you for your patience. We have finished the evaluation on fully human annotated set from videos from a different source. The relevent section has been added to [Appendix A.12: Out-of-Distribution Evaluation Set]. We used the MSR-VTT [1] dataset for the video source and created human-labeled unanswerable QA pairs for the evaluation.

In Table 6 of Appendix A.12, we show that the aligned model outperforms the unaligned model even on the out-of-distribution evaluation set, highlighting its ability to generalize beyond the training distribution. Although the performance gains are somewhat smaller compared to the in-distribution evaluation results in Table 1, the aligned model consistently demonstrates improved performance.

[1] Xu et al. "MSR-VTT: A Large Video Description Dataset for Bridging Video and Language", CVPR 2016

Moreover, we have also addressed the Weakness 5 from Reviewer GEbz (Human Evaluation) and added this into [Appendix A.11: Human Evaluation of Model Prediction]. The result from the human evaluation confirms that the aligned model, using our framework, outperforms the unaligned model.

We sincerely thank the reviewer once again for the time and thoughtful feedback.

[Response to Weaknesses & Questions]

Weakness 1: Missing references

Thank you for the valuable comment. Following the reviewer’s suggestion, we have added a new subsection, “Unanswerability in Question Answering”, in [Section 2: Related Work]. This subsection incorporates the references provided by the reviewer, discussing prior works on unanswerability in both LLM settings and image-based VLM settings.

---

Weakness 2: In-depth analysis of UVQA dataset by unanswerability type

Thank you for the comment. Following the reviewer’s suggestion, we have added an analysis section in [Appendix 8: Performance Analysis by Unanswerability Type], which examines the performance of answerability across different categories of unanswerability.

Specifically, we analyzed performance on three primary types of unanswerability: object, relation, and attribute.

For relation-related unanswerability, we further subdivided questions into two categories:

1. Intra-object relationships (involving a single object) and
2. Inter-object relationships (involving multiple objects).

Both of these categories were further classified into static and dynamic relationships. (Examples of each category can be found in the paper).

Our findings indicate that object and relation-related unanswerable questions achieve comparable performance, whereas attribute-related unanswerable questions pose a greater challenge. (Note that this difference in performance is not due to data imbalance in the unanswerable UVQA dataset, as we ensured the dataset was carefully balanced across these three categories during training, as noted in Section 5.1.)

On the other hand, for the subcategories of the relation-related unanswerable questions, we did not observe a consistent pattern where one type of relationship consistently outperformed or underperformed across different models.

[Response to Weaknesses & Questions]

Weakness 1-(a): The problem of refusing to answer feels a bit small and artificial.

We appreciate the reviewer’s feedback. While the problem of refusal behavior may initially seem small or artificial, we believe it has significant implications for the application of Video-LLMs in real-world scenarios. In particular, Video-LLMs are increasingly being considered for use in assistive technologies, such as answering questions about streaming video content for visually impaired or blind individuals. In these contexts, ensuring that the model appropriately refuses to answer when uncertain or when the input exceeds its capabilities is critical for maintaining user trust and preventing misinformation.

---

Weakness 1-(b) & 2: Does the problem persist on SOTA Video-LLMs with bigger model sizes, and can the problem be mitigated by using an unanswerability prompt

Thank you for the insightful comments and suggestions. To address the reviewer's concerns, we have conducted a series of additional experiments, detailed below.

[Does Model Size Affect Answerability?]

In Figure 1-(b), we have added a comparison between a 7B model and a 72B model across different variants of open-sourced Video-LLMs (LLaVA-OneVision [1] and LLaVA-Video-qwen2 [2]). While scaling up model size improves performance on traditional answerable QA benchmarks, we observe that the inability of these models to handle unanswerable questions persists regardless of model size. This indicates that scaling alone does not address the issue of unanswerability.

[1] Li, et al. LLaVA-OneVision: Easy Visual Task Transfer, arxiv 2024
[2] Zhang, et al. Video Instruction Tuning With Synthetic Data, arxiv 2024

[Can Prompting Improve Answerability?]

To explore the reviewer’s suggestion, we conducted experiments on whether explicitly adding the prompt, "If the question cannot be answered using the video content, state that it is unanswerable and provide a reason.” can improve the handling of unanswerable questions by open-sourced VLMs. These results are included in [Appendix A.9: Can Prompting Detect Unanswerability?]. We find that explicit prompting does not significantly improve answerability performance for these models, regardless of their size. This underscores the importance and relevance of the problem we address in this work.

[How About Closed-Sourced VideoLLMs Like GPT-4o?]

Our main experiments focused on open-sourced Video-LLMs, as described in the paper. To provide a broader perspective, we extended our analysis to include a closed-source Video-LLM (GPT-4o). Interestingly, GPT-4o demonstrated some ability to handle unanswerable questions, achieving an accuracy of 0.50 on the UVQA dataset, compared to 0.57 by the DPO model of VLM-RLAIF (7B) trained using our framework. This suggests that GPT-4o likely underwent internal alignment processes for answerability, as open-sourced models without such alignment consistently fail in this area. Alignment is particularly crucial for closed-source models deployed as services, as effective handling of unanswerable questions is essential for trustworthiness and safety.

While closed-source models may address this issue internally, their methods remain undisclosed. Our work emphasizes the limitations of open-sourced Video-LLMs in handling unanswerable questions and provides an open-source dataset and framework to tackle this challenge.

---

Weakness 3: Trade-offs between answerability performance and standard VideoQA performance.

As is often the case in multi-objective optimization, there is a trade-off in our setup as well. This trade-off is an inherent limitation, which we have explicitly acknowledged and addressed in our work. The metrics we propose, such as $S\_{\text{ex-ref}}$ (Eq. 5), are designed to evaluate models while considering such trade-offs. For example, $S\_{\text{ex-ref}}$ quantifies the extent to which models, after alignment, decline to answer answerable questions that it was previously capable of answering correctly. A lower $S_{\text{ex-ref}}$ score is better, and unaligned models naturally achieve a score of 0. However, after alignment through SFT or DPO, models exhibit $S\_{\text{ex-ref}}>0$, indicating some trade-off in performance on previously answerable questions.

To further address this, we added a section in [Appendix A.10: Pareto Front Analysis] to comprehensively illustrate the trade-off. Specifically, we observe that alignment for answerability improves accuracy on the unanswerable UVQA dataset (0.0 $\rightarrow$ 0.57) but slightly decreases accuracy on the answerable QA dataset (0.49 $\rightarrow$ 0.47). Notably, the Pareto front analysis demonstrates that the aligned models (SFT, DPO) achieve a better balance, positioning them at an optimal point where performance on unanswerable questions improves without excessively compromising performance on answerable questions.

Weakness 3: The metric is defined between aligned model and unaligned model. However, if one wants to test the ability of a given model regarding unanswerable questions, the proposed metric cannot be used.

To evaluate the performance of a given model, we can use the straightforward accuracy metric $S\_\text{acc}$ defined in Eq. (4). This metric calculates the overall accuracy by considering how well a given model responds to both the answerable and unanswerable questions. However, it can also be decomposed to focus on these two cases separately:

$$S\_{\text{acc, ans}} = \frac{N_1 + N_4 + N_7}{N_{1-9}}$$ $$S\_{\text{acc, unans}} = \frac{N_{12} + N_{15} + N_{18}}{N_{10-18}}$$

Here, $S\_{\text{acc, ans}}$ measures the accuracy of a given model on answerable questions, while $S\_{\text{acc, unans}}$ measures the accuracy of a given model on unanswerable questions. Along with our proposed metric defined between unaligned and aligned models, we also use these kinds of absolute performances, which evaluate the performance of a single given model, in our main result (Table 1).

---

Weakness 4: Using LLM-in-the-loop to create the dataset may create incorrect QA pairs.

While using an LLM-in-the-loop to create the dataset may introduce some noise, we have implemented filtering processes to mitigate issues such as incorrect QA pairs, as it had been detailed in [Appendix A.4.2: Dataset Filtering].

For the training dataset, we perform an automatic filtering step to remove instances where the altered description is too semantically similar to the original description. This step minimizes the risk of incorrect labeling, where the generated unanswerable question remains answerable.

For the evaluation dataset, we apply a human filtering process to ensure high quality. Specifically, we manually review all samples and remove any questions that remain answerable after the alteration process. This ensures that the evaluation dataset reflects the intended characteristics of unanswerable questions.

These steps collectively reduce noise and improve the quality of both the training and evaluation datasets.

---

Weakness 5: Human performance on the evaluation set is not presented.

We have used the $\text{LLM}\_\text{score}$ in Table 1 as a proxy for human evaluation, following a common practice in recent research where LLMs are used to evaluate generated content. This approach has been shown to correlate strongly with human preferences, as demonstrated in LLM-as-a-Judge [1].

That said, we agree that explicitly including a human evaluation is a valuable addition to the results. In response to your suggestion, we are currently in the process of performing a human evaluation. We will update on this once they are finished.

[1] Zheng et al., Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena,NeurIPS 2023 Datasets and Benchmarks Track

---

Question 1: More qualitative examples of UVQA dataset and prediction results of existing & and aligned models

Thank you for the suggestion. In response to your feedback, we have added more qualitative examples of unanswerable questions from the proposed UVQA dataset in [Appendix A.4.3: Examples of the Generated UVQA Dataset]. Additionally, we have included more qualitative examples of the predicted responses from both the existing and aligned models in [Appendix A.7: Additional Examples of Model Predictions].

[Response to Weaknesses & Questions]

Weakness 1: The proposed QAs in the dataset seem to be mostly geared toward detection capabilities without requiring much reasoning.

We appreciate the reviewer’s observation and would like to clarify that our dataset does include more complex question forms than the examples originally shown in the paper. In Appendix A.4.3, we have added more qualitative examples of unanswerable questions from our proposed UVQA dataset.

For instance, the examples include questions such as: "Why is the man with the blue-green jacket pointing at something in the video?", "In which sequence does the pug eat from a blackish dog dish in the video?", which are question forms that require contextual understanding and temporal understanding, respectively.

However, as the reviewer noted, determining unanswerability for these questions still primarily relies on the detection capabilities of the Video-LLM. We would like to note that because we are dealing with questions that go beyond the information boundaries of the video, it is natural that we rely on the detection capabilities of the Video-LLMs to discern these questions.

That said, we believe it is important to emphasize that, as shown in Figure 1-(a) and (b), open-sourced Video-LLMs, regardless of parameter size (up to 72B), consistently fail to address even these detection-based unanswerable questions. This highlights a significant pitfall in current models’ capabilities, even for what might seem like simpler unanswerable questions.

We also agree with the reviewer that extending this work to include unanswerability requiring more advanced reasoning would be a valuable future direction. Our work aims to provide a foundational step toward this goal, setting the groundwork for future research to tackle increasingly complex unanswerable scenarios.

---

Weakness 2: The metric is rather complex. Why not use F1-Score?

[Why we didn't use F1-score]

We chose not to use the F1-score because our evaluation framework extends beyond a binary classification of questions as simply answerable or unanswerable. Specifically, if the model predicts a question as unanswerable, we only consider the prediction correct if the reasoning for unanswerability is also correct. This distinction is reflected in Figure 2, where we differentiate between "unanswerability correct" ($\text{unanswerable}\_{\text{c}}$) and "unanswerability wrong" ($\text{unanswerable}\_{\text{w}}$). Similarly, when the model predicts a question as answerable and provides an answer without any "unanswerable" indicators, correctness is determined by how well the generated response aligns with the ground truth answer.

However, we agree that a simplified evaluation of binary correctness for answerable vs. unanswerable predictions can offer additional insights. Therefore, we have included this metric, using F1-score, in the revised manuscript (Table 1).

 

[Why accuracy alone is insufficient]

Moreover, we would like to clarify why we proposed a seemingly complex metrics (Eq. 5-7) rather than solely evaluating on the accuracy value (Eq. 4). Accuracy, as a standalone metric, fails to capture essential details introduced by the alignment process. For instance, consider an answerable question $Q_A$:

1. The unaligned model correctly answers $Q_A$.
2. After alignment for answerability, the aligned model either incorrectly answers $Q_A$ or predicts it as unanswerable.

A naive accuracy measurement cannot differentiate whether the aligned model's error arose from an incorrect answer or an incorrect "unanswerable" prediction. This limitation makes it difficult to analyze how alignment (e.g., the alignment algorithm $f$ in Eq. (3) or the dataset used for alignment) impacts the model's performance.

This motivated us to devise a metric that provides a more detailed and explainable evaluation of the alignment process. Our proposed metric captures various aspects, including the case described above, offering deeper insights into the alignment's effects on the model.

As detailed in our response to Weakness 5, we have completed the human evaluation and included the results in [Appendix A.11: Human Evaluation of Model Prediction]. The results show a high correlation between the human evaluation and the $\text{LLM}\_\text{score}$, further validating the use of an LLM evaluation as a proxy for human evaluation. Additionally, the human evaluation confirms that the aligned model, using our framework, outperforms the unaligned model.

We sincerely thank the reviewer once again for the insightful feedback.

• Appendix A.11 has been added to show the human evaluation of model predictions.
• Appendix A.12 has been added to show the performance on the out-of-distribution evaluation set, which uses videos from a different source and includes fully human-annotated unanswerable QA pairs.

We sincerely thank the reviewer for their valuable time, thoughtful comments, and constructive feedback, which have greatly helped improve our work.