Unraveling Cross-Modality Knowledge Conflicts in Large Vision-Language Models

Thanks for your valuable suggestions and insightful questions!

1. W1 & Q1: The study evaluates the proposed methods on only two knowledge-intensive datasets. This narrow scope raises concerns about the generalizability of the findings across different domains and types of data. The paper should consider adding other evaluations like MMBench, Vibe Eval, and Mistral evals to justify that the method can generalize well.

We appreciate the suggestions. Our study focuses on the cross-modality knowledge conflicts in VLMs. We have looked into your suggested datasets. Please understand that these general benchmarks do not provide data to evaluate cross-modality knowledge conflict. Therefore, we have to use two knowledge-intensive dataset, i.e., the ViQuAE dataset and the InfoSeek dataset, to facilitate the study of this problem.

---

2. W2: The introduction and explanation of the contrastive metric could be more detailed. Additional clarification on how it effectively distinguishes conflicting samples would strengthen the paper.

The contrastive metric quantifies the differences between the log probabilities of predicted tokens generated by the visual and textual modalities. This difference serves as a metric for the extent of conflicting knowledge between the two modalities. We included an overall figure across datasets and model scales to prove its effectiveness. From the figure, we can observe a large difference of the contrastive metric between the consistent samples and the conflicting samples.

---

3. Q2: More justification about why choosing this decoding method?

We have the following three reasons to do so.

First, the contrastive decoding method can effectively eliminate undesired logits from the original logits, thereby reducing the interference caused by conflicting parametric knowledge.

Second, since the detection process generates logits as its output, it is a natural step to work directly with these logits to refine the decoding process.

Finally, DCD dynamically adjusts the decoding process, ensuring that the final output is primarily influenced by the modality with higher confidence. This is to utilize the nature of the confidence that higher confidence from one modality may indicate that the modality could contain more relevant information than the other. At the same time, it leaves room for the other modality to contribute more significantly when its influence is necessary.

---

4. Q3: While the paper proposes using a decoding method to solve the problem, more analysis should be given to study whether the problem comes from model training or inference. While a decoding based is lightweight, I wonder whether improving LVLM training can solve the problem too.

Thank you for your valuable suggestion. We agree that cross-modality knowledge conflicts may originate during the training phase. These conflicts often result from the separate training of visual encoders and language models on distinct datasets with differing objectives, leading to misaligned representations between the modalities. And our study focuses on detecting and interpreting these cross-modality knowledge conflicts during inference time. Thus, we did not focus on the training-time solutions. Discussing further, improving the training process could potentially address these conflicts. For instance, joint training of visual and language components on unified datasets with shared objectives might foster better alignment between modalities.

Thanks for your valuable suggestions and insightful questions!

1. W1: In particular, you may need to consider Image Caption or CoT Reasoning tasks, like Coco, ScienceQA, M3CoT.

We appreciate the suggestions. We have looked into your suggested dataset. Please understand that these datasets do not fit the scope of our study. We are studying the cross-modality knowledge conflicts between the language and vision components. Thus, we want the task to require less on other abilities except for knowledge. These datasets often require the model to focus on some details of the given image, or require other visual abilities, which further widen the performance gap defined in our paper.

---

2. Q1: Add discussion of knowledge conflicts on how to multimodally generate tasks.

For the generation task, we have already included some cases of the generated answers in the appendix. In Appendix A, we present several examples of conflicting generated contents given the questions. These cases suggest that conflict still happens when given inputs from different modalities. We also sample 200 cases and manually calculate the flip rate of the generated contents. The results show that 40 cases (20%) exist of knowledge conflicts in the generation form.

Thanks for your valuable suggestions and insightful questions!

1. W1&Q2: Soundness of the cross-modality parametric knowledge conflict explanation and how is $\Delta$Acc calculated?

Thank you for your detailed analysis and thoughtful question.

First, to answer your question about how $\Delta$Acc is calculated, the $\Delta$Acc is calculated on the recognized evaluation set. However, there is a misunderstanding in your calculation. As defined in Section 3, the Flip Rate (FR) does not consider the correctness of the answers. This means that when both the textual and visual answers are incorrect but inconsistent with each other (e.g., the gold answer is A, the textual answer is B, and the visual answer is C), they still contribute to the FR. Consequently, the FR includes some cases classified as Type 4 in your analysis. We have clarified this point in the revised version of our paper.

Second, regarding the intuition behind calculating the lower bound of the Conflict Rate (CR), Section 4 aims to detect cross-modality knowledge conflicts by examining the consistency of answers across different modalities. Due to the complexity and entanglement of various factors in the generation process, it is challenging to precisely determine which inconsistencies are caused specifically by cross-modality knowledge conflicts. Hence, in this section, we first filter out those samples where VLMs cannot recognize it (lack of knowledge) and then eliminate the influence of other factors by subtracting the $\Delta$Acc from the FR, which estimates the lower bound of the conflict rate because some of the $\Delta$Acc may be caused by the cross-modality knowledge conflicts, as suggested by our Figure 2. These two steps approximately estimate the lower bound of the conflict rate by disentangling two critical factors (lack of knowledge and performance gap) from the generation process. It ensures that our estimation of the conflict rate is less affected by unrelated factors.

Thus, in this context, it’s important to clarify that the classification of the four types of cases does not fully align with the objective of this section. Specifically, this classification only counts whether the answers are correct given textual and visual inputs, not considering why the answer is flipped. Some are caused by cross-modality knowledge conflicts, while others may attribute to the model’s poor reasoning ability. Therefore, calculating the lower bound of CR is to estimate how many conflicting cases are likely to be caused by knowledge conflicts. For example, in a scenario where no cross-modality knowledge conflict exists, after confirming that the VLM can recognize the entity in the input image, the accuracy difference between the textual and the visual answer could only be caused by the performance gap defined in our paper. Thus, subtracting the accuracy difference from the flip rate ensures that these factors are eliminated.

Finally, to go beyond estimating the lower bound of the conflict rate, we introduced a contrastive metric in Section 5. This metric provides a more precise quantification of the extent of cross-modality knowledge conflicts, complementing the lower-bound analysis in Section 4.

Thanks again for your insightful analysis. We have clarified that we are reporting the lower bound of the conflict rate in the table and added the detailed explanation of the metrics in our revised version.

Thanks for your valuable suggestions and insightful questions!

1. Q1: Why was an LLM used instead of a VLM to generate incorrect distractor choices during dataset construction?

We appreciate your question regarding whether the model has access to the complete original question when synthesizing relevant choices. To clarify, the LLM is provided with the named entity from the question during the distractor generation process. This ensures the model to have the necessary context to create plausible options. Choosing LLMs is due to the fact that they are trained on extensive and diverse textual data, equipping them to generate contextually relevant and semantically appropriate distractors. Their proficiency in language processing ensures that the distractors are not only linguistically coherent but also semantically related to the correct answer. As a result, the generated multiple-choice questions maintain high quality and effectively serve their intended purpose.

---

2. Q2: What is the rationale behind down-sampling the InfoSeek dataset to match the size of the ViQuAE dataset? Comparisons should focus on models, not datasets.

The original InfoSeek is over 70k, which will take more than 20 hours for us to evaluate one 7b model, let alone a 13b or 34b model. Thus, for the sake of efficiency and budget consideration, we down-sampled the InfoSeek dataset, also to match the size of the ViQuAE dataset.

---

3. Q3: Does the flip rate account for cases where both the visual and text answers are incorrect (but not equal)? For subsets entirely consisting of such cases, N_p=N×0=0 and N_f=N×1=N resulting in CR=1. This scenario is more likely attributable to a lack of knowledge or weak reasoning abilities in the LLM rather than knowledge conflicts.

Yes, the flip rate does account for cases where both the visual and text answers are incorrect but not equal.

Exactly as you mentioned, in order to amply assess the performance degradation attributed to cross-modality knowledge conflict instead of the VLM’s lacking of knowledge or other factors, we first filter out those samples that VLM cannot recognize the named entity in it (a lack of visual knowledge) and then minus the delta accuracy from the flip rate (performance gap, as we suggested, also includes weak reasoning abilities). Through these steps, the computed conflict rate should reasonably reflect knowledge conflicts rather than other factors like knowledge deficits or reasoning weaknesses. Additionally, to quantitatively interpret knowledge conflicts, we propose the contrastive metric in Section 5, which offers a nuanced interpretation of these conflicts.

---

4. Q4. In Table 2, why is the R.Acc lower than the Acc for the Qwen2-VL-7b model on the InfoSeek dataset? Could the authors provide a possible explanation for this observation?

Thank you for pointing out this intriguing observation.

Intuitively, one would expect the R.Acc to be higher than Acc, as recognizing the entity typically indicates that the VLM possesses some knowledge about it. However, if the visual components contain incorrect knowledge, it will also cause performance drop. In the case where Qwen2-VL-7b is evaluated on the InfoSeek dataset, we believe that the visual components may contain knowledge that is inconsistent with the language model's parametric knowledge. This misalignment between the modalities could result in a lower R.Acc, as the recognized entity might lead to incorrect answers rather than improving performance. In general, issues like this highlight the necessity of checking the consistency of the language and vision components of VLMs.

---

5. Q5. In Section 4.3, the authors claim that the knowledge conflict rate (CR) is consistent across datasets, regardless of model scale. However, Table 2 shows that for the InfoSeek dataset, as the LLaVA model size increases, the CR decreases, with the reduction becoming more pronounced with larger models. This observation appears to contradict the authors' claim.

While it is true that CR decreases as the LLaVA model size increases, we note that the lower bound of the CR remains consistently high across datasets. This suggests that scaling up models enhances their cross-modality competency and overall performance, but does not entirely resolve cross-modality knowledge conflicts. Therefore, despite a reduction in CR with larger models, a significant level of conflicts persists, indicating that model scaling alone is insufficient to eliminate these conflicts.

Moreover, this conclusion is supported by the contrastive metric analysis presented in Figure 3. The consistently significant gap in the contrastive metric between consistent and conflicting samples demonstrates that cross-modality knowledge conflicts are a persistent issue across datasets, irrespective of the model scale. This further supports our conclusion that model scaling alone is insufficient to address these conflicts comprehensively.