MM-R$^3$: On (In-)Consistency of Multi-modal Large Language Models (MLLMs)

Adapter Overhead: What is overhead brought by the adapter?

Adapter parameters: BLIP-2 adapter parameters are 376M and LLaVA adapter parameters are 268M. The original number of parameters in BLIP-2 is 12.1B and in LLaVA is 7B. This means that the overhead of the adapter is only 3.1% and 3.8% respectively.

Inference speed: We run 100 examples on the original LLaVA and adapter + LLaVA (ours). The processing time for the original LLaVa is 89.4 seconds and ours is 91.2 seconds on a single GPU with batch size = 1.

Impact on Downstream Tasks: Does the adapter module impact the models' performance on other downstream tasks?

We evaluate the OKVQA dataset to validate the performance of the downstream task before and after the adapter-based fine-tuning of the MLLM model.

• Original LLaVa 1.5M (temperature = 0): Acc = 58.04
• Finetune LLaVa 1.5M (temperature = 0): Acc = 57.12

The number does not drop after adding the adapter, indicating the proposed adapter can not only preserve the ability of MLLM but also improve the consistency.

Error Analysis: Can you provide more insights into the types of inconsistencies observed in the models and how the adapter addresses them? For example, are there specific patterns or error types that the adapter helps mitigate?

While it is difficult to quantify specific trends due to variability of tasks and questions. One interesting behaviour that we observe are inconsistencies in numeric responses and how the adapted model is able to address them by inducing consistency. Some examples are below:

Example 1:

gt: 36

original: 250, 10%, 69%

finetune: 50%, 40%, 42%

Example 2:

gt: $2,114.99

original: no., 1200, $2698

finetune: $2,926,

$2,823,

$2,355

Could you provide an analysis of how different pretraining strategies or model architectures impact consistency?

We notice that trends with respect to architecture and training strategies depend on types of perturbation that we consider. For perturbations to visual inputs (stylizations), we notice that even though the performance of the BLIP-2 model and LLaVa model are similar in terms of accuracy (Tab3: image restyling) the consistency of the BLIP-2 model is much less compared to the LLaVa model. This points to the fact that models without instruct tuning, such as BLIP-2, are weaker and more susceptible to stylizations-based perturbations to input images and less consistent than LLaVa family counterparts.

We notice from Table 5 that in terms of architecture, scaling the MLLM language decoder to a larger size (13B vs 7B) helps make the model more consistent overall. We attribute this to the fact that using a larger language decoder during the fine-tuning stage of LLM training helps with more effective knowledge transfer between visual and language modality making the models less susceptible to changes to input which leads to more consistency.

On the other hand, we notice that in the case of occluded objects that require reasoning based on the overall semantic context of the scene, the BLIP-2 model is much more consistent than the LLaVa models. We attribute this to the hybrid of multiple losses (image-text matching and image-text contrastive learning) used pretraining of Qformer of Blip-2 which helps them capture the overall semantic context of the image better compared to LLaVa family of models which does not involve pretraining with these losses.

While the results are intensive, it is a bit overwhelming to look at each of the three tasks and four metrics one by one. Is there a metric that can be a good proxy of all the results, or can the average be a good representative?

This is an excellent suggestion. Motivated by works in generalized few-shot recognition, we propose to use the Harmonic mean of correctness and consistency as a single good proxy metric. We first calculate the average of Acc and S_GT, two metrics that evaluate correctness against the ground truth. Next, we compute the average of Con and Sc, two metrics assessing the consistency of generated responses. Finally, we combine these two averages into one single score using the harmonic mean, as we believe this approach can reduce bias when averaging values with large disparities. We use harmonic mean since ideally we want a model to be both correct and consistent and it helps balance the performance between these two key aspects.

Final_score = Harmonic_mean(mean(Acc + SGT), mean((Con + Sc))

After adding the adapter, does the performance on standard datasets drop?

We evaluate the OKVQA dataset to validate the performance of the downstream task before and after the adapter-based fine-tuning of the MLLM model.

• Original LLaVa 1.5M (temperature = 0): Acc = 58.04
• Finetune LLaVa 1.5M (temperature = 0): Acc = 57.12

The number does not drop after adding the adapter, indicating the proposed adapter can not only preserve the ability of MLLM but also improve the consistency.

This paper is not the first one to study “consistency” in VQA/MLLMs. More discussion and comparisons with existing works should be provided.

To the best of our knowledge, our paper is the first to do an in-depth study of diverse MLLMs and improve consistency in the realm of multimodal vision language models which integrate a visual encoder with an LLM and give continuous textual captions/responses to queries. We are aware of papers that study this property in LLMs, which we discuss in related work. We will be happy to provide additional discussions and comparisons if the reviewer can identify specific works with respect to which such discussions and comparisons should be made.

Fairness of comparison.

For a fair comparison, we train our adapter only on the original unmodified OKVQA + InfographicVQA dataset (for rephrasing task); and the original unmodified Indoor Scene + Google Landmarks Dataset v2 (for the image restyling task). We call this variant (task adapted). Note that (task adapted) and (consistency adapted – reported in the paper) models have identical architectural structure and are both optimized. In other words, the comparison is no longer with zero-shot (original). Due to the time limit, we conducted the experiment using the LLaVa model only. Nevertheless the trends are clear, while training with task-specific data increases the accuracy on the task (an expected behavior), it does very little to improve the consistency (see Con measure). Training the adapter with rephrasing and restyling data substantially improves the consistency, while not negatively impacting, and typically marginally improving, the accuracy. Specifically the improvement in consistency is 33.2 -> 43.2 and 22.5 -> 32.6, over 10 points for each of the tasks. The lack and quantification of consistency in original models and improvement of consistency through training of the adapter are our key contributions.

Why choose different source datasets?

The choice of datasets stems from our desire to maintain diversity in our benchmark, as correctly pointed out by the reviewer, as well as availability of annotations in various source datasets. MSCOCO provides instance segmentation masks, making it ideal for context reasoning task (which requires object masking). MSCOCO itself does not include question-answer pairs (which would be needed for the rephrasing task) or clear scenes to recognize (which is how we organize the restyling task). Some of the MSCOCO images do appear in the VQAv2 dataset, so questions could potentially be obtained from there. We have done some preliminary experiments using such data during the rebuttal, and results are consistent with those obtained on the proposed dataset. Mainly, the relative ranking of models is nearly identical for both accuracy and consistency. The overall accuracy, however, tends to be considerably higher for MSCOCO images (e.g., for Qwen-VL-Chat by as much as 22 to 30 points for rephrasing and restyling), showing that our originally chosen dataset is actually a lot more challenging.

Evaluation of the downstream task.

We evaluate the original unmodified OKVQA dataset to validate the performance on the downstream task before and after the adapter-based fine-tuning of the MLLM model.

• Original LLaVa 1.5M (temperature = 0): Acc = 58.04
• Finetune LLaVa 1.5M (temperature = 0): Acc = 57.12

The number does not drop after adding the adapter, indicating the proposed adapter can not only preserve the ability of MLLM but also improve the consistency.

Evaluation procedure.

1. Calculation of Accuracy (Acc): To clarify line 244, we do case-insensitive substring matching to validate the response. This works because GT responses tend to be single words or short phrases. Consider the example in Figure 5, the answers in the question rephrasing task from LLaVa are “columbia”, “north face”, and “no” and the ground truth answer is “north face”. Hence, Acc for three answers is 0/100/0. As a result, the average score for this example will be 33.3 as reported.
2. Calculation of similarity with GT (S_GT): As the exact match criterion has some limitations, i.e. it may inaccurately categorize semantically similar responses as incorrect, we use a similarity metric in the form of Sentence BERT embeddings.
3. Calculation of ​​Consistency Accuracy (Con): We compute the pairwise similarity scores between responses using Sentence BERT and utilize a threshold of 0.7 to delineate semantic consistency. Consider again the example in Figure 5, the answers in the question rephrasing task from LLaVa are “columbia”, “north face”, and “no”. Since none of these are semantically similar to one another, the pair-wise Sentence BERT scores are 0.27/0.14/0.24 — all below 0.7 threshold and resulting in Con of 0.
4. Calculation of Consistency Similarity (SC): We compute the pairwise similarity scores between responses and average them. Using the same example above, the SC score will be (0.27+0.14+0.24)/3 = 0.21.

Adversarial robustness vs. consistency.

We appreciate the relationship pointed out by the reviewer. We are aware of the adversarial robustness literature but did not think it was sufficiently close to be discussed. In retrospect, we agree that we should have discussed it as part of the related work. We will revise the manuscript to add such discussion. That said, there are important differences between adversarial robustness (and works cited by the reviewer) and consistency we study in this paper. Specifically, while there is a great variety of works in adversarial robustness, let us contrast them with our work in terms of their main tenets:

1. Most adversarial robustness approaches [A, B, C, D, E] operate in classification settings. Models such as [F, G, H] operate on LLMs that have no entitlement of vision and language, and [I] only studies CLIP (which is a particularly simple contrastive VLM variant). Notably, none of the models deal with VLMs with continuous text outputs.
2. They assume the presence of an adversary agent that attempts to find small, local, and often imperceptible, perturbations to inputs (e.g., [E] propose pixel level noise perturbations for vision models; [G] propose typos and synonyms for LLMs; [H] propose character and word level deletions, repetition, etc.), that “produce an incorrect response” [G] or a “decrease in overall classification performance” [I]. In other words, robustness is closely tied to accuracy; i.e., robustness only makes sense in the context of a capable model, for samples that the original model is able to classify correctly.
3. Adversarial robustness models, particularly those that attempt to provide theoretic guarantees, quantify worst-case performance under an adversary attack.

In contrast, in studying consistency in VLMs we:

1. Focus on a broad class of VLM models that produce open-world textual outputs (including both open- and closed-sourced); this is well beyond CLIP discussed in [I] (which is the closest among suggested citations).

2. We focus on semantic input perturbations (rephrasing and restyling) of both visual and lingual modalities and semantic output equivalence. This is much harder to achieve and quantify. This also goes significantly beyond local word/character perturbations in LLMs or pixel noise perturbations in vision robustness literature.

   Importantly, the notion of consistency is entirely devoid of the accuracy or correctness of the original model. Specifically, we study consistency for both all responses and specifically failure cases (see supplementals). A model can be trivially consistent by always responding with the same phrase, irrespective of the input, however, such a model would not generally be considered either accurate or robust under most standard definitions of those two properties. Further, consistency does not assume an adversary, but rather a cooperative agent. In other words, the only perturbations we consider are those likely to be generated by a “typical” user (not one that tries to fool a model). Overall, consistency does not guarantee robustness.

   On the other hand, a robust model may also not necessarily guarantee consistency, because typical robustness measures ability for an adversary to flip the decision from correct to incorrect. In more complex tasks (e.g., VQA, captioning), there may be multiple correct answers and also many ways to be incorrect. Consistency measures semantic equivalency even within these classes, which robustness typically does not.

3. Finally, consistency as we define it, is a measure of average performance under semantic perturbation, not one of worst-case performance.

We will elaborate on these connections and differences in the revised manuscript.

Validation of dataset quality.

We appreciate the concern raised by the reviewer. We would like to clarify that 100 samples are 0.2% of the dataset and not 0.002% as stated by the reviewer. Nevertheless, the point stands. Since validating a large portion of the dataset (with 87,000 samples) manually would be exceedingly costly and time-consuming, we adopt two alternate strategies to further evaluate the quality of our dataset and address the concern. (1) We human validate additional 200 samples during the rebuttal period and find the statistics (on over 300 random samples now) are not very different (93% semantic equivalency for language rephrasing vs. 92% reported on 100 samples in the paper; 85% semantic equivalence for restyling vs. 86% reported on 100 samples in the paper). These additional results illustrate that the quality metrics reported are stable and reflective of the dataset as a whole. (2) we use the InternVL-26B [1] model (a strong VLM not part of our analysis, with capabilities exceeding GPT4-o in many cases) to automatically validate ALL of the data for the rephrasing task and find it to be 88% semantically equivalent according to InternVL. Note that this is likely a lower bound as InternVL itself is not perfect. However, this further validates the quality of the dataset.

[1] InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks, Z. Chen, J. Wu, W. Wang, W. Su, G. Chen, S. Xing, M. Zhong, Q. Zhang, X. Zhu, L. Lu, B. Li, P. Luo, T. Lu, Y. Qiao, J. Dai, CVPR 2024.

Why exotic visual perturbations? Why require an MLLM to correctly guess an occluded object?

These tasks are designed to probe the “property” of consistency in MLLM models since humans inherently exhibit this property in their responses. Specifically, our definition of consistency is motivated by human and social psychology and the Cialdini’s Principle of Consistency. The Cialdini’s consistency principle states that people are motivated toward cognitive consistency and will change their attitudes, beliefs, perceptions, and actions to achieve it. In other words, humans in certain experimental settings prefer consistency over more objective measures. We believe that it is important for models to also exhibit consistency in order to operate convincingly in tandem with human users. This would also go a long way towards closing the gap in building trust and MLLM use for, and in, decision making processes. Further, please note that we do NOT require models to guess correctly, we mainly require and measure their ability to respond consistently. In other words, a model that responds incorrectly but the same for the various perturbations will be deemed 100% consistent.

Whether the task is realistic or not, is somewhat irrelevant for our ability to measure this property. For example, our restyling task measures whether models respond consistently when texture cues are removed via stylization. Previous work [1,2] shows that humans are shape-biased (and do not change decisions when texture cues are modified via stylization). Operating similar to humans, i.e., being shape-biased has various downstream benefits like better recognition performance and overall robustness. Similarly, humans can guess masked objects from the context of the scene, and even if the guess is incorrect, they would persist on the chosen answer irrespective of the type of mask. The essence of our task is not to directly solve a downstream problem but a way to probe a property of MLLMs that humans inherently exhibit in their decision-making/responses. In fact, we argue that asking more open-ended questions in an ill-posed task is conducive to enhancing our ability to measure the intrinsic consistency/inconsistency of such models. The key insight is that even for these (perhaps unrealistic) perturbations of an image and ill-posed tasks, a given person will respond consistently, and we should expect MLLM models to do the same. This is the key metric that we measure.

[1] ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness, R. Geirhos, P. Rubisch, C. Michaelis, M. Bethge, F. Wichmann, W. Brendel, ICLR 2019.

[2] Does enhanced shape bias improve neural network robustness to common corruptions, C. Mummadi, R. Subramaniam, R. Hutmacher, J. Vitay, V. Fischer, J. Metzen, ICLR 2021

Why choose BLIP-2 and LLaVa?

As mentioned in Section 5.2 Implementation Details (line 445), we choose BLIP-2 and LLaVa 1.5M for consistency improvement experiments because they are widely used models, have low consistency compared to other models and allow us to show the efficacy of our approach on different types of MLLM families (i.e., ones that use only CLIP v.s Qformer based architectures). These models have also served as the foundation of many newer SoTA open-source MLLM models (such as, BLIP3 and LLaVA-NEXT). Generally, we view our adapter and experiments in the paper as a proof of concept that one can improve consistency of MLLM models without necessarily impacting their accuracy. While we show that the adapter can be used to improve consistency of pre-trained models effectively, ultimately we imagine that consistency objectives would be embedded into RLHF fine-tuning and other mechanisms of training MLLMs in the future.

Image restyling is still low compared to other models.

We agree that the image restyling task showed less improvement compared to the question rephrasing and context reasoning tasks. We believe this could be due to the inherent difficulty of the task for MLLMs, which have generally not seen images of this form.

Figures 2, 3, and the embedded figures are too small to view clearly.

Thanks for the suggestion. We will enlarge the figures in the revised version as much as space permits.

How does the stochasticity affect the measured inconsistency based on rephrasing, restyling and context reasoning tasks?

As shown in Figure 3: Impact of Entropy, consistency in BLIP-2, BLIP-3 and Qwen-VL-chat are less affected by the stochasticity, while LLaVa 1.5, MoE-LLaVa and mPlug-Owl2 are more sensitive to the entropy parameters.

How does the proposed adapter address the consistency caused by the stochasticity?

To address this question, we run different entropy on the LLaVa 1.5M model with the proposed adapter. Expectantly, higher temperature does lead to lower accuracy and consistency. However, adapter counterparts perform significantly better in consistency, compared to non-adopted counterparts, for the same temperature. Specifically, we see 10.63 point improvement at temperature of 0.7 and 13.81 improvement at temperature of 0.2 in terms of Con metric; improvement in Sc are similar ~10%.

Thank you for answering the questions. I will keep the original rating.