SURE-VQA: Systematic Understanding of Robustness Evaluation in Medical VQA Tasks

Thank you again for your valuable comments and for taking the time to read our general reply, as well as considering our point-by-point comments here:

---

W1. [...] There may be a case that distribution shifts indeed exist in the dataset but the dataset distribution is totally included in the pre-training data of VLMs [...]. I think a definition regarding 'distribution shifts' here is required [...]

• Thank you for bringing up this important point. We think this is a misunderstanding because we totally agree with you: defining i.i.d. and OoD is challenging, particularly in the context of foundation models due to the extensive scope of training data, as also discussed in lines 508-511 of our manuscript (Generalizability of the Framework).
• Thus, in our study, we specifically define i.i.d. versus OoD in terms of the distribution shift between the fine-tuning dataset and the test data, allowing us to control for the shifts in the data. Further, we ensured that the datasets used for fine-tuning and evaluation are sourced independently from those used during the pre-training, eliminating any potential data leakage.
• To clarify our approach, we have updated the manuscript at lines 150-154 of our manuscript to more prominently highlight our definitions of i.i.d. and OoD within the scope of this study.

---

Q1. In human rater study, the kendall correlation between two human raters is supposed to be the highest and the Mistral is expected to be closest to humans' scores. But in figure 3, the correlation between humans is even the worst in SLAKE and not the highest in OVQA. How to understand this phenomenon?

• Thank you for highlighting this observation. Notably, with our more comprehensive human rater study (see general reply), this behaviour is no longer observable. Generally, we attributed this behavior to the fact that the metrics fall between the evaluations of the two human raters, resulting in a stronger correlation with the average human rating than between the individual raters themselves.
• However, we would like to highlight once again that increasing the number of raters seems to reduce this effect, making the study design overall more robust.

---

Q2. In Figure 2, the authors define several distribution shifts. I was wondering if these shifts are commonly ackonwledged in OoD fields [...]?

• We mainly base our shifts on the taxonomy of Castro et al. [1], a highly cited work in the OoD field, with the taxonomy also being used in other papers, e.g. [2], [3], [4]. The only shift type that is not based on this taxonomy is the question type shift, which we introduce in order to have shifts in both modalities. This taxonomy is also positively perceived by reviewers 5eB8 and UkoV.
• As also mentioned by Castro et al. “these terms correspond roughly to particular dataset shift scenarios studied in general ML literature, namely ‘covariate shift’, ‘concept shift’, ‘target shift’, ‘conditional shift’ and ‘domain shift’ [...].” [1]
• Relating to [Gulranjani et al.], our shifts focus on “domains arising from natural processes”.
• To highlight the relevance of our selected shifts, we have added citations to these works in lines 242-244 of our manuscript.

[1] Castro et al. “Causality matters in medical imaging”
[2] Bungert et al. “Understanding Silent Failures in Medical Image Classification”
[3] Roschewitz et al. “Automatic correction of performance drift under acquisition shift in medical image classification”
[4] Choi et al. “Translating AI to Clinical Practice: Overcoming Data Shift with Explainability”

---

Q3. [...] The selected shifts, though realistic, could be expanded to reflect more clinically meaningful scenarios. Shifts in patient demographics [...] are useful here, but modality and especially question type shift may not correlated with medical setting.

• We agree that there may be additional shifts worth investigating to further enhance clinical relevance and added it as a dimension to investigate in line 527 of our manuscript.
• Regarding modality shift, we acknowledge that it is a rather extreme shift that may not directly occur in medical practice, but rather more subtle versions of it, like encountering new scanner types from different manufacturers. However, we included it to represent shifts of varying severities, allowing us to evaluate the model's robustness under more extreme conditions.
• In contrast, we consider the question-type shift to be clinically relevant. It is plausible in a medical setting that a physician might ask about aspects not covered in the model's training or fine-tuning data. As medical knowledge and practices evolve over time, new question types may emerge that the model has not encountered before.

---

Thank you once more for your constructive feedback. As we believe to have addressed your concerns, please let us know if there are any remaining concerns that would hinder a recommendation for acceptance.

Thank you again for your valuable comments and for taking the time to read our general reply, as well as considering our point-by-point comments here:

---

W1. The article [...] lacks a more explicit and comprehensive definition of the robustness of VLMs in Med VQA tasks. It may necessary to further compare with the existing work on the evaluation of VLM generalization ability [...]

• Thank you for your feedback. We have revised the manuscript to address your concern about providing a more comprehensive definition of the VLM robustness in Med VQA tasks. We have added an explanatory text to lines 043–045.
• For a detailed discussion on existing VLM generalisation work, we kindly refer you to sections P1-P3 in the manuscript where we analyse the current literature and discuss the pitfalls.

---

W2. [...] The authors list several scenarios [...], which appear to resemble an evaluation of the model's domain adaptation capability. Are there any related works on VLM domain adaptation capabilities in the medical field [...]?

• Thank you for your comment. We would like to clarify that our study is more closely aligned with Domain/OoD generalisation rather than domain adaptation. While both address domain shifts, domain adaptation utilises target domain data to adapt models, whereas Domain/OoD generalisation works without access to target domain data and focuses on detecting shifts [1,2].
  • "DA assumes the availability of labelled or unlabeled target data (i.e., unsupervised DA, UDA) for model adaptation" [1]
  • "For OoD generalisation, one only has supervised data from the underlying data distribution on the training domain." [2]
• We acknowledge the importance of highlighting this connection more explicitly. To address this, we have included a discussion on this topic in the revised manuscript at lines 043-045.

[1] Yoon et al. "Domain Generalization for Medical Image Analysis: A survey"
[2] Liu et al. "Learning Causal Semantic Representation for Out-of-Distribution Prediction"

---

W3. Regarding the [...] the model's robustness [...] where performance shows opposing results between SLAKE and OVQA, is there an explanation for this? [...]

• We appreciate the reviewer’s observation. The behaviour differences in SLAKE and OVQA stem from how OoD questions align with the training data. While OoD questions are absent from the training set, training questions can provide enough context for effective generalisation during testing. In SLAKE, open-ended questions like "What is the largest organ in the picture?" align with training data such as "What is the main organ in the image?", enabling better generalisation, whereas the closed-ended question "Which is smaller, [x] or [y]?" require reasoning not covered in training, leading to weaker performance.
• In OVQA, the pattern is reversed, with the closed-ended question "Is this a study of [x]?" aligning with training tasks including questions like "Does this image show a normal [organ]?" and showing strong generalisation, while open-ended questions like "What bones are evaluated primarily?" require detailed reasoning not supported by the training data, resulting in poorer performance.
• We acknowledge the importance of this point and have clarified it in the manuscript (lines 478-482).

---

W4. [...] It would be beneficial to summarize the key factors that significantly influence the robustness of VLMs in Med VQA tasks. Additionally, further discussion on methods for improving model robustness on out-of-distribution (OoD) data, based on the analysis of the impact of PEFT methods on model robustness, would be valuable. Relevant literature from general domains [...] could provide useful insights.

• We appreciate the reviewer’s insightful feedback and have updated our results section accordingly. Specifically, in lines 520–523, we now explicitly state that robustness alone is not a determining factor when selecting fine-tuning methods and different shifts may uniquely challenge model performance.
• We also agree that the methods discussed by Yoon et. al. and Ma et. al. are relevant to enhance robustness, thus we cite them as a starting point of future work to enhance robustness and benchmark the methods within our framework in lines 533-535.

---

Thank you once more for your constructive feedback. As we believe to have addressed your concerns, please let us know if there are any remaining concerns that would hinder a recommendation for acceptance.

Thank you again for your valuable comments and for taking the time to read our general reply, as well as considering our point-by-point comments here:

---

W1. Although the paper aims to create more realistic shifts [...] only the shift in one modality is presented. Therefore, the SURE-VQA dataset [...] does not provide novel insights into the robustness of VLMs against multimodal distribution shifts [...]

• We find this suggestion as an insightful addition to our paper. Therefore we have conducted an ablation study on OVQA dataset focusing on multimodal distribution shifts. This study is added to our Appendix E. Looking at the results, we can confirm the assumption that multimodal shift lowers robustness compared to unimodal shift since they are stronger.

---

W2. The experiments in the paper only focus on one VLM [...]

• We acknowledge this limitation and have addressed it in our manuscript. We focused on LLaVA-Med v1.5 since it is SOTA with highest community recognition (443 citations vs. 162 for Med-Flamingo). Expanding our study to multiple VLMs was infeasible due to the computational demands of extensive experiments (e.g. 189 experiments for hyperparameter tuning). Please refer to the general reply for more detailed discussion.

---

W3. It would be beneficial to introduce some mechanism to control the shifts [...] For example, control the number of shifted samples from each type in the test set.

• Thank you for your comment, in our study we create two test sets where one only contains i.i.d. data and the other one contains only OoD data. The reason why we used such a setting is because this approach aligns with existing literature, which often examines robustness in separate i.i.d. and OOD splits [1,2,3].
• We would like to clarify that our study focuses on using realistic shifts rather than synthetic ones. While synthetic shifts allow for explicit control of severity, realistic shifts are more challenging to control for severity but better reflect real-world scenarios.

[1] Shirnin et. al. "Analyzing the Robustness of Vision & Language Models"
[2] Qiu et. al. "Benchmarking Robustness under Distribution Shift of Multimodal Image-Text Models"
[3] Chen et. al. "Benchmarking robustness of adaptation methods on pre-trained vision-language models".

---

W4. The use of large language models as evaluators [...] may not be the best choice [...].

• We have taken your comments into consideration and made several updates to our study to address your concerns.
• To improve the robustness of our human evaluation, we increased the number of raters from two to five, resulting in a more reliable assessment of human consensus.
• We observe that across all datasets, the human-human correlation is the highest, indicating strong alignment among the human raters compared to other metrics.
• For SLAKE and OVQA, the correlation between human evaluations and Mistral is higher than that with traditional metrics, indicating Mistral's effectiveness as an evaluation tool. In contrast, for the MIMIC dataset, traditional metrics show higher correlation than Mistral due to MIMIC's structured format, where token matching works well. For detailed discussion please see concern 3 in our general reply.

---

Q1. With multiple LLMs available, is there a specific reason for choosing the Mistral model as the evaluator?

• Thank you for your question. We selected Mistral as our evaluator since at the time of our development, Mistral was among the state-of-the-art (SOTA) LLMs available in the open-source community, demonstrating significant improvements over models like Llama1 and Llama2 as stated in their paper [1] .

[1] Jiang et. al. Mistral 7B

---

Q2. Why does the paper focus on analyzing different PEFT methods using the constructed dataset instead of different VLMs?

• Thank you for your question. It is important to note that our framework is flexible and not limited to PEFT methods alone. It can indeed be used to compare different VLMs in terms of their robustness.
• Our focus on analysing different PEFT methods using the constructed dataset is motivated by the growing importance of efficient fine-tuning techniques for large models, especially in domains like the medical field with limited resources [1]. That’s why it’s both interesting and important to look into how the choice of PEFT strategy might affect a model’s robustness.
• With this motivation, we see testing additional VLMs rather as an extension than a replacement, to our FT study. However, this extension is infeasible due to computational constraints. For details, please refer to our general reply.

[1] Dutt et al. “Parameter-Efficient Fine-Tuning for Medical Image Analysis: The Missed Opportunity”

---

Thank you once more for your constructive feedback. As we believe to have addressed your concerns, please let us know if there are any remaining concerns that would hinder a recommendation for acceptance.

Thank you again for your valuable comments and for taking the time to read our general reply, as well as considering our point-by-point comments here:

---

W1. Only LLaVA-Med is evaluated in the experiments [...].

• We generally agree with your point and acknowledge that evaluating multiple VLMs could provide additional insights
• However, as detailed in our general reply, the scope of our work focuses on proposing a general framework for evaluating VLM robustness, with the analysis of fine-tuning methods serving as an exemplary use case to demonstrate its utility
• Extending the study to multiple VLMs would have exceeded the scope and computational budget but represents an important direction for future research
• Please refer to our general reply for a more comprehensive discussion

---

W2. The experimental section focuses heavily on PEFT methods [...]. This narrow focus could limit insights [...] that might arise from other fine-tuning or training approaches [...]

• Thank you for your valuable suggestion regarding incorporating additional training schemes into our analysis. Based on your feedback, we have now included full fine-tuning as an additional training approach in our experiments
• With these new results, we further underline the effectiveness of PEFT methods in the medical domain, which aligns with findings in [1]
• We hope this extension to our analysis addresses your concern and believe it further strengthens the relevance of our findings. Please refer to our general reply and our updated manuscript for further details on the results of the newly added fine-tuning method.

[1] Dutt et al. “Parameter-Efficient Fine-Tuning for Medical Image Analysis: The Missed Opportunity”

---

W3. The primary contribution is the SURE-VQA framework itself. Although it establishes a valuable structure for evaluation, the paper lacks contributions in model development or methodological advancements [...]

• We think there is a misunderstanding about the scope of ICLR. We specifically submitted to the primary area “datasets and benchmarks”, a dedicated area where the novelty does not lie in new methods.
• The legitimacy is also evident in numerous ICLR papers sharing our format and type of contribution: E.g. Kahl et al. ICLR24 (oral), Jaeger et al. ICLR23 (Top 5%), Galil et al. ICLR23a (Top 25%), Galil et al. ICLR 2023b, Wiles et al. ICLR 2022 (oral), Zong et al. ICLR23 (Top 25%).
• Motivation: The main observation we make is that currently there is a lack of literature highlighting key factors for evaluating robustness in the medical VQA task
• Novelty: We are the first ones to
  • Evaluate the robustness of (medical) VLMs on realistic data shifts inherent to the data
  • Define key factors that should be adhered for a meaningful investigating robustness in VLMs
  • Thereby, the focus of the aspect that influences the robustness can be varied by the concrete studies (in out case the focus was on fine-tuning methods)
• Concrete Impact:
  • Researchers can use our framework for
    • benchmarking other components that might influence the robustness, e.g. the model architecture
    • developing new methods that enhance the robustness
  • Practitioners can use the findings of our benchmark as a guideline in method selection
    • One concrete finding of our study is that no PEFT method consistently outperforms others in terms of robustness, and all are similarly robust, making the choice of PEFT method independent of robustness needs. We specifically highlight this in our updated manuscript in lines 520-522.

---

W4. The paper uses Mistral [...] to evaluate semantic similarity, but it may not thoroughly address possible biases or limitations inherent in these evaluators [...].

• We believe that a reliable metric should bring the predictor as close as possible to the gold standard, which, in the context of free-text evaluation are human raters [1,2,3]. Rather than analyzing potential biases of individual metrics, we demonstrate that the proposed metric aligns more closely with this gold standard, making it a preferable choice. We therefore leverage Mistral as an LLM evaluator, aiming to capture the semantic understanding that traditional metrics lack
• Further, as outlined in our general reply, we made our human rater study now even more comprehensive by adding more human raters for a more robust evaluation of Mistral as an LLM-based metric
• For a detailed discussion, please refer to the general reply

[1] Celikyilmaz et al. “Evaluation of Text Generation: A Survey”
[2] Yang et al. “Advancing Multimodal Medical Capabilities of Gemini”
[3] Fu et al. “GPTScore: Evaluate as You Desire”

---

Thank you once more for your constructive feedback. As we believe to have addressed your concerns, please let us know if there are any remaining concerns that would hinder a recommendation for acceptance.