MMedPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization

Thank you for your valuable feedback to help us improve our paper. All tables and images referenced in this rebuttal can be found in https://anonymous.4open.science/r/ICML_rebuttal-0304/README.md

Q1: More diverse clinical datasets could strengthen the evaluation.

A1: We conducted experiments on a diverse dataset PMC-VQA [R1], which includes various medical imaging modalities. As shown in Table R9, our method demonstrates significant improvements over several baseline methods, further validating the generalizability of our approach.

[R1] Zhang X et al. PMC-VQA. Nature Communications Medicine 2025.

---

Q2: ... doesn't present significant theoretical proofs…

A2: Due to the rebuttal length constraints, we were unable to include the theoretical proofs in the rebuttal. We plan to add theoretical justifications in the revised PDF to demonstrate that our method can theoretically enhance model performance based on DPO. We aim to formally analyze the interplay between noise, relevance, and preference optimization — potentially building connections to existing theories.

---

Q3: Lacks comprehensive implementation specifications, including baseline method details, hyperparameter selection, and ablation studies.

A3: We will include detailed implementation specs in the revised manuscript. Baselines like DPO, SFT, STLLaVA-Med, FiSAO, and others follow official or original pipelines. Our method uses LoRA (rank 128, alpha 256), a 2e-5 learning rate for the multimodal projector, cosine schedule with 3% warmup, and DeepSpeed ZeRO-3. Hyperparameters align with prior work (e.g., LLaVA-Med) for fair comparison. Our key distinction is incorporating clinical relevance into preference tuning, guiding the model toward medically meaningful outputs. Detailed ablations will also be added.

---

Q4: The authors should discuss recent papers.

A4: We agree and will revise the related work section to include recent Med-LVLM alignment efforts, domain-specific preference learning methods, and clinical validation metrics. This will better contextualize our contributions and highlight key distinctions.

---

Q5: The novelty … feature-level [1], image-level [2], and text-level noise injection [3]...

[1] Strengthening… Optimization [2] Self-Supervised … Alignment [3] Aligning … Fine-Tuning [4] SILKIE

A5: MMedPO introduces clinically meaningful corruptions targeting disease-relevant regions, enabling fine-grained, domain-aware supervision. Unlike prior methods using global noise (e.g., BPO, POVID), MMedPO integrates clinical context directly into both preference construction and optimization, achieving more precise and effective alignment for medical tasks.

---

Q6: give a short explanation of what the 'tools' represent in Figure 2' captions

A6: Tools refers to external medical visual grounding models used to extract region-level information from the image. Specifically, we utilize MedKLIP as the visual tool $T(x_v)$, which predicts disease-related local regions $h = T(x_v)$ for each medical image $x_v$. This enables the model to better attend to clinically relevant visual features during alignment and scoring. We will revise the figure caption to clarify this point.

---

Q7: How does MMedPO specifically address medical domain challenges that general MLLM alignment methods cannot?

A7: Unlike general MLLM alignment methods, MMedPO is tailored for medical tasks through two key components: (1) visual grounding via tools like MedKLIP to focus on disease-relevant regions, and (2) a clinical relevance scorer to prioritize responses that are not just fluent but medically meaningful. This ensures alignment better suited to the precision demands of the medical domain.

---

Q8: What safeguards ensure the clinical relevance scoring accurately reflects medical expertise?

A8: We validated the scoring by comparing Med-LLM outputs with ratings from three clinical experts on 100 samples. The strong correlation (Table R6) confirms alignment with expert judgment. Notably, the multi-LLM setup outperformed single-LLM scoring, demonstrating that our approach approximates expert-level evaluation with credible accuracy.

---

Q9: How was the performance validated with actual healthcare professionals?

A9: We conducted an expert evaluation on 50 samples, where three medical professionals rated reports from our method and baselines using a 5-point scale (Excellent, Good, Fair, Poor, Very Poor). As shown in Table R8, our method consistently received higher scores, confirming its clinical effectiveness.

---

Q10: What specific metrics were used to evaluate clinical relevance beyond standard VQA metrics?

A10: We use a Med-LLM-based clinical relevance score that assesses the appropriateness and usefulness of answers in context. Unlike standard VQA metrics (e.g., accuracy), this score reflects real-world clinical value and aligns with expert judgment.

Thank you for your constructive comments and suggestions. All tables and images referenced in this rebuttal can be found in https://anonymous.4open.science/r/ICML_rebuttal-0304/README.md

---

Q1: The performance improvement with SFT is much smaller.

A1: As clarified in Section 4.2, we report MMedPO’s performance in both with- and without-SFT settings. Even when combined with SFT, MMedPO yields consistent gains across all four datasets, with an average improvement of 10.5%. While gains may be smaller in absolute terms, these results demonstrate the robustness and modularity of our method, highlighting its compatibility with other training strategies.

---

Q2: The quality of the generated dispreferred data pair [a].

[a] Yan, Qianqi, et al. Worse than random? arXiv 2024.

A2: Dispreferred responses are generated via GPT-4o with guided hallucinations to ensure clinical plausibility, while preferred ones are real reports. Expert review gave dispreferred samples an average score of 6.4/10, confirming suitability for preference optimization. We’ll include example pairs in the final version for transparency.

---

Q3: The relevance score of these generated reports is also evaluated by text-only LLM..

A3: In our setting, the evaluated reports are intentionally perturbed via GPT-4o to introduce plausible errors. Therefore, the role of the text-only LLM is not to verify image-text alignment, but to assess the clinical severity and plausibility of these injected faults. In other words, the LLM is used to judge whether a faulty report still retains clinical value (e.g., contains common or benign errors) or if it includes clear factual mistakes that are unlikely to be made by competent practitioners. This scoring helps us assign lower training weights to less harmful dispreferred responses, and higher weights to truly misleading ones — ultimately improving the effectiveness and safety of preference optimization.

---

Q4: The evidence provided in Figure 5 …the image.

A4: In Figure 5, our method enhances the model's attention to the image, suggesting improved focus on relevant regions. To make this more intuitive and convincing, we have added visualizations of the attention weights as overlays on the image (Image R1-R5). These visualizations clearly illustrate where the model is attending within the image. We will include several examples of these attention visualizations in the revised version of the PDF.

---

Q5: One additional scaling experiment on VLMs of different sizes ....

A5: As shown in Table R7, our method has been successfully extended to Med-VLMs based on both VILA-M3-8B and VILA-M3-13B [R1]. In both cases, we observe consistent and significant performance improvements, demonstrating the generalizability and scalability of our approach across models of different sizes.

[R1] Nath V, et al. VILA-M3. CVPR 2025.

---

Q6: Some of the designs for the method… expert evaluations..

A6: We conducted expert evaluation on 50 samples, where three clinicians rated reports using a 5-point scale (Excellent, Good, Fair, Poor, Very Poor). As shown in Table R8, our method consistently received higher scores than baselines, supporting the effectiveness of our design choices.

---

Q7: The evaluation in Table 4 compared global noise...random noise masking.

A7: We incorporated random noise masking and observed performance gains. As shown in Table R5, while different noise types yield comparable results, the improvement largely stems from lesion-aware preference optimization, underscoring the value of localized, clinically meaningful perturbations. Thanks!

---

Q8: Providing more examples of the generated depreferred data.

A8: We will provide more examples of dispreferred data and more cases in the future version. We will show more examples like Figure 6 in the future version.

---

Q9: There is no discussion on the reliability of these composing VLM/LLMs used in the method.

A9: We agree reliability is crucial. While we use LLaVA-Med v1.5 for its strong benchmark performance, we recognize its limitations in complex cases. To enhance robustness, we adopt a multi-agent setup with cross-checking. We’ll revise the manuscript to discuss these concerns and cite [a–c], while highlighting future directions like human-in-the-loop and confidence calibration.

---

Q10: How does the generated dispreferred data look like? Is it possible to evaluate the quality of these data quantitatively?

A10: Dispreferred responses are generated by injecting plausible hallucinations into preferred ones using GPT-4o. We will include more examples in the final version. For evaluation, we use a clinical relevance score to assess each dispreferred response, which also serves as a weight in preference learning. We agree that human expert validation is important and plan to include it in future work to further assess clinical soundness.

Thank you for reviewing our paper and for your valuable feedback. Below, we address your concerns point by point. We would appreciate it if you could let us know whether your concerns are addressed by our response. All tables referenced in this rebuttal can be found in https://anonymous.4open.science/r/ICML_rebuttal-0304/README.md

---

Q1: Given the sensitive nature of the task and the data involved, is there any human involved when quantifying the relevances during the Multi-Agent step? Even though Medical AI agents are highly capable these days, one can't be 100% sure if only relying on LLMs can guarantee the most expertise evaluations. If there is no human-in-the-loop, what is the reason behind such a decision in design?

A1: To ensure reliability, we additionally involved human experts in the evaluation process. Specifically, we selected 100 samples and invited three experienced clinical experts to assign clinical relevance scores to the preference data. We then compared these human scores with those generated by Med-LLMs using multiple correlation metrics. As shown in Table R6, the Med-LLM scores exhibit strong correlation with human judgments, validating their effectiveness. Moreover, the multi-LLM setting outperforms the single-LLM approach, showing better alignment with expert assessments. This human-in-the-loop validation supports the credibility of our automated scoring mechanism while demonstrating that well-designed multi-agent LLM setups can approximate expert-level evaluations with reasonable accuracy.

---

Q2: When assigning CRS to the curated preference data, is the preferred response also given weights by the agents, presumably always larger than its dispreferred counterpart? Please clarify if CRS's scale is 'relevant' (where the preferred is always assumed to be 1.0), or 'absolute'; if CRS is a 'relevant' scale, it might not best reflect how a response is objectively good in terms of clinical response quality.

A2: Thank you for the question. To clarify, we assign the Clinical Relevance Score (CRS) only to the dispreferred response in each DPO preference pair. The preferred response is not explicitly scored, as its superiority is already established through the preference annotation. Therefore, the CRS is defined in an absolute manner rather than a relative scale. That is, it directly reflects the clinical quality of the dispreferred response, independent of its counterpart. This allows us to distinguish between cases where the dispreferred response is still reasonably acceptable (e.g., CRS ≈ 0.8) versus cases where it is clearly irrelevant or incorrect (e.g., CRS ≈ 0.2).

---

Q3: L171, the reference to Chan et al is apparently missing the year.

A3: Thank you for pointing this out. We will correct the citation and include the missing year for the reference to Chan et al.

Thank you for your valuable feedback to help us improve our paper. We detail our response below and please kindly let us know if our response addresses your concerns. All tables referenced in this rebuttal can be found in https://anonymous.4open.science/r/ICML_rebuttal-0304/README.md

---

Q1: The definition of hallucination in Med-LVLMs is unclear in this paper, and the reasons behind hallucination causes remain ambiguous.

A1: We define hallucination as a response that contradicts the image content given the question. As noted in the introduction, a key issue is modality misalignment [R1-R2], where the model overly relies on text rather than grounding responses in visual content.

[R1] Zhou Y, et al. Analyzing ...Models. ICLR 2024.

[R2] Chen J, et al. Detecting ...models. arXiv 2024.

---

Q2: Some empirical results appear inconsistent… a) The reported performance of the base model...is much worse than in its original paper [1]... b) Table 7 shows extremely low BLEU-2/3/4 scores on MIMIC-CXR ... fine-tuning?

[1] LLaVA-Med: Training … in One Day. NeurIPS 2023.

A2: a) The performance drop of the base model (LLaVA-Med v1.5) on SLAKE and VQA-RAD compared to the original paper [1] is due to differences in experimental setup. While prior works used fully fine-tuned LLaVA-Med v1.0 checkpoints, such checkpoints are not available for v1.5. Therefore, in our experiments, we used the LLaVA-Med v1.5 pre-trained checkpoint and performed LoRA fine-tuning, which naturally leads to some performance gap compared to full fine-tuning. b) The low BLEU-2/3/4 scores in Table 7 were due to the specific BLEU settings used in our initial evaluation. We appreciate your attention to this and have since updated our BLEU configuration. The revised results (Table R1) are more consistent with expectations. Importantly, this correction does not affect the overall trends or conclusions.

---

Q3: Multi-agent collaboration for clinical relevance scoring...questioning the necessity of this step.

A3: We discuss the benefits of using multiple Med-LLMs for clinical relevance scoring in Section 4.3.2. As shown in Table 3, multi-agent discussion yields a 3.6% improvement in clinical relevance scores over single-LLM scoring, suggesting that incorporating diverse perspectives can enhance the robustness, even if it is heuristic in nature. It is worth noting Table 8 reports results for the report generation task, not clinical relevance scoring. While generation performance appears similar across single- and multi-LLM setups, Table 3 shows that the multi-agent approach yields meaningful gains in clinical relevance evaluation.

---

Q4: Why is an additional clinical relevance weight necessary?...weighting may be redundant...better to conduct a baseline without this step.

A4: While the Preference Data Curation step ensures dispreferred responses include hallucinations or region-level neglect, not all errors are equally harmful—some are clearly incorrect and easily avoidable, while others are subtle and clinically significant. We introduce clinical relevance weighting to capture this distinction and better guide optimization. To validate its effectiveness, we added experiments in Table R2 and Table R3 comparing models trained with and without relevance-based weighting. The results show consistent performance drops without weighting, confirming its benefit.

---

Q5: The math formula of weighted DPO needs consistency checking...Algorithm 1 is not described in the main text.

A5: Thanks for pointing this out. We will make sure to correct the inconsistencies in the mathematical notations across the text, equations, and Algorithm 1 in the final PDF version. Additionally, we will add a proper reference and description of Algorithm 1 in the main text to ensure clarity and completeness.

---

Q6: Lack of clinically meaningful evaluation metrics...using NLG metrics (BLEU, ROUGE-L, METEOR), but these do not reflect medical accuracy. More relevant clinical efficacy (CE) metrics...should be reported, following prior work like METransformer [2]. This is especially crucial since ...without CE metrics, it is unclear whether improvements are meaningful.

[1] LLaVA-Med, NeurIPS 2023. [2] METransformer, CVPR 2023.

A6: We have included clinical efficacy metrics—precision, recall, and F1-score—in Table R4, comparing our method against baselines. Our approach consistently improves both NLG and clinical efficacy metrics, indicating better diagnostic relevance and clinical accuracy of the generated reports.

---

Q7: Would other types of perturbations...improve lesion-based preference optimization?

A7: We explored various noise types (Table R5) and found that while Diffusion and Gaussian noise performed similarly, random noise offered a slight improvement. Many thanks for your valuable advice.