QoQ-Med: Building Multimodal Clinical Foundation Models with Domain-Aware GRPO Training

We thank Reviewer mAXD for finding our method interesting, the mathematical formulation rigorous and the multimodal architecture novel.

W1/Q1: We further ran the main experiments 4 times under each setting for statistical analysis, and included the updated results below:

The standard deviation for all methods is around ~1% across runs. In terms of overall performance, our method (DRPO) significantly outperformed all baseline methods (SFT, ReMax, RE++, RLOO and GRPO) with p<0.0001.

W2: We evaluated the fairness of the following eight datasets from CLIMB, where the demographic information is available: CheXpert, Vindr-Mammography, HAM10000, Hemorrhage, COVID-BLUES, ISIC2020, PAD-UFES-20. In particular, these datasets contain the age, gender and parent birthplace of the participants. We bin the age into 20-year brackets, treat each demographic information as a group, and evaluate the standard deviation of the accuracy and F1 scores across different groups. The results are as follows (lower is better):

In general, we do not observe a significant difference in fairness between GRPO and DRPO (our method). Our method gives a more homogeneous F1 score across groups, while the standard deviation in accuracy across groups for GRPO is lower. We will conduct a more thorough analysis of the fairness of these models and training methods as a part of our ongoing effort.

Q2: The number of clusters in the model is determined automatically via the elbow method, as described in Sec. 3.3 and App. B. 2, with the possibility to set an upper limit on the number of clusters. In the original experiment, the cluster limit for each domain is set to 10. We tested the model with 1 (no clustering), 3, 10 and 20 clusters. The results are as follows:

In general, we observe that having no cluster or a very low cluster limit will cause a decrease in performance. A higher cluster limit, however, does not seem to hurt the performance, as the elbow method automatically chooses a lower cluster count than the limit. This allows the algorithm to remain efficient under arbitrary cluster limits.

Q3: The core contribution of the hierarchical scaling scheme is that it performs better for rare conditions and rare datasets. As described in Figure 2, compared to GRPO, DRPO performs particularly better in clinical modalities where data is rare. In addition, the gap between training and validation accuracy is similar across all methods and modalities, which indicates that DRPO does not cause the model to overfit to smaller datasets.

W3: We agree that synthetic segmentation labels may introduce errors for some rare cases. BiomedParse reported an average dice score of 0.91-0.98 across clinical domains. We randomly selected 100 images evenly across the eight imaging domains where ground truth segmentations are not available. We observed that the segmentations for most classes are highly consistent with the true abnormalities, like Lung Opacity, Acute pulmonary embolism and Pituitary Tumor, with an estimated dice score of >0.8. However, the segmentations for some rarely seen or novel classes, like COVID-19 and malignant lesions in breast ultrasound, are less accurate, with an estimated dice score in the 0.3-0.4 range. Due to the restriction of visual content in the rebuttal, we will provide a detailed visual and quantitative analysis of the generated segmentations in the final manuscript.

In the final manuscript, we will discuss the environmental impact in App. D.1. Our A100 SXM has a TDP of 400W. Assuming CPUs and other components have a combined TDP of 600W, training of a 7B model requires an electricity of 182.4kWh. Under the same assumption, training a 32B model requires an electricity of 571.2 kWh. We would also like to point out that releasing public models as we did has a positive environmental impact, as researchers can directly use or finetune the model on the domain data, instead of doing the pretraining repeatedly.

Thanks a lot for reviewing our paper and giving constructive feedbacks! We hope our rebuttal has addressed your concerns. As the discussion period is about to end, please let us know if you have any other questions that remains to be addressed.

We thank reviewer L5Uq for finding our DRPO method interesting, our task important, and our experiments comprehensive. We are excited to see what researchers will build upon our released multimodal disease diagnosis models. We agree that a more sophisticated fusion mechanism may be helpful in aligning multiple modalities. There have been some recent efforts to tackle this challenge. FUSION, for example, utilizes similarity loss for the alignment between vision and text embeddings. And EMMA proposed a lightweight module for cross-modality alignment. Most of these efforts are centered around texts and images. In our cases, fusion of time series, vision and texts is a non-trivial challenge. While it is out of the scope of this work, we will explore this in our future work, and we welcome community efforts to build better multimodal representations.

We thank Reviewer pph3 for finding our multimodality task important, evaluation comprehensive and writing well-structured.

Weakness 1: We further ran the main experiments 4 times under each setting for statistical analysis, and included the updated results below:

The standard deviation for all methods is around ~1% across runs. In terms of overall performance, our method (DRPO) significantly outperformed all baseline methods (SFT, ReMax, RE++, RLOO and GRPO) with p<0.0001.

Weakness 2: In general, we found that the weight of each reward does not have a significant impact on the final performance. In particular, the auxiliary rewards on formatting saturate shortly in the early stages of training. They have effectively no impact on the later stages due to normalization. We tested different combinations of accuracy rewards: semantic alignment rewards. The results are as follows:

Decreasing the weight of the accuracy reward gives a drop in overall performance and performance in most domains, but results are still significantly better than all baselines, which demonstrates the robustness of DRPO.

Q1: We included a sample reasoning trace in the anonymous repo under datasets/. Due to the size limit of the anonymous repo and the restriction on external links, we are not able to share the entire reasoning traces now, but we will include them in the final manuscript.

Q2/3: We included examples of reasoning traces that lead to incorrect answers in App. E and Sec. 4.3. Generally, there are 2 areas where the current model seems to struggle. First, the model seems to struggle to spot minor objects like support devices. In addition, the model was not particularly good at diagnosing novel diseases like COVID from ultrasound, where the most common misdiagnosis is pneumonia. From our randomly selected samples, there is no instance where the reasoning evidence is not aligned with the image. We will conduct a more thorough error analysis in the appendix of the final manuscript.

We thank Reviewer MhFk for the constructive feedback and suggestions.

W1/Q1: The reasoning training from QoQ style training is highly generalizable, as we do not presume any reasoning structure during the training process. The supervision is only on the final diagnosis. This follows the finding from the Deepseek R1 paper, where the authors demonstrated how simple, flexible supervision elicits models’ pretraining knowledge, helps prevent reward hacking and enables more holistic learning. In Sec. 4.3 and App. E, we detailed how the model reasons across different modalities. In particular, we found that the model is able to focus more on the color for the reasoning of dermoscopy, with examples like “slightly raised, pinkish, and slightly elevated area.” On the other hand, the model focuses more on the brightness, patterns and textures for ultrasound videos, with examples like “The hyperechoic nature of the lesion is often associated with consolidation.”

W2: As described in Sec. 4.3, we collaborated with expert clinicians to annotate the relevance of the reasoning traces to the diagnosis. The expert annotation in App. E indicates that around 93.8% of the reasoning traces are relevant to the diagnosis of the disease. In particular, around 45.3% of the reasoning is rated as highly relevant, and 100% of the rated samples contain highly relevant reasoning traces.

W3/Q2: We would like to clarify that our model QoQ-Med does support pathology slides, as well as 3D scans (CT, MRI) and videos (ultrasound). The performance is demonstrated in Table 2 (attached above for reference). For pathology slides, DRPO achieves an F1 score of 0.449, representing an 11.1% improvement over the next best baseline GRPO (0.404). In 3D imaging modalities, our method shows strong results on MRI with DRPO achieving 0.652 F1 score, a substantial 20.7% improvement over the next best baseline PPO's 0.540. For CT scans, DRPO without KL divergence demonstrates the best performance with a 0.365 F1 score, improving by 42.0% over SFT and PPO's 0.257.