Sincerely thanks for your meaningful and detailed review. Your positive feedback is a huge encouragement to our team, including 1) theoretical framework, 2) CPO method, 3) experiments across multiple clinical benchmarks, 4) contributing the CCF dataset and 5) novel techniques. In response to your concerns, we have provided a detailed explanation below and revised our manuscript accordingly.

1. (Weakness 1) Motivation of concept drift. Thanks for your kind and helpful suggestions. Our motivation is based on our Observation 1.1 in our manuscripts, where we impose a small perturbation on the CoT reasoning process that does not affect the semantics, but the distribution of the results will be greatly deviated, which is the concept drift, as shown in our Fig.1 of our manuscripts.

   In terms of real-world medical environments, diagnosis requires strong robustness to resist noise, bias, and other issues from the data. However, the current MLLM will bring biased results by adding only small perturbations that do not affect the semantics to CoT. It poses a huge challenge to the application of MLLM in the medical field. We have added more details to clarify our motivation for concept drift in real-world medical environments. Thanks for your helpful suggestions.

2. (Weakness 2 & 3) Contributions. Many thanks for your comments. We are sincerely sorry for the misunderstanding. As for contributions:

   • First, we formalized the issues we found in Observation 1.1 using the concept drift framework, regarding the autoregressive CoT generation as a stream of next-token prediction actions, providing a solid theoretical foundation to study the non-stationary reinforced custom-tuning.

   • Second, within the concept drift framework, counterfactual causal inference is used to further tighten the model's decision boundary under the non-stationary reinforced custom-tuning, leading to our proposed Counterfactual Preference Optimization (CPO). By embedding learnable concept graphs as the expert,our approach automates the generation of adversarially-constrained reasoning trajectories, substantial decoupling between beneficial distribution adaptation and detrimental concept drift.

   • Third, we conduct comprehensive empirical validation to verify our proposed method across various clinical benchmarks, including disease classification, report generation and zero-shot generalization. The superior results demonstrate statistically significant improvements in robustness, generalization, and accuracy of our method under non-stationary custom-tuning.

   • Fourth, we contribute the CXR-CounterFact dataset, with over 320,000 counterfactual reasoning trajectories, facilitating future studies on counterfactual reasoning in medical applications.

   In terms of differences between our proposed CPO and DPO, although CPO and DPO both utilize a preference-based optimization framework, CPO is fundamentally distinct from DPO. DPO compares human-preferred responses against dispreferred ones. CPO instead contrasts factuals with counterfactuals generated under explicit causal interventions, specifically designed to isolate the causal effect. CPO has a tighter decision boundary than DPO, as shown in the results of our additional experiments in the answer of 4. Experiment of DPO.

   As for the motivation of choosing chest diagnostics as the application, it is due to the abundance of public professional medical diagnosis reports, which serve as rich CoT reasoning process. This domain provides an ideal platform for studying MLLM reasoning processes and validating the robustness of our proposed CPO in real-world settings. We will explore additional multimodal applications in the future. And we have added related discussions in the section of Outlooks.

   We have added more details and descriptions about our contributions in the section of Introduction. Many thanks for your helpful suggestions.

3. (Weakness 2) Relationships among concepts. In terms of relationships of different concepts** such as concept drift, counterfactual inference, reinforcement custom-tuning, they are strongly related to each other in our work. We initially discovered the problem of concept drift in reinforcement custom-tuning of chest diagnosis, so counterfactual causal inference is used to solve it. We have added more details about their relationships to clarify them in the section on Methodology. Thanks for your kind review.

4. (Question) Experiment of DPO. we apprecaite your kind and helpful suggestions. We have added additional experiments about DPO on multi-label chest diseases classification on MS-CXR-T as shown in Table 2. Our proposed CPO is significantly superior to DPO with the same number of training data, proving the performance of our propsoed counterfactual preference optimization. We have added discussions about DPO in the section of Methodology and Experiments. Thanks for your sincere review.

   	Con.	PE	Pna.	Pnx.	Ede.	Avg.
   SFT	54.9	71.7	70.0	95.9	76.5	73.8
   DPO	63.2	72.4	76.7	93.5	76.3	76.4
   CPO	77.7	72.7	87.4	95.8	75.3	81.8

Thank you once more for generously dedicating your time to provide a thorough review. Your feedback is tremendously valuable, and we are open to hearing from you at any time. If you find our response satisfactory, we would greatly appreciate your assistance in improving our rating score.

Dear Reviewer 4zYr:

Thank you for your time and valuable feedback on our submission. As the response period will conclude soon, we want to kindly check if you have any further questions or need clarification on our rebuttal. We are happy to provide additional details if needed.

Best wishes,

Authors.

Thank you very much for your kind and helpful review, especially for summarizing the strengths of our work, including 1) raising the problem of concept drift in CoT of MLLMs and solution, and 2) theoretical analysis and experimental support. In response to your concerns, we have provided a detailed explanation below and revised our manuscript accordingly.

1. (Weakness 1) Background and problem. Thank you very much for your sincere review. Our background and problem are based on our Observation 1.1 in our manuscripts, where we impose a small perturbation on the CoT reasoning process that does not affect the semantics, but the distribution of the results will be greatly deviated, as shown in our Fig.1 of our manuscripts. Specifically,

   • Concept drift denotes unpredictable distribution changes in data streams [1], where concept drift manifests itself as a large deviation in the distribution due to a small perturbation in CoT reasoning, which is viewed as a sequential token stream generation process.

   • Non-stationary environment refers to the CoT reasoning process, which is generated by the autoregressive decoding paradigm inherent to MLLMs, vulnerable to the interference from the data characteristics. And we posit that it is characterized as a sequential token stream generation process.

   We have added more details and explanations of our background and proposed problem in the section of Introduction. Thanks for your kind review.

2. (Weakness 2) Transferred to other fields. Thanks very much for your detailed and thoughtful review. Our framework can be transferred to other fields.

   • First, chest diagnostics was chosen as the application domain in our manuscript due to the wealth of publicly accessible professional diagnostic reports, which constitute valuable CoT reasoning data. This makes it particularly suitable for studying MLLM reasoning and assessing the real-world robustness of our CPO framework.

   • Second, we mainly focus on the methodology for solving the concept drift problem in the CoT reasoning process. Our implementation is also highly automated and can be easily transferred to other similar fields.

   We have added more discussion about our future work, exploring additional multimodal fields in the section on Outlooks. Many thanks for your kind comments.

3. (Weakness 3) Training costs. We appreciate the reviewer's valuable feedback regarding computational costs. As noted, the primary overhead arises from the additional LLM expert that generates counterfactual samples. We mitigate this through two key strategies:

   • Targeted Counterfactual Generation: Leveraging our hierarchical concept graph, we statistically identify features susceptible to concept drift. This allows targeted perturbation of high-drift-risk features instead of random perturbation. Consequently, we generate only two highly relevant counterfactual samples per original instance, significantly reducing computational demands.

   • Static Dataset Construction: The targeted approach enables pre-generation of all counterfactuals offline. Thus, the entire counterfactual dataset, our published CCF dataset, is built once via LLM inference prior to training, eliminating dynamic generation costs during model optimization. This process required approximately 5 days using 4 A100 GPUs. While still time-consuming and computationally intensive, this one-time cost is feasible and manageable for real-world deployment. We are sincerely sorry for your confusion of each step involving four inferences by MLLMs. We have added more details in the section of Methodology.

   Many thanks for your kind and thoughtful suggestions.

4. (Question 1) Positive and negative samples. Thanks for your kind question. We are sincerely sorry for your misunderstanding. In terms of positive samples, MIMIC-CXR not only provides the diagnosis results of the disease, but also provides a detailed diagnosis report issued by the doctor. In the diagnosis report, the doctor explains the reasons for the diagnosis results. Therefore, we regard the report issued by the professional doctors as the chain-of-thought preferred by humans, which are positive samples. As for negative samples, we generate counterfactual CoT for diagnostic report instances as our negative samples, that is, perturbing specific radiological features to interfere with the diagnosis results according to our proposed concept graph. We have added more details of our positive and negative samples in Line 202-203. Many thanks for your detailed review.

5. (Question 2) Typos. Thanks very much for your corrections. We have corrected these typos and double-checked the entire manuscript to make sure there are no further typos.

We appreciate your time and thorough review. Your feedback is highly valuable to us, and we welcome further communication from you. If you are satisfied with our response, we would be grateful for your support in enhancing our rating score.

[1] Lu, Jie, et al. "Learning under concept drift: A review." IEEE transactions on knowledge and data engineering 31.12 (2018): 2346-2363.

Sincerely thanks for your meaningful and detailed review. Your positive feedback is a huge encouragement to our team, including: 1) solid theoretical foundation and well-designed experiments and 2) practical significance and valuable dataset. And, we have provided detailed responses below to address your questions one by one, and made revisions to our manuscript according to your feedback:

1. (Weakness 1) Computational overhead. Many thanks for your helpful and thoughtful suggestions. We acknowledge the reviewer's valid concern regarding computational costs and appreciate the opportunity to clarify the practical feasibility of our approach. The additional computational overhead is mainly concentrated in the generation of counterfactual samples by LLM, as pointed out by the reviewers. We mainly alleviated it from the following two points:

   • Targeted Counterfactual Generation instead of Random Perturbation. The constructed hierarchical concept graph establishes disease entities, radiographic features, clinical relationships and taxonomies. By analyzing the statistical relationship between radiographic features and disease entities in the concept graph, we can identify the features that are prone to drift and make targeted perturbations instead of random perturbations. Therefore, for each sample, we only generated the two most relevant counterfactual samples, which greatly reduces the number of counterfactuals generated and reduces computing resources.

   • Static Generation. Because we perform targeted counterfactual generation, we do not need to generate them dynamically during training. Therefore, we only need to build the entire dataset once through the inference of LLMs, which is the CCF dataset we published. The entire dataset construction process took about 5 days on 4 A100 GPUs. Although this is still very time-consuming and computationally intensive, it is affordable and feasible in real-world deployment scenarios.

   We have added more detailed discussions about the computation and generation of counterfactuals in the section on Implementation Details. Thanks so much for your elaborate and thoughtful review.

2. (Weakness 2) Generalizability of concept graph construction. We sincerely thank the reviewer for raising the points about the generalizability of our concept graph construction.

   • First, we added more technical details of this extraction process in the Implementation Details. The prompt we used for automated extraction from the MIMIC-CXR dataset is as follows:

     TASK ROLE: You are a senior chest radiologist. I will provide a large number of chest DR case report texts. Please automatically extract key imaging feature words related to various chest diseases from these reports, and use these features to build a structured imaging knowledge graph to reveal the association between different diseases based on common imaging features.
     
     CORE REQUIREMENTS:
     1. Standardized feature extraction:
         - Extract all abnormal imaging descriptors from reports.
         - Normalize terminology using Radiology Lexicon (RadLex) and Fleischner Society guidelines.
     2. Disease-feature mapping:
         - Link standardized features to diagnoses diseases per report.
         - Identical features MUST use identical normalized terms across diseases.
     3. Knowledge graph construction:
         - Nodes: 
             Diseases, such as Pneumothorax, Atelectasis and Pneumonia;
             Features, such as lung opacity and air bronchograms.
         - Relationship:
             (Disease)-[HAS_FEATURE]->(Feature);
             (Feature)-[ASSOCIATED_WITH]->(Disease)
         - Semantics:
             Diseases sharing one more identical feature node are interconnected.
     
     OUTPUT REQUIREMENTS:
     Please output the final constructed radiological knowledge graph and the standardized feature-disease association data extracted from the report in a structured JSON format. The format is as follows:
     {
         "data":[
             {"diseases":[], "features":[]},
             // ... cases
         ],
         "disease_feature_map":{
             "Pneumonia": ["Consolidation", "Ground-glass opacity", "Air bronchogram", "Pleural effusion"],
             // ... relationships
         }
     }
     
     

     As can be seen,we have achieved highly automated concept graph extraction and construction through LLM. Then, one chest radiology expert assisted us in verifying the constructed concept graph.

   • Second, in terms of Broader Applications, we have achieved highly automated concept map extraction and construction through LLM, with minimal manual verification. We argue that our method can be transferred to other multimodal fields for broader application. We will continue to expand more multimodal applications in future works. Thanks for your sincere review.

   • Third, The motivation for choosing chest diagnostics as the application is due to the abundance of public professional medical diagnosis reports, which serve as rich CoT reasoning process. This domain provides an ideal platform for studying MLLM reasoning processes and validating the robustness of our proposed CPO in real-world settings. Future work will explore additional multimodal applications. And we have added related discussions in the section of Outlooks. Many thanks for your kind comments.

3. (Weakness 3) Typos. Thanks very much for your corrections. We have corrected these typos and double-checked the entire manuscript to make sure there are no further typos.

Thank you for your continued time and thorough review. Your feedback is immensely valuable to us, and we are always eager to hear from you. If you find our response satisfactory, could you please consider helping us improve our rating score?

We sincerely appreciate your insightful, thoughtful and detailed review. Your positive feedback serves as a great source of encouragement for our team, especially recognizing the strengths: 1) novel theoretical connection addressing a significant gap in the stability of reasoning in MLLMs, 2) valuable contribution of the CXR-CounterFact dataset, and 3) methodologically robust way of our proposed method. Furthermore, we have diligently responded to each of your questions and incorporated your feedback into our manuscript revisions:

1. (Weakness 1) The drift in visual modality. Thanks so much for your sincere and constructive suggestions. We acknowledge that this work does not address concept drift within the visual modality, as rightly pointed out by the reviewer. The motivation for focusing on the concept drift in language models is based on our Observation 1.1 in the manuscript. We find that when MLLMs are performing CoT reasoning within the language model, even if a slight perturbation does not change semantics, it will cause concept drift. Consequently, our current analysis and proposed methods focus primarily on this textual reasoning aspect. We sincerely thank the reviewer for this valuable insight. In further work, we will focus on the drift in visual modality.

2. (Weakness 2) The scope of the abstract. Thank you for your thoughtful and helpful feedback. We have revised the abstract to narrow the scope: "This paper uncovers a critical yet overlooked phenomenon in multi-modal large language models (MLLMs), especially for chest diagnosis: detrimental concept drift within chain-of-thought (CoT) reasoning during non-stationary reinforcement fine-tuning (RFT), where reasoning token distributions evolve unpredictably, thereby introducing significant biases in final predictions". The motivation for choosing chest diagnostics as an application is that, there are a large number of public medical diagnosis reports as professional CoT. It is particularly helpful in studying the reasoning process of MLLMs and verifying the robustness of our proposed CPO in real environments. We will continue to expand more multimodal applications in future works. Thanks for your constructive suggestions.

3. (Weakness 3 & 4) CPO algorithm. Many thanks for your kind and astute suggestions. While CPO and DPO share a preference-style optimization framework, CPO is fundamentally distinct. DPO contrasts human-preferred vs. dispreferred responses, which are random negative samples. CPO instead contrasts factuals with counterfactuals generated under explicit causal interventions, specifically designed to isolate the causal effect. CPO has a tighter decision boundary than DPO. Crucially, we added additional experiments that replace CPO's counterfactuals with DPO-style random negatives on multi-label chest diseases classification on MS-CXR-T as shown in Table 2. Our proposed CPO is significantly superior to DPO with the same number of training data, proving they are not functionally equivalent. We have added discussions about DPO in the section of Methodology and Experiments. Thanks for your sincere review.

   	Con.	PE	Pna.	Pnx.	Ede.	Avg.
   SFT	54.9	71.7	70.0	95.9	76.5	73.8
   DPO	63.2	72.4	76.7	93.5	76.3	76.4
   CPO	77.7	72.7	87.4	95.8	75.3	81.8

4. (Weakness 5) Pre-trained or fine-tuned Baseline. Many thanks for your comments. We are sincerely sorry for your misunderstanding. We follow the benchmark settings from the BioViL-T[7], and most of the compared methods have been pre-trained and fine-tuned. For example:

   Methods	Pre-trained Datasets	Fine-tuned Datasets
   BioViL	PubMed，MIMIC-III, MIMIC-CXR	RSNA，MIMIC-CXR
   BioViL-T	MIMIC-CXR v2	Chest ImaGenome, MS-CXR, RSNA, MS-CXR-T
   Med-ST	MIMIC-CXR	MS-CXR-T，RSNA，COVIDx
   TempA-VLP	MIMIC-CXR, Chest ImaGenome	MS-CXR-T，Chest Imagenome
   CoCa-CXR	MIMIC-CXR,Chest ImaGenome	MIMIC-CXR, Chest ImaGenome
   PromptMRG	MIMIC	MIMIC, IU X-ray
   MambaXray	CXPMRG	MIMIC-CXR，CheXpert Plus, IU X-ray
   Ours		MIMIC-CXR, MS-CXR-T

   Significantly, our method outperforms the baselines despite utilizing less data, indicating that the performance improvement is largely attributable to CPO, not simply backbone model optimization. We have added more discussion of the comparison method in the section of Implementation Details. Thanks again!

5. (Weakness 6) Training steps of SFT and CPO. Thank you for raising this point. We appreciate your attention to the training details. To clarify, both the SFT and CPO models were trained for exactly one epoch, as suggested by Qwen2.5-vl [9]. The difference in the number of training steps arises solely because the CPO dataset is approximately three times larger than the SFT dataset due to the addition of counterfactual trajectories. Therefore, both models completed their training after seeing their respective datasets once, indicating comparable convergence points in terms of epoch count, rather than SFT being stopped prematurely. We have also added a discussion about the training steps in Implementation Details. Thanks for your detailed review.

6. (Weakness 7) Inspiration of Long CoT. Thank you very much for your constructive and thoughtful inspiration. We agree that LLMs employing long CoT reasoning with self-reflection strategies represent significant advancements and can indeed help mitigate certain forms of reasoning errors. However, self-reflection strategies primarily act as a reactive correction mechanism during the model's own inference process. In contrast, our counterfactual samples are a proactive intervention during the training phase. This actively shapes the model's internal representations and decision boundaries to be more robust against causal fallacies from the outset, reducing the need for later correction. Thus, we view these approaches as highly complementary rather than mutually exclusive. Moreover, we have cited these papers [1-2], and added more discussion in the Outlooks.

7. (Weakness 8) Discussion about GRPO. Thank you for your insightful comment. We noticed GRPO's excellent efficiency and reasoning ability, especially on DeepSeekMath and DeepSeek-R1. We also tried to use GRPO to train Qwen2.5-vl and DeepSeek-VL2 on MIMIC-CXR, but both encountered reward collapse that led to failed training. Inspired by DAPO paper[8], we attribute this to GRPO's reliance on sparse final-answer rewards, where suboptimal reward assignment obscures high-quality samples. In contrast, CPO provides denser causal trajectories, significantly improving generalization and robustness by operating controlled counterfactual interventions. We have also added a discussion about the GRPO in the section on Experiments. Thanks for your detailed review.

8. (Weakness 9) Case studies. We appreciate the reviewer's insightful feedback. We have added the case studies based on Observation 1.1 in our manuscripts. The CPO-finetuned logit results for the case under Observation 1.1 are presented below.

   	Atelectasis	Cardiomegaly	Lung Opacity	Pneumonia	Pneumothorax
   "lung opacity"	0.19	0.22	0.83	0.75	0.08
   "opacity"	0.19	0.17	0.88	0.80	0.04

   Compared to the results shown in Fig. 1 of our manuscripts, CPO shows consistent performance even when slight perturbations are applied during CoT inference. In addition, our model assigns higher logits to the predictions of “lung opacity” and “pneumonia” inferred from the “opacity” attribute, and lower logits to all other diseases. This indicates that our model successfully performs causal inference.

   Besides, we also provided comparative experimental results of DPO in our answer to (Weakness 3 & 4) CPO algorithm. Our superior experimental statistical results illustrate that the performance is attributed to the learn conceptual causal relationships, rather than acquiring more knowledge. Thanks for your thoughtful review.

9. (Weakness 10) References. Many thanks for your detailed review. We have cited these papers [1-6], and added more discussion about the multimodal reasoning in the section on Introduction and Related Works.

Thanks again for your valuable time and thorough review. Your feedback is immensely valuable to us, and we welcome further communication from you. If you find our response satisfactory, could you please consider helping us improve our rating score?

[7] S. Bannur et al., “Learning To Exploit Temporal Structure for Biomedical Vision-Language Processing,” CVPR 2023.

[8] Yu, Qiying, et al. "Dapo: An open-source llm reinforcement learning system at scale." 2025.

[9] Bai, Jinze, et al. "Qwen technical report." arXiv preprint arXiv:2309.16609 (2023).

Dear Reviewer 3kr3:

Thanks again for your detailed and insightful ​​reviews​​! We would like to double-check and see if our responses have addressed your concerns, as the deadline of rebuttal is approaching. If you have any further questions, please feel free to let us know. We are more than happy to clarify more about our paper and discuss it further with you.

Best wishes, Authors.