Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Models

We sincerely thank the reviewer for the encouraging and constructive feedback. We appreciate your recognition of our strong empirical results, thorough evaluations and ablations, and the potential impact of our code and dataset release. Below, we address your specific concerns in detail.

W1: "The novelty of our work."

A: Thanks. We would like to clarify that our work is not a direct extension or simple combination of existing methods. Instead, it introduces a novel and unified post-training methodology specifically tailored to the core challenges of visual reasoning, such as cognitive rigidity and generalization to domain shift in dynamic visual reasoning.

In contrast to previous works that either (i) focus exclusively on text-based reasoning ([1]); (ii) adopt only-SFT ([2,3]) or only-RL ([4,5,6]) strategies for post-training in isolation; or (iii) involve a few concurrent works ([7,8]) that combine SFT and RL but remain restricted to static visual tasks with narrowly defined reward types—our approach is the first to introduce a generalizable training framework that systematically integrates SFT and RFT in a tightly coupled manner, effectively addressing both static (e.g., structural perception) and dynamic (e.g., temporal visual counting, spatial transformation) visual reasoning under a unified reward formulation.

The novelty of our approach lies in the following aspects:

• An adaptive training sample allocation strategy that dynamically balances intermediate reasoning supervision and reward-guided exploration. This approach strengthens the coupling between SFT and RFT, enabling a unified learning process aligned with visual reasoning goals—unlike prior methods that treat SFT and RL as separate stages.

• A unified visual reasoning reward formulation that generalizes across diverse task formats. Beyond replicating standard reward templates, we introduce a principled reward design that captures reasoning correctness and completeness in a task-agnostic manner:

  • Structured-format reward (including summary, caption, thinking and answer tags) to enhance the quality and coherence of reasoning chains.
  • Discrete-value reward for tasks with discrete-value outputs (e.g., classification and counting).
  • Tolerance-based reward for tasks requiring continuous-value outputs (e.g., geometric estimation, visual grounding).
  • Function-matching reward for tasks with sequence-like outputs (e.g., action planning or tool calling).

  These reward formulations are designed to be task-agnostic and applicable, moving BEYOND handcrafted, task-specific designs commonly used in prior work.

• A comprehensive visual reasoning evaluation protocol that spans in-domain and domain-shifted settings across both synthetic and real-world environments. It systematically benchmarks key reasoning skills—such as dynamic reasoning, geometric perception, and arithmetic—to assess generalization and robustness.

[1] DeepSeek-R1, ArXiv'25.

[2] LLaVA-CoT, ArXiv'24.

[3] LlamaV-o1, ACL'25.

[4] Visual-RFT, ICCV'25.

[5] VLM-R1, ArXiv'25.

[6] R1-Zero’s “Aha Moment”, ArXiv'25.

[7] R1-Onevision, ICCV'25.

[8] Vision-R1, ArXiv'25.

---

W2: "About the CoT length."

A: Thanks. First, we clarify that our tasks are far from trivial. Even top proprietary models like GPT-4o and Gemini 1.5 Pro scored below 50% across all tasks. For example, GPT-4o achieved only 35.9% on the Spatial Transformation task, revealing a key limitation in current VLMs. In contrast, our Reason-RFT 2B model reached 67.6%, demonstrating the effectiveness of our approach.

As for CoT response length, we note that visual reasoning fundamentally differs from traditional text reasoning (e.g., math problems). Our tasks rely on perceptual inference and visual comparison, not verbose textual elaboration. For instance, in Spatial Transformation, models must recognize visual state changes and infer spatial relations—this rarely requires long-form output. Thus, an average output length of ~70 tokens is both sufficient and appropriate, and longer responses do not necessarily imply stronger reasoning.

---

W3: "More experiments on more benchmarks."

A: Thanks. Our primary goal is NOT to achieve the best results on general VQA benchmarks, but to explore an important challenge: how to enhance domain-specific complex reasoning without severely degrading general capabilities.

Reason-RFT achieves more stable performance than other methods, with smaller drops across general benchmarks. For instance, on RealWorldQA, it reduces the accuracy drop to only –5.9 (vs. –19.1 for ANS-SFT and –30.7 for CoT-SFT). On MathVision, it even improves over the zero-shot baseline (+1.3). These results demonstrate Reason-RFT’s potential in balancing specialized reasoning and general capability.

Further details will be included in Appendix C of the revised version.

---

Q1: "More experiments on Qwen2.5-VL."

A: Thanks. To further validate the effectiveness of our method, we conducted experiments on Qwen2.5-VL-3B-Instruct. The results are summarized below:

• (1) Clear Improvement in Visual Reasoning:

  • Reason-RFT outperforms:
    • ANS by +22.5 (T1), +15.4 (T2), +3.8 (T3)
    • COT by +8.1 (T1), +5.1 (T2), +8.1 (T3)
    • RFT-ZERO by +17.9 (T1), +3.2 (T2), +10.1 (T3)

• (2) Strong Generalization under Domain Shift:
  Specifically, on the T1-DSM subset +44.0 over RFT-ZERO

• (3) Data Efficiency:
  Similar to Qwen2VL, the Reason-RFT training paradigm on Qwen2.5VL demonstrates high data efficiency:
  achieving ~70% of the final performance of Reason-RFT-ZERO using only <5% of the data.

We will add detailed results and analysis for Qwen2.5-VL in Appendix C as a new subsection: Additional Results on Alternative Backbones.

---

Q2: "Ablation experiments on RL without the format reward."

A: Thanks for the suggestion. To assess the impact of the format reward during RL, we conducted additional experiments on Qwen2.5VL-3B and Qwen2VL-2B across three tasks, comparing RFT training with and without the format reward. The results are summarized below:

We draw two main conclusions from these results:

1. Performance drop without format reward: Removing the format reward consistently reduced accuracy across all tasks and models, highlighting its importance in guiding RL training.

2. Model size affects convergence: For Qwen2.5VL-3B, convergence was stable without the format reward due to its strong instruction-following ability. However, Qwen2VL-2B showed a ≈200-step delay, indicating that smaller models rely more on explicit format supervision for structured prompts like <think>.

We will include these additional results and discussions in a new subsection of Appendix C: More Experiment Results on Format Reward.

---

Q3: "More analysis of CoT."

A: Thank you. In our pipeline, content within <think> represents the model’s CoT reasoning. To evaluate its improvement post-RL, we compare the Reason-RFT model (Stage 2) with the CoT-SFT baseline (Stage 1) on the Structure Perception task:

Qualitative Analysis:

1. The post-RL model shows stronger logical coherence, with fewer reasoning breaks. For instance, Fig. 16 (case 2) highlights a missing calculation step in pre-RL output that is corrected after RL.

2. We observe reflective reasoning patterns (e.g., "let me double check") emerging post-RL, absent in pre-RL outputs (e.g., Fig. 1).

Quantitative Analysis:

1. Step Count: GPT-4o analysis on 100 samples shows post-RL responses have 2.7 more reasoning steps on average.

2. Lexical Markers: We categorized two types of expressions:

   • Prompting words: "oh I see", "let me think step by step", "let me double check"
   • Logical connectives: "so", "therefore", "first", "but", "moreover" After RL, Use of prompting phrases and logical connectives increased by 14% and 23%, respectively.

3. Answer Accuracy: As shown in the Structure Perception subplot of Fig. 13, answer accuracy improves by +20.6% post-RL.

These results confirm that RL substantially enhances the quality and depth of CoT reasoning.

---

We hope our response addresses your concern and we look forward to further discussion with you.

Dear Reviewer yjNn,

Thanks again for your diligent effort in reviewing our submission. We have carefully addressed the concerns raised and conducted the requested experiments. As the discussion phase deadline is approaching, we sincerely hope you can consider positively recommending our work if your concerns are solved. If you still have further comments/suggestions, please don't hesitate to let us know.

Best regards.

Dear Reviewer yjNn,

We're grateful for your thoughtful recommendation to incorporate the new results and explanations into the next version of the paper. We will make sure to include them to improve the clarity and value of our work for readers.

We are also sincerely thankful for your willingness to adjust your score based on our response. Your constructive feedback has been invaluable in strengthening our submission.

Best regards.

We sincerely thank the reviewer for the thoughtful and encouraging feedback. We will address your suggestions in detail below and incorporate the relevant improvements into the revised manuscript.

---

W1 (Q1): "The novelty and contributons of our work."

A: We would like to clarify that our work is not a simple combination of existing methods. Instead, we introduces a novel and unified post-training methodology specifically tailored to the core challenges of visual reasoning, such as cognitive rigidity and generalization to domain shift in dynamic visual reasoning.

In contrast to previous works that either (i) focus exclusively on text-based reasoning ([1]); (ii) adopt only-SFT ([2,3]) or only-RL ([4,5,6]) strategies for post-training in isolation; or (iii) involve a few concurrent works ([7,8]) that combine SFT and RL but remain restricted to static visual tasks with narrowly defined reward types—our approach is the first to introduce a generalizable training framework that systematically integrates SFT and RFT in a tightly coupled manner, effectively addressing both static (e.g., structural perception) and dynamic (e.g., temporal counting, spatial transformation) visual reasoning under a unified reward formulation.

The novelty of our approach lies in the following aspects:

• An adaptive training sample allocation strategy that dynamically balances intermediate reasoning supervision with reward-guided exploration. This strategy enables more effective coupling between the SFT and RFT stages, supporting a unified learning process that aligns model behavior. Unlike prior methods that treat SFT and RL as disjoint stages, our design ensures their interaction is coherent and purpose-driven.

• A unified visual reasoning reward formulation that generalizes across diverse task formats. We introduce a principled reward design that captures reasoning correctness and completeness in a task-agnostic manner:

  • Structured-format reward (including summary, caption, thinking and answer tags) to enhance the quality and coherence of reasoning chains.
  • Discrete-value reward for tasks with discrete-value outputs (e.g., classification and counting).
  • Tolerance-based reward for tasks requiring continuous-value outputs (e.g., geometric estimation, visual grounding).
  • Function-matching reward for tasks with sequence-like outputs (e.g., action planning or tool calling).

  These reward formulations are designed to be task-agnostic and applicable, moving BEYOND handcrafted, task-specific designs commonly used in prior work.

• A comprehensive visual reasoning evaluation protocol, covering both in-domain and domain-shifted scenarios, and spanning synthetic and real-world visual environments. The evaluation rigorously benchmarks multiple dimensions of reasoning ability—including dynamic reasoning, geometric perception, and mathematical arithmetic—enabling us to assess generalization and robustness in a systematic manner.

[1] DeepSeek-R1, ArXiv'25.

[2] LLaVA-CoT, ArXiv'24.

[3] LlamaV-o1, ACL'25.

[4] Visual-RFT, ICCV'25.

[5] VLM-R1, ArXiv'25.

[6] R1-Zero’s “Aha Moment”, ArXiv'25.

[7] R1-Onevision, ICCV'25.

[8] Vision-R1, ArXiv'25.

---

W2: "Query about the separation between ID and DS data."

A: We clarify that the ID/DS split in our benchmark is not artificial, but based on a principled distinction: whether object categories, scene compositions, and camera viewpoints were seen during training.

• ID data: aligns with training distributions in terms of object identity, layout, and viewpoint.
• DS data: introduces systematic shifts—e.g., novel 3D assets, unseen viewpoints, or altered scene configurations.

This separation reflects practical generalization challenges in embodied visual reasoning, where agents often face unfamiliar and dynamic environments. For example, in our Visual Transformation benchmark, the DS set simulates real-world deployment where agents encounter new perspectives or object arrangements.

The necessity of this split is further empirically validated. For example, in the Visual Counting zero-shot setting, Qwen2-VL-2B's accuracy drops sharply from ID (82.40) to DS-D (42.67) to DS-M (0.00), highlighting the difficulty and importance of DS evaluation in diagnosing robustness and transferability.

---

W3-1 (Q1): "CoT data construction."

A: The CoT data is built through a combination of automated generation and manual verification to ensure both diversity and correctness:

• Diversity: We use reasoning-guided prompts (e.g., “Let’s think step by step”) with GPT-4o and Gemini-Pro, applying temperature sampling (temperature=0.7, top-k=50, top-p=0.9) and adding hand-crafted exemplars to enhance variety and depth.

• Correctness: Outputs are filtered through manual review and automated scripts. Low-quality samples (e.g., incomplete reasoning or incorrect answers) are removed. For each task subtype, 10% of the samples are randomly selected for human validation and refinement if needed.

---

W3-2: "Impact of Reason-RFT on downstream reasoning quality."

A: We compare the Qwen2.5VL-3B model trained with Reason-RFT (Stage 2, with RL) and the one trained only with CoT-SFT (Stage 1, without RL) on the Structure Perception task. Key findings are as follows:

Qualitative Analysis:

1. The RL-trained model shows stronger logical coherence and fewer broken reasoning chains.
   • E.g., in Fig. 16 (case 2), the pre-RL model omits a crucial “×2” step, which is rare post-RL.
2. Post-RL responses include reflective phrases (e.g., “let me double check”) not seen in pre-RL outputs (Fig. 1).

Quantitative Analysis:

1. Reasoning steps: On 100 samples, the RL-trained model generates +2.7 more steps on average (per GPT-4o analysis).
2. Lexical patterns: Post-RL responses include +14% more prompting cues and +23% more logical connectives.
3. Answer accuracy: As shown in Fig. 13 (Appendix), final accuracy improves by +20.6% after RL.

Overall, Reason-RFT significantly enhances reasoning fluency, structure, and downstream performance—not just correctness.

---

W3-3: "Is CoT-SFT necessary, given Reason-RFT-Zero's strong performance?"

A: While CoT-SFT brings the largest gains on structurally formatted tasks like Visual Transformation, its benefits are broader and foundational. It supports Reason-RFT in the following ways:

• Format Alignment: CoT-SFT teaches structured outputs (e.g., spatial transformation), reducing RL exploration cost and improving convergence.

• Knowledge Injection: In domain-shifted tasks like visual counting, CoT-SFT boosts generalization. Compared to RL-only, Reason-RFT improves by +23.2 (2B) and +14.4 (7B) on VC (DS-H).

• Sample Efficiency: CoT-SFT acts as a policy prior. As shown in Fig. 3, Reason-RFT reaches 70% of Reason-RFT-Zero’s performance using only 3% of its training data.

In short, CoT-SFT is not just a warm-up—it is crucial for format learning, generalization, and efficient RL adaptation.

---

Q1: "CoT data construction."

A: Plase refer to W3-1.

---

Q2: "Decoding strategy and temperature settings."

A: We used greedy decoding (temperature = 0.0) for all methods reported in the main text. To evaluate robustness under different decoding strategies, we further tested temperature = 0.5 and 0.7 using Qwen2.5-VL-3B-Instruct on three representative tasks.

Overall, greedy decoding yields the best absolute scores due to its determinism. However, relative performance rankings remain consistent across temperatures, and Reason-RFT consistently outperforms all baselines.

These results confirm that our method is robust and effective under varying decoding strategies, and the main conclusions of our paper remain unchanged.

---

Q3: "Report standard deviation and clarify number of runs."

A: Thank you for your question. To ensure reproducibility, we ran each method three times with different seeds on Qwen2VL-2B. Results (mean ± std) are shown below:

Across the 12 metrics above, all standard deviations are under 2.0, with most under 1.0, indicating high stability. Due to space constraints, we omit the Qwen2VL-7B results here but will include them in the revised version’s appendix.

---

We hope our response addresses your concern and we look forward to further discussion with you.

Dear Reviewer nCpk,

Thanks again for your diligent effort in reviewing our submission. We have carefully addressed the concerns raised and conducted the requested experiments. As the discussion phase deadline is approaching, we sincerely hope you can consider positively recommending our work if your concerns are solved. If you still have further comments/suggestions, please don't hesitate to let us know.

Best regards.

We sincerely thank the reviewer for the constructive and encouraging comments. We are grateful for your recognition of our work as a solid study on the roles of SFT and GRPO in VLM post-training, the value of our proposed evaluation tasks and datasets for future research, and the insightful experimental analysis. Your feedback is highly appreciated, and we address your specific concerns in detail below. We will carefully reflect your suggestions in the revised manuscript.

---

W1: "Limited base models. Can the authors provide analysis with other base models beyond Qwen-series?"

A: Thanks for your suggestion. To validate the effectiveness of our proposed method, we conducted additional experiments on InternVL3-2B-Instruct using the Visual Counting task. The results are shown below:

The results demonstrate that Reason-RFT consistently enhances performance on the InternVL backbone. Key observations:

• (1) Substantial Gains in Visual Reasoning:
  Compared to ANS, COT, and Reason-RFT-ZERO, our Reason-RFT approach leads to notable improvements:

  • InternVL3-2B:
    • +23.16 over ANS
    • +8.53 over COT
    • +18.06 over Reason-RFT-ZERO

• (2) Robustness under Domain Shift (T1-DSM):
  The largest gains occur in domain-shifted settings:

  • InternVL3-2B: +43.60 over Reason-RFT-ZERO on T1-DSM

• (3) Data Efficiency:
  Similar to Qwen2VL, Reason-RFT shows high data efficiency on InternVL3:
  achieving ~70% of the final performance of Reason-RFT-ZERO using only <5% of the data.

We will include the full results and analysis for InternVL in the revised version, under Appendix C: More Experimental Results on Different Backbones.

Additionally, based on your reference to the interesting findings in Qwen-series models and RLVR in [1], we further investigated the effect of random reward in the RL training stage. We conducted experiments using both Qwen2.5VL-3B and InternVL3-2B, applying discrete random rewards (0/1) and continuous random values in [0,1]. Results are as follows:

Qwen2.5VL-3B:

InternVL3-2B:

Based on these results, we conclude that random reward schemes do not enhance model performance. In fact, they degrade the accuracy for both Qwen2.5VL and InternVL3. For example, with random reward (0~1), performance drops by -9.67 and -11.25 on Task1 respectively.

---

Q1: "Why does SFT lead to worse results than the zero-shot performance of Qwen2VL-7B in visual counting (ID and DS-D)?"

A: Thanks for your question—this is indeed an interesting phenomenon. SFT tends to guide models toward memorizing domain-specific response patterns, as demonstrated in [1]. For strong base models like Qwen2VL-7B, which already perform well on visual counting tasks in the zero-shot setting (98.60), further SFT may not enhance reasoning ability but instead induce cognitive rigidity, where the model aligns too rigidly with training examples, leading to diminished generalization capability on unseen test cases. This rigidity can degrade performance on ID and DS-D test sets.

This also explains why Reason-RFT employs only a small number of samples (1.6k) during the Stage 1 CoT-SFT phase—sufficient to inject new task knowledge and activate novel reasoning patterns without cognitive rigidity.

[1] SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training, ICML'25.

---

Q2: "Why does ANS-SFT outperform, while pure RL underperforms on spatial transformation?"

A: Thanks for your question. Spatial transformation requires specific spatial reasoning and output patterns. When the base model lacks this ability (as evidenced by the low zero-shot scores of the 2B and 7B models—4.35 and 13.1, respectively), pure RL tends to explore ineffective regions of the reasoning space and output formats during the early stages of training. Additionally, due to the lack of specialized knowledge in pretraining, pure RL struggles to overcome the knowledge barrier—especially for smaller models like 2B—resulting in poor learning signals and ineffective reasoning. In contrast, ANS-SFT provides direct supervision to quickly learn the desired answering format, achieving readily achievable performance. Therefore, a two-stage SFT+RL (Reason-RFT) process is necessary to ensure stronger and more stable performance.

---

Q3: "How are different visual reasoning tasks coupled with SFT and RL strategies?"

A: Thank you for the insightful follow-up. Indeed, we observe a clear task-specific coupling between visual reasoning tasks and post-training strategies:

• SFT is more effective for tasks where format learning or domain-specific answer templates dominate performance—such as spatial transformation or counting tasks with structured outputs. In these cases, direct supervision helps the model quickly acquire the correct response pattern, especially when the base model lacks such capabilities in the zero-shot setting.
• RL, on the other hand, is more beneficial for tasks requiring multi-step reasoning—such as latent action reasoning or transformation sequence planning—where the model already shows basic task understanding. In such cases, RL fine-tuning helps reinforce coherent reasoning trajectories. However, for tasks the model has not yet grasped, pure RL often fails to provide effective learning signals, limiting its usefulness when used alone.

While neither SFT nor RL alone is sufficient, our Reason-RFT framework demonstrates that carefully coupling the two—via CoT-based cognitive activation and reward-guided refinement—yields robust improvements. We also find that tuning the data ratio and reward design plays a critical role in maintaining adaptability without overfitting. Additional task-wise analyses and ablations will be incorporated in the appendix of the revised version.

---

Q4: "How much does the ratio of data size for SFT and RL affect the performance?"

A: Thanks for your question. We found that for visual reasoning tasks, the optimal range of CoT samples used for SFT activation lies between 0.8k and 2k. Increasing the number of CoT samples beyond 2k brings marginal gains. Interestingly, when the proportion of CoT samples exceeds approximately 40% of the RL training data, the model performance begins to decline. Once it surpasses 80%, the performance tends to degenerate toward that of CoT-SFT alone (i.e., cognitive rigidity).

This is a particularly interesting phenomenon. We will include a performance trend plot and further discussion on this effect in the appendix of the revised version.

---

We hope our response addresses your concern and we look forward to further discussion with you.

Dear Reviewer krtY,

Thanks again for your diligent effort in reviewing our submission. We have carefully addressed the concerns raised and conducted the requested experiments. As the discussion phase deadline approaches, we sincerely hope that you will consider maintaining a positive recommendation for our work if you find your concerns sufficiently resolved. If you still have further comments/suggestions, please don't hesitate to let us know.

Best regards.

We sincerely thank the reviewer for the thoughtful and encouraging feedback. We are especially grateful for the recognition of our two-stage training framework, as well as the improved performance and transparency on visual reasoning. We deeply value your comments and suggestions, and address them in detail below. We will carefully incorporate the relevant clarifications and improvements into the revised manuscript.

---

W1 (Q1): "The novelty of our work."

A: Thank you for raising this point. We would like to clarify that our work is not a direct extension or simple combination of existing methods. Instead, it introduces a novel and unified post-training methodology specifically tailored to the core challenges of visual reasoning, such as cognitive rigidity and generalization to domain shift in dynamic visual reasoning.

In contrast to previous works that either (i) focus exclusively on text-based reasoning ([1]); (ii) adopt only-SFT ([2,3]) or only-RL ([4,5,6]) strategies for post-training in isolation; or (iii) involve a few concurrent works ([7,8]) that combine SFT and RL but remain restricted to static visual tasks with narrowly defined reward types—our approach is the first to introduce a generalizable training framework that systematically integrates supervised fine-tuning (SFT) and reward-based fine-tuning (RFT) in a tightly coupled manner, effectively addressing both static (e.g., structural perception) and dynamic (e.g., temporal visual counting, spatial transformation) visual reasoning under a unified reward formulation.

The novelty of our approach lies in the following aspects:

• An adaptive training sample allocation strategy that dynamically balances intermediate reasoning supervision with reward-guided exploration. This strategy enables more effective coupling between the SFT and RFT stages, supporting a unified learning process that aligns model behavior with visual reasoning objectives. Unlike prior methods that treat SFT and RL as disjoint stages, our design ensures their interaction is coherent and purpose-driven.

• A unified visual reasoning reward formulation that generalizes across diverse task formats. Beyond replicating standard reward templates, we introduce a principled reward design that captures reasoning correctness and completeness in a task-agnostic manner:

  • Structured-format reward (including summary, caption, thinking and answer tags) to enhance the quality and coherence of reasoning chains.
  • Discrete-value reward for tasks with discrete-value outputs (e.g., classification and counting).
  • Tolerance-based reward for tasks requiring continuous-value outputs (e.g., geometric estimation, visual grounding).
  • Function-matching reward for tasks with sequence-like outputs (e.g., action planning or tool calling).

  These reward formulations are designed to be task-agnostic and applicable, moving BEYOND handcrafted, task-specific designs commonly used in prior work.

• A comprehensive visual reasoning evaluation protocol, covering both in-domain and domain-shifted scenarios, and spanning synthetic and real-world visual environments. The evaluation rigorously benchmarks multiple dimensions of reasoning ability—including dynamic reasoning, geometric perception, and mathematical arithmetic—enabling us to assess generalization and robustness in a systematic manner.

[1] DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning, ArXiv'25.

[2] LLaVA-CoT: Let Vision Language Models Reason Step-by-Step, ArXiv'24.

[3] LlamaV-o1: Rethinking Step-By-Step Visual Reasoning in LLMs, ACL'25.

[4] Visual-RFT: Visual Reinforcement Fine-Tuning, ICCV'25.

[5] VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model, ArXiv'25.

[6] R1-Zero’s “Aha Moment” in Visual Reasoning on a 2B Non-SFT Model, ArXiv'25.

[7] R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization, ICCV'25.

[8] Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models, ArXiv'25.

---

W2: "CoT data construction."

A: Thanks for your question. The CoT data is constructed through a combination of automated generation and manual verification, with both diversity and correctness rigorously ensured. Specifically:

• Diversity assurance: We begin by leveraging a set of carefully designed reasoning-guided templates (e.g., "Let's think step by step", "Break down your reasoning", etc.) to prompt automatic generation using models like GPT-4o and Gemini-Pro. To increase stylistic diversity, we apply temperature sampling (temperature=0.7, top-k=50, top-p=0.9) and include manually crafted exemplars as few-shot demonstrations to enhance generation quality and depth.

• Correctness assurance: All generated outputs undergo filtering via both manual review and automated scripts. Low-quality samples—such as overly brief reasoning steps or answer inconsistencies with ground truth—are discarded. For each task subtype, we randomly sample 10% of the data for human validation. Subtypes with deficiencies are further corrected and supplemented, yielding a reliable and comprehensive dataset.

We will include a detailed description of the CoT data construction process in the appendix of the revised version.

---

W3: "Zero-shot setting consistency in Table 1 models."

A: Thanks for your suggestion. The open-source models reported in Table 1 are evaluated under the same zero-shot setting to ensure fair comparison. This includes identical evaluation datasets, question prompts, and generation parameters (e.g., temperature=0.0 for greedy sampling).

---

W4: "Generalization performance on generic benchmarks."

A: We thank the reviewer for pointing out the generalization performance in Table 6. Our primary goal is NOT to achieve the best results on general VQA benchmarks, but to explore an important challenge: how to enhance domain-specific complex reasoning without severely degrading other capabilities.

As shown in the table above, taking Qwen2VL-7B as example, while Reason-RFT does not significantly improve general VQA scores, it maintains a more stable general performance compared to other training paradigms (e.g., ANS-SFT, CoT-SFT, Reason-RFT-Zero), which often cause larger performance drops. For example, on RealWorldQA benchmark, ANS-SFT leads to a -19.09 drop, CoT-SFT -30.73, and Reason-RFT-Zero -22.09, while Reason-RFT reduces the gap to only -5.88, showing a much more stable performance behavior. Moreover, in some cases, Reason-RFT even brings slight gains (e.g., on MathVision, +1.30). This indicates its potential in balancing multiple abilities during training.

We believe this is a valuable insight for future work on developing more balanced and robust multimodal models. Further clarification will be included in the appendix of the revised version.

---

W5: "Add missing citations to prior neuro-symbolic methods."

A: Thank you for your suggestion. We will include the missing citations (e.g., NMN [9], IPRM [10]) in the revised version under the neuro-symbolic methods section of the related work.

[9] Neural Module Networks，CVPR'16.

[10] Learning to Reason Iteratively and Parallelly for Complex Visual Reasoning Scenarios, NeurIPS'24.

---

Q1: "CoT data construction."

A: Please refer to W2.

---

Q2: "Comparison of CoT quality before and after RL."

A: Thank you for your question. To assess the improvement in CoT quality after RL, we compared the reasoning outputs of the Qwen2VL-3B model trained with Reason-RFT (Stage 2, post-RL) and the one trained with only CoT-SFT (Stage 1, pre-RL) on the Structure Perception task as example. The findings are as follows:

Qualitative Analysis:

1. Despite high textual similarity, the post-RL model exhibits stronger logical coherence between reasoning steps, with fewer broken reasoning chains. For example, in Fig. 16 (case 2) of the main text, the pre-RL model correctly infers a formula but omits the multiplication by 2 in the subsequent step—an issue less common after RL.
2. The post-RL model demonstrates reflective reasoning behaviors (e.g., "let me double check"), which we did not observe in pre-RL outputs (e.g., Fig. 1).

Quantitative Analysis:

1. Reasoning Step Count: We sampled 100 examples and used GPT-4o to count reasoning steps. The post-RL model produced, on average, 2.7 more steps than the pre-RL model.

2. Lexical Usage: We categorized two types of expressions:

   • Prompting words: "oh I see", "let me think step by step", "let me double check"
   • Logical connectives: "so", "therefore", "first", "but", "moreover"

   After RL, the average frequency of prompting words and logical connectives increased by 14% and 23%, respectively.

3. Answer Accuracy: As shown in the Structure Perception subplot of Fig. 13 in the appendix, the final answer accuracy improved by an average of +20.56% after Stage 2 training.

These results collectively confirm that RL enhances both the coherence and effectiveness of CoT reasoning.

---

We hope our response addresses your concern and we look forward to further discussion with you.

Dear Reviewer rmS6,

Thank you once again for your thoughtful and constructive feedback on our submission. We have carefully addressed the concerns you raised and conducted the requested experiments. If our response has resolved your concerns, we would sincerely appreciate it if you could consider increasing your score. If there are still any remaining questions or suggestions, please feel free to let us know—we would be happy to further clarify.

Best regards.

We sincerely thank the reviewer for the constructive comments and for acknowledging our well-motivated methodology and effective reward design. Below, we address your specific concerns in detail.

---

W1 (Q1): "The novelty of our work."

A: Thank you for raising this point. We would like to clarify that our work is not a direct extension or simple combination of existing methods. Instead, it introduces a novel and unified post-training methodology specifically tailored to the core challenges of visual reasoning, such as cognitive rigidity and generalization to domain shift in dynamic visual reasoning.

In contrast to previous works that either (i) focus exclusively on text-based reasoning ([1]); (ii) adopt only-SFT ([2,3]) or only-RL ([4,5,6]) strategies for post-training in isolation; or (iii) involve a few concurrent works ([7,8]) that combine SFT and RL but remain restricted to static visual tasks with narrowly defined reward types—our approach is the first to introduce a generalizable training framework that systematically integrates SFT and RFT in a tightly coupled manner, effectively addressing both static (e.g., structural perception) and dynamic (e.g., temporal visual counting, spatial transformation) visual reasoning under a unified reward formulation.

The novelty of our approach lies in the following aspects:

• An adaptive training sample allocation strategy that dynamically balances intermediate reasoning supervision with reward-guided exploration. This strategy enables more effective coupling between the SFT and RFT stages, supporting a unified learning process that aligns model behavior with visual reasoning objectives. Unlike prior methods that treat SFT and RL as disjoint stages, our design ensures their interaction is coherent and purpose-driven.

• A unified visual reasoning reward formulation that generalizes across diverse task formats. Beyond replicating standard reward templates, we introduce a principled reward design that captures reasoning correctness and completeness in a task-agnostic manner:

  • Structured-format reward (including summary, caption, thinking and answer tags) to enhance the quality and coherence of reasoning chains.
  • Discrete-value reward for tasks with discrete-value outputs (e.g., classification and counting).
  • Tolerance-based reward for tasks requiring continuous-value outputs (e.g., geometric estimation, visual grounding).
  • Function-matching reward for tasks with sequence-like outputs (e.g., action planning or tool calling).

  These reward formulations are designed to be task-agnostic and applicable, moving BEYOND handcrafted, task-specific designs commonly used in prior work.

• A comprehensive visual reasoning evaluation protocol, covering both in-domain and domain-shifted scenarios, and spanning synthetic and real-world visual environments. The evaluation rigorously benchmarks multiple dimensions of reasoning ability—including dynamic reasoning, geometric perception, and mathematical arithmetic—enabling us to assess generalization and robustness in a systematic manner.

[1] DeepSeek-R1, ArXiv'25.

[2] LLaVA-CoT, ArXiv'24.

[3] LlamaV-o1, ACL'25.

[4] Visual-RFT, ICCV'25.

[5] VLM-R1, ArXiv'25.

[6] R1-Zero’s “Aha Moment”, ArXiv'25.

[7] R1-Onevision, ICCV'25.

[8] Vision-R1, ArXiv'25.

---

W2-1 (Q2): "CoT data construction."

A: Thank you for the question. Our CoT data is built through a combination of automated generation and manual verification, with careful attention to both diversity and correctness.

• Diversity: We use reasoning-guided templates (e.g., "Let's think step by step", "Break down your reasoning") to prompt models like GPT-4o and Gemini-Pro, aided by temperature sampling (T=0.7, top-k=50, top-p=0.9) and include manually crafted exemplars as few-shot demonstrations to enhance generation quality and depth.

• Correctness: Outputs are filtered via both human review and scripts. We remove brief or inconsistent responses, and manually validate 10% of each task subtype. Subtypes with issues are corrected or extended to ensure overall reliability.

A full description of this pipeline will be included in the appendix of the revised version.

---

W2-2 (Q2): "Trajectory length and problem complexity."

A: We determine the trajectory length of each subtask by combining 50% manually constructed representative samples and 50% model-generated average reasoning length. During the automated filtering stage of CoT samples, we exclude those with lengths less than 60% or greater than 140%of the target trajectory length.

Statistical analysis shows the average trajectory lengths for the three tasks are:

• Visual Counting: μ ≈ 70, σ ≈ 30,
• Structure Perception: μ ≈ 180, σ ≈ 80,
• Spatial Transformation: μ ≈ 400, σ ≈ 120

For problem complexity, we define problem complexity based on the number of reasoning steps in the CoT process. We sampled 100 instances per task and used GPT-4o to evaluate the number of reasoning steps. Results indicate that CoT-SFT samples range from 3 to 10 steps, with an average of 5 steps.

We will include the statistical details of CoT-SFT construction in the appendix of the revised version.

---

W3 (Q3): "Reward assignment."

A: Thank you for your question. Reward types are predefined during data construction based on each question’s ground truth format. Each instance is annotated with its expected answer type (e.g., discrete label, numerical value, or structured output), and the corresponding reward function (e.g., discrete-valued, mathematical, or functional-based) is selected to ensure alignment between output format and training feedback.

---

W4 (Q4): "Experiments on Qwen2.5-VL."

A: Thanks for your suggestion. To further validate the effectiveness of our method, we conducted experiments on Qwen2.5-VL-3B-Instruct. The results are summarized below:

The results demonstrate that Reason-RFT consistently improves model performance even when applied to the more capable Qwen2.5-VL base models. Key Observations are as follow. Taking Qwen2.5VL-3B as example:

• (1) Clear Improvement in Visual Reasoning:

  • Reason-RFT outperforms:
    • ANS by +22.5 (T1), +15.4 (T2), +3.8 (T3)
    • COT by +8.1 (T1), +5.1 (T2), +8.1 (T3)
    • RFT-ZERO by +17.9 (T1), +3.2 (T2), +10.1 (T3)

• (2) Strong Generalization under Domain Shift:
  Specifically, on the T1-DSM subset +44.0 over RFT-ZERO

• (3) Data Efficiency:
  Similar to Qwen2VL, the Reason-RFT training paradigm on Qwen2.5VL demonstrates high data efficiency:
  achieving ~70% of the final performance of Reason-RFT-ZERO using only <5% of the data.

We will add detailed results and analysis for Qwen2.5-VL in Appendix C as a new subsection: Additional Results on Alternative Backbones.

---

W5: "Stage1 CoT-SFT’s benefit beyond spatial transformation."

A: Thanks for your question. While Stage 1 CoT-SFT brings the most evident gains on visual transformation tasks due to its structured format, its benefits extend across reasoning scenarios:

• Format Alignment: In tasks with compositional or structured output formats, such as spatial transformations, CoT-SFT helps models quickly adapt to expected answer format, avoiding inefficient RL exploration and accelerates convergence.
• Domain Knowledge Injection: On domain-shifted tasks (e.g., VC-DS), where zero-shot performance is poor, CoT-SFT injects domain-specific reasoning traces. Compared to RL-only, Reason-RFT improves by +23.2 (2B) and +14.4 (7B).
• Data Efficiency: CoT-SFT serves as an effective policy prior. In Fig. 3 in the main text, Reason-RFT reaches 70% of full-RFT performance using only 3% of its training data.
• Generalization Retention: CoT-SFT acts as a regularizer to retain broad reasoning ability (Tab. 6 in the main text), mitigating overfitting seen in RL-only.

W6: "The novelty of Reward Design."

A: Thank you for the question. We would like to clarify the novelty of reward design as follow:

• We propose a structured format reward (Summary + Caption + Thinking + Answer) to assess reasoning chain quality, with ablation studies confirming its effectiveness across different paradigms.
• We design fine-grained function-based rewards that go beyond sequence-level overlap, incorporating item-level exact, partial, and function-only matches. These are further evaluated under different credit assignment strategies (e.g., exact match only, partial credit, penalized partial). Such nuanced reward designs are rarely explored in previous work.

Moveover，we emphasize that our primary contribution in this part is NOT the design of intricate and complex reward functions, but the introduction of a general and practical reward formulation that supports broad applicability across diverse visual reasoning tasks with discrete-value, continuous-value, or agent-based structured outputs.

---

We hope our response addresses your concern and we look forward to further discussion with you.

Thank you again for your thoughtful follow-up and for considering our additional results and explanations helpful.

As you rightly emphasized, it's essential to clearly describe both the automated and manual components of our CoT-SFT data construction. While our rebuttal already outlined the pipeline (see W2-1 and W2-2), we will further expand and detail these aspects in the revised version, including:

(1) Automated Generation

We use reasoning-guided templates such as:

• "Let's break down the problem step by step..."
• "To answer this, we need to consider..."

These templates are combined with model prompting using GPT-4o and Gemini-Pro, under temperature-controlled sampling (T=0.7, top-k=50, top-p=0.9). To improve quality and reasoning depth, we also insert hand-crafted few-shot exemplars tailored to each subtask.

We will include the full list of prompt templates and few-shot examples in the appendix of the revised version.

(2) Automated Filtering

Each CoT response is filtered based on:

• Length range: We compute the target trajectory length for each subtask using 50% human-written samples and 50% model-generated examples. Samples falling below 60% or above 140% of this average length are discarded.
• Inconsistency: We exclude samples containing inconsistencies with the known ground truth.

Average trajectory lengths across tasks are:

• Visual Counting: μ ≈ 70, σ ≈ 30
• Structural Perception: μ ≈ 180, σ ≈ 80
• Spatial Transformation: μ ≈ 400, σ ≈ 120

These statistics will be visualized and added to the appendix.

(3) Human Verification

We randomly sample 10% of CoT samples from each subtask for manual review. Reviewers focus on coherence, logical validity, and alignment between reasoning and answer.

For instance, in spatial or geometry-based tasks, we occasionally observe errors like:

• Correct final answer with an incorrect reasoning chain (e.g., solving for triangle area using the Pythagorean theorem).
• Inconsistent steps that contradict earlier statements. For example, a CoT response might begin by stating "There are 3 red blocks on the left and 2 on the right," but later conclude that the total is 6—contradicting the initial count.

In a follow-up quality check, we found that prior to human verification, approximately 3.8% of samples contained critical logical flaws. After verification, the residual error rate was reduced to below 1%, indicating high post-cleanup reliability.

---

We will ensure all the above details, examples, and statistics are included in the revised manuscript's appendix. We also commit to open-sourcing all prompt construction templates and associated code for reproducibility and community use.

If these clarifications have addressed your remaining concerns, we would be truly grateful if you would consider updating your score. Please do let us know if there's anything further we can provide. Thanks again for your time and thoughtful feedback.