Fast-Slow Thinking GRPO for Large Vision-Language Model Reasoning

We’d like to thank the reviewer for acknowledging the novelty and effectiveness of our proposed approach. We address your raised points as follows.

W1,Q1: Experiments stop at 7B parameters, scalability on larger models remains untested.

We agree with your point that validating the scalability of our approach on larger models is important.

While we noted the computational constraints in our original manuscript's limitations section, to directly address your concern, we train FAST-GRPO on Qwen-2.5-VL-32B with the same training dataset. Due to time and compute constraints, we stopped training after three epochs (1200 GPU hours), resulting in a likely sub-optimal checkpoint. We compare our method against several strong baselines on 6 benchmarks including the newly introduced MM K12 benchmark proposed in MM-Eureka [1], a comprehensive reasoning benchmark of 2,000 problems spanning math, physics, chemistry, and biology. The results are presented below:

These new experimental results demonstrate the effectiveness and scalability of our approach:

1. Comparable or Superior Reasoning Accuracy: Compared to strong "slow-thinking" models like Vision-R1-32B and MM-Eureka-32B, FAST-32B achieves comparable or even better average accuracy across various reasoning and VQA benchmarks.
2. Significant Efficiency Gains: The average response length of FAST-32B (422.1 tokens) is nearly 40% shorter than that of Vision-R1-32B (710.6 tokens) and 22% shorter than that of MM-Eureka-32B (546.0 tokens), achieving a balance for both performance and efficiency.

---

Q2: Improving the reliability of the difficulty estimation.

Thank you for your question regarding the reliability of our difficulty estimation. Our difficulty estimation framework is composed of two complementary components to ensure its reliability:

1. Extrinsic Difficulty: This metric is derived from the model's performance (pass@k on a given problem). As this metric is derived directly from the model's actual performance, it accurately reflects the model's empirical problem-solving ability.
2. Intrinsic Difficulty: This metric captures the inherent visual complexity of an image using GLCM and ViT entropy. The reliability of our intrinsic metric is validated from two perspectives:
   • The GLCM component is established to have strong human alignment ($SRCC=0.75, PLCC=0.77$)[2].
   • As detailed in Section 5.4 Validation of Image Complexity, our human annotation study confirmed that our combined image complexity metric has a moderate and statistically significant correlation with human judgments ($SRCC=0.49, PLCC=0.50$).

By combining these two metrics, our approach provides a comprehensive and reliable estimation of problem difficulty.

---

W2,Q3: Evaluation focues on math/diagram reasoning; Generalisation to open-domain VQA, captioning or dialogue is uncertain.

While our initial focus was on enhancing visual reasoning across math, science, VQA, and figure understanding, we agree that demonstrating broader generalization capability strengthens the value of our work.

First, we would like to clarify that, as GRPO typically requires verifiable reward signals, our training dataset did not include open-domain VQA samples. Despite this, our method was evaluated on the MM-Vet benchmark, an open-domain VQA benchmark, where FAST-7B shows gains over the base model (71.2% vs. 67.1%).

Second, to address your concerns, we have conducted a comprehensive set of experiments on 5 additional benchmarks, which cover diverse tasks including open-domain VQA, hallucination analysis, low-level visual perception, and scientific reasoning. The additional benchmarks are:

• Bingo Score [3]: An open-domain VQA benchmark designed for holistic hallucination analysis.
• MMHal Benchmark [4]: An open domain VQA benchmark, evaluating both hallucination and informativeness to calculate an overall socre.
• MMVP [5]: A diagnostic benchmark that evaluates low-level visual perception capability of LVLMs.
• MMEval-Pro [6]: A comprehensive calibrated evaluation benchmark, adapted from sources including MathVista, MMMU, ScienceQA and manually crafted questions.
• MM K12 [1]: A Human-annotated multimodal scientific reasoning benchmark across math, physics, chemistry and biology.

We compare FAST-3B and FAST-7B against strong slow-thinking baselines, with the results below:

These results on new benchmarks demonstrate that:

1. Our method has strong generalization capabilities in scientific reasoning. The performance of FAST on MM K12, which covers physics, chemistry, and biology, shows that our method generalizes well beyond math to other scientific domains, outperforming strong baselines.
2. Our method outperforms slow-thinking LVLMs in open-domain VQA tasks. On the hallucination benchmarks (Bingo Score and MMHALU), visual perception, and general VQA and benchmarks (MMVP and MMEval-Pro), slow-thinking LVLMs tend to significantly degrade, while our FAST models achieve higher scores. This observation corroborates the findings of Liu et al. [7], who noted that lengthy reasoning chains in slow-thinking models can amplify visual hallucinations.

[1] Meng, Fanqing, et al. "Mm-eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning." arXiv:2503.

[2] Zhang, Jinjin, et al. "Diffusion-4k: Ultra-high-resolution image synthesis with latent diffusion models." CVPR. 2025.

[3] Cui, Chenhang, et al. "Holistic analysis of hallucination in gpt-4v (ision): Bias and interference challenges." arXiv preprint arXiv:2311.03287 (2023).

[4] Sun, Zhiqing, et al. "Aligning Large Multimodal Models with Factually Augmented RLHF." ACL. 2024.

[5] Tong, Shengbang, et al. "Eyes wide shut? exploring the visual shortcomings of multimodal llms." CVPR. 2024.

[6] Huang, Jinsheng, et al. "MMEVALPRO: Calibrating Multimodal Benchmarks Towards Trustworthy and Efficient Evaluation." NAACL, 2025

[7] Liu, Chengzhi, et al. "More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models." arXiv:2505.

Dear Reviewer,

We hope this message finds you well. As the discussion period is nearing its end, we wanted to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you'd like us to consider, please let us know. Your insights are invaluable to us, and we're eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper.

Dear Reviewer NQ6C,

Thank you once again for your valuable comments on our submission. As the discussion phase is approaching its end, we would like to kindly confirm whether we have sufficiently addressed all of your concerns. Should there be any remaining questions or areas requiring further clarification, please do not hesitate to let us know. If you are satisfied with our responses, we would greatly appreciate your consideration in adjusting the evaluation scores accordingly.

We sincerely look forward to your feedback.

Dear Reviewer,

The authors have already submitted their response to your review, and we are approaching the deadline for author-reviewer discussions (August 8, 11:59 PM AoE).

Could you please share your feedback with the authors regarding whether their response sufficiently addresses the concerns raised in your review, and if any issues remain that require further clarification or elaboration?

As a key reminder, active engagement in the discussion process is critical at this stage. Reviewers must participate in discussions with authors (e.g., by posting follow-up feedback in the discussion thread), and then submit a Mandatory Acknowledgement to confirm your participation.

Best regards,

AC

We’d like to thank the reviewer for acknowledging the novelty and effectiveness of our proposed approach. We address your raised insightful points as follows.

W1, Q1: The potential conflicting reward signal could hinder the model's ability to learn such challenging problems. Have you experimented with alternative formulations, such as a weighted sum?

To ensure clarity, we will adopt the reviewer's terminology for this response, where $S_{difficulty}$ refers to the model's extrinsic difficulty (termed $S_{extrinsic}$ in our paper) and $S_{complexity}$ refers to the final combined score (termed $S_{difficulty}$ in our paper).

In our method design, the corner case you proposed, a problem with high difficulty ($S_{difficulty}$) but low visual complexity ($H_{image}$) will indeed result in a low final complexity score ($S_{complexity}$). However, we will demonstrate that this does not create a conflicting reward signal.

Our analysis is twofold:

1. Our reward design avoids conflicting signals from this corner case.
2. Empirical evidence shows that using a weighted sum yields nearly identical results, suggesting the practical impact is negligible.

---

Analysis 1: Reward Design Avoids Conflicting Signals

As detailed in the below reward equation (Equation 4 in paper), our difficulty-aware length reward ($r_t$) is applied based on three distinct cases:

\begin{cases} 1 - \frac{L}{L_{avg}} & \text{if } (S_d < \theta) \wedge (r_a = 1) \\\\ \min\left(\frac{L}{L_{avg}} - 1, 1\right) & \text{if } (S_d \ge \theta) \wedge (r_a = 0) \\\\ 0 & \text{otherwise} \end{cases}$$ Case 1: When the solution is **correct** and the question is categorized as **"Not Complex"** ($S_{complexity} < \theta$), the length reward $r_t$ encourages shorter responses. Specifically, it penalizes responses where the length $L$ exceeds the batch average length $L_{avg}$. Case 2: When the solution is **incorrect** and the question is categorized as **"Complex"** ($S_{complexity} \ge \theta$), the reward $r_t$ moderately encourages longer responses. Case 3: Otherwise. This category includes two scenarios: **a) the solution is incorrect and the question is "Not Complex"**, and b) the solution is correct and the question is "Complex". In these cases, $r_t$ is 0, providing neither a reward nor a penalty. First, since the problem is inherently difficult for the model, the majority of its sampled responses will be incorrect ($r_a=0$). This situation falls into **Case 3**. In this case, the length reward $r_t$ is zero. This means the model is **not penalized for generating longer, exploratory reasoning chains for most of its attempts**. This directly avoids the conflicting reward you were concerned about. Second, in the rarer event that the model produces a correct solution ($r_a=1$), it falls into **Case 1**. Here, a modest length penalty is only applied if the response is longer than the batch average length $L_{avg}$. Additionally, **the final reward is dominated by the primary accuracy reward ($r_a=1$)**. This is because, as defined in our reward ($r = r_a + \lambda_r \cdot r_f + \lambda_t \cdot r_t$), other rewards (format and length) are scaled by coefficients of 0.5. --- **Analysis 2: Empirical Evidence Shows This Corner Case Has Negligible Impact** To validate our design, we ranked 1,000 random training samples to analyze the correlation between different scoring metrics. We focused on two key aspects: **(1)** the ranking correlation between $S_{difficulty}$ and $H_{image}$ to evaluate potential corner cases, and **(2)** the ranking correlation between multiplicative and the weighted sum. Our correlation metrics included Spearman's Rank Correlation Coefficient (SRCC), Pearson's Linear Correlation Coefficient (PLCC), and Intersection@top-200. The results provide insights about our data distribution. * **First**, there is a strong positive correlation between $S_{difficulty}$ and $H_{image}$, which suggests the reviewer's proposed corner case is infrequent in our data. * **Second**, the two formulations (multiplicative vs. weighted sum) have a high correlation and identify 193 of the same top 200 most complex questions. | |SRCC| PLCC | Intersect@top 200 | |:----:|:-:| :-: | :-: | | **$S_{difficulty}$ vs. $H_{image}$** | 0.75 | 0.68 | 156 | | **Multiplicative vs. Weighted sum** | 0.94 | 0.92 | 193 | Finally, we **experimented FAST-GRPO with the weighted-sum formulation ($S_{complexity}=S_{difficulty}+ H_{image}$)**. As shown below, a specific formulation has no significant impact on the model's performance. | | MathVision | MathVista | MathVerse | Avg. Len. | | :--- | :---: | :---: | :---: | :---: | | **Multiplicative** | 30.6 | 73.8 | 50.6 | 175.5 | | **Weighted Sum** | 29.1 | 73.9 | 50.2 | 183.8 | In summary, **our analysis shows that while the corner case is theoretically possible, it is rare in our data distribution, and our method's performance is robust to the specific formulations.** We will incorporate these results and analysis into the revision. --- > **W2: Filtering strategy is not laid out well.** To filter our initial 500K-question dataset, we applied a rigorous filtering process. First, we deduplicated all samples with duplicate images and removed any questions that overlapped with our evaluation benchmarks. Next, to satisfy the GRPO algorithm's need for verifiable rewards, we standardized the data to include only closed-form questions—such as multiple-choice, numerical, or single-word answers—using regular expressions and custom rules. Finally, we filtered by extrinsic difficulty. We measured our base model's success rate for each question (pass@8) and removed those that were either too easy (pass@8 = 1) or too hard (pass@8 = 0), as these provide a poor learning signal for training. Due to limited space, further specifics will be in the revision. --- > **W3, Q2: A graded approach to switching from slow to fast thinking.** In response to this feedback, we have designed and conducted experiments to compare our original binary strategy with a **continuous Slow-to-Fast** curriculum. In this new setup, we introduce a linear scheduling for sampling "Easy" questions. The probability of sampling an easy question, $P_{easy}$, increases linearly with the training epoch $e$ (from $0$ to total epoches $E=10$), following $P_{easy}(e) = P_{max} \cdot (e/E)$, where $P_{max}$ is a hyperparameter denotes the max sampling probability of easy questions, set to 0.4. This ensures a gradual transition from focusing on hard and medium samples to incorporating more easy samples as training progresses. We evaluated this continuous approach on three benchmarks, with the other settings identical to our main experiments. The results are presented below: | | MathVista | MathVision | MathVerse | Avg. Length | | :--- | :---: | :---: | :---: | :---: | | **Binary** | 73.8 | 30.6 | 50.6 | 175.5 | | **Continuous** | 74.4 | 30.9 | 51.0 | 221.2 | As shown in the table, the continuous curriculum can **surpass the binary method in accuracy, while with longer responses.** We hypothesize this is because the $P_{max}$ is not sufficiently high to sample adequate easy questions. We will incorporate results into the revision. --- > **W4: FAST-GRPO on Larger-scale model.** **Thank you for the question. As this point is similar to the one raised in comment W1,Q1 by Reviewer NQ6C, we respectfully refer you to our detailed response provided there.** --- > **W5: Lack qualitative analysis.** We'd like to respectfully point out that we did provide qualitative analyses in the appendix, which are: * **Case Study of Fast-Slow Thinking (Appendix H, Figure 10):** This section provides a direct, side-by-side comparison of our FAST model against the Base Model and a "slow-thinking" model (R1-OneVision) on two distinct problems. * On a simple question (coordinate identification), our model provides a correct and concise 59-token answer, while R1-OneVision exhibits overthinking with a needlessly verbose 349-token response. * On a relatively complex geometry problem, the Base Model fails, and while R1-OneVision is correct, it is again verbose (676 tokens). In contrast, our FAST model adaptively adopts slow thinking, producing a correct and significantly more efficient 375-token solution. * **Analysis of Naive GRPO Failure (Appendix F, Figure 8):** We show a case where naive GRPO produces a correct final answer but with a logically flawed chain-of-thought, which highlights the overfitting problem that FAST-GRPO can effectively alleviate. **Due to the rebuttal's restrictions (e.g., no figures), we will provide more qualitative analysis, including case study and failure pattern in the revised version.** --- > **Q3: Final gains from dataset curation or FAST-GRPO algorithm.** First, to clarify the point raised from **Table 5 in Section 5.3 Ablations**, the term **Data Sampling** refers specifically to the Slow-to-Fast sampling strategy used during training. This is **distinct from the one-time data filtering** we performed as a preprocessing step to create the 18K training set used across all relevant experiments. Second, to precisely isolate and quantify the contribution of our algorithm, we designed the **Naive GRPO** baseline. Specifically, this baseline is trained on the **same 18K curated dataset**, without the core components of our method. The results from Table 5 demonstrate a significant performance uplift: Compared to the naive baseline, FAST-GRPO simultaneously boosts accuracy by +3.4 (MathVista), +2.5 (MathVision) points, while also reducing the response length from 243.6 to 175.5 tokens. In conclusion, while a well-curated dataset provides a strong foundation, **the FAST-GRPO algorithm itself mainly contributes to improvement in reasoning accuracy and efficiency**. --- > **Format Concern: boundaries are not defined in the algorithm** The boundary is half of the total epochs. We will revise it in the revision.

Dear Reviewer,

We hope this message finds you well. As the discussion period is nearing its end, we wanted to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you'd like us to consider, please let us know. Your insights are invaluable to us, and we're eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper.

Thank you for your valuable and constructive feedback.

More Qualitative Analysis and Error Breakdown (1/2)

Beyond the case studies in appendix, we further qualitatively analyse the model's bahavior. We first present a systematic breakdown of the model's failure modes.

Our error breakdown of failure cases of FAST-7B, R1-OneVision-7B and base model (Qwen2.5-VL-7B) on MathVista identifies three predominant categories:

• Visual Perception Failure: Models incorrectly extract or interpret visual information.
• Reasoning Error Propagation: Models often can't backtrack to fix a mistake made midway through their reasoning.
• Knowledge Confliction & Knowledge Gap: Models either prioritises internal knowledge over conflicting visual evidence or hallucinate a response when faced with a knowledge gap.

Insight Analysis

• Reduction in Reasoning & Knowledge Errors: Compared to the baseline, FAST-7B reduces failures from Reasoning Error Propagation and Knowledge Confliction by ~27% and ~19%, respectively. This is attributed to its adaptive thinking: shorter logical chains minimize opportunities for error propagation and effectively suppress hallucinations caused by long reasoning chains that contradict visual evidence.

• Visual Perception as the Primary Bottleneck: Over 51% of all failures in FAST-7B are due to Visual Perception Failure. This demonstrates that if the model initially "sees" the context incorrectly, even a flawless reasoning process will lead to a wrong answer. Therefore, enhancing fundamental visual capability is a crucial factor for future improvement.

---

We provide specific cases for each failure mode below.

Failure Mode 1: Visual Perception Failure

• Case A: Misinterpreting Visual Context

  • Question: "What is the highest amount this class measures?"
  • Image: A beaker with measurement lines up to 400ml and a label indicating a 600ml total capacity.
  • Error: The model sees both "400ml" and "600ml" but fails to distinguish between a marked measurement and the container's total capacity, incorrectly answering 600.

• Case B: Incorrect Data Extraction from Charts

  • Question: "How many bars have value below 40?"
  • Image: A bar chart.
  • Error: The model correctly identifies three bars below the threshold but erroneously includes a fourth bar (value 42.1) in its count, resulting in a wrong total.

---

Failure Mode 2: Reasoning Error Propagation

• Case A: Mid-Reasoning Comparison Error

  • Question: "Is the sum of the smallest two bars greater than the largest bar?"
  • Image: A bar chart.
  • Error: The model correctly identifies the values and calculates their sum (9.29% + 12.51% = 21.8%). However, it fails at the final logical comparison (comparing 21.8% to the largest bar, 21.37%), leading to the wrong 'Yes'/'No' answer.

• Case B: Hallucinating a Geometric Rule

  • Question: A geometry problem asking to find an angle.
  • Image: A diagram with parallel lines and a triangle.
  • Error: The model abandons a valid reasoning path to invent a false geometric rule. It then performs a correct calculation based on this flawed premise and fails to self-correct, producing an incorrect answer.

---

Failure Mode 3: Knowledge Confliction & Knowledge Gap

• Case A: Confident Assertion of Incorrect Knowledge

  • Question: "In which part of the mold are the cylindrical ports located?"
  • Image: A diagram clearly showing the ports in the lower half.
  • Error: The model ignores the direct visual evidence and instead asserts its incorrect "general knowledge" from its language model that ports are "typically located in the upper part," confidently selecting the wrong answer.

• Case B: Flawed Application of Domain-Specific Knowledge

  • Question: "What is the ACK number at message 6?"
  • Image: A TCP transmission sequence diagram.
  • Error: The model understands the TCP protocol in theory but fails to apply that knowledge correctly to the specific data flow in the diagram. It performs a flawed calculation using the wrong values.

We’d like to thank the reviewer for acknowledging the novelty and effectiveness of our proposed approach. We address your raised points as follows.

W1, Q1: Generalizability of the proposed FAST-GRPO method on reasoning tasks in other domains.

We thank the reviewer for raising the important question regarding the generalizability of FAST-GRPO to other reasoning domains.

First, we would like to clarify that generalizability was a key consideration in our original design from dataset construction to evaluation.

On dataset construction: Our training dataset was intentionally curated for diversity, encompassing a wide range of tasks beyond mathematics. The dataset is composed of:

• Mathematics (~40%), with samples from sources including MathV360K and Geometry3K.
• Science (~20%), with key sources including AI2D and ScienceQA.
• Visual QA (~20%), sourced from datasets including ShareGPT4V and Vizwiz.
• Figure Understanding (~20%), drawing from datasets including DocVQA and ChartQA.

On evaluation: This method was validated not only on math benchmarks but also on general-purpose evaluations like MM-Vet (achieving 71.2% vs. base model's 67.1%) and multimodal reasoning evaluations like MathVista (73.8% vs. base model's 68.2%), which contains 10.7% scientific reasoning questions covering physics and biology.

Second, to more directly and comprehensively address your valuable concern, we have evaluated FAST-3B and FAST-7B on the challenging MM-K12 benchmark proposed in MM-Eureka[1]. MM-K12 contains 2,000 reasoning questions evenly distributed across math, physics, chemistry, and biology (25% each). We present the results below against strong baselines.

The results demonstrate the strong generalizability of our approach. Specifically:

• Strong Generalization Across Science Domains: FAST-7B significantly outperforms its base model (Qwen-2.5-VL-7B) across all science domains: Physics (+8.4), Chemistry (+7.0), and Biology (+8.8). This confirms that FAST-GRPO is effective beyond mathematical reasoning.
• Competitive Accuracy with Efficiency: Compared to strong slow-thinking models like R1-Onevision-7B and OpenVLThinker-7B, FAST-7B achieves better accuracy (62.2% vs 60.6%) in science domains while being more efficient, reducing the average response length by approximately 33.8% (371.2 vs. 561.0 tokens). Even when compared to MM-Eureka-7B, trained on the MM-K12 train set, FAST-7B achieves comparable accuracy (62.2% vs. 64.5%) with a 30% reduction in response length (371.2 vs. 537.8 tokens).
• Scalability: The improvements are consistent across model sizes, with FAST-3B outperforming a larger model Qwen-2.5-VL-7B (55.1% vs 53.6%) with more efficient reasoning (318.1 vs 477.6 tokens).

We believe this new evidence supports the generalizability of FAST-GRPO. We will incorporate these results and their corresponding analysis into our revision.

[1] Meng, Fanqing, et al. "Mm-eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning." arXiv:2503.

---

W2: Section 5.4 could be restructured to provide more clear insights.

We sincerely appreciate this valuable suggestion. Restructuring Section 5.4 would improve its clarity and narrative flow. In the revised manuscript, we will adopt this advice and reorganize the section. Specifically, we will group the analyses on 'Validation of Image Complexity', 'Difficulty Threshold Analysis', and 'Extrinsic Difficulty Analysis' under a new, cohesive sub-subsection titled 'Analysis of Image-Text Question Difficulty'.

Dear Reviewer,

We hope this message finds you well. As the discussion period is nearing its end, we wanted to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you'd like us to consider, please let us know. Your insights are invaluable to us, and we're eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper.

Dear Reviewer A2xT,

Thank you once again for your valuable comments on our submission. As the discussion phase is approaching its end, we would like to kindly confirm whether we have sufficiently addressed all of your concerns. Should there be any remaining questions or areas requiring further clarification, please do not hesitate to let us know.

We sincerely look forward to your feedback.

Dear Reviewer,

The authors have already submitted their response to your review, and we are approaching the deadline for author-reviewer discussions (August 8, 11:59 PM AoE).

Could you please share your feedback with the authors regarding whether their response sufficiently addresses the concerns raised in your review, and if any issues remain that require further clarification or elaboration?

As a key reminder, active engagement in the discussion process is critical at this stage. Reviewers must participate in discussions with authors (e.g., by posting follow-up feedback in the discussion thread), and then submit a Mandatory Acknowledgement to confirm your participation.

Best regards,

AC