We sincerely thank the reviewer for their thorough evaluation and insightful feedback. The comments are highly constructive and have helped us improve the quality of our work.

Below, we conducted additional experiments and analysis to address the questions and provide revision plans for the areas of improvement.

1. On Quantifying Reflective Behavior and Metacognition (Weakness 2, Question 1)

This is an excellent point. We were aware of this qualitative analysis of reflections but encountered challenges in properly defining reflections and recognizing reflections in the responses: in many occasions, model reflections cannot be detected by pre-defined reflective phrases.

Below we attempted a qualitative analysis by focusing on the reflections triggered by our "forced rethinking". We conducted our analysis using VL-Rethinker-32B inferred responses on the validation data, selecting a subset of queries that keep the correctness of responses balanced. We defined problem difficulty based on the mean accuracy of responses within the same query group. We then measured the frequency of rethinking (i.e., generating the "Wait" trigger), the success rate (initial answer was wrong, final answer is correct), and the failure rate (initial answer was correct, final answer is wrong).

Our findings are summarized in the table below:

We draw the following key observations:

• Adaptive Reflection: The model’s rethinking frequency is positively correlated with problem difficulty. It adaptively triggers reflection more often on harder problems where its initial answer is more likely to be incorrect.
• Effective Rethinking: While the success rate of rethinking is higher for easier problems and lower for harder problems, rethinking is overall beneficial. The model consistently improves its answers across all difficulty levels via rethinking, with very low degradation rates. The degradation rate is low because the model tends to use self-verification for correct answers, and the outcome reward distinguishes the degraded rethinking response from a beneficial rethinking response.

Here we acknowledge the limitation that problem difficulty is bootstrapped from our model's own performance. We will refine this analysis using more objective judgement in our revision. Nonetheless, these statistics provide quantitative support for our claim of emergent metacognitive behavior. The model shows evidence of when to rethink and this reflection leads to consistent, positive performance gains.

2. On the Modality-Agnostic Claim of SSR (Weakness 3, Question 2)

We thank the reviewer for asking for clarification on this point. The "vanishing advantages" problem is general in GRPO when training queries is easy to master, leading to vast majority of training rollouts being zero advantages. This issue is particularly pronounced for vision-language reasoning domains because there are still a lack of publicly available high-quality challenging reasoning-related queries. In the text-only math reasoning domain, the vanishing advantages problem is mitigated by a few high-quality challenging query dataset, for example, DeepMath. The table below shows the performance improvement on GSM8K and Math500 when training with different collection of training queries (easier-to-master Math8K vs. the higher-quality challenging DeepMath). The values shown are absolute pass@1 improvements.

Key Observations: The benefit of SSR is more pronounced on the Math8K dataset. This is because the problems in Math8K are easier to master, making the baseline GRPO more susceptible to saturation. On the more uniformly challenging DeepMath dataset, the baseline is stronger, but SSR still offers a reliable, albeit smaller, improvement.

3. On Error Analysis of Rethinking (Question 3)

This is a crucial analysis to understand whether rethinking improves visual perception or primarily corrects textual-symbolic reasoning. Due to the time-intensive nature of manually inspecting and annotating numerous generation rollouts, we were unable to complete this analysis during the rebuttal phase.

However, we have a clear plan to incorporate this into the final version of our paper. The analysis will involve manually categorizing corrected errors into groups such as:

1. Logic-Specific Corrections: The initial visual perception was correct, but the model corrected a flaw in its mathematical or logical reasoning.

2. Vision-Specific Corrections: The model initially misinterpreted or failed to locate a key visual element, and the reflection process corrected this perceptual error.

3. Combined Corrections: Both the visual interpretation and the subsequent logic were flawed and were corrected.

4. On the Areas of Improvements

• Outcome Reward and Rationale Quality. We thank the reviewer to pointing out this critical point. Since this work is a preminimary study of slow-thinking in VLMs, the paper follows the prevailing setting of using outcome reward. However we agree this will be a important direction for furture work and we will add this point in the limitation and future work section in the revision.

• Downsides of SSR. We agree with the reviewer that discussing the potential downsides of SSR is important. In the meantime, we would like to clarify a detail of our implementation designed to mitigate this risk: SSR prioritizes trajectories based on absolute advantage, not just raw rewards. This prevents the policy from merely overfitting to simple, high-reward samples and instead encourages it to focus on trajectories where the policy's performance significantly exceeded its expected value.

  However, we acknowledge that the full impact of SSR on the policy's long-term exploration remains an open area for investigation. We will add a discussion in the paper to elaborate on this potential limitation and frame it as a direction for future research into balancing exploration and exploitation in VLM-based RL.

We thank the reviewer once again for their valuable and actionable feedback, which has undoubtedly strengthened our paper.

We thank the reviewer for their thoughtful feedback and insightful questions. We are glad to address them below.

1. Regarding the inclusion of "slow-thinking" data in the original Qwen-VL training.

This is an excellent question. Based on our analysis of the base model (Qwen2.5-VL), the model does not intrinsically generate reflective reasoning process (see line 152), which is precisely the motivation for "forced rethinking".

There are several potential reasons for not including "slow-thinking" data in Qwen2.5-VL. First, the release of Qwen2.5-VL is within one month after the release Deepseek-R1, so the training data might not include much slow-thinking data. Besides, Qwen-VL are engineered for comprehensive performance across a wide spectrum of tasks, including many "fast-thinking" perception-focused benchmarks (e.g., OCR, grounding) where slow-thinking is less necessary. Second, creating a well-balanced instruction-tuning dataset that effectively mixes "slow-thinking" data for reasoning with "fast-thinking" data for perception is a significant challenge. In fact, a very recent work, MiMo-VL, highlights this difficulty and proposes a complex four-stage tuning process to empower VLMs with slow-thinking capabilities, underscoring the non-trivial nature of this problem.

2. On the performance gap between VL-Rethinker and OpenAI-o1 on certain benchmarks.

The reviewer correctly observes the performance discrepancy on benchmarks like MMMU-Pro and MEGA-Bench. These benchmarks are exceptionally broad, covering a massive range of multidisciplinary knowledge and long-tail, real-world visual-language tasks. The sheer diversity of domains and instructions in these tests requires vast world knowledge that is not well-represented in publicly available academic datasets.

Proprietary models like OpenAI's GPT-4o are trained on datasets of a scale and quality that are orders of magnitude larger than what is available to the public. This extensive, high-quality private data likely provides them with a decisive advantage on these benchmarks.

3. On the speculative reasons why RL incentivizes reflection differently in multimodal contexts.

Our primary hypothesis, mentioned briefly in the paper (line 152), is that the base VLM generate insufficient reasoning processes that include self-reflection in its native sampling distribution. In more technical terms, during the RL exploration phase, the model's rollouts overwhelmingly consist of direct, concise answers rather than multi-step, reflective thought processes.

This creates an exploration bottleneck. Since the model rarely, if ever, spontaneously produces the "slow-thinking" behaviors we want to reward, the RL algorithm has few opportunities to learn and reinforce these complex reasoning pathways. This contrasts with text-only settings, where the base models have seen slow-thinking data more frequently in pre-training and instruction tuning, and samples these reflective responses during RL more frequently.

4. Regarding evaluation on additional benchmarks.

We appreciate the reviewer suggesting these highly relevant benchmarks. Our selection of the seven benchmarks used in the paper was deliberate, aiming to cover a representative set of reasoning skills. For instance, MathVerse, MathVista, MathVision assess quantitative reasoning skills similar to those in MMStar. Likewise, MathVerse, EMMA, MMMU involves evaluating diagram and chart understanding capabilities that overlap with AI2D and ChartQA.

We acknowledge the value of broadening our evaluation and will certainly consider including these benchmarks in future iterations of our work to provide an even more comprehensive assessment.

We are very grateful to the reviewer for their positive and exceptionally thorough feedback. We are thrilled that they recognized our work as "excellent" and appreciated the novelty of our contributions. Their insightful questions and constructive criticism have been invaluable in helping us identify areas for improvement and strengthen the paper.

Below, we address the weaknesses and questions raised.

1. On the Justification of the SFT Loss

Thank you for raising this important point. We realize that the role of the auxiliary SFT loss was not sufficiently justified in the manuscript.

The SFT loss is applied concurrently with the RL objective (GRPO) during the training phase. Its primary purpose is to accelerate the learning of the "forced rethinking" behavior. While the model could learn to generate the thinking process through the RL objective alone (we tried), we observed that this approach is significantly slower. This is because the trigger tokens for rethinking (e.g., "wait") initially have a very low probability under the policy's distribution. As a result, the discrepancy distribution on the forced tokens is large in the initial training phase and gets clipped. It is until an extended training process that the policy adopts the rethinking tokens. In contrast, the auxiliary SFT loss provides a more direct and stable gradient signal, guiding the policy to adopt the rethinking syntax much more efficiently. We will add a dedicated justification to clarify the motivation for the auxiliary SFT loss in our revision.

2. On the Analysis of Reasoning Outputs

This is an excellent suggestion. We agree that a deeper analysis of the model's reasoning process is crucial to fully demonstrate the benefits of our approach.

Revision Plan: We will add a new analysis section to the Appendix. Due to time constraints during the rebuttal period, we have completed preliminary analyses and will include a comprehensive version in the revision. This new section will include:

1. Rethinking Efficacy: A quantitative breakdown of rethinking success and failure rates across different tasks (which we have already completed a preliminary study as per .Reviewer EyWX's suggestions)
2. Error Analysis: A comparative error analysis of final answers with and without rethinking to pinpoint the types of mistakes our method corrects.
3. Reasoning Length vs. Difficulty: A correlation study between the length of the thinking trace (i.e., number of tokens) and problem difficulty and types, providing insight into how the model dynamically allocates computational effort.

3. On the Impact of the SSR Buffer Size

The reviewer raises a valid concern regarding the SSR buffer size and the risk of learning from stale, off-policy data. A very large buffer can indeed degrade performance by increasing the discrepancy between the behavior policy and the current policy. While a full ablation on buffer size was not feasible during the rebuttal period due to computational limitations, we recognize its importance. We commit to including this ablation and a discussion on it in the final version of the paper.

4. On the Robustness of Hyperparameters ($\alpha$ and Rethinking Fraction)

We appreciate the feedback on Figure 9 and the general question of hyperparameter robustness.

• Alpha ($\alpha$) Sensitivity: The range for $\alpha$ in Figure 9 was selected to demonstrate performance stability across a set of reasonable values. But we realize the current manuscript does not justify this range is reasonable by connecting it to the sharpness of the temperature-annealed softmax distribution. We will refine this in our revision.
• Rethinking Hyperparameters: we will add a brief discussion in the Appendix on the sensitivity to the rethinking fraction and the selection of triggers, providing practical guidance for applying our method to new problem settings.

5. Clarifications and Minor Points

• Figure 2 Training Setting: Thank you for requesting clarification. This figure illustrates a 72B-Instruct model trained with standard GRPO on publicly available datasets. We will update the caption and text to make this clear. The figure's purpose is to motivate our work by showing that even large models can quickly saturate their performance on publicly available VL reasoning queries that are potentially noisy, and easy-to-learn for larger models.
• Minor Weaknesses: We are grateful for these detailed corrections. We will remove the incorrect bolding of MMMU in Table 1, add the missing subscript for $\pi$ on L134, and add direct cross-references to the Appendix sections as suggested.

Thank you once again for your constructive and valuable feedback.

We sincerely thank the reviewer for their patience of reading the paper and their constructive feedback. The suggestions have greatly helped us improve the clarity and impact of our paper. Below, we address each of the questions raised.

1. Definition of 'effective queries' in Figure 2.

We appreciate the reviewer's request for clarification. In the context of Figure 2, "effective queries" refers to queries for which the corresponding training batch contains at least one non-zero advantage value, thus effectively contributing to the gradient update.

More specifically, the GRPO method computes advantages within each query group. A key scenario arises when all responses within a single query group receive an identical reward (e.g., all rewards are 0 or all are 1). In this case, the calculated advantages for all samples in that group become exactly zero. Consequently, these samples do not provide a learning signal (i.e., they contribute zero gradient) during the policy update.

Currently the direct definition of "effective queries" is missing, which can lead to readers' confusions. Since the notion of "effective queries" is indeed equivalent to the number of "effective training samples" (query-responses with non-zero advantages), in our revision, we will use the more unambiguous term "effective training samples" to avoid any confusions.

2. Clarification and Improvement of Figure 6.

We thank the reviewer for this suggestion. The current Figure 6 was generated via a post-hoc analysis: we used a static set of training rollouts and applied three distinct filtering methods (no filtering, zero-advantage filtering, and our SSR) to this fixed dataset to plot the resulting advantage histograms.

We agree that a dynamic plot tracking advantage metrics over the course of training would be far more compelling and would directly substantiate our claim that advantage values diminish rapidly without SSR. In the revised manuscript, we will replace the current figure with a new plot following the reviewer's suggestions.

3. Rationale for Figure 4.

The word cloud in Figure 4 was generated by sampling responses from our trained VL-Rethinker on the validation set and visualizing the most frequent words. We agree with the reviewer's assessment that this figure is primarily illustrative and does not contribute significant technical value to the paper's core claims. To streamline the paper and focus on the most impactful results, we will remove this figure in our revision.

4. The Role of the Additional SFT Loss. The SFT loss is an auxiliary loss that is applied concurrently with the RL objective during the RL training phase. Its purpose is to directly incentivizes the policy to adopt the "forced rethinking" responses. While the model can learn this behavior through the RL objective alone, we found that the learning process is significantly slower. This is because the "wait" token initially has a low probability in the policy's native distribution, causing the policy ratio in the PPO objective to be frequently clipped. The auxiliary SFT loss thus provide more direct gradients that accelerates the adoption of this rethinking token.

5. Typos and Nitpicks

We are grateful to the reviewer for their careful reading and for catching these issues. We will correct them in the revised version.

Dear Reviewer

Could you please check the rebuttal and update the final score ?

Best, AC