Enhancing the Outcome Reward-based RL Training of MLLMs with Self-Consistency Sampling

Thank you for the detailed feedback. We apologize for the places where our expression was unclear and for the notation inconsistencies. Below we (1) give a concise overview of the motivation and rationale of the paper (2) summarize the theoretical guarantee which explains why SCS works, (3) highlight additional experiments that verify our method, (4) provide detailed responses to each of the reviewers’ points.

The overview of the motivation and rationale of the paper

The paper targets a common weakness of outcome reward-based RL training trained on multiple‑choice datasets: models often arrive at the correct option while following wrong reasoning trajectories. Aiming at mitigating this without the heavy computational burden of an external reward model, we propose a lightweight self‑improvement scheme that (i) introduces a sampling strategy and (ii) couples the standard accuracy reward with a self‑consistency reward. Then we prove the effectiveness of our method through mathematical derivations. We prove that models tend to choose reliable reasoning trajectories with SCS methods. Finally, we validate the theory with experiments.

The theoretical derivation and proof of the SCS method

Next, we describe our modeling and derivation procedure. We see the entire reasoning process as a tree structure and make three key assumptions:

1. Uniqueness of the correct trajectory. There is exactly one correct reasoning trajectory in the tree. Justification of the assumption: This is plausible because, for most problems, the solution is unique.
2. Leaf/Choice Alignment. Every trajectory of the reasoning tree ends at one of the answer choices. Justification of the assumption: In the majority of cases, models ultimately outputs a single option as its solution, whether it is correct or not.
3. Relationship between correct/incorrect reasoning trajectories and answer options. If the model follows the correct trajectory, it must arrive at the correct option; if it follows an incorrect trajectory, it may still pick either the correct option or a wrong one. Justification of the assumption: With correct reasoning the model just retrieve the correct answer, whereas an incorrect reasoning chain can lead to multiple outcomes (e.g., guessing when the correct answer is not reachable).

Under these assumptions, we first prove that the traditional reward scheme—which rewards the model solely on whether it selects the correct option—still leaves a portion of probability that the optimized model will follow an incorrect trajectory. Specifically, using the tree structure, we compute the Bayesian probability that the answer is correct while the reasoning trajectory is wrong. Because an error trajectory can still deliver the correct option and thus receive a reward, this probability is non‑negligible.

We then prove that introducing a consistency reward guides the model toward the correct reasoning trajectory. Specifically, we calculate the expected consistency reward on both the correct and incorrect trajectories during optimization and show that the expectation is much higher on the correct trajectory. Since reinforcement learning pushes the model toward higher‑reward trajectories, the optimized model becomes more likely to select the correct reasoning trajectory.

Experimental validation of the effectiveness of our method

Building on the theoretical guarantee, we ran the full protocol on six benchmarks and four RL baselines: Self‑Consistency Sampling consistently lifted answer accuracy by ≈ +7 pp on the weaker baseline and +2 pp on stronger ones. More qualitative analyses and case‑study evaluations show that, beyond boosting scores, our approach genuinely reduces instances of unfaithful reasoning.

Responses for the questions

Q1: The explanation for why SCS can overcome unfaithful reasoning problem.

On the theoretical level, as noted above, we prove that once the consistency reward is applied, the expected reward for the correct reasoning trajectory surpasses that of any incorrect trajectory that still happens to reach the correct option. To be specific, during optimization we evaluate the expected consistency reward for both the correct and the incorrect reasoning trajectories, and demonstrate that the correct trajectory attains a markedly greater expected reward. Consequently, the model becomes more inclined to follow the correct trajectory. At the empirical level, when the model follows an incorrect reasoning trajectory it often resorts to guessing, so the final answer is largely unrelated to that trajectory. As a result, if we truncate the chain and re‑run the reasoning multiple times, the answers show low consistency. By contrast, answers generated along the correct trajectory remain highly consistent. We also introduce visual perturbations, which further decrease consistency on erroneous trajectories, making the model more robust.

Q2: The inconsistent use of notations.

We apologize for the confusing notation. We have now unified all symbols , and rewritten both subsections for clarity. We will update our paper in our final version. The key explanations and corrected definitions are summarized below.

1. Thank you for spotting this inconsistency. Yes $p_{\theta}(\tau \mid x)$ in Line 126 refers to the same autoregressive policy denoted as $\pi_{\theta}$ in Line 104. We will use $\pi_{\theta}$ throughout the revised manuscript and note the change in the notation table.
2. We apologize for the confusion; however, it seems the sentence may have been misunderstood.The original sentence in the paper is " Since current models have sufficient instruction following ability, we assume that the probability of option model achieved is 1 when the reasoning trajectory $\tau = \tau^+$", which illustrates one of our assumptions mentioned above: if the reasoning trajectory is the correct one, it will inevitably arrive at the correct answer option (probability 1). "Option model" is not a phrase.
3. $\tau^+$denotes the correct reasoning trajectory. As we mentioned above, we assume that if the model follows the correct trajectory, it must arrive at the correct option. Our justification is that models can easily retrieve the correct answer when they already have the whole right reasoning trajectory.
4. Equations (5) and (6) are derived directly from Equation(3). Here,$|A|$ denotes the size of the set of distinct answer options reached by the sampled trajectories. For a correct trajectory, our assumption implies that only one option is reachable, so $|A|=1$. For an incorrect trajectory, we prove in the appendix that the expected value satisfies $\mathbb{E}(|\mathcal{A}^-|)>1$. Sorry for an error here—the formula of $R_{con^{+}}$is missing two multiplicative factors. The correct expression should be: $R_{con^{-}} = \frac{1}{N} * (N - 1)* \lambda$.
5. The reward that appears in Equation 1 is the reward used in the original GRPO algorithm. We include Algorithm 1 only as part of preliminaries which introduce our four baseline RL algorithms; it is not part of our proposed method.
6. Sorry for the inconsistency again, we will use $R_{con}$ throughout the revised version.
7. The original sentence in the paper is "The consistency reward $r_{\text{cons}} = \lambda(N - |\mathcal{A}|)$ is computed based on the diversity of the sampled answers— if the answers are highly consistent, indicating stable reasoning given the same prefix, the value of $r_{\text{cons}}$ is high (since $|A|$is low). " It means that when consistency is high, the set A contains few distinct options, thus $|A|$is low.

Q3: Understating baseline RL’s significant performance gains over SFT.

We apologize for the confusing phrasing. Our results suggest that the vanilla RL did not fully unlock the model’s capabilities. The two baselines we tested—REINFORCE++ and RLOO yielded only small gains. We agree that stronger RL variants such as GRPO and REINFORCE++ can achieve more accuracy gain. But SCS still lifts them further (+0.9  and 1.7 pp, respectively), showing that SCS complements even the best‑performing RL protocols.

We will change the analysis in L196 from "Replacing SFT with the baseline RL protocol produces only marginal improvements. " to the analysis above.

We sincerely thank the reviewers for their careful reading and constructive comments, and we apologize for the ambiguities, typos, and notation inconsistencies. We have thoroughly revised the manuscript to clarify our motivation, unify notation, fill in missing definitions, and expand both theoretical and empirical evidence.The experiment results show that our SCS is a robust method, supported by both theoretical derivation and practical experiments. Thanks for your comments again. We sincerely hope that our above response has clearly addressed your questions.

We thank the reviewer for the valuable feedback and the appreciation of our contributions, including the clear analysis of the phenomenon, the lightweight and reproducible method, strong empirical gains, and robust ablation results. Below is our point-by-point response to each of the comments.

W1 & Q2: The quantitative analysis of about improved reasoning reliability with SCS.

Thanks for pointing out this problem. To give concrete proof that our SCS method improves reasoning reliability, we conducted a quantitative analysis. To be specific, for each benchmark we randomly sampled 100 cases that models answered with the correct option before and after training with SCS. Then we manually checked each case whether the model's output is aligned with publicly provided solutions and count times of unfaithful reasoning. Besides, we also asked two strong closed‑source LLMs (OpenAI o3‑mini and Gemini 2.5 Flash) to rate the same traces. The results are shown below:

Table R1. Quantitative analysis of the improvement in reasoning reliability after applying SCS. The numbers in the table represent the occurrences of unfaithful reasoning in 100 correctly answered questions.

The results show a around 15% in faithfulness across all three datasets, demonstrating that our central claim that SCS not only boosts answer accuracy but also produces more reliable reasoning. We will include this table in the revised version of our paper.

W2 & Q1: Absence of multi runs and error bars for reported results.

Thank you for pointing this out—we now include full statistical reporting based on our repeated‑run data. Based on Tables 2 and 3, we carried out three additional runs under the same experimental setup as in the paper; the resulting scores and their 95% confidence intervals are presented in the table below.

Table R2. 95 % confidence intervals for the results of different RL algorithm experiments.

Table R3. 95 % confidence intervals for the results of ablations experiments for two components in SCS (TR = Truncation–Resampling, VP = Visual-Perturbation).

As the table shows, even with only three repeated runs, the confidence intervals remain small, indicating a robust positive effect. We will conduct more runs and update the table in our nexr version.

W3 & Q3: lacking validation of SCS on models with different architectures or scales.

Thank you for pointing out the problem, we have run the complete RLOO + SCS protocol on two additional models with different architectures and scales:

Table R4. The results of our method applied in models of different size and structure. Applying our method, both two models achieve marked performance gains.

For different architecture, we train InternVL3-8B, and SCS still yields a +4.2 points gain. For different model scales, we apply our SCS on Qwen2.5-VL-3B-Instruct, which also shows a 1.6% performance gain. These new results confirm that our SCS is applicable to a variety of model architectures and parameter scales.

W4: The analysis of effect of visual‑perturbation on image semantics.

Thank you for pointing this out. We injected only mild Gaussian noise ($\sigma_{max}=0.3$). To verify the effect on image semantics, we conducted a small human study (5 annotators checking 200 images), both confirming negligible semantic distortion. Furthermore, our case study shows that after applying the visual‑perturbation method, the model not only retains its understanding of the image’s semantics but also gains a stronger capacity for fine‑grained reasoning. We will include these results in the revision.

Q4: The measurement of additional compute costs.

Thank you for bringing up this issue. The extra overhead introduced by SCS method lies almost entirely in the sampling phase Leveraging modern high‑performance inference engines such as vLLM, these process are batched and run in parallel, so the wall‑clock impact grows sub‑linearly with N and remains well‑controlled. Under identical hyperparameters on 8 * A100 GPU (N = 4, truncation ratio=0.8), we observed:

Table R5. Comparison of training time costs after applying SCS.

Thus, with advanced inference backends, the time cost of SCS is both predictable and acceptable given the performance gains it enables.

Q5: Evidence of SCS’s generalization and performance on open-ended reasoning tasks.

We greatly appreciate your feedback. As mentioned above, in open‑ended settings such as fill‑in‑the‑blank or free‑form generation, the output space is so large that brittle shortcut patterns become exceedingly rare. Consequently, the base RLVR algorithm already exhibits strong robustness [1,2,3]. And In Figure 2(a) we have conduct an experiment to verify this. While our SCS reward can still be applied in principle, we expect its incremental benefit to be smaller because agreement among independently sampled long‑form answers is naturally harder to achieve and hard to be verified. Extending SCS to these domains is an interesting direction we leave for future work.

Q6: The inconsistent use of notation.

Thank you for catching this inconsistency. The notation in Figure 5 were legacy placeholders used during early drafting. We will correct the axis titles and the figure caption in the revised version so that all symbols match the definitions in Section 3. Sorry for our mistake, and this does not affect any results or conclusions.

Limitations

Thank you for bringing up this issue, which we are happy to elaborate on. We think that given the performance improvements SCS delivers, its computational overhead remains both foreseeable and well within an acceptable range.

[1] Cohen, J., Rosenfeld, E., & Kolter, Z. (2019, May). Certified adversarial robustness via randomized smoothing. In international conference on machine learning (pp. 1310-1320). PMLR.

[2] Yue, Y., Chen, Z., Lu, R., Zhao, A., Wang, Z., Song, S., & Huang, G. (2025). Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?. arXiv preprint arXiv:2504.13837.

[3] Wan, G., Wu, Y., Chen, J., & Li, S. (2024). Reasoning aware self-consistency: Leveraging reasoning paths for efficient llm sampling. arXiv preprint arXiv:2408.17017.

We thank you for your time and valuable feedback. We’re especially grateful for the recognition of our work’s strengths, including the clear problem identification, strong empirical results across benchmarks, and informative ablation studies. And we took every question seriously and provide full responses to all of them below.

W1: Apparent conflict between the “differentiable” claim and the discrete cardinality formulation of the consistency score

Thank you for catching this issue. What we mean is that the consistency score is utilized as reward, which is used to calculate the advantage and is therefore differentiable during RL training. We agree that calling the score itself “differentiable” could be misleading, and we have rephrased the sentence to “a consistency score as part of reward” which is more accurate.

W2: Unreported hyperparameter ablation for other algorithms

Thanks for pointing this out, now we have conducted some hyperparameter analysis across all RL methods. ( GRPO, REINFORCE ++ and REINFORCE ++-baseline). We apologize that, owing to constraints in time and computational resources, we could not conduct a hyperparameter ablation study as extensive as the one presented in the paper. We will complete these experiments in a future revision. The experiment results are shown in the table:

Table R1. Hyperparameter ablation study for GRPO with SCS.

Table R2. Hyperparameter ablation study for REINFORCE++ with SCS.

Table R3. Hyperparameter ablation study for REINFORCE++–baseline with SCS.

From the tables, we obtained trends that closely mirror those reported for RLOO:

• GRPO + SCS. Fixing the ratio (r=0.2), the performance grows as the increase of the the number of resampled trajectories, and finally reaches the peak at 64.4 (r=0.2,m=8).
• REINFORCE++ + SCS. When fixing the ratio, the same unimodal pattern as GRPO appears. When fixing number of resampled trajectories, the performance metric rises at first and then declines as the ratio increases(61.9 → 62.7 → 61.1 for r=0.2,0.4,0.8).
• REINFORCE++–baseline+ SCS. Accuracy peaks at 63.1 for (r=0.2,m=6). When fixing r=0.2, performance first rises as the number of resampled trajectories grows (63.0 to 63.1 when truncation ratio from 0.2 to 0.4). then slips when the number of resampled trajectories larger (r=0.8,63.2).

Nearly all three algorithms every (r,m) configuration still outperforms its vanilla counterpart by 0.6–2.2 pp, indicating that the consistency reward delivers a uniform benefit on different algorithms and hyperparameter settings.

W3: Lack of verification for assumptions.

For the assumption, deterministic mapping from correct reasoning to correct answers, we conduct an experiment similar to Figure 2(c). First, we manually select 100 cases which are solved with correct reasoning trajectories by Qwen2.5-VL-7B-Instruct for each benchmark. Then we remove the final option answer part (e.g., Answer: A.) for each initial response, and continue to generate from the truncation point. For each case, we do 4 resamples and count for the average number of final options for each question. The results are as follows:

Table R4. The average number of final options for each question in different benchmarks.

The results show that nearly all resamplings finish with the exact correct answer, illustrating that when the model follows the correct trajectory, it almost certainly arrives at the correct option. We think this experiment can justify our assumption of the deterministic mapping from correct reasoning to correct answers.

For the claimed differentiability of the consistency score, we have answered in W1, please refer to it.

W4: Comments Suggestions And Typos

Thank you for catching these issues. We will perform a thorough proof‑reading and correct all typos and formatting errors in the revised version.

We sincerely thank you for your thoughtful feedback. We appreciate the recognition of our work’s strengths, including the motivation, the simplicity and generality of our method, the adequacy of the experiments and the clarity of the writing. Besides, we have carefully replied your questions and concerns below.

W1: Separating Established Techniques from Our Novel Contributions

Thank you for the opportunity to clarify. Below we separate what we adopt, what we adapt, and what is new in our method:

1. Adopted ideas (directly reused)

   • Self‑consistency sampling — drawing multiple reasoning trajectories and comparing their answers, as explored by Wang [1], Wang [2], and Xu[3].
   • Perturbation for robustness — adding Gaussian noise to images to test semantic stability [4,5].

2. Adapted ideas (repurposed for RL)

   • Truncation–resampling: originally a few‑shot prompting trick, we reinterpret it for RL by freezing the high‑confidence prefix of a trajectory and resampling only the tail, turning it into a low‑variance exploration scheme.
   • Visual perturbation: instead of data augmentation, we use perturbations to create paired trajectories whose agreement becomes a training signal, not merely an evaluation metric.

3. New contribution (unique to this work)

   • We introduce a self‑consistency reward: a differentiable signal that measures agreement across perturbed, truncation‑resampled trajectories and feeds this value directly into policy gradients. This shifts consistency from post‑hoc voting to an intrinsic learning objective, enabling the policy to shape its own trajectory distribution for greater robustness—an approach that, to our knowledge, has not appeared in outcome‑reward RL literature.

We will revise the paper to make these three layers—adopt, adapt, innovate—explicit and add the missing citations noted above.

W2: Consistency rewarding may drive the model toward coherent but factually wrong reasoning

Thank you for raising this important point. As mentioned in our paper, we address this concern in two ways that together prevent the models drifting toward “consistent but wrong” solutions:

1. First, acc reward and consistency reward are both applied. Thus the accuracy component continually lead the policy to externally correct behavior, while the consistency term lead toward reliables strategies instead of replacing the ground truth.
2. Second, we apply a small coefficient c=0.4 for consistency reward, ensuring that the model cannot “hack” its way to high reward by neglecting real task performance.

W3: The systematic analysis of about improved reasoning reliability with SCS.

Thanks for pointing out this problem. To give concrete proof that our SCS method improves reasoning reliability, we conducted a quantitative analysis. To be specific, for each benchmark we randomly sampled 100 cases that models answered with the correct option before and after training with SCS. Then we manually checked each case whether the model's output is aligned with publicly provided solutions and count times of unfaithful reasoning. Besides, we also asked two strong closed‑source LLMs (OpenAI o3‑mini and Gemini 2.5 Flash) to rate the same traces. The results are shown below:

Table R1. Quantitative analysis of the improvement in reasoning reliability after applying SCS. The numbers in the table represent the occurrences of unfaithful reasoning in 100 correctly answered questions.

The results show a around 15% in faithfulness across all three datasets, demonstrating that our central claim that SCS not only boosts answer accuracy but also produces more reliable reasoning. We will include this table in the revised version of our paper.

W4: Lack of Confidence Intervals in Reported Scores

Thank you for pointing this out—we now include full statistical reporting based on our repeated‑run data. Based on Tables 2 and 3, we carried out three additional runs under the same experimental setup as in the paper; the resulting scores and their 95% confidence intervals are presented in the table below.

Table R2. 95 % confidence intervals for the results of different RL algorithm experiments.

Table R3. 95 % confidence intervals for the results of ablations experiments for two components in SCS (TR = Truncation–Resampling, VP = Visual-Perturbation).

As the table shows, even with only three repeated runs, the confidence intervals remain small, indicating a robust positive effect. We will conduct more runs and update the table in our next version.

Q1: Settings confined to multimodal multiple‑choice tasks limits evidenced generality

Thank you for your thoughtful question. Our goal is to curb unfaithful reasoning. In practice, we observe this failure mode most frequently in multiple‑choice settings, where the models can exploit option‑specific answers. In more open‑ended formats—fill‑in‑the‑blank or free‑form generation—the space of outputs is much larger and such shortcut patterns appear far less often, so the original RLVR algorithm alone already behaves robustly [3,7,8]. And In Figure 2(a) we have conducted an experiment to verify this. For that reason, we concentrated our methodological contribution on multiple‑choice tasks, where it delivers the most benefit.

At the same time, we think that the multi-choice paradigm is not that narrow: it takes the a large part of RL datasets such as M3cot, ScienceQA, comt and so on. These cover natural images, diagrams, videos, and so on; domains range from science to daily-life questions. Therefore, although our experiments are focused on multi-choice questions, the application scope spans a wide variety of real‑world reasoning scenarios.

Q2: Limited evidence on how SCS candles open‑ended scenarios.

We greatly appreciate your feedback. As mentioned above, in open‑ended settings such as fill‑in‑the‑blank or free‑form generation, the output space is so large that brittle shortcut patterns become exceedingly rare. Consequently, the base RLVR algorithm already exhibits strong robustness [3,6]. While our SCS reward can still be applied in principle, we expect its incremental benefit to be smaller because agreement among independently sampled long‑form answers is naturally harder to achieve and hard to be verified. Extending SCS to these domains is an interesting direction we leave for future work.

[1] Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., ... & Zhou, D. (2022). Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171.

[2] Wang, H., Prasad, A., Stengel-Eskin, E., & Bansal, M. (2024). Soft self-consistency improves language model agents. arXiv preprint arXiv:2402.13212.

[3] Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., ... & Zhou, D. (2022). Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171.

[4] Cohen, J., Rosenfeld, E., & Kolter, Z. (2019, May). Certified adversarial robustness via randomized smoothing. In international conference on machine learning (pp. 1310-1320). PMLR.

[5] Hendrycks, D., & Dietterich, T. (2019). Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12261.

[6] Yue, Y., Chen, Z., Lu, R., Zhao, A., Wang, Z., Song, S., & Huang, G. (2025). Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?. arXiv preprint arXiv:2504.13837.

Thank you again for your valuable feedback. We have responded to your questions in the rebuttal. We hope our response addresses your concerns. If you have any further questions or require clarification, we would be happy to provide additional information.

Thank you for the feedback and your appreciation for our motivation and our Self-Consistency Sampling method. Additionally, you raised several important concerns and questions, which we address in detail below.

Q1: The percentage of unfaithful reasoning phenomenon.

Thanks for your question. For the same benchmarks we tested in the paper, we manually check 100 items which are correctly answered by Qwen2.5-VL-7B and count the number of times this situation occurs. We have illustrated the results in Appendix, and we put it again below:

Table R1. Statistics of unfaithful reasoning across different benchmarks. The table show the occurrences of unfaithful reasoning in 100 correctly answered questions.

Comprehensively analyzing the table and Figure 2c, We can conclude that the unfaithfull reasoning phenomenon is universal, and it is notable in some benchmarks (64% unfaithfull reasoning rate for mathvision), which further confirms our motivation.

Q2 & W1: The performance gains from SCS are not consistent across different RL algorithms.

Thanks for your very insightful question. The effectiveness of SCS appears to be limited to a specific algorithm, and one of the reasons is that SCS works by introducing resampling and perturbation which reduce the variance of gradient estimation. RLOO, which uses a leave-one-out advantage, tends to suffer from higher variance, so the variance reduction brought by SCS yields more significant gains. In contrast, methods like GRPO and REINFORCE++ already incorporate normalization that reduces variance, leaving limited room for further improvement by SCS. To be specific, we calculate the variance of advantage. Let the episodic rewards $R_1, \ldots, R_N$ with mean $\mu$ and variance $\sigma_R^2$. For RLOO, the advantage is
$A_{\mathrm{RLOO}}^{(i)}=R^{(i)}-\frac{1}{N-1} \sum_{j \neq i} R^{(j)}$ Because $R_i$ is not included in the leave‑one‑out baseline, the two terms are independent. Hence, $\operatorname{Var}\left[A_{\mathrm{RLOO}}\right]=\sigma_R^2\left(1+\frac{1}{N-1}\right)$ For GRPO and REINFORCE++, since they have already applied normalization, the variance is small. Because RLOO inherently has high variance, the variance‑reduction effect of SCS is significant, yielding a clearer improvement in model performance. In contrast, the other two algorithms already exhibit low variance, so the benefit is much less noticeable.

W2: The advantage of truncation-resampling over total-resampling.

Thank you for pointing out this important detail. Truncation‑resampling strikes a compute‑efficient balance: instead of regenerating full trajectories like total‑resampling, we resample only the uncertain tail, cutting sampling time and GPU memory. What' more, this efficiency does not come at the expense of quality—our ablation in Figure 5 shows that when we change the truncation ratio r, the best performance occurred at r=0.8, higher than r=0.2 (nearly total‑resampling). In a word, our SCS both saves resources and improves performance.

Q3& W2: Explanations of the principles and necessity of visual‑perturbation (VP).

Thank you for your thoughtful question. For the explanation of visual‑perturbation, we think it can be seen as a part of resampling. These perturbed inputs further de‑correlate the individual rollouts, so the results can aggregate more independent samples rather than near‑duplicates. We still think VP is necessary and effective. First, Table 3 already confirms that both truncation‑resampling (TR) and visual‑perturbation (VP) are individually beneficial —raising accuracy from 57.8 % to 63.0 % and 62.8 %, and their combination yields an additional boost beyond either component alone, reaching 65.5 %. Furthermore, our case studiy shows that after applying the visual‑perturbation method, the model not only retains its understanding of the image’s semantics but also gains a stronger capacity for fine‑grained reasoning. So using them together can bring further performance gains.

W3: Additional computation costs caused by SCS.

Thank you for bringing up this issue. The extra overhead introduced by SCS method lies almost entirely in the sampling phase Leveraging modern high‑performance inference engines such as vLLM, these process are batched and run in parallel, so the wall‑clock impact grows sub‑linearly with N and remains well‑controlled. Under identical hyperparameters on 8 * A100 GPU (N = 4, truncation ratio=0.8), we observed:

Table R2. Comparison of training time costs after applying SCS.

Thus, with advanced inference backends, the time cost of SCS is both predictable and acceptable given the performance gains it enables.

Limitations:

As for limitations, our SCS has not been extensively applied to LLMs and more MLLMs to verify its generality. The SCS method is primarily suited for multiple-choice style training datasets and does not adapt well to open-ended or fill-in-the-blank questions. Therefore, it is currently mainly applicable to outcome-based RL algorithms. Additionally, the method should be tested on more datasets and benchmarks.

Thank you for providing thoughtful responses to my questions and comments. The responses helped me understand the paper better. Here are my thoughts on the author responses:

• It is interesting that there are 64 unfaithful reasoning processes in 100 correctly answers questions in MathVision dataset. This observation may have led the method of visual-perturbation.
• It is unfortunate that Self-Consistency Sampling (SCS) is effective only for RL algorithms with high variance such as RLOO.
• Both truncation-resampling and visual-perturbation appear to be effective. However, it is unfortunate that the effect is not amplified when used together.

After reviewing the author responses, I maintain my initial score.