Point-RFT: Improving Multimodal Reasoning with Visually Grounded Reinforcement Finetuning

Thank you for your valuable advice and comments!

Comparison with Prior SOTA

We have added comparisons with LLaMA Vision 3.2 and Molmo. Specifically, we also trained our Point-RFT using LLaMA Vision 3.2 as the backbone.

The experiments demonstrate the effectiveness of our method across different models. In the revised version, we will add more model comparisons and qualitative discussions.

General Image QA Understanding Tasks

We additionally introduced the MMVet dataset as a benchmark, as it is a standardized task suite covering a wide range of multimodal reasoning tasks, further validating our method’s generalization to general images.

The results show that our method significantly improves performance across various scenarios, strongly supporting our findings. Although we are unable to include images, we observed that pointing-based thought generalizes natural images well. We will include more examples in the revision.

Out-of-Domain Clarification

Sorry for the confusion. Some of the evaluation data categorized as out-of-domain (OOD) also appear in the SFT data. This is because our definition of OOD is based on whether the data is used in the RL phase. Since directly applying reinforcement learning to vision-language models often leads to performance degradation on out-of-domain tasks, using RL data as the criterion allows us to better assess whether our method maintains robust performance beyond the scope of RL-trained data (e.g., ChartQA). We include a broader mix of data types in the SFT stage to train a more generalizable capability, thereby enhancing the model’s usability in real-world scenarios. We will clarify this more explicitly in the revised version.

We appreciate your constructive advice!

New Insights or Innovations

As stated in L61, our core contribution lies in being the first to propose a multimodal language model (VLM) that is outcome reward trained through visually grounded thought. We demonstrate the advantages of visually grounded chain-of-thought (CoT) reasoning, including:

• Better scalability compared to pure text-based reasoning in L, shown in Tables 2, 3, and 5;
• Improved generalization ability, shown in Tables 2 and 3;
• Better interoperability, shown in Section 4.4.3.

The datasets and engineering implementations are means to achieve these goals, but they are not the core of our aforementioned conceptual contributions.

The core finding in our paper is that, unlike in the text domain, after RL fine-tuning on specific tasks, VLM models exhibit significant in-domain performance gains but often suffer large performance drops on unseen out-of-domain (OOD) tasks, indicating poor generalization. Point-RFT demonstrates that the RL process for VLMs must effectively align reasoning with visual evidence to achieve OOD capabilities. To this end, we train the model to generate “point-and-think” trajectories via SFT and then apply RL, thereby enabling chain-of-thought generalization in RL comparable to LLMs in pure text domains.

As a result, we not only propose an effective dataset and model but also introduce a VLM RL paradigm with cross-task generalization. The real-world examples presented in Section 4.4.3 validate that our method successfully transfers in-domain reasoning ability to real-world domains, effectively addressing the generalization problem of RL in VLMs.

Additional Benchmarks on Different Domains

IconQA and PlotQA are related to abstract images and scientific documents instead of standard chart tasks, which demonstrates generalization to document-like content and abstract visual inputs.

We additionally introduced the MMVet dataset as a benchmark, as it is a standardized task suite covering a wide range of multimodal reasoning tasks, further validating our method’s generalization to natural images.

The results show that our method significantly improves performance across various scenarios, strongly supporting our findings. Although we are unable to include images, we observed that pointing-based thought generalizes natural images well. We will include more examples in the revision.

Clarification on Improvement Margins

The difference in average scores between the two models (42.77 vs. 35.94) is primarily driven by the large performance gap on the counting task (68 vs. 21). The average scores on the other datasets are (36.46 to 38.90) and (39.68 to 46.80), respectively, and show consistent performance.

So, why does the counting task exhibit such an outlier value? This is because Qwen2-VL-7B tends to give direct and correct answers, while Qwen2.5-VL often generates chain-of-thought (CoT) responses that introduce hallucinations. Since the counting task is highly sensitive to hallucinations, Qwen2.5-VL performs worse. However, after applying Point-RFT, Qwen2.5-VL-7B shows reduced hallucinations and its stronger reasoning abilities become more effective, resulting in greater performance gains.

This also demonstrates the advantage of our Point-RFT: through grounded CoT, our method reduces hallucinations and improves reasoning ability during RL.

Clarification on OOD Evaluation

Sorry for the confusion. Some of the evaluation data categorized as out-of-domain (OOD) also appear in the SFT data. This is because our definition of OOD is based on whether the data is used in the RL phase. Since directly applying reinforcement learning to vision-language models often leads to performance degradation on out-of-domain tasks, using RL data as the criterion allows us to better assess whether our method maintains robust performance beyond the scope of RL-trained data (e.g., ChartQA). We include a broader mix of data types in the SFT stage to train a more generalizable capability, thereby enhancing the model’s usability in real-world scenarios. We will clarify this more explicitly in the revised version.

Evaluation on Other RL Algorithms

We have tried the PPO algorithm on Qwen2.5-VL-7B with Point-RFT.

We can find that PPO obtained similar results, showing that the key factor in achieving strong model performance is not the specific RL algorithm itself, but rather our design of the “point-and-think” trajectory and the corresponding outcome reward.

Thanks for your comments!

Lower Evaluation Scores

This is because, to maintain consistency across all comparisons, we followed the reasoning output setting (i.e., CoT) as described in Lines 202–204 of the main text, which is also the standard setup in prior RFT studies (refer to Section B of supplementary material):

This drop, however, stems primarily from the model’s in-domain short-answer post-training, not from an intrinsic weakness of long-form reasoning—a phenomenon echoed in other RFT studies (Deng et al., 2025; Wang et al., 2025a; Huang et al., 2025; Meng et al., 2025; Peng et al., 2025; Chen et al., 2025; Zheng et al., 2025).

We applied the same evaluation standard to all models to ensure a fair comparison (the original score can be found in the next response). Further explanations and comparisons without CoT can be found in Section B of supplementary material.

MMVet and Other Benchmarks

We choose to report on MMVet because it covers a wider range of question categories and is generally representative of other commonly used benchmarks. Following your suggestion, we have expanded the comparison to include MMVet, MMVista, and MMBench under their original benchmark settings, and found consistent trends across these datasets. Moreover, the results on MMVet particularly highlight our model’s strong ability to transfer reasoning skills from document images to natural images.

Positive Effect Beyond Specific Scenarios

As demonstrated in Table 2 of the main text, Point-RFT brings substantial improvements across all tasks, not only in counting-related scenarios. Notably, most tasks are unrelated to counting, and the gains are in some cases even more significant on non-counting tasks, further underscoring the generality of our approach. Unless the task is purely textual, multimodal tasks inherently involve perception. In such cases, the CoT generated by Point-RFT consistently improves performance and is not limited to specific scenarios. Our sub-category results on MMVet further support this conclusion.

Looking forward to further discussing with you!

We are truly delighted to hear that all your concerns have been resolved!

In summary, we have discussed the following original concerns and resolved them as follows:

• Novelty: Clarified that our innovation lies in being the first to combine visually grounded chain-of-thought (CoT) reasoning with reinforcement learning to train VLMs, which significantly enhances cross-task generalization.
• Generalization: Broadened evaluation to MMVet, MMVista, and MMBench, confirming robustness on natural images and varied question types.
• Performance Differences: Showed that Point-RFT dampens CoT-induced hallucinations, with pronounced gains on counting tasks.
• OOD Definition: Defined out-of-distribution as data unseen during the RL phase, ensuring fair generalization tests.
• RL Algorithms: Revealed that improvements stem from the “point-and-think” trajectory design, not the GRPO or PPO.
• Task Specificity: Showed that Point-RFT improves performance across a wide range of tasks and is not limited to any particular scenario.
• Benchmark Scores: Clarified that lower absolute scores were explained by the adoption of unified CoT-based scoring. Detailed comparisons are provided in Suppl. Section B.

We would greatly appreciate it if you could consider raising our score, as all concerns have now been resolved. Should you have any further concerns or questions, we would be more than happy to discuss with you.

Thank you for your valuable feedback and suggestions!

Improvements by Format Training

This is because even in out-of-domain scenarios, using Point-SFT alone can significantly reduce perception and alignment errors, thereby lowering the overall error rate. However, standalone SFT remains constrained by data quality. In contrast, our Point-RFT method based on outcome reward modeling further enhances the quality of visual grounding, leading to improved performance.

To further explore why format training leads to such improvements, we conducted additional experiments where volunteers manually categorized errors on 200 samples that were predicted incorrectly by the base model.

We simply categorized the errors into four types: perception errors, reasoning errors, alignment errors, and other errors. Perception errors represent mistakes in visual perception, such as failing to correctly identify objects or details in an image. Reasoning errors refer to mistakes in logical reasoning or understanding of the text. Alignment errors indicate incorrect referencing of visual information that has been correctly perceived during the reasoning process. Other errors encompass various other types of mistakes.

We found that through format training, the model generates "point-and-think" trajectories during reasoning, significantly improving the alignment between visual information and text reasoning. This leads to a substantial reduction in perception errors and alignment errors, thereby decreasing the overall error rate. This also explains why base-RFT without format training is ineffective.

Results on Standard VQA Tasks

We additionally introduced the MMVet dataset as a benchmark, as it is a standardized task suite covering a wide range of multimodal reasoning tasks, further validating our method’s generalization to natural images.

The results show that our method significantly improves performance across various scenarios, strongly supporting our findings. Although we are unable to include images, we observed that pointing-based thought generalizes natural images well. We will include more examples in the revision.

Thanks for your invaluable advice!

Contribution and Novelty Beyond New Dataset

As stated in L61, our core contribution lies in being the first to propose a multimodal language model (VLM) that is outcome reward trained through visually grounded thought. We demonstrate the advantages of visually grounded chain-of-thought (CoT) reasoning, including:

• Better scalability compared to pure text-based reasoning in L, shown in Tables 2, 3, and 5;
• Improved generalization ability, shown in Tables 2 and 3;
• Better interoperability, shown in Section 4.4.3.

The datasets and engineering implementations are means to achieve these goals, but they are not the core of our aforementioned conceptual contributions.

The core finding in our paper is that, unlike in the text domain, after RL fine-tuning on specific tasks, VLM models exhibit significant in-domain performance gains but often suffer large performance drops on unseen out-of-domain (OOD) tasks, indicating poor generalization. Point-RFT demonstrates that the RL process for VLMs must effectively align reasoning with visual evidence to achieve OOD capabilities. To this end, we train the model to generate “point-and-think” trajectories via SFT and then apply RL, thereby enabling chain-of-thought generalization in RL comparable to LLMs in pure text domains.

As a result, we not only propose an effective dataset and model but also introduce a VLM RL paradigm with cross-task generalization. The real-world examples presented in Section 4.4.3 validate that our method successfully transfers in-domain reasoning ability to real-world domains, effectively addressing the generalization problem of RL in VLMs.

Difference with Visual-CoT and Sketchpad

Our approach relies on outcome incentivization rather than process design or supervision. This enables broader generalization to diverse tasks, beyond problems with specific handcrafted designs. Specifically, Visual-CoT is limited to one bounding box detected in a narrow domain, and Sketchpad requires external tool, making them difficult to generalize.

Diverse Benchmarks on Different Domains

IconQA and PlotQA are related to abstract images and scientific documents instead of standard chart tasks, which demonstrates generalization to document-like content and abstract visual inputs.

We additionally introduced the MMVet dataset as a benchmark, as it is a standardized task suite covering a wide range of multimodal reasoning tasks, further validating our method’s generalization to natural images.

The results show that our method significantly improves performance across various scenarios, strongly supporting our findings. Although we are unable to include images, we observed that pointing-based thought generalizes natural images well. We will include more examples in the revision.

Evaluation on Other Model Families

We choose Qwen2.5-VL as it is a widely used practice for VLM learning. To further validate the effectiveness of our method, we also conducted experiments on LLaMA 3.2 Vision, which also has different model scales.

The results demonstrate that our method adapts well to different architectures.