Unveiling the Compositional Ability Gap in Vision-Language Reasoning Model

We thank the reviewer for the constructive feedbacks, and answer the questions raised as follows:

Q1: Is RL-Ground overly tailored to synthetic tasks? How scalable is progress reward, and how robust is caption-before-thinking to errors?

We thank the reviewer for raising these important concerns. While our benchmark is synthetic for controllability and diagnosis, the underlying challenges—visual grounding, multi-hop reasoning, and compositional capability—reflect real-world VQA situations. RL-Ground is not tightly coupled to our benchmark; instead, it proposes two broadly applicable directions:

1. Enforcing explicit visual grounding (<caption>) before reasoning to trigger reasoning with text, and
2. Rewarding intermediate progress to encourage visual-grounded reasoning.

Regarding the generalizability of progress reward:
We acknowledge that defining sub-goals in open-domain tasks can be challenging. However, many real-world VQA tasks contain implicit intermediate signals (e.g., object detection, caption generation, OCR) that could be repurposed into reward shaping. We envision future work leveraging pretrained vision models (e.g., SAM, GroundingDINO, BLIP-2, YOLO) to estimate such sub-goals in-the-wild as the progress reward.

On the robustness of the <caption> step:
We appreciate the concern regarding potential brittleness of the <caption> step. To better understand how RL-Ground contributes to compositional generalization, we introduce two supporting grounding tasks that measure the model’s ability to extract critical visual information:

• Shape Area Grounding: asks for the area of a specified shape
• Grid Position Grounding: requires locating a shape on the grid using (row, column) indices

These tasks serve as diagnostic tools for analyzing how well the model grounds visual entities before reasoning over them.

Table: Evaluation of VLMs across tasks and their associated visual grounding metrics

As shown in the above table, RL-Ground achieves high grounding scores (Shape Area Grounding: 96.2%, Grid Position Grounding: 88.6%) compared to all other post-training strategies, along with the best performance on the compositional task (52.8%). This confirms that the <caption> step, combined with step-wise reward, enables the model to better structure and retain relevant visual cues, thereby enhancing multi-step compositional reasoning. In contrast to the concern that caption errors could cascade, our results indicate that RL-Ground actually reduces such failure cases by explicitly guiding the grounding process before reasoning begins.

To further mitigate possible risks from incorrect or misleading captions, we identify one promising future direction: introducing a stricter supervision mechanism that treats the caption as a self-contained summary and only assigns reward if the downstream answer can be derived from the caption alone, without using the image. Such an extension would encourage more semantically complete and robust visual summarization. Nonetheless, we emphasize that RL-Ground is not proposed as a final solution, but rather as a light-weight and effective direction that highlights how enhancing atomic visual alignment and grounding can unlock compositional capabilities in VLMs.

Q2: Does the compositional gap and RL/SFT difference generalize beyond Qwen?

We believe the observed compositional gap and the relative robustness of RL over SFT are not specific to Qwen, but indicative of broader challenges in current VLM post-training.
We plan to validate our findings across architectures such as Gemma, InternVL, and Phi-4, as future work.
To clarify our current scope, we will more explicitly acknowledge the architectural limitation in the paper’s Limitations section.

Q3: Is RL-Ground’s improvement specific to synthetic language patterns? Can it generalize to real-world VQA?

To assess the generalization beyond our benchmark, we conducted a new zero-shot evaluation on the GeoQA dataset (754 test samples), a realistic, natural-language VQA benchmark, using the checkpoint trained with our synthetic dataset and the original Qwen2.5-VL-3B-Instruct base. We evaluated:

• Qwen2.5-VL-3B-Instruct: 14.8% accuracy
• SFT: 9.2% accuracy
• RL: 17.8% accuracy
• RL-Ground: 21.2% accuracy

These results confirm that RL-Ground improves performance on naturalistic questions despite being trained only on synthetic data. This highlights that the proposed strategies—captioning and progress reward—enhance reasoning robustness and are not limited to benchmark-specific language patterns.

We will include this experiment and analysis in the revised version to reinforce the practical relevance of RL-Ground beyond ComPABench.

The rebuttal has significantly strengthened the paper's claims and overall contribution. Consequently, I will be raising my score.

We thank the reviewer for the constructive feedback. Below we address each concern in turn.

Weakness 1: Limited model scale and architecture diversity

We acknowledge this limitation and will explicitly state it in the Limitation section. Due to compute constraints, our experiments focus on Qwen models (3B and 7B). Evaluating compositional generalization across other model families such as LLaMA, Gemma, and even closed-source models like GPT-4o is an important direction, and we plan to include these in future work using the same benchmark setup.

Weakness 2: Organization and motivation of RL-Ground

We appreciate the suggestion regarding paper organization. Our current structure reflects the probing nature of this work. Our primary goal is to reveal and analyze compositional gaps in MLLMs, a dimension that has received limited attention in prior work. We do not position RL-Ground as a novel algorithmic contribution, but rather as a practical guidepost that highlights how compositional ability may emerge from vision-to-text conversion and visually grounded intermediate reasoning.

We agree that clearer separation of contributions and evaluation will improve readability. In the revised version, we will move benchmark details into a standalone “Benchmark” section and place result analysis, including RL-Ground, into a separate “Results and Findings” section.

Weakness 3: RL-Ground novelty, RQ1 impact, and experimental details

We thank the reviewer for this feedback. We emphasize that our goal is not to propose a new algorithm, but rather to highlight two practical directions that improve compositional generalization in RL training:
(i) explicit conversion from vision to text to trigger subsequent reasoning with text, and
(ii) incorporating progress-based intermediate rewards to ground visual perception tasks.

Regarding its effectiveness on RQ1 (Cross-Modality Compositional Generalization), we note that RQ1 uses models trained on pure-text corpora, while RL-Ground is a post-training scheme applied to multimodal models. However, to probe the potential transfer of its design principles, we also experimented with applying the caption-before-think prompting pattern at inference time during RQ1 evaluation. As shown below, we observe consistent improvements across both tasks:

These results suggest that the structured prompting design in RL-Ground may also benefit models evaluated on RQ1, even when trained in purely textual settings.

As for implementation details, RL-Ground uses the same hyperparameter setup as the RL baseline, with two key changes:

1. A modified format reward that only gives credit when outputs follow the full <caption>...</caption><think>...</think><answer>...</answer> structure.
2. An additional progress reward for intermediate reasoning steps, increasing the maximum possible reward from 2 to 3.

We will clarify these implementation specifics in the revised version.

Thank you again for your thoughtful comments and for recognizing the value of our work on diagnosing compositional generalization in MLLMs. We especially appreciate your suggestion to evaluate GPT-4o, and have conducted experiments accordingly.

Below, we compare GPT-4o with Qwen2.5-VL-7B-Instruct (which has been evaluated in the paper) on our benchmark:

We would like to emphasize that the focus of our paper is not on evaluating the inherent compositional capability of different models, but rather on probing how different post-training strategies influence compositional generalization. Our experimental design centers on this goal, including the development of a synthetic benchmark that enables controlled training and evaluation of such effects.

In this context, Qwen2.5-VL-7B-Instruct serves as a zero-shot reference model without any fine-tuning or post-training on our synthetic datasets. It is used only to illustrate the general trend of performance drop on multimodal compositional tasks when no task-specific post-training is applied.

The results from GPT-4o, while clearly stronger in absolute performance, still show a significant gap between unimodal and multimodal compositional tasks. This supports our central claim: cross-modal compositional generalization remains an open challenge, even for top-performing proprietary models.

We would also like to highlight additional insights discussed in our rebuttal to other reviewers, where we conduct fine-grained error analysis on atomic grounding tasks such as area calculation and grid localization for a single shape. These analyses demonstrate that:

• SFT struggles to improve grounding capabilities,
• RL helps models learn grounding behavior,
• And our proposed RL-Ground further enhances visual grounding and thus improves compositional generalization.

To ensure that our findings generalize beyond synthetic setups, we additionally evaluate our approach on a realistic benchmark (GeoQA), further reinforcing the practical value of our conclusions.

We hope these additional experiments and clarifications demonstrate the strength of our contribution. We kindly invite you to reconsider your assessment in light of these updates.

Thank you once again for your constructive and encouraging feedback.

We thank the reviewer for their constructive feedback and thoughtful suggestions. Below, we provide responses to the key concerns.

Weakness 1–2: Lack of qualitative error analysis and the low SFT performance on the compositional task

We thank the reviewer for these important observations. To better understand the behaviors of different post-training strategies, we introduce and analyze two auxiliary tasks:

• Shape Area Grounding evaluates whether the model can correctly localize a specific shape and extract its relevant geometric attributes (e.g., base, height) to compute its area. For example, given a prompt like "What is the area of the purple trapezoid?", the task assesses whether the model correctly identifies the purple trapezoid in the image and performs the area computation accordingly.

• Grid Position Grounding measures the model’s ability to spatially localize a target shape on a 2D grid. For instance, given the question "What is the position of the orange right_triangle?", the model is expected to identify the correct grid coordinates of the shape in the image.

These grounding evaluations help us identify where failures occur and how they correlate with compositional generalization.

Table: Evaluation of VLMs across tasks and their associated visual grounding metrics

SFT overfits to training patterns and fails to adapt to new task prompts

As shown in the table above, SFT achieves high accuracy on individual reasoning tasks such as Shape Area (73.6%) and Grid Position (91.8%). However, it shows extremely lower grounding scores (1.2% for Shape Area), and its compositional accuracy drops to just 7.2%. This suggests that SFT learns to associate surface-level prompt features with memorized reasoning templates seen during training, rather than learning to flexibly combine reasoning skills.

Example failure case

Question: There are 3 shapes on a 6x6 grid. Find the shape that is nearest to the red trapezoid in terms of Manhattan distance, what is the total area of the nearest shape to the target shape and the target shape?
SFT Output:
<think>Target shape: red trapezoid at (2, 4)
Distance from target shape to blue square at (4, 1): 5
Distance from target shape to green square at (1, 2): 3
Closest shape is the green square at (1, 2) with distance 3
Total area of closest shape and target shape: 3 + 8 = 11</think>
<answer>11</answer>
Ground Truth: 105

In this case, SFT correctly identifies the nearest shape (e.g., the green square) but hallucinates the areas of both the target and the nearest shape, outputting an incorrect total without grounding either value in the actual image. This illustrates that the model overfits to previously seen single-task prompt templates, failing to adapt when the task requires integrating multiple skills or resolving an unfamiliar instruction composition.

RL improves grounding and partial composition

Pure RL shows consistently high grounding accuracy (96.6% for Shape Area, 74.8% for Grid Position), enabling better generalization on the compositional task (31.2%). This indicates that RL encourages more faithful visual attention and reasoning behavior. However, the performance gap shows that RL alone is insufficient for strong compositionality without further inductive guidance.

SFT-init-RL underperforms due to inherited SFT biases

Although SFT-init-RL performs well on atomic tasks, its grounding scores remain low (e.g., 1.2% for Shape Area Grounding), and its compositional accuracy collapses to 1.0%. This suggests that starting RL training from an SFT checkpoint may preserve the shallow reasoning habits learned during SFT, making it harder for RL to correct them later.

RL-Ground achieves robust and generalizable behavior

By enforcing a <caption> step and applying step-wise rewards to guide intermediate progress, RL-Ground attains high grounding accuracy (96.2% and 88.6%) and the best overall compositional performance (52.8%). These results offer deeper insight: RL-Ground enhances compositional ability by strengthening the model’s atomic visual grounding capability.

Weakness 3: Artificial benchmark design and lack of real-world tasks

We appreciate the reviewer’s concern. The synthetic nature of ComPABench is intentional and serves a clear diagnostic purpose: it allows us to precisely control task structures, systematically vary compositional difficulty, and isolate failure modes such as visual grounding, spatial localization, and multi-hop reasoning. While not a replacement for open-domain evaluation, ComPABench complements existing real-world benchmarks by providing a controlled lens to probe compositional reasoning—an ability often underrepresented in current VLM benchmarks.

To assess the generalization of our findings beyond synthetic data, we conducted a zero-shot evaluation on GeoQA, a more natural visual QA benchmark consisting of 754 test samples. Despite being trained only on synthetic data, RL-Ground significantly improves performance:

This evaluation provides three key insights:

1. SFT underperforms even the base model, dropping from 14.8% to 9.2%. This supports our central claim that SFT tends to overfit to synthetic prompt distributions and weakens out-of-distribution generalization.

2. RL recovers and surpasses base model performance, demonstrating that reinforcement learning improves robustness through optimizing the path of thinking more efficiently.

3. RL-Ground (ours) achieves the best generalization, outperforming both RL and SFT. Its design—explicit <caption> followed by progress-guided reasoning—encourages the model to extract and structure visual information more effectively, leading to better real-world QA performance even without exposure to in-domain examples.

We will include this GeoQA result and comparative analysis in the revised version to clarify that: While ComPABench is synthetic by design for diagnostic control, the caption-before-thinking strategy and progress-based reward design introduced in RL-Ground can transfer effectively to more naturalistic, multi-task reasoning scenarios, improving model generalization even without training on in-domain examples.

Moreover, we are actively working to extend ComPABench with real-world image sources as future plan.

Weakness 4-5: Visualization issue in Figure 4 and Figure 1

Thank you for the helpful suggestions. We agree that a bar chart is more appropriate for comparing discrete conditions and will revise Figure 4 accordingly to improve readability. For Figure 1, we will adopt a clearer layout with improved labeling and a legend to better illustrate the task structure and highlight the compositional gap.

Summary of Planned Revisions

We thank the reviewer again for highlighting both the strengths and areas for improvement. We will:

• Include qualitative error cases in the final version
• Clarify the significance of low SFT performance with empirical evidence
• Revise Figures 1 and 4 for better readability
• Emphasize that ComPABench is a diagnostic tool, complementary to open-world benchmarks, and reveal the conclusions drawn in our paper are transferable to real-world datasets.

We hope these improvements and clarifications address your concerns and merit a higher score.

Dear Reviewer 3k4H,

Thank you again for taking the time to review our submission. We deeply appreciate your thoughtful feedback and would welcome the opportunity to continue the discussion. In our rebuttal, we have carefully addressed all the concerns you raised, including detailed error analyses, failure case studies (e.g., on SFT), and the generalization on realistic dataset. We hope that our responses help clarify the contributions and impact of our work more clearly.

As the discussion phase is drawing to a close, we would be grateful if you could take a moment to review our responses and share any further questions or concerns. We would be more than happy to continue the discussion and provide additional clarification if needed.

Thank you for your consideration.

Thanks for the author's discussions. The reviewer may still have some concerns and decided to maintain their current score.

[w1-w2] While the authors provide an example to illustrate the failure mode of SFT, the analysis remains limited in scope and does not convincingly explain the unusually low compositional performance. Additionally, the reported behavior of SFT-init-RL appears inconsistent with common empirical patterns observed in similar training paradigms. This raises the question of whether the underlying base model is too weak. Have the authors explored using a stronger base model to verify whether the observed trends still hold?

[w3] The reviewer strongly recommend to include these results in the next version of the paper which can strenghthen the experimental completeness.

We thank the reviewer for the thoughtful comments and detailed feedback. And we answer the questions raised by the reviewer as follows:

Q1: Dataset Leakage – Are ComPABench prompts/images derived from public datasets? How is overlap with Qwen-VL filtered?

All ComPABench images and prompts are synthetically generated in-house using deterministic rendering pipelines (see Supplementary Sec. A). Our datasets are constructed from parametric shape definitions and grid layouts with predefined color palettes, ensuring no overlap with real-world datasets. Therefore, leakage is strictly avoided.

Q2: Sample Size Sensitivity – Did you compute confidence intervals or run multiple seeds?

We acknowledge the concern regarding statistical robustness. As noted in the NeurIPS checklist, we ran inference 3 times for all evaluation settings and report average results. We will state it clearly in the revision, which confirms that the key trends remain stable under variance.

Q3: Reward Shaping – Can the authors release the reward code and ablative weights?

Yes, we have released the complete reward shaping code in the supplementary zip, and will publicly host it upon acceptance. As detailed in Supplementary Sec. B (Table 2), we conducted ablations isolating the impact of:

• Caption format only: Improves grounding but insufficient alone for compositional reasoning.
• Progress reward only: Boosts compositional accuracy from 31.2% to 39.6%.
• Full RL-Ground: Combines both, achieving 52.8% compositional accuracy.

These results validate the design of RL-Ground as both principled and effective.

Q4: Scale Trend – Does RL’s advantage persist at larger scales (e.g., 32B)?

Due to compute constraints, our study focuses on 3B and 7B models. While we have not yet evaluated larger models (e.g., 32B), prior work [1] demonstrates that SFT tends to overfit even at 11B, whereas RL provides better generalization under distribution shifts. We leave large-scale evaluation as an important direction for future work.

[1] Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V. Le, Sergey Levine, and Yi Ma. SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-Training. arXiv preprint arXiv:2501.17161, 2025.

Limitations – Synthetic domain and narrow reasoning types

We explicitly discuss this in Supplementary Sec. 1. ComPABench is designed as a controlled diagnostic suite (like CLEVR), not as a naturalistic benchmark. While it lacks open-world visual complexity, it systematically isolates reasoning failure modes that broader datasets often conflate. We view it as complementary to large-scale evaluations.

Typo – Line 49 “Firgure”

Thank you for noticing. We will correct this in the final version.

In summary, we appreciate the reviewer’s recognition of our benchmark design, training insights, and actionable findings. We hope our clarifications, ablations, and reproducibility efforts address the reviewer’s concerns and justify a higher score.

We thank the reviewer for the effort in reviewing our paper. We also want to let you know that, as mentioned in the discussion with other reviewer, we also conducted experiments on GeoQA, which is a realistic dataset with natural language diversity.

• Qwen2.5-VL-3B-Instruct: 14.8% accuracy
• SFT: 9.2% accuracy
• RL: 17.8% accuracy
• RL-Ground: 21.2% accuracy

The results demonstrate that our conclusion can generalize beyond our synthetic datasets even without training on the task-specific data. We will include all these clarifications and improvements in our revised version. Thanks again for your recognition of our work.