Pixel Reasoner: Incentivizing Pixel Space Reasoning via Curiosity-Driven Reinforcement Learning

We thank you for your thoughtful questions and constructive feedback. Below, we address the questions.

1. On Generalization Beyond Zoom-In and Select-Frame

We thank the reviewer for requesting us for clarification on this crucial question. We agree that the current set of operations is focused, a deliberate choice we acknowledge in the paper's limitations section. We would like to clarify our rationale.

1. Rationale for a Focused Scope: The space of potential visual operations is vast, including rotation, drawing, filtering, segmentation, and more. Rather than attempting an exhaustive implementation, which is neither feasible nor necessarily productive for an initial investigation, we strategically selected ZOOM-IN and SELECT-FRAME. These were chosen because they are fundamental primitives for interacting with static images and dynamic videos, respectively. This focused set allowed us to rigorously study the core challenges of the pixel-space reasoning paradigm in a controlled yet broadly representative setting. These two operations cover the most of the use cases. The other operations like segmentation, etc are not contributing much values to our primary goal, i.e., to first establish the importance and viability of this paradigm.

2. Extensibility of the Framework: Our framework is designed from the ground up to be extensible. We envision future work building upon this foundation by designing bespoke operations tailored to specific downstream tasks (e.g., line drawing for geometric problem-solving, or fine-grained segmentation for autonomous driving). The core mechanisms for incorporating new operations would remain consistent:

   • Skill Acquisition via Instruction Tuning: The model would be fine-tuned on instruction-following datasets to learn the syntax and semantics of any new visual operation.
   • Exploration via Curiosity: The curiosity-based reward mechanism is agnostic to the specific operation and would continue to encourage the model to explore the utility of the new tool.
   • Task-Specific Rewards: The primary adaptation would be in the task-success reward function. For example, extending to a GUI grounding task would involve incorporating an Intersection-over-Union ($IoU$) reward, while a document understanding task might require a novel layout-aware reward metric.

In summary, while the scope of the demonstrated operations is focused, this does not detract from the paper's main contribution: revealing the importance of pixel-space reasoning and providing a viable, scalable method to unlock this capability in VLMs.

2. On Error Analysis and Failure Cases

We appreciate the reviewer's request for a more detailed discussion of failure cases. As noted, we provide a qualitative analysis in Appendix D.2. The primary failure modes observed were:

1. Hallucination: In some cases, the model pretends the visual operation is performed successfully and hallucinate with a conclusion and continue its reasoning.

2. No-Reaction: Another failure mode is the model's inability to properly react to the errors from visual operations. The model noted the errors and then proceed with textual reasoning without attempting to fix the errors.

Insights and Future Work: This paper is a preliminary study of pixel-space reasoning and it follows the common setting of using only outcome rewards. A key insight from analyzing these failures is that intermediate supervision could benefit more accurate and reliable pixel-space reasoning. Currently, the model relies primarily on the outcome reward, so the correctness of intermediate steps, e.g., correcting errors from visual operations, will not be properly encouraged. This points to a clear path for future improvement: developing methods to provide denser supervisory signals for the intermediate pixel-space reasoning steps. This could involve designing reward shaping for handling errors from visual operations, or providing more signals for the correctness of visual operations, e.g., rewarding the grounding of a relevant visual cues.

We sincerely thank the reviewer for their detailed, insightful, and constructive feedback. We are encouraged that they recognized our work as a "significant and timely contribution" and appreciated the rigor of our methodology and the meaningful performance gains our model achieves.

We would like to begin by highlighting what we see as the core contribution of this work: to formally identify, operationalize, and incentivize pixel-space reasoning. We believe this is a distinctive capability that underpins the powerful visual analysis seen in frontier models like OpenAI o3. Our paper reveals a key challenge in acquiring this new ability and proposes an effective framework to overcome it. We will frame our responses to the specific points below with this central contribution in mind.

Addressing Weaknesses

1. On the Scope of Visual Operations: We agree with the reviewer that the current set of visual operations is focused, a point we also acknowledged in the limitation section. This was a deliberate choice for this initial work, and we wish to clarify our rationale:

• The space of potential visual operations is vast (e.g., rotation, line drawing, segmentation, and more). It is neither feasible nor necessarily productive to try and enumerate and implement them all.
• Rather than attempting an exhaustive implementation, we selected ZOOM-IN and SELECT-FRAME as they are fundamental and broadly representative operations for interacting with static images and dynamic videos, respectively. These two operations cover the most of the use cases. The other operations like segmentation, etc are not contributing much values to our primary goal, i.e., to show the importance of the pixel-space reasoning paradigm.
• Our framework is designed to be extensible. Based on the current core set, we envision future work designing bespoke operations for specific downstream tasks, such as line drawing for geometric problem-solving or fine-grained segmentation for autonomous driving, etc.

Therefore, while the scope of the demonstrated operations is focused, we believe this does not detract from the paper's main contribution: revealing the importance and challenges of pixel-space reasoning and providing a viable method to unlock this capability in VLMs.

2. On the Technical Contribution of the Training Pipeline: We appreciate the reviewer's point that we indeed "stand on the shoulders of giants". We will refine the contribution stated in line 83 by clarifying that we innovatively address the challenges of learning pixel-space reasoning by proposing the approach, but not directly inventing instruction-tuning from scratch.

3. On the Quality of the SA1B Dataset and Potential for Brittle Strategies: We thank the reviewer for raising this important point about data quality. We share the concern regarding the potential noise in automatically generated datasets and took several deliberate steps to mitigate this risk, which we would like to clarify:

• Diverse Data Sources: We did not rely solely on SA1B. Our instruction tuning data is curated from a variety of sources, including FineWeb and STARQA, to provide rich textual and contextual diversity (Lines 117-119).
• Controlled Synthesis: Our data generation process utilizes a template-based synthesis approach (Figure 3, Lines 137-141). This provides strong structural control over the reasoning traces, ensuring that the generated visual operations are logically sound and contextually necessary for the task. This moves beyond simple noisy labels and enforces a coherent reasoning structure.
• Empirical Generalization: The strongest evidence against the model learning brittle strategies lies in its performance. Our model demonstrates consistent and significant generalization across a wide range of benchmarks and media formats (images and videos), many of which have distributions distinct from the training data.

These factors together ensure that the model learns robust and generalizable reasoning skills rather than superficial shortcuts. We thank the reviewer for highlighting this concern, and we will consider more measures to facilitate data quality assurance.

4. On the Fixed-Rate Constraint (RaPR) and Potential Over-Regularization: This is a crucial question, and we thank the reviewer for giving us the opportunity to clarify a nuanced but central aspect of our method. There may be a misunderstanding that the RaPR constraint is a rigid, dataset-wide mandate. In fact, it is a dynamic reward bonus to that monitors at the query level and rewards at the response level.

Here is a more precise breakdown of how it functions:

• RaPR monitors the query-level exploration. It tracks the proportion of rollouts that uses visual operations within the same query group.

• A curiosity bonus is added to a response's reward if and only if two conditions are met: (1) the response contains a visual operation, and (2) the current query-level RaPR is too low, i.e., below the target ratio.

• The model is never forced to use a visual operation. On the one hand, if the model determines that visual operations are not beneficial for a query and consequently explores no such actions in its rollouts, no penalty is incurred. On the other hand, if the model finds visual operations useful occasionally and its exploration satisfies the curiosity, no further reward bonus is given, thus preventing the model from adding gratuitous operations.

• The curiosity bonus acts as a temporary learning scaffold, primarily to help the model escape the initial "learning trap." As Figure 6 shows, this reward signal diminishes as the model learns the intrinsic value of the actions. The primary reward for reasoning correctness always remains dominant.

Therefore, there is no permanent mandate that could lead to over-regularization. We will revise the methodology section to make this dynamic, query-level exploration mechanism clearer to prevent misunderstanding.

5. On Data Release: We are fully committed to open science. The data,code,models were released (as stated in Line 253), but rebuttal disallow any form of link sharing. So we will share the link after the paper decision is out on the Openreview platform. We also appreciate the suggestion to include data samples and will add representative examples to the appendix in our revision.

6. On Table 1 and Missing Baselines:

• SEAL Baseline: We thank the reviewer for this point and would like to gently clarify that SEAL is included in Table 1 and cited in our related work (Line 355). To improve clarity, we will explicitly name "SEAL" in our discussion of "visual guided search" in Line 355 of the related work section.
• Missing Tool-Use Model Results: The empty cells for some tool-using models exist because the original papers for these models did not report performance on those specific benchmarks. Adapting these highly specialized visual search models to the diverse reasoning benchmarks we use is a significant engineering effort beyond the scope of this work. We believe the current table provides a fair comparison on the visual search benchmark (V-Star) where results are available.
• Distinction from tool-using models: Our work focuses on enabling a new reasoning paradigm where the model directly interacts with visual inputs to guide its internal thought process. This is distinct from direct tool-use or visual search, where the goal is typically to retrieve external information. The objective of our visual operations is to improve the quality of reasoning itself. We will clarify this distinction further in the related work section.

7. On the Absence of Qualitative Examples in the main paper: We agree that qualitative examples are essential for a paper introducing a new reasoning ability. Due to page constraints, we placed these examples, including both successes and failures, in the appendix. In our revision, we will restructure the paper to move several key qualitative examples into the main body for better visibility.

Addressing Comments and Questions

• Figure Readability: We agree and will revise Figures 3 and 8 for the final version to improve readability.

• Intrinsic Mechanisms for Pixel-Space Reasoning: We are grateful for this insightful and thought-provoking question. The reviewer's observation that pixel-space reasoning is not "intrinsically self-reinforcing" perfectly captures the motivation behind our work. This is precisely the "learning trap" we aimed to solve. Our use of curiosity-driven RL is a pragmatic and effective method to provide an initial "intrinsic motivation" and bootstrap the learning process, allowing the model to discover the long-term utility of these actions.

We wholeheartedly agree that developing more sophisticated intrinsic mechanisms is an exciting and important frontier for research. We believe our work is a crucial first step in this direction. We will add a discussion of this promising future direction to our conclusion to inspire further research in the community.

• Broader Impacts Section: We apologize for the confusion. A broader impacts statement was included in the NeurIPS checklist (Lines 703-710). We will add a formal "Broader Impacts" section to the appendix for better visibility.

We thank the reviewer again for their time and their thoughtful, high-quality feedback. This review has provided us with clear, actionable guidance that will undoubtedly help us improve the quality and clarity of our paper.

We sincerely thank the reviewer for constructive suggestions. Below, we address the reviewer's specific questions.

1. Regarding the inclusion of the ChartQAPro benchmark.

We thank the reviewer for bringing the interesting ChartQAPro benchmark to our attention. We agree that evaluating our method on such tasks is a valuable direction. However, we did not include it in the current submission for two primary reasons:

• Timing: The ChartQAPro benchmark was released in mid-April 2025. Given the NeurIPS submission deadline in May, we didn't know about this new work at the moment of submission.
• Scope: As we state in our paper (L34, L85), our primary research goal is to demonstrate the significance of pixel-space reasoning in visually-intensive, general-purpose scenarios. We therefore prioritized benchmarks like InfographicsVQA, V-Star, MVBench, which include visually-rich scenarios like videos, high-resolution scenes and infographics.

2. Regarding the limitations of the synthetic dataset.

We appreciate the reviewer's insightful comment on the limitations and scalability of our synthetic dataset. We acknowledge that its current form is focused on demonstrating the core capabilities on visually-rich tasks and does not yet cover complex, multi-step numerical or logical reasoning often required by chart-based VQA.

To address this and to outline a path toward a more comprehensive dataset, we will add a discussion on this limitation to the paper. We will also propose a concrete pipeline for extending our synthetic data generation to tasks like ChartQA, which includes the following steps:

1. Leverage a state-of-the-art VLM (e.g., GPT-4o) to identify key visual elements and tabular data within a chart.
2. Generate question-answer pairs that require targeted reasoning about these identified visual cues.
3. Synthesize both expert demonstration and self-correction trajectories based on these generated pairs, teaching the model to ground its reasoning in specific visual regions.

3. Regarding the analysis of the 'learning trap'.

We thank the reviewer for highlighting the need for a deeper analysis on the 'learning trap'.

In our current work, we provided a deep empirical analysis of this "learning trap" problem in Section 5.2. Our experiments demonstrate that without instruction tuning and intrinsic rewards, the model defaults to its well-established textual reasoning abilities and fails to effectively learn the novel pixel-space reasoning skill, which is precisely the "learning trap" problem. We also show that the fundamental reason for this problem is the significant proficiency gap between the two capabilities.

To make this clearer, we will revise the manuscript to explicitly frame Section 5.2 as an empirical investigation into the 'learning trap'.

4. Regarding the ablation study on self-correction data.

We thank the reviewer for raising this point, as it gives us the opportunity to clarify the precise role of the self-correction data. We believe the potential confusion arises from our choice to use a negative framing in the paper (L302). To emphasize the data's impact, we described the model’s failures without it, rather than its benefits with it.

To state the mechanism directly and positively: the self-correction trajectories are designed to teach the model to handle errors from the visual operations. When a visual operation yields an unexpected or erroneous outcome, this data provides explicit demonstrations of how to proceed with a corrective action, rather than leading to model's bypassing of these errors from visual operations.

In our revision, we will rephrase this section to state this positive mechanism more explicitly, ensuring the contribution of the self-correction data is unambiguous.

5. Regarding the advantages of using RL for pixel-space reasoning.

We thank the reviewer for requesting a more thorough justification for using Reinforcement Learning (RL). As we note in L52, our approach follows the standard paradigm of post-training with instruction tuning and RL to instill new, complex abilities. The reason this two-stage process is necessary is that instruction tuning alone is insufficient for this task.

As our results in Table 1 empirically demonstrate, a model trained with only instruction tuning is fragile and struggles to generalize. RL is not just a means of making learning more efficient; it is necessary for developing a robust reasoning policy. RL enables the model to explore the vast space of possible visual operations and learn to assign credit to actions over long sequences. The intrinsic reward signal encourages the model to move beyond simply mimicking expert demonstrations and to discover for itself which policies lead to correct answers, making it more resilient to novel situations not seen during supervised fine-tuning. We will clarify this motivation in the revised manuscript.

Thank you to the authors for the concise response. However, I did not find any informative response to the raised major concerns,such as limited experimental results.

We have now completed our experiments on CharQAPro Bechmark and list our results below.

We compare PixelReasoner-7B against its base model (Qwen-VL2.5-7B), and other leading models reported in the ChartQAPro paper. For PixelReasoner-7B, the values in parentheses next to each score indicate the rate of its pixel-space reasoning.

Our analysis highlights the following key findings:

• State-of-the-Art Open-Source Performance: PixelReasoner-7B establishes a new state-of-the-art among open-source models with an overall score of 47.97. It outperforms its base model, Qwen-VL2.5-7B, by 4.7 absolute points on the overall score.

• Generalization: This strong performance is achieved without any fine-tuning on chart-related datasets. This result underscores the excellent generalization capability of our model's inherent pixel-space reasoning mechanism for a complex task it was not explicitly trained for.

• Computational Efficiency: These performance gains are realized without introducing significant computational overhead. The overall RaPR is 15%, making our approach both effective and highly efficient compared to the base model.

• Competitive with Proprietary Models: While Claude 3.5 Sonnet leads overall, our open-source model is highly competitive, surpassing other powerful proprietary models like GPT-4o (41.68) and Gemini 1.5-Flash (45.97).

• Limitation on Unanswerable Questions: A detailed error analysis revealed a key limitation: PixelReasoner-7B currently fails on most unanswerable questions within the benchmark. Addressing this failure mode is a clear priority and a promising direction for future work.

Hope our new experimental results can resolve your concerns regarding our submission.

We thank the reviewer for their valuable feedback and insightful questions.

1. Efficiency Trade-offs (Tokens & Visual Operations)

Thank you for this excellent question about the efficiency of our method. We reported RaPR (rate of pixel-space reasoning) and tokens per query-response for each benchmark to clarify the computational trade-offs.

As shown, tasks requiring fine-grained detail, like video analysis (MV-Bench) and visual search (V*), trigger visual operations most frequently. In contrast, tasks like InfographicsVQA, where the evidence is often easier to found in the single page infographics, have a much lower rate of visual opeartion. These observations also demonstrate PixelReasoner's ability to allocate computation efficiently and autonomously.

2. Extending with More Visual Operations

This is a great question about the extensibility of our framework. Additional operations like line-drawing for geometric problems, region highlighting/masking for grounded reasoning, or optical character recognition (OCR) for document understanding would be highly beneficial. Our framework is designed to be easily extensible to new tools through three core mechanisms:

• Skill Acquisition: New operations are incorporated by fine-tuning the model on instruction-following datasets that demonstrate the tool's syntax and function.
• Agnostic Exploration: Our curiosity-driven reward mechanism is tool-agnostic. It naturally encourages the model to explore when and how a new tool can be used to solve a task, without needing manual reprogramming.
• Task-Specific Rewards: The primary adaptation is in the task-success reward function. For example, a GUI navigation task could use an Intersection-over-Union (IoU) reward for object grounding, while a document task might use a layout-aware edit distance. This modular design allows our method to easily incorporate new operations for a wide range of multimodal problems.

3. Correction in Table 1 We sincerely thank the reviewer for identifying the error in Table 1 and the imprecise description of the VPD model. We will correct them accordingly in our revision.

To further evaluate our Pixel Reasoner's OOD generalization capabilities on broader domains, we have conducted a new experiment on ChartQAPro. We list our results as follows:

We compare PixelReasoner-7B against its base model (Qwen-VL2.5-7B), and other leading models reported in the ChartQAPro paper. For PixelReasoner-7B, the values in parentheses next to each score indicate the rate of its pixel-space reasoning.

Our analysis highlights the following key findings:

• State-of-the-Art Open-Source Performance: PixelReasoner-7B establishes a new state-of-the-art among open-source models with an overall score of 47.97. It outperforms its base model, Qwen-VL2.5-7B, by 4.7 absolute points on the overall score.

• Generalization: This strong performance is achieved without any fine-tuning on chart-related datasets. This result underscores the excellent generalization capability of our model's inherent pixel-space reasoning mechanism for a complex task it was not explicitly trained for.

• Computational Efficiency: These performance gains are realized without introducing significant computational overhead. The overall RaPR is 15%, making our approach both effective and highly efficient compared to the base model.

• Competitive with Proprietary Models: While Claude 3.5 Sonnet leads overall, our open-source model is highly competitive, surpassing other powerful proprietary models like GPT-4o (41.68) and Gemini 1.5-Flash (45.97).

• Limitation on Unanswerable Questions: A detailed error analysis revealed a key limitation: PixelReasoner-7B currently fails on most unanswerable questions within the benchmark. Addressing this failure mode is a clear priority and a promising direction for future work.

The evaluation on ChartQAPro has significantly strengthened our paper by demonstrating the robust generalization of PixelReasoner. We believe these new results underscore the value of our contribution, and we will integrate this analysis into the revised manuscript.