We sincerely appreciate the Reviewer for providing valuable feedback and suggestions. We have responded to each of your comments and questions as follows:

• Q1. Clarification and explanation regarding "specialist model”.

  Thank you for your question regarding whether Perception-R1 is a single generalist model or a collection of specialist models. In fact, the core contribution of Perception-R1 is to propose a new, perception-first RL post-training paradigm. Our focus is more on the effectiveness and generality of this method rather than build a single unified model. For the sake of training efficiency, we trained different tasks separately using only a very small amount of data in our experiments. However, it is entirely feasible to mix the rewards from different tasks to perform joint training, and some recent work [1] has already done so. To further demonstrate this feasibility, we also conducted a simple mixed-reward experiment, as shown in the table below:

  case	data	Visual Grounding			Visual Counting
  		RefCOCO	RefCOCO+	RefCOCOg	Pixmo_val	Pixmo_test
  single reward	5k grounding	89.1	81.7	85.7	-	-
  single reward	5k counting	-	-	-	78.1	75.6
  mix reward	5k grounding + 5k counting	88.6	81.1	85.2	78.0	75.2

  We can observe that mixing rewards from different tasks for joint training does not cause significant negative impact on task performance. Therefore, it is entirely feasible to train a single unified model if training efficiency is not a primary concern.

  [1] Ma, Yan, et al. "One RL to See Them All: Visual Triple Unified Reinforcement Learning."

• Q2. How was the RL checkpoint chosen for evaluation?

  All experiments were evaluated using the final checkpoint of the trained model.

• Q3. Are the results consistent across seeds and training steps?

  Regarding the training seed and the number of training steps, we compiled the model's performance under different training seeds and numbers of training steps, as shown in the table below:

  exp	seed value	Visual Grounding
  		RefCOCO	RefCOCO+	RefCOCOg
  1	123	88.7	81.0	85.1
  2	124	89.1	81.5	85.3
  3	125	88.8	81.0	85.4
  Average	--	88.9	81.2	85.3

  exp	training step	Visual Grounding
  		RefCOCO	RefCOCO+	RefCOCOg
  4	100	88.5	80.9	84.5
  5	200	88.7	81.3	85.1
  6	300	88.8	81.5	85.7
  last	357	89.1	81.7	85.7

  we can observe that model training is not sensitive to seed settings (performance fluctuation does not exceed one point). Additionally, RL training steps show a positive correlation with final performance, and as training steps increase, performance gradually converges to an upper limit.

• Q4. Clarify of some abbreviations and missing prior work.

  Thank you very much for your meticulous review. Based on your feedback, we have carefully examined all occurrences of technical term abbreviations to ensure that full explanations are provided upon first appearance. We have also added citations for the missing prior work you identified. These updates will be presented in the next version.

• Q5. Why not make model automatically learn to skip thinking?

  Thank you for this interesting question. We would like to answer your query from two aspects:

  1. First, as our experiments in the paper demonstrate, an explicit linguistic thinking process is not beneficial for visual perception tasks. We observed that including such a process consistently degraded the model's final performance. This finding prompted us to disable the explicit linguistic thinking process and instead allow the model to learn an implicit "visual logic." For a more detailed explanation of "visual logic" and "linguistic logic," please refer to our response to Q5 of Reviewer # z4S9.

  2. Second, it is indeed feasible to have the model autonomously learn to skip the thinking process. We conducted experiments on this as well (shown in the table below). Specifically, we enabled an explicit reasoning process training on the RefCOCO by incorporating <think> and <answer> special tokens. We then tracked the number of tokens generated within the <think> tags as training progressed. We can observe that during the RL process, the model tends to gradually shorten the length of its "thinking" tokens. However, this approach is highly inefficient, and the final performance is not as good as when we disable the thinking process directly.
     | exp | training step | thinking_length | |------|------|------------------| | 1| 10 | 72| | 2| 50 | 39| | 3| 100| 27| | 4| 150| 25| | 5| 200| 20|

  Therefore, we opted to bypass the explicit reasoning steps, prompting the model to generate the final answer directly.

We sincerely appreciate your recognition of our work, as well as your thoughtful and professional review. The suggestions you provided are highly valuable for the further improvement of our work. We have responded to each of your comments and questions as follows.

• Q1. Lack of conceptual rigor including Perception Policy and Perceptual Perplexity.

  We are sorry that we did not provide detailed formal definitions in the main text due to the page limit. We offer the following clarifications below and will add them to the next revision.

  First, we aim to provide an RL-centric definition of Perception Policy. In the context of Perception-R1, the learned policy $\pi_{\theta}$ is the MLLM itself (e.g., Qwen2-VL). The learning process uses the GRPO algorithm to update the model’s parameters $\theta$ based on rule-based rewards calculated from generated outputs.

  During this process, the observation space $O$ can be defined as $O=(I,P)$, where $I$ is the image input, $P$ is the text-based prompt that specifies the task. For object detection, for example, the prompt is a request to output the bounding box coordinates and the class names of objects. The state at step $t$ can be formally defined as: $s_t = (O, a_1, a_2, ..., a_{t-1})$, where $(a_1, a_2, ..., a_{t-1})$ is the action sequence of tokens based on next-token prediction mechanism. For the specific task of object detection, the action sequence must conform to a strict structure encodes both object locations and their categories that looks like:{"bbox_2d": [127, 54, 471, 406], "label": "laptop"}. For the formal equation of Perceptual Perplexity, given a perceptual task dataset, the perceptual perplexity P of this task can be represented as: $P = \prod_{i=1}^{N} m_i!$ where $m_i$ is the number of ground-truth objects belonging to category $i$, $i$ ∈ {1, 2, ..., N} N is the total number of possible object categories for the task. The sum of objects across all categories must equal the total number of objects: $\sum_{i=1}^{N} m_i = M$. For visual grounding and OCR task, $M = N = 1$. For visual counting task, $M > 1, N = 1$. For object detection, $M > 1, N > 1$.

• Q2. Comparison between policy-gradient approach like GRPO and preference-based approach like DPO.

  Thank you for your valuable suggestion. While the relative merits of policy-gradient RL like GRPO versus preference optimization like DPO have already been extensively studied in prior work [1][2][3], our work focuses primarily on their application within visual perception contexts. In this domain, we identify two principal advantages of GRPO over DPO:

  1. Dynamic on-Policy exploration. Through a dynamic and interactive training process, GRPO samples data from the current policy model and continuously updates the policy online. This enables it to explore the policy space, learn from its own mistakes, adapt to its evolving capabilities, and constantly create new, more challenging "curricula" for itself. This implies that GRPO affords the model a broader exploration space, enabling it to discover superior visual perception polices. In contrast, DPO, as an offline method, relies on a fixed preference dataset, which leads to a "distribution shift" problem and limits its further performance improvement.
  2. Generation of high-quality hard negatives. GRPO's online sampling naturally generates high-quality negative examples that are relevant and challenging. Since the model explores at the edge of its capabilities, its failures are meaningful mistakes rather than random errors. These "hard negatives" are incorrect but plausible solutions that provide maximum learning value. DPO is limited by its pre-collected dataset, where the quality of negative examples may not target the model's specific weaknesses as it evolves. GRPO therefore creates a more effective learning process by continuously finding its own most challenging examples.

  As for the work on PerPO [4], although it is specifically optimized for visual perception tasks through "discriminative rewarding" and "listwise preference optimization”, its approach still falls within the paradigm of DPO. Consequently, it cannot fundamentally resolve the two aforementioned deficiencies compared to GRPO.

  [1] Wang, Bo, et al. "Implicit Reward as the Bridge: A Unified View of SFT and DPO Connections."
  [2] Tang, Yunhao, et al. "Understanding the performance gap between online and offline alignment algorithms."
  [3] Lanchantin, Jack, et al. "Bridging Offline and Online Reinforcement Learning for LLMs."
  [4] Zhu, Zining, et al. "Perpo: Perceptual preference optimization via discriminative rewarding."

• Q3. More systematic study to support the claim of generalization enhancement.

  Based on your suggestion, we ran additional experiments to test our generalization claim. We used models post-trained on different tasks. Note that we excluded OCR because its focus on text extraction differs fundamentally from the other tasks, which require general visual capabilities like spatial perception and object identification. The results are presented below.

  	post-training task	MMBench	MMVet	MMStar	ScienceQA	SeedBench	MME		LLaVA-Bench	AI2D
  		Avg	Avg	Avg	Avg	Avg	Cognition	Perception	Avg	Avg
  Qwen2-VL-2B	--	71.9	45.6	46.3	74.0	72.7	418.5	1471.1	46.5	71.6
  Perception-R1	visual counting	71.8	48.9	45.7	73.4	73.0	430.0	1473.9	58.2	71.8
  	visual grounding	71.4	49.2	46.6	73.6	72.9	425.4	1457.2	52.0	71.3
  Qwen2.5-VL-3B	--	71.4	57.7	53.3	75.8	70.9	560.7	1470.8	62.7	79.0
  	object detection	78.5	56.4	55.7	78.1	73.7	611.4	1546.7	65.2	80.5

  The results show that the Perception-R1, when trained on diverse visual tasks, consistently enhances general visual understanding and reasoning. This demonstrates the effectiveness and generality of our method.

• Q4. More experimental results based on Qwen2-VL and Qwen2.5-VL.

  In response to your suggestion, we have conducted additional experiments comparing our Perception-R1 framework with the Qwen2-VL and Qwen2.5-VL baselines. The experimental results are presented as follows:

  case	base model	Visual Grounding			OCR	Visual Counting		Detection
  		RefCOCO	RefCOCO+	RefCOCOg	PageOCR	Pixmo_val	Pixmo_test	COCO2017
  Perception -R1	Qwen2-VL-2B	89.1	81.7	85.7	98.2	78.1	75.6	28.0
  	Qwen2.5-VL-3B	89.5	83.5	86.5	97.1	83.8	81.1	31.9

• Q5. Does the difference in reasoning between Perception-R1 and DeepSeek-R1 imply that their visual and linguistic abilities rely on separate mechanisms, each requiring its own alignment strategy?

  Yes, as we argue in the paper, this difference arises because visual perception tasks are primarily oriented toward visual logic rather than semantic or linguistic logic. For tasks like object counting or localization, a model learns implicit patterns directly from image pixel. Imposing an explicit, language-based reasoning chain introduces an unnecessary and potentially detrimental step that degrades visual perception capabilities. Similar findings are also documented in [1].

  Furthermore, we wish to clarify that although the reasoning paradigms for visual perception and linguistic reasoning differ, they can still be unified. By designing and combining distinct reward functions, a model can be trained to accommodate both modes of thinking, an avenue that has already been explored in [2].

  [1] Wei, Yana, et al. "Open Vision Reasoner: Transferring Linguistic Cognitive Behavior for Visual Reasoning."
  [2] Ma, Yan, et al. "One RL to See Them All: Visual Triple Unified Reinforcement Learning."

• Q6. Provide statistical significance such as with different random seeds.

  We appreciate you highlighting the importance of statistical significance. In response, we conducted stability experiments for Perception-R1 on the RefCOCO/+/g. Each experiment utilized a different random seed to assess the robustness of our findings, and the results are shown below.

  case	seed value	Visual Grounding
  		RefCOCO	RefCOCO+	RefCOCOg
  1	123	88.7	81.0	85.1
  2	124	89.1	81.5	85.3
  3	125	88.8	81.0	85.4
  Average	--	88.9	81.2	85.3

  We can observe that under different random seed settings, the model's performance fluctuation on benchmarks does not exceed 1 point, indicating that Perception-R1 possesses sufficient stability to support reproducibility.

• Q7. Negative societal impacts of creating more powerful perception models.

  We appreciate you raising the critical issue of societal impact. We agree this was an important omission and, following your suggestion, have added a discussion about this as follows: “It is essential to acknowledge the broader societal impacts and dual-use nature of our work. Like many AI technologies, the perception models that offer benefits in visual grounding, OCR, and object detection can also be exploited. For instance, highly accurate person detection could fuel mass surveillance and threaten civil liberties, while powerful OCR could scan sensitive documents without consent. As these models become foundational to AI systems, ethical considerations become paramount. We urge the community to lead the conversation on responsible development, establish clear guidelines, and create robust safeguards to mitigate these risks.”

We greatly appreciate the Reviewer for the recognition of our work and the valuable comments and suggestions you provided. We have responded to each of your comments and questions as follows.

• Q1. The "Perception-R1" name and its scope for broader perception tasks.

  Thanks for your suggestion. We would like to emphasize that the core contribution of Perception-R1 is to propose a new, perception-first RL post-training paradigm. So we name our framework “Perception-R1”. To demonstrate its effectiveness, we selected four foundational visual perception tasks: visual grounding, OCR, visual counting, and object detection. Theoretically, our framework can be adapted to any visual perception task by defining an appropriate reward function. In fact, some latest works have already attempted to use rule-based RL techniques for segmentation [1] and tracking [2], which can be viewed as extensions of our work. In the future, we will also explore applying Perception-R1 to 3D visual perception tasks (such as 3D object detection and tracking, depth estimation, etc.) to further expand its application scope.

  [1] You, Zuyao, and Zuxuan Wu. "Seg-R1: Segmentation Can Be Surprisingly Simple with Reinforcement Learning."
  [2] Wang, Biao, and Wenwen Li. "R1-track: Direct application of mllms to visual object tracking via reinforcement learning."

• Q2. Theoretical explanation of why rule-based RL is well-suited for visual tasks.

  This question is similar to the Q1 of Reviewer # S8da. To further elaborate on our reasoning, we highlight the following points:

  1. RL conducts better negative supervision compared to SFT. Visual representation learning and downstream visual task like object detection are highly dependent on the quantity and quality of negative samples [1][2]. In the context of post-training, the negative sampling spaces and strategies for RL and SFT differ significantly. With RL, any action that yields a low reward is treated as a "negative sample." This allows RL model to effectively explore and learn from a wide range of trajectory samples on-line. While SFT methods only sample one trajectory (answer text) off-line and the negative supervision is very limited (only from a few tokens that are not strictly aligned with the answer).
  2. Policy gradient allows models to explore the perception pattern freely not just clone the expert behavior. SFT is fundamentally a form of behavior cloning (BC), where the model learns to maximize the likelihood of an expert's demonstration (the ground-truth text). This approach can suffer from covariate shift: when the model makes a mistake, it enters a state distribution not seen during training, leading to compounding errors. In contrast, RL, particularly through policy gradient methods, optimizes the policy $\pi$ to maximize the expected cumulative reward. This objective encourages the model to explore and discover novel strategies or implicit ”perception patterns" that lead to high rewards, even if those patterns deviate from the expert's specific demonstration. It learns to solve the task robustly, rather than simply mimicking one particular solution path.
  3. RL allows for more flexible and task-aligned reward shaping. The objective for SFT is fixed: minimizing the cross-entropy loss against a reference text. This is an indirect proxy for the actual task performance. For many visual tasks, performance is better measured by non-differentiable or complex metrics. RL's reward function $R$ can be designed to directly incorporate these metrics. For instance, in object detection, the reward can be the Intersection over Union (IoU) score, directly optimizing for localization accuracy. For image editing, the reward could be based on a combination of a CLIP score for semantic alignment and a score from an aesthetic quality model. This flexibility allows RL to align the model's learning process much more closely with the true end-goal of the visual task.

  [1] He, Kaiming, et al. "Momentum contrast for unsupervised visual representation learning."
  [2] Lin, Tsung-Yi, et al. "Focal loss for dense object detection."

• Q3. More baseline results.

  Following your advice, we add more baselines including Qwen2-VL, Qwen2.5-VL, and LLaVA to apply our Perception-R1 framework. The results are shown as following:

  case	base model	Visual Grounding			OCR	Visual Counting		Detection
  		RefCOCO	RefCOCO+	RefCOCOg	PageOCR	Pixmo_val	Pixmo_test	COCO2017
  LLaVA-1.5-7B	-	57.2	49.8	50.2	-	-	-	-
  Perception -R1	LLaVA-1.5-7B	61.8	54.0	55.6	-	-	-	-
  Qwen2-VL-2B	-	86.8	77.1	83.3	94.4	60.2	50.5	0.1
  Perception -R1	Qwen2-VL-2B	89.1	81.7	85.7	98.2	78.1	75.6	28.0
  Qwen2.5-VL-3B	-	86.7	79.3	84.3	94.9	60.8	57.4	16.1
  Perception-R1	Qwen2.5-VL-3B	89.5	83.5	86.5	97.1	83.8	81.1	31.9

• Q4. Reward functions are still tailored per task and depend on carefully tuned heuristics, which may not scale easily.

  Thank you for your concerns regarding the reward function design. The reward function is central to rule-based RL. Given the discriminative nature of visual perception tasks, most reward functions are discriminative metrics such as Intersection over Union (IoU) or Euclidean distance. These different task-specific reward functions are not mutually exclusive, rather, they can be mixed and scaled up. Recent work [1] has explored this approach, demonstrating the effectiveness and scalability of a unified RL framework for multiple downstream tasks.

  Furthermore, from an RL perspective, this multi-component reward structure functions can be regarded as task-aligned reward shaping. As discussed in the response of Q2, for many visual tasks, performance is better measured by non-differentiable or complex metrics. RL's reward function $R$ can be designed to directly incorporate these metrics. By providing a dense and multifaceted objective rather than a single sparse signal, it inherently mitigates the risk of policy exploitation (i.e., reward hacking), thereby compelling the model to learn more robust and generalizable policies.

  [1] Ma, Yan, et al. "One RL to See Them All: Visual Triple Unified Reinforcement Learning."

• Q5. Limited evaluation on real-world applications.

  Thanks for your advice. To address your concerns, we have conducted additional evaluations in real-world and robotic scenarios. Specifically, we select two challenging real-world understanding and reasoning benchmarks, i.e., RealWorldQA, and OmniSpatial [1] to evaluate Perception-R1. The results are presented in the table below.

  models	RealWorldQA	OmniSpatial
  Baseline (Qwen2.5-VL-3B)	60.26	41.81
  Perception-R1	61.83	42.07

  We can observe that Perception-R1 still demonstrates its effectiveness in noisy real-world scenarios.

  [1] Jia, Mengdi, et al. "OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Models."

• Q6. Open code and model weights.

  Thank you for this valuable suggestion. In fact, we are already prepared to make our work open-source including data, code, and model weight. In order to adhere to the NeurIPS double-blind review policy, we have omitted this information in the current submission. We will be sure to include a direct link to the open-source repository in the final camera-ready version.

• Q7. Why SFT + RL performs worse than RL-only?

  Thank you for your question. We attribute the reason to two primary factors:

  1. Our base model, i.e., Qwen2-VL and Qwen2.5-VL, have already undergone a comprehensive SFT stage. Applying an additional SFT phase on this foundation risks disrupting the model's originally learned data distribution.
  2. As we analyze in the response for Q2, SFT is fundamentally a form of behavior cloning (BC) [1]. This suggests the model can become constrained by the rigid mimetic patterns acquired during the SFT phase, thereby impeding effective exploration in the subsequent RL stage.

  As a result, performance can actually degrade, making the direct application of RL a more effective approach.

  [1] Chen, Jierun, et al. "The Synergy Dilemma of Long-CoT SFT and RL: Investigating Post-Training Techniques for Reasoning VLMs.”

Thank you very much for recognizing our work and for providing such valuable comments and suggestions. We have responded to each of your comments and questions as follows:

• Q1. Theoretical explanation about why RL outperforms SFT without CoT process.

  Thanks for your question. This is a profound and valuable topic. Indeed, some existing works [1][2][3] have already explored the relative merits of RL and SFT. In the context of visual perception, we think the following key reasons are crucial:

  1. RL conducts better negative supervision compared to SFT. Visual representation learning and downstream visual task like object detection are highly dependent on the quantity and quality of negative samples [4][5]. In the context of post-training, the negative sampling spaces and strategies for RL and SFT differ significantly. With RL, any action that yields a low reward is treated as a "negative sample." This allows RL model to effectively explore and learn from a wide range of trajectory samples on-line. While SFT methods only sample one trajectory (answer text) off-line and the negative supervision is very limited (only from a few tokens that are not strictly aligned with the answer).
  2. Policy gradient allows models to explore the perception pattern freely not just clone the expert behavior. SFT is fundamentally a form of behavior cloning (BC), where the model learns to maximize the likelihood of an expert's demonstration (the ground-truth text). This approach can suffer from covariate shift: when the model makes a mistake, it enters a state distribution not seen during training, leading to compounding errors. In contrast, RL, particularly through policy gradient methods, optimizes the policy $\pi$ to maximize the expected cumulative reward. This objective encourages the model to explore and discover novel strategies or implicit ”perception patterns" that lead to high rewards, even if those patterns deviate from the expert's specific demonstration. It learns to solve the task robustly, rather than simply mimicking one particular solution path.
  3. RL allows for more flexible and task-aligned reward shaping. The objective for SFT is fixed: minimizing the cross-entropy loss against a reference text. This is an indirect proxy for the actual task performance. For many visual tasks, performance is better measured by non-differentiable or complex metrics. RL's reward function $R$ can be designed to directly incorporate these metrics. For instance, in object detection, the reward can be the Intersection over Union (IoU) score, directly optimizing for localization accuracy. For image editing, the reward could be based on a combination of a CLIP score for semantic alignment and a score from an aesthetic quality model. This flexibility allows RL to align the model's learning process much more closely with the true end-goal of the visual task.

  And this conclusion still holds true for the simple image classification task [6].

  [1] Chen, Jierun, et al. "The Synergy Dilemma of Long-CoT SFT and RL: Investigating Post-Training Techniques for Reasoning VLMs.”
  [2] Zhu, Ke, et al. "Continual sft matches multimodal rlhf with negative supervision."
  [3] Wang, Bo, et al. "Implicit Reward as the Bridge: A Unified View of SFT and DPO Connections."
  [4] He, Kaiming, et al. "Momentum contrast for unsupervised visual representation learning."
  [5] Lin, Tsung-Yi, et al. "Focal loss for dense object detection."
  [6] Li, Ming, et al. "Think or not think: A study of explicit thinking in rule-based visual reinforcement fine-tuning."

• Q2. The definition and justification of the perceptual-perplexity metric.

  We are sorry that we did not provide detailed formal definitions in the main text due to the page limit. We offer the following clarifications of perceptual perplexity below and will add them to the next revision.
  Definition: Given a perceptual task dataset, the perceptual perplexity P of this task can be represented as: $P = \prod_{i=1}^{N} m_i!$ where $m_i$ is the number of ground-truth objects belonging to category $i$, $i$ ∈ {1, 2, ..., N} N is the total number of possible object categories for the task. The sum of objects across all categories must equal the total number of objects: $\sum_{i=1}^{N} m_i = M$.
  Justification: For visual grounding and OCR task, $M = N = 1$. For visual counting task, $M > 1, N = 1$. For object detection, $M > 1, N > 1$. Consequently, the perceptual perplexity associated with these tasks increases from low to high.

• Q3. Experiment setting in Table 1-4 and add Qwen2.5-VL baseline result.

  The results for Perception-R1 presented in Tables 1-4 were obtained in a zero-shot setting. Concurrently, we report the SFT-only results in Table 5. Following your advice, we have added the results of Qwen2.5-VL as a baseline, as shown in the table below. These results will be included in the next version.

  case	base model	Visual Grounding			OCR	Visual Counting		Detection
  		RefCOCO	RefCOCO+	RefCOCOg	PageOCR	Pixmo_val	Pixmo_test	COCO2017
  Qwen2-VL-2B	-	86.8	77.1	83.3	94.4	60.2	50.5	0.1
  Qwen2.5-VL-3B	-	86.7	79.3	84.3	94.9	60.8	57.4	16.1
  Perception -R1	Qwen2-VL-2B	89.1	81.7	85.7	98.2	78.1	75.6	28.0
  Perception-R1	Qwen2.5-VL-3B	89.5	83.5	86.5	97.1	83.8	81.1	31.9

• Q4. Issue of the comparison between SFT-only and baseline setting in Table. 5.

  We sincerely apologize that there is an oversight in the OCR column of Table 5. In fact, the correct baseline metric for row 3 in the OCR column is 94.4, which is consistent with the Qwen2-VL baseline reported in Table 3. With this correction, it is clear that the performance in the SFT-only setting is generally higher than the baseline performance. We will rectify this in the revised manuscript.

• Q5. Confusion about the statement at line 256-257.

  Thank you for pointing out the ambiguity of this sentence. To clarify, our intention is to convey that for tasks with low perceptual perplexity, such as visual grounding and OCR, the RL-only setting does not yield a significant performance improvement over the SFT-only method. The gain is only about one point, which is why we describe the results as "comparable”. In the next version, we will revise this statement to be more precise and avoid any ambiguity.