Perception in Reflection

We thank the reviewer for the thoughtful feedback and for recognizing the value of our method and experiments. We carefully address the concerns and clarify potential misunderstandings as follows:

Q1: Alignment with Human Visual Focus

We address this concern by clearly defining “ground-truth human attention” and conducting human evaluations:

“Ground-truth human attention” refers to the qualitative visual focus humans naturally adopt when answering questions—typically prioritizing the main subject over background elements. An attention map that better reflects human perception tends to activate more semantically relevant image tokens and highlight regions aligned with human focus patterns [1].

To further support this claim, we conducted a human evaluation to compare the image attention of RePer and LLaVA-1.5.

• Setup: Six annotators (PhD/Master’s students in CV/NLP) were shown anonymized and shuffled attention maps from both models on 100 randomly sampled test images. They were asked to choose: “Which attention map better reflects your own visual focus if you were to answer this question?” The interface is shown in Re-Fig. 2 at https://reper-vl.github.io/ICML-Rebuttal/.
• Metric: We report the win rate—the percentage of cases where RePer’s map was preferred over LLaVA-1.5’s.
• Result: Image attention maps from RePer was preferred in 70.27% of cases over LLaVA-1.5, indicating stronger alignment with human visual focus.

We will revise the paper to clarify the definition and more accurately scope this claim based on the supporting evidence.

[1] Huang et al. OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation, CVPR 2024.

Q2: Advantages in Complex Reasoning

We appreciate the reviewer’s suggestion and would like to clarify a potential misunderstanding: our work focuses on perception-oriented tasks, while complex reasoning is positioned as a future direction. As noted at the end of the Introduction, we see strong perception as a foundation for advanced reasoning capabilities.

To further explore this potential, we evaluated 13B models on three reasoning benchmarks—ScienceQA, MMMU, and AI2D—using VLMEvalKit. As shown below, RePer consistently achieves the highest performance, demonstrating stronger reasoning capabilities and generalization.

Q3: Novelty of RePer

1. Reflection in LVLMs vs. LLMs (Importance & Uniqueness): While reflection has been explored in LLMs, it holds distinct importance in the context of LVLMs. Unlike language models, where input tokens are discrete and semantically stable, visual perception involves high uncertainty. LVLMs are more prone to hallucinating nonexistent objects, misinterpreting visual cues, or overlooking salient regions—challenges that extend beyond typical language ambiguity. Our reflection-based method directly addresses these issues by enabling iterative refinement. Beyond improving output quality, it also enhances human-aligned visual attention and reduced hallucination, underscoring its unique value for multimodal perception.

2. Dual Methodological Contributions:

• First, we propose reflective perception learning to LVLMs. where image-grounded, step-wise corrections provide fine-grained rewards for aligning vision and language. This is fundamentally distinct from language-only feedback and proven highly effective.
• Second, we introduce a reward-weighted unlikelihood training to reinforce high-quality responses while explicitly penalizing suboptimal ones. This mitigates the response collapse observed in prior works (e.g., RISE, SCoRe), where early-stage answers dominate learning regardless of quality, and enables stronger first-turn performance.

3. Data Construction innovations: While not extensively emphasized in the main paper, our data construction pipeline introduces key innovations by combining VLM-based and rule-based reward signals to construct high-quality reflective conversations. This dual-reward structure improves the precision and interpretability of supervision, making a clear improvement over prior self-reflection datasets (e.g., RISE, SCoRe). Notably, it also resonates with both RM-based and RM-free RL paradigms.

Finally, regarding the notion of novelty itself, we borrow a perspective from Novelty in Science [1]:

If you hear a good idea, there is a moment of surprise and then, the better it is, the more obvious it may seem. If it is easy to explain and obvious in hindsight, this in no way diminishes the creativity (and novelty) of the idea.

[1] Black. Novelty in Science. Medium. https://medium.com/@black_51980/novelty-in-science-8f1fd1a0a143

We thank the reviewer for the positive feedback, particularly for recognizing our work as a “well-motivated method” with solid experiments. Below, we carefully address each of your concerns and clarify potential misunderstandings.

Q1: LLaVA-1.5 BoN vs RePer

We appreciate the suggestion and clarify a possible misunderstanding: the critic model is not reqired but optional for RePer during inference. As discussed in Sec. 4.6 (Lines 370–381), RePer already outperforms LLaVA-1.5 in the first turn without any external feedback, demonstrating a fair comparison that reflects the effectiveness of our reflective perception training.

To further address the reviewer’s concern, we conducted another fair comparison on DetailCaps-4870 by implementing a Best-of-N (N=1,3,5,6) baseline for LLaVA-1.5, where we sample N responses and use the same critic (LLaVA-Critic-7B) to select the best one for computing the final score.

As shown in the table, even with a larger sampling budget, LLaVA-1.5-13B Best-of-6 still falls short of RePer-13B’s performance using just a single-turn response. This underscores RePer’s efficiency and effectiveness in answer generation: rather than relying on brute-force sampling, RePer produces strong initial responses through reflective perception learning mechinism, and further enhances them via feedback-driven refinement.

We will clarify the fairness of the comparison more explicitly in the revision.

Q2: Alignment with Human Visual Focus

To address this concern, we conducted a human evaluation to further support the claim that RePer’s image attention better aligns with human visual focus.

• Setup: We recruited 6 annotators (PhD/Master’s students in computer vision and NLP) and randomly sampled 100 images from the test set. For each image, we collected image attention maps (as shown in Fig. 4) generated by RePer and LLaVA-1.5 during inference. The attention maps were anonymized and randomly shuffled before being shown to annotators, who were asked: “Which attention map better reflects your own visual focus if you were to answer this question?” The annotation interface is shown in Re-Fig. 2 on our supplementary website: https://reper-vl.github.io/ICML-Rebuttal/.
• Metric: We report the win rate, defined as the percentage of cases where RePer’s image attention map was preferred over LLaVA-1.5’s by annotators.
• Result: Image attention maps from RePer was preferred in 70.27% of cases over LLaVA-1.5, indicating stronger alignment with human visual focus.

Q3: Discussion about RISE

Please refer to Lines 196–206 for our discussion on the connection and differences with RISE. While both methods leverage iterative supervision, RePer introduces a distinct imitation learning formulation based on reward-weighted reflective unlikelihood training. Moreover, it explicitly proposes a reflective perception mechanism aimed at addressing a key limitation of current LVLMs—the unrealistic assumption of perfect initial responses.

Q4: Correlation Between Feedback and Next-Round Response

We clarify that the next-round responses are indeed correlated with the critic feedback. As illustrated in Figure 8, during data construction, each successive response in the conversation is selected based on a higher GPT-4o score than the previous one. Concretely, the first response hallucinates non-existent objects like “bottle or vase” which the critic points out. The second response removes these errors and correctly identifies the image as a book. In the third turn, the model further corrects the authorship and improves conciseness, directly following the critic’s feedback. This step-by-step correction process demonstrates a strong alignment between the feedback and the revised responses, providing effective supervision for learning from reflection.

Thank you very much for constructive feedbacks and for recognizing our presentation quality and experimental design. We carefully address each of your concerns below.

Q1: Evaluation Reliability

To address concerns about potential bias from using the same LLM for both data construction and evaluation, we followed the suggested direction and conducted: (1) a human evaluation comparing LLM ranking with human preferences, and (2) GAPE evaluation using alternative LLMs not involved in data construction. These results collectively show:

• Our proposed GAPE evaluation aligns well with human judgment, indicating minimal bias.
• RePer consistently achieves the highest preference, both in human evaluations and in GAPE scores across different LLM evaluators.
• The strong alignment confirms GPT-4o as an effective discriminator, which provides a “discrimination to generation” gradient that guides the model to learn accurate preferences.

1. Human Evaluation

• Setup: This human study uses captions generated by three 13B models on 119 GAPE samples. For each image, three anonymized captions were randomly ordered and shown to six annotators (PhD/Master’s students in CV), who ranked them based on the GAPE criteria (Fig. 7). The interface is shown in Re-Fig.1 at https://reper-vl.github.io/ICML-Rebuttal/.
• Metric:
  • Mean Rank: The average ranking position across all cases.
  • Top-1 Rate: The percentage of times a model’s caption was ranked best by humans.
• Results: As shown below, both human evaluation metrics exhibit a consistent model ranking, aligning with the ranking from the GPT-4o-based GAPE benchmark. Notably, RePer is ranked as the top-1 in 63.87% of cases, validating its effectiveness. |Model|Mean Rank↓| Top-1↑|GAPE↑| |-|-|-|-| |LLaVA-1.5| 2.46|15.13%|77.37| |Volcano|2.01|35.29%|78.17| |RePer|1.53|63.87%|82.54|

2. Evaluation with Alternative LLMs

• To further rule out evaluator bias, we replaced GPT-4o with Claude and Gemini for GAPE scoring on 13B models. As shown below, RePer consistently outperforms baselines, reinforcing the reliability of observed improvements. |Model|Gemini↑|Claude↑|GPT-4o↑| |-|-|-|-| |LLaVA-SFT|78.55|74.64|74.88| |LLaVA-RLHF|78.89|74.18|75.36| |LLaVA-1.5|80.65|75.87|77.37| |Volcano|81.59|76.70|78.17| |RePer|83.39 |78.41|82.54|

Q2: Latency & Cost

We justify the practicality of RePer from two angles:

1. RePer already outperforms baselines with its initial response (Tab. 3), achieving better performance under the same inference cost.
2. As suggested, we evaluate RePer’s efficiency on 100 samples from the DetailCaps, measuring latency (ms/token) and cost (total generation time). As shown below, the added cost from extra turns is modest (e.g., +11 ms/token from 1 to 3 turns), and multi-turn refinement remains optional based on budget constraints or quality demands. |Model|#Turn|Gen. Token|Cost (ms)|Latency (ms/token)|CAPTURE| |-|-|-|-|-|-| |LLaVA-1.5|1|85.83|4750.0|55.34|51.23| |RePer|3|275.42|18377.5|66.73|55.55|

Q3: Reliance on Large-scale LLMs

We believe breakthroughs are possible without relying on >100B LLMs.

In RePer, core improvements stem from the reflective perception mechanism and RPL paradigm—not from reliance on large-scale LLMs. Both the reward scoring and the critic model can be implemented without large models: Fig. 2 (Step-3) illustrates rule-based scoring aligning vision and text, and Tab. 3 shows a smaller LLaVA-Critic-7B can be an effective critic.

A promising direction beyond large LLMs is to build more reliable and efficient supervision systems. Large models are convenient for data generation but often introduce hallucinations that cap smaller models’ performance. Combining expert models for pre-/post-processing, or incorporating lightweight human-in-the-loop feedback [1] for corretion/validation, offers a cost-effective way to improve supervision quality.

Another direction is to pursue efficient modeling paradigms. Recent work such as OpenAI-O1/Deepseek-R1 employ RL methods like PPO/GRPO on verified data, enabling models to better exploit internal capabilities and self-generate high-quality reasoning trajectories.

[1] Garg R et al. (2024). Imageinwords: Unlocking hyper-detailed image descriptions. Google.

Q4: Relations to Iterative Learning Strategies

RePer shares conceptual ties with active and curriculum learning: it uses feedback to refine predictions and progressively improves responses from coarse to accurate. Unlike traditional approaches, this progression is self-structured within the RePer’s own decision loop. As a future direction, integrating uncertainty-based reflection or organizing learning trajectories from simple to complex corrections could further enhance adaptability.

Q5: Reference & Typo

Thank you for the helpful references. We carefully considered them for designing our human evaluation, and will include the citations and fix the typo in the revision.