DP²O-SR: Direct Perceptual Preference Optimization for Real-World Image Super-Resolution

We thank the reviewer 4d1m for his/her valuable feedback. For brevity, we use W# for weaknesses and Q# for questions (e.g., W1, Q1) throughout our response.

---

[W1 and W2]: Does metric overlap bias evaluation toward the optimized score rather than real perceptual gain? Lacks human evaluation.

Thank you for the valuable comments.

We acknowledge the concern that using the same metrics for both optimization (via CPS) and evaluation may introduce bias. To address this, we conducted a user study based on human perceptual judgments (Appendix Sec. 5). The results show that CPS aligns well with human preferences, and our method is consistently preferred over baselines in terms of perceived visual quality.

In addition, we have already included extensive visual comparisons in the supplementary material (Appendix Sec. 4), which further illustrate the perceptual improvements of our method.

---

[W3]: Technical novelty is interesting but lacks strong evidence.

Thank you for the insightful feedback.

While CPS and our weighting strategy are conceptually simple, they are carefully designed to address two key challenges in RealSR: (1) balancing perceptual quality and fidelity, and (2) modeling the strength of human preferences more accurately.

Beyond these, we introduce a systematic framework for Preference Pair Curation (PPC), where we study how increasing noise samples and selecting top-$N$/bottom-$N$ pairs affects the quality of offline supervision—a direction rarely explored in prior work.

To evaluate the practical impact of each component, we conducted ablation studies with Table 2 (c) (also below). The results show that each component—PPC, HSW—brings consistent CPS improvements, supporting the effectiveness of our design.

We will revise the paper to better clarify the motivation and significance of these contributions within the context of preference-based learning for low-level vision.

---

[Q1 and Q2]: Please report metric std deviations before and after fine-tuning. Miss PSNR results.

Thank you for the helpful suggestions.

1. Sensitivity to noise seed (standard deviation):
We have added a table reporting the mean and standard deviation of 13 FR and NR IQA metrics across multiple noise seeds, both before and after fine-tuning on the Syn-Test and RealSR datasets.

We observe that the standard deviations consistently decrease after applying our DP²O-SR method. This indicates that our method not only improves average perceptual quality, but also yields more stable outputs under varying noise conditions—a desirable trait for practical deployment.

2. PSNR reporting (distortion):
Although PSNR is a widely used fidelity metric, it primarily reflects pixel-level similarity and often favors overly smooth outputs. In contrast, we place greater emphasis on perceptual fidelity metrics such as LPIPS, which better capture human-perceived similarity and perceptual realism.

In response to your suggestion, we have now included PSNR and SSIM results in the table below for completeness. We believe this trade-off is reasonable: while our method may yield slightly lower PSNR, it significantly improves perceptual quality, as reflected in LPIPS, and other no-reference metrics.

These results, along with the standard deviation analysis, will be included in the revised manuscript.

---

[Q3]: Why not vary CFG?

Thank you for the insightful question. We agree that varying CFG is a natural way to generate diverse outputs.

However, we found that changing CFG introduces a predictable trade-off: higher CFG values tend to improve perceptual quality (as measured by no-reference metrics), but degrade fidelity (as measured by full-reference metrics), and vice versa. This makes it difficult to identify samples that are simultaneously strong in both aspects.

In contrast, sampling multiple outputs under the same CFG but different noise seeds allows us to capture a richer range of natural variations, including rare but valuable samples that are high in both fidelity and perceptual quality. These “lucky” samples help define the upper bound of model capability.

Since our goal is to guide the model toward outputs that balance both perceptual quality and fidelity, we fix CFG and vary the noise seed to build a more expressive and informative preference space for reward learning.

Empirically, this design is supported by the results in Table 1 (main paper), where our preference-tuned models consistently outperform baselines on both full-reference and no-reference metrics.

We will clarify this rationale in the revised manuscript.

Dear Reviewer 4d1m,

Many thanks for your time in reviewing our paper and your constructive comments. We have submitted the point-to-point responses. We appreciate if you could let us know whether your concerns have been addressed, and we are happy to answer any further questions.

Best regards,

Authors of paper #5066

We thank the reviewer Rx2M for his/her valuable feedback. For brevity, we use W# for weaknesses and Q# for questions (e.g., W1, Q1) throughout our response.

---

[W1]: Report generation cost; consider online RL (e.g., GRPO) to address DPO drift.

Thank you for the valuable comments.

For offline candidate generation, we use 8×A100 GPUs in parallel. Generating 20 samples per LR image over 4,000 images takes approximately 7 hours with C-SD2 and 18 hours with C-FLUX. After sampling, we apply IQA models to label the 80,000 generated images, which takes an additional 3 hours using the same GPU setup.

We acknowledge that DPO may introduce distribution mismatch, as the policy can deviate from the reference model used to generate training samples. While online RL methods such as GRPO may alleviate this drift, they often come with trade-offs in terms of training stability, data efficiency, and implementation complexity. We opt for an offline approach due to its simplicity and scalability, but view online RL as a promising future direction.

---

[W2]: Provide user study or correlation analysis with human judgments.

Thank you for the important question.

We have conducted a user study to validate whether CPS aligns with human perceptual preferences. As described in Appendix Sec. 5, participants were asked to compare model outputs across several methods. The results show that CPS is strongly correlated with human judgments, and our method is consistently preferred over the baselines.

This supports CPS as a reliable proxy for perceptual quality.

---

[W3]: Report PSNR/SSIM; include DRealSR to test generalization.

Thank you for the thoughtful suggestion.

We have included PSNR and SSIM results in the table below to examine the fidelity aspect of our method.

To assess generalization, we also report results on the DRealSR dataset. We observe that DP²O-SR consistently outperforms the base models (C-SD2 and C-FLUX) across most perceptual and fidelity metrics, demonstrating the robustness of our method.

The table below summarizes results on three datasets: Syn-Test, RealSR, and DRealSR. We will include this expanded evaluation in the revised manuscript.

---

[W4]: Provide more visuals to support perceptual diversity from noise variation.

Thank you for the suggestion. We will include more visual results in the next revision to better support the motivation regarding perceptual diversity from noise variation.

---

[Q1]: Can the RL strategy extend to one-step SR methods without noise inputs?

Thank you for the insightful question.

Our current method relies on stochastic generation (via noise inputs) to produce diverse candidates for offline reward modeling. Therefore, it is not directly applicable to deterministic one-step SR methods that lack inherent randomness.

However, if controlled randomness is introduced—e.g., by injecting noise during training—our RL framework could potentially be adapted to such settings. Exploring such extensions remains an interesting direction for future work.

Dear Reviewer Rx2M,

Many thanks for your time in reviewing our paper and your constructive comments. We have submitted the point-to-point responses. We appreciate if you could let us know whether your concerns have been addressed, and we are happy to answer any further questions.

Best regards,

Authors of paper #5066

We thank the reviewer Jm8D for his/her valuable feedback. For brevity, we use W# for weaknesses and Q# for questions (e.g., W1, Q1) throughout our response.

---

[W1 & Q1]: Can you provide MOS or external FR metrics to validate gains?

Thank you for the helpful comment. To mitigate the risk of circular evaluation with CPS, we include extensive qualitative comparisons (Appendix Sec. 4) and a user study (Appendix Sec. 5) based on human perceptual preferences. The study confirms that our method consistently outperforms baselines in terms of visual quality, providing strong evidence beyond the optimization metric.

We will clarify this point in the main paper to better highlight the role of human evaluation in validating perceptual improvements.

---

[W2 & Q2]: Report CPS, LPIPS, PSNR, SSIM for M=1.

Thank you for the valuable suggestion. To evaluate the model under realistic single-sample conditions, we report CPS, LPIPS, PSNR, and SSIM for the M=1 setting (i.e., one stochastic sample at inference). As shown in the table below, the results are very close to the M=20 averages reported in the main paper, indicating that our model maintains stable performance even without sampling multiple outputs.

This also aligns with our analysis in Fig. 4(a), where increasing M mainly expands perceptual diversity, while average quality remains consistent.

We will clarify this finding in the revised paper to emphasize the practicality of our method under single-sample inference.

---

[W3]: CPS and HSW are incremental novelty.

Thank you for the comment. While CPS builds on existing IQA metrics, its novelty lies in unifying full- and no-reference scores into a normalized preference signal tailored for Real-ISR—a setting where such integration has not been explored. This allows leveraging complementary strengths of multiple IQA metrics.

HSW is inspired by importance sampling, but we design it specifically to balance intra-pair contrast and inter-group diversity within limited candidate sets—an essential factor for stable and effective DPO training in Real-ISR tasks.

Although both components draw from existing ideas, their integration into a unified preference-driven framework, along with demonstrated consistent improvements across models and benchmarks, represents a practical and impactful contribution. We will clarify this in the revised version.

---

[W4]: Preference alignment in diffusion has been explored (e.g., Diff-DPO, VideoDPO); this adaptation seems incremental.

Thank you for the thoughtful comment.

While our work is inspired by preference-based alignment frameworks like Diff-DPO and VideoDPO, we believe our contributions go beyond a direct adaptation and address challenges unique to RealSR:

1. Task-specific reward design: RealSR demands a careful balance between perceptual quality and fidelity. We propose CPS, a unified reward that combines full- and no-reference IQA metrics—unlike prior work that focuses on semantic alignment (e.g., text-image relevance).

2. Systematic study of preference data construction: We analyze how preference quality varies with sampling count and selection strategies (e.g., top-N vs. bottom-N), which is largely unexplored in prior diffusion-based alignment literature.

3. Hierarchical Significance Weighting (HSW): Our weighting scheme accounts for both intra-pair contrast and inter-group variance, offering a more structured and informative signal than the binary win/loss formulation in standard DPO.

Together, these elements constitute a meaningful extension of preference-based alignment methods to low-level vision tasks. We will clarify these distinctions in the revised manuscript and better highlight the novelty relative to prior work.

---

[W5]: No significance tests or seed variance reported.

Thank you for pointing this out.

To address the concern about variability and robustness, we report the mean and standard deviation (± std) of 13 IQA metrics across 20 random seeds. The table below provides a direct measure of output stability under stochastic sampling. All models use the same set of seeds for fair comparison.

While we did not conduct formal significance tests, we believe that reporting seed-based variance serves a similar purpose—demonstrating that the improvements are consistent and robust rather than due to random chance. We will clarify this intent in the revised manuscript.

---

[Q3]: Do CLIP-based CPS components favor oversharpened or hallucinated details?

Thank you for the insightful question.

We have observed that CLIP-based components in CPS (e.g., CLIPIQA+, Q-Align) can indeed favor oversharpened or hallucinated textures when used in isolation.

To address this, CPS integrates both FR and NR IQA metrics. The FR components serve to anchor the reward signal in ground-truth consistency, effectively mitigating the tendency toward hallucinated or overly sharp artifacts.

We will include visual examples in the revised version to illustrate how CPS achieves a better balance between perceptual quality and fidelity.

---

[Q4]: Ablate HSW into intra-pair and inter-group to isolate contributions.

Thank you for the helpful suggestion.

We have conducted separate ablation studies to isolate the effects of the two components in HSW:

• (i) Intra-pair only (IP)
• (ii) Inter-group only (IG)

The table below shows the CPS score under different configurations:

We find that both components individually contribute to performance gains, and their combination yields the best result.

We will include this ablation in the revised manuscript to clarify the individual contributions of intra-pair and inter-group weighting.

---

[Q5]: Please provide justification of CPS composition and normalization.

Thank you for the helpful question.

CPS is designed to balance fidelity and perceptual quality by combining both FR and NR IQA metrics:

• FR metrics (e.g., LPIPS, TOPIQ-FR, AFINE-FR) emphasize structural and content fidelity.
• NR metrics (e.g., CLIPIQA+, MUSIQ, Q-Align) encourage perceptual realism and aesthetics.

Using only FR tends to suppress fine details, while NR-only optimization may lead to hallucinated or oversharpened artifacts. The combination helps avoid these extremes.

To ensure fair contribution from each metric, we apply min-max normalization across each comparison set, aligning their numeric ranges. Additionally, we assign equal weights to the FR and NR groups, rather than individual metrics, to avoid imbalance due to differing group sizes.

Metrics such as PSNR and SSIM are excluded from CPS due to their known poor correlation with human perception.

Due to rebuttal format constraints, we are unable to include illustrative examples here, but we will provide detailed visual comparisons in the revised version to further justify the design of CPS.

Dear Reviewer Jm8D,

Many thanks for your time in reviewing our paper and your constructive comments. We have submitted the point-to-point responses. We appreciate if you could let us know whether your concerns have been addressed, and we are happy to answer any further questions.

Best regards,

Authors of paper #5066

We thank the reviewer iowj for his/her valuable feedback. For brevity, we use W# for weaknesses and Q# for questions (e.g., W1, Q1) throughout our response.

---

[W1, Q1 and Q3]: Why use all IQA metrics? Why not only use FR or NR metrics as reward? Why are FR and NR metrics given equal importance in CPS formulation?

We thank the reviewer for the insightful questions.

We do not use all IQA metrics indiscriminately. Instead, we carefully select 3 full-reference (FR) and 6 no-reference (NR) metrics commonly used in perceptual quality assessment.

FR metrics assess fidelity and structural similarity to ground truth, but relying solely on them can produce less vivid outputs. NR metrics promote perceptual quality without reference, encouraging more appealing visuals. However, NR metrics alone may lead to reward hacking or favor unrealistic generations—a concern also raised by Reviewer Jm8D (Q2) regarding CLIP-based NR metrics.

To investigate further, we conducted ablation studies using only FR or only NR metrics in CPS. As shown below, FR-only rewards improve fidelity but hurt perceptual quality, while NR-only rewards do the opposite. Combining both gives a better overall balance:

To avoid imbalance from the unequal number of FR and NR metrics, we assign equal total weight to each group in the CPS formulation. This prevents overemphasizing the NR group and ensures that both fidelity and perceptual quality are equally represented during optimization.

We agree that visual examples would help illustrate these trade-offs. Due to rebuttal constraints, we will include qualitative comparisons in the final version.

---

[W2]: Why are CPS scores compared across models if CPS is defined as relative within a set?

Thank you for the thoughtful comment and for highlighting this potential confusion.

To clarify, CPS is not computed by normalizing each model’s outputs independently. Instead, for each input image, we collect outputs from all models under comparison (e.g., those in the same table), and apply min-max normalization jointly across this full set.

This ensures CPS reflects the relative perceptual quality across models, making scores directly comparable within the same evaluation group. Thus, CPS values in Tables 1–3 are valid for intra-table comparison, but not across different tables or experiments.

We appreciate the reviewer’s attention and will clarify this computation procedure in the revised manuscript.

---

[W3]: HSW lacks detail in the main paper, relying on the supplement.

Thank you for the suggestion. We will add the key HSW steps and equations to the main paper to improve clarity and self-containment.

---

[W4]: The purpose of Figure 3a-b is unclear, and the downward arrow is unexplained.

Thank you for the feedback. We will revise Figure 3 to clarify its message: (a) shows that increasing sampling times during preference data construction improves performance with best/worst-of-1 pairs; (b) illustrates a trade-off between data quality and quantity when using best/worst-of-N, with optimal performance at a moderate N.

The downward arrow indicates increased sampling during the preference data collection stage.

---

[Q2]: Which IQA metric aligns most closely with CPS?

Thank you for the thoughtful question. We analyzed which IQA metrics align most closely with CPS and whether a reduced metric set could suffice for reward design.

CPS is computed as the average of normalized FR and NR metrics. To ensure equal group contributions, each FR metric is weighted 1/6 and each NR metric 1/12. The table below shows the normalized scores of our model (DP²O-SD2), the weights, and each metric’s contribution:

Among FR metrics, AFINE-FR, LPIPS, and TOPIQ-FR contribute most to CPS. For NR metrics, MUSIQ, TOPIQ-NR, and QALIGN are the top contributors.

To test if a reduced set suffices, we conducted ablations using only the top-k metrics from each group. This reflects the RealSR task’s need to balance fidelity (FR) and perception (NR). Results are shown below:

While CPS is most influenced by a few strong metrics, using only top-k metrics consistently underperforms compared to using all. This suggests that the additional metrics provide complementary, non-redundant signals, supporting the need for a diverse set in reward design.

We will include this analysis and tables in the revised manuscript.

---

[Q4]: What's the performance drop when replacing HSW with random pairing?

Thank you for the helpful question.

Our HSW strategy builds on the PPC sampling framework:

• The baseline uses a fixed top-1 vs. bottom-1 pairing.
• PPC introduces diversity by randomly pairing samples within top-N and bottom-N candidates.
• HSW further improves this by assigning higher weights to more informative pairs during training.

As shown in Table 2(c), PPC improves CPS from 0.626 (baseline) to 0.632, and adding HSW further raises it to 0.641.

We do not perform fully random pairing (i.e., sampling any two outputs arbitrarily), as our method relies on reward-ranked candidates to ensure meaningful preference signals.

We will clarify this distinction and the ablation structure in the revised manuscript.

---

[Q5 & Q6]: How do intra-pair strategies and class imbalance affect model performance? Is a simple top-1 vs. bottom-N pairing sufficient?

Thank you for the insightful questions.

For Q5:
We tested a top-1 vs. bottom-N pairing strategy, where the best sample is paired with multiple negatives. As shown in the table (e.g., Top1+Bottom10, CPS = 0.685), this setup performs reasonably well but consistently lags behind our full method (CPS = 0.703).

This suggests that fixing the anchor to top-1 limits intra-pair diversity. In contrast, sampling from top-N and bottom-N introduces richer variation, which benefits preference learning.

For Q6:
We agree that imbalance can bias training. To mitigate this, we maintain symmetry between positive and negative pools (top-N vs. bottom-N). While exhaustive testing is infeasible, our experiments with asymmetric settings (e.g., Top1+Bottom4/7/10) show diminishing gains, supporting the effectiveness of balanced sampling.

Overall, our strategy balances informativeness and robustness, which we find crucial for stable optimization. We will clarify these points in the revised version.

Dear Reviewer iowj,

Many thanks for your time in reviewing our paper and your constructive comments. We have submitted the point-to-point responses. We appreciate if you could let us know whether your concerns have been addressed, and we are happy to answer any further questions.

Best regards,

Authors of paper #5066