FedSR: Frequency-Aware Enhancement for Diffusion-based Image Super-Resolution

1. Lack of Novelty in Motivation: The motivation presented in Figure 1 is not novel and has appeared in numerous low-level vision papers, including but not limited to:

• Exploring Fourier Prior for Single Image Rain Removal.

• Deep Fourier-based Exposure Correction Network with Spatial-Frequency Interaction

These papers are not discussed in the paper either. The insights in the contribution section (page 107) seem incremental.

2. Although FedSR+ enhances existing SR methods on metrics like CLIP-IQA, it significantly lowers fidelity on PSNR and SSIM, which is contrary to the paper’s motivation. For instance, in Table 1, using the proposed method reduces StableSR’s PSNR by 0.67dB, which is quite unacceptable.

3. Lack of Complexity Analysis: The paper does not discuss the computational complexity introduced by FedSR, such as parameters and FLOPs. If the method significantly increases computational overhead while offering only marginal improvements, it would have limited practical deployment value.

4. Potential Unrealistic Details:

• Figure 4 shows images without ground truth (GT).
• It appears that applying the proposed method may introduce “unrealistic” texture details, thus improving metrics like CLIP-IQA, which are less sensitive to fidelity, while decreasing PSNR, which is fidelity-sensitive.

5. Need for User Study: It is crucial to conduct a user study to validate that the enhanced details provided by the proposed method are “real” and consistent with the GT. Emphasizing fidelity is extremely important in SR tasks; without it, the task may as well be a generation task.
6. There are several existing papers on real-SR in the frequency domain, but the paper lacks discussion and comparison in this regard, including but not limited to:

• Image Super-resolution Via Latent Diffusion: A Sampling-space Mixture Of Experts And Frequency-augmented Decoder Approach.
• Wavelet-based Fourier Information Interaction with Frequency Diffusion Adjustment for Underwater Image Restoration.
• Waving Goodbye to Low-Res: A Diffusion-Wavelet Approach for Image Super-Resolution.

1. The authors argue that the amplitude information and high-frequency information in the image are crucial for addressing the image degradation problem through experimental analysis. However, it is questionable whether these components always play a pivotal role at different time step T in the denoising process, because the function of the diffusion model is different at different time steps.
2. Although the proposed method achieves good results in the no-reference metrics, there are negative optimization results in the visual comparison shown in Figure 4. For example: (1) within the results of StableSR+ours, DiffBIR+ours, and SeeSR+Ours concerning image1, the animal hair appears more artificial and unrealistic; (2) within the results of SeeSR+Ours concerning image1, the background area where the plants are located is incorrectly generated; (3) within the results of SeeSR+Ours concerning image4, the loss of high-frequency details pertaining to the plants in the lower left corner contrary to the viewpoint proposed in the paper.
3. How are the values of P_b and r_thresh determined in Equation 8? The authors are encouraged to conduct additional experimental analyses to enhance comprehension surrounding these parameters.

1. The observations presented in Figure 1, such as the significance of high-frequency information for image recovery, are well-known in the field and thus cannot be considered novel contributions to this study.
2. As shown in Tables 1, 2, and 3, the proposed method exhibits no improvement—and in some cases, a decline—in performance on reference-based metrics. Although pixel-aware metrics like PSNR and SSIM may not fully align with human perception, perceptual metrics such as LPIPS and DISTS should be included to better evaluate the fidelity of the results. Relying solely on non-reference metrics is insufficient for assessing recovery quality, as these metrics can be influenced by artifacts and noisy textures, which also applies to the results in Table 4.
3. Frequency-based approaches for super-resolution have become quite prevalent, as exemplified by methods like [1][2]. The lack of novelty in the frequency-focused aspect of the proposed method limits its contribution to the field.
4. In Figure 4, particularly the first, third, and fourth rows of the squirrel example in the first column, the textures of the squirrel pelts have transformed into pseudo-textures after applying the proposed method, raising further doubts about the reliability of the improved non-reference metrics.

[1] Image Super-resolution via Latent Diffusion: A Sampling-space Mixture of Experts and Frequency-Augmented Decoder Approach

[2] Frequency-Domain Refinement with Multiscale Diffusion for Super Resolution

1. There are many papers on improving image recovery from the frequency domain perspective, and frequency domain feature selection mechanisms have also been proposed.
2. The PASD results in Table 2 and Table 3 are quite different from the PASD original paper. I did not carefully check the experimental results of other methods.
3. There are already many methods for super-resolution based on diffusion models. The frequency domain algorithm proposed in this paper is just a combination of past algorithms and diffusion models, and there is no special innovation.
4. Like this method, there are already many methods that use frequency enhancement to generate details and downstream tasks. For example, FreeU: Free Lunch in Diffusion U-Net