Image Super-Resolution with Text Prompt Diffusion

1. The work lacks comparison with state-of-the-art methods [1,2,3].
2. The work should conduct experiments on more real-world datasets, e.g. DRealSR dataset.
3. The work does not show how user-friendly and flexible the prompt is. To some extent, it is also flexible to directly give the user a 0-1 value as the strength of each degradation.
4. Using a text encoder to encode discrete degradations is somewhat redundant. Does the method still work when the degradation description is changed (e.g., heavy blur -> very blurry)?

[1] Scaling Up to Excellence:Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild. CVPR 2024. [2] SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution. CVPR 2024. [3] CoSeR: Bridging Image and Language for Cognitive Super-Resolution. CVPR 2024.

1. Limited Novelty: The idea of degradation-guided RealSR has been extensively explored in numerous low-level vision papers, including but not limited to:

   • Efficient and Degradation-Adaptive Network for Real-World Image Super-Resolution (DASR)
   • Textual Prompt Guided Image Restoration
   • Dcs-risr: Dynamic Channel Splitting for Efficient Real-World Image Super-Resolution
   • DaLPSR: Leverage Degradation-Aligned Language Prompt for Real-World Image Super-Resolution

2. What is the necessity of the text?: The proposed integration of degradation information into the SR network is not significantly different from the approach used in DASR. The text used in this paper, aside from providing degradation classification information, does not offer any additional value. Thus, the necessity of integrating text and a text encoder into the SR network is questionable. It is possible that using DASR’s degradation features could achieve similar results.

3. Lacks comparisons with many popular SR methods: The paper almost entirely omits comparisons with diffusion-based SR methods. There are many open-source and popular diffusion-based methods, including but not limited to:

   • [IJCV2024] Exploiting Diffusion Prior for Real-World Image Super-Resolution
   • [CVPR2024] SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution
   • [ECCV2024] Pixel-Aware Stable Diffusion for Realistic Image Super-Resolution and Personalized Stylization
   • [CVPR2024] CoSeR: Bridging Image and Language for Cognitive Super-Resolution
   • [CVPR2024] Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild
   • [CVPR 2024] SinSR: Diffusion-Based Image Super-Resolution in a Single Step This makes it difficult to be convinced of the proposed method’s superiority.

This paper appears to be outdated, as the field of RealSR has advanced rapidly. I don’t believe this paper contributes value to the field. Considering the limited novelty and the insufficient experiments, I decided to give it a rejection.

1, The proposed prompt text includes some degradation, such as Blur, Resize, Noise, Compression. The reviewer wonders to know why contains the scale factor? Is it flexible to embed the scale factor to SR model?

2, The reviewer would like to know the inference time.

3, Do the authors consider the our-of-distribution case when inference? For example, the testing image contains blur and noise, but the text prompt when inference only has 3 text prompts, e.g. blur, noise, compression.

4, Do the authors consider the our-of-distribution case when training? For example, the training text contains blur and noise, but the testing image contains 3 degradations, e.g., blur, noise, and compression.

1. The use of textual prompts in image super-resolution tasks is not novel, yet the paper lacks discussion and comparison with existing methods like PASD [1] and SeeSR [2], which also employ textual prompts.

2. In Section 3.1.2, the paper claims that textual prompts depicting degradation are superior to prompts based on image content for conditioning the denoiser network, referencing Figure 3 as evidence. However, it does not clarify how the "overall caption" result was generated, so further explanation is needed. Additionally, a comprehensive analysis and comparison with related works [1, 2] is necessary rather than relying on a single example to assert that semantic textual descriptions are redundant.

3. The paper suggests using a pre-trained MLLM for generating the degradation description for real-world super-resolution inference. But it doesn’t analyze how often the MLLM-generated prompts match the true data degradation procedure. It’s not clear why the paper assumes that the MLLM can give a good description about the image degradation in real-world use-cases. It would be beneficial to report the accuracy of the MLLM-generated prompts.

4. Given that BSRGAN [3] outperforms or closely matches the proposed method on some metrics in Table 7, the paper needs to also include BSRGAN in qualitative results.

5. The second comparison in Figure 7 includes the DAN result, while the first comparison lacks. The paper needs to make a proper qualitative comparison with all methods.

6. The paper lacks a user study both on synthetic and real-world datasets.

References

[1] Yang, Tao, Rongyuan Wu, Peiran Ren, Xuansong Xie, and Lei Zhang. "Pixel-aware stable diffusion for realistic image super-resolution and personalized stylization." arXiv preprint arXiv:2308.14469 (2023).

[2] Wu, Rongyuan, Tao Yang, Lingchen Sun, Zhengqiang Zhang, Shuai Li, and Lei Zhang. "Seesr: Towards semantics-aware real-world image super-resolution." In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 25456-25467. 2024.

[3] Zhang, Kai, Jingyun Liang, Luc Van Gool, and Radu Timofte. "Designing a practical degradation model for deep blind image super-resolution." In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4791-4800. 2021.

[4] Huang, Yan, Shang Li, Liang Wang, and Tieniu Tan. "Unfolding the alternating optimization for blind super resolution." Advances in Neural Information Processing Systems 33 (2020): 5632-5643.