DDR: Exploiting Deep Degradation Response as Flexible Image Descriptor

We thank the reviewer for the thoughtful response to our paper. We address specific points below:

Q1. Comparison with Q-bench:

Thank you for the suggestion. There are key differences in target and methodology between the proposed DDR and Q-bench [r1] (in this response letter):

• Target: Our goal is to measure the deep degradation response in the feature space as an image descriptor, while Q-bench aims to evaluate the low-level capabilities of existing Multi-modal Large Language Models (MLLMs).

• Methodology: Our proposed method uses CLIP to encode degradation prompts and calculates the response of image deep features to these text-driven degradation representations. In contrast, Q-bench directly asks MLLMs to generate text describing low-level information within the images.

Based on your suggestion, we will emphasize these differences in our revision.

[r1] Wu H, Zhang Z, Zhang E, et al. Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Vision.

Q2. Comparison with mentioned papers:

Thank you for the helpful suggestion. Reference [r2] (in this response letter) reveals that a pre-trained Super-Resolution (SR) network is capable of extracting degradation-related representations. Similarly, [r3] (in this response letter) uses a pre-trained SR network to extract degradation representations from images with various types of degradation. In comparison, our proposed method uses the text encoder of the CLIP model to encode degradation prompts as degradation representations. We will cite and discuss [r2] and [r3] in our revision and revise lines 69 and 70 accordingly.

[r2] Liu Y, Liu A, Gu J, et al. Discovering distinctive ‘semantic’ in super-resolution networks.

[r3] Liu Y, He J, Gu J, et al. Degae: A new pretraining paradigm for low-level vision.

Q3. Network Setting for SISR:

Thank you for the helpful suggestion. This work focuses on single-image super-resolution (SISR), where low-resolution (LR) and high-resolution (HR) images are with the same resolution, whereas HAT and RCAN focus on classical SISR that expands the resolution of the input image. Based on your suggestion, we conducted experiments using SwinIR. The results are shown in Table 3 (in separate PDF file), where we observe that the proposed DDR also achieves the best performance.

Q4. Formatting of Figures and Table:

Thank you for the helpful suggestion. We will carefully reformat the results Tables and Figures.

Q5. More experiment details:

According to your suggestion, we provide more experimental details here and will also explain them in the revision. Unlike classical single-image super-resolution (SISR), we conduct experiments on real-world image super-resolution datasets that low-resolution (LR) and high-resolution (HR) images are with the same resolution. Therefore, we empirically train the model at a resolution of 128 × 128. For the learning rate, we adhere to the official settings for NAFNet and Restormer. Specifically, the initial learning rate for NAFNet is set to 1e-3, and for Restormer, it is set to 3e-4. We also adopted a cosine annealing strategy for both models.

Thank you for your thoughtful feedback. We address specific points below:

Q1. Comparison with other BIQA metrics:

Thank you for the suggestion. The proposed DDR is an Opinion-Unaware Blind Image Quality Assessment (OU-BIQA) metric, which does not require training with human-labeled Mean Opinion Score (MOS) values. In contrast, the LIQE, UNIQUE, and TreS metrics mentioned all require training with MOS values from the IQA dataset. Therefore, we compare DDR with other state-of-the-art OU-BIQA metrics to ensure a fair comparison. Based on your suggestion, we will discuss these works in our revision.

Q2. Performance comparison between DDR and LPIPS:

We address this question in the following two parts:

• Q2.1: Possible reason for DDR outperforms LPIPS

  We also find it very interesting that DDR outperforms LPIPS. The possible reasons for this may be as follows: 1) DDR leverages the rich multi-modality prior knowledge of CLIP to build joint image-text guidance as a loss for model training; and 2) DDR is a self-supervised learning objective that enhances the model's generalization ability. As a result, the model trained with DDR may perform better on the test set, which could include unseen data distributions.

• Q2.2: Does the performance gains result from a stronger visual backbone?

  As shown in Table 2 (in separate PDF file), a stronger backbone in DDR does not always lead to improved performance. For instance, RN50x16 outperforms RN50x64, and ViT-B/32 and ViT-B/16 both outperform ViT-L/14. We anticipate this is because a larger visual model may not necessarily enhance the ability to understand low-level textures. Therefore, we argue that the performance gains achieved by DDR are not due to a stronger vision backbone compared to LPIPS. For the possible reasons please refer to Q2.1.

Q3. Why DDR can assess image quality?

Thank you for the thoughtful question. We propose DDR to measure the disparity in image features after introducing degradation in the feature domain. Our findings suggest that images with less degradation (i.e., higher quality) are more sensitive to newly introduced degradation, while images with more degradation (i.e., lower quality) experience less change in their features after further degradation. As shown in Figure 1 and Figure 4 (in the original manuscript), for blurred images, a lower DDR indicates greater blurriness, while a higher DDR corresponds to clearer images. Therefore, we believe DDR effectively quantifies the degree of degradation in images, which enables the assessment of image quality. The extensive experimental results on OU-BIQA presented in Table 3 (in the original manuscript) demonstrate the effectiveness of the proposed DDR as an OU-BIQA model.

Q4. Does a high DDR indicate that the image distribution is robust to the feature interference caused by $T_d$?

We agree that there is a correlation between DDR and robustness to feature interference caused by $T_d$. However, a high DDR indicates a significant change in deep image features after introducing degradation in the feature domain, suggesting that the image features are sensitive to the interference caused by $T_d$. In contrast, a low DDR implies that the image is more robust to the interference from $T_d$.

Thanks for authors' response on the comments. Most of my concerns have been addressed.

We thank the reviewer for the thoughtful response to our paper. We address specific points below:

Q1. Clarifying of some technical details:

• Explanation of Figure 2: We measure the amount of images with different Degradation Response (DDR) values to represent the distribution of DDR. Specifically, we divided the range of DDR into multiple intervals. For each point on the curve, we measured the number of images whose DDR values fall within the corresponding interval. The horizontal axis in Figure 2 represents the numerical values of DDR, while the vertical axis represents the number of images. By adjusting the levels of handcrafted degradation, DDR demonstrates varying performance on the Opinion-Unaware Blind Image Quality Assessment (OU-BIQA) task. We conducted experiments across a range of degradation levels, selecting the level with the best performance on OU-BIQA as “optimal”. “Low” and “high” represent the lowest and highest degradation levels, respectively. When the degradation level is too low, there is only a subtle difference between the $\mathcal{F}$ and $\mathcal{F}_d$ for all images. In contrast, when the degradation level is too high, most images demonstrate an overly strong response. Using our text-driven “adaptive” strategy, DDR demonstrates a similar value distribution and performance to the manually set “optimal” degradation level. This result shows the effectiveness and flexibility of the proposed method.

• Clarification of metric M: We use the cosine distance in our method, which is defined as:

  $$\mathcal{L}\_{cos}(x, y)=1-\mathcal{S}\_{cos}(x, y),$$

  where $\mathcal{S}_{cos}(x,y) = \frac{x.y}{||x||||y||}$ represents the cosine metric or cosine similarity between $x$ and $y$. Therefore, the Ln norm and cosine distance adopted in this paper have the same meaning for low/high values. Specifically, a larger Ln norm or cosine distance implies a greater difference, indicating that the image features undergo a more significant change after introducing degradation. Conversely, a smaller Ln norm or cosine distance indicates a smaller discrepancy.

• Usage of pre-trained CLIP features: Yes, we directly use the pre-trained CLIP features. We will provide a detailed discussion in response to your Question 2. 

• Definition of degradation types: We define the degradation types according to popular papers [r2, r3] (in this response letter). Specifically, “color” degradation refers to unnatural or unpleasant color distortions, such as contrast errors. “Content” degradation refers to issues that result in unclear content, such as down-sampling and JPEG compression artifacts.

Q2. Whether DDR contains useful degradation information:

Thank you for your valuable question. We agree with you that CLIP features contain rich high-level information. However, recent works [r1, r2, r3] (in this response letter) have demonstrated that the pre-trained CLIP encoder also possesses a certain ability to understand low-level degradation features. Following these works, we propose to use the pre-trained CLIP encoder to obtain degradation representation in this paper. We acknowledge that fine-tuning CLIP with degradation prompts may lead to better performance. However, a key point of our paper is to reveal the effectiveness of degradation response, and we will explore how to better obtain degradation representations in future work.

[r1] Wang, Jianyi, Kelvin CK Chan, and Chen Change Loy. "Exploring clip for assessing the look and feel of images." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 37. No. 2. 2023.

[r2] Wu, Haoning, et al. "Towards explainable in-the-wild video quality assessment: a database and a language-prompted approach." Proceedings of the 31st ACM International Conference on Multimedia. 2023.

[r3] Zhang, Weixia, et al. "Blind image quality assessment via vision-language correspondence: A multitask learning perspective." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.

Q3. Discussion of more BIQA works:

Thank you for the clarification. We agree with you that the proposed DDR is highly relevant to non-reference image quality assessment. Specifically, our proposed DDR belongs to Opinion-Unaware Blind Image Quality Assessment (OU-BIQA), which does not require training with human-labeled Mean Opinion Score (MOS) values. Based on your suggestion, we will discuss more relevant prior papers in Section 2.

Q4. Selection of degradation prompt:

The proposed DDR is robust to the selection of degradation prompts. We demonstrate this robustness by evaluating DDR with a set of similar words on the OU-BIQA task. As shown in the Table 1 (in separate PDF file), altering positive and negative words results in only minor changes in the SRCC metric for the OU-BIQA task. This observation indicates the robustness of DDR to the choice of positive/negative words. Therefore, we simply select the words that represent each type of degradation with the best performance in our experiments.

Q5. Potential negative societal impact:

Thank you for the suggestion. We will discuss this point in our revision.