Towards Ultra-High-Definition Image Deraining: A Benchmark and An Efficient Method

We appreciate the efforts and valuable suggestions provided by the reviewer. We will address the concerns outlined below:

[W1]

According to the reviewer's suggestion, we have added a section to discuss with the closely-related method (see Appendix A.6). Compared to the Global Feature Modulation Layer (GFML) in MixNet, our SFRL has several main differences: (1) Flexible Rearrangement: Unlike the fixed swapping of adjacent elements in GFML, SFRL rearranges the positions of all elements in the 3D feature map during each dimension transformation. This allows our method to flexibly capture global spatial information. (2) Multi-Scale Integration: While GFML operates at a single scale, SFRL is embedded within a multi-scale encoder-decoder network backbone, enabling more effective exploration of multi-scale information, which is crucial for image deraining. (3) Enhanced UHD Restoration: As discussed in Section 4.3 of the main paper, MixNet directly downsamples UHD images to learn spatial features, leading to detail loss. Our method addresses this issue by introducing the Frequency-aware Feature Modulation Layer (FFML), which enhances UHD image restoration quality through joint learning in both spatial and frequency information.

[W2]

Thank you for the reviewer's suggestions. We have added more implementation details for dataset construction in the revised version (see Appendix A.1).

We model the generation of rain streak layers as a motion blur process, leveraging two essential characteristics of rain streaks: their repeatability and directionality. Formally, this can be represented as:

where $\mathbf{N}$ represents the rain mask derived from random noise $n$, where we utilize uniform random numbers combined with thresholds to adjust the noise level. The parameters $l$ and $\theta$ correspond to the length and angle of the motion blur kernel $\mathbf{K} \in \mathbb{R}^{p \times p}$. Additionally, a rotated diagonal kernel is applied with Gaussian blur to control the rain streak thickness $w$. The values for noise quantity $n$, rain length $l$, rain angle $\theta$, and rain thickness $w$ are sampled from the ranges $[100,500]$, $[10,40]$, $[0^{\circ},60^{\circ}]$, and $[3,7]$, respectively. The symbol $*$ denotes the spatial convolution operator.

We utilize alpha blending to combine the rain layer with the background layer. Here, the alpha value of each pixel in a layer determines the extent to which colors from underlying layers are visible through the current layer's color. Mathematically, this process is expressed as:

where $\alpha$ represents the blending ratio, which is empirically set within the range $[0.8, 0.9]$. The symbol $\odot$ indicates element-wise multiplication. The $R$, $G$, and $B$ channels are handled independently during the process. We show an example of synthesizing rainy images with different blending ratios in Figure 8 in Appendix A.1. It is evident that when the blending ratios are set to 0.8 and 0.9, the rain streaks become invisible in the white sky region, resulting in greater harmony with real rainy images.

[W3]

According to the reviewer's suggestion, we calculate the runtime of our method and recent image deraining methods on 4K images. We conduct on a machine equipped with an NVIDIA RTX 4090 GPU. The table below shows that our method achieves lower inference time. We have added this result in the revised paper.

[Q1]

The visual examples compared with MAXIM are intended to evaluate deraining performance on real rainy scenes, rather than on the low-resolution Rain13k dataset. Here, we further provide the quantitative results of our method and MAXIM on real-world image deraining in Table 6. Furthermore, more visual examples are included in Figure 15. These quantitative and qualitative results demonstrate that our method exhibits better generalization capability to real-world scenarios. We have added these results in the revised version.

[Q2]

As stated on L322-323, for all deep models, we use an NVIDIA TESLA V100 GPU with 32GB memory to test UHD images. Our experiments reveal that methods such as JORDER-E, RCDNet, SPDNet, Restormer, and DRSformer are unable to infer full-resolution results on UHD images. This limitation arises from their high memory consumption and computational complexity. Due to the diverse architectures of deep learning networks, it is challenging to define a precise threshold for determining whether a model is unsuitable for UHD inference. For models that cannot directly process UHD images, we can adopt a splitting-and-stitching strategy to overcome this limitation, enabling them to handle ultra-high-resolution inputs.

[Q3]

Yes. For fair comparison, all models are trained for an equal number of epochs, and we select the weights from their final training epoch for testing purposes. We have clarified this in the revised version.

[Q4]

All methods are used with their respective default settings. Following previous UHD studies [1,2], we do not adjust the parameters of the existing models to ensure a fair comparison. We have clarified this in the revised version.

[1] Zheng et al. Ultra-High-Definition Image Dehazing via Multi-Guided Bilateral Learning, CVPR 2021

[2] Li et al. Embedding Fourier for Ultra-High-Definition Low-Light Image Enhancement, ICLR 2023

[Q5]

According to the reviewer's suggestion, we have provided a failure case in Appendix A.10. As shown, our method is able to effectively remove rain streaks but fails to handle the presence of fog-like rain accumulation in real rainy scenes. In addition, the splashing effect of raind on the ground is also worth paying attention to in future work, which is crucial for downstream autonomous driving.

We appreciate the efforts and valuable suggestions provided by the reviewer. We will address the concerns outlined below:

[W1]

Thank you for the reviewer's suggestions. We have added more implementation details for dataset construction in the revised version (see Appendix A.1). We will open-source the code for our data synthesis process to serve as a reference for the community.

The motivation for introducing geometric transformation operation comes from real-world observations. In the physical world, rain streaks are naturally fine and elongated, reflecting their slender geometry and high-speed movement. In high-resolution 4K images, with the increased pixel density and more detailed spatial representation, these characteristics can be captured more accurately. Each rain-affected region is represented with greater precision, allowing the streaks to appear longer and thinner, consistent with their real-world appearance. Conversely, in low-resolution images, the reduced pixel information limits the ability to preserve these fine details. Rain streaks that are physically the same length may appear shorter and thicker due to the loss of spatial resolution and the effects of blurring. This results in a coarser and more ambiguous depiction of rain streaks, where fine structures and elongated shapes are less distinguishable. This disparity highlights the importance of resolution in faithfully representing rain streaks and motivates the development of high-resolution synthesis approaches.

We first generate an initial rain streak layer, then perform geometric transformation using scaling operations, and finally use alpha blending techniques to synthesize the rainy images. Specifically, we model the generation of rain streak layers as a motion blur process, leveraging two essential characteristics of rain streaks: their repeatability and directionality. Formally, this can be represented as:

where $\mathbf{N}$ represents the rain mask derived from random noise $n$, where we utilize uniform random numbers combined with thresholds to adjust the noise level. The parameters $l$ and $\theta$ correspond to the length and angle of the motion blur kernel $\mathbf{K} \in \mathbb{R}^{p \times p}$. Additionally, a rotated diagonal kernel is applied with Gaussian blur to control the rain streak thickness $w$. The values for noise quantity $n$, rain length $l$, rain angle $\theta$, and rain thickness $w$ are sampled from the ranges $[100,500]$, $[10,40]$, $[0^{\circ},60^{\circ}]$, and $[3,7]$, respectively. The symbol $*$ denotes the spatial convolution operator.

We utilize alpha blending to combine the rain layer with the background layer. Here, the alpha value of each pixel in a layer determines the extent to which colors from underlying layers are visible through the current layer's color. Mathematically, this process is expressed as:

where $\alpha$ represents the blending ratio, which is empirically set within the range $[0.8, 0.9]$. The symbol $\odot$ indicates element-wise multiplication. The $R$, $G$, and $B$ channels are handled independently during the process. We show an example of synthesizing rainy images with different blending ratios in Figure 8 in Appendix A.1. It is evident that when the blending ratios are set to 0.8 and 0.9, the rain streaks become invisible in the white sky region, resulting in greater harmony with real rainy images.

[W2]

Thank you to the reviewer for pointing out the issue with the reference format. We have updated the reference format in the revised version.

We appreciate the efforts and valuable suggestions provided by the reviewer. We will address the concerns outlined below:

[W1]

According to the reviewer's suggestion, we calculate the #FLOPs on 4K images. The table below shows that our method achieves lower FLOPs.

[W2]

According to the reviewer's suggestion, we calculate the runtime of our method and recent image deraining methods on 4K images. We conduct on a machine equipped with an NVIDIA RTX 4090 GPU. The table below shows that our method achieves lower inference time. We have added this result in the revised paper.

[W3]

For some methods (JORDER-E, RCDNet, SPDNet, Restormer, and DRSformer) that are unable to infer full-resolution results on UHD images, existing UHD studies [1,2] typically adopt two strategies: (1) downsample the input to the largest size that the model can handle and then resize the result to the original resolution, (2) split the input into multiple patches and then stitch the result. Researchers in [1] demonstrate that employing the second strategy often leads to undesired boundary artifacts in the output images, as it does not account for the complete structure of the image. Thus, we adopt the first strategy in the original manuscript.

According to the reviewer's suggestion, we further adopt the splitting-and-stitching strategy to evaluate the deraining performance. The table below shows that our method still achieves the best quantitative results, thanks to its ability to perform direct inference on full UHD images. We have updated these results in the revised paper.

[1] Zheng et al. Ultra-High-Definition Image Dehazing via Multi-Guided Bilateral Learning, CVPR 2021

[2] Li et al. Embedding Fourier for Ultra-High-Definition Low-Light Image Enhancement, ICLR 2023

[W4]

According to the reviewer's suggestion, we further employ the advanced non-reference IQA metric MANIQA to evaluate the quality of different deraining images. The table below shows that our method achieves better perceptual quality with higher MANIQA values.

[W5]

We compare the performance of our model trained on the proposed dataset and existing low-resolution dataset (Rain13k) and then test on real-world 4K rainy images. We have provided the comparison results in Appendix A.4 of the revised version. It can be observed that the model trained on our 4K-Rain13k performs better on real-world rainy images, indicating that our dataset effectively reduces the domain gap and enables the model to generalize better to real-world scenarios.

Thanks to the authors for the answer. The authors have addressed my concerns well, and I consider the paper acceptable.

[W3]

As stated on L161-186, we find that the geometric inconsistency in the synthesis of low resolution and high-resolution rainy images, with noticeable discrepancies in the length and thickness of rain streaks. Conventional methods may struggle to preserve realistic attributes such as texture, position, and perspective relationships at higher resolutions. To overcome this, our geometric transformation approach explicitly considers these factors, ensuring that the generated rain streaks align naturally with the scene and exhibit proper perspective and scaling characteristics.

As stated on L159, to mitigate the appearance of rain streaks as a separate layer, we incorporate alpha blending into the synthesis process. This blending technique ensures a smooth integration of rain streaks with the background, preserving the physical realism of rain’s interaction with the scene (e.g., partial occlusion, transparency effects). We have added more details for dataset construction in the revised version (see Appendix A.1).

In Appendix A.3, we have provided a quantitative analysis of the proposed dataset. Specifically, we adopt the Kullback-Leibler Divergence (KLD), also known as relative entropy, to measure the difference between two probability distributions (i.e., synthetic image and real-world image). Figure 10 presents the comparison results of the representative synthetic benchmarks and our benchmark, showing that our 4K-Rain13k is close to the distribution of real-world rainy images. The reason behind this is that 4K-Rain13k fully considers the realism and diversity of the rain patterns, thereby narrowing the domain gap between synthetic and real images.

[W4]

Thank you for the reviewer's suggestions. We compare the performance of our model trained on the proposed dataset and existing low-resolution dataset (Rain13k) and then test on real-world 4K rainy images. We have provided the comparison results in Appendix A.4 of the revised version. It can be observed that the model trained on our 4K-Rain13k performs better on real-world rainy images, indicating that our dataset effectively reduces the domain gap and enables the model to generalize better to real-world scenarios.

We appreciate the efforts and valuable suggestions provided by the reviewer. We will address the concerns outlined below:

[W1]

I'm sorry for any confusion caused. For some methods (JORDER-E, RCDNet, SPDNet, Restormer, and DRSformer) that are unable to infer full-resolution results on UHD images, existing UHD studies [1,2] typically adopt two strategies: (1) downsample the input to the largest size that the model can handle and then resize the result to the original resolution, (2) split the input into multiple patches and then stitch the result. Researchers in [1] demonstrate that employing the second strategy often leads to undesired boundary artifacts in the output images, as it does not account for the complete structure of the image. Thus, we adopt the first strategy in the original manuscript.

According to the reviewer's suggestion, we further adopt the splitting-and-stitching strategy to evaluate the deraining performance. The table below shows that our method still achieves the best quantitative results, thanks to its ability to perform direct inference on full UHD images. We have updated these results in the revised paper.

[1] Zheng et al. Ultra-High-Definition Image Dehazing via Multi-Guided Bilateral Learning, CVPR 2021

[2] Li et al. Embedding Fourier for Ultra-High-Definition Low-Light Image Enhancement, ICLR 2023

[W2]

Firstly, directly training on full 4K images poses significant challenges due to the limited memory capacity of current GPU devices. In image restoration, scaling up the resolution leads to prohibitively high GPU memory consumption, making it infeasible to process such high-resolution data in an end-to-end training framework. Therefore, training on small patches is a widely adopted and practical approach in the field. In our experiment, we choose $768 \times 768$ as the maximum patch size based on GPU memory limitations.

Secondly, although the network is trained on patches, our dataset is constructed using full 4K-resolution images. As stated on L177-180, we fully consider the unique attributes of rain streaks on 4K high-resolution images during the dataset construction process. Training on patches sampled from this dataset enables the network to effectively learn these features, as the rain streak patterns are preserved within the patches.

Thirdly, while the network is trained on patches for computational efficiency, the testing and evaluation are conducted on full 4K images. This ensures that the model's performance is directly assessed on the target resolution, demonstrating its ability to handle UHD image.