Random Is All You Need: Random Noise Injection on Feature Statistics for Generalizable Deep Image Denoising

Thank you so much for your insightful comments and the recognition of our approach. We greatly appreciate the time and effort you've invested in reviewing our paper. We are eager to address the concerns raised by the reviewer as outlined below and are open to further discussions to resolve any remaining issues. The issues are renumbered as 1, 2, 3, 4, etc., according to their original order presented by the reviewer.

A1 $\rightarrow$ W1 & W2: Thank you for your suggestion. The additional ablation study is included in Appendix A.2, where we adjusted the intensity of the random noise injected into feature statistics by applying a scaling factor. We plan to include LPIPS in the tables in the altered version.

A2 $\rightarrow$ Q1: Regarding this key question raised by the reviewer, we recently conducted preliminary exploratory experiments to address this aspect. Specifically, we trained our RNINet on Gaussian noise first, followed by training on Poisson noise, and refer to this version as RNINet-g2p. The total number of iterations for RNINet-g2p was kept identical to the iterations used for training RNINet exclusively on Gaussian noise (referred to as RNINet-original). While we observed a very few cases with improvements, the overall results showed a significant performance drop, as presented below:

In conclusion, we believe that training solely on Gaussian noise could obtain overall satisfactory results. However, there is considerable potential for further exploration into the optimal combination of training noise types to achieve the best results. This remains an interesting and open topic that we are actively considering. We are optimistic that future research will address this phenomenon in greater depth, both theoretically and experimentally.

We hope we have adequately addressed all the reviewer's concerns through this rebuttal. Should there be any issues that remain unaddressed, please let us know, we look forward to further discussions with you.

Best regards,

Authors of #1857

A3 → W3: Our work does not aim to address MT's limitations from a purely theoretical perspective. Instead, we tackle these limitations through a three-step approach: observation, model design with explanation, and experimentation.

1. Observation: We observed that MT tends to oversmooth image content, a side-effect that results in the loss of high-frequency details and a drop in PSNR (L52–L53 and Fig. 1).

2. Model Design and Explanation: We proposed RNINet, a framework fundamentally different from masked training. In Appendix A.3, we provide a theoretical explanation demonstrating the rationale and effectiveness of our design, proving that the key components of our approach are well-founded.

3. Experimentation: We conducted thorough experiments to address MT's limitations. The significant performance improvements achieved by our method substantiate its success in overcoming the challenges posed by MT.

A4 → W4 & W2 (partially): Thank you for your notification and suggestion. We find that in Fig. 5, some differences require zooming in to be noticeable. Considering the significant PSNR/SSIM improvements achieved by our RNINet, including metrics like PSNR directly below each image is a very good suggestion. This would enhance the clarity and persuasiveness of the visual comparisons. We plan to apply this improvement to both Fig. 5 and Fig. 8, while also including more cases in Fig. 8. We are actively working on these updates and will reflect them in the final version of our manuscript.

We hope we have adequately addressed all the reviewer's concerns through this rebuttal. Should there be any issues that remain unaddressed, please let us know, we look forward to further discussions with you.

Best regards,

Authors of #1857

Thank you so much for your insightful comments and for recognizing the superior performance of our approach. We greatly appreciate the time and effort you've invested in reviewing our paper. We are eager to address the concerns raised by the reviewer as outlined below and are open to further discussions to resolve any remaining issues. The issues are renumbered as 1, 2, 3, 4, etc., according to their original order presented by the reviewer.

A1 → W1: Thank you for your comments. We would like to clarify that, to the best of our knowledge, our method is the first to introduce feature-level noise injection to enhance the generalizable performance of image denoising. Compared with previous methods, our method consistently outperforms them in both denoising performance and efficiency, with significant improvements. It is worth noting that, with the rapid evolution of research, discovering entirely unseen basic elements has become extremely hard. Consequently, the novelty in research should be increasingly recognized for its well-articulated motivation and innovative perspective. For instance, even renowned works like GANs, VAEs, and their applications often share similar basic components. However, this does not lessen the novelty of these studies due to their superior performance and the fresh perspectives they bring to their target tasks. We sincerely hope the reviewer will consider the novelty aspect of our work based on these points, which we believe align with the standards of most existing publications.

A2 → W2 & Q1: Thank you for recognizing that our method outperforms other approaches in the SIDD evaluation. The zero-shot denoising is a highly challenging task on generalization capability and different from pipelines trained and tested on the same dataset (e.g., training on SIDD and testing on SIDD), these models can achieve exceptionally high quantitative metrics on specific datasets but they are overfitting them. For example, while the SIDD-pretrained Restormer model achieves around 40 dB PSNR on the SIDD testset, its performance drops significantly when simply tested on Speckle Noise, as shown in the table below:

These results show that the performance of the SIDD-pretrained Restormer is also relatively poor compared to its 40 dB PSNR on the SIDD testset. Therefore, the outcomes heavily depend on the specific scenarios and how we treat the results in a fair and consistent manner. In the zero-shot generalization scenario (where model has no prior exposure to SIDD), we think our performance is satisfactory, as it surpasses previous methods. We are optimistic that future methods will continue to improve upon these results and achieve even greater performance.

For additional experimental results on other real-world benchmarks, we would like to align with the suggestion from Reviewer Xna9, who brought LAN [1] to our attention. In LAN [1], two zero-shot denoisers are introduced and evaluated on two real-world datasets, Poly and CC (Nam), our work also targets this scenario. Following the experimental setup outlined in LAN [1], we compare our performance with the two zero-shot denoisers and the previous SOTA method, MT. We think this comparison provides a more convincing evaluation. The results are as below:

We believe the results above provide additional evidence of our method's strong generalization capability in handling zero-shot real-world denoising conditions. The DND dataset doesn't provide ground truth data but use online service for testing (we met a lot of troubles and request limitations), we currently have obtained results that: RNINet (34.85/0.8391) and MT (33.75/0.8310), with our method outperforming the previous SOTA, MT, by a significant PSNR margin (+1.1db).

[1] LAN: Learning to Adapt Noise for Image Denoising. CVPR 2024.

[2] Real-world noisy image denoising: A new benchmark. ArXiv 2018.

[3] A holistic approach to cross-channel image noise modeling and its application to image denoising. CVPR 2016.

Dear Reviewer kYmE,

Greetings!

We would like to inform you that we have updated several figures and tables in our new version based on your suggestions. For example, we included more showcases in Figure 8 and added PSNR values below the images in Figure 5 to enhance the clarity and persuasiveness of the visual comparisons.

Please refer to Appendix A.5 for details. You will find new figures and tables highlighted with "Supplementary for Rebuttals." Kindly review them for further information. Together with the previous rebuttal contents, we hope we have adequately addressed all of your concerns. Should there be any remaining issues, please let us know. We look forward to further discussions with you.

Best regards,

Authors of #1857

Dear Reviewer kYmE,

Thank you very much for your valuable suggestions. We have carefully followed your recommendations and incorporated the corresponding changes in both our rebuttal and the revised manuscript. We hope your concerns have been thoroughly addressed.

As you may be aware, the discussion period is nearing its deadline, leaving us with very limited time for further exchanges. We would greatly appreciate it if you could take a moment to review our responses. If you find our updates satisfactory, we kindly hope you might consider updating your score.

Should you have any additional concerns or require further clarification, we are more than happy to address them promptly.

Thank you once again for your thoughtful feedback and support.

Best regards,

Authors of #1857

Thank you so much for your thoughtful comments and recognition of the innovative perspective and superior performance of our work. We greatly appreciate the time and effort you've invested in reviewing our paper. We are eager to address the concerns raised by the reviewer as outlined below and are open to further discussions to resolve any remaining issues.

A1 $\rightarrow$ W1: Thank you for your suggestion. We have conducted experiments on two additional real-world noise datasets, PolyU [2] and Nam [3], aligned with the suggestion from Reviewer Xna9, who brought LAN [1] to our attention. In LAN [1], two zero-shot denoisers were introduced to evaluate performance under zero-shot real-world denoising conditions. Following the experimental setup outlined in LAN [1], we compare the performance of these two zero-shot denoisers with the previous state-of-the-art method, MT, and our proposed method. The results are presented below:

We believe these results provide additional evidence of our method's strong generalization capability in handling zero-shot real-world denoising conditions.

[1] LAN: Learning to Adapt Noise for Image Denoising. CVPR 2024.

[2] Real-world noisy image denoising: A new benchmark. ArXiv 2018.

[3] A holistic approach to cross-channel image noise modeling and its application to image denoising. CVPR 2016.

A2 $\rightarrow$ Q1: For this question, we would like to provide our perspectives from two steps (Q&A).

1. Why do current renowned methods (SwinIR, Restormer, DRUNet, etc.) tend to conduct extensive evaluations on Gaussian noise rather than other types of noise?

Interestingly, many widely used models provide pretrained versions specifically for Gaussian noise, accompanied by extensive experiments. In our view, this preference can be attributed to two potential reasons:

i) In many natural and technical processes, noise can often be regarded as the accumulation of multiple small effects. According to the Central Limit Theorem, when the sum of independent random variables becomes sufficiently large, their distribution converges to a normal (Gaussian) distribution.

ii) Gaussian noise is characterized by a mean of 0 and a fixed variance, with a uniform spectrum across all frequencies. This means Gaussian noise can randomly and uniformly affect all inputs to a model, making it more robust to small input variations and enhancing its generalization capability. In other words, Gaussian noise is inherently more challenging to remove compared to other noise types, which may explain its frequent use in evaluations.

2. Can Gaussian denoisers address the complexities of real noise?

Real-world noise encompasses a diverse range of conditions and complexities, which are not fully captured by datasets like SIDD, PolyU, or Nam, as these primarily focus on camera sensor noise. In practice, other types of noise, such as Speckle noise, Poisson noise, and Gaussian noise, also occur in real-world scenarios. General Gaussian denoisers are optimal for Gaussian noise but can address the complexities of real noise only moderately well, particularly in a zero-shot generalization context. However, adjusted and generalizable Gaussian denoisers (i.e. RNINet, MT) can offer improved zero-shot solutions due to their robust generalization capabilities, making them better equipped to handle various forms of complex real-world noise. We are optimistic that there will be future research to thoroughly explore this intriguing question through both theoretical analysis and experimental validation.

We hope we have adequately addressed all the reviewer's concerns through this rebuttal. Should there be any issues that remain unaddressed, please let us know, we look forward to further discussions with you.

Best regards,

Authors of #1857

Dear Reviewer 41dF,

Thank you for your valuable suggestions, recognition, and continued support of our work. In response to your comments, we have conducted additional experiments on two real-world datasets, PolyU and Nam, and compared them with two zero-shot denoisers in LAN. These experiments demonstrate the superior performance of our methods compared to previous approaches under very practical conditions. Furthermore, we have elaborated on your question regarding the use of Gaussian denoisers for real noise. For additional insights on this issue, you may also refer to the findings in our rebuttal to Reviewer YmqT, who found them quite interesting. We hope our rebuttal effectively addresses your concerns and enhances our original explanations.

As you may be aware, the discussion period after extension is nearing its deadline, leaving us with limited time for further exchanges. Your feedback is immensely valuable to us, and we are eager to hear your thoughts. If you find our updates satisfactory, sincerely, we wonder if you could consider updating/raising your score/confidence to support our work?

Should you require any further clarification or additional information, please don’t hesitate to reach out.

Thank you again for your support and consideration.

Best regards,

Authors of #1857

A5 → Q3 & Q4: Thank you so much for bringing this brilliant work of LAN [1] to our attention. Upon detailed review of LAN [1], we found that it aims to bridge the noise distribution gap across various real-world denoising datasets to enhance generalization capabilities. Given that these datasets may have closely related noise distributions, LAN [1] also explores zero-shot denoising in Section 4.3 where the training and test noise distributions significantly differ. Our work also targets this scenario, thus we included comparisons with two zero-shot denoisers from LAN [1] alongside the previous SOTA method MT in our evaluations, we follow the experimental setup in LAN [1] to process PolyU [2] and Nam [3] testset, the results are presented as below:

Our method demonstrated superior zero-shot generalization performance compared to MT and two zero-shot denoisers, which we believe provides strong evidence that our approach can generalize well to real-world noise. We would like to cite LAN [1] and other related benchmarks [2][3] and include the new evaluation results in our final version to further enhance the credibility of our findings.

[1] LAN: Learning to Adapt Noise for Image Denoising. CVPR 2024.

[2] Real-world noisy image denoising: A new benchmark. ArXiv 2018.

[3] A holistic approach to cross-channel image noise modeling and its application to image denoising. CVPR 2016.

Supplement to Q1 & Q2: We disable the noise injection on feature statistics and instead inject the noise into the image space, which is equivalent to training a general Gaussian denoiser as described in A2. Below, we present the test results on Speckle Noise:

The results show that RNINet with feature-space noise injection outperforms RNINet with image-space injection. This is because RNINet with image-space injection essentially degrades into a Gaussian denoiser. However, it still achieves reasonable results on Speckle Noise, which we believe demonstrates the effectiveness of the streamlined encoder-decoder framework in mitigating overfitting, thereby also contributing to the overall superior performance of our RNINet.

We hope we have adequately addressed all the reviewer's concerns through this rebuttal. Should there be any issues that remain unaddressed, please let us know, we look forward to further discussions with you.

Best regards,

Authors of #1857

Thank you so much for your insightful comments and for recognizing the superior performance of our approach. We greatly appreciate the time and effort you've invested in reviewing our paper. We are eager to address the concerns raised by the reviewer as outlined below and are open to further discussions to resolve any remaining issues. The issues are renumbered as 1, 2, 3, 4, etc., according to their original order presented by the reviewer.

A1 → W1 & W2: Thank you for your comments. We would like to clarify that, to the best of our knowledge, our method is the first to introduce feature-level noise injection to enhance the generalizable performance of image denoising. Compared with previous methods, our method consistently outperforms them in both denoising performance and efficiency, with significant improvements. It is worth noting that, with the rapid evolution of research, discovering entirely unseen basic elements has become extremely hard. Consequently, the novelty in research should be increasingly recognized for its well-articulated motivation and innovative perspective. For instance, even renowned works like GANs, VAEs, and their applications often share similar basic components. However, this does not lessen the novelty of these studies due to their superior performance and the fresh perspectives they bring to their target tasks. We sincerely hope the reviewer will consider the novelty aspect of our work based on these points, which we believe align with the standards of most existing publications.

A2 → W3 & Q2: We guess there might be some misunderstandings regarding the random noise used in our experiments. In line with the task's requirements in previous works, we trained exclusively on Gaussian noise while testing on various unseen noise types. The random noise tensors injected into the feature statistics were also sampled from a normal/Gaussian distribution (L253-L254). For a fair comparison, all noise used in this task should be restricted to Gaussian noise. Therefore, adding random noise to the input data across training datasets (W3), or injecting noise into the image space (Q2), both equate to adding Gaussian noise to the input image, which is the standard pipeline for training a general Gaussian denoiser. However, our experiments have shown that a general Gaussian denoiser does not generalize as effectively as our approach. We include Appendix A.3 to enhance the interpretability of our method's effect. It does not solely demonstrate the effectiveness of our approach, but it does so in conjunction with the validation provided by our experiments. Thank you again for your notification, we would like to clarify these points to avoid any misunderstandings in our final version.

A3$\rightarrow$W4: Thank you for this notification. To align with A1, we would like to revise this statement and emphasize our technical contributions as offering a fresh perspective by introducing noise injection for enhanced denoising generalization capability.

A4 → W5 & Q1: While not explicitly mentioned in our manuscript, we have tested our framework using a transformer-based encoder-decoder previously, which however resulted in a average performance drop of approximately 0.5dB and less satisfactory efficiency. We realize that the large model capacity with overtraining might cause overfitting to the training noise [a][b]. Therefore, we opted for a streamlined encoder-decoder framework as depicted in Fig. 3. The experiments demonstrate that the streamlined encoder-decoder associated with noise injection module contributes collectively to the superior performance in terms of both effectiveness and efficiency for our model.

[a] Evaluating the generalization ability of super-resolution networks. TPAMI 2023.

[b] Masked image training for generalizable deep image denoising. CVPR 2023.