Revolutionizing EMCCD Denoising through a Novel Physics-Based Learning Framework for Noise Modeling

We thank Reviewer V6zx for the constructive comments and address the concerns as follows:

1. Theoretically speaking1:

Blooming occurs when the charge in a pixel exceeds the saturation level and the charge starts to fill adjacent pixels.

In traditional CCD cameras, this phenomenon occurs prior to read-out circuits, which precludes $N_r$ and $N_q$ that happen relatively downstream in image formation. However, the exact definition we included in our formula (4) & (5) is determined by the process of how we calibrate blooming factor. As shown in Fig.5, calibration frames for this factor are the end results of the entire image acquisition, which means we are unable to verify intermediate results to see when each components would be added into the frame. Having said that, as Fig.5 (b) has already shown relative good linearity of blooming effect, we strip blooming effect from all components from actually acquired image $D$, as indicated in equation (4). In equation (5), it is the synthetic image for network training. As long as blooming removal in equation (4) is in accordance with its calibration from equation (3), we believe it is a coherent reasoning, which is corroborated by results of effective blooming removal in Fig.6.

2. FPN Calibration: We acquired FPN calibration frames multiple times, all of which can be fitted in S-shaped curve, with an accuracy of less than 1 digital number margin of error averaged over all pixels for exposure between 0.006s and 0.1s. However, there do exist different transition smoothness between different acquisitions, which cannot be fully explained with our current observations, and we will continue to explore its reasons. On the other hand, piecewise linear regression for FPN representations produce slightly worse results than our S-shaped regression version: PSNR of piecewise regression yields 47.98dB vs 48.59dB in our paper. Even though piecewise function can have better fitting performance, it is possible that such representation is more prone to overfitting of FPN dark frames than using a more general and smooth S-shaped regression.

3. Ablation Study: We genuinely appreciate the suggestion on our ablation study. We conducted finer-grained ablation studies to evaluate contributions from preprocessing components like FPN and blooming removal. The table below summarizes the results:

These results indicate that our preprocessing is distinct from PMN. Notably, blooming removal contributes significantly to performance gains, while the combination of FPN and blooming removal provides a substantial boost.

We thank Reviewer M8ik for the valuable feedback and offer the following clarifications and responses:

1. Regarding the notation in Equation (1), most of the variables are 2D images, i.e. $D, I, N_p, \text{FPN}, N_r, N_q \in \mathbb{F}^{M \times N}$ (with $M$ and $N$ being height and width of the image), while $K$ and $f$ are scalars. The transformation $B[\cdot]$ maps one image to another. All $\times$ symbols represent element-wise multiplication between a matrix and a scalar. We appreciate the suggestion to make the notation more informative. In the final version, vectors and matrices will be bolded, and their dimensionalities will be explicitly defined to prevent confusion.

2. From Information Theory perspective, noise patterns such as $N_r$ and $N_q$ inevitably cause content loss, manifesting as visual distortions like blurriness in spatial domains. The authenticity of these noise components can be verified by comparing our synthesized noisy images with real noisy images and through the performance gains achieved by our pipeline over existing methods.

3. In the context of low-photon count imaging, the negative binomial distribution is a method to explain variabilities larger than mean values of the signals, since Poisson statistics dictates equality between mean and variance. However, Poisson distribution is only for photon shot noise, which is the signal-dependent components in our paper. Low-photon count imaging can suppress read-out noise from imaging circuits, but not able to eliminate all such noises. Thus, our additive noise components are not simply for explanation of long-tail traffic of photon incidences, but rather a description of possible independent noise patterns from specific device physics.

4. Key differences between EMCCD and other CMOS noise modeling are reflected in Fig.2 of our paper as uniqueness of our model, and also summarized here as follows:

   • EMCCD noise parameters are strongly influenced by exposure time, as accumulated heat affects the sensors, unlike CMOS models that primarily depend on system gain or ISO. • EMCCDs exhibit blooming effects, which are largely absent in CMOS models. • EMCCDs benefit from simpler read-out circuits, whereas CMOS models involve complex noise correlations and banding patterns.

5. For fair comparisons, we retained the original Uformer architecture, only modifying its input/output dimensions to handle single-channel grayscale EMCCD images. A tailored network to further improve denoising is part of our future work.

6. We collected real world data shot on an EMCCD device as shown in Fig. 8 in our appendix. All validation results (Tab.1 & 2), as well as qualitative demonstrations (Fig. 8, 9, 14, 15), are produced by training a network on synthetic data and testing it on real world data, so that risk of overfitting to synthesized data can be minimized in our conclusions.

We thank Reviewer e1uJ for the insightful comments and appreciate the opportunity to address the questions raised.

1. While there are similarities between our method and typical CMOS noise modeling, our objective was not to entirely redefine the physics-based noise modeling framework. Notable works, such as those by LLD[1] and PNNP[2], have sought to enhance the performance of existing models within the same framework. In contrast, the primary goal of our research is to revolutionize the denoising of EMCCD sensors by introducing novel noise modeling techniques alongside comprehensive experiments and analyses. This approach aims to fully leverage the potential of EMCCD technology, which has yet to be thoroughly explored.

2. The banding patterns observed in ELD-simulated noisy samples arise because ELD treats channel biases as scalars. As shown in Fig. 3, FPN exhibits distinct spatial variations. Using a single scalar to compute bias values results in residual deviations from zero—for example, the pixel values in the top and bottom rows of a frame remain farther from zero even after DC bias correction. These residual deviations are the root cause of the banding patterns, as they are encapsulated in row mean values, leading to elevated $\sigma_{\text{row}}$ levels.

3. We conducted a comparison experiment to LRD, which yields a PSNR of 45.79dB, between theoretical noise analysis based synthesis (Ryan et al. 2021) and SRDTrans (Li et al. 2023), all of which worse than our results. Possible reasons for LRD's shortcomings include differences between CMOS and CCD sensors, its inability to handle spatially correlated noise patterns (such as combined FPN and blooming effects), and its reliance on extensive metadata to map noise to camera parameters. By contrast, EMCCD images offer limited metadata, making LRD less effective in this context. Our uniquely designed EMCCD noise model addresses these challenges effectively.

4. To minimize differences between CMOS and EMCCD sensors, we converted SID clean images into single-channel images before generating synthetic noisy data. We quantitatively evaluated all methods on a macroscopic scene captured with a genuine EMCCD device. Since the SID dataset contains similar content, our experiments sufficiently demonstrate the effectiveness of our pipeline as a proof of concept. Furthermore, our validation on a microscopic dataset underscores the pragmatic and ready-to-use nature of our method.

References

[1] Y. Cao, M. Liu, S. Liu, X. Wang, L. Lei, and W. Zuo, ‘Physics-Guided ISO-Dependent Sensor Noise Modeling for Extreme Low-Light Photography’, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 5744–5753.

[2] H. Feng, L. Wang, Y. Huang, Y. Wang, L. Zhu, and H. Huang, ‘Physics-guided Noise Neural Proxy for Practical Low-light Raw Image Denoising’, arXiv preprint arXiv:2310. 09126, 2023.

[3] F. Zhang et al., ‘Towards General Low-Light Raw Noise Synthesis and Modeling’, in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 10820–10830.