We appreciate your valuable suggestions and comments. Here are our detailed responses to your feedback:

1. Novelty Limited Due to the Need for Noise-Clean Paired Data:

   We want to clarify that the pretrained model aims to provide a good initialization and does not have strict restrictions on the data used for training (e.g., we can use data from different devices or noise-only data). Specifically, our pretrained dataset, SID (captured by a DSLR camera), differs significantly from the RawNeRF dataset (captured by an iPhone X) in terms of color space and bit depth. To further verify the impact, we also experimented with a self-supervised pretraining method like Neighbor2Neighbor with noise-only images in SID. The results, shown in Table R1, indicate no significant difference, supporting that noise-clean paired images are not strictly necessary during the pretraining stage.

2. Comparison with Baselines and RawNeRF:

   We want to clarify that in the few-shot settings (which are more common in real-world scenarios like 3D reconstruction using multi-camera setups or for autonomous driving) on the RawNeRF dataset, we achieved a 3-4 dB gain compared to the rawnerf loss baselines as presented in Figures 5 and 6. In full-view settings, we also observed around a 0.9 dB gain compared to 3DGS with the rawnerf loss baseline. While our method's reconstruction quality is similar to RawNeRF, we significantly reduced the training time by 100x and boosted the rendering speed by 5000x, as shown in Table R2, making it more feasible for real-world applications.

Once again, we sincerely thank you for your constructive feedback, which has been crucial in refining our work.

Tables:

Table R1: Ablation Study of F Pretraining on RawNeRF Datasets (Full Training Views)

Table R2: Quantitative Comparison on RawNeRF Datasets (Full Training Views)

We appreciate your valuable suggestions and comments. Here are our detailed responses to your feedback:

1. Lambertian Surface Assumption:

   We use the Lambertian surface assumption to simplify our analysis. However, our method can also be applied to non-Lambertian surfaces, such as mirror reflections on a piano. The core of our method is the integration of a noise model to combine the noise prior with 3DGS reconstruction. Since 3DGS supports non-Lambertian surfaces, our method is also applicable in such scenarios.

2. Effect of $L_{cov}$:

   The $L_{cov}$ term ensures that noise remains independent across pixels and maintains the standard deviation of the noise distribution within a pixel. We want to clarify that the ablation of $L_{cov}$ ($\lambda_{cov}$) is already presented in Table 4 of our paper. We also provide a visual illustration in Figure R3 of our submitted PDF. As shown, $L_{cov}$ ensures the noise remains independent across different pixels and adjusts the extracted noise distribution within a pixel to follow a standard Gaussian distribution after normalization. Since $L_{nd}$ only ensures the mean of extracted noise is zero, it does not control the standard deviation. Therefore, constraining the diagonal of the covariance matrix in $L_{cov}$ is necessary to ensure the standard deviation is one.

3. Home-like Setting:

   We have found that the noise model can also be calibrated in a home-like setting as follows:

   a. Flat Frames Capture: Use a MacBook screen to capture flat frames. Adjust the focus of the mobile phone (iPhone X) camera to infinity, attach it tightly to the screen, and adjust the screen brightness to capture a series of flat frames at different ISO and exposure levels. This process is illustrated in Figure R4(a) of the submitted PDF.

   b. Dark Frames Capture: Use a drawer and a headphone to capture dark frames. Connect the mobile phone (iPhone X) with a (wired/wireless) headphone, open the camera app, place the phone in the drawer, close the drawer, and use the headphone's play button to remotely take photos. Capture 50 dark frames at different ISOs. This setup is shown in Figure R4(b) of the submitted PDF.

   These settings can be easily achieved in a home environment. We found that the noise parameters calibrated this way are very close to those calibrated in a lab setting (dark room) ($\sigma_{read}$ 7.38 vs. 7.06). The impact on the final 3DGS reconstruction performance on RawNeRF is less than 0.2dB. Additionally, we have developed a semi-automatic labeling app for Android mobile phones, as shown in Figure R4(c) of the submitted PDF, which we will release upon the paper's acceptance. We hope this tool will enable more users to enjoy 3DGS reconstruction in low-light conditions using their own mobile phones.

Once again, we sincerely thank you for your constructive feedback, which has been crucial in refining our work.

Dear reviewer,

The authors have submitted a rebuttal to your comments. Could you please read it and respond? The deadline for your reply is within one day. Thank you.

AC

We sincerely thank you for your thorough review and valuable suggestions. Here are our detailed responses to your comments:

1. Comparison with NeRF Baselines:

   We acknowledge the importance of comparing our method with NeRF-based methods. We have conducted a comparison between NeRF and HDR Scaffold-GS baselines and our proposed method in Table R1. We observed that the unmodified 3DGS (HDR Scaffold-GS) exhibited a gap in rendering quality compared to the NeRF-based method (RawNeRF), as 3DGS is more prone to noise. This is discussed in our manuscript and illustrated in Figure R2 of our submitted PDF. Our method bridges this gap, achieving similar performance to RawNeRF while significantly reducing training time (by a factor of 100) and rendering speed (by a factor of 5000), making raw 3D reconstruction feasible in real-world applications.

2. Ablation of the Noise Extractor F:

   We want to clarify that pretraining the noise extractor F is necessary because U-Net struggles to converge and extract high-frequency noise within 30,000 iterations during 3DGS training. Pretraining F ensures it can effectively extract high-frequency noise. Our pretrained dataset, SID (captured by a DSLR camera), differs significantly from the RawNeRF dataset (captured by an iPhone X) in terms of color space and bit depth. To verify the impact, we also experimented with a self-supervised method like Neighbor2Neighbor (using one output as the clean image) with noise-only images in SID. The results, shown in Table R2, indicate no significant difference, supporting noise-clean paired images are not strictly necessary during the pretraining stage.

3. Clarification of Figure 5:

   We apologize for the unclear presentation in Figure 5. The solid line in Figure 5 represents the one-stage method, corresponding to the last three rows in Table 1. The dashed line denotes the two-stage method, corresponding to the rest of Table 1. The one-stage method directly trains the 3DGS on the noisy images with different loss functions. In the two-stage method, the input is first denoised using a pretrained denoiser, and then the 3DGS is trained with RawNeRF loss functions. We will clarify this in the revised version of our manuscript for better clarity.

Once again, we sincerely thank you for your constructive feedback, which has been crucial in refining our work.

Tables:

Table R1: Quantitative Comparison on RawNeRF Datasets (Full Training Views)

Table R2: Quantitative Comparison on RawNeRF Datasets (Full Training Views)

We sincerely thank you for your review and valuable suggestions on our paper. In response to your suggestion regarding comparisons on more sparse-view settings, we have conducted additional quantitative comparisons using the LLFF dataset with simulated noisy raw images. Below are the details of our approach and findings:

To address the more sparse-view settings (3-views), we used the inverse ISP proposed in [1] to convert the LLFF dataset RGB images into RAW images. We then added synthetic noise following the iPhone X noise model and adhered to the baseline methods' settings [2] for a fair comparison. Specifically, we used FSGS [2] as the 3D representation baseline and only modified the loss function to our proposed $L_{\text{nrr}}$. The results, presented in Tables R1, demonstrate that our method consistently outperforms other methods even with limited view settings.

Once again, we appreciate your constructive feedback and hope that our response addresses your concerns.

Tables and References:

Table R1: Quantitative Comparison on LLFF Datasets (3 Training Views)

* * denotes the downsample ratio of the resolution following the same settings as in [2].

References:

[1] Reverse Imaging Pipeline for Raw RGB Image Augmentation; Samu Koskinen, Dan Yang, and Joni-Kristian Kämäräinen, ICIP 2019

[2] FSGS: Real-Time Few-Shot View Synthesis using Gaussian Splatting; Zehao Zhu, Zhiwen Fan, Yifan Jiang, and Zhangyang Wang, ECCV 2024

We deeply appreciate your thorough review and valuable feedback on our submission. Here are our detailed responses:

1. More Quantitative and Qualitative Results:

   Given the limitations of the RawNeRF dataset, we conducted additional quantitative comparisons using the LLFF dataset with simulated noisy raw images and provided qualitative comparisons for the training scenes in the RawNeRF dataset. Specifically, we used the inverse ISP proposed in [1] to convert LLFF RGB images into RAW images, added synthetic noise following the iPhone X noise model, and adhered to baseline methods' settings [2]. We used FSGS [2] as the 3D representation baseline, modifying only the loss function to our proposed $L_{\text{nrr}}$. The results are listed in Table R1. Our method consistently outperforms others even with limited view settings and diverse training scenes. For qualitative results, please refer to Fig. R1 in our submitted PDF, which shows that our method produces smoother results and reduces artifacts.

2. Connection Between the Analysis and Main Text:

   Our analysis highlights an overlooked issue: 3DGS is more vulnerable to noise in few-shot settings compared to neural radiance field-based 3D reconstruction. Unlike RawNeRF, 3DGS requires explicit noise regularization. Specifically, in Sec. 4.1, we illustrate the connection between (N) and the bias of the 3DGS optimal target and real-world point color. As (N) decreases, the bias increases, necessitating a denoiser to prevent 3DGS points from overfitting to noisy training views, as shown in Fig. 6 of our manuscript. This issue is common in scenarios like multi-camera 3D reconstruction and street scene reconstruction in autonomous driving. Although the effect diminishes with larger (N), it remains significant, as evidenced by our results with 100-view settings (Fig. 7 and Table 1).

3. Novelty of First Contribution:

   The impact of noise is unique in 3DGS compared to NeRF-based methods. MLPs in NeRF can perform low-frequency filtering, making it challenging to overfit to high-frequency noise, as explored in [3]. RawNeRF requires ~5 days to train a single scene and 2 minutes to render a new view in 4K resolution, whereas 3DGS can achieve training in 2 hours and 50 FPS rendering speed. However, 3DGS tends to generate thin-flat noise, degrading rendering speed and quality. Visual details comparing NeRF and 3DGS can be found in Fig. R2 of our submitted PDF. We revise our first contribution in the intro. as follows:

   We explore the unique impact of noise on 3DGS and its relationship with the number of input views. We highlight that 3DGS is more vulnerable to noise than neural radiance field-based 3D reconstruction, especially in few-shot settings, and provide a detailed analysis of how noise impacts the optimization of 3DGS, modeling its relationship with the number of training views and the noise distribution.

4. Training ELD/PMN with iPhone X Noise Model:

   There is a gap between simulated iPhone X noise on DSLR images and real noisy iPhone X images due to differences in bit depth (14-bits vs. 12-bits). Following your suggestion, we simulated noise-clean paired data using the iPhone X noise model and performed denoising first, followed by reconstruction. The results, listed in Table R2, do not show significant performance improvement, likely due to sensor differences between the iPhone X and DSLR cameras.

5. Comparison with 3DGS and RawNeRF:

   We agree that a comparison with 3DGS and RawNeRF is valuable. We have added this comparison in Table R2 under full-view settings. Our method shows that the original 3DGS is more prone to noise, resulting in lower reconstruction quality. Although NeRF-based methods achieve similar results due to their low-pass characteristics, they require significantly more time (5000x for rendering a single image and 100x for training) compared to our method, making them impractical for real-world applications.

6. Pretrained Checkpoint and Loss Function:

   The denoising method in Table 1 is used to first denoise the input, and then the loss function is applied to 3DGS for the 3D reconstruction loss. Thus, the loss function is only used during the 3D reconstruction training period.

Once again, we sincerely thank you for your constructive feedback.

Tables and References:

Table R1: Comparison on LLFF Datasets (3 Training Views)

• denotes the downsample ratio of the resolution, as same as [2].

Table R2: Comparison on RawNeRF Datasets (Full Training Views)

References:

[1] Reverse Imaging Pipeline for Raw RGB Image Augmentation

[2] FSGS: Real-Time Few-Shot View Synthesis using Gaussian Splatting

[3] DINER: Disorder-Invariant Implicit Neural Representation

Dear Reviewer m8Xp,

We sincerely thank you for your review and feedback.

Color bias

We believe that the ISP used in RawNeRF for training views are different from the ones we used, this shall be the cause of the color bias. The detailed reasons are as follows:

• Even the visualizations of the input images are different from the ones in RawNeRF as shown in Fig. 9 of their arixv paper.
• We observed the same color bias as shown in our Fig. R1 input images using provided ISP w/o color correction default viewer of Windows, default viewer of MacOS, and rawpy with in-camera white balance.
• We achieved different and inaccurate white balances using rawpy with its auto white balance.

Since the ISPs of training views are not released (the provided ISP for testing needs GT images for color correction as you mentioned), we will be grateful if you have any idea to obtain the same results in their paper and we will update the results in the revised version.

Even if there are differences in the usage of ISP in training views, we use the same provided ISP for the test views in the test scenes so all the results in the paper do not have this issue.

Loss of details

• We acknowledge that some details are lost in the grass of the second row. However, as seen in other figures—including those in the manuscript and rebuttal PDF, such as the spokes of the bicycle wheel and the text on the board—we generally preserve details well.

• The loss of detail in the grass in the second row of Fig. R1 might be coincidental. If it were due to the over-smoothing nature of our method, this effect would likely be visible in all other figures as well.

• However, our method achieves better PSNR, SSIM, and LPIPS compared with HDR Scaffold-GS. The quantitive results demonstrate that our method can achieve overall better detail preserving, and over-smoothing is not a common behavior of our method.

• By removing the needle-like artifact, we can significantly improve the rendering FPS, e.g., increases from 53 FPS to 134 FPS even for full-view training in Fig. R2 and ~4x FPS increase for 4-view training.

We hope this clarifies the unique contributions and impact of our work.

Best regards,

Authors