6DGS: Enhanced Direction-Aware Gaussian Splatting for Volumetric Rendering

Thank you for your detailed feedback and insightful suggestions. Below, we address your comments.

W1. View-dependent Opacity and Supplemental Video

We have validated the effectiveness of view-dependent opacity in our ablation studies (see rows labeled "No $\lambda_{opc}$" in Tables 4 and 5). The results clearly show the importance of this component in enhancing rendering quality for view-dependent scenes.

Additionally, we have provided a supplemental video showcasing real-time rendering using a customized splatviz tool [1] on a standard laptop. This video demonstrates the practical capabilities of 6DGS in real-time applications and offers visual confirmation of its rendering quality for novel views.

Reference:

[1] Barthel, Florian, et al. "Gaussian Splatting Decoder for 3D-aware Generative Adversarial Networks." arXiv preprint (2024).

W2. Geometry and Novel View Synthesis

We appreciate your concern about the trade-off between view dependency modeling and geometry reconstruction. However, our results indicate that 6DGS achieves a favorable balance:

• Generalization: All our quantitative and qualitative evaluations are conducted on test set novel views, highlighting 6DGS's strong generalization capabilities.
• Geometry Accuracy: Visualizations in Figure 3 (last three columns) illustrate that 6DGS produces more faithful geometry compared to 3DGS and N-DG.
• Efficiency: By using significantly fewer Gaussian points than 3DGS (e.g., 67,490 vs. 115,111 points on average in the 6DGS-PBRT dataset), 6DGS avoids overfitting while improving both geometric and rendering accuracy.

These results confirm that 6DGS effectively enhances view-dependent effects without compromising geometric fidelity or novel view synthesis quality.

Q1. Real-world Scenes from NeX

We have newly conducted extensive experiments on three real-world datasets: Deep Blending [1], Tanks&Temples [2], and Shiny from NeX [3]. Compared with 3DGS and N-DG:

• Deep Blending: 6DGS achieves comparable quality (28.59 vs. 29.48 PSNR) to 3DGS with 33% fewer points.
• Tanks&Temples: 6DGS outperforms 3DGS in quality (23.50 PSNR vs. 23.42 PSNR) while reducing the number of points by 22%.
• Shiny: In scenes with strong view dependency (e.g., cd and lab), 6DGS achieves PSNR gains of +2.88 and +2.97, respectively, while utilizing only ~50% of the Gaussian points.

These results demonstrate that 6DGS effectively generalizes to real-world scenarios, particularly excelling in scenes with strong view-dependent effects.

In comparison to NeX, a method focused on implicit representations and specular effects, 6DGS achieves faster training times (48 minutes vs. >18 hours) with slightly lower average PSNR (23.01 vs. 26.45). This highlights 6DGS's trade-off between real-time performance and fidelity.

Reference:

[1] Hedman, Peter, et al. "Baking neural radiance fields for real-time view synthesis." ICCV 2021.

[2] Knapitsch, Arno, et al. "Tanks and temples: Benchmarking large-scale scene reconstruction." ACM ToG 2017.

[3] Wizadwongsa, Suttisak, et al. "Nex: Real-time view synthesis with neural basis expansion." CVPR 2021.

CONTINUE ...

Thank you for your thoughtful feedback. In response to your concern, we have included a new supplemental video featuring the lego scene from NeRF-Synthetic. Due to supplementary size limitations, we could only include one additional example. This visualization demonstrates that 6DGS achieves high-quality novel view synthesis. Notably, for novel views at the bottom of the lego model, 6DGS shows empty Gaussian points (as the bottom is invisible in training images), whereas 3DGS exhibits obvious artifacts.

Additionally, we have added qualitative results in Section A6 of the Appendix, as shown in Figures 7 and 8. These figures provide visualizations from three real-world datasets: Deep Blending [1], Tanks & Temples [2], and Shiny [3]. Across these datasets, 6DGS demonstrates comparable or superior visual quality compared to baseline methods.

We hope these additions address your concerns. Please feel free to let us know if you have further questions or suggestions. Thank you once again.

Thank you for your insight and for maintaining a positive score.

Thank you for your thoughtful feedback and constructive suggestions. Below, we address each point in detail:

W1. Synthetic NeRF and Real-world Scenes from NeX

We would like to respectively clarify the reviewer zNRe that we have already included the Synthetic NeRF dataset, as shown in Table 3 and Table 5. Additionally, we have newly conducted extensive experiments on three real-world datasets: Deep Blending [1], Tanks&Temples [2], and Shiny from NeX [3]. Compared with 3DGS and N-DG:

• Deep Blending: 6DGS achieves comparable quality (28.59 vs. 29.48 PSNR) to 3DGS with 33% fewer points.
• Tanks&Temples: 6DGS outperforms 3DGS in quality (23.50 PSNR vs. 23.42 PSNR) while reducing the number of points by 22%.
• Shiny: In scenes with strong view dependency (e.g., cd and lab), 6DGS achieves PSNR gains of +2.88 and +2.97, respectively, while utilizing only ~50% of the Gaussian points.

These results demonstrate that 6DGS effectively generalizes to real-world scenarios, particularly excelling in scenes with strong view-dependent effects.

In comparison to NeX, a method focused on implicit representations and specular effects, 6DGS achieves faster training times (48 minutes vs. >18 hours) with slightly lower average PSNR (23.01 vs. 26.45). This highlights 6DGS's trade-off between real-time performance and fidelity.

References:

[1] Hedman, Peter, et al. "Baking neural radiance fields for real-time view synthesis." Proceedings of the IEEE/CVF international conference on computer vision. 2021.

[2] Knapitsch, Arno, et al. "Tanks and temples: Benchmarking large-scale scene reconstruction." ACM Transactions on Graphics (ToG) 36.4 (2017): 1-13.

[3] Wizadwongsa, Suttisak, et al. "Nex: Real-time view synthesis with neural basis expansion." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.

W2. Challenges of Camera Pose

We have evaluated 6DGS on three real-world datasets (Deep Blending, Tanks&Temples, and Shiny) where camera poses were estimated using COLMAP, presenting challenging real-world conditions. Our strong results on these datasets--particularly the improvements on scenes with strong view-dependent effects like 'cd' (+2.88 dB PSNR) and 'lab' (+2.97 dB PSNR)--demonstrate that 6DGS is robust to camera pose estimation uncertainties in real-world scenarios. This validation extends beyond synthetic datasets and confirms our method's effectiveness under practical deployment conditions.

Q1. N-DG Settings

Yes, we use the released code with default settings provided by N-DG, which uses 6D Gaussian representation for volumetric radiance fields. While N-DG also supports a 10D representation with additional features (albedo $rgb$ and roughness $r$), it is only used to demonstrate the application of Global Illumination.

CONTINUE ...

I have read the other review comments and the rebuttal. The rebuttal has addressed most of my concerns; however, I still look forward to seeing the qualitative results in real-world scenarios. Therefore, I maintain my rating as marginally above the acceptance threshold.

Thank you for highlighting the strengths of our work and the compatibility of 6DGS with existing frameworks. Below, we address your feedback and questions in detail.

W1. Real-world Scene Evaluations

To address the concern regarding real-world datasets, we have conducted additional experiments on three challenging datasets: Deep Blending [1], Tanks & Temples [2], and Shiny from NeX [3]. The results demonstrate that 6DGS generalizes well to real-world scenarios and excels in modeling strong view-dependent effects:

• Deep Blending: 6DGS achieves comparable quality (28.59 vs. 29.48 PSNR) to 3DGS with 33% fewer points.
• Tanks&Temples: 6DGS outperforms 3DGS in quality (23.50 PSNR vs. 23.42 PSNR) while reducing the number of points by 22%.
• Shiny: In scenes with strong view dependency (e.g., cd and lab), 6DGS achieves PSNR gains of +2.88 and +2.97, respectively, while utilizing only ~50% of the Gaussian points.

References:

[1] Hedman, Peter, et al. "Baking neural radiance fields for real-time view synthesis." Proceedings of the IEEE/CVF international conference on computer vision. 2021.

[2] Knapitsch, Arno, et al. "Tanks and temples: Benchmarking large-scale scene reconstruction." ACM Transactions on Graphics (ToG) 36.4 (2017): 1-13.

[3] Wizadwongsa, Suttisak, et al. "Nex: Real-time view synthesis with neural basis expansion." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.

W2. Visual Characteristics and Dataset Variety

Our 6DGS-PBRT dataset is designed to test a diverse range of optical phenomena using six carefully crafted scenes rendered with the PBRT engine "Cycle." These scenes address various challenging light transport effects:

• Volumetric Effects:

  • Scenes: cloud-bunny, cloud, smoke, explosion, ct-scan
  • Effects: Complex volumetric absorption and scattering
  • Goal: Test the model’s capability to handle varying densities and light transport through participating media

• Surface Effects:

  • Scenes: suzanne
  • Effects: Translucence and refraction through glass BSDF material, along with surface scattering
  • Goal: Evaluate the model’s ability to manage intricate light-material interactions

To further validate the versatility of our method, we added a new dragon scene that combines subsurface scattering (SSS) and surface scattering effects. As shown in the table below, 6DGS achieved the best results with the fewest Gaussian points. Visualizations of the dragon scene are provided in the Supplementary Material. These additional evaluations demonstrate the adaptability of 6DGS to diverse scenarios involving both volumetric and surface-based light transport phenomena.

CONTINUE ...

Thank you for your insightful feedback and constructive comments. Below, we address each point in detail:

W1. Real-world Scene Evaluations

We have newly conducted extensive experiments on three real-world datasets: Deep Blending [1], Tanks&Temples [2], and Shiny from NeX [3], as shown in Table 5 in the Appendix. Our key observations include:

• Deep Blending: 6DGS achieves comparable quality (28.59 vs. 29.48 PSNR) to 3DGS with 33% fewer points.
• Tanks&Temples: 6DGS outperforms 3DGS in quality (23.50 PSNR vs. 23.42 PSNR) while reducing the number of points by 22%.
• Shiny: In scenes with strong view dependency (e.g., cd and lab), 6DGS achieves PSNR gains of +2.88 and +2.97, respectively, while utilizing only ~50% of the Gaussian points.

These results demonstrate the robustness of 6DGS in generalizing to real-world settings and its ability to excel in modeling strong view-dependent effects.

References:

[1] Hedman, Peter, et al. "Baking neural radiance fields for real-time view synthesis." ICCV 2021.

[2] Knapitsch, Arno, et al. "Tanks and temples: Benchmarking large-scale scene reconstruction." ACM ToG 36.4 (2017): 1-13.

[3] Wizadwongsa, Suttisak, et al. "Nex: Real-time view synthesis with neural basis expansion." CVPR 2021.

Q1. $\lambda_{opa}$ Parameter Analysis

We agree that the parameter $\lambda_{opa}$ is a critical component in 6DGS. As shown in Table 4 (ablation study), even when naively setting $\lambda_{opa}=1$, 6DGS significantly outperforms 3DGS, achieving +8.52dB PSNR (36.05 vs 27.53) while using 48.7% fewer Gaussian points. We introduce $0 < \lambda_{opa} < 1$ as a hyper-parameter to fine-tune view-dependency effects, where higher $\lambda_{opa}$ leads to stronger view-dependency with fewer Gaussian points (Appendix Figure 4). With our default setting of $\lambda_{opa}=0.35$, we further improve PSNR to 37.49 (+1.44dB) with a slight increase in points (66,803), still 42% fewer than 3DGS. Making $\lambda_{opa}$ learnable achieves the best PSNR (37.61) while maintaining an efficient point count (67,492). Importantly, our method achieves state-of-the-art performance across multiple datasets using the default $\lambda_{opa}=0.35$ without dataset-specific tuning.

CONTINUE ...

Q2. Dynamic Scenes

We thank the reviewer for raising this question, which we have been considering with too. Our 6DGS improves the view-dependency modeling ability of 3DGS and can be naturally extended to dynamic scenes in two ways. First, we can directly integrate our directional component into existing 3DGS-based dynamic methods [4, 5]. Second, we can extend 6DGS with an additional temporal dimension to model a 7D spatial-angular-temporal Gaussian:

where $\mu_p \in \mathbb{R}^3$, $\mu_d \in \mathbb{R}^3$, and $\mu_t \in \mathbb{R}$ represent the means in position, direction, and temporal space respectively, and $\Sigma \in \mathbb{R}^{7\times7}$ captures the full covariance. While extending 6DGS to dynamic scenes is an exciting future direction, it is beyond the scope of our current work, which focuses on demonstrating the effectiveness of 6DGS for static scenes.

References:

[4] Wu, Guanjun, et al. "4d gaussian splatting for real-time dynamic scene rendering." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.

[5] Yang, Ziyi, et al. "Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.

Q3. Initialization Phase

We respectfully disagree with the concern about computational complexity in initialization. In fact, our initialization is computationally simpler than 3DGS:

• Position and opacity: Uses the same initialization method as 3DGS
• Scale initialization: Simply sets diagonal elements in the covariance matrix to 0.1, while 3DGS requires computing point distances
• Rotation initialization: Sets off-diagonal elements to zero, similar to 3DGS's zero rotation

Therefore, 6DGS initialization has similar or lower computational cost compared to 3DGS.

Q4. CUDA Implementation

We acknowledge that implementing Algorithm 1 in CUDA could further enhance speed. However, the performance gain would not be significant since Gaussian rasterization remains the primary computational bottleneck. Instead, we focused on accelerating the rendering pipeline by adopting FlashGS, resulting in 6DGS-Flash which significantly improves performance from 154.5 FPS to 326.3 FPS (2.1× speedup) as shown in Table 2. We are also considering implementing Algorithm 1 in CUDA to further optimize our training and rendering pipeline.

We hope these clarifications address your concerns and appreciate your valuable feedback.