EGGS: Exchangeable 2D/3D Gaussian Splatting for Geometry-Appearance Balanced Novel View Synthesis

We thank Reviewer wtGw for their thorough reading of our manuscript, valuable feedback, and for recognizing the contributions of EGGS. The reviewer highlighted several positive aspects of our work: they considered our method novel and insightful, effectively combining the advantages of 3D and 2D Gaussians; found the evaluation to be comprehensive, sufficiently validating the effectiveness of our approach; and acknowledged its potential impact across multiple subfields in 3D reconstruction. Below, we address the reviewer’s concerns.

Q1: Improving the Videos and Demo

We appreciate the reviewer’s suggestion regarding improving the visualization videos and demo website. At the time of submission, we were constrained by the use of an anonymous GitHub repository, which has limited storage and does not support HTML or embedded MP4 files. As a result, we had to convert videos to lower resolution GIFs and rely on markdown for presenting results, which impacted both quality and flexibility.

In the revised version, we will make efforts to enhance the demo website by including high-resolution videos across different scenes, making comparisons more accessible and intuitive.

Q2: Visualization of Gaussian type distribution in rendered images

We thank the reviewer for the careful reading of our manuscript and for recognizing our analysis of Gaussian type distribution during training. We will include additional visual examples showing the spatial distribution of 2D and 3D Gaussians in rendered images to better support our hypothesis.

We sincerely appreciate the reviewer’s suggestions. Due to the rebuttal policy, we are not permitted to submit additional figures or videos or update the anonymous links at this stage. However, we will incorporate these visualizations in the revised version to better illustrate the key ideas and performance of our method.

We thank Reviewer bqq2 for careful reading of our manuscript, insightful suggestions and recognizing positive aspects of our submission, including combining the benefits of 3D and 2D Gaussians and improving the performance across datasets. Below, we address the reviewer's concerns.

Q1: Insufficient Evaluation

Comparison with HybridGS. As noted in Appendix B, HybridGS did not release its training code at the time of our submission, making it difficult for a direct comparison. More importantly, the two approaches target different problem settings and propose orthogonal optimizations.

• HybridGS specifically focuses on dynamic scenes with transient objects. HybridGS represents the 3D scene entirely with 3D Gaussians and leverages single-view image-space 2D Gaussians to fit transient objects. The 3D Gaussians and image-space 2D Gaussians are treated separately and are not exchangeable.
• In contrast, EGGS focuses on general reconstruction settings and represents the 3D scene using an exchangeable hybrid of 2D and 3D Gaussians. Our hybrid representation can be integrated into the HybridGS framework by replacing its 3D Gaussians with our exchangeable hybrid representation.

For completeness, we provide quantitative comparisons on two dynamic scenes of different occlusion levels from the NeRF On-the-go dataset. "3DGS" and "EGGS" denote direct reconstruction of the dynamic scene without decoupling transient objects and "+ HybridGS" means HybridGS is used to decouple the static scene and transient objects. We note that at the time of rebuttal, the authors of HybridGS had released only their trained models, not the training code. Therefore, we directly leverage the single-view image-space Gaussians from their checkpoints to remove the transients in "EGGS+HybridGS".

As shown in the above table, directly reconstructing dynamic scenes with 3DGS leads to inferior appearance performance. This is because transient objects lead to significant inconsistency across different viewpoints. HybridGS mitigates the viewpoint discrepancy by decoupling transient objects from the 3D scene, improving the appearance by a significant margin. EGGS alone still suffers from transient objects. However, EGGS outperforms the 3DGS counterpart when transients are removed by HybridGS.

Standard for choosing baselines. Gaussian Splatting-based methods are developed with a varying focus, some emphasizing appearance fidelity while others prioritize geometric accuracy. To provide a comprehensive evaluation, we consider both aspects. For appearance and geometry, we select state-of-the-art methods that excel in the respective aspect as baselines, allowing us to highlight the strengths of our proposed method.

We note that while most Gaussian Splatting-based methods report appearance metrics such as PSNR, only a subset with an explicit focus on geometry report metrics like Chamfer Distance. In previous works, geometry-focused methods such as SuGaR, 2DGS, and GOF are commonly used as baselines for benchmarking geometry accuracy (e.g., Table 2 in [1], Table 1 in [2]). In line with this practice, we adopt these methods as baselines in Table 2 to assess the geometry performance of our approach.

Q2: Geometry Evaluation

Qualitative comparison on Tanks&Temples. In Figure 9, we present rendered RGB images and depth maps on the Tanks&Temples dataset. The main artifacts in our depth maps appear near the sky regions. However, depth in these areas is typically unreliable in Gaussian Splatting-based methods, and is often explicitly masked out during evaluation (e.g., Figure 7 in [3]). In our current visualization, we did not apply such masking, which likely exaggerates the perceived artifacts. We believe the inaccuracies near the sky do not meaningfully reflect overall depth quality. While not as clean as 2DGS, our method produces more accurate depth maps than 3DGS for foreground objects and buildings.

We provide a quantitative comparison below to further verify the geometry accuracy. We will also update the visualizations and include additional depth map videos in the revised version to improve clarity.

Quantitative comparison on Tanks&Temples.
We provide a quantitative geometric comparison on the Tanks&Temples dataset. Following prior works, we report the F1 score based on alignment between the predicted and ground-truth point clouds.

As shown in the above table, our method consistently outperforms 3DGS in F1 score across all scenes. While 2DGS has a slightly higher F1 score, 2DGS has inferior appearance performance, as shown in Table 1 in our manuscript. While geometry-focused methods like 2DGS and GOF prioritize geometry accuracy for surface reconstruction tasks, they often lead to decreased appearance fidelity even compared to the 3DGS baseline. This aligns with our analysis on the DTU dataset in Table 2. In contrast, our method achieves a better balance, improving the appearance while maintaining high geometry accuracy.

Q3: Dataset used in Table 4

The ablation study in Table 4 is conducted on the LLFF dataset. We will clarify this in the revised version.

Q4: More Visualization

Due to the rebuttal policy, we are not permitted to submit additional figures or videos at this stage. However, in addition to the quantitative comparisons already provided in our manuscript and in the response to Q2, we will make sure to include more qualitative visualizations in the revised version and on the demo website. These will include depth map videos, normal map videos, and rendered meshes.

We sincerely appreciate the reviewer’s constructive feedback. In the revised version, we will include the comparison with HybridGS and additional geometry evaluation on Tanks&Temples. We will also provide more visual results to highlight the reconstructed geometry. We hope our response addresses the reviewer’s concerns, and we would be grateful for a positive reconsideration of the score.

[1] Huang, Binbin, et al. "2D Gaussian Splatting for Geometrically Accurate Radiance Fields."

[2] Hyung, Junha, et al. "Effective Rank Analysis and Regularization for Enhanced 3D Gaussian Splatting."

[3] Cheng, Kai, et al. "GaussianPro: 3D Gaussian Splatting with Progressive Propagation."

We thank Reviewer 2p4H for their thorough reading of our manuscript, constructive feedbacks and recognizing several positive aspects of our submission, including the novelty of our idea and the improved performance of EGGS in both appearance and geometry. Below, we address the main concerns raised in the review.

Q1: Comparison with SoTA baselines

We thank the reviewer for acknowledging that EGGS improves reconstruction quality and achieves a better balance between appearance and geometry. We provide quantitative comparisons with two additional strong baselines: the SoTA 3DGS-based method 3DGS-MCMC and the NeRF-based method Mip-NeRF360. We evaluated the performance on the Mip-NeRF 360 and LLFF dataset. We also incorporate 3DGS-MCMC into our framework by replacing the vanilla 3DGS component.

As shown above, our method achieves comparable performance with both 3DGS-based and NeRF-based SoTA baselines. More importantly, by replacing vanilla 3DGS with 3DGS-MCMC (i.e., Ours + 3DGS-MCMC), our method can outperform the SoTA baselines. This suggests that while current EGGS is built on the vanilla 2DGS and 3DGS, more advanced variants can be incorporated to further improve the performance.

Q2: Rendering Speed

We provided an analysis of the inference speed in terms of frames per second (FPS) in Table 10 of the supplementary materials (included in the .zip file). For convenience, we provide the results below.

The rendering speed is mostly determined by the rasterization technique. While 2DGS typically has the smallest model size, as shown in Table 3 in our manuscript, its inference speed is slower than 3DGS-based methods. This is because the ray-splat intersection used in 2DGS incurs more intensive computation than affine approximation-based rasterization in 3DGS. Our method combines the two rasterization techniques and has a higher FPS than 2DGS. We note that while EGGS has a lower FPS than pure 3DGS-based method, our method achieves a better balance between model size, training and inference efficiency and reconstruction quality. We will add an additional column to Table 3 to clearly indicate the inference efficiency of each method in our revised version.

Q3: Ablation on Geometry

In Table 4, we perform the ablation study on LLFF dataset. Since LLFF dataset does not provide ground-truth point clouds or depth maps, we did not report geometry metrics in that table. We provide additional ablation results on the DTU and Tanks&Temples dataset, which include ground-truth point cloud. Following the common practice in geometry accuracy evaluation, we report Chamfer Distance (CD) on DTU and F1 score on Tanks&Temples.

As shown in the above table, merely mixing 2D and 3D Gaussians (row ii) or directly adding frequency regulation (row v) can hardly improve the geometry accuracy. Our hybrid rasterizer (row iii) improves both CD and F1 score by a significant margin. This indicates the importance of the ray-splat intersection based rasterization. Incorporating adaptive type exchange (row iv) and frequency decoupled optimization (row vii) allows a more flexible representation, more effectively exploiting the geometric accurateness of 2D Gaussians.

Q4: Scope of related works

We appreciate the reviewer’s suggestion to broaden the related work section. Purely reconstruction-focused methods such as DUSt3R are indeed valuable, particularly in settings where camera poses are not available. In this work, we adopt a commonly used setting for rendering-based methods, where camera poses are assumed to be known. In practice, such poses are often obtained through tools like COLMAP or more recent methods like DUSt3R[1] and VGGT[2].

In the revised version, we will expand the related work section to acknowledge and discuss recent purely reconstruction methods in more detail.

Q5: The functionality of $\lambda_z$.

Our scale modulation consists of two components: Eq.(7), which applies a soft gating to the raw $z$-scale $s_z$, and Eq.(6), which modulates the opacity using the activated $z$-scale $s_{z*}$. Each plays a distinct role.

• Eq.(7) controls when the $z$-scale becomes active by smoothly mapping $s_{z}$ to $s_{z*}$ using a sigmoid gate. When $s_z$ is negligibly small, the gating suppresses the activation and $s_z*$ approaches zero. In this case, the Gaussian behaves like a 2D primitive. As $s_z$ increases beyond the threshold, $s_z*$ effectively approaches the original scale $s_z$.
• Eq.(6), on the other hand, defines how much the activated $z$-scale influences opacity. The parameter $\lambda_z$ controls the strength of this correlation: as $s_{z*}$ increases, the opacity decreases smoothly, regulated by $\lambda_z$. This formulation enables opacity to act as a proxy signal to optimize $s_z$ for 2D Gaussians, which otherwise do not receive gradient updates to their $z$-scale.

In summary, Eq.(7) controls the activation of the $z$-scale, while Eq.(6) introduces a soft correlation between scale and opacity to facilitate learnable type transitions. Although it is possible to combine them into a single equation, we separate them for clarity in both concept and implementation.

Q6: Clarification on scale modulation

In vanilla 2DGS, the $z$-scale is not involved during optimization. The primary purpose of our scale modulation is to enable 2D Gaussians to transition into 3D Gaussians when necessary. Our intuition is that this transition typically occurs when a Gaussian needs to develop volumetric capacity or represent semi-transparent regions. To support this, we introduce a correlation between $z$-scale and opacity for 2D Gaussians, which allows the $z$-scale to receive meaningful gradient updates. We believe that scale modulation is essential for enabling dynamic type exchange between 2D and 3D Gaussians, and for supporting a more flexible representation, instead of merely an engineering trick.

Q7: Figure margin

We appreciate the reviewer’s suggestion. We will increase the margins around Figure 5 and Figure 6 to improve readability.

We sincerely appreciate the reviewer’s constructive suggestions. In the revised version, we will incorporate additional evaluations with SoTA methods, include efficiency and geometry metrics, expand the discussion on purely reconstruction-based methods, and clarify the scale modulation design. We hope these additions and clarifications address the reviewer’s concerns and would be grateful for a positive reconsideration of the score.

[1] Wang, Shuzhe, et al. "DUSt3R: Geometric 3D Vision Made Easy."

[2] Wang, Jianyuan, et al. "VGGT: Visual Geometry Grounded Transformer."

We thank Reviewer 5biL for their constructive feedback and for recognizing the contributions of our work, including the soundness of our methods and superior qualitative and quantitative performance. Below, we address the main concerns raised in the review.

Q1: The relationship between opacity and z-scale

Representing non-planar yet solid structures. We appreciate the reviewer’s insight regarding the potential importance of 3D but opaque Gaussian primitives for representing non-planar yet solid structures. This observation actually aligns with our design: the scale modulation between opacity and z-scale applies only to 2D Gaussians. Non-planar yet solid structures can still be effectively captured by 3D Gaussians, which remain unaffected by this modulation. During training, the model can either densify 3D Gaussians or convert suitable 2D Gaussians into 3D ones to better represent such regions.

Empirical analysis on the relationship of opacity and z-scale. The primary motivation for modulating the z-scale of 2D Gaussians is to enable their transition into 3D Gaussians when necessary. Our intuition is that such a transformation typically occurs when the model needs to acquire volumetric capacity or represent semi-transparent regions. To facilitate this, we introduce an inverse relationship between opacity and z-scale only during training.

To verify the rationale behind scale modulation, we conducted training without explicit z-scale modulation. In this setup, we employed affine approximation-based rasterization for 2D Gaussians to enable gradient flow to the z-scale. While this rasterization is not geometrically accurate, it suffices to observe the dynamics between opacity and z-scale during optimization. We analyze the cosine similarity between the gradient of opacity ($\nabla_{\alpha}$) and z-scale ($\nabla_{s_z}$) for 2D Gaussians.

As shown in the table, $\nabla_{\alpha}$ and $\nabla_{s_z}$ have negative cosine similarity across different training iterations. This suggests 2D Gaussians with increasing z-scale tend to exhibit decreasing opacity. Such behavior supports our design of soft modulation and validates the effectiveness of our design.

Q2: Alternative initialization of Gaussian type

While more advanced initialization strategies may improve reconstruction quality, designing such initialization is non-trivial. For instance, wavelet-based decomposition can effectively decouple frequency information in the 2D image space. However, during initialization, 2D pixels do not directly correspond to 3D points, which makes it challenging to apply Discrete Wavelet Transform for initializing Gaussian type.

In our response to Q3 below, we show that different initializations often converge to Gaussian models with similar type distributions and comparable appearance performance. This empirical observation suggests that random initialization serves as a reasonable and effective starting point. As mentioned in our limitations section, 3D point segmentation could potentially enhance initialization. For example, we can segment the initial 3D points into distinct regions and assign different Gaussian types to different semantic segments. This might lead to improved initialization and more efficient training. However, determining an optimal mapping between semantic labels and Gaussian types remains challenging. We thus leave the development of more advanced initialization techniques to future work.

Q3: Can 2D Gaussians change to 3D if initialization is incorrect?

A key feature of EGGS is its exchangeable representation, which allows each Gaussian primitive to change its type as needed, regardless of its initial type. To verify this capability, we conduct a simple experiment by investigating three initialization scenarios: we initialize all Gaussians as 2D, all as 3D, or with random type assignments, and observe the distribution of Gaussian types throughout training. Below, we show the percentage of 3D Gaussians at different iterations.

As shown in the above table, even when the model is initialized entirely with 2D Gaussians, part of the Gaussians are converted to 3D Gaussians during training, leading to a hybrid model in the final stage. This demonstrates that 2D Gaussians can indeed transition to 3D types during training, despite potentially incorrect initialization, and vice versa.

Q4: Improve readability

Due to space constraints, we included an introduction to the mathematical formulation of 2DGS and 3DGS rasterization in Appendix B. In the revised version, we will add a brief preliminary section at the beginning of Section 3 to further improve readability.

We sincerely appreciate the reviewer’s constructive suggestions. We will include detailed discussions on the scale modulation and initialization in our revised version to offer more insights into our method.