Holistic Large-Scale Scene Reconstruction via Mixed Gaussian Splatting

We thank the reviewer for thorough evaluation and constructive feedback. We address each concern below:

Q1: Specific Cases of the Problems

1. Complex Parameter Tuning

Prior divide-and-conquer approaches rely heavily on carefully crafted view assignment strategies for each individual block, which introduces considerable tuning complexity. For example, VastGaussian requires setting a visibility threshold to determine whether a particular camera view should contribute to a block, while CityGaussian introduces an intersection threshold for selection. Beyond that, parameters must also be decided for how many blocks to divide the scene into and define the splitting intervals along each axis. These hyperparameters vary across scenes and typically require manual adjustment or grid search, significantly increasing implementation burden and making reproduction more challenging.

2. Loss of Global Information

Existing large‑scale scene reconstruction methods employ a divide‑and‑conquer strategy, partitioning the scene into independent blocks for multi‑GPU training. Since each block is optimized in isolation, multiview constraints do not span block boundaries, which can lead to visible artifact or texture misalignment at those boundaries. Methods such as CityGaussian, CityGaussianV2, and VastGaussian attempt to mitigate these artifacts by increasing the overlap between adjacent blocks. The more recent DOGS framework further addresses boundary inconsistencies through a dedicated “consensus step.” As stated in the DOGS paper, below information acknowledges that boundary inconsistencies persist in prior methods.

“Our method presents higher‑fidelity rendering results than VastGaussian near the splitting boundary, which validates the effectiveness and importance of the consensus step” (DOGS, page.17).

Due to independent optimization of blocks breaks multi-view constraints across boundaries, such post-processing cannot fundamentally resolve the boundary inconsistency.

Regarding qualitative comparison, due to NeurIPS policy preventing us from including boundary issue visualizations in this rebuttal, please refer to DOGS’s Figures 8 and 11, which illustrate these boundary effects.

To provide a quantitative comparison, we follow DOGS’s partitioning strategy and select 300 viewpoints across block boundaries in both the Rubble and Building scenes. We render each viewpoint using the DOGS model and our MixGS method, then compute PSNR, SSIM, and LPIPS for both in below table. Despite DOGS’s consensus refinement, MixGS consistently yields higher quality at block interfaces, effectively eliminating inconsistencies at the source rather than relying on post-processing.

Compared with other methods, MixGS optimizes the entire scene as a single entity: all Gaussians are updated under global multi-view constraints, and no block partitioning is performed. This holistic design eliminates boundary inconsistencies at the source, rather than relying on post-processing alignment. We appreciate the reviewer’s suggestion: in the revised manuscript, we will include a clearer discussion of block‑boundary issues and provide illustrative visualizations in the supplementary material.

[1] Dogs: Distributed-oriented gaussian splatting for large-scale 3d reconstruction via gaussian consensus. NeurIPS 2024.

---

Q2: Processing Areas with Extremely Sparse Initialization

To ensure that richly textured areas are faithfully reconstructed, MixGS employs two complementary mechanisms.

Adaptive Densification: On the one hand, in the coarse training stage, Gaussians are duplicated and split in regions with high-frequency textures and detailed structure (as in 3DGS [1]). As a result, areas with complex details are initially covered by a denser set of coarse Gaussians. This seeding provides a strong prior for the decoded Gaussians to refine fine details.

Offset-Driven Refinement: On the other hand, benefiting from the offset pool, the decoded Gaussians will move toward under-reconstructed and high-frequency zones during training, instead of locating near the corresponding coarse Gaussians. We show the center location of the coarse and decoded Gaussians in Figure 4 of the manuscript. It can be observed that the decoded Gaussians shift toward more complex areas (e.g., bush and rooftop tiles in Fig.4), further increasing the Gaussian density exactly where it is needed most.

Moreover, MixGS can generate multiple decoded Gaussians per coarse Gaussian by simply modifying the decoder's output dimension. We evaluated different ratios K (i.e., $N_{\Phi} : N_v$) on the Rubble scene:

Higher ratios improve quality (PSNR reaches 27.02 dB at 3:1) but increase memory usage. We use the 1:1 setting for optimal balance between performance and memory efficiency on a single 24GB GPU.

[1] 3D Gaussian Splatting for Real-Time Radiance Field Rendering. TOG 2023.

---

Q3: Different from Scaffold-GS

Scaffold‑GS builds a hierarchical, region‑aware representation by introducing fixed anchor points and dynamically spawning Gaussians around them based on view coverage and feature saliency. In contrast, MixGS explores spatial structure through a unified multi‑resolution hash encoding of Gaussian centers, enabling implicit feature learning at multiple scales.

Moreover, MixGS encodes not only each Gaussian’s position but also its auxiliary attributes, such as rotation, scale, and camera pose, into a high‑level feature vector. These rich, view‑aware features are then decoded by a lightweight MLP to generate fine‑detail Gaussians, whose positions are refined via an offset pool. Finally, our novel mixing mechanism combines the original “coarse” Gaussians with the decoded “fine” Gaussians into a single set for rendering, ensuring both global consistency and local fidelity across an entire large‑scale scene.

In summary, while Scaffold‑GS adaptively assigns Gaussians per region and view, MixGS holistically optimizes the full scene representation, jointly learning coarse structure and fine details, without any block partitioning or anchor placement. We will add this comparison in the revision to clarify these conceptual and technical distinctions.

---

Q4: Why LPIPS Is Relatively Higher?

Our MixGS choice to prioritizes holistic consistency (critical for large scenes), instead of trivial LPIPS optimization. Hence, this does not reflect poor detail recovery. Methods like DOGS and CityGaussian use divide-and-conquer, which can create "pixel-perfect but lighting inconsistent" local regions (lower LPIPS for small patches) but fail to align textures/lighting across block boundaries (e.g., grassland suddenly shifts in the 4 rows and 2 columns in Fig. 1 of supplementary). Our holistic framework avoids this by enforcing global consistency, which may introduce minor local color adjustments to align large structures, and these adjustments are what LPIPS occasionally penalizes.

Additionally, we evaluate MixGS on the MatrixCity and Campus scenes. Both datasets are recognized for their challenging scale and high-frequency details. As shown in the table below, MixGS achieves the highest rendering quality across PSNR, SSIM, and LPIPS. We will include these quantitative results and corresponding qualitative comparisons in the revised manuscript.

---

Q5: Rendering Speed Comparison on the Same Platform

Due to hardware constraints, we are unable to benchmark MixGS on A100 GPUs. Instead, we measure all methods on an NVIDIA RTX 3090 to ensure a fair comparison. The results are shown in the Table below. We will include this direct comparison in the revised manuscript for greater clarity.

---

Q6: Typo Error

Thank you for your careful review. We will perform a comprehensive proofread of the manuscript and correct all grammatical and spelling mistakes in the revised version.

We thank the reviewer for thorough evaluation and constructive feedback. We address each concern below:

Q1: Training Time Efficiency

We acknowledge the reviewer’s concern about training time. The longer training duration in our original results is primarily due to hardware limitations: while other methods (as shown in Table 2 of the supplementary material) were trained using five high-end GPUs (5 × RTX 6000 48GB), our experiments were conducted on a single RTX 3090 24GB. This significant difference in hardware configuration directly impacts training efficiency.

To provide a more fair comparison, we integrate MixGS into the open-source distributed training framework [1] and retrain our model using 5 × RTX 3090 GPUs. The training time comparison is summarized below:

With five GPUs, training time of MixGS drops to a range comparable with or better than existing methods.

Furthermore, we confirm that distributed training preserves rendering quality. Below are the quantitative results comparing the original single-GPU model and our multi-GPU version. These results demonstrate that MixGS can scale effectively under distributed GPU training while preserving high rendering quality. We appreciate the reviewer’s suggestion and will include this discussion and additional results in the revised version.

[1] gsplat: An open-source library for Gaussian splatting. JMLR 2025.

---

Q2: Deep analysis of the proposed method

1. Differences Between PSNR and LPIPS

We appreciate the reviewer’s attention to metric consistency. However, it is critical to note that PSNR and LPIPS measure fundamentally distinct aspects of reconstruction quality, and their divergence does not inherently indicate a failure in detail recovery:

• PSNR quantifies pixel-level intensity/color error (MSE-based). A high PSNR confirms that models accurately match the ground truth in terms of low-level color/brightness fidelity, which is a prerequisite for detail recovery, as fine textures rely on precise pixel values.

• LPIPS measures perceptual similarity using pre-trained CNN features (e.g., VGG), which are sensitive to high-level structural cues (e.g., edge continuity, semantic consistency) but may overpenalize certain benign discrepancies (e.g., subtle lighting variations in large scenes that humans barely notice).

In large-scale scenes, LPIPS can be elevated by factors unrelated to "lack of details": for example, global illumination consistency (a strength of our holistic framework) may introduce minor local color adjustments to align lighting across the scene, which LPIPS sometimes flags despite preserving fine textures.

2. Fine Detail Recover of MixGS

We directly demonstrate the capability of MixGS for detail recovery:

2.1 Qualitative Proof: According to the policy for NeurIPS 2025, we are unable to include additional links or figures. Please refer to Row 3 and Row 5 of Figure 1 in the supplementary materials. These clearly demonstrate that our method reconstructs richer details in the areas indicated by arrows, particularly for the rooftop tiles and blue building, despite exhibiting marginally lower LPIPS scores in certain scenes.

2.2 Quantitative Proof: In experiments where we removed auxiliary view-dependent information from the Rubble scene, i.e., camera poses and rotation/scaling parameters of Gaussian splats, the results presented in the table below demonstrate that reducing view-dependent information consistently degrades PSNR while simultaneously increasing LPIPS. Consequently, it would be inaccurate to simplistically conclude that incorporating more view-dependent information inherently deteriorates LPIPS performance.

Additionally, we evaluate MixGS on the MatrixCity and Campus scenes. The MatrixCity dataset comprises 5,620 training views and 741 test views, while the Campus subset of UrbanScene3D includes 5,850 training images and 21 test images. Both datasets are recognized for their challenging scale and high-frequency details. As shown in the table below, on both MatrixCity and Campus, MixGS achieves the highest rendering quality across PSNR, SSIM, and LPIPS. We will include these quantitative results and corresponding qualitative comparisons in the revised manuscript.

3. Why LPIPS Is Relatively Higher?

Our MixGS choice to prioritizes holistic consistency (critical for large scenes), instead of trivial LPIPS optimization. Hence, this does not reflect poor detail recovery. Methods like DOGS and CityGaussian use divide-and-conquer, which can create "pixel-perfect but lighting inconsistent" local regions (lower LPIPS for small patches) but fail to align textures/lighting across block boundaries (e.g., grassland suddenly shifts in the 4 rows and 2 columns in Fig. 1 of supplementary). Our holistic framework avoids this by enforcing global consistency, which may introduce minor local color adjustments to align large structures, and these adjustments are what LPIPS occasionally penalizes.

4. Conclusion

The observed phenomenon of higher PSNR with slightly elevated LPIPS reflects a trade-off in our framework to maintain global consistency in large-scale scenes, rather than indicating any failure in detail recovery. Furthermore, our MixGS approach achieves reasonable scene modeling through an explicit coarse Gaussian training phase combined with implicit decoded Gaussians, while simultaneously incorporating view-specific representation information. This cooperative combination ultimately produces superior rendering results. We appreciate the reviewer's comments and will include additional discussion to eliminate confusing description, along with more representative comparative visualizations in our revision.

We thank the reviewer for thorough evaluation and constructive feedback. We address each concern below:

Q1: Position of Decoded Gaussians

A1: We agree that decoded Gaussians may shift significantly relative to their coarse counterparts, because our offset pool tends to move primitives toward high‑frequency, texture‑rich regions within the current view frustum (e.g., bush and rooftop tiles panels in Fig.4) to improve local rendering quality.

Crucially, we observe through experiments that the decoded Gaussians remain strictly inside the original view frustum. This behavior emerges naturally during training: any decoded Gaussian shifting outside the view frustum would not project onto the current image plane (by definition of the frustum), thus contributing nothing to the rendered pixel values. Consequently, such Gaussians receive zero loss gradient during backpropagation, and the network naturally learns to restrict decoded Gaussians to regions that affect the current view. For the current camera pose, coarse Gaussians outside the frustum are irrelevant to rendering, so their exclusion does not sacrifice information needed for the current view. In other words, decoded Gaussians can roam freely within the view frustum to capture scene information, but they will not wander outside the frustum, since such movement would not improve (and indeed would harm) the training objective.

---

Q2: Should we expand the view frustum?

A2: Expanding the view frustum brings more coarse Gaussians into the visible set and thus yields additional decoded Gaussians for rendering the same camera view. In other words, a larger frustum promotes more primitives contributing to the image and can improve quality.

As suggested by the reviewer, we expand the frustum by scaling the camera’s translation by a factor of 1.05, simulating raising a drone's altitude to capture the scene. Note that during the rendering process, all the camera poses remain unchanged so that the rendered images still correspond to the same ground truth. The table below compares standard MixGS to the expanded view frustum (EV) on four benchmark scenes. The expanded view frustum could consistently improves reconstruction quality for Rubble and Residence scenes, demonstrating the benefit of including more Gaussians in the frustum. Due to time constraints, we could not explore this idea more deeply. We appreciate the reviewer’s insightful suggestion, which is interesting for our future research.

---

Q3: Post-Processing of Gaussians

A3: If the coarse Gaussians are edited, such as being repositioned or having their appearance changed, our MixGS framework can adapt efficiently. In cases of moderate changes, the decoder can simply relearn the offsets without requiring retraining from scratch. Since the decoder is conditioned on the updated coarse Gaussians, it can reassign detail representations to match the new configuration.

For more substantial edits, we can freeze the coarse Gaussians and fine-tune only the decoder in the affected regions. This localized adaptation enables MixGS to flexibly handle post-processed scenes without globally re-optimizing the full representation, making it particularly suitable for downstream editing tasks.

---

Q4: Does the method produce smooth results with a moving camera?

A4: On the one hand, all the quantitative results reported in our paper are evaluated on the official test set, which contains viewpoints unseen during training. Despite this, our method consistently achieves high rendering quality, demonstrating its generalization ability.

On the other hand, due to NeurIPS policy, we are unable to provide an anonymous link to show qualitative videos rendered under a densely sampled, unseen camera trajectory.

To further validate the smoothness of our method under continuous camera motion, we conduct experiments on the large‐scale MatrixCity dataset [1]. MatrixCity comprises 5,620 training images and 741 test images, with the test set designed to provide densely sampled, consecutive viewpoints. We evaluated our approach alongside several baselines and report the results in the table below. It can be observed that our method achieves the highest results across all metrics, demonstrating its robustness in producing smooth, high‐quality renderings as the camera moves continuously. We appreciate the reviewer’s comments and will integrate video results into our revision.

[1] Matrixcity: A large-scale city dataset for city-scale neural rendering and beyond. ICCV 2023.

We thank the reviewer for thorough evaluation and constructive feedback. We address each concern below:

Q1: Inconsistencies at Block Boundaries in Divide‑and‑Conquer Methods

A1: Existing large‑scale scene reconstruction methods employ a divide‑and‑conquer strategy, partitioning the scene into independent blocks for multi‑GPU training. Since each block is optimized in isolation, multiview constraints do not span block boundaries, which can lead to visible artifact or texture misalignment at those boundaries. Methods such as CityGaussian, CityGaussianV2, and VastGaussian attempt to mitigate these artifacts by increasing the overlap between adjacent blocks. The more recent DOGS framework further addresses boundary inconsistencies through a dedicated “consensus step.” As stated in the DOGS paper, below information acknowledges that boundary inconsistencies persist in prior methods.

“Our method presents higher‑fidelity rendering results than VastGaussian near the splitting boundary, which validates the effectiveness and importance of the consensus step” (DOGS, page.17).

Due to independent optimization of blocks breaks multi-view constraints across boundaries, such post-processing cannot fundamentally resolve the boundary inconsistency.

Regarding qualitative comparison, due to NeurIPS policy preventing us from including boundary issue visualizations in this rebuttal, please refer to DOGS’s Figures 8 and 11, which illustrate these boundary effects.

To provide a quantitative comparison, we follow DOGS’s partitioning strategy and select 300 viewpoints across block boundaries in both the Rubble and Building scenes. We render each viewpoint using the DOGS model and our MixGS method, then compute PSNR, SSIM, and LPIPS for both in below table. Despite DOGS’s consensus refinement, MixGS consistently yields higher quality at block interfaces, effectively eliminating inconsistencies at the source rather than relying on post-processing.

Compared with other methods, MixGS optimizes the entire scene as a single entity: all Gaussians are updated under global multi-view constraints, and no block partitioning is performed. This holistic design eliminates boundary inconsistencies at the source, rather than relying on post-processing alignment. We appreciate the reviewer’s suggestion: in the revised manuscript, we will include a clearer discussion of block‑boundary issues and provide illustrative visualizations in the supplementary material.

[1] Dogs: Distributed-oriented gaussian splatting for large-scale 3d reconstruction via gaussian consensus. NeurIPS 2024.

---

Q2: Evaluation on Additional Datasets

A2: As suggested, we have evaluated MixGS on the MatrixCity and Campus scenes. The MatrixCity dataset comprises 5,620 training views and 741 test views, while the Campus subset of UrbanScene3D includes 5,850 training images and 21 test images. Both datasets are recognized for their challenging scale and high-frequency details.

On both MatrixCity and Campus, MixGS achieves the highest rendering quality across PSNR, SSIM, and LPIPS, demonstrating its robustness across diverse large-scale scenes. We will include these quantitative results and corresponding qualitative comparisons in the revised manuscript. Thanks.

---

Q3: Fair Comparison Setup Regarding Appearance Embedding

A3: The Appearance Embedding (AE) module used in VastGaussian introduces a lightweight CNN that refines rendered images from the 3D Gaussian pipeline as input and encodes the discrepancy between these renderings and the ground-truth images. This component serves as an image-level post-processing refinement that operates on the rendered output, rather than directly enhancing the underlying 3D Gaussian representation. Therefore, in our main manuscript, we followed the same evaluation protocol as DOGS by disabling the AE module to ensure a fair comparison of the underlying 3D representation and rendering quality.

As suggested by the reviewer, we also implement and test the AE module for further comparisons. The complete results are shown in the table below:

These results show that even with AE enabled, MixGS maintains superior performance across all metrics, confirming that its advantage stems from the holistic 3D modeling framework rather than the absence of post-processing.

---

Q4: Limited Number of Gaussians in Highly Textured Regions

A4: To ensure that richly textured areas are faithfully reconstructed, MixGS employs two complementary mechanisms.

Adaptive Densification: On the one hand, in the coarse training stage, Gaussians are duplicated and split in regions with high-frequency textures and detailed structure (as in 3DGS [1]). As a result, areas with complex details are initially covered by a denser set of coarse Gaussians. This seeding provides a strong prior for the decoded Gaussians to refine fine details.

Offset-Driven Refinement: On the other hand, benefiting from the offset pool, the decoded Gaussians will move toward under-reconstructed and high-frequency zones during training, instead of locating near the corresponding coarse Gaussians. We show the center location of the coarse and decoded Gaussians in Figure 4 of the manuscript. It can be observed that the decoded Gaussians shift toward more complex areas (e.g., bush and rooftop tiles in Fig.4), further increasing the Gaussian density exactly where it is needed most.

Moreover, MixGS can generate multiple decoded Gaussians per coarse Gaussian by simply modifying the decoder's output dimension. We evaluated different ratios K (i.e., $N_{\Phi} : N_v$) on the Rubble scene:

Higher ratios improve quality (PSNR reaches 27.02 dB at 3:1) but increase memory usage. We use the 1:1 setting for optimal balance between performance and memory efficiency on a single 24GB GPU.

[1] 3D Gaussian Splatting for Real-Time Radiance Field Rendering. TOG 2023.