LangSplatV2: High-dimensional 3D Language Gaussian Splatting with 450+ FPS

Thank you for the constructive feedback and the positive assessment of our work! Below, we detail our responses to the review questions.

Q1:Quantitative Analysis of Training Cost

Below, we provide a quantitative comparison of training cost between LangSplatV2 and other methods on the LERF dataset, using a single NVIDIA A100 GPU. We report the total training time until convergence, peak GPU memory usage, and the number of training iterations needed to reach comparable performance:

As we can see, LangSplatV2 indeed incurs a higher training cost compared to LangSplat and LEGaussians due to the need to construct and optimize high-dimensional semantic fields. However, our main contribution lies in significantly improving test-time performance, which is more critical for real-world deployment.

Specifically, LangSplatV2 achieves 47× faster inference than LangSplat and 14× faster than LEGaussians under similar memory usage. It also yields higher 3D semantic segmentation accuracy, demonstrating a better quality-performance trade-off.

Additionally, compared to the training cost of a naive version of LangSplat trained directly with 512D language features, LangSplatV2 is more efficient (3h vs. 45h), thanks to the introduction of a global codebook and sparse coefficient field, which reduces training complexity without sacrificing feature expressiveness.

Q2:Statistical Significance of Model Accuracy Improvements

Thank you for the suggestion. Following your advice, we conducted additional experiments on the LERF and 3D-OVS datasets using different random seeds to evaluate the stability and statistical significance of our results. For each scene, we repeated the training of LangSplatV2 three times and report the mean and standard deviation of the mIoU scores in the table below.

As shown, LangSplatV2 exhibits low variance in most scenarios. In cases where relatively higher variance is observed, such as in the Kitchen and Bed scenes, all repeated runs still significantly outperform the LangSplat baseline. Overall, these results validate the statistical significance and consistency of our reported improvements, and reinforce the robustness of LangSplatV2 under different random initializations.

Q3: Further Clarification of Originality and Comparison with Concurrent Work

Thank you for the comment. We respectfully disagree with the implication that our method is similar to LEGaussian. While both methods utilize a codebook-based representation, LangSplatV2 is fundamentally different in design and motivation. It is precisely this novel direction that enables us to achieve significantly better rendering speed and accuracy compared to prior work. Below, we elaborate on the key differences and technical advantages of LangSplatV2 over LEGaussian:

1. Global 3D Codebook vs. 2D Codebook

LEGaussian first builds a 2D codebook by quantizing features in the image space, then learns a 3D model to predict the index of the codebook via classification. In contrast, LangSplatV2 directly learns a global 3D codebook shared across all Gaussians in true 3D space. This allows our method to learn quantization directly in the 3D spatial domain, rather than in projected 2D images, resulting in lower quantization error and more accurate 3D representation modeling.

2. No MLP Bottleneck

LEGaussian relies on an MLP to map Gaussian features to a codebook index, which introduces computational overhead and acts as a bottleneck. LangSplatV2 eliminates the MLP altogether, removing this bottleneck and achieving much faster rendering and training performance.

3. Efficient Sparse Coefficient Splatting

We propose a new Efficient Sparse Coefficient Splatting mechanism, which assigns only a small number of nonzero coefficients per Gaussian, reducing the effective rendering dimension to $K$. In contrast, LEGaussian renders with full high-dimensional feature vectors, which is less efficient and limits scalability in practice.

Moreover, LEGaussian operates in a compressed latent space and does not preserve the full expressiveness of high-dimensional language features. Its 2D codebook design also fails to fully leverage 3D spatial priors. LangSplatV2, by contrast, learns and renders high-dimensional language features directly in 3D, leading to more semantically meaningful and accurate reconstructions.

As shown in Table 7 of our main paper, LangSplatV2 achieves superior semantic segmentation accuracy, faster inference speed, and comparable memory usage across three benchmark datasets, compared with LEGaussian, demonstrating the practical impact of our novel formulation.

Thank you for the constructive feedback and the positive assessment of our work! Below, we detail our responses to the review questions.

Q1: My main concern is the related work section which seams like an arbitrary list of work related to 3DGS in some parts and misses related work which also aims to speed up rendering of 3DGS.

Thank you for the helpful suggestion. We will revise and expand the Related Work section in the main paper accordingly. Specifically, we will reorganize the current content and include a dedicated paragraph discussing 3D Gaussian Splatting acceleration, including relevant works such as Compressed 3D Gaussian Splatting that also adopt codebook-based strategies for faster rendering. We will include the following discussion in the related work section.

To reduce computational overhead and improve rendering speed, recent works have proposed various strategies to accelerate 3D Gaussian Splatting (3DGS). One line of work compresses the Gaussian representation using vector quantization and codebooks. For instance, Compressed 3D Gaussian Splatting [1] adopts sensitivity-aware clustering and quantization-aware training to compress both directional colors and Gaussian parameters, achieving up to 31× compression and 4× rendering speedup. CompGS [2] similarly applies K-means-based quantization to achieve 40–50× storage reduction and 2–3× faster rendering with minimal quality loss.

Other approaches focus on structural regularization or architectural simplification. Self-Organizing Gaussian Grids [3] reorganize Gaussian parameters into spatially coherent 2D grids, reducing redundancy via local homogeneity. LightGaussian [4] uses pruning based on global significance and vector quantization, achieving 15× compression and 200+ FPS rendering. [5] proposes to reduce the number of Gaussians via a learnable mask and introduces compact view-dependent color encoding, achieving a 25× reduction in storage. Techniques like Speedy-Splat [6] and SORT-Free Splatting [7] improve runtime performance through optimized rasterization and differentiable approximations of alpha blending.

In contrast to these works that focus on accelerating RGB-based scene rendering, our method tackles the unique challenge of handling high-dimensional Gaussian features (e.g., 512D). Extending prior compression techniques [1, 4] to our setting would require learning and quantizing such high-dimensional attributes per Gaussian, leading to excessive memory usage. Moreover, directly rendering with full 512D features remains computationally demanding. In contrast, our proposed method circumvents the need to directly learn and render high-dimensional features for 3D Gaussian points.

[1] Niedermayr, Simon, Josef Stumpfegger, and Rüdiger Westermann. "Compressed 3d gaussian splatting for accelerated novel view synthesis." CVPR 2024.

[2] Navaneet, K. L., et al. "Compgs: Smaller and faster gaussian splatting with vector quantization." ECCV 2024.

[3] Morgenstern, Wieland, et al. "Compact 3d scene representation via self-organizing gaussian grids." ECCV 2024.

[4] Fan, Zhiwen, et al. "Lightgaussian: Unbounded 3d gaussian compression with 15x reduction and 200+ fps." NeurIPS 2024.

[5] Lee, Joo Chan, et al. "Compact 3d gaussian representation for radiance field." Proceedings of the CVPR 2024.

[6] Hanson, Alex, et al. "Speedy-splat: Fast 3d gaussian splatting with sparse pixels and sparse primitives." CVPR 2025.

[7] Hou, Qiqi, et al. "Sort-free gaussian splatting via weighted sum rendering." ICLR 2025.

Q2: Training numbers should be mentioned in the paper (not just reference to appendix)

Thanks for the suggestion. We will mention it and provide a detailed analysis in the main paper in the revised version.

Thank you for the constructive feedback and the positive assessment of our work! Below, we detail our responses to the review questions.

Q1: Video Demo

Thank you for the suggestion. Due to NeurIPS rebuttal guidelines, we are unable to upload images or videos during the rebuttal phase. However, we will publicly release video demos that show visual comparisons with the baselines upon acceptance.

Q2: Performance Gap in the Kitchen Scene

Thanks for the question. Actually LangSplatV2 attains a more accurate 3D language field in the Kitchen scene.

To better understand the performance gap, we encourage referring to Table 2 of our main paper, where LangSplatV2 significantly outperforms LangSplat in the 3D Semantic Segmentation task (59.1% vs. 44.5%). This task provides a more reliable and comprehensive assessment of the learned 3D language field, as it considers the overall semantic consistency of the entire 3D space rather than relying on a single point prediction.

It's important to note that the 3D Object Localization task uses the location with the maximum similarity as the predicted object center, which can be disproportionately affected by noise and local artifacts. In the Kitchen scene specifically, this sensitivity—combined with the limited number of test samples—results in high variance. In fact, when we ran multiple evaluations on the Kitchen scene, the average localization accuracy of LangSplat was 79.5%, compared to 86.4% for LangSplatV2, demonstrating the superiority of LangSplatV2.

Finally, Figure 5 qualitatively validates that LangSplatV2 exhibits more semantically accurate and spatially coherent activations, reinforcing that it indeed learns a better 3D language field in the Kitchen scene.

Q3: Workarounds for the Training Time

Thank you for your question. The increased training time compared to LangSplat primarily stems from the need to learn high-dimensional 3D semantic fields in LangSplatV2, which naturally incurs more computational overhead. We acknowledge this trade-off and have actively explored ways to mitigate it.

Among the techniques we tested, one of the most effective was the initialization strategy for the global codebook. Instead of using random initialization, we first extract all semantic features from the input 2D images and perform clustering. The resulting cluster centers are then used to initialize the codebook. This initialization significantly improves convergence speed.

As shown in the table below, training with this initialization (denoted as “w init”) achieves much lower training loss at early iterations compared to the version without initialization (“w/o init”). Specifically, with only 10K iterations, the “w init” version reaches a similar loss level as the “w/o init” version after 30K iterations. As a result, we were able to reduce LangSplatV2’s training iterations from 30K (used in LangSplat) to just 10K, effectively narrowing the training time gap. We recognize that training efficiency remains an important concern and plan to further investigate strategies to accelerate training. We believe this will be a valuable and interesting direction for future work.

Q4:Will you open-source the code?

We commit to releasing the complete source code, training scripts, and model weights to ensure full reproducibility upon acceptance.

Q5:Typo

Thanks! We will fix it in the revised version.

Thank you for the constructive feedback and the positive assessment of our work! Below, we detail our responses to the review questions.

Q1: Runtime Analysis

Thanks for the suggestion. We provide a detailed runtime comparison in the table below.

The results are obtained on the LERF dataset with one A100 GPU. It can be observed that LangSplatV2 achieves the fastest speed at every stage, leading to a substantial improvement in inference efficiency. Furthermore, Tables 2, 3, and 4 in the main paper demonstrate that LangSplatV2 attains state-of-the-art performance across multiple datasets. In summary, LangSplatV2 not only achieves superior or competitive query accuracy but also exhibits significantly higher speed.

Q2:It would also be good to see code open-sourced.

Yes, we will release the complete source code, training scripts, and model weights to ensure full reproducibility upon acceptance.

Q3:Input Image Resolutions

Thank you for the question. The input image resolutions used in our evaluation are: 730×980 for LERF, 1080×1440 for 3D-OVS, and 1080×1620 for Mip-NeRF360. These datasets cover a wide range of resolution settings. As shown in the table below, our method LangSplatV2 consistently outperforms prior approaches such as GS-Grouping and LEGaussian in terms of segmentation accuracy, query speed, and memory efficiency across all these datasets. Despite the varying input resolutions, LangSplatV2 maintains real-time query performance (as low as 2.6 ms on LERF, 4.8 ms on 3D-OVS, and 3.8 ms on Mip-NeRF360) while also achieving the highest Segmentation IoU on all three benchmarks. This demonstrates that our method is robust to input resolution changes and can deliver high-quality, efficient performance across different scene configurations.