BevSplat: Resolving Height Ambiguity via Feature-Based Gaussian Primitives for Weakly-Supervised Cross-View Localization

We thank Reviewer WBEg for the detailed review and constructive suggestions. We address the major concerns below.

1. Writing

We sincerely thank you for your careful review and valuable feedback. We would like to provide explanations for the points you raised.

More Explanation of IPM.

IPM was defined on page 1, line 32 and visualized in Figure 1 to motivate our work. It assumes that all points on the ground plane, and exploit the ground plane Homography for the ground-to-satellite projection. We will highlight this more clearly in the revision.

Concepts of "Height Ambiguity".

Thank you for the question. We use the term "height ambiguity" to describe the challenge of projecting a 2D ground-level image to a Bird's-Eye View (BEV). Because a single 2D pixel could represent points at various depths and heights, its true 3D position is ambiguous.

Methods like IPM resolve this ambiguity by assuming a flat ground plane. This introduces significant errors for any object with height, causing the characteristic distortions and smearing we aim to solve. We will clarify this definition in our revision

Different Localization Task Definitions in Related Works

We will clarify these terms in the paper.

• One-to-many localization: this task aims to determine the coarse location of a ground camera by retriving its similar satellite image counterparts from the database. The similar satellite image counterparts can be several, and thus is named as one-to-many localization.
• Pixel-level localization: once the coarse location of a ground camera is given, the pixel-level localization aims to determine which pixel of the satellite image corresponds to the precise ground camera pose. This paper solves for the pixel-level localization.

Rework the presentation of the paper's contributions.

Thank you for the suggestion. Our contribution is re-worked below:

• We introduce BevSplat, a new framework that synthesizes a Bird's-Eye View (BEV) representation by modeling the ground scene as a collection of feature-based 3D Gaussian primitives.
• Our approach explicitly models 3D geometry to resolve the critical issue of height ambiguity, a limitation of traditional projection methods like IPM that leads to severe BEV distortions.
• We demonstrate that BeVSplat achieves new state-of-the-art performance on the challenging KITTI and VIGOR datasets, significantly outperforming prior methods within the practical weakly-supervised localization paradigm.

Rotation Estimation

Yes, our rotation estimation module is adopted directly from G2SWeakly. We will make this explicit in our revised pipeline description.

2. Why not compare with "View from Above", will claims like "in cross-area evaluations, our method even surpasses supervised approaches" (line 224) still hold?

The primary reason we did not include a direct comparison with "View from Above (VFA)" [20] was the unavailability of its official source code and pre-trained models, which prevents a direct and fair reproducible comparison. We note that even the most recent state-of-the-art supervised method, FG² [21] (CVPR'25), also omits a comparison to VFA for the same reason.

To address your crucial question about whether our claim on line 224 still holds, we have benchmarked our weakly-supervised method against this new fully-supervised SOTA method, FG²[21]. The results are presented below:

It can be seen that, although it outperforms our methos on same-area evaluation, our method shows superior cross-area evaluation results, which are consistent in the conclusions in our submission.

3. Comparison with other BEV projection Baselines, e.g. [5] and [20].

We thank the reviewer for this suggestion. To provide a comprehensive analysis, we have tried our best during this rebuttal period to re-implement "View from Above" [20] and adapt OrienterNet [5] to our weakly-supervised framework. The following sections present a direct comparison of performance, efficiency, and methodology.

Performance Comparision.

Runtime and Memory Analysis.

Implementation Details.

For a rigorous comparison, we adapted the baselines to our weakly-supervised setting.

• View from Above [20]: Due to the unavailability of public code, we re-implemented the method based on the paper's descriptions.
• OrienterNet [5]: We used the official code, modifying only its satellite feature extractor to match ours for a controlled comparison.
• Loss Adaptation: Both baselines, originally designed for full supervision, were adapted to our weakly-supervised tasks (λ1=0: No GPS, λ1=1: Noisy GPS) using our loss framework (Eq. 7) to ensure a meaningful evaluation.

Analysis of Experimental Results.

View from Above [20].

• Impractical Horizon Assumption: Relies on a fixed horizon height τ, which is not robust to real-world camera poses and leads to incorrect feature separation.
• Requires Strong Supervision: Its keypoint selection mechanism needs precise camera poses for supervision, making it unsuited for our weakly-supervised task. This explains its severe performance drop, especially with No GPS (λ1=0).
• Inefficient: Its iterative refinement design results in excessive computational costs. Our re-implementation required 62GB VRAM for training and 209ms for inference (the original paper reported 68GB and 222ms).

OrienterNet [5].

• Poor Generalization for Occluded Regions: OrienterNet produces a dense BEV by "hallucinating" or "in-painting" features for occluded regions. While this can work when guided by an accurate GPS label in same-area, it fails to generalize well to unseen environments, explaining its poor performance in cross-area evaluations.
• Ineffective Occlusion Handling: It uses simple weighted averaging for BEV projection, which is less accurate for handling vertical occlusions than BeVSplat's principled alpha blending.
• Computationally Heavy: OrienterNet uses a complex h x w x d polar depth representation to perform an attention-based sum over ground features. This implicit, high-dimensional process incurs substantial computational and memory overhead to produce a denser BEV.

Consistent with Broader Findings.

Our findings align with broader trends for complex models in this domain. For instance, our baseline G2SWeakly[1] also benchmarked against a transformer-based counterpart[4] (see Table 2 in their paper). They similarly found that while the transformer had an advantage with Noisy GPS (λ1=1), its performance degraded significantly in the No GPS (λ1=0) setting. This supports our conclusion that complex architectures often struggle to generalize without strong supervisory signals.

4. Conclusion.

In summary, BevSplat outperforms these strong baselines because it avoids their simplifying assumptions and computationally expensive designs. Our explicit 3D Gaussian representation is more robust, generalizes better, and is inherently suited for weak supervision. We acknowledge a minor speed trade-off compared to the simpler G2SWeakly, which we identify as a direction for future work.

References

[4] Shi, Yujiao, et al. "Boosting 3-dof ground-to-satellite camera localization accuracy via geometry-guided cross-view transformer." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023. [21] Xia, Zimin, and Alexandre Alahi. "FG^2: Fine-Grained Cross-View Localization by Fine-Grained Feature Matching." Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.

We thank Reviewer aKDE for the positive evaluation and insightful comments. We have corrected the typos (In Figure 1 and on line 77) in the revised manuscript. Our responses are below.

1. Carefule Explanation of the Weakly Supervised Setup

Thank you for the question. We focus on a weakly-supervised setting because precise GPS data is often unavailable in the real world. To do this, we adopt the weakly-supervised setup from G2SWeakly [1], which defines two scenarios:

λ=0: the error of the location labels for ground images in the training dataset is the same as the error that the model aims to refine during deployment. For example, the error of location labels for ground images in the training data set is +/- 20m. During testing, the model is also given a location of query images with error up to 20m and aim to reduce this error.

λ=1: relatively more accurate location labels for ground images in the training data are available than the poses we aim to refine during employment. For example, the model was trained with images whose location labels have an error of +/- 5m. During testing, the query images have an initial location estimate with errors up to 20m, and the model aim to reduce this error.

2. On "Same Area" vs. "Cross Area"

We apologize for the lack of clarity.

• Same-Area: The test query images are from the same geographical regions as the training set, but are not the same images.
• Cross-Area: The test images are from entirely new geographical regions not seen during training, testing the model's generalization.

3. On the Benefits in Supervised vs. Weakly-Supervised Settings

We thank the reviewer for this excellent question. Although our paper focused on the weakly-supervised setting, our framework also demonstrates strong performance with full supervision. To illustrate this, we trained our model and the G2SWeakly baseline in a fully supervised setting on the KITTI dataset. The results are as follows:

As the results show, while switching to a fully supervised setting, our model is highly competitive, even outperforming the most recent SOTA, FG² (CVPR 2025)[21], in the challenging cross-area task. However, we found that our weakly-supervised model (λ1=1) achieves even better cross-area performance than our own fully-supervised version. This suggests that precise supervision may cause overfitting to the training domain's biases, which is contrary to our goal of building a more generalizable system.

Therefore, our paper's focus on the weakly-supervised setting is twofold. First, it addresses the practical challenge that high-quality GPS data is often unavailable in the real world. Second, it is the setting where our method paradoxically achieves its best and most robust generalization performance.

4. Convincing Evidence Regarding Efficiency

Thank you for this insightful comment regarding the efficiency of our 3D representation. We would like to clarify why our model, despite its cost, is essential for the task.

A core premise of our work is that explicitly modeling height is necessary to obtain a high-fidelity BEV representation. Methods that ignore 3D information, like Inverse Perspective Mapping (IPM), create distorted BEVs by "smearing" tall objects across the ground plane (as shown in our Fig. 1 & 4). This geometric corruption makes precise localization exceptionally challenging. Our method uses Gaussian primitives specifically to model this essential 3D structure and produce a clean, distortion-free BEV.

While our approach requires slightly longer inference time and larger GPU memory than traditional IPM, however, when compared to other approaches—such as the transformer-based LSS and OrienterNet, or the RAFT-optimized DenseFlow—our method is significantly more efficient. The results on KITTI dataset are shown bellow:

5. Key Factors of Performance Improvement

While the strong FiT3D DINO backbone contributes to a significant improvement, our method still demonstrates a clear improvement over both IPM and direct point cloud projection using the same backbone, as shown in Table 4. This indicates that our proposed BEV synthesis module is crucial to the performance gains.

6. What Motivates the Use of Fine-tuning with a DPT-like Module [48]?

Thank you for pointing out this question. Using DPT-like models is a common practice for 3D vision tasks; for example, VGGT (CVPR 2025) utilizes DPT for point cloud reconstruction, depth estimation, and feature matching. Although we are the first to apply DPT-DINO for feature extraction in the specific sub-field of cross-view localization, our motivation is to similarly obtain features that are rich in 3D information. This is analogous to human navigation, where in addition to semantic information, an understanding of the real 3D scene is also crucial for localization.

However, obtaining such 3D-aware features to bridge the significant ground-satellite domain gap is non-trivial. Simpler fine-tuning methods like direct end-to-end training or LoRA[62] fail, as they either lose crucial texture details or are not powerful enough to adapt the foundation model. We use a DPT-like module because its multi-scale feature fusion architecture is uniquely suited for this challenge. It successfully adapts the backbone by preserving both the low-level texture and high-level semantic information required for matching across these different domains.

To clarify the motivation that was missing from our original Table 4, we provide the following ablation study on the KITTI dataset which was previously omitted:

7. Azimuth Values Exactly Match Those of G2SWeakly

Thank you for your observation. The azimuth results are identical because our work focuses on improving translational localization, and we directly adopt the rotation estimator from our baseline, G2SWeakly [1], without any changes.

We did mention this on lines 212-214, but we agree this could be made much clearer. We will explicitly state this in our method description and add a note to Table 1 in the revision.

References

[60] Philion, Jonah, and Sanja Fidler. "Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d." European conference on computer vision. Cham: Springer International Publishing, 2020.

[62] Hu, Edward J., et al. "Lora: Low-rank adaptation of large language models." ICLR 1.2 (2022): 3.

We sincerely thank Reviewer arEj for the thorough review and constructive feedback. We address the major concerns below.

1. Comparison with Lift-Splat-Shoot[60].

We thank the reviewer for this excellent question and for the suggestion to compare our method with an important baseline like Lift-Splat-Shoot (LSS) [60]. To provide a direct and fair comparison, we adapted the official LSS open-source code for our cross-view localization task. The key modifications were changing the input to a single ground camera, adding a satellite feature branch, and training the model with our loss function.

The performance and efficiency results on the KITTI dataset are presented below.

Performance Comparision.

Runtime and Memory Analysis.

Analysis of Experimental Results.

Regarding the reviewer's point on computational cost, our analysis shows that our approach is actually more efficient than our adapted LSS baseline, particularly in training memory and inference time. We attribute this to the different 3D representations: LSS constructs a dense H x W x D point grid, which creates a larger computational bottleneck during the splatting stage compared to our sparser H x W set of Gaussian primitives.

In terms of localization accuracy, we believe the performance difference stems from two key design choices. First, our method's use of alpha blending allows for a more principled handling of occlusions compared to LSS's pillar-based pooling. Second, LSS's design generates a dense BEV, which forces the network to "hallucinate" features for occluded areas. This is challenging in a weakly-supervised setting that lacks a BEV ground truth. Our method avoids this by generating a sparse but more faithful representation of only the visible scene.

We believe these design choices make our approach particularly well-suited for the high-precision, weakly-supervised localization task. We will add this detailed comparison to our paper and thank you for prompting this valuable analysis.

2. Multi-frame Fusion Comparison: BevSplat vs. IPM and Direct Point Cloud Projection.

We thank the reviewer for this valuable suggestion. To demonstrate our method's consistency and fusion capabilities, we have conducted a new multi-frame comparison against both IPM and direct point cloud projection baselines.

The tables below present the performance on the KITTI dataset using a sequence of six frames. The values in parentheses show the percentage improvement from fusing six frames over the single-frame results

Results for λ1 = 0：

Results for λ1 = 1：

Our BevSplat framework demonstrates superior multi-frame fusion capabilities for the following reasons:

• Inverse Perspective Mapping (IPM): BEV representations generated via IPM are prone to significant artifacts and distortions, which fundamentally hinder effective temporal fusion.
• Direct Point Cloud: Projections of raw point clouds result in sparse representations that handle occlusions poorly, often causing background features to bleed through foreground objects. This issue is not resolved well by aggregating multiple frames.
• Our Method: In contrast, BevSplat utilizes adaptive Gaussian primitives to create a dense, coherent BEV that faithfully represents the road topology. This provides a robust foundation for multi-frame fusion, as shown in Fig. 9 of our supplement.

We will incorporate these visualizations into our revised manuscript.

3. Metric Depth of DepthAnythingV2.

We appreciate the opportunity to clarify this important implementation detail, and we apologize our manuscript was not sufficiently clear on this point.

The DepthAnythingV2 project provides models that support both relative depth estimation (on arbitrary images) and metric depth estimation (when trained on datasets with ground-truth metric depth). We utilize the latter for our experiments. By using a model that directly outputs depth in meters, the scale ambiguity challenge is addressed at the input stage, providing our localization framework with the necessary metric information.

4. High Computational Cost on Representation.

Thank you for this insightful comment regarding the necessity of our 3D representation. We would like to clarify why this component, despite its cost, is essential for the task.

A core premise of our work is that explicitly modeling height is necessary to obtain a high-fidelity BEV representation. Methods that ignore 3D information, like Inverse Perspective Mapping (IPM), create distorted BEVs by "smearing" tall objects across the ground plane (as shown in our Fig. 1 & 4). This geometric corruption makes precise localization exceptionally challenging. Our method uses Gaussian primitives specifically to model this essential 3D structure and produce a clean, distortion-free BEV.

While our approach requires slightly longer inference time and larger GPU memory than traditional IPM, however, when compared to other approaches—such as the transformer-based LSS[60] and OrienterNet[5], or the RAFT-optimized DenseFlow[18]—our method is significantly more efficient. The results on KITTI dataset are as shown below:

[60] Lift, Splat, Shoot: Encoding Images from Arbitrary Camera Rigs by Implicitly Unprojecting to 3D, ECCV 2020

Dear Reviewer arEj,

We sincerely appreciate your time and effort in reviewing our manuscript and offering valuable suggestions. We noticed that you have submitted your final score.

May we kindly ask if our response has addressed your concerns? Please feel free to let us know if you have any other questions. We would be more than happy to address them.

Best regards,

The Authors

We thank Reviewer gAUg for the constructive feedback and insightful suggestions. We address the raised points below.

1. Sensitivity to Depth Prediction Quality and Failure Cases.

We thank the reviewer for this suggestion. We evaluate our framework's performance with three different depth foundation models: DepthAnythingv1, ZoeDepth, and DepthAnythingv2:

The results demonstrate that while a more accurate depth model improves performance, the overall system is not highly sensitive to the choice of different depth estimators. This robustness stems from our end-to-end differentiable design, which optimizes the initial 3D Gaussian positions during training, compensating for minor discrepancies between different depth priors. Our framework can seamlessly leverage future advancements in monocular depth estimation. We will include this analysis in our paper.

2. Robustness in Adverse Environmental Conditions

We thank the reviewer for this crucial point. To test our method's robustness, we generated synthetic Rain, Fog, and Night data for KITTI (following the methodology of "Robust-Depth"[61], ICCV 2023). This allows for a controlled comparison against the G2SWeakly baseline under challenging conditions.

The results show that while both methods are affected by adverse conditions, our approach demonstrates greater relative robustness. The reason lies in how each method handles corrupted input:

• IPM-based methods like G2SWeakly project visual artifacts (e.g., rains, fog) directly onto the BEV, creating severe geometric distortions that corrupt the final representation.
• In contrast, our method, while starting with a less accurate depth map in these conditions, still preserves a stable underlying 3D structure. It avoids the fundamental stretching errors of IPM and is better able to ignore atmospheric noise, leading to a more graceful degradation in performance.

We will add this analysis to our paper. We also believe that as depth estimation foundation models continue to improve, the robustness of our method will increase further. Thank you for prompting this important investigation.

3. Is Np Sensitive to Specific Dataset?

We conducted an ablation study on the number of Gaussian primitives per pixel ($N_{p}$) across both the KITTI and VIGOR datasets to validate our design choice. The results for KITTI are presented in Figure 5 of our main paper, and the new results for the VIGOR dataset are provided below.

The results on VIGOR are consistent with our findings on KITTI: performance is optimal when $N_{p}$=3. As we discuss in our paper:

• Using too few primitives can limit the model's ability to fill gaps in sparse regions.
• Using too many primitives can make training more difficult.

This finding is also consistent with prior work. Our design was inspired by PixelSplat(CVPR 2024), which similarly found Np=3 to be a robust and effective setting across multiple datasets. Therefore, we conclude that Np=3 is a well-justified hyperparameter that should be generally applicable.

4. Runtime and Memory Analysis.

We apologize for not providing this direct comparison in the original manuscript. As we discussed in our conclusion and supplementary material (lines 53-61), we acknowledge that our 3D reconstruction incurs a modest computational overhead, resulting in slightly higher inference costs than the simpler G2SWeakly [1] baseline. We consider this a necessary trade-off for the significant gains in localization accuracy. However, our method remains highly efficient compared to more complex approaches like the RAFT-based DenseFlow [18] and the transformer-based OrienterNet [5]. We will add this direct comparison to our revised manuscript for a clearer context.

5. Clarification on Occlusion Handling in the Differentiable Blending Process

Forward Pass: Following Equation (3), we sort all contributing Gaussians by depth and render them from front to back

Backward Pass: The process is fully differentiable, allowing the loss to guide how the scene should be structured. For the feature vector ($f_{b}$): The gradient for the feature of the b-th Gaussian is computed as:

$\frac{\partial L_{all}}{\partial f_{b}} = \frac{\partial L_{all}}{\partial F_{BEV}} * T_{b} * \alpha_{b}$

The learning signal is scaled by transmittance ($T_{b}$) and opacity ($α_{b}$). This means the features of the most visible (least occluded) and most solid Gaussians are prioritized for updates.

For the opacity ($α_{b}$): The gradient of the b-th Gaussian is computed as:

$\frac{\partial L_{all}}{\partial \alpha_{b}} = \frac{\partial L_{all}}{\partial F_{BEV}} * T_{b} * (f_{b} - f_{b}^{accum})$

The gradient depends on the difference between the current Gaussian's feature ($f_{b}$) and the accumulated features behind it ($f_{b}^{accum}$). This trains the model to make a Gaussian opaque if it is needed to hide a conflicting background, effectively learning to form solid, occluding surfaces.

In short, this mechanism is directly analogous to how the original 3DGS handles RGB colors, and we have repurposed it to optimize feature representations for localization.

[61]Saunders, Kieran, George Vogiatzis, and Luis J. Manso. "Self-supervised monocular depth estimation: Let's talk about the weather." ICCV 2023.