GTR-Loc: Geospatial Text Regularization Assisted Outdoor LiDAR Localization

Thank you for your insightful comments. We have addressed each point and will integrate all corresponding revisions into the final version of the paper.

Response to W1&Q1. (1) Why "text-assisted" localization? The focus of our method is not on processing natural language, but on learning a novel geospatial text regularization to solve localization ambiguities. We use the term "text" because we convert discrete geospatial labels into text strings (e.g., "District 50, North"), which are then processed within the text modality. This textification is critical because it leverages the rich semantic knowledge embedded in the CLIP text encoder. If we use discrete IDs (e.g., 50) as categorical labels instead, the model needs to learn complex relationships between these meaningless symbols and point features from scratch. This not only presents significant optimization challenges but also leads to overfitting. The model cannot comprehend inherent spatial relationships like the adjacency of "District 50" and "District 51" or the opposition of "North" and "South."

In contrast, by converting the labels into text with semantic meaning (e.g., "District 50, North"), we leverage the vast world knowledge and text understanding capabilities embedded in pre-trained VLMs. The powerful text encoders of traditional VLMs (e.g., CLIP) have excellent robustness and generalization capabilities for text. CLIP understands that "North" and "Northeast" are semantically related. This significantly reduces the dependency on task-specific labeled data, and because of the flexibility of text, a fixed network architecture can easily adapt to different datasets or changing data categories. Hence, the proposed geospatial text is not a simple feature engineering.

(2) Re-framing the contribution. We acknowledge that the framing could be clearer. In the final version, we will refine our paper to better articulate that we use CLIP to process our novel geospatial text, rather than proposing a new natural language processing.

Response to W2&Q2. (1) Why use CLIP? We use a pre-trained CLIP encoder to leverage its rich prior, even for simple, structured text. A randomly initialized embedding layer will treat district and direction IDs as independent and arbitrary tokens. Hence, it needs to learn relationships from scratch, leading to less efficiency and poor generalization. In contrast, CLIP already understands that "North" and "South" are opposites and "Northeast" is an intermediate geographical concept. This meaningful relational structure can adapt to scene changes (e.g., adding districts or changing datasets) without retraining. CLIP can also better handle the composition of tokens like "District 50" and "North," rather than simple concatenation.

(2) Replace CLIP with embedding layers. We conduct new ablation studies by replacing the CLIP text encoder with embedding layers (ELs). Specifically, we learn two separate, learnable embedding layers (nn.Embedding) initialized from scratch: one for the 100 district IDs and another for the 16 direction IDs. The resulting vectors are concatenated and passed through an MLP to match the feature dimension of the original CLIP embedding. The results shown below indicate that using simple learnable embeddings only leads to a small performance improvement. Although it is still effective, its performance is noticeably inferior to the full GTR-Loc that uses CLIP. Hence, using a powerful, pre-trained CLIP text encoder is not overkill.

Response to W3. (1) Coarse partitioning on large-scale maps. Coarser partitions create overly large districts that can encompass multiple visually similar yet geographically separate locations. Then, the text would lose its power to disambiguate. Since the dataset scale cannot be increased, we use fewer partitions to illustrate the same situation. The original results in Table 5 show that as partitions decreases from z=100 to z=49, localization accuracy decreases. As shown below, we conduct new ablation studies with fewer partitions (z=25 and z=9). The results indicate that coarser partitions can not provide effective discriminative cues. However, as the district number is a hyperparameter, it can be scaled with the map size.

(2) Sensitivity to fine-grained partitioning. Excessively fine partitions risk overfitting due to very sparse data in each district, which can turn the textual cue into a rigid lookup key instead of a soft regularizer. However, our design demonstrates robustness to this. Results in Table 5 and the table below, respectively, show that as we increase partitions to z=144, and even to extreme values like z=2500 and z=10000, performance plateaus rather than degrades sharply. It suggests that the model is not overly sensitive to overfitting. This resilience stems from the regularizing effect of MRD distillation, which effectively smooths out noise from the sparse data within these fine-grained partitions.

Response to Q3. (1) Performance gap analysis. The small performance gap shows that knowledge distillation successfully transfers the teacher's reasoning, not just its results. By mimicking the teacher's text-regularized approach, the student learns to resolve subtle geometric ambiguities. This is highly effective on datasets focused on geometric ambiguities like QEOxford/Oxford, where the student's position error gap is minimal (as low as 1.33%/0.39%). The gap is larger on NCLT, likely due to less stable data from its Segway collection platform. Across all datasets, the consistent gap in orientation error (more than 5%) suggests that distilling knowledge for precise orientation is more challenging than for position.

(2) Failure scenario for the student. The student model's failures are confined to rare, extreme scenarios where LiDAR data contain almost no distinctive geometric features. For example, on a very long street in the Oxford dataset, the building facades, streetlights, and other structures are completely identical. In these few instances (<1% of the trajectory), the teacher model succeeds by leveraging geospatial text, an external cue that the student lacks. While these specific failures can produce large errors, their statistical impact becomes negligible when averaged over the entire dataset. In the vast majority of cases, the student effectively utilizes subtle geometric cues, achieving performance nearly identical to the teacher's.

Thank you for your insightful comments. We have addressed each point and will integrate all corresponding revisions into the final version of the paper.

Response to W1&Q1. (1) Inconsistent and indirect descriptive texts. Human-like text generation, such as TOD3Cap in Figure 3, tends to describe target-environment-relationship, such as vehicles or vegetation. These free-form scene descriptions are inherently unstable; dynamic objects and environmental changes lead to inconsistent characterizations of the same location over time, providing an unreliable description. Furthermore, these descriptions offer only indirect cues for 6-DoF pose estimation, with irrelevant details acting as noise that can impede learning. This challenge is compounded by the sparse, geometric-only nature of LiDAR point clouds, which, unlike images, lack rich texture and color.

(2) Stable and direct GTG. In contrast, our GTG produces text (e.g., "District 99, West-Southwest") that yields a standardized representation of universal pose information, creating a signal that is stable, repeatable, and directly relevant to pose estimation. The text's invariance to dynamic content provides the consistency needed to resolve localization ambiguities via regularization. Then, the LATER module refines GTG-generated text by fusing it with the current point cloud's geometry, creating viewpoint-specific embeddings that enrich the final training feature with diverse visual-geometric information beyond the core pose.

(3) Experimental validation and proof. In Table 4, we demonstrate that GTG is more effective than free-form scene descriptions. We also present this table here. We replace GTG with a SOTA 3D scene captioning model, TOD3Cap, to generate rich text. This free-form text mainly describes object motion, appearance, relationship, and environment. However, using free-form descriptions from TOD3Cap yields no significant performance gain over the vanilla model. Our template-based GTG leads to significant improvements.

Response to W2&Q2 Time evaluation. Our method is real-time and highly efficient during inference. On the Oxford/QEOxford datasets, the average processing time per sample is 29ms (34 FPS), and on the NCLT dataset, it is 48ms (21 FPS). These speeds are well within the respective scanning frequencies of 20 Hz for Oxford/QEOxford and 10 Hz for NCLT, highlighting the model’s ability to maintain high accuracy during real-time operation. We also report the runtime comparison with SCR baselines, as shown in the table below. Our runtime outperforms SGLoc and LiSA, while being on par with LightLoc. However, our superior performance demonstrates the advantage of GTR-Loc in both accuracy and efficiency.

Our model's complex self/cross-attention modules are confined to the teacher network (blue flow, Fig. 2), which is used only during offline training. This design ensures that our final inference model, obtained via MRD distillation, is a lightweight, LiDAR-only network (red flow, Fig. 2) with efficiency comparable to baselines like LightLoc. To better highlight this, we will move the detailed time analysis from the appendix into the main text.

Response to W3&Q3. (1) Is text integration the bottleneck? First, considering our text input as an "approximate ground truth location" may be a misunderstanding. Taking the QEOxford dataset as an example, its area is approximately 1000m x 1500m, which we divide into a 10x10 grid. This means each geospatial district is about 100m x 150m in size. For a localization task requiring meter- or even sub-meter-level accuracy, a regional label spanning hundreds of meters is far from an "approximate ground truth." In addition, our framework is SCR, which predicts world coordinates for each input point. The ground truth label is coordinate labels rather than pose labels.

Second, we would like to respectfully clarify that GTR-Loc achieves superior performance on all three datasets. Our method outperforms all baselines on QEOxford, achieves SOTA position accuracy on Oxford and NCLT, and ranks second in orientation estimation. Specifically, on QEOxford, GTR-Loc (0.75m/1.03°) achieves a significant 9.64%/8.04% improvement in both position and orientation over the previous SOTA, LightLoc (0.83m/1.12°). GTR-Loc consistently outperforms LightLoc on both the Oxford (2.59m/1.19° vs. 2.67m/1.25°) and NCLT (1.40m/2.62° vs. 1.46m/2.80°) datasets. The absolute improvement in position accuracy over previous leading SCR methods, LiSA and LightLoc, is significant. The results in Table 7 mainly demonstrate the effectiveness of our MRD distillation, with the LiDAR-only model achieving performance nearly on par with the LiDAR-text version.

(2) Does text regularization merely accelerate training? Our method does not simply accelerate training; it fundamentally alters the learning process. Traditional SCR methods, using per-point loss, can get trapped in local minima when encountering geometrically similar scenes. To address this, our method introduces text regularization as a unified, global constraint. For instance, the text "District 50, North" mandates that all predicted coordinates are macroscopically consistent with this region, creating a more favorable optimization landscape when combined with the original per-point loss. Therefore, this text guidance is not merely a training accelerator but a fundamental tool for resolving localization ambiguities.

We conduct new ablation studies by taking the baseline model (the LiDAR-only network architecture without a teacher and MRD) and extending its training time by 3x (from 25 to 75 epochs on QEOxford/Oxford and 30 to 90 epochs on NCLT). As shown in the table, simply training for longer yields nearly no performance gain without MRD. It comes nowhere close to matching the results of GTR-Loc w MRD (normal epoch) under its standard training schedule. In addition, training GTR-Loc for a longer time also will not improve performance because the network has already converged. This provides strong evidence that our method reshapes the learning process, rather than merely accelerating it.

Response to Q3: Thank you for your further feedback on our rebuttal. We appreciate the opportunity to clarify this point and will make it clearer in the revision.

Using geospatial text for inference may be impractical because generating it requires gt poses or pose-relevant information, the very thing we aim to estimate. To solve this, we design the MRD distillation. We use a teacher-student framework for training. The teacher (LiDAR+Text) learns to resolve ambiguities, while the student (LiDAR-only) mimics the teacher's outputs. Through distillation, the LiDAR-only student inherits the teacher's text-assisted knowledge, learning to "narrow the search space" on its own using only point clouds.

Importantly, our deployed model (a distilled, LiDAR-only GTR-Loc) requires no text during inference and uses only a single LiDAR scan. However, we further investigate the case where text is used during inference. We first test a hypothetical scenario where gt poses are available to generate perfect text for the teacher model (blue flow, Fig. 2) at inference. The ablation study in Table 7 demonstrates that the distilled model's performance (row 4 of the table below) is nearly identical to the teacher (row 2 of the table below) that uses perfect text during testing. This proves that our MRD successfully transfers the vast majority of the text's benefits into the LiDAR-only model.

Since gt poses are usually unavailable during inference, we then try to train an auxiliary classification network to predict district and direction categories at test time. The predicted categories are used as district and direction IDs for text generation. The network comprises a LightLoc encoder and 4 FC layers. The results shown in the appendix (row 3 of the table above) indicate that the teacher with predicted text inputs is not effective. It yields virtually no performance benefits at the cost of higher computational complexity and longer test times. Inaccurate position and orientation classification (the table below) leads to wrong text generation, thus resulting in poor localization.

Thank you for your insightful comments. We have addressed each point and will integrate all corresponding revisions into the final version of the paper.

Response to W1 Assumptions on Spatial Discretization. (1) Long-term environmental changes. Although our method relies on a predefined map partition, our partitioning scheme is designed to be robust to environmental changes. The partition is not based on semantic features of the environment (like roads or buildings), but on an abstract, universal coordinate grid. Hence, with long-term environmental changes, such as construction and demolition, seasonal and vegetation changes, the partition is still stable. Results on the NCLT dataset, known for its data collection spanning several months, also highlight our method's robustness to long-term variations.

(2) Large-scale deployment. Our design is also methodologically scalable for large-scale deployment. While we use a 10x10 uniform grid in this paper, it is not a limitation of the method. For larger deployments (e.g., an entire city), the strategy can be seamlessly extended. For example, we can adopt standard practices from geospatial applications, such as a hierarchical UTM grid, to programmatically partition an area of any size. In addition, our ablation study (Table 5) also demonstrates the robustness of our approach (e.g., increasing from 100 to 144 districts). This is critical for scalability, as a reasonable, predefined partition is sufficient, making the framework practical for real-world use.

Response to W2&Q1 Vision-language fusion. (1) Novelty of LATER. The novelty of our approach stems from both its design purpose and specific architectural details. Previous VLMs bridge vision and language via methods like CoOp's learnable prompts, BLIP's contrastive-fusion learning, or Flamingo's gated cross-attention for injecting visual tokens into frozen LLMs. They are designed for image understanding tasks such as image captioning or zero-shot classification, while our task is LiDAR localization. LATER is specifically designed to dynamically refine a pose-aware text embedding based on the unique geometric features of the current LiDAR scan, thus regularizing localization.

In addition, our novel module details are also different from previous VLMs. Following the well-established paradigm set by VLMs, LATER also utilizes a Transformer architecture and learnable prompts. However, LATER does not simply fuse the point cloud and text; instead, it independently trains a Transformer to dynamically refine learnable prompts ${v_n}$ using point features $F_p$. Subsequently, it concatenates these visually-tuned prompts $v'_n$ with a static text embedding that contains coarse pose information from our custom generator.

(2) Advantages of LATER. Our LATER is preferable because it is a tailored solution to solve view-dependent ambiguities in LiDAR localization. Directly adopting a fusion module from models like CoOP or Flamingo would be suboptimal for our task. They are designed to fuse dense images with rich, descriptive texts, aiming to maximize the semantic understanding of scene contents. However, our inputs are sparse LiDAR point clouds and structured texts, and we aim to regularize localization. We also conduct a new ablation study by replacing LATER with CoOP and Flamingo. We retain the original fusion configuration, replacing only the fusion module itself. The results shown in the table below indicate that with these fusion mechanisms, we can not achieve optimal results. This proves that the unique dynamic refinement mechanism of LATER is both effective and necessary.

Response to Q2 Result variability. In our initial submission, we follow the common practice within the LiDAR localization field, where most prior works report mean errors from a single, deterministic run. This ensures that our results are directly and fairly comparable to the existing SOTA. To further strengthen our conclusions, we have conducted additional experiments to report the variance and confidence interval. We re-run our method (GTR-Loc) and the strongest baseline (LightLoc) for 5 iterations using different random seeds on the QEOxford and NCLT datasets. The results (mean ± standard deviation) of position/orientation errors are presented in the table below.

As shown in the table above, the standard deviations across multiple runs are very small, indicating that the performance of both methods is highly stable. As shown in the table below, GTR-Loc's variance (Var) of position error and orientation error is consistently and robustly superior to that of LightLoc across all 5 runs. The 95% confidence intervals (CI) of our method's performance do not overlap with the baseline's, confirming that the reported improvements are statistically significant and not an artifact of random chance.

Response to Q3 Adaptivity of discretization. (1) Learnable discretization schemes. In our current work, we use a fixed, uniform grid partitioning primarily for its interpretability and efficiency. A uniform grid is a simple, structure-agnostic approach. To explore the potential benefits of adaptive partitioning, we conduct new ablation studies with a learnable strategy. Specifically, we use learnable parameters, nn.Parameter, to learn districts z and directions d, with initial z=100 and d=16. The results are shown below, where the learnable discretization scheme is not better than ours. We think the reason is that a simple learnable scheme may introduce optimization challenges, as this important parameter is difficult to learn.

(2) Failure cases. First, coarse districts containing geometrically similar but distinct locations (e.g., parallel streets) may receive an identical text signal that fails to provide disambiguation. In such cases, performance may revert to the baseline, though this text is benign rather than actively harmful. Second, partition boundaries cutting across long roads can cause textual labels to hop during training. However, this issue is mitigated by our LATER module, which leverages real-time visual input, and is further smoothed by distillation, ensuring minimal impact on the final inference model.

We thank the reviewers for their constructive and positive comments. We have addressed all points individually and will incorporate all corresponding revisions into the updated manuscript. We hope and have endeavored to resolve all of the reviewers' concerns.

Thank you for your insightful comments. We have addressed each point and will integrate all corresponding revisions into the final version of the paper.

Response to W1&Q2. (1) Disadvantages of rich text for LiDAR localization. Human-like text generation, such as TOD3Cap in Figure 3, tends to describe target-environment-relationship, such as vehicles or vegetation. These free-form scene descriptions are inherently unstable; dynamic objects and environmental changes lead to inconsistent characterizations of the same location over time, providing an unreliable description. Furthermore, these descriptions offer only indirect cues for 6-DoF pose estimation, with irrelevant details acting as noise that can impede learning. This challenge is compounded by the sparse, geometric-only nature of LiDAR point clouds, which, unlike images, lack rich texture and color.

(2) Advantages of GTG for LiDAR localization. In contrast, our GTG produces text (e.g., "District 99, West-Southwest") that yields a standardized representation of universal pose information, creating a signal that is stable, repeatable, and directly relevant to pose estimation. The text's invariance to dynamic content provides the consistency needed to resolve localization ambiguities via regularization. Then, the LATER module refines GTG-generated text by fusing it with the current point cloud's geometry, creating viewpoint-specific embeddings that enrich the final training feature with diverse visual-geometric information beyond the core pose.

(3) Experimental validation and proof. In Table 4, we demonstrate that GTG is more effective than free-form scene descriptions. We also present this table here. We replace GTG with a SOTA 3D scene captioning model, TOD3Cap, to generate rich text. This free-form text mainly describes object motion, appearance, relationship, and environment. However, using free-form descriptions from TOD3Cap yields no significant performance gain over the vanilla model. Our template-based GTG leads to significant improvements.

Response to W2. The proposed GTG does not focus on specific environments. First, as discussed in the previous response, the GTG is not tied to any specific environmental features like roads or buildings. It partitions any given map into discrete districts and directions, which is a universal geospatial cue that can be applied to any environment that can be mapped. Second, the proposed GTG and the entire text modality are used exclusively during training and are completely absent during inference. We propose MRD distillation to conduct LiDAR-only localization at inference, without requiring text inputs. A localization model that does not use text at inference avoids overfitting to the templates of any specific environment.

Response to W3. Performance analysis. We would like to respectfully clarify that GTR-Loc achieves superior performance on all three datasets. Our method outperforms all baselines on QEOxford, achieves SOTA position accuracy on Oxford and NCLT, and ranks second in orientation estimation. Specifically, on QEOxford, GTR-Loc (0.75m/1.03°) achieves a significant 9.64%/8.04% improvement in both position and orientation over the previous SOTA, LightLoc (0.83m/1.12°). GTR-Loc consistently outperforms LightLoc on both the Oxford (2.59m/1.19° vs. 2.67m/1.25°) and NCLT (1.40m/2.62° vs. 1.46m/2.80°) datasets. The absolute improvement in position accuracy over previous leading SCR methods, LiSA and LightLoc, is significant.

The primary goal of GTR-Loc is to solve localization ambiguity, where a model incorrectly identifies its location due to geometrically similar scenes. The Oxford dataset, including QEOxford (a version with enhanced ground-truth), features long, repetitive urban roads with highly similar geometric structures, which creates significant ambiguities. Hence, the margin of improvement is most pronounced on QEOxford and still significant on Oxford. The NCLT dataset, gathered via a Segway, is less stable than data from vehicle platforms, making localization more challenging. Nevertheless, our superior performance against the SOTA method, LightLoc, validates the effectiveness of using geospatial text regularization.

Response to W4&Q1. (1) If inference with text. It is crucial to distinguish our final model—a distilled, LiDAR-only regressor (red flow shown in Figure 2) requiring only a single LiDAR scan for a fair comparison—requires no text at inference. To address the reviewer's concern and strengthen our results, we have conducted additional experiments using GTR-Loc with text during inference. Specifically, we first train an auxiliary classification network to predict district and direction categories from LiDAR point clouds. We set this network's architecture as a LightLoc encoder followed by four FC layers. During inference, the pre-trained classification network predicts the district and direction ID for text generation. This generated text, along with the original point cloud, is then fed into the LiDAR-text Regressor (blue flow shown in Figure 2) to perform prediction.

The results indicate that this approach is not effective. It delivers virtually no performance gains (the table above) while increasing both test time and computational complexity due to the additional classification network and multimodal regression. Inaccurate classification (the table below) leads to wrong text generation, thus resulting in inaccurate localization. Therefore, it further demonstrates that transferring knowledge via MRD into a LiDAR-only model is more efficient and effective. These additional experiments have also been reported in Tables 3 and 4 of the appendix.

(2) Additional information for inference. There is no additional text information used for GTR-Loc during inference. Our model's input is identical to that of previous SCR methods like LightLoc—only the LiDAR point cloud. The proposed MRD distillation allows LIDAR-only inference. Once training is complete, this model has no access to GTG, text, or pose information.

Thank you for your constructive feedback. We have addressed W4&Q1 by including additional experimental setups and results. We welcome any further discussion and deeply appreciate your guidance in improving this paper.

Using geospatial text for inference creates a circular dependency, as its construction requires the very pose information we aim to estimate. Therefore, we propose MRD to perform LiDAR-only localization during inference. Nevertheless, we evaluate its potential by experimenting with text generated from (1) predicted and (2) gt pose-relevant information.

(1) As explained for W4&Q1, we generate text by using a classification network to predict pose-relevant categories (district/direction). However, even with the GTG template, errors from this classification (row 5) lead to flawed text that hinders localization. Consequently, it yields no discernible performance gain (row 2, consistent with our W4&Q1 response).

(2) We then test a hypothetical scenario where gt poses are available for text generation at inference. This better result (row 3) establishes the theoretical upper bound for performance achievable with text. However, it is impractical, as gt poses are unavailable during real-world inference. Therefore, using MRD distillation is a more viable solution.