Q1 and Q2: Lack of novelty in the method and framework.

Thanks for your question. The main contribution of this work relies on two aspects.

First, we attempt to build an ability that can be located immediately upon entering an unknown area for the first time, without GPS, without the pre-built map. To achieve this, our work has two settings that are significantly more challenging compared to traditional place recognition works: (1) High-resolution remote sensing imagery, which is easy to acquire, is applied for the first time as reference maps for retrieval in conjunction with lidar point cloud queries. (2) We have expanded the retrieval range to $100 km^2$, rather than searching along trajectories. For this cross-view, cross-domain retrieval problem, under such a strict setting, most previous works fail to provide satisfied performance, and we achieve a retrieval rate higher than $80\%$ for the first time.

Second, at the technical level, our proposed retrieval framework based on semantic domains is a principled and effective solution, not a naive combination of existing modules. We conducted comprehensive ablation and comparative experiments on each component of the framework, demonstrating that our method provides a more robust and innovative integration (Table 6 - Table9). The core contribution lies in the proposed Spatial-Temporal Particle Estimation (STPE) module, which fully leverages both spatial and temporal continuity in sequential queries. This leads to a substantial improvement in retrieval accuracy, boosting Recall@1 from $20.07\%$ to $83.27\%$.

Based on this, we believe that the exploration and technique contribution in this work will bring beneficial inspiration to the community.

Table 6: Performance comparison on XA-L&RSI and Oxford-L&RSI. (Table 3 in the main paper)

Recall@1 and Recall@10 (%)↑ are reported. The best result is rendered in bold.

Table 7 : Ablation study about effectiveness of components. (Table 4 in the main paper)
Results include Recall@1 and Recall@30 (%)↑ on Test Set-S and 11-14-02-26 are reported. Sem.: Semantic segmentation. STPE: Spatial-Temporal Particle Estimation. Orient.: Orientation.
The best results are in bold.

Table 8: Ablation study about the selection of the semantic encoder. (Table 1 in Appendix)
Recall@1 and Recall@10 (%)↑ on XA-L&RSI and Oxford-L&RSI are reported.
† indicates that the input image size is 518×518. The best result is rendered in bold.

Table 9: Ablation study about the compatibility of STPE with other existing networks. (Table 2 in Appendix)
Recall@1 and Recall@10 (%)↑ on XA-L&RSI and Oxford-L&RSI are reported. The best result is rendered in bold.

Q3: The impact of the motion model's failure.

As noted in Lines 205–209 of the main paper, the motion model for each time step} is directly obtained using FastGICP combined with magnetometer-based heading correction.

We acknowledge that dynamic objects (e.g., vehicles, pedestrians) can indeed introduce noise to motion estimation. To mitigate this, we deliberately avoided prolonged proximity to large moving vehicles during data collection. Additionally, we filtered out point clouds within a 3-meter radius of the data collection vehicle to reduce self-interference.

Empirically, the submaps constructed via FastGICP show high quality in visualizations, and the particle aggregation mechanism in STPE over multiple time steps helps to neutralize occasional noisy estimates. This robustness is also reflected in the strong performance of place recognition observed in our experimental results (Table 6 and Table 7).

Q4: The XA-L&RSI dataset open source.

Thanks for acknowledging the value of our dataset. We fully agree with your view on the importance of open data, especially given the scarcity of datasets in this research area — a challenge we are also deeply aware of. Before submitting the paper, we only acquired a usage license (rather than an open license) for the high-resolution ($0.5m$) satellite imagery of Xiang'an District, Xiamen, which prevented us from releasing the dataset in the short term. This was clearly stated in the checklist.

However, we are pleased to share that the licensing issues have recently been resolved. We now hold the appropriate rights to release the imagery, and we plan to open-source the XA-L&RSI dataset upon acceptance of the paper.

Dear Reviewer 4DTw,

Thanks for your effort in reviewing our work. We have addressed your concerns in the rebuttal and summarize our response as follows:

1. Lack of novelty in the method and framework (W1, W2): We have clarified that the primary contribution of our work lies in both the problem setting and the proposed framework. we are the first to address the challenge of large-scale (over 100 km²) urban cross-view LiDAR-based place recognition with high-resolution remote sensing imagery. The integration of Spatial-Temporal Particle Estimation (STPE) provides a principled and innovative solution by exploiting both spatial and temporal continuity in sequential queries, which significantly improves retrieval accuracy (Recall@1 increased from 20.07% to 83.27%).

2. The impact of the motion model's failure (W3 ): We explained the motion model estimation method, discussed how we mitigate the interference of dynamic objects on the FastGICP algorithm in practice, and the experimental results in the main paper also demonstrate the robustness of the STPE algorithm to random noise in the motion model.

3. The XA-L&RSI dataset open source (W4 ): We have clarified the reason for filling in "Data cannot be open sourced in the short term." in the checklist, and we also confirmed that the datasets used in our work will be made publicly available to the research community upon acceptance of the paper.

Additional Experiment on Motion Model's Failure(W3):

We have just completed an additional experiment to address your concerns about the impact of the motion model's failure. We introduce different levels of yaw angle noise $\epsilon_{yaw}(^\circ)$ and translation noise $\epsilon_{xy}(m)$ into the estimated motion model. The translation noise is defined by the noise in the x-direction and y-direction as follows: $\epsilon_{xy}=\sqrt(\epsilon_x^2+\epsilon_y^2)$

In this experiment, we take $\epsilon_x=\epsilon_y$ for convenience. The average Recall@1(%) of three repeated experiments on Test Set-G ($100km^2$) of XA-L&RSI are shown in the following table:

Please note that the disturbances in the table represent intentionally introduced additional noise, based on the noise inherently caused by the FastGICP algorithm and the magnetometer used in the system.

$\epsilon_{yaw}=\pm5^\circ$ and $\epsilon_{xy}=\pm1$m indicates that, in addition to the inherent noise, random yaw angle noise within a range of $\pm5^\circ$ and random translation noise within a range of $\pm1m$ are introduced.

The parentheses report the performance drop compared to the additional noise-free setting.

The results demonstrate that our method can tolerate a certain level of additional noise in the estimated motion model ($\epsilon_{yaw}<\pm10^\circ$ and $\epsilon_{xy}<\pm1$m), which is typically introduced by dynamic objects affecting the FastGICP algorithm. In practice, we found that excluding the points of the data collection vehicle itself from the raw point cloud effectively reduces this noise to an acceptable level.

We are eager to engage in further discussion if you have any additional concerns.

Best Regards,

Authors

Q1: The related work is incomplete.

Thanks for raising this concern. Firstly, among the nine references mentioned as missing, four are already cited in the introduction or related work sections, including (LC)$^2$ [4], VXP [6], Tang et al. [9], and the journal version of Tang et al. [1].

Secondly, the motivation of this work is to develop a capability for immediate localization upon first entry into an unknown environment, without relying on GPS or a pre-built 3D map. Therefore, we choose globally available and easily accessible remote sensing imagery as the reference. In terms of related work, we focus on cross-view and cross-modal retrieval. The remaining five papers in the list are as follows:

As shown in Table 3, since [2], [3], [5], and [7] all rely on a pre-built 3D map of the target area constructed from ground-level or low-altitude observations, they are not directly related to our work.

[8] is more accurately described as a geo-tracking method. It requires an initial absolute pose and uses multiple ground cameras and a LiDAR sensor to match against a small ($100\times100m^2$) satellite image. This setup is completely unrelated to our problem.

Table 3: The task settings of the mentioned Related Works.

In summary, since these works are not strongly related to our approach, we did not cover the entire set of literature from those fields. Nevertheless, we will include [2], [3], [5], and [7] in the Related Work of the camera-ready version.

Q2: Lack of comparison with the methods of Lidar to aerial images.

Table 4: Supplementary performance comparison.

$Recall@1$ and $Recall@10$ (%)↑ are reported.

Thanks for the suggestion. However, the recommended baselines [2], [3], and [4] address the place recognition between LiDAR and street-view camera rather than LiDAR and aerial image, and are not directly related to our work.

Among them, [3] and [4], the proposed methods are tightly coupled with monocular depth estimation from street-view imagery. However, high-quality depth estimation is not feasible for satellite imagery due to the lack of reliable geometric cues. As a result, these methods cannot be directly applied to our setting, and a fair comparison is not possible.

[2] projects LiDAR into a range image and then performs image-to-image retrieval against street-view imagery. Given the substantial viewpoint discrepancy between range images and satellite imagery, along with the constraint in [2] that restricts retrieval to the vehicle's trajectory, it is expected that directly transferring this method to large-scale, city-level retrieval would result in poor performance. In Table 4, we retrained and compared both the official implementation of [2] and a variant where the range image is replaced with a magnetometer-corrected semantic BEV image in XA-L&RSI, to provide a more fair and meaningful comparison. They both significantly underperform our method across all benchmarks.

It is important to highlight that, to the best of our knowledge, our work is the first to tackle LiDAR-to-Remote-Sensing-Imagery place recognition in a city-scale area ($100km^2$). In the list of Q1, [1] and [9] are the most relevant. Unfortunately, neither of them is open-sourced, and [9] does not even provide a network architecture diagram, which significantly increases the difficulty of reproduction.

Moreover, both [1] and [9] are designed for trajectory-constrained retrieval, meaning they assume localization occurs only along pre-defined driving routes. As a result, their performance degrades quickly as trajectory length increases, which limits their applicability in large-scale, unconstrained scenarios.

During the rebuttal period, we made our best effort to re-implement the approach proposed in [1], and the results are reported in Table 4. The performance remains substantially below that of our method.

Q3: Mathematical derivation of STPE.

Thank you for carefully reviewing our mathematical derivation.

• Yes, the state $**x**_j^{[k]}$ of a particle refers to the 2D coordinate $(x, y)$ of a candidate submap in the remote sensing database retrieved by the query $Q_j$.

• By "aggregating the particles", we refer to propagating particles from multiple past time steps (t−L to t−1) to time step t using the motion model, and then constructing a probability density function at time step t based on the combined particle distribution.

• We agree that Eq. 4 and Eq. 5 may cause confusion. Our original intention was to express, in probabilistic terms, how the particles from time step t−1 transition to time step t. However, as described from L205 to L209 of the main paper, the motion of the particles are directly obtained using FastGICP and magnetometer-based heading correction. Therefore, we have removed Eq. 4 and Eq. 5 to avoid ambiguity. This change does not affect the overall flow of the derivation.

• We assume the x and y dimensions are independent, and thus the diagonal covariance matrices are used in Eq.6 and subsequent Gaussian mixture models :

• The noise term v in Eq. 3 originates from the motion estimation process based on FastGICP combined with magnetometer-based heading correction. This procedure inherently produces noisy estimates of the motion model. When aggregating many particles over multiple time steps, the effect of such noise tends to average out. Therefore, we model the distribution at time t empirically as a mixture of multiple gaussian models without explicitly including a separate noise term in Eq. 10. Empirically, this approximation works well in our experiments. (Table 3 and Table 4 in the main paper, please see Table 6 and Table 7 included in the response to Reviewer 4DTw for convenient reference.)

Q4: Data enhancement.

Yes, we applied data augmentation. Specifically, we performed random rotations along the z-axis within $\pm 15^\circ$ to simulate heading noise and improve robustness to magnetic disturbances.

Q5: Limitations.

Thank you for the comment. As noted in the Limitations (Appendix A.1), our method relies on magnetometer-based heading priors and is designed for urban environments. Performance may degrade in semantically sparse rural areas. We acknowledge these limitations and plan to address them in future work.

Q6: Threshold of $30m$ in evaluation.

Table 5: Ablation study about the distance threshold of Retrieval θ.

$Recall@1$ and $Recall@10$ (%)↑ are reported.

We appreciate the reviewer’s suggestion. The choice of a distance threshold $\theta=30m$ is a deliberate trade-off. During training, a larger threshold reduces the spatial overlap between LiDAR and positive remote sensing images, which weakens the effectiveness of contrastive learning. A smaller threshold, however, requires denser remote sensing submap sampling to ensure every LiDAR query has valid positives.

During inference, increasing the threshold simplifies the retrieval task and leads to higher recall, but with weaker spatial alignment. Our $30m$ threshold ensures at least $50\%$ overlap between LiDAR BEV images and positive remote sensing images to reflect a more realistic and challenging setting.

For reference, we provide results under a threshold $\theta=40m$ in Table 5, which show higher recall due to relaxed retrival criteria.

Q7: L192: top-C should be top-K?

Thank you for the question. We intentionally use "top-C" to refer to the number of retrieval candidates produced by global place recognition for a single current query ($C=200$ in our setting). In contrast, "K" denotes the number of top retrieval results selected from each historical query in the sequence during the STPE refinement stage ($K=30$ in our setting). If $L$ denotes the number of historical queries in the sequence, the STPE module operates on $K\times L$ particles in total and re-ranks the top-C retrieval candidates of the current query.

Thank you for your thoughtful and detailed feedback on our rebuttal.

Main Comments:

the link between the paper and related work (this paper is not the first to address lidar to aerial image place recognition but the previous methods did not release their code)

Thank you for recognizing our efforts during the rebuttal period. We do not claim ours is the first one to address lidar to aerial image place recognition. But we want to emphasize that we are the first to address the challenge of large-scale (over 100 km²) urban cross-view LiDAR-based place recognition with high-resolution remote sensing imagery. The previous methods [1] and [9] differ from ours in a fundamental way: They limit the retrieval range to a predefined trajectory instead of an open geographic area, assuming a known driving route, making these methods impractical for real-world use.

the derivation of the particle filter....

We would like to clarify the difference between the STPE algorithm and the traditional particle filter. The particle filter relies on an initial particle set that decays over time steps, requiring reinitialization once the particles disappear.

In contrast, the STPE algorithm operates differently. In our approach, particles are maintained in a queue. For each current query, only the retrieval results of the previous $L$ queries, including the current query itself, are used as particles for propagation. This mechanism ensures the smooth transmission of spatio-temporal information, without the need for reinitialization.

Q1: How is the retrieval on the first query run i.e. the initialization step? at t=0, the range [t-1, t-L] is not valid

A1: Thanks for your insightful question. The STPE algorithm is not applicable at $t=0$. In fact, the results reported in the paper all begin from $t=L-1$, which does not impact fairness. In practical applications, the target area is an unknown region that has never been visited before, and our ultimate goal is to achieve accurate localization right from the start. However, currently, a small distance (around 250m) must be traveled first, during which only single-query retrieval results can be provided.

Q2: Why are the retrieval candidates for $Q_j^t$ defined as the aerial images with the most similar network embedding vector not included in the derivation?

A2: Thanks for your careful review. In the main paper, we stated that the time steps in the STPE algorithm sequence range from $t-L$ to $t-1$. However, the correct range should be from $t-L+1$ to $t$. The total number of queries in the sequence remains $L$. We sincerely apologize for the misunderstanding caused by the typographical error. We will correct the derivation and description in Section 4.3 in the camera-ready version, including a revision of Eq. 11 as follows:

$P_t(x, y) = \frac{1}{L} \sum_{j=t-L+1}^t \tilde{P}_j(x, y)$

Q3: all the database images of the past past L queries $Q_j^{t2}$ with $t2 \in t-1, t-L$ define the particles used in the particle estimation. How are they initialized? For example, are the particle used at time t initialized with their last value during the estimation at t-1? Or are they initialized with the position associated to the retrived aerial image?

A3: Thank you for your insightful question. The former is mostly correct: the particles used at time $t$ are initialized with their values from the estimation at time $t-1$. However, since our particle set is maintained as a queue, the particles at time $t$ are updated by adding the retrieval candidates of current query itself, while removing the particles from time $t-L$.

Again, thank you for taking the time and energy to help us improve the paper.

Q1 and Q5: Sec. 4.3 is difficult to understand.

Thank you for your suggestion. We acknowledge that the current presentation of Section 4.3 can be improved in terms of clarity and the use of mathematical notations.

Section 4.3 introduces the STPE algorithm, which is designed to alleviate the unreliability of individual query retrieval results by leveraging information from a sequence of past queries.

We utilize a sequence of past queries. For each query, we treat its top-K retrieval results as particles, where the state of each particle is defined by its x and y coordinates. Eq. 3 illustrates that, if the motion model is known, each particle can be propagated forward in time, with added random noise reflecting uncertainty. In practice, the motion model at each time step is directly obtained using FastGICP combined with magnetometer-based heading correction.

The original intention of Eq. 4 and Eq. 5 was to express the probabilistic distribution of a particle's state after motion in a single time step. However, these equations may hinder understanding rather than clarify it, so we propose to remove them.

Eq. 6 models the initial distribution of particles at a single past time step, which we assume follows a Gaussian Mixture Model (GMM). Since the x and y dimensions are independent, we use a diagonal covariance matrix (this step was omitted for brevity).

Eq. 7, Eq. 8, and Eq. 9 describe how the key parameters of the GMM are estimated. Eq. 10 models the distribution of particles propagated from a single past time step to the current time step. Finally, Eq. 11 aggregates the distributions from multiple past time steps to form the probability density function at the current position. Eq. 12 then computes the probability score for each candidate remote sensing submap, which is used to re-rank the retrieval results for the current query.

In the camera-ready version, we will carefully revise this section to provide a clearer explanation of the particle aggregation process. We will also include intermediate steps and intuitive descriptions to better guide readers through the derivation, ensuring that this part is more accessible and readable in the final version.

Q2 and Q6: The datasets only covers two cities.

Table 1 : Zero-shot performance comparison on Kitti-360 Seq. 00 and Oxford-L&RSI .
Recall@1 and Recall@10 (%)↑ are reported.

We appreciate the concern of the reviewer. We are also willing to perform broader evaluations. However, existing mainstream localization datasets do not provide corresponding high-resolution remote sensing images. Due to time constraints during the rebuttal period, we selected Sequence 00 from the KITTI-360 and manually constructed a new zero-shot test set by equipping it with corresponding satellite imagery. The data processing is consistent with that of XA-L&RSI and Oxford-L&RSI. The new test set contains $1532$ Lidar submaps, $2747$ remote sensing submaps, and remote sensing images covering an area of $2.28km^2$. As shown in Table 1, our method still achieves the best performance on this new benchmark. We will provide additional results trained and evaluated on KITTI-360 in the camera-ready version.

Q2 and Q6: Dependence segmentation models

Table 2: Ablation study about different semantic estimators.

Recall@1 and Recall@10 (%)↑ are reported.

Thanks for your question. To evaluate the robustness of our method, we replaced SphereFormer with HSSC [1], and observed only a minor drop in performance. This indicates that our framework is not sensitive to the specific choice of segmentation model, as shown in Table 2.

[1] Chen, S., Yang, B., Xia, Y., et al. Bridging LiDAR gaps: A multi-LiDARs domain adaptation dataset for 3D semantic segmentation. In Proc. of IJCAI, 2024, pp. 650–658.

Q3: Missing metadata information of satellite images

We thank the reviewer for pointing this out. For XA-L&RSI, remote sensing image data was captured in 2022 and 2023, covering the Xiang'an District of Xiamen, China (approx. $24^\circ32^\prime N$–$24^\circ41^\prime N$, $118^\circ10^\prime E$–$118^\circ20^\prime E$). For Oxford-L&RSI, remote sensing image data was captured in 2019, covering the Xiang'an District of Xiamen, China (approx. $51^\circ44^\prime N$–$51^\circ45^\prime N$, $1^\circ14^\prime W$–$1^\circ16^\prime W$). All the remote sensing image data was collected by the Beijing-3 satellite, using sensors BJ3N1, BJ3N, BJ3N2, and BJ3A1.

Q4: Format issues noticed.

Thank you for your careful review. We will thoroughly check the entire paper for formatting and spelling issues, and make the necessary corrections in the camera-ready version.

Thank you for your kind decision to raise your score and for the encouraging feedback. We deeply appreciate your recognition of the progress we've made.

Your suggestions regarding enhancing the clarity and presentation of our paper are truly valuable, and we agree that implementing them will improve both the paper's quality and its readability.

As we incorporate these improvements, please let us know if any other aspects concerning clarity or other points require further attention.​​ We remain open to further discussions to ensure that the highest standards are achieved.

Thank you once again for your valuable time and feedback.

Q1: How to reduce STPE overhead for real-time use?

Thank you for the insightful question. In practice, since our L2RSI framework incorporates the efficient FastGICP algorithm, the STPE module does not need to be activated for every query. Instead, it can be triggered periodically for high-precision place recognition. Between two STPE activations, relative poses can be propagated using FastGICP. This trigger-based design significantly reduces the computational overhead introduced by STPE in real-time applications.

Q2 and Q4: Dependence on magnetometer data.

Table1 : Ablation study about effectiveness of components. (Table 4 in the main paper)
Results include Recall@1 and Recall@30 (%)↑ on Test Set-S and 11-14-02-26.
Sem.: Semantic segmentation. STPE: Spatial-Temporal Particle Estimation. Orient.: Orientation.
The best results are in bold.

As noted in our Limitations section (A.1 in the Appendix), the proposed framework relies on magnetometer data to provide a coarse orientation prior. This is crucial because, at the city scale (covering an area of $100 km^2$), LiDAR-to-remote-sensing-image retrieval is highly challenging. Even a rough orientation estimate significantly facilitates the learning of cross-view and cross-modal global descriptors. As shown in Table 1 (third row), removing this directional input significantly degrades the performance of place recognition. In future work, we plan to address this limitation by exploring orientation-invariant or self-supervised alternatives.

Q3: Datasets open source.

Thank you for your interest in our dataset. We have already obtained the necessary open-source licenses for remote sensing imagery. The XA-L&RSI and Oxford-L&RSI datasets will be released to the research community upon the official publication of the paper.

Q5: End-to-end latency during real-time robotic operation.

Thanks for your valuable question. The total inference latency of the proposed L2RSI pipeline is approximately $290ms$, which includes around $200ms$ for point cloud semantic segmentation, $50ms$ for generating the semantic BEV image, and $40ms$ for retrieval and STPE refinement per query. It is worth noting that our current implementation focuses on offline evaluation to validate the core contributions, and has not yet been optimized for deployment. In practical applications, the system can adopt a trigger-based design, where high-accuracy place recognition is performed periodically, which is sufficient to meet the requirements of typical urban autonomous localization tasks.

We appreciate the reviewer’s thoughtful comments regarding runtime efficiency and its implications for real-world deployment.

As mentioned, our current reported latency of ~290 ms per query was obtained on an RTX 3090 GPU using an unoptimized Python implementation. This version includes substantial debug functionality, logging, and data dumping routines, which are not representative of a production system. Therefore, the reported time reflects a conservative upper bound.

In practice, we acknowledge that achieving real-time performance is critical for on-vehicle deployment. To this end, we foresee multiple engineering directions to significantly reduce latency, including:

1. Codebase Refactoring and Compilation: Our current implementation is entirely in Python. Migrating core modules to C++ or leveraging PyTorch’s JIT compilation can substantially reduce overhead.
2. Triggered Execution Mode: In practical systems, Fast-GICP runs in real-time to provide relative poses between consecutive point cloud scans. The proposed L2RSI module only needs to operate in a triggered manner, intermittently providing accurate global localization results to align the relative poses from Fast-GICP to a global coordinate frame. As shown in Table 5 of the main paper, the STPE algorithm can function effectively even when the sequence sampling rate is reduced to 30%. Therefore, under the triggered execution mode, the point cloud semantic segmentation, BEV image generation, and retrieval components of L2RSI operate at a low frequency. The STPE algorithm is only executed upon trigger events.

Table 5: Ablation study about the sampling rate $\lambda$.
Recall@1 (%) on Test Set-S is reported. The runtime (ms) here only involves STPE. The best result is in bold.

3. We fully agree with you that reaching practical deployment speeds—especially for embedded platforms—will require substantial engineering effort. Our long-term goal is to achieve a 10 Hz localization rate. These optimizations will be continuously incorporated and tracked via our GitHub repository.

Thank you once again for your valuable feedback and thoughtful comments.

Thank you for the time and effort you have dedicated to reviewing our work. We truly appreciate your valuable feedback and thoughtful suggestions. Should you have any concerns or points that require further attention, we would be happy to continue the discussion and address them to ensure the highest standards are achieved.