SafeMap: Robust HD Map Construction from Incomplete Observations

We sincerely thank the reviewer for your thoughtful and detailed feedback. We appreciate your recognition of our "design", "formulation", "experimental rigor", and "supplementary material". Below, we address each of your comments and suggestions in detail.

---

Q1: "Investigate which views are most critical for HD map construction."

A: Thank you for this insightful suggestion. We conducted a detailed analysis of individual camera importance by systematically removing each view and observing performance drops (mAP) across MapTR and HIMap on nuScenes, and MapTR on Argoverse2. The results reveal a clear hierarchy:

• The front-center view is most critical (e.g., MapTR: 50.3 → 31.3 mAP; HIMap: 65.5 → 38.0);
• Back views show moderate importance (e.g., MapTR: 28.5 mAP when back is missing);
• Front-side views have smaller but non-negligible impact;
• Side views are least critical (e.g., Argoverse2: 58.7 → 56.7 mAP).

These findings highlight the need for greater robustness to front/back view losses and suggest differentiated optimization strategies for peripheral cameras. We have integrated this analysis into the revised version.

---

Q2: "Additional experimental results for MapTRv2."

A: Thanks for your comment. We conducted experiments comparing SafeMap to MapTRv2 under various single-view-missing conditions on nuScenes. As shown below, SafeMap consistently improves robustness across all view types, demonstrating strong generalization and reliability in incomplete observation settings:

We have included these findings in the revised manuscript to further reinforce our effectiveness.

---

Q3: "Impact of temporal information (e.g., StreamMapNet, DTCLMapper) on view missing."

A: Thanks for your suggestion. We conducted experiments using StreamMapNet to evaluate the role of temporal cues in mitigating view-missing issues. Results on single-view-missing settings (nuScenes) are shown below:

While temporal fusion provides moderate gains, it does not fully address view-missing challenges. In contrast, SafeMap explicitly reconstructs missing views and corrects BEV features, leading to significantly stronger performance. We have included these experiments and analyses in the revised revision.

---

Q4: "Can SafeMap improve robustness under other corruption scenarios (e.g., MapBench)?"

A: Thanks for your question. Yes. We evaluated SafeMap using the MapBench benchmark [1], which includes 8 real-world camera corruptions covering environmental conditions (e.g., Fog, Snow), sensor artifacts (e.g., Motion Blur), and total sensor failures. As shown in Table 9 in the main paper, SafeMap achieves:

• +1.9% mRR / +9.4% mCE over MapTR;
• +6.2% mRR / +16.8% mCE over HIMap.

These results confirm our robustness beyond view-missing scenarios, handling diverse sensor degradations encountered in real-world deployments.

---

Q5: "What are the training and inference costs of SafeMap?"

A: Thanks for asking. SafeMap is efficient in both training and inference:

• Training: ~32.5 minutes/epoch on 8× RTX 3090 GPUs.
• Inference (Table 8): Compared to MapTR, SafeMap runs at 21.4 FPS vs. 21.5 FPS, with negligible differences in memory (2300 MB vs. 2298 MB) and model size (39.5M vs. 39.1M parameters).

These results demonstrate that SafeMap introduces minimal computational overhead while delivering substantial robustness and accuracy gains.

---

Q6: "Related Work — Comparison with POP-Net."

A: Thank you for highlighting this relevant work. We have included a discussion comparing SafeMap to POP-Net in the revised manuscript. Our key distinctions are:

1. Flexible Gaussian-based sampling vs. rigid panoramic encoders;
2. BEV feature distillation vs. token decomposition and aggregation;
3. Efficiency: SafeMap avoids the added latency and complexity of POP-Net's dual-encoder design.

These distinctions reflect our emphasis on modularity, scalability, and robustness. We appreciate your suggestion and have clearly positioned our contributions relative to POP-Net in the Related Work section.

---

Reference:

[1] Hao et al. Is Your HD Map Constructor Reliable under Sensor Corruptions? NeurIPS, 2024.

We sincerely thank the reviewer for the detailed and constructive feedback. We are encouraged that you found "the experimental results convincing" and "the supplementary material informative". Below, we address your insightful concerns point by point and will incorporate the clarifications into the revised version of the manuscript.

---

Q1: "How does the method compare to other sparse-view HD map reconstruction approaches?

A: Thank you for this important question. Most existing HD map reconstruction methods—including MapTR and HIMap—assume a full multi-view setup (6 views in nuScenes, 7 in Argoverse2), and do not explore sparse-view conditions. To evaluate SafeMap under such settings, we simulated 1–5 randomly missing views (out of 6) and compared against MapTR.

As presented in Table 7 of the main paper, SafeMap demonstrates three key advantages:

1. Only 2.3 mAP drop with 3 missing views, vs. 8.1 mAP drop for MapTR.
2. Maintains robust performance even with 4 missing views (66% sparsity).
3. Under extreme cases, i.e., 5 missing views, SafeMap outperforms MapTR by 50% in mAP.

These results confirm our generalizability to sparse inputs, supporting its practicality in real-world applications with limited or degraded sensor coverage.

---

Q2: "More experiment results on real-world camera sensor corruptions."

A: To comprehensively evaluate real-world robustness, we adopt MapBench [1], which includes eight realistic corruptions:

1. Environmental conditions: Brightness, Low-Light, Fog, and Snow;
2. Sensor artifacts: Motion Blur and Color Quantization;
3. Sensor failures: Camera Crash and Frame Loss.

As shown in Table 9 in the main paper, SafeMap outperforms MapTR by +1.9% mRR / +9.4% mCE and HIMap by +6.2% mRR / +16.8% mCE, confirming that our approach is robust not only to missing views but also to partial occlusion, adverse lighting, and noisy signals.

---

Q3: "How does SafeMap compare to data augmentation methods that simulate missing/corrupted views?"

A: Thanks for your question. We conducted controlled experiments with two baselines:

• Random Mask Patch: randomly masks image patches;
• Random Mask View: randomly drops one full view.

As shown below, SafeMap outperforms both:

While masking-based augmentation improves robustness, it lacks explicit mechanisms to reconstruct missing features. In contrast, SafeMap:

• Explicitly learns spatial relationships through the Gaussian-based PVR module;
• Leverages complete panoramic BEV priors via the D-BEVC module.

This targeted feature recovery provides significantly more effective supervision, especially under severe view dropouts. As suggested, we have added this study in the revised version.

---

Q4: "Why define Gaussian in image space (pixels) instead of 3D space using camera parameters?"

A: Thank you for raising this point. While it is true that pixel space is not linearly aligned with 3D space, we adopt a Gaussian in image space for two practical reasons:

• Efficiency: Deformable attention operates in the image plane; defining Gaussian sampling here avoids computational overhead and enables seamless integration with pixel-based encoders.
• Effectiveness: Our Gaussian prior focuses on adjacent views, which in multi-camera setups are spatially proximate. Despite 3D nonlinearity, the horizontal image plane is a strong proxy for angular continuity.

Moreover, our sensitivity analysis (Appendix C) empirically confirms that Gaussian sampling in image space improves reconstruction accuracy. We agree that incorporating 3D-aware priors is an exciting direction and plan to explore this in future work.

---

Q5: "Why use a separate module to predict missing views rather than directly predicting from available ones?"

A: Great question. SafeMap separates the process into two stages:

• G-PVR reconstructs missing views, guided by spatial priors;
• D-BEVC corrects BEV features using distilled panoramic features.

This two-stage design offers a clear advantage over end-to-end prediction from incomplete inputs. As shown in Table 7 of the main paper:

• With 3 missing views: SafeMap achieves 22.8 mAP vs. 19.5 mAP with direct prediction;
• With 5 missing views: SafeMap achieves 6.0 mAP vs. 4.0 mAP (a 50% relative improvement).

These gains highlight that explicit view completion and correction better preserve geometric consistency, especially under extreme sparsity. As suggested, we have made this discussion more explicit in the revised version.

---

Reference:

[1] Hao et al. Is Your HD Map Constructor Reliable under Sensor Corruptions? NeurIPS, 2024.

We sincerely thank the reviewer for the positive evaluation and thoughtful comments. We appreciate your recognition that our work "provides new insights into fault-tolerant HD map reconstruction for autonomous driving". Below, we address your specific concerns in detail and will incorporate corresponding improvements into the revised manuscript.

---

Q1: "More experiments on random missing views and real-world camera sensor corruptions are needed."

A: Thanks for your suggestion. We have conducted extensive robustness evaluations addressing both random view drops and real-world sensor corruptions:

• Random Missing Views:

  • As detailed in Table 7 of the main paper, we simulated 1 to 5 randomly missing camera views using the 6-camera setup in nuScenes. Each missing-view setting involves multiple combinations (e.g., 6 for 1 missing view, 15 for 2 missing views, 20 for 3, 15 for 4, and 6 for 5), and SafeMap is trained to generalize across all these cases, maintaining high performance with minimal degradation—credited to its ability to infer missing cues via G-PVR and D-BEVC.

• Real-World Sensor Corruptions:

  • We adopt MapBench [1], which includes eight realistic corruptions across:
    1. Environmental conditions: Brightness, Low-Light, Fog, and Snow;
    2. Sensor artifacts: Motion Blur and Color Quantization;
    3. Sensor failures: Camera Crash and Frame Loss.
  • As shown in Table 9 of the main paper, SafeMap outperforms MapTR by +1.9% mRR / +9.4% mCE and HIMap by +6.2% mRR / +16.8% mCE, demonstrating robustness beyond simple view dropouts and toward real-world deployment scenarios.

---

Q2: "In Eq (2), why do pixel px and pixel py follow different noise distributions?"

A: Thank you for this insightful question. Our design choice is based on empirical observation and domain priors in autonomous driving:

• The horizontal coordinate px follows a Gaussian distribution centered on the middle of the panoramic perspective view. This reflects the fact that adjacent views to the missing one contain the most informative features for reconstruction. The Gaussian distribution allows us to focus attention on these nearby views while still softly exploring farther regions.
• The vertical coordinate py follows a uniform distribution. This is because vertical information (e.g., lane boundaries, poles, road textures) is uniformly important across different heights in the image, especially in BEV-centric tasks like HD map construction.

This hybrid sampling strategy balances spatial focus and coverage—allowing G-PVR to direct more attention toward informative horizontal regions while still preserving vertical completeness. Appendix C provides a sensitivity analysis showing that this setup offers superior performance over uniform sampling in both dimensions.

---

Q3: "The backbone builds on MapTR. What is the motivation of the architecture?"

A: We appreciate this concern. While SafeMap builds upon MapTR for its base encoder-decoder structure, our contributions lie in robust map construction under incomplete observations, which MapTR and existing works are not designed to address. Specifically, our architectural novelty includes:

• The G-PVR module, which performs dynamic reference point sampling using Gaussian priors and deformable attention to reconstruct missing views;
• The D-BEVC module, which distills global spatial priors from complete panoramic views and corrects BEV representations derived from incomplete inputs.

These modules are model-agnostic and can be plugged into other backbones (e.g., StreamMap), offering a general solution for improving robustness. As shown in the experiments below, SafeMap consistently outperforms both MapTR and StreamMap variants when camera views are missing:

• Table. Performance on individual missing views (nuScenes, MapTR backbone): |View Missing|Front|Front-L|Front-R|Back| Back-L|Back-R | :-|:-:|:-:|:-:|:-:|:-:|:-:| |MapTRv2|40.3|53.7|54.2|41.2|56.3|57.4|50.5| |SafeMap|53.6|59.7|59.1|52.3|60.1|60.8 | $\Delta$ |+12.3|+6.0|+4.9|+11.1|+3.8|+3.4

• Table. Extension to StreamMap (temporal BEV backbone): |View Missing | Front | Front-L |Front-R|Back| Back-L | Back-R| | :-|:-:|:-:|:-:|:-:|:-:|:-:| |StreamMap (w/o temp)|13.2|17.4|18.0|14.1|16.3|17.1 |StreamMap (w/ temp)|14.3|18.2|19.3|15.3|17.5|18.5 |SafeMap (w/ temp)|18.7|21.5|21.2|19.3|19.8|22.3

These improvements verify that SafeMap is not merely an application of MapTR, but rather a general framework that enhances robustness and reliability under real-world constraints, a critical capability not addressed by prior methods.

---

Reference:

[1] Hao et al. Is Your HD Map Constructor Reliable under Sensor Corruptions? NeurIPS, 2024.

We sincerely thank the reviewer for the thoughtful and constructive feedback. We greatly appreciate your recognition of our "technical novelty", "robustness in incomplete scenarios", "experimental design", and "plug-and-play applicability". Below, we address your main questions and concerns in detail.

---

Q1: "Extending SafeMap to incorporate multi-sensor fusion."

A: Thank you for the insightful suggestion. To explore the effectiveness of SafeMap in multi-sensor fusion settings, we extended our framework to MapTR-Fusion, a recent camera-LiDAR fusion model, and evaluated it under 13 real-world multi-sensor corruptions introduced in MapBench [1].

The results, presented in the following table, indicate that our SafeMap method not only enhances performance in scenarios involving camera view loss but also significantly boosts the efficacy of multi-sensor fusion methods.

These results further strengthen the practical value of our approach in real-world autonomous driving systems. As suggested, we have added this discussion and table to the revised version.

---

Q2: "Could the authors discuss real-world deployment scenarios?"

A: We appreciate your comment on real-world applicability. While large-scale real-world deployments are resource-intensive, we adopted MapBench [1] to systematically simulate real-world conditions. This benchmark includes eight realistic camera sensor corruptions, categorized into:

1. Exterior environmental conditions (Brightness, Low-Light, Fog, and Snow).
2. Interior sensor artifacts (Motion Blur and Color Quantization).
3. Complete sensor failures (Camera Crash and Frame Loss).

As presented in Table 9 of the main paper, SafeMap consistently outperforms baselines under these settings:

• +1.9% in mRR and +9.4% in mCE over MapTR.
• +6.2% in mRR and +16.8% in mCE over HIMap.

These results clearly demonstrate our robustness under practical corruptions and affirm its suitability for real-world deployment. As suggested, we have included a more explicit discussion in the revised manuscript.

---

Q3: "More detailed explanations of Gaussian-based sparse sampling."

A: Thanks for your comments. Our G-PVR module employs Gaussian-based sparse sampling to guide reference point selection in the reconstruction of missing camera views. The intuition is to prioritize sampling locations near adjacent views, which typically contain the most informative cues for reconstructing the missing view.

Concretely, the horizontal coordinate of reference points is sampled as:

$p_x ∼ 𝓝(μ, σ²),$

and the vertical coordinate is sampled as:

$p_y ∼ 𝕌(0, H),$

where:

• $μ$ is set to the center of the panoramic perspective view $(μ = (N_a × W) / 2)$.
• $σ²$ is a hyperparameter controlling the horizontal spread (tuned via ablation in Appendix C).
• $H$ is the vertical resolution of the input image.

These coordinates define a 2D sampling grid, where reference points are more densely sampled around the central horizontal region (typically where adjacent views reside), and uniformly along the vertical axis.

This sampling pattern is utilized in the Deformable Attention module [2]. This mechanism allows the model to dynamically focus on the most informative regions, concentrating attention near adjacent views while still capturing a broader context. As shown in Table 5 and Figure 4 in the main paper, this targeted sampling strategy leads to improved reconstruction quality compared to uniform or local sampling strategies.

---

Reference:

[1] Hao et al. Is Your HD Map Constructor Reliable under Sensor Corruptions? NeurIPS, 2024.

[2] Zhu et al. Deformable DETR: Deformable Transformers for End-to-End Object Detection, ICLR 2021.