We appreciate the reviewer’s acknowledgment of the importance of uncertainty estimation in safety-critical applications like autonomous driving and their recognition of the interesting motivation behind our work. Below, we provide a detailed response to their feedback, addressing each concern while further clarifying our contributions and methodology.

• The paper is a bit hard to follow. Most of the content describes different settings for different subtasks. However, the key contribution is hard to find, rather than applying uncertainty estimation to 3D object detection.

We also recognize that the readability could be improved by better highlighting the key contribution and streamlining the presentation of tasks, which we will ensure in the final revised version of the paper.

The key contribution lies in our novel uncertainty estimation framework, specifically designed for 3D object detection, which integrates evidential deep learning (EDL) with Bird’s Eye View (BEV) representations. This versatile framework enables computationally efficient (real-time) uncertainty quantification at multiple levels—scene, bounding box, and missed objects—while uniquely accounting for both classification and localization uncertainties simultaneously, a capability not addressed by other methods.

The relabeling pipeline and the uncertainty use cases, such as OOD detection, bounding box quality assessment, and missed object identification, serve to demonstrate the effectiveness and practical applicability of our approach. These examples highlight how our method can improve the robustness and reliability of 3D object detection systems in real-world scenarios.

• Experimentally, conducting experiments simultaneously on camera-only, LiDAR-only, and fusion-based methods seems unnecessary. The input modality is unlikely to affect the main conclusion of the paper.

The inclusion of experiments on LiDAR-only and fusion-based methods serves to validate the generality of our uncertainty estimation framework across a variety of sensor setups commonly used in 3D object detection. While the input modality does not directly influence the core contribution of our method, using diverse scenarios ensures that the observed quality improvements are consistent across different setups. This consistency demonstrates that our improvements are not due to random variations from retraining the model but reflect a systematic and reliable enhancement, further reinforcing the robustness of our approach for real-world applications.

• Including the Waymo dataset is only for constructing out-distribution data, which seems unnecessary.

We used the Waymo dataset as out-of-distribution (OOD) data because a similar setup has been previously employed in studies such as [1] which highlights the practical importance of evaluating performance across different data distributions. That paper observed a significant drop in performance when models trained on one dataset were tested on a different one, such as NuScenes and Waymo, due to the OOD nature of the data.

The nuScenes versus Waymo setup is not only well-established [1] but also aligns with standard protocols in domain adaptation [2, 3] and OOD generalization frameworks [4] in 3D object detection. Including Waymo in our evaluation follows this standard protocol, allowing us to demonstrate how our uncertainty estimation framework effectively identifies and handles OOD scenarios. This is a critical capability for real-world autonomous systems, where encountering OOD scenarios is common. We will revise the paper to emphasize these points and include additional references to clarify this choice.

[1] Wang, Yan, et al. "Train in germany, test in the usa: Making 3d object detectors generalize." CVPR 2020.

[2] Wang, Yan, et al. "Ssda3d: Semi-supervised domain adaptation for 3d object detection from point cloud." AAAI 2023

[3] Tsai, Darren, et al. "Ms3d++: Ensemble of experts for multi-source unsupervised domain adaptation in 3d object detection." IEEE Transactions on Intelligent Vehicles 2024

[4] Hegde, Deepti, et al. "Multimodal 3D Object Detection on Unseen Domains." arXiv 2024

We sincerely thank the reviewer for their thoughtful evaluation and detailed feedback on our paper. We deeply appreciate the recognition of our work addressing the critical problem of uncertainty quantification in 3D object detection. We are particularly grateful for the acknowledgment of the efficiency of our evidential learning-based approach and the effectiveness of the prior-based regularization in mitigating model overconfidence. We have carefully considered all comments and suggestions and provide detailed responses to each concern below.

• This paper lacks novelty, the method of quantifying uncertainty is not much different from that in paper [1].

We appreciate the reviewer bringing up the comparison with the mentioned paper [1]. While we acknowledge certain similarities between the two approaches, we respectfully highlight several key distinctions that underscore the novelty and uniqueness of our work:

• Focus on Epistemic Uncertainty: While [1] primarily integrates uncertainty into the inference pipeline to enhance detection quality, therefore focusing largely on aleatoric uncertainty, our approach prioritizes epistemic uncertainty. In safety-critical applications like autonomous driving, epistemic uncertainty is essential for understanding model reliability in out-of-distribution scenarios [2, 3], complex or unknown scene conditions [4], and situations with limited training data [5]. This emphasis on epistemic uncertainty not only distinguishes our contribution but also aligns naturally with active learning setups, where accurately quantifying uncertainty enables efficient selection of informative samples for labeling, ultimately improving model robustness and reliability.

• Different Distribution Formalization: The mentioned paper adopts a Dirichlet distribution for modeling objectness heatmap predictions. In contrast, our approach employs a Beta distribution formulation, which is tailored to the requirements of modern 3D object detection architechtures. This distinction is important, as our formulation required deriving specific theoretical adjustments and practical modifications to integrate effectively within 3D detection pipelines. These differences make it unsuitable to directly borrow techniques from [1].

• Model Integration and Efficiency: Our approach requires minimal architectural changes by adding a small component to the existing model, making it lightweight and easy to integrate into 3D detection pipelines. On the other hand, the mentioned approach [1] retrains the entire model to incorporate uncertainty estimation, requiring specialized losses and significantly modified architectures. Unlike this, our method is compatible with any model utilizing BEV features and works seamlessly with various training setups. This distinction underscores the efficiency of our method and highlights its practicality for large-scale applications, particularly in resource-constrained environments such as autonomous driving.

• 3D vs. 2D Detection Context: Quantifying uncertainty in 3D object detection poses fundamentally different challenges compared to 2D detection. While 2D detection also involves spatial relationships and object localization, it is limited to projecting objects onto a 2D plane, where the relationships between objects are inherently simpler. In contrast, 3D detection requires reasoning about spatial relationships in a three-dimensional environment, where factors like depth, scale, occlusion, and perspective significantly increase the complexity. Additionally, 3D detection often involves multi-modal inputs (e.g., LiDAR and camera), which must be fused effectively. Our method addresses these complexities by leveraging Bird's Eye View (BEV) features tailored for the 3D context, which, to the best of our knowledge, has not been done before in this way.

We believe these points illustrate the distinct contributions of our work and its relevance to advancing uncertainty quantification for 3D object detection. We hope this clarifies the novelty and impact of our approach.

Review #n1q7

We are grateful to the reviewer for their detailed review and valuable suggestions regarding our paper. We also deeply appreciate their recognition of of the clarity in our writing and the thoroughness of our experiments, while also considering strong aspects of our method. We have carefully addressed all of the concerns and suggestions raised, as discussed below.

• The proposed method does not adequately address the limitations of traditional approaches, necessitating continued extensive re-annotation efforts without a significant leap in detection efficiency.

We appreciate the reviewer’s observation regarding annotation requirements. Indeed, our method does not eliminate the need for labeled data, and we did not claim it would. Effective training, especially for complex tasks like 3D object detection, inherently benefits from additional labeled data, and our approach aligns with this understanding.

The core contribution of our paper is an improved uncertainty estimation method that outperforms traditional approaches in efficiency and effectiveness. Specifically, we demonstrate its advantages in an active learning setup, where our method allows for more strategic data selection, thereby reducing the volume of new annotations needed compared to existing uncertainty methods. In other words, while labeling remains essential, our approach allows for a more efficient, targeted annotation process,

• The key to enhancing performance in real-time robotics applications, particularly in autonomous driving, lies in the effective management of uncertainty during the detection phase, rather than the re-annotation process.

Our approach is designed to effectively manage uncertainty during inference. We will clarify the wide applicability of our framework in the revised paper to ensure this point is more evident. While our final experiment focuses on a re-annotation task, our initial experiments on OOD detection, bounding box quality assessment, and missed object identification can also directly be used to drive real-time processes in robotics and autonomous driving. For example, our method effectively flags high-uncertainty scenes, enabling autonomous systems to hand control back to human drivers—a feature that directly enhances operational safety. These showcases underscore our approach’s value in dynamically managing uncertainty during detection, making it highly applicable for real-time settings.

• The paper does not provide clear evidence on how the uncertainty estimation directly benefits real autonomous driving applications, such as motion planning.

We will revise the paper to better emphasize how our method effectively benefits real-world applications, including motion planning, as mentioned by the reviewer. Our paper demonstrates how improved uncertainty estimation directly benefits real-world applications like motion planning. By identifying uncertain regions or detections during inference, our approach enables adaptive responses, such as human intervention or planning adjustments in high-risk situations. This supports safer and more informed decision-making, which is critical for autonomous driving. While enhancing the planner itself is outside the scope of this work, as it is a distinct and complex task, our method provides valuable uncertainty insights that can improve the safety and reliability of the planning process through effective uncertainty management.

Thank you for providing the details of auto-label pipeline.

We sincerely appreciate the reviewer’s thoughtful feedback on our work. We are especially grateful for the recognition of the theoretical foundation of our method, the meaningful experimental results, and the innovation and applicability of our approach to real-world challenges in autonomous driving. Below, we carefully address all the concerns and suggestions raised.

• The method mainly focuses on providing uncertainty estimates without apparent direct improvement in detection accuracy itself.

Indeed, the primary goal of our method is not to directly improve detection accuracy but to provide reliable uncertainty estimates while also maintaining the accuracy of the considered model. Comparisons between the original model and our method confirm that the accuracy remains consistent, ensuring that our approach adds uncertainty estimation capabilities without compromising detection performance. However, our active learning setup demonstrates how uncertainty estimation can make the labeling process for self-driving applications more efficient. By strategically prioritizing high-uncertainty regions for relabeling, our approach indirectly improves detection accuracy by ensuring higher-quality training data.

• While experiments cover LiDAR and LiDAR+Camera setups, a significant portion of current detection models are Camera-only; hence, the framework lacks some supportion in experiment to demenstrate the generality in this area.

While it is true that we focused more on setups that include LiDAR information in inference, our method operates at the BEV level, which is agnostic to the type of inputs used to generate predictions. The choice of input modalities primarily affects the quality of the BEV features, which is beyond the scope of our work, as we directly use the BEV representations provided by the backbone.

That said, we agree that including experiments with camera-only setups would further strengthen the demonstration of our method’s generality. However, conceptually, such experiments would not alter the main conclusions of the paper, as also pointed out by reviewer #7H8K. We will consider this extension in future work to provide a more comprehensive evaluation.

• The paper claims efficiency advantages over traditional methods, suggesting suitability for real-time applications like driving; however, it does not provide a direct comparison in terms of computational efficiency. It remains unclear how much this added uncertainty estimation slows down the original model.

Our method introduces minimal computational overhead, as it only modifies a small part of the model: the heatmap prediction head. This component consists of a few lightweight convolutional layers applied directly on top of the BEV features. As a result, the additional computational cost is negligible compared to the overall model inference time, amounting to roughly 1% of the original inference time.

In contrast, standard sampling-based approaches require multiple forward passes (N times the inference time), making our method significantly more efficient. This negligible overhead underscores the suitability of our approach for real-time applications like autonomous driving.

• It's well understood that traditional methods like MC Dropout and Deep Ensembles have high computational costs due to repeated inference, which is why they’re increasingly less popular in uncertainty quantification for autonomous driving. Given this shift, the question is: compared to existing UQ directly modeling methods, such as [1], does your evidential learning approach offer any distinct advantages?

Evidential learning has been gaining popularity recently due to its ability to efficiently estimate uncertainty while providing high-quality epistemic uncertainty estimates—a notable weak spot for many existing sampling-free methods. Unlike sampling-based approaches, which are well-suited for estimating epistemic uncertainty but come with prohibitive computational costs, evidential learning offers an appealing trade-off by combining efficiency with robust epistemic uncertainty estimation.

Regarding [1], while the proposed method falls under the category of sampling-free uncertainty estimation, it directly predicts predictive variance. This approach is known to poorly represent epistemic uncertainty, as it often conflates aleatoric and epistemic uncertainty, leading to potential limitations in its applicability. Evidential learning, in contrast, provides a principled way to model epistemic uncertainty explicitly, making it a compelling alternative in scenarios requiring efficient and reliable uncertainty quantification.

[1] Su, Sanbao, et al. "Uncertainty quantification of collaborative detection for self-driving." ICRA 2023.