Distil-E2D: Distilling Image-to-Depth Priors for Event-Based Monocular Depth Estimation

Q1: Mechanisms to prevent trivial collapse in the confidence weighting module.

We thank the reviewer for highlighting this potential failure mode. To prevent collapse of $w_t^d$, we incorporate two stabilizing mechanisms: base clamping and a sparse confidence regularizer.

Specifically, in base clamping we reparametrize:

$\tilde{w}_t^d = \alpha + (1 - \alpha) w_t^d,$

with $\alpha = 0.1$, where $\tilde{w}_t^d$ is the final confidence weights. This enforces $\tilde{w}_t^d \in [0.1, 1]$ to avoid vanishing gradients. Additionally, the sparse confidence regularizer applies an L1 regularization loss:

$\mathcal{L}_{\text{reg}} = \beta\| w_t^d - \hat{w}_t^s \|_1,$

where $\beta=0.5$ and $\hat{w}_t^s$ is the min-max normalized sparse confidence map from Equation (10). This regularizer encourages consistency with high-confidence regions and promotes spatially adaptive weighting. We will include these implementation details in the final supplementary material.

Q2: Request for visualizations or quantitative statistics of ACE confidence maps.

We report summary statistics of the dense confidence map $w_t^d$ computed over the MVSEC training set:

These statistics demonstrate that $w_t^d$ exhibits variation and spans the full expected dynamic range, and does not collapse to a trivial solution. Although visualizations of the confidence maps cannot be presented in the rebuttal, we will provide them in the final supplementary material.

Q1: Inconsistencies in Figure 1 annotations and equations.

We thank the reviewer for pointing out the errors in the figure. The coefficient in $L_{align}$ between Calibrated Depth and LiDAR Depth should be $w^d_t$ instead of $W_c$. Furthermore, the arrow between DepthAnythingV2 and MDE Prediction should not exist. We will have the figure corrected in the camera-ready paper.

Q2: Inconsistent acronym usage and duplicate references.

We thank the reviewer for the feedback. We will align the naming and correct the references.

Q3: Justification for using logarithmic depth in Equation (7).

We use log depth instead of true depth because it offers numerical stability and better captures relative depth relationships, which is crucial when aligning synthetic and LiDAR depths. Depth values in driving scenes can span several orders of magnitude from very close objects (e.g., less than 2 meters) to distant background regions (tens of meters). Applying a log transform compresses this dynamic range into a bounded interval (approximately 0 to 1 in our normalized setup). This ensures that both near and far depths contribute comparably to the loss.

Q4: Lack of detailed encoder and decoder architecture descriptions.

Please refer to our response to Reviewer qS9Q Q2 for full architectural details, which will also be included in the final supplementary material.

Q5: Clarification on whether training data includes low-light sequences.

The training includes only well-illuminated sequences. For example, we use the outdoor_day2 sequence in MVSEC for training, which is also the standard training set for other competing algorithms. We refrain from using nighttime sequences for training because of the degraded quality of monocular depth estimates in low light.

Dear Authors:

Thank you for your detailed rebuttal. I have carefully reviewed your responses to my previous comments and find that my concerns have been adequately addressed.

I now regard the paper as a solid contribution to the field. I will accordingly increase my review score.

Best regards

Q1: Fairness of using DAV2 in E+I experimental protocol.

We thank the reviewer for raising this point regarding the fairness of the E+I experiments. Using DAV2 as the image encoder is a natural and principled design choice. It avoids the need to train a separate image encoder from scratch with limited data by using the rich priors of foundational depth models. Importantly, incorporating DAV2 does not trivially solve the depth estimation problem. APS frames often yield inaccurate depth predictions when relying solely on DAV2, particularly in challenging nighttime scenes. Additional challenges arise from depth misalignment between synthetic supervision and real-world data. In our E+I framework, events play a crucial role by offering high-temporal-resolution cues that complement DAV2 features. This leads to improved performance in scenarios where image-only models struggle. We thus view our approach as a meaningful extension of existing models instead of being an unfair advantage. This is because events remain essential to achieving accurate depth estimation.

Q2: Missing implementation details on encoder and DAV2 architecture.

We use DepthAnythingv2-L for the frame encoder in the E+I setting and for synthetic depth generation. The architecture of our event encoder, context encoder, and RDE/MDE decoder is as follows:

Event Encoder

Context Encoder

Decoder (RDE and MDE Decoders)

Skip connections are applied between the event encoder and RDE/MDE decoders. We will include these architectural details in the supplementary material.

Q3: Inference time comparison between ERE and MDDE+.

We thank the reviewer for this observation. Upon revisiting our runtime measurements, we verified that runtime of ERE is as reported (35ms). However, we realized that our reported MDDE+ runtime (24.2ms) had mistakenly included pre-processing overhead. In our revised measurement, MDDE+ achieves approximately 8ms on our setup. This confirms that it is indeed much faster than ERE. We will update the runtime results accordingly.

Q4: Limited evaluation of proxy supervision using only one VFM.

We thank the reviewer for this insightful suggestion. Although our study focused on a single VFM to maintain a controlled analysis, exploring other VFMs such as MoGe, DepthPro, and Lotus, or even a mixture-of-experts design could potentially further improve depth distillation. We acknowledge this as a valuable direction for future work.

Q5: Scalability with large unlabelled datasets.

We agree that proxy pretraining on large-scale unlabelled event-image data is a highly promising direction. Our framework is inherently compatible with such scaling and has a strong potential to yield generalizable EMDE models when applied to large-scale data. Although the construction and processing of such datasets requires resources beyond the scope of this work, we view this as a natural and impactful extension of our approach and an important direction for future research.

Q6: Use of relative versus metric foundational models for depth supervision.

We ran some initial experiments with Depth Pro as a source of metric supervision. However, we found that the predicted depths often exhibited inconsistent global scaling and do not align well with the LiDAR ground truth across different scenes. In contrast, relative depth models such as Depth Anything V2 produced more temporally stable output with sharper object boundaries and more coherent segmentation of surfaces. These qualitative differences resulted in a better spatial structure, which we then calibrated to metric space using our NDC module. Our two-stage design, which first captures rich relative geometry before calibrating to metric space, offers a more reliable and modular alternative to directly relying on metric predictions from VFMs.

Q1: Potential distortions induced by the context branch under accelerated motion.

We analyze whether the context branch introduces distortions under acceleration on the MVSEC test set using event-only input. The test frames are segmented into low, medium, and high motion bins based on the average ground truth optical flow magnitude with thresholds defined by the 33rd and 66th percentiles. For each bin, we report the mean depth error at 30m.

The results show that with the context branch, the accuracy decreases with increasing motion due to the growing misalignment between frames, which reduces the effectiveness of temporal fusion. Nonetheless, the context branch continues to provide valuable temporal cues and structural continuity, yielding improved accuracy over the baseline across low to high motion magnitudes.

Q2: Clarification on APS, its distinction from RGB, and the appearance of saturation in Figure 1.

APS (Active Pixel Sensor) frames are captured from the same imaging plane as events and are inherently co-aligned. In contrast, RGB frames are acquired from an external camera and require extrinsic calibration. As seen in Figure 1, APS frames may appear saturated under challenging lighting due to limited dynamic range. Although APS and events share the same imaging plane, they operate on distinct circuitry and sensing principles. Specifically, event data more robust to extreme illumination since APS measures absolute intensity while events encode log-intensity changes.

Q3: Notation error in Equation (5).

Thank you for highlighting this. We will correct the notation in the camera-ready version.

Q4: Discussion about model generalization.

Similar to existing EMDE models, our method currently does not generalize to open-world scenes and remains dataset-specific. This work focuses primarily on improving the quality of supervision through dense depth distillation. However, we acknowledge that the lack of large-scale event datasets remains a key bottleneck. We hope that our work motivates future efforts to acquire such datasets. This would enable our work and other future work to be extended to generalizable event-based monocular depth estimation.