Repurposing Marigold for Zero-Shot Metric Depth Estimation via Defocus Blur Cues

We thank the reviewer for appreciating the clarity of our work and the motivation to explore other depth cues in conjunction with foundation model-based depth estimators. We want to emphasize that our work aims to demonstrate that incorporating camera physics, particularly defocus blur, into a data-driven Monocular Depth Estimation model at inference time can effectively address both texture coupling and incorrect metric scale estimation modes without requiring any re-training (L:34-39, main paper). We have validated both these claims through synthetic toy examples (Fig. 3) and also on real camera hardware data. We agree that our method is closely related to defocus blur-based multi-shot approaches, so comparing it with them does contextualize things better. To that end, we have added comparisons with relevant previous works below. However, we would like to reiterate that developing the best-performing single / multi-shot method was not our primary goal or claim, and that the results/validations presented in the manuscript adequately validate our claims (L:34-39).

Comparison with previous camera physics + deep learning based work.
Most of the previous methods (L:98-L:110) train a deep learning network, but on relatively smaller datasets, while also requiring >=4 images as inputs. This severely limits generalizability and performance outside the training data distributions. While these methods [A, B] attempt to use priors such as total variation or task-specific loss functions while training the network, they are not strong enough to overcome the ill-posedness of the depth from defocus blur inverse problem with only aperture supervision and 2 images as input -- which is the regime our method operates in. We demonstrate this quantitatively in Table 1 below.

As suggested, we trained [B] and the direct depth supervision baseline mentioned in [B] with the same inputs as our method on the NYUv2 dataset [E], which is a well-known RGBD public benchmark for Monocular Depth Estimation. NYUv2 contains AIF images and GT depth pairs (654 test, 795 training). For our method, we synthesize the blurred input image at aperture (F/8) and focus distance (1.5m) using our forward model for a simulated comparison.

The results in Table 1 below show that our method achieves comparable performance with existing SOTA metric depth foundation model baselines (MLPro, UniDepth, Metric3D) reported in our work, while previous camera physics + deep learning methods (last 2 rows, Table 1) struggle in performance.

We would like to emphasize that the leading methods above in Table 1 are trained on Millions of Image-depth pairs to predict metric depth (Metric3D: 8M, UniDepth: 3M), while our method performs comparably to them without any re-training using defocus cues with a scale-invariant monocular depth estimator (Marigold).

While our method does use gradient-based optimization, we would like to remind the reviewer that this gradient-based optimization happens in the latent space of Marigold, which lets us leverage internet-data scale priors in Marigold that are strong enough to achieve competitive results for metric depth estimation, without re-training, using only 2 input images. Previous approaches struggle in this scenario, as shown in Table 1( last 2 rows) above. Our work aims to demonstrate (L:34) that incorporating physics cues with a pre-trained foundation model can be a powerful unlock for tasks that it hasn't been explicitly trained on, but lie in the vicinity of the base task. The current evidence presented in our paper validates this claim.

GT Data Quality.
Acquiring very high-quality depth maps from real hardware is non-trivial, and often requires complex structured light setups [C]. While we acknowledge that RealSense is not as precise as LiDAR or structured-light scanners, it is a much cheaper, hence academically viable alternative. It provides dense, metrically accurate depth within its operating range (0.2–3.8 m) using stereo cues, and has been used as ground truth in prior monocular depth from defocus work [D]. We clarify that our goal is not to achieve sub-millimeter accurate ground truth, but rather to evaluate relative and metric consistency in practical indoor scenarios. RealSense has a much larger field of view compared to the DSLR camera used to capture the all-in-focus (AIF) image. As a result, while plotting in 2D, the details in the depth map look slightly blurry. As suggested by the reviewer, we have visualized the error maps for our method and MLPro against the ground truth. We will add these visualizations to the camera-ready / appendix, but the observations from the visualizations are as follows: The visualizations show that MLPro (which has sharp boundaries in its predictions) struggles to predict correct depth scales for some areas in the scene, resulting in significant errors on planar regions. Our visualizations indicate that there is no particular bias towards high error on sharp boundaries in our collected data, making it suitable as ground truth for comparison.

Lack of real-world datasets with multiaperture inputs.
Our method requires an AIF and a blurred image as inputs, and a ground truth depth map for evaluation. Most public RGBD datasets only contain the RGB image and ground truth depth map, but not the blurred image. So it's hard to evaluate our method on those. However, as suggested by the reviewer, we have also evaluated our method and other baselines from our work on the test set (654 images) of the well-established NYUv2 test set (see Table 1 above).

Is Marigold Necessary? Extending to other methods.
Our method is generally applicable to any diffusion-based model like Marigold, since this provides access to a differentiable mapping from the diffusion model latent space to our depth estimates and thus allows for optimizing these latents at test time, which is required to incorporate defocus cues without re-training the base model. Additionally, our choice of Marigold was also motivated by the fact that the Marigold authors released a Latent Consistency Model (LCM) version of Marigold, which allows sampling the diffusion model in a single step, which is beneficial for faster inference. However, our method is plug-and-play and can be, in principle, applied to any diffusion-based estimator, which provides access to the latent space of the model. We demonstrate this by extending our method to Geowizard as well. We are referencing Table 3 here from our response to 1d2m.

GeoWizard [F] is a diffusion-based monocular depth + normal predictor, trained on a larger and more complex data distribution by fine-tuning the stable diffusion UNet. We observe that the relative depth from geowizard after inference time optimization exhibits minor patch-level artifacts, potentially due to stronger texture-depth coupling. Unlike Marigold, Geowizard does not provide a latent consistency model (LCM) checkpoint; hence, it requires more sampling steps. The metrics for Geowizard obtained in Table 2 are with 50 optimization iterations, and 5 sampling steps for Geowizard per iteration. Geowizard is 4x slower than the Marigold-LCM checkpoint on an A-40 GPU, but still achieves comparable results. This highlights the flexibility of our approach: as stronger or faster diffusion models become available, our method can seamlessly leverage them for improved performance.

We will add these numbers, visualization, and discussion in a revision.

[A] Depth Estimation via Reconstructing Defocus Image: CVPR 2023.
[B] Aperture Supervision for Monocular Depth Estimation: CVPR 2018.
[C] High-Accuracy Stereo Depth Maps Using Structured Light: CVPR 2003.
[D] Passive snapshot coded aperture dual-pixel RGB-D imaging: CVPR 2024.
[E] NYU Depth Dataset V2: ECCV 2012.
[F] GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image: ECCV 2024.

We are glad that our rebuttal addressed the concerns raised by MnCO in their initial review regarding the real dataset and extension of our method to other backbones such as Geowizard. We note the concerns raised by MnCO in their new comment regarding a fairer comparison with potential solutions.

Please see our comments in response to 1d2m, titled Regarding baseline Comparisons: Part 1 and Baseline Comparisons: Part 2 -- Comparing with fine-tuning-based methods. For convenience, we have reproduced these responses below as separate comments with the same content.

In these comments, we address concerns regarding baseline comparisons raised by both 1d2m and MnCo, and present comparisons with the suggested fine-tuning-based methods.

We have summarized our results in Table 4 and Table 5 of the comment Baseline Comparisons: Part 2 -- Comparing with fine-tuning-based methods. Our results demonstrate the adaptability of inference-time methods to out-of-distribution domains without any re-training, unlike fine-tuning-based approaches that can struggle due to the domain gap between the fine-tuning and test data. The comments above detail our experimental setup and provide supporting quantitative evidence.

We thank the reviewer for appreciating the novelty of our approach, the clarity of the paper, and our toy experiments and validation. We agree that handling dynamic scenes with the current approach is challenging and would be an interesting avenue for future work. We will expand our collected dataset to add more test scenes to facilitate future research in this area.

Discussion on adapting the method to feedforward depth models.
Our method requires that the inference process of our depth model works by differentiably mapping from some latent embedding space to the output depth, such as the one implicitly provided by diffusion models from noise to data (GANs and VAEs could in principle similarly work). This enables inference time optimization of the latent vector at test time. However, discriminative or autoregressive transformer-based approaches, such as DepthAnything, don’t provide access to such a latent space. Adapting pre-trained autoregressive transformers to new datasets/domains typically involves test-time adaptation [A], which requires collecting a sizable dataset and then adding some extra layers, which are then fine-tuned with this dataset while keeping the rest of the weights frozen. Such an approach can be potentially used to extend methods like DepthAnythingv2 for handling multi-aperture input. But unlike our method, which is training-free, that would require fine-tuning / re-training the adaptation layers in the model. We are unaware of any large-scale datasets with multi-aperture defocus captures, but this approach might become viable in the future if such datasets emerge. We will add this discussion in a revision.

Real dataset size.
Our motivation to collect this data was to create a validation set to verify our framework on real hardware. We agree that a larger dataset with depth maps, all in focus, and multiple aperture inputs would be useful for the community, and we plan to continue adding more examples to our released dataset. As suggested by other reviewers, we have also evaluated our method through simulated measurements on the NYUv2 dataset (654 test images), which consists of ground truth AIF and depth maps, and demonstrate comparable performance to SOTA metric depth foundation models. (Table 1 in MnCO rebuttal response).

[A] Rapid network adaptation: ICCV 2023.

We thank 1d2m for their detailed critique and for recognizing the novelty of our work. Our work demonstrates that Marigold, a generative model not built to incorporate defocus cues, can be repurposed to include them in a principled training-free manner. This achieves robust generalization without labeled depth datasets or retraining. We have validated our claims and the proposed framework through real-world experiments on a self-collected dataset, which also demonstrates the importance of correctly modeling the defocus blur (Fig. 2, Tab.1, L:253-257 in the main paper). We have addressed the questions from 1d2m below:

Weakness 1: Misleading SOTA comparison and input modality, comparison with multiaperture methods.
Misleading SOTA Comparison.
We would like to emphasize that our work aims to establish an existence proof of the proposed framework, and that our intent was not to suggest that we establish a new state-of-the-art for multi-shot depth imaging. If such an impression was conveyed, it was entirely unintentional, and we will clarify this more clearly in the paper. We intended to showcase the effectiveness of our approach within the context of our framework, without implying any misrepresentation of its standing relative to prior work.
Comparison with Multi-Aperture Methods:.
We show that incorporating defocus cues at test time enables Marigold to achieve zero-shot metric depth, which is competitive in quality with zero-shot metric depth foundation models by simply using an additional blurred image as input along with the all-in-focus (AIF) image. We agree that this does make our method multishot and have presented comparisons with relevant baselines below, as we believe that contextualizing our performance to multi-shot methods is valuable. However, we emphasize that we don't claim to be the best multi-shot/defocus base method. While these additional comparisons are valuable, they do not affect the claims (L:34) and the demonstrated validation in our paper.

While our work indeed falls in the category of monocular metric depth estimation with more than 1 image, we are different from DFD-based methods, since we use 2 images with different apertures, but the same focus distance. This is different from the majority of existing DfD methods that require at least 4-5 images captured at different focus distances [H, A]. We attempted to adapt DEReD [A], a SOTA self-supervised learning based DFD baseline, but observed that it performs very poorly and fails to converge when given only an AIF and a blurred image as input, as recovering metric depth from 2 images is a highly ill-posed problem, unless very strong priors or really large datasets are used – neither of which has been used in most DfD works. We also discuss this in more detail in the response to MnCO.

Our setup is more similar to [B], which also relies on two images captured at the same focus distance: an AIF and a blurred image as inputs. They propose a differentiable model that takes an AIF and simulates defocus blur by interpolating pre-blurred AIFs at discrete depth levels, with weights predicted by a CNN that implicitly encodes scene depth. The network is trained to match the synthesized image to the captured (ground truth) blurred image.

We implemented their method, and also a depth-supervision-based baseline mentioned in their work, which involves directly regressing metric depth from a network with the AIF image stack as input. Following their training-based framework, we trained and validated these baselines on NYU-v2 [E], a popular indoor RGBD benchmark. We observe in Table 2 that the methods show poor performance metrics on the NYU-v2 test set. We observe that the Depth supervision baseline produces blurry results, as observed in [B], and the method proposed in [B] struggles with strong texture-depth coupling, resulting in inaccurate metric depth.

Repeating Fig 6.
In Fig. 6, we used a single blurred image as input to Marigold, not as a baseline but just to check if a single image is sufficient for metric depth estimation (RMSE: 1.36). However, as suggested, we used 2 blurred images (f/16 and f/8) as input, with f/16 as the proxy to AIF (RMSE: 0.6). While better than using a single blurred image, F/16 is a less accurate proxy for our AIF (f/22 vs f/8), which leads to a higher error (RMSE: 0.341). This highlights the importance of a high-quality AIF and shows that performance degrades gracefully with increased blur in the AIF.

Weakness 2: Limited Practicality.
Applying our method to GeoWizard.
We agree with the reviewer and show results below from applying our method with the Geowizard model (Table 3), demonstrating the plug-and-play nature of our method. GeoWizard is similar to Marigold but trained on a larger and more complex data distribution by fine-tuning the stable diffusion UNet. We observe that after the inference time optimization, Geowizard relative depths exhibit minor patch-level artifacts, potentially due to stronger texture-depth coupling. We will add these numbers and visualizations in a revision.

Computational Latency.
We agree that our method is slower than feedforward depth models; however, this trade-off does not diminish our core contribution (L:34-39). We are similar in runtime to other diffusion-based inference-time optimization methods [I], and future advances in this space will also benefit our runtime.

Weakness 3: Justification for defocus blur.
Defocus blur provides metric depth cues and is readily available in images as it's a fundamental property of the camera lens. Compared to other cues, such as stereo (2 cameras), defocus blur allows metric depth estimation with a single camera, and specifically without requiring calibration for the camera extrinsics, which is required for stereo. Moreover, while stereo cameras are widely used, their applicability is limited in compact settings like endoscopy. Other camera physics methods, such as coded apertures, require manufacturing custom hardware. In comparison, defocus cues are more accessible, as they are present in images captured with any standard camera lens. While Metric3D resizes images based on camera focal lengths to ensure stable training over datasets of different depth scales, it doesn’t provide a physically grounded depth cue like defocus blur, and thus cannot be leveraged for per-scene inference time optimization.

Unique Challenges and benefits of Marigold + defocus blur.
Defocus blur offers a differentiable forward model using an AIF and depth map, making it suitable for integration with Marigold at test time. To address mismatches with real cameras (L234, Fig. 2), we adopt a more accurate disk kernel over a Gaussian. While this model simplifies gradient computation, it ignores occlusions, leading to minor errors at depth discontinuities.

Weakness 4: Marigold Specific Advantages:.

1. Unlike feedforward methods such as DepthAnythingv2 or UniDepth, diffusion-based methods like Marigold provide access to a low-dimensional latent manifold that maps from a noise vector to a relative depth map differentiably. This enables test-time refinement of Marigold’s predictions by incorporating physical cues like defocus blur through differentiable forward models, as shown in Fig. 3.
2. Marigold is obtained by fine-tuning StableDiffusion [G] (trained on internet-scale RGB data), allowing it to inherit strong real-world generalization capabilities from StableDiffusion. Most other feedforward models are trained only on RGBD datasets, which are relatively smaller compared to the stable diffusion training dataset.
3. Marigold also retains the well-known benefits of diffusion-based depth predictors, such as better fine detail recovery [F] compared to feedforward methods. Additionally, Marigold also provides a latent consistency checkpoint, which speeds up inference compared to using Geowizard.

[A] Depth Estimation via Reconstructing Defocus Image: CVPR 2023.
[B] Aperture Supervision for Monocular Depth Estimation: CVPR 2018.
[C] High-Accuracy Stereo Depth Maps Using Structured Light: CVPR 2003.
[D] Passive snapshot coded aperture dual-pixel RGB-D imaging: CVPR 2024.
[E] NYU Depth Dataset V2: ECCV 2012.
[F] GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image: ECCV 2024.
[G] SDXL: ICLR 2024.
[H] Bridging Unsupervised and Supervised Depth From Focus via All-in-Focus Supervision: ICCV 2021.
[I] DITTO: Diffusion Inference-Time T-Optimization for Music Generation : ICML 2024

We thank the reviewer for their response and are glad that our rebuttal clarified many of their concerns. Below, we address the follow-up questions on Figure 6 and focus breathing raised by the reviewer in their official comment. Please see our next set of comments for our response to the concerns regarding baseline comparisons raised by 1d2m and MnCo in their official comment.

Figure 6 clarification: In our method, Marigold takes in AIF as input (captured at F/22), and predicts a relative depth map. We then synthesize a blurred image, using our forward model (eq 7 in the paper) that uses the all-in-focus (AIF), relative depth from marigold, and the learnable scale-offset parameters. We then compare the synthesized blurred image with the captured blurred image (at F/8), which contains sufficient blur to discern metric depth cues.

Of these two inputs, it is well known [A] that the blurred image provides metric depth cues and thus is an important input. To investigate the impact of the AIF in this setup, we don’t provide the AIF as input, and use only a modestly blurred image (F/16) as input in our setup, for both Marigold as well as for comparison with the synthesized blurred image. We will clarify in the paper that the experiment for Fig. 6 was meant to be an ablation study, to inspect the impact of the AIF image in our setup, and whether our proposed approach can work with a single, modestly blurred image.

As discussed in the main paper (Fig. 6) and the rebuttal response to 1d2m, we observe that Marigold performs poorly with even modest aperture blur, resulting in significant inaccuracies as compared to using two images. This motivates the two-image approach: the AIF provides high-quality depth from Marigold, and the blurred image provides metric depth cues.

Focus Breathing: We would like to clarify that focus breathing is the change in field of view (FoV) caused by adjusting the focus distance of the camera lens [C]. This effect arises from variations in optical magnification with focus distance. Methods such as [A], which rely on multiple inputs captured at different focus distances (focal stacks), require alignment to account for these FoV shifts.

Importantly, focus breathing does not occur when varying the aperture; it happens only when adjusting the focus distance.

By definition [D], the aperture/aperture stop is a surface in the system whose size affects only the area over which light is collected, and not the angle (FoV). Since our method uses inputs captured at different apertures but the same focus distance, no FoV alignment is needed. This gives our method an advantage: unlike focal stack-based approaches, we avoid both FoV misalignment and the need for extrinsic calibration. We have also demonstrated competitive performance at different focus distances (0.8m in Table 1 of the main paper and 1.5m in Table 1 in response to MnCO’s review).

[A]: Fully Self-Supervised Depth Estimation from Defocus Clue: CVPR 2023.
[B]: Aperture Supervision for Monocular Depth Estimation: CVPR 2018.
[C]: An Implicit Neural Representation for the Image Stack: Depth, All in Focus, and High Dynamic Range: Siggraph Asia 2023.
[D] Ch 7.1 in Fundamentals of Optics 4th Edition, by Francis A Jenkins, Harvey E. White.

Comparisons with multi-aperture methods 1d2m rightly pointed out in their initial review that our approach relies on multi-aperture inputs, and suggested that we compare our method to previous DfD-like / multi-aperture methods. We added those comparisons in (Table 1, 1d2m response), as they offer valuable context for comparing our method with previous work.
Baseline [B] in our rebuttal utilizes the same type of multi-aperture inputs as our method. Like [B], our method also doesn’t directly use depth supervision, as our optimization tries to minimize the error between the synthesized and observed blurred image. Given these points, [B] can be considered a fair comparison to our method.

Comparison with fine-tuning-based methods.
As suggested by MnCO and 1d2m in their initial reviews, we provided comparisons with a camera physics + deep learning based method in our rebuttal (Table 1, MnCO response and Table 2, 1d2m response) with 2 baselines. Our method relies on a strong relative depth foundation model (RDFM), and incorporates defocus cues at test-time, without re-training the model. Hence, in the initial submission, we did not compare with fine-tuning-based methods, as they are outside the premise of our claims.

While such a comparison would be valuable, a fair comparison would be between our method, i.e., RDFM + test time optimization, and Metric depth models obtained by fine-tuning RDFMs (DepthAnythingv2 [A1] / Marigold) on Metric depth datasets with a restricted depth range (L:225, 290 in paper). We have not added a comparison with ZoeDepth as it is less performant than DepthAnythingV2 [A1] and MLPro [A2] (which we have already compared with and whose publications already compare to ZoeDepth). As noted by 1d2m in their comment, inference time optimization (our method) can be valuable in zero-shot settings, where fine-tuning-based methods would struggle due to the domain gap. We validate this by evaluating 3 additional fine-tuning-based methods on our collected real dataset in a zero-shot setting. Specifically, we consider the following baselines -

1. DepthAnythingv2 fine-tuned on Hypersim (metric depth), as suggested by MnCO. For this baseline, we used the pre-trained model fine-tuned on this configuration released by the authors.
2. DepthAnythingv2 fine-tuned on NYU-v2 (metric depth). We used the official fine-tuning code in the DepthAnythingv2 repo to fine-tune the base model for 25 epochs.
3. Marigold Fine-Tuned on NYUv2, as suggested by 1d2m. We fine-tuned Marigold for 60 epochs using the official training code. To obtain both depth and RGB latents, Marigold uses the same encoder, requiring inputs normalized to [0,1]. This makes direct fine-tuning with metric depth supervision non-trivial. As a practical proxy, we normalize the GT depth maps (for training supervision) using a fixed global minimum and maximum depth for the full NYUv2 dataset (0.1 m, 10 m), rather than the originally used per-image normalization. Developing optimal fine-tuning strategies for Marigold with true metric depth supervision is beyond the scope of this work; our setup here serves solely as a baseline for comparison and context.

We summarize our results in Table 4 (our dataset) and Table 5 (nyuv2 dataset) in the next comment (Baseline Comparisons Part 2), which validates the utility of our method in zero-shot scenarios, where fine-tuning-based methods can potentially suffer in performance due to the domain gap (Table 4). On our real dataset (Table 4), we outperform all the fine-tuning baselines. On the NYUv2 dataset (Table 5), we either outperform or are comparable to all the baselines except DepthAnythingV2 fine-tuned on NYUv2, which is expected, as the fine-tuning dataset is in the same distribution as the test dataset. Please see the captions of Table 4 and Table 5 in the next comment for more details.

[A1] Depth Anything V2: Neurips 2024.
[A2] Depth Pro: Sharp Monocular Metric Depth in Less Than a Second: ICLR 2025.
[B]: Aperture Supervision for Monocular Depth Estimation: CVPR 2018.

[A1] Depth Anything V2: Neurips 2024.
[A3] DITTO: Diffusion Inference Time T Optimization: ICML 2024.
[A4] End-to-End Diffusion Latent Optimization Improves Classifier Guidance: ICCV 2023.

We thank the reviewer (W1fb) for appreciating our experiments on real-world data and our insights on incorporating defocus cues with pretrained diffusion models, and have addressed their questions below.

Major Weakness 1: Concerns about limited applicability.
1a: Restricted depth range, dynamic scenes.
We agree with W1fb that our method is best applicable to scenes with a restricted depth range, which is predominantly true for indoor scenes, but also possible for outdoor scenes (e.g., STAIRS in our dataset). We will clarify this in L:289.

While dynamic scenes are not the focus of our work, we acknowledge that handling dynamic scenes in our framework would be valuable. Our current assumption of static scenes aligns with standard practice in monocular depth estimation, as followed by prior works [A, B, E, D]. Note that a two-exposure imaging method does not preclude application to scenes containing any motion. In principle, short exposures can be used with rapid actuation of a camera aperture to capture our two input images with a delay that is on the order of 2x slower than a single image alone. This would only require the motion to be reduced by 2x vs. a single image method set to capture with minimal motion blur. Admittedly, some inter-exposure alignment would likely be needed, but we leave that to future work.

1b: Variable aperture cameras are also readily available on commodity devices.
Our method only requires a single variable aperture camera, which is readily available in the form of handheld commodity devices like Canon DSLRs (as used in our work) and even in some smartphones such as the Samsung Galaxy S9, S10, and upcoming S26. While fixed aperture cameras are more common, variable aperture is still widely accessible and avoids the need for custom hardware like phase or aperture masks used in prior methods [D, F].

Major Weakness 2: 3D metrics and availability of camera Intrinsics.
2a: Additional 3D metrics.
We agree that 3D point cloud metrics such as Chamfer Distance and F1 score (derived from Chamfer Distance) [A, B] are also representative metrics for 3D metric depth; we have added them to Table 1 below for our method and the competitive baselines. We will add these numbers in a revision. Our reported metrics (L:234 in main paper) are also well accepted in the literature [B, E] for depth accuracy – $\delta_1, \delta_2, \delta_3, RMSE, log10$ capture metric 3D depth accuracy.

Table 1: Results with 3D Metrics (CD: Chamfer Distance), (FA: Aggregated F1 score as used in UniDepth [A]) on our dataset. On the newly added 3D metrics CD (chamfer distance) and FA (aggregated F score), our method outperforms the existing baselines on our real dataset. This is consistent with the trends observed with the other metrics as well. $\uparrow, \downarrow$ indicate whether higher or lower is better, respectively.

2b: Availability of Camera Intrinsics.
Our capture setup involves capturing an in-focus and a blurred image from a real DSLR camera. The metadata in these images contains EXIF data (camera intrinsics) by default. As the reviewer noted, this is standard practice, with most methods similarly relying on EXIF metadata [G, H]. Hence, we do not focus on the case of unknown camera intrinsics in our work, as that information is always going to be available by design in our capture setup.

Major Weakness 3: Processing the Intel RealSense depthmap to prepare our dataset.
The input images in our setup are captured with an RGB camera, and the ground truth depth map is captured from a RealSense camera (see Fig. 2c in the paper for the capture setup). These two cameras are physically close but not co-located; as a result, the depth from the RealSense does not geometrically correspond to the RGB camera (due to geometric misalignment).

To handle this misalignment, the cameras need to be calibrated using standard camera calibration. This is a standard data pre-processing step [C, D], and is also done by real public RGBD datasets like NYUv2 [C], which allows them to provide geometrically aligned RGB images and depth maps. This allows a fair comparison between the predicted depth from the RGB camera and the GT depth in the same coordinate system. In L:239-240 of our work, this is what we refer to as alignment.

At a high level, first, the pose between the RGB and RealSense cameras is estimated via standard camera calibration (section C of the supplement, L:241 main paper), similar to [D]. The RGB depth map is warped to the GT depth camera’s frame using the estimated pose, enabling comparison between the predicted and GT depth in a shared geometric space.

This alignment is crucial for a principled and accurate comparison between GT depth and predicted depth, and is the only alignment we use in our work – both for our method and the baselines. Therefore, all reported metrics in the rebuttal and Table 1 of the main paper are computed directly on the raw metric depth outputs.

Minor Weakness 1: Exact Inference Algorithm, Learning Rates.
We have included these details in section A of the supplement, which we reference in L:214 of the paper. We used the Adam optimizer in PyTorch for our optimization, and section A of the supplement contains learning rates and other optimization details.

Minor Weakness 2: Dependence of output depth on scale and shift parameters ($\alpha$/$\beta$).
The scale and shift parameters $\alpha,\beta$ are parameterized such that $\alpha\in[0, s_{max}]$ and $\beta\in[0,s_{min}]$, where $s_{min}, s_{max}$ denote the maximum possible values for the lower and upper bounds of the scene depths. We assume $s_{min}, s_{max}$ to be known since we operate in a restricted depth range. (L 184:186, main paper). To ensure both boundedness, and differentiability during optimization, $\alpha, \beta$ are parameterized as (L:184, L:185 main paper) –

\alpha = s_{\text{max}} \cdot \sigma(a)~~~\textrm{and}~~~ \beta = s_{\text{min}} \cdot \sigma(b),$$ where $a,b$ are the learnable unbounded variables, and $\sigma(\cdot)$ denotes the sigmoid activation. For the performance ablation plot with $\alpha,\beta$ values (Fig. 5), we mention in the figure caption that the $\alpha,\beta$ values are normalized to 0-1 ($\alpha \rightarrow \frac{\alpha}{s_{max}}, \beta \rightarrow \frac{\beta}{s_{min}}$) for plotting. We will update the figure caption to clarify this. [A] UniDepth: Universal Monocular Metric Depth Estimation. CVPR 2024. [B] Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Image: ICCV 2024. [C] NYU Depth Dataset V2: ECCV 2012. [D] Passive snapshot coded aperture dual-pixel RGB-D imaging: CVPR 2024. [E] Depth Pro: Sharp Monocular Metric Depth in Less Than a Second: ICLR 2025. [F] Learning Phase Masks for Passive Single View Depth Estimation: ICCP 2019. [G] Depth Estimation via Reconstructing Defocus Image: CVPR 2023. [H] Aperture Supervision for Monocular Depth Estimation: CVPR 2018.