Overview

We thank you for your feedback. We appreciate that you recognized our work as "clearly described" and an "interesting and structured approach." Regarding the concerns raised, we found the feedback on our ablation studies and computational cost analysis invaluable for strengthening the paper. However, on the core assessment of our novelty/transparency, we must respectfully beg to differ. Due to space constraints, some tables below are included in other reviewers' rebuttals, and the ablation table only highlights essential metrics.

On Novelty: Differentiating from Prior Art

Weaknesses 1. Questions 1.

Jasmine is the first SD-based SSMDE framework, and our contribution can be located at Lines 65-71. For the MIR:

1. We have made numerous modifications: within each training batch, the same SD model alternately reconstructs synthesized or real images, as well as predicts depth maps through a switcher; the reconstructed images are supervised using a photometric loss rather than the traditional latent MSE loss.
2. Our motivations also differ fundamentally from those of SD: Self-supervision tends to erode the prior knowledge embedded in SD, making it unsuitable for direct application as in Marigold. Under this motivation, we introduced MIR to preserve this prior , with important modifications for SSMDE.
3. We conducted extensive ablation experiments on the use of auxiliary images, which yielded valuable insights. It is worth noting that MIR is only one aspect of our overall contribution.

For synthetic images usage, Marigold must use synthetic data to keep its latent space intact, as noise-free, dense depth GT is extremely rare. In contrast, our Jasmine model employs real KITTI images for its depth training, and its auxiliary images can be flexibly selected from either real or synthetic sources. This is because Jasmine's self-supervised process does not require high-quality GT, but supervised Marigold does. Therefore, we actually do NOT mirror Marigold, but a pioneering work that effectively integrates diffusion priors into SSMDE.

In addition to MIR, the improvement of SSG is also the first work in self-supervision to analyze the difference between the two types of depth de-normalization. We provide a detailed analysis of these differences in Sec B, which, to our knowledge, is unprecedented in the literature. The SST module uses cross-attention to learn the scale/shift parameter from the last iteration prediction, as referred to the reviewer SPog rebuttal for more details.

Finally, we note that the reviewer arKR highlighted our work as 'the first to integrate diffusion priors into SSMDE' and as 'establishing a new paradigm for the task.' Reviewer SPog remarked that our approach 'addresses a longstanding challenge in self-enhanced depth estimation,' and reviewer fCDa stated that we 'introduce a novel approach to harness the SD visual priors for SSMDE.' Collectively, these comments affirm that our work is innovative and makes meaningful contributions to the field.

Missing Ablation Studies

Weaknesses 2. Questions 2.

We would like to clarify that $L_{GDS}$, $L_{e}$, and $L_{sky}$ are NOT the core contributions of our work, which is why they are presented in the appendix. SSMDE is a well-established field, and many of its challenges have already been addressed by prior research. The loss functions mentioned in the appendix are mature, widely adopted solutions, and our intention is NOT to reinvent these established methods.

Second, we have already visualized the ablation results for $L_{sky}$ and $L_{e}$ in Appendix Fig. 7 (b-2, b-3, d-3, d-4). To further address the reviewer's concerns, we have also provided ablation results in Table G. Without these losses, Jasmine remains at the SoTA level.

Table G. Ablation study on $L_{GDS}$,$L_{sky}$ and $L_{e}$, evaluation rules are same with Table 1 in the paper.

Finally, another experiment is designed to compare the performance of our improved SD priors against traditional frameworks, both building upon these losses. In fact, in our paper, Table 1 (Jaeho et al.*) shows the performance of the MonoViT framework with these mature Losses. Our method consistently outperforms Jaeho et al.*. Here, we further include a zero-shot performance comparison in Table H, which confirms that incorporating the improved SD prior significantly enhances the model's generalization. These results show that the performance depends mainly on our new paradigm, rather than on these mature Losses.

Table H.

Training complexity

Weaknesses 3.

Due to the rebuttal policy, we regret that we are forbidden to release our full code at this time. However, to enhance transparency, we have provided detailed pseudocode in our rebuttal and commit to releasing our code upon paper acceptance.

SD_UNet, PoseNet, SSG, VAE = init_models()
optimizer = init_optim(...)
VAE.eval()
for inputs in dataloader:
 #MIR
 s=[[0,1],[1,0]]#switcher
 z^I_t, z^I_m, z^n = VAE.encode(inputs['I_t'],inputs['I_m'],noise)
 D_ssi = VAE.decode(SD_UNet([z^I_t,z^n], t=999,s=s[0]))
 I_rec= VAE.decode(SD_UNet([z^I_m,z^n], t=999,s=s[1]))
 #SSG
 D_list = SSG(D_ssi, z^I_t) #D_list=[D_ssi,D_1,D_si]
 #SSMDE Pose
 pose = PoseNet(inputs['I_t'], inputs['I_(t')'])
 #Loss
 loss = compute_loss(inputs, D_list, pose,I_rec)
 #Optim
 loss.backward(), optimizer.step()
def compute_loss(inputs,D_list, pose,I_rec):
 D_si,D_ssi=D_list['D_si'],D_list['D_ssi']
 reproj_img = reproject(inputs['I_(t')'],D_si, pose)
 L_ph = ph_loss(reproj_img, inputs['I_t'])
 L_s = ph_loss(I_rec, inputs['I_m'])
 L_tc= tc_loss(D_list,inputs['D_tc'],type='berhu')#Eq 11
 L_a= compute_L_a(inputs,D_si,D_ssi)
 loss =L_ph + L_s + L_tc+L_a*0.008
 return loss
def compute_L_a(inputs,D_si,D_ssi):
 L_GDS = gds_loss(inputs['I_t'],inputs['seg'],D_si)
 L_SKY = sky_loss(D_ssi, inputs['sky_mask']) 
 L_e = edge_loss(D_si,D_ssi)
 return L_GDS+L_SKY+L_e

Our training process is fully end-to-end. While multiple network modules are involved, only 3 core components require optimization: the U-Net of Stable Diffusion, PoseNet, and our proposed SSG. All modules are jointly updated using a single optimizer optimizer.step(), as illustrated in the provided pseudocode. Although there are multiple loss functions, they are all based on well-established and open-source prior works in the field(`ph_loss-> Monodepth2, sgd_loss->From Ground to Objects). This design choice ensures that our method remains conceptually straightforward to implement. With the provided pseudocode and clear references to existing methods, we believe our approach is logically transparent.

Computational costs and actual deployment

Weaknesses 4. Question 5.

We acknowledge that SD-based methods introduce higher computational overhead, inference latency and affect the fairness of comparisons with traditional CNN-based SSMDE. But we argue this has a limited impact on our contribution.

First, we report the computational requirements and inference latency in Table B (refer to rebuttal for reviewer arKR), comparing Jasmine with both self-supervised and SD-based models.

Second, we wish to position our work in current MDE research trends. Recently, a series of leading works represented by Marigold (CVPR 2024 Best Paper Candidate), Lotus(ICLR 2025) and DepthAnything (CVPR/NIPS 2024) have established a research paradigm within the community: there is a willingness to accept a certain level of computational cost at this stage in exchange for the unprecedented generalization and quality afforded by Large Vision Models(LVMs). Our work shares the same philosophy as these studies. Therefore, the main contribution of our paper is not to pursue the ultimate inference speed, but to explore and successfully introduce the powerful prior of SD into an SSMDE framework for the first time, aiming to solve the long-standing problems of weak generalization and poor detail recovery in this field.

Third, for Application Potential, the rapid adoption of Marigold by 30+ downstream tasks demonstrates that even computationally intensive models can find widespread application if their performance is significant enough. Similarly, we believe Jasmine, which does not require depth GT, has strong potential for real-world applications where annotated data is limited but leveraging LVMs is desirable.

Others

Question 3.

Tables A and E (see rebuttal for reviewers arKR and fCDa) provide a detailed performance comparison at 640×192 and 1216×352 resolutions. Results on the Cityscapes dataset are already reported in Table 2 of the main paper. Since we have performed zero-shot evaluations on several street-view datasets, we believe that additional testing on a similar Waymo dataset (less used in this community) would offer limited insights. Instead, to better address your generalization concerns, we conducted evaluations on indoor datasets, as presented in Table C (see rebuttal for reviewers arKR).

Question 4.

Table J reports the results when using monodepth2 as the teacher model, as well as the performance of our method without any teacher model. Due to time constraints, we are unable to perform comprehensive training with additional teacher models and choose the monodepth2 (a relatively worse one that puts our model at a disadvantage). We find that the teacher model has a trivial impact on final results, as it serves to stabilize the initial training phase. However, omitting the teacher model leads to suboptimal performance.

Table J

Dear Reviewer s3Km,

Thank you very much for taking the time to read our rebuttal and for your acknowledgement. We greatly appreciate your detailed review and valuable feedback.

We hope that our response has effectively addressed your previous concerns regarding the novelty of our work, the ablation studies for key modules ($L_{GDS}$ and $L_{SKY}$), the complexity of the training procedure, and the computational overhead.

We sincerely look forward to your further feedback. If you feel that our rebuttal has sufficiently resolved your concerns, we would kindly ask you to consider re-evaluating our paper's rating in your final assessment.

Of course, we highly value any opportunities to address any potential remaining concerns before the discussion closes, which might be helpful for improving the rating of this submission. Please do not hesitate to comment on any further concerns. Your feedback is extremely valuable!

Thank you once again for your time and professional engagement.

Sincerely,

The Authors of Paper #14390

Overview

We sincerely thank you for your time and for providing this critical feedback. We apologize that some of the preliminaries and figure presentation were unclear. Despite these major issues with clarity, we are encouraged that the reviewer still recognized that our paper addresses a long-standing challenge in this area and shows strong experimental results. Following the reviewer's guidance, we try to address every concern as follows.

Weaknesses: Writing Logic & Figure Presentation

In the paper, we first present our motivation: SSMDE suffers from poor generalization and limited prediction detail, whereas SD priors offer a promising solution. However, when introducing SD priors, we encountered 2 challenges: (1) self-supervised signals can disrupt SD priors, and (2) there are distributional differences between self-supervised depth and SD outputs (SI-SSI). We addressed them with MIR and SSG, ultimately enabling the effective integration of SD priors, which led to significant improvements in performance, generalization, and prediction detail. In the method section, after introducing the preliminaries on traditional SSMDE and SD protocol, we provide a detailed explanation of MIR and SSG. For MIR, we first discuss the challenges (1) principle, then describe the implementation details, and further expand MIR with diverse auxiliary images usage. For SSG, we also begin with the challenges (2) principle, then elaborate on the design details. We hope this clarifies the overall writing logical of our paper. Questions regarding the figures will be addressed in 1️⃣,2️⃣,3️⃣ below.

How self-supervision introduce to SD

Weaknesses: Unclear Technical Motivation Questions 1. Questions 3.

First, it is important to clarify that the self-supervision used here is distinct from the unsupervised learning of foundational vision models like DINOv2. In our context, self-supervision means the training process leverages the temporal and spatial relationships between sequential images to train the depth estimator, without GT depth.

Intuitively, integrating SD with self-supervision fundamentally changes the source of supervision for the SD output. In the standard single-step denoising paradigm, the original SD model directly predicts depth, and during training, GT depth is used to supervise the U-Net’s output via an MSE loss. In our approach, however, the supervisory signal for the SD output comes from a reprojection loss. Specifically, we predict the relative pose between adjacent frames and use SD output + pose to reproject the adjacent frames onto the current input image. If the SD-predicted depth is accurate, the reprojected/reconstructed image should closely match the original input image. This photometric consistency serves as the self-supervised signal to guide depth learning.

For Question 3, in supervised SD-based depth estimation (e.g., Marigold), $y$ indeed represents the GT depth map. However, in our self-supervised framework, $y$ should be interpreted as the target output that the model aims to generate, rather than a pre-existing supervised signal. Specifically, as described in Sec 3.1, in our single-step denoising strategy, the U-Net $f_\theta^z$ takes pure noise $\mathbf{n}$ and an image latent variable $z^I$ as input, and the model's task is to learn a mapping that transforms $\mathbf{n}$ into a meaningful depth map latent $z_0^y$. Thus, the model never sees and never requires the 'correct answer' $z^y$.

How self-supervision compromise SD prior

Questions 4. 1️⃣Specific to the figures 3 (a)

For clarity, we first revisit the core definition of disparity: when capturing 2 images of the same scene from different camera positions, the same point will appear at different pixel coordinates in each image. This difference is known as disparity. Disparity and depth are inversely proportional and correspond one-to-one.

From this perspective, the loss function in Eq. 2 is designed to find the most similar point in the source view $I_{t'}$ for each point in the target view $I_t$. From the coordinate difference between these points $I_{t'\rightarrow t}$, we obtain the depth supervision (Eq.1). However, if the corresponding point in $I_t$ is occluded in $I_{t'}$, the optimization process is forced to select the "least bad" alternative, resulting in an incorrect match and, consequently, erroneous depth estimation.

Taking Fig 3(a) as an example, the scene consists of an infinite background (depth=$\infty$), an orange rectangle (20m), and a light blue tree (10m). Here, the camera only shifts horizontally between the target and source views. For point q (tree), the correct disparity is 10 pixels (10m). For point p,r (rectangle), the correct disparity should be 5 pixels (20m).

Due to camera movement, the tree in the source view occludes the correct matching point of p. To minimize photometric loss, the algorithm searches for the most similar region nearby and once again finds the orange area at p', which is 10 pixels (10m).

This error causes the expected depth edge on the right side of point p to disappear (the right side's depth is all 10m, same reason as p), resulting in a blurred boundary in the depth map. When such a depth map is used as a supervisory signal to guide the SD model, it effectively introduces "noisy" data. This forces the SD model to learn and reproduce these incorrect and blurred boundaries, thereby undermining its strong prior knowledge of clear object boundaries.

Others

2️⃣Specific to Fig 4(a)

Since we cannot include the revised figure in this rebuttal, we will provide a detailed textual explanation of the process below.

For DepthHead We want to clarify that the relationship of $(D_k,h^k)$ is same to $(y,z^y)$ in the SD model. The SSG network does not directly process the original image, but the encoded features. The difference is that SD uses VAE for encoding and decoding, while GRU only uses one convolutional layer, so we ignored this structure in the figure. We thank you for pointing this out and will include it in the final version.

Arrows from nowhere In fact, Fig 4(a) corresponds to the gray rectangle shown in Fig. 2, standing for an iteration within two consecutive ones. Thus, the origins and destinations of 'nowhere' arrows indicate connections to the preceding and following SSG iterations, respectively.

$z^I$ integration and $D_k$ update To better clarify these questions and Fig. 4(a), we detail the first iteration (k=0) below:

1. We first encode the SD output $D_0(D_{SSI})$ to obtain the hidden state $h^0$.
2. We use learnable $Q_{SC}/Q_{SH}$(as query) to interact with the $h^0$ (as key/value) with cross-attention and generate scale $s_c$ and shift $s_h$.
3. The input $x^0$ is formed by concatenating the image features $z^I$ and the updated depth $s_c*D_0+s_h$.
4. The GRU takes $h^0$ and $x^0$ as input and outputs next hidden state $h^1$. After decoding, depth residual $D_{\delta}$ is obtained. Finally, we use Eq 10 to calculate the refined depth map $D_1$.

3️⃣Specific to Fig 3(g)

We agree that Figure 3(g) should be more self-contained, and we will revise its caption in the final version, like:

In sub-figure (g), the x-axis ($\lambda$) is the proportion of the KITTI dataset used in training(e.g., 0.3 indicates a KITTI: Hypersim ratio of 3:7 in the MIR training data), the y-axes are the metrics of the performance, including AbsRel (Absolute Relative Error, lower is better) and A1 (Accuracy, higher is better), definitions detailed in Sec E. We show the performance variations under different $\lambda$ settings for photometric supervision (Eq. 5, better) and latent supervision (Eq. 4, worse).

Questions 2.

The task switcher itself cannot solve the SD prior perturbation problem, but the surrogate task (MIR) enables can. As we explained in "How self-supervision compromises SD priors," self-supervision introduces "noisy" supervision signals. The role of the switcher is to introduce "clean" ones. Specifically, the training target of the image reconstruction task is the input image itself, which contains the most perfect, sharpest, and structurally correct visual details. Therefore, it guides the model to focus on and preserve realistic edges, textures, and structures. During training, the "bad gradients" from the self-supervised loss and the "good gradients" from the image reconstruction task are mixed together. These "good gradients" act as a normalization or an anchor, constantly pulling the model's parameters back to a "correct" region capable of generating sharp, realistic details, effectively counteracting the negative impact of the "bad gradients" on the model's priors.

Other Problems 1.

This claim originates from the foundation work on self-supervised depth estimation by Godard et al. (2017), titled "Unsupervised Monocular Depth Estimation with Left-Right Consistency" (CVPR 2017).

other problems 2.

At L155-L160, the paper states that we utilize the synthetic Hypersim dataset as auxiliary image. However, using synthetic data does NOT mean they must be used. As demonstrated in our experiments (Table 3, (d), (13), L325-l327), using high-quality real images can achieve similarly effective results. So they are not contradictory.

other problems 3.

We will fix it in the final version.

Dear Reviewer SPog,

Thank you for your prompt reply and for confirming that our explanations have addressed your concerns. We truly appreciate the time and effort you have dedicated throughout this entire review process. Your insightful questions and constructive feedback have been invaluable in helping us strengthen the paper.

We look forward to your final recommendation after your careful re-evaluation. Should any further questions arise during this time, we would welcome the opportunity to clarify them.

Thank you once again for the very productive discussion.

Best Regards,

The Authors of Paper #14390

Overview

We sincerely thank you for your thorough and highly constructive feedback on our manuscript. We are greatly encouraged by your positive assessment and are pleased that you found our work to be a "novel approach" (S01) with appealing motivation and well-justified analysis(S02), supported by "strong experimental results" (S03).

We especially appreciate the valuable and actionable suggestions for improvement. In our revision, we have worked to address all the weaknesses and questions raised, including:

W01, Q01

Fair comparison with SD-based methods under specific evaluation resolution.

First, the reviewer noted that there is a performance gap between the results reported in our paper and those in the original work. It is primarily due to differences in the GT used for evaluation, NOT the input resolution. We will address this issue in detail in the W03 analysis.

Second, we would like to clarify that for SD-based supervised models—particularly those trained on the vKITTI dataset (cropped to $1216 \times 352$) and the Hypersim dataset (resized to $480 \times 640$), such as Marigold and Lotus—the impact of input resolution on depth estimation performance is relatively limited. In contrast, self-supervised models are relatively sensitive to resolution changes due to the strong dependence of photometric reprojection losses on image details.

As illustrated in Table D, the performance of supervised models remains close across resolutions of $1024 \times 320$ and $1216 \times 352$, whereas self-supervised models experience a significant drop in accuracy at higher resolutions. (note: the $1216 \times 352$ resolution follows the Marigold, images cropped from the original ones.)

Table D. Performance comparison at different resolutions. Marigold/Lotus are SD-based supervised models, and MonoViFi/MonoViT are self-supervised models.

Despite the inherent disadvantages of self-supervised training under these conditions, we leveraged the strengths of SD priors and evaluated our model at $1216 \times 352$ (while it was trained at $1024 \times 320$). As shown in Table E, our approach still outperforms SD-based supervised models, highlighting the effectiveness of our method.

Table E. Performance comparison under the Lotus $1216 \times 352$ evaluation protocols on the KITTI benchmark.

W02, Q02

New experiments and analysis on indoor datasets.

As the first work to introduce SD priors into self-supervised depth estimation, our experiments were initially focused on the most established self-supervised setting—the street view dataset. The primary challenge in indoor self-supervised depth estimation arises from frequent and significant rotational motion (whereas KITTI is predominantly translational), which necessitates improvements to the pose network. However, this aspect is beyond the scope of our core contribution, as our work does not modify the pose estimation network.

Importantly, we argue that evaluating the benefit of strong SD priors for indoor SSMDE tasks is less about training on indoor data and more about assessing the generalization capability on it. To this end, we performed a zero-shot evaluation of several self-supervised models on indoor datasets, as presented in Table C. Despite the substantial distribution shift between the training and testing data, Jasmine continues to exhibit strong performance, further underscoring the effectiveness and generalizability of our approach.

Table C. Zero-shot evaluation on the NYUv2 dataset, test on the official test split.

W03, Q03

Evaluation and Comparison at the KITTI improved GT

Thank you for the additional reminder! As shown in Table D, we report a performance comparison between Jasmine, SD-based methods, and self-supervised models on the KITTI improved GT benchmark, and we have consistently achieved the SoTA level. The inconsistency in metrics mentioned in W01 has also been resolved here.

Table F. Quantitative comparison between Jasmine and SD-based/self-supervised methods on the KITTI dataset using improved GT.

W04

Experiment on SD prior perturbation.

The motivation for protecting SD priors stems from the observation that self-supervised training can disrupt the integrity of these priors. The supervised approaches, like Marigold, can successfully leverage synthetic data to keep the SD latent space intact (Original sentence from Marigold paper). However, even models like Marigold are NOT directly applicable to the original image generation task, as their adaptation for depth estimation does not preserve the generative pathway required for RGB pixel synthesis. Instead, the inheritance of SD priors in Marigold is reflected through improved prediction details and enhanced generalization ability. And both of these points are embodied in and ablated in our paper, undoubtedly proving the protection of SD priors.

However, we agree that a more direct evaluation of SD prior perturbation — through image reconstruction metrics — would provide additional insights, and it should be done in image generation fine-tuning. We consider this a valuable direction for future work.

End

We believe these additions have significantly strengthened the paper. The above results will be included in the final manuscript.

Overall

We sincerely thank you for your time and effort in providing such a constructive and detailed review. We are greatly encouraged that you found our contributions meaningful, our methodology sound, and that our work establishes a new paradigm for self-supervised depth estimation.

Your suggestions regarding the fairness of benchmark comparisons, the inclusion of runtime analysis, and further evaluation of generalization on more diverse domains like indoor scenes are highly insightful. We fully agree that addressing these points will significantly strengthen our paper's clarity and completeness. Below, we provide a point-by-point response to your comments and will incorporate all the valuable changes into our final manuscript.

On Fairness of Benchmarking and Additional Experiments (W1, W2)

We have carefully checked Table 1 and ensured that all of the baseline results are reported at the same resolution of 1024x320.

We have included the performance evaluation of our retrained model at a resolution of 640 × 192 in Table A, alongside comparisons with other latest self-supervised models. The results continue to demonstrate the SoTA performance of our method.

Table A. Quantitative results on the KITTI dataset with 640x192 resolution. The best results are marked in bold.

On Inference Latency and Runtime Performance (W3)

We present the inference latency and computational complexity of the Jasmine model in Table B, alongside comparisons with both SD-based and self-supervised methods. Our results indicate that the inclusion of SSG has minimal impact on inference delay, as Jasmine and Lotus exhibit comparable inference time. Although incorporating SD does lead to a notable increase in computational complexity relative to other self-supervised approaches, Jasmine strikes a trade-off between performance and computational cost.

Table B. Computational efficiency comparison. MACs and Runtime are measured under a single image with 1024x320 resolution on one RTX 4090 card. MACs is the number of core multiply-accumulate operations and MACs≈FLOPs/2.

On Generalization to More Diverse Domains (Q1)

As the first work to introduce SD priors into self-supervised depth estimation, in the paper, our experiments were initially focused on the most established self-supervised setting—the street view dataset. But to address the reviewer’s concerns, we performed a zero-shot evaluation of several self-supervised models on indoor datasets, as presented in Table C. Despite the substantial distribution shift between the training and testing data, Jasmine continues to exhibit strong performance, further underscoring the effectiveness and generalizability of our approach.

Table C. Zero-shot evaluation on the NYUv2 dataset, test on the official test split.