Delving into Cascaded Instability: A Lipschitz Continuity View on Image Restoration and Object Detection Synergy

We sincerely appreciate your insightful comments. In response, we clarify the low-light setting and evaluate robustness to noise (Q1), clarify the role of regularization and gradient dynamics (Q2), revise visualizations and expressions for improved clarity (Q3–Q4), and compare with pretrained enhancement methods (Q5). Detailed responses are provided below.

Q1: The low-light images are synthesized by the gamma correction. However, in real scenes, low-light images are typically accompanied by noise.

A1: Thanks for the insightful comment. We follow standard protocols [a, b, c] using gamma correction to ensure consistency with existing benchmarks.

We agree that real-world low-light images often contain noise. To evaluate robustness under such conditions, we introduce Poissonian-Gaussian noise during testing. As shown in the table below, our method maintains strong performance, indicating good generalization to more realistic low-light scenarios.

[a] Image-Adaptive YOLO for Object Detection in Adverse Weather Conditions, AAAI 2022

[b] Rethinking Image Restoration for Object Detection, NeurIPS 2022

[c] Unsupervised Variational Translator for Bridging Image Restoration and High-Level Vision Tasks, ECCV 2024

Q2-1: It seems that the detection loss can also suppress the Lipschitz constant, as its gradient is generally aligned with the Jacobian norm. Suppressing the Lipschitz constant doesn't seem to be solely driven by the restoration loss.

A2-1: The effect of detection loss on Lipschitz suppression is inconsistent. While it can occasionally reduce the Lipschitz constant when its gradient aligns with the Jacobian norm (i.e., $\xi(t) < 0$), object detection involves non-smooth components such as bounding box matching, which often introduce sharp gradient changes. These can result in $\xi(t) > 0$, increasing the Lipschitz constant and destabilizing training.

In contrast, the restoration loss plays a primary and consistent role in suppressing the Lipschitz constant by providing smooth, stable gradients. As shown in Figure 5, its inclusion lowers and stabilizes both Jacobian and gradient norms.

We will add this discussion to Line 206 to clarify the effect of the detection loss.

Q2-2: I am concerning that the penalty of $\lVert \nabla_{\mathbf{\theta}} f_{\mathbf{\theta}}(\boldsymbol{x}) \rVert$ is enough to harmonize the instability.

A2-2: We clarify that $\lVert \nabla_{\theta} f_{\theta}(\boldsymbol{x}) \rVert$ enforces Lipschitz regularization by promoting smoothness in the parameter space. When applied alone, it can improve performance over the baseline. For example, YOLOv10 improves from 46.0 to 47.2 mAP on RTTS.

Furthermore, when combined with input-space regularization via the restoration loss $\mathcal{L}_{\text{res}}$, performance further increases to 49.2 mAP: a further gain of 2.0%. This indicates the complementarity of the two regularizers: gradient norm stabilizes parameters, while restoration loss smooths input supervision.

Figure 5 also supports this, LR-YOLOv8 (ours) achieves lower Jacobian norms than LR-YOLOv8* (using only $\mathcal{L}_{\text{res}}$), leading to more stable training.

Q3: Figure 2 (a) does not clearly explain why the purple dot cannot be seen in 'ours'.

A3: The points corresponding to $x$ (purple) and $x + \Delta x$ (yellow) are nearly overlapping, appearing as dark yellow. To improve clarity, we will update the Figure 2 using distinct marker shapes to better distinguish the two points.

Q4: In Row 203, what does 'small' mean?

A4: Thanks for your meticulous comment. We have fixed this typo:

$\lVert \nabla\_{\mathbf{\theta}\_b} \mathcal{L}\_{\mathrm{res}}(g\_{\mathbf{\theta}\_b, \mathbf{\theta}\_r}) \rVert \le G$ for small $G < \lVert \nabla\_{\mathbf{\theta}\_b} \mathcal{L}\_{\mathrm{det}}(f\_{\mathbf{\theta}\_b, \mathbf{\theta}\_d}) \rVert$

We refer to the gradient norm of the restoration loss, $\lVert \nabla\_{\mathbf{\theta}\_b} \mathcal{L}\_{\mathrm{res}}({g}\_{\mathbf{\theta}\_b, \mathbf{\theta}\_r}) \rVert$, being smaller than that of the detection loss, $\lVert \nabla\_{\mathbf{\theta}\_b} \mathcal{L}\_{\mathrm{det}}(f\_{\mathbf{\theta}\_b, \mathbf{\theta}\_d}) \rVert$.

Q5: Using the pretrained image enhancement methods to yield clear images for training and testing.

A5: Our method achieves consistently better performance than the pretrained image enhancement methods. As suggested, we include the Transformer-based pretrained method OneRestore [a] and the Diffusion-based pretrained method PromptFix [b] for evaluation. Please see the results in the table below.

[a] OneRestore: A Universal Restoration Framework for Composite Degradation, ECCV 2024

[b] PromptFix: You Prompt and We Fix the Photo, NeurIPS 2024

Q6: No potential negative societal impact.

A6: We discuss potential societal impacts in Appendix G: Broader Impacts. Specifically:

1. Improved detection capabilities in low-visibility conditions may enhance public safety but also raise privacy concerns in surveillance applications.

2. The proposed method’s effectiveness in challenging environments may attract interest from security-related applications, which, while unintended, highlights the importance of responsible deployment.

Dear Reviewer KmkB,

We appreciate your valuable feedback, which has greatly contributed to improving our work.

Best regards,

Authors

Thanks to the author's detailed answer, which addressed my concerns, I chose to raise my score to 5: Accept.

Thank you very much for the insightful comments. In response, we extend our evaluation to diverse degradation types (Q1), validate our framework on additional detection architectures (Q2), and discuss its relevance to camouflaged object detection (Q3). Detailed responses are provided below.

Q1: Generalization to other adverse conditions (rain, snow, mixed weather)

A1: Our method naturally extends to a range of adverse conditions. To validate this, we extended our evaluation to additional adverse conditions, including motion blur, rain, snow, and mixed degradations (e.g., haze + rain).

Across all settings, our approach consistently outperforms existing baselines. Please refer to the results provided in the tables below for detailed comparisons.

We construct new training and validation sets under each degradation setting. All models are retrained on the respective training sets and evaluated on the corresponding validation sets.

The above results will be included in Section 5.3.

Q2: Extend to diverse detection architectures.

A2: Our LROD framework is both compatible with and effective across different detection paradigms. During rebuttal, we applied it to the shared backbone of both the transformer-based RT-DETR [a] and the two-stage Faster R-CNN [b]. In both cases, LROD consistently remains effective.

Please refer to the detailed results presented in the tables below.

All models are trained on VOC_Haze_Train and evaluated on both the synthetic dataset VOC_Haze_Val and the real-world dataset RTTS.

• 1. Transformer-based detector RT-DETR:

• 2. Two-stage detector Faster R-CNN Results:

We will include the above experiments in Section 5.3.

[a] DETRs Beat YOLOs on Real-time Object Detection, CVPR 2024

[b] Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks, NeurIPS 2015

Q3: If some interesting findings also in Camouflaged Object Detection [1] ... high-resolution is beneficial for object detection [2]

A3: Thanks for raising this discussion. We believe that our analysis on the Lipschitz continuity disparities between restoration and detection networks extends naturally to camouflaged object detection (COD), where similar challenges in boundary stability arise.

In particular, COD involves detecting objects with ambiguous, low-contrast boundaries, posing challenges similar to those in cascaded systems under adverse conditions. Our findings suggest that instability around object boundaries in COD may stem from the non-smooth behavior of detection networks. The high-resolution preservation enabled by restoration modules can help address this, as fine edge and texture details are essential for accurate localization. This observation is consistent with [2], which highlights the importance of high-resolution features in improving COD performance.

We will incorporate the above discussion and include the related works [1, 2] in Lines 320-325.

[1] Camouflaged Object Detection

[2] High-resolution Iterative Feedback Network for Camouflaged Object Detection

Dear Reviewer AYoA,

Thank you for your positive feedback and support. We appreciate your suggestions for improving our paper.

Kind regards,

The Authors

Thank you very much for the constructive comments. We validate the effectiveness of our framework across different detector paradigms (Q1-1) and degradation scenarios (Q3), and conduct ablation studies on regularization strategies (Q2-1) and architectural designs (Q2-2). Below, we provide our point-by-point responses.

Q1-1: Generalizability to other detection paradigms (e.g., DETR, RT-DETR) or two-stage detectors like Faster R-CNN?

A1-1: Our LROD framework is both compatible with and effective across other detection paradigms. Following the suggestion, we applied it to the shared backbone of both the transformer-based RT-DETR and the two-stage Faster R-CNN. In both cases, LROD remains effective, as demonstrated by the results reported in the tables below.

All models are trained on VOC_Haze_Train and evaluated on both the synthetic dataset VOC_Haze_Val and the real-world dataset RTTS.

• 1. Transformer-based detector RT-DETR:

• 2. Two-stage detector Faster R-CNN Results:

We will include the above experiments in Section 5.3.

Q1-2: Any gradient flow or optimization issues with the models mentioned above?

A1-2: We observe that RT-DETR and Faster R-CNN exhibit similar gradient flow and optimization issues to those in YOLO. Both models suffer from sharp gradient transitions and unstable optimization behavior. Our proposed regularization facilitates smoother gradient propagation, and the overall training process is more stable.

This is evidenced by the upper bounds on the Lipschitz constant, as presented in the table below.

Q2-1: Ablation on alternative regularization strategies.

A2-1: During the rebuttal, we conducted an ablation study comparing our method with two alternative regularization strategies: Spectral Norm Regularization (SNR) [a] and the adversarial training method Projected Gradient Descent (PGD) [b].

Our approach consistently outperforms both SNR and PGD, as shown in the table below.

All models are trained on VOC_Haze_Train and evaluated on RTTS to assess out-of-domain generalization.

[a] Spectral Normalization for Generative Adversarial Networks, ICLR 2018

[b] Towards Deep Learning Models Resistant to Adversarial Attacks, ICLR 2018

We further discuss the advantages over the two regularization techniques as follows,

First, compared to SNR, which constrains weights globally, our method penalizes $\lVert \nabla_{\mathbf{\theta}} f_{\mathbf{\theta}}(x) \rVert$, reducing output sensitivity to parameter changes and enabling input-aware smoothness. Further, the restoration network promotes feature smoothness aligned with detection.

Second, compared to adversarial training (e.g., PGD), which requires generating perturbed inputs and increases training cost, our approach achieves implicit robustness without adversarial examples, resulting in more stable and efficient training. Moreover, while PGD often compromises performance on clean images, our method maintains accuracy on both clean and degraded inputs.

Q2-2: Ablation on deeper architectural variants.

A2-2: To explore the impact of backbone sharing depth, we conducted an ablation study using different configurations of shared encoder stages between the detection and restoration heads.

Our method shares the first three stages (F1–F3), aiming to strike a balance between feature smoothness and task-specific specialization. As shown in the table below, shallower sharing (F1–F2) limits regularization, while deeper sharing (F1–F4) introduces task interference, supporting our design choice of the first three stages (F1–F3).

All strategies are trained on VOC_Haze_Train and evaluated on RTTS to assess out-of-domain generalization.

The above two ablation studies will be included in Section 5.3.

Q3: Include other forms of degradation (e.g., motion blur, rain, snow, or a combination of these).

A3: As suggested, we evaluated our method on additional degradation types, including motion blur, rain, snow, and haze–rain mixtures.

Across all scenarios, our method consistently outperforms existing approaches, demonstrating its versatility and robustness under diverse adverse conditions, as shown in the tables below.

We construct new training and validation sets under each degradation setting. All models are retrained on the respective training sets and evaluated on the corresponding validation sets.

The above results will be included in Section 5.3.

Q4: Clarification of Restoration Loss Contribution During Inference: Is the restoration module used during inference, or is it only for training regularization? If it’s discarded after training, how does its supervision affect feature stability during inference?

A4: The restoration and detection networks share a common backbone. During inference, since the primary task is object detection, the restoration head can be removed without affecting detection performance. Furthermore, if both detection results and restored images are desired, the restoration head can be retained to generate the restored outputs.

We clarify that the primary role of restoration supervision is during training, which helps suppress the model’s sensitivity to input perturbations, thereby constraining the Lipschitz constant of the detection network (Figure 5) and enhancing both its robustness and feature stability at inference time (Table 4).

We sincerely appreciate your constructive comments. In response, we extend our evaluation to diverse degradations (Q1), provide analyses on Lipschitz estimation and robustness (Q2), conduct ablations on regularization and architecture (Q3), and validate generalizability across detectors (Q4). Our detailed point-by-point responses are provided below.

Q1: Extend to other degradation types (rain, motion blur, snow, etc.).

A1: Our method effectively extends to a variety of adverse conditions, including motion blur, rain, and snow. As shown in the table below, our method consistently outperforms existing approaches across all these scenarios.

We construct new training and validation sets under each degradation setting. All models are retrained on the respective training sets and evaluated on the corresponding validation sets.

The above results will be included in Section 5.3.

Q2-1: Empirical Jacobian norms estimate Lipschitz continuity, which may be noisy or dataset-specific ... Discussion on the Lipschitz constant estimation variance.

A2-1: We agree that empirical Jacobian norms can be sensitive to noise and data distribution. To reduce this effect, we follow prior work [a, b] and compute norms over many randomly sampled validation inputs, which helps stabilize estimates and limit dataset-specific bias.

[a] Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach, ICLR 2018

[b] Some Fundamental Aspects about Lipschitz Continuity of Neural Networks, ICLR 2024

Furthermore, we include an analysis of the estimation variance and report confidence intervals to assess robustness in the table below. To complement the empirical approach, we also compute certified upper bounds using the dataset-independent LipSDP method [c]. Together, these results validate the use of Jacobian norms as a stable proxy for estimating Lipschitz continuity while accounting for variance and distribution sensitivity.

[c] Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks, NeurIPS 2019

Q2-2: Analysis about adversarial robustness is preferred.

A2-2: As suggested, we analyze adversarial robustness by evaluating performance under varying levels of Gaussian noise.

Our method maintains higher absolute performance and exhibits the smallest drop from the clean setting, demonstrating stronger resilience to input perturbations. Results are provided in the tables below.

Q3-1: Ablation Study on the Backbone-Sharing Strategy.

A3-1: As suggested, we conducted an ablation study on backbone sharing and found that sharing the first three stages (F1–F3) yields the best balance between stability and task synergy. Shallower sharing (F1–F2) limits regularization, while deeper sharing (F1–F4) introduces interference. This supports our design choice of using the first three stages.

Please refer to the detailed results presented in the tables below.

All strategies are trained on VOC_Haze_Train and evaluated on RTTS to assess out-of-domain generalization.

Q3-2: Ablation Study on the Regularization Coefficients.

A3-2: As suggested, we conducted an ablation study on the regularization coefficients $\lambda$ (input space) and $\lambda_p$ (parameter space). The results show that our method consistently outperforms the baseline across a range of coefficient values, with only minor variation in performance.

All settings are trained on VOC_Haze_Train and evaluated on RTTS.

The above two ablation studies will be included in Section 5.3.

Q4: Include non-YOLO detectors as baselines.

A4: Our LROD framework is both compatible with and effective across other detection paradigms. During the rebuttal, we integrated it into the shared backbone of both the transformer-based RT-DETR [a] and the two-stage Faster R-CNN [b]. In both settings, it consistently enhances performance, as evidenced by the results in the tables below.

All models are trained on VOC_Haze_Train and evaluated on both the synthetic dataset VOC_Haze_Val and the real-world dataset RTTS.

• 1. Transformer-based detector RT-DETR:

• 2. Two-stage detector Faster R-CNN Results:

We will include the above experiments in Section 5.3.

[a] DETRs Beat YOLOs on Real-time Object Detection, CVPR 2024

[b] Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks, NeurIPS 2015

Q5: Clarity on how gradients flow jointly through detection and restoration heads.

A5: We clarify that the detection and restoration heads share the first three backbone stages, and their losses jointly update these parameters. The detection loss provides task-specific supervision, while the restoration loss contributes smooth gradients that regularize training. As shown in Figure 5, incorporating the restoration loss (LR-YOLOv8*) stabilizes both Jacobian and gradient norms, highlighting the benefit of joint gradient flow.

We will also include a gradient flow diagram in Figure 6 for clarity.

Q6: The authors have listed a few limitations of their work, however I don't see discussion about potential negative societal impact.

A6: We discuss potential societal impacts in Appendix G: Broader Impacts. Specifically:

1. Improved detection capabilities in low-visibility conditions may enhance public safety but also raise privacy concerns in surveillance applications.

2. The proposed method’s effectiveness in challenging environments may attract interest from security-related applications, which, while unintended, highlights the importance of responsible deployment.