You Only Look Around: Learning Illumination-Invariant Feature for Low-light Object Detection

We sincerely thank you for your insightful and positive comments.

---

How does the method perform in comparison to recent related methods?

We complement more detailed comparison experiments as presented in Table 1, which includes runtime, model size, and performance. Note that our YOLA still achieves the best performance and speed among the evaluated methods.

Besides, ReForDe[7] didn’t release their detailed training data, so we can not implement it. However, compared to the improvements reported by ReForDe in their paper ($\leq$0.2 mAP on ExDark using YOLOv3), our YOLA demonstrates a more substantial enhancement, achieving improvements of 1.7 mAP and 2.7 mAP on YOLOv3 and TOOD, respectively.

Additionally, FeatEnHancer[1] didn't release their code, we thus follow the FeatEnHancer’sexperimental setting to implement the RetinaNet-based detectors as shown in Table 2. We can see that even though our baseline implementation on the ExDark dataset is inferior to FeatEnHancer’s, the integration of YOLA enables our method to achieve the best performance (1.9 mAP significant improvement compared to baseline). For DarkFace dataset, FeatEnHancer decreases the baseline performance by 0.1 mAP, which is attributed to hierarchical features that failed to be captured by RetinaNet, as claimed in [1]. In contrast, our YOLA, triggered from the physics-based model perspective without elaborate design, surpasses the baseline with a remarkable improvement of 2.5 mAP. It strongly suggests the generalizability and effectiveness of YOLA.

---

The proposed method assumes neighboring pixels exhibit high similarity of illumination. However, how does it hold true when meeting uneven light at night?

In the main text, we assume that the illumination values of neighboring pixels are equal, which allows us to eliminate the influence of illumination when extracting features. However, in cases of uneven illumination, this assumption's constraint is weakened but still helps reduce the impact of illumination, as shown in teaser (b). Our method is detection-driven and can further mitigate the influence of such uneven illumination during the learning process, as illustrated in teaser (d). Additionally, please refer to the appendix. When the actual distance between neighboring pixels is too large, the assumption may not hold. Therefore, we propose IIloss to constrain the extraction of illumination-invariant features to as close a region as possible, mitigating the impact of uneven lighting.

---

Table 1: Quantitative comparisons of the ExDark dataset based on TOOD detectors

---

Table 2: Quantitative comparisons (YOLA vs. FeatEnHancer), $\dagger$ indicates our implemented baseline.

---

Reference:

[1] Khurram et al. Featenhancer: Enhancing hierarchical features for object detection and beyond under low-light vision, ICCV 2023.

[2] Cui et al. You only need 90k parameters to adapt light: a light weight transformer for image enhancement and exposure correction. BMVC 2022.

[3] Liu et al. Image-adaptive yolo for object detection in adverse weather conditions. AAAI 2022.

[4] Sanket et al. Gdip: Gated differentiable image processing for object detection in adverse conditions. ICRA 2023.

[5] Qin et al. Denet: Detection-driven enhancement network for object detection under adverse weather conditions. ACCV 2020.

[6] Yin et al. Pe-yolo: Pyramid enhancement network for dark object detection. ICANN 2023.

[7] Sun et al. Rethinking image restoration for object detection. NeurIPS 2022.

[8] Cui et al. Multitask aet with oerthogonal tangent regularity for dark object detection, ICCV 2021.

We sincerely thank you for your constructive comments.

---

In the pipeline, the work is basically low-light enhancement + detector. In evaluations, only object detection is evaluated, why not evaluate on low-light enhancement datasets/benchmarks?

First and foremost, we need to emphasize that our framework is designed to enhance the performance of object detectors in low-light scenarios, rather than for visual brightening or denoising .

Human visual alignment can improve detection performance, but it is not the only solution. From a machine vision perspective, detection does not depend on achieving image quality that aligns with human vision. Despite the presence of noise and blurriness in low-light images, this does not hinder our method from improving performance in detection tasks.

Essentially, our method is object detection rather than image enhancement, which means comparing our results based on image enhancement benchmarks is unfair.

More specifically, our framework does not employ any extra low-lit and normal-lit image pairs or other image restoration-related loss to enhance the input images. We only leverage detection loss to guide IIM in producing task-specific illumination invariant features. Therefore, improving visual quality is not our goal. Interestingly, we found that the subsequent enhanced image yielded by FuseNet tends to display increased brightness.

---

In experiments, LLIE methods fail to achieve satisfactory performance due to inconsistency between human visual and machine perception. The enhancement methodologies prioritize human preferences. However, it is important to note that optimizing for enhanced visual appeal may not align with optimized object detection performance. It is hard to understand without visual examples.

As we have shown, some of the images we present are cloudy, which exemplifies the discrepancy between machine vision and human vision. In LLIE (Low-Light Image Enhancement) methods, many loss functions are designed based on human prior assumptions, such as illumination smoothness and color consistency in Zero-DCE, or TV loss in denoising. These losses often lead to the loss of many details while preserving details preferred by human perception. This difference may result in LLIE methods performing poorly in detection tasks, as discussed in related works[1, 2, 3].

---

RG/RB/GB-chromaticity is widely used in computer vision; learning-based methods for intrinsic images or inverse rendering also adopt the constraint. It deals with light intensity but does not try to solve the image noises in low light conditions. Does this paper consider noise reduction in the pipeline?

Illumination invariant features based on chromaticity have ambiguities between colors with the same chromaticity, e.g. dark red or light red. Such limitations are not discussed.

Enhancing image quality, such as reducing noise and improving chromaticity distinction, is a practical strategy, but it is not the only one for improving detection performance. In this work, We are committed to finding an end-to-end approach to enhance downstream detection tasks, rather than focusing on image restoration.

Besides, improving both image quality and detection performance typically requires paired annotated datasets, which presents significant challenges for practical applications. Therefore, we strongly believe that developing simpler and more efficient end-to-end methods to reduce the burden of data annotation will greatly benefit the community.

---

The proposed features are claimed to fit many tasks. More tasks other than detection can be demonstrated.

We present our evaluation of YOLA on the instance segmentation task in Table 1. We report the quantitative comparisons of several advanced LLIE and low-light object methods using Mask R-CNN on the low-light instance segmentation (LIS[10]) dataset. We can see that our YOLA achieves the best performance across all metrics, indicating that YOLA not only facilitates low-light object detection but also low-light instance segmentation. For more visual comparison, please refer to our appendix.

---

Table 1: Quantitative Comparisons.

Reference

[1] Cui et al. Multitask aet with oerthogonal tangent regularity for dark object detection, ICCV 2021.

[2] Khurram et al. Featenhancer: Enhancing hierarchical features for object detection and beyond under low-light vision, ICCV 2023.

[3] Qin et al. Denet: Detection-driven enhancement network for object detection under adverse weather conditions. ACCV 2020.

[4] Yin et al. Pe-yolo: Pyramid enhancement network for dark object detection. ICANN 2023.

[5] Guo et al. Zero-reference deep curve estimation for low-light image enhancement. CVPR 2020.

[6] Jiang et al. Enlightengan: Deep light enhancement without paired supervision. TIP 2021.

[7] Liu et al. Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement. CVPR 2021.

[8] Ma et al. Toward fast, flexible, and robust low-light image enhancement. CVPR 2022.

[9] Yang et al. Implicit neural representation for cooperative low-light image enhancement. CVPR 2023.

[10] Chen et al. Instance segmentation in the dark. IJCV 2023.

We sincerely thank you for your insightful and constructive comments.

As a plug and play module, I think the lighting invariant module should be integrated into more detectors to prove its effectiveness.

We report more detectors as shown in Table 1. It contains the Anchor-based detector (Faster-RCNN[1], RetinaNet[2], YOLOV3[3]), and the Anchor-free one (TOOD[4], Sparse-RCNN[5]). By integrating YOLA, the performances of all detectors are significantly improved, demonstrating the generalization capability of YOLA. Besides, please refer to the appendix, we also evaluate our YOLA on the instance segmentation task, demonstrating YOLA’s effectiveness on Mask R-CNN detector.

---

The authors claim that they learned light invariant features, and then models trained in low light conditions should be able to generalize directly to normal light conditions. And vice versa. The authors should provide more experiments to demonstrate the generalization ability of their light invariant features.

We complement the more generalization experiments as presented in Table 2, where $GAP$ represents mAP$_{50}$ gap between in-domain and cross-domain. We trained detectors separately in low-light (ExDark) and normal-light(Pascal VOC 2012[6]) conditions and then tested them in the opposite conditions. We can see that our YOLA achieves the best performance across different domains and shows a smaller performance drop when tested on cross-domain.

---

The authors should provide evaluation content such as runtime and memory used by the run, it seems that their approach is more lightweight.

We have included a comparison of runtime, model size, and performance in Table 3. The inference speeds, evaluated on an RTX 2080Ti using the MMdetection toolbox, indicate that YOLA introduces the fewest additional parameters (0.008M) while achieving the highest FPS and outstanding performance among the evaluated detectors, excluding the baseline.

---

Table 1: YOLA with more detectors.

---

Table 2: Generalization comparison.

---

Table 3: Quantitative comparisons of the ExDark dataset based on TOOD detectors

---

Reference:

[1] Ren et al. Faster r-cnn: Towards real-time object detection with region proposal networks. NeurIP2015.

[2] Lin et al. Focal loss for dense object detection. ICCV 2017.

[3] Redmon et al. Yolov3: An incremental improvement. ArXiv.

[4] Feng et al. Tood: Task-aligned one-stage object detection. ICCV 2021.

[5] Sun et al. Sparse r-cnn: End-to-end object detection with learnable proposals. CVPR 2021.

[6] Everingham et al. The {PASCAL} {V}isual {O}bject {C}lasses {C}hallenge 2012 {(VOC2012)

[7] Cui et al. You only need 90k parameters to adapt light: a light weight transformer for image enhancement and exposure correction. BMVC 2022.

[8] Liu et al. Image-adaptive yolo for object detection in adverse weather conditions. AAAI 2022.

[9] Sanket et al. Gdip: Gated differentiable image processing for object detection in adverse conditions. ICRA 2023.

[10] Qin et al. Denet: Detection-driven enhancement network for object detection under adverse weather conditions. ACCV 2020.

[11] Yin et al. Pe-yolo: Pyramid enhancement network for dark object detection. ICANN 2023

We would like to thank the reviewer for carefully reading our submission and providing many insightful comments.

The authors assume uniform illumination between neighboring pixels to eliminate the influence of the positional term $m$ in Equation 1. However, images captured in real-world scenes often exhibit uneven lighting. I question the validity of this assumption.

In the main text, we assume that the illumination values of neighboring pixels are equal, which allows us to eliminate the influence of illumination when extracting features. However, in cases of uneven illumination, this assumption's constraint is weakened but still helps reduce the impact of illumination, as shown in teaser (b). Our method is detection-driven and can further mitigate the influence of such uneven illumination during the learning process, as illustrated in teaser (d). Additionally, please refer to the appendix. When the actual distance between neighboring pixels is too large, the assumption may not hold. Therefore, we propose IIloss to constrain the extraction of illumination-invariant features to as close a region as possible, mitigating the impact of uneven lighting.

---

The paper does not provide definitions for the symbols used in Equation 2.

We apologize for the oversight. In the revised version, we will include the corresponding explanations. $B$ denotes the blue channel, whereas $R$ represents the red channel.

---

What does "Baseline" refer to in Tables 1 and 2, and why is the object detection performance based on low-light enhancement methods worse than the Baseline?.

'Baseline' refers to the detector that our method is built upon. The difference between 'Baseline' and 'Ours' is that we utilized the IIM (Illumination Invariant Module). Lowlight image enhancement methods may fail to achieve satisfactory performance due to the task of primarily satisfying human vision instead of machine perception. This phenomenon is also claimed by many related works[1,2,3].

---

Many end-to-end low-light face detection algorithms have been proposed. The authors should compare their method not only with general object detection algorithms but also with these specialized low-light face detection algorithms on the DarkFace dataset.

We explored several specialized low-light face detection algorithms [4,5,6]. However, most of these algorithms are evaluated on the DarkFace test set that does not provide Ground Truth, or lacks open-source code. Consequently, we selected some representative face detection algorithms adopted by these methods as our baselines, including DFSD[7] and PyramidBox[8]. We implemented our YOLA using their default settings for a fair comparison, as shown in Table 1. Our YOLA outperforms the baselines, achieving 0.6 and 0.8 higher mAP on PyramidBox and DFSD, respectively, demonstrating its generalization ability in the face detection task.

---

There are some questions regarding the training loss. The paper does not provide the training loss of the model. Are the components for learning illumination-invariant features and the object detection network trained jointly, or is the illumination-invariant feature learning component trained separately? If trained separately, considering that the ExDark and DarkFace datasets consist only of annotated low-light images without corresponding normal-light images, is this training unsupervised? The authors should provide detailed information about the training process in the paper.

Our YOLA model is trained without any additional image pair annotations using an end-to-end joint training fashion. Specifically, the features produced by the IIM are supposed to be inherently illumination invariant at initialization, based on the Lambertian assumption. Thus, we do not require any additional normal light images or other loss such as Brightness loss to guide its learning. We only employ detection loss to guide the IIM in producing task-specific illumination invariant features for downstream tasks.

---

Table 1: Face detection algorithms on DarkFace Dataset

---

Reference:

[1] Cui et al. Multitask aet with oerthogonal tangent regularity for dark object detection, ICCV 2021.

[2] Khurram et al. Featenhancer: Enhancing hierarchical features for object detection and beyond under low-light vision, ICCV 2023.

[3] Qin et al. Denet: Detection-driven enhancement network for object detection under adverse weather conditions. ACCV 2020.

[4] Wang et al. Unsupervised face detection in the dark. T-PAMI 2022.

[5] Wang et al. Hla-face: Joint high-low adaptation for low light face detection. CVPR 2021.

[6] Yu et al. Single-stage face detection under extremely low-light conditions. ICCVW 2021.

[7] Li et al. Dsfd: dual shot face detector. CVPR 2019.

[8] Tang et al. Pyramidbox: A context-assisted single shot face detector. ECCV 2018.