End-to-End Low-Light Enhancement for Object Detection with Learned Metadata from RAWs

First of all, thank you very much for your professional and thoughtful review. We will do our best to incorporate your suggestions to further improve the quality of our manuscript. Regarding your concerns, we hope the following point-by-point responses adequately address them.

Q1. Concerns regarding the contribution of the constructed dataset

Ans.: Thank you for your comments. First, we would like to emphasize that the core contribution of our work lies in efficiently extracting the minimal yet essential RAW information required by downstream machine vision tasks, with careful consideration of transmission overhead. The dataset serves as just one component supporting this broader objective.

Regarding the dataset, our primary motivation was to create a resource for cross-dataset validation, which is crucial not only for our method but for all RAW-based object detection approaches. To the best of our knowledge, at the time of submission, only one other real-night RAW dataset with three overlapping object categories existed—ROD (Toward RAW Object Detection : A New Benchmark and A New Model, CVPR 2023). However, it only contains RAW images with corresponding annotations (no sRGB counterpart), preventing it from rigorous cross dataset evaluation. In this context, our constructed dataset fills an immediate gap and provides a meaningful contribution to the low light research community.

Q2. Provides discussion or evaluation on RAW data from sensors with different CFAs

Ans.: Thanks for your constructive and insightful suggestions. First, we agree that camera-specific details, such as the CFA pattern, are indispensable for fidelity-oriented RAW reconstruction tasks. However, our objective is different: we extract only the metadata that downstream machine vision models require but cannot obtain from sRGB images, and we deliberately omit fidelity or perceptual supervision to minimize the amount of RAW information that needs to be transmitted. Accordingly, the MV IR module applies an inverse gamma step followed by a learnable remapping to recover the information suppressed by gamma correction and compression, which depend only minimally on the camera model. In this way, our method achieves satisfactory performance with little to no reliance on the full set of camera-dependent parameters.

However, at the current stage, we are not able to obtain such an open source dataset to conduct a rigorous cross dataset evaluation, and we appreciate your further suggestions.

Moreover, we sincerely appreciate your constructive suggestions. As noted in our paper, the construction of the RID dataset is an ongoing project, and we plan to incorporate additional subsets captured with diverse smartphones (e.g., HUAWEI and iPhone) to further contribute to the broader machine vision community.

Q3. Comparing with other RAW-based enhancement methods.

Ans.: Thanks a lot for your suggestions. First, we acknowledge that recent RAW-based methods have demonstrated overwhelming advantages over sRGB-based approaches; we will incorporate more RAW-based methods in the related-works section and experiments of our manuscript. In fact, we have already included the cutting-edged RAW-based enhancement method RAOD (CVPR 2023) which is particular designed for object detection tasks, and our approach achieves comparable performance while offering unique advantages in terms of transmission overhead.

Meanwhile, we would like to emphasize that in comparisons with RAOD and other RAW-based methods, our focus is not solely on achieving higher performance, which is unrealistic given that those approaches have full access to the entire RAW data. Our primary objective is to minimize the transmission and storage requirements of RAW-based pipelines, enabling broader deployment in edge-to-cloud scenarios. Therefore, evaluating methods based solely on downstream accuracy constitutes an unfair comparison, especially considering that RAW images typically require 48 bpp for storage and transmission.

Following your suggestion, we have added two RAW-based methods, AdaptiveISP (Learning an Adaptive Image Signal Processor for Object Detection，NeurIPS2024) and SID (Learning to See in the Dark, CVPR 2018). Due to time and computational constraints, we are only able to report results on the YOLOv3 backbone.

However, quite surprisingly, our method still outperforms these newly incorporated RAW based methods, which can be mainly attributed to the analytical design of our MV-IR module. It establishes a clear and efficient restoration path built upon prior knowledge of the ISP pipeline.Note that RAWformer outputs RAW images, which would require substantial modifications to downstream detectors; therefore, we do not include it in this comparison.We appreciate your understanding and will include more comprehensive comparisons with cutting-edge RAW-based methods in the revised manuscript.

Q4. Compare the computational overhead and inference time.

Ans.: Thanks for your valuable and constructive suggestions. It is indeed important and necessary to analyze the running speed and computational overhead. As our method is mainly proposed for reducing the transmission cost in edge-to-cloud scenarios, where the source images (sRGB or RAW) also need to be compressed and transmitted, we benchmark both stages separately for a fair assessment of runtime and computational overhead. The resulting speed and computational-cost statistics are reported below.

1)Imaging end: We compare the CMCE encoder with extra two image codecs (Encoder part only): TIC (Transformer-based Image Compression, DCC 2022), and MLIC++ (Linear-Complexity Multi-Reference Entropy Modelling, arXiv 2023).;

2)Application end: we evaluate the MV-IR module against the five enhancement methods already employed in our manuscript.

First of all, we want to express our sincere appreciation for your responsible and thoughtful review. Your suggestions not only help improve the quality of our current manuscript but also inspire our future work. Below, we answer your questions point-by-point in the hope of addressing your concerns, and we welcome any further suggestions.

Q1: Concerns about the effectiveness of the proposed modules and the ablation study settings.

Ans.: Thank you for your valuable and insightful suggestions, which made us realize that our current ablation studies are insufficient to demonstrate the effectiveness of our design.

For the ablation of CMCE, we conducted an additional study that replaces the referenced sRGB images with plain images (containing no information) at the encoding side, while keeping all other components identical. This eliminates cross-modal correspondence during RAW-metadata compression, and we retrained the entire framework on the YOLO-V3 backbone. The results are listed below:

As clearly shown, omitting the correspondence between the sRGB and RAW images greatly increases the transmission-resource requirement and results in a remarkable performance drop, confirming that CMCE successfully leverages cross-modal redundancy. As for MV-IR, we conducted comprehensive ablation studies by alternately disabling the remapping and inverse-mapping steps to demonstrate their effectiveness on the YOLOv3 backbone; the results are shown below.

Thank you again for your insightful suggestions. We will incorporate these new ablation studies into the revised manuscript.

Q2. Evaluate the effectiveness on recent detectors.

Ans.: Thank you for your suggestions. In response, we have added the cutting-edge object detector YOLOv11 as a new baseline and compared it with other restoration-then-detection schemes. The corresponding results are listed below.

The results show that our method still obtains remarkable advantages over these enhancement-then-detection methods on recent detectors. We will include additional results using other SOTA detectors in the supplementary materials of the revised manuscript, as this is essential for a comprehensive demonstration of our method’s effectiveness. Thanks again for your valuable suggestion.

Q3. Comparison with recent enhancement models

Ans.: Thanks a lot for your suggestions. Following your suggestion, we have added two RAW-based enhancement methods AdaptiveISP (Learning an Adaptive Image Signal Processor for Object Detection，NeurIPS 2024) and SID (Learning to See in the Dark, CVPR 2018) for a more comprehensive comparison. Owing to time and computational-resource constraints, we are able to report results only on the YOLOv3 backbone; these results are shown below. We appreciate your understanding.

Herein, we’d like to emphasis that compared with exiting RAW-based methods, we are not focused solely on achieving higher performance, which would be unrealistic because those approaches have full access to the entire RAW data. Our primary goal is to minimize the transmission and storage requirements of RAW-based pipelines so they can be deployed widely in edge-to-cloud scenarios, given that RAW images typically require 48 bpp for storage and transmission.

However, quite surprisingly, our method still outperforms these newly incorporated RAW based methods, which can be mainly attributed to the analytical design of our MV-IR module. It establishes a clear and efficient restoration path built upon prior knowledge of the ISP pipeline. Moreover, we will incorporate more recent RAW-based methods into our related-work section to better contribute to the community.

Q4. Concerns about the generalization evaluation across different cameras and the validity of cross-camera comparisons.

Ans.: Thanks a lot for your kind and valuable suggestions. Herein, we would first like to emphasize that both the camera we used and the one used for the LOD dataset are Canon cameras. Although they are different models, the RAW images were captured at the same format. Moreover, our model is designed to extract only the essential metadata from the information that is discarded by the ISP. Since most information loss in the ISP occurs during gamma correction and compression (processes that depend only minimally on the camera model) the MV-IR module applies an inverse-gamma operation followed by a learnable remapping to recover the information lost. From this perspective, we hold a positive attitude toward our model’s cross-camera generalization capability.

Q5. Explanation for not utilizing exposure time and ISO

Ans.: Thanks for your comments. First, we acknowledge that exposure time and ISO are important and are widely used in RAW reconstruction and low-light enhancement studies which pursues high reconstruction fidelity or perceptual quality. However, our method focuses on fulfilling machine-vision needs while minimizing transmission cost, as we apply no fidelity or perception-oriented supervision.

Moreover, we would like to claim that with the access to both the sRGB images and the RAW counterpart, these parameters can be easily derived analytically, in other words, the obtained sRGB images have already contained such information.

Accordingly, we do not employ these camera parameters directly; instead, we rely on end-to-end training to automatically extract the most essential metadata.

Q6. Concerns regarding the unexpectedly low mAP75 performance of RAOD

Ans.: Thanks a lot for your comments. According to our analysis, the discrepancy stems from the training preferences of different object detection models. Factors such as higher NMS thresholds, the choice of GIoU/DIoU/CIoU losses, and bounding box regression bias can influence mAP50 and mAP75 differently. Since most detectors prioritize the mAP50 metric, their hyperparameters are often tuned to favor it. As a result, performance comparisons between mAP50 and mAP75 may exhibit different trends. Moreover, we strictly adhere to the principle of reproducibility and will open-source all checkpoints and training scripts for the baseline models.

Q7. Concerns regarding the accuracy of the reported bpp values

Ans.: Thanks for your kind comments. We apologize for any confusion caused by our unclear representation. In Tables 1–3 of our manuscript, the bpp value accounts only for the additional bits required to transmit the RAW metadata. In fact, as noted in lines 263 and 272 of our manuscript, the bpp values for the sRGB and RAW images are 0.15 and 48, respectively. Therefore, the total cost in our setting is 0.15 (sRGB)+ 4.88e-4(RAW Metadata). We will revise the manuscript to make this point clearer.

Moreover, the bpp values in our manuscript are measured at test time and accurately represent the actual transmission cost in practical deployments. During inference, the RAW metadata generated by the CMCE module is coded into a binary bit-stream and transmitted to the application end. Its bpp is calculated as the bit-stream size (in bits) divided by the image resolution. The bpp for both the sRGB and RAW reference images is computed in the same way, using each file’s size (in bits) divided by its resolution.

Q8. Conclude the limitations of this work

Ans.: Thanks a lot for your kind suggestion. We ‘d like to conclude our limitations as follows: First, as mentioned in Sec. 3.4, our RID dataset is an ongoing project. Inspired by your suggestions, we believe it is necessary to expand the dataset by incorporating images captured from different cameras and smartphones, and to further investigate the variation in RAW metadata requirements across different ISP pipelines. Second, our MV-IR module is currently optimized for object detection, and its effectiveness for other machine vision tasks (e.g., semantic segmentation) remains unexplored. Creating corresponding annotations and either tailoring the module for specific tasks or developing a generalized version are valuable directions for future work. Third, inspired by your comments, we find it interesting to investigate the differences in required metadata between machine vision needs and human perception–oriented needs. Furthermore, designing a hierarchical structure to represent both machine- and perception-tailored metadata, and incorporating a scalable compression strategy to fulfill both requirements, would be an interesting and valuable direction for extending our current work.

Q9. Minor issues related to typos and missing references

Ans.: Finally, we would like to express our deepest gratitude for your comprehensive and responsible review, which has greatly helped improve the quality of our manuscript. We will revise the relevant sections accordingly, incorporate the necessary references, and conduct thorough proofreading to avoid such issues.

First of all, we would like to express our deepest gratitude for your positive attitude and valuable suggestions, which are very important and a great encouragement to us. We hope that our following responses adequately address your concerns.

Q1. Explain the bpp calculation.

Ans.: Thanks for your thoughtful review, and we apologize for any confusion caused by the missing details. The bpp values in our manuscript are measured at test time and accurately represent the actual transmission cost in practical deployments. During inference, the RAW metadata generated by the CMCE module is coded into a binary bit-stream and transmitted to the application end. Its bpp is calculated as the bit-stream size (in bits) divided by the image resolution. The bpp for both the sRGB and RAW reference images is computed in the same way, using each file’s size (in bits) divided by its resolution.

Q2. Compare the computational cost against other models.

Ans.: Thanks for your valuable and constructive suggestions. It is indeed important and necessary to analyze the running speed and computational overhead, which are critical in real-world applications. As our method is mainly proposed for reducing the transmission cost in edge-to-cloud scenarios, where the source images (sRGB or RAW) also need to be compressed and transmitted, we benchmark both stages separately for a fair assessment of runtime and computational overhead.All GPU experiments run on an NVIDIA RTX 6000. The resulting speed and computational-cost statistics are reported below.

1)Imaging end: We compare the CMCE encoder with extra two image codecs (Encoder part only): TIC (Transformer-based Image Compression, DCC 2022), and MLIC++ (Linear-Complexity Multi-Reference Entropy Modelling, arXiv 2023).;

2)Application end: we evaluate the MV-IR module against the five enhancement methods already employed in our manuscript.

Q3. Concerns of the absence of YOLA and RAOD in ablation studies.

Ans.: Thanks for your kind suggestions. We apologize for any misunderstanding caused by our unclear representation. To address your concern, we would like to clarify that in Tables 1–3 we compare the commonly employed restoration-then-detection paradigm: that first apply a low-light enhancement model to obtain high-quality images, then feed these images to downstream object-detection models. In fact, both YOLA and RAOD are machine-vision-tailored low-light enhancement methods; thus, we did not employ them in the ablation study. After carefully proofreading the manuscript, we realized that our statement in line 235 may mislead readers, and we will modify the corresponding part in the revised manuscript to avoid this issue.

First of all, we would like to express our sincere appreciation for your responsible and comprehensive review. Your positive attitude toward our RAW-metadata-guided, plug-and-play design is a great encouragement to us. Regarding your concerns, we hope the following point-by-point responses address them, and we would appreciate any further suggestions.

Q1. Concerns about the fundamental differences from traditional enhancement methods.

Ans.: Thanks for your comments. First, we would like to emphasize that, compared with traditional Retinex-based methods whose heuristic illumination estimation is agnostic to camera settings and recovers only approximate reflectance, our approach begins in the RAW domain to reconstruct the linear sensor-radiance domain. The inverse-gamma and remapping operations are derived analytically from the RAW images, making the mapping physically interpretable.

Besides our work, recent RAW-based low-light-enhancement methods have become mainstream and demonstrate overwhelming advantages over traditional approaches. Compared with these RAW-based methods, we focus on practical edge-to-cloud scenarios, where RAW images cannot be directly accessed, and aim to minimize transmission cost by identifying and extracting only the most essential RAW data.

For intuitive demonstration, we include two Retinex-based methods, one conventional method LIME (LIME: Low-Light Image Enhancement via Illumination Map Estimation，TIP 2016) and one learning-based method RetinexFormer (Retinexformer: One-stage Retinex-based Transformer for Low-light Image Enhancement, ICCV 2023). We first compare their performance with ours on the LOD dataset, then feed their outputs into our pipeline for additional RAW-metadata-guided enhancement; the corresponding results are shown below.

As shown, our method outperforms the conventional methods and can further enhance their performance, demonstrating that RAW metadata offers remarkable advantages over these approaches.

Q2. Explain the specifically leveraged specific cross-modal redundancies and improvement upon standard learned image compression.

Ans.: Thanks for your comments. Cross-modal redundancy denotes the contextual correspondence between a RAW image and its compressed sRGB counterpart, both of which are available at the imaging end. Because the sRGB image is a degraded yet closely correlated representation of the same scene, the two modalities share substantial mutual information. This allows our method to transmit only a small amount of RAW-specific data, since the decoder already possesses the sRGB frame. In contrast to standard learned image codecs, which handle a single modality and remove only intra-image redundancy, we take as input both RAW and sRGB and improve the entropy model to exploit their correspondence, thereby further reducing the required bitrate.

Moreover, our method does not aim for signal-level fidelity, meaning it does not reconstruct the RAW image on the decoding side. The end-to-end strategy uses compactness and task-specific performance as its constraints, so it extracts only the key information needed by downstream tasks. These task-oriented adaptations, compared with existing learned image codecs, enable an extremely low bitrate.

Q3. Evaluate the effectiveness on recent detectors.

Ans.: Thanks a lot for your kind suggestion. Due to time and computational-resource limitations, we were only capable of adopting only YOLOv11 (-x version) as the extra benchmark, we hope your understanding. Comparisons were conducted on the LOD dataset, with all settings kept exactly the same as in our manuscript for fair comparison; the results are shown below.

The results show that our method still obtains remarkable advantages over these enhancement-then-detection methods on recent detectors. We will include additional results using other SOTA detectors in the supplementary materials of the revised manuscript, as this is essential for a comprehensive demonstration of our method’s effectiveness. Thanks again for your valuable suggestion.

Q4. Analyze the runtime and computational overhead.

Ans.: Thanks for your valuable suggestions, it is indeed important and necessary to analyze the running speed and computational overhead for reference.

As our method is mainly proposed for reducing the transmission cost in edge-to-cloud scenarios, where the source images (sRGB or RAW) also need to be compressed and transmitted, we benchmark both stages separately for a fair assessment of runtime and computational overhead.All GPU experiments run on an NVIDIA RTX 6000. The resulting speed and computational-cost statistics are reported below.

1)Imaging end: We compare the CMCE encoder with extra two image codecs (Encoder part only): TIC (Transformer-based Image Compression, DCC 2022), and MLIC++ (Linear-Complexity Multi-Reference Entropy Modelling, arXiv 2023).;

2)Application end: we evaluate the MV-IR module against the five enhancement methods already employed in our manuscript.

Q5. Concerns about the performance gains to both YOLA (sRGB-based) and RAOD (raw-based) methods. 

Ans.: Thanks for your comments. 1)As for for the sRGB-based method YOLA, we outperform it in 17 of 18 comparison groups (2 datasets $\times$ 3 backbones $\times$ 3 metrics); notably, in cross-dataset testing our method achieves a 4 % mAP50 gain across all three backbone detectors, clearly demonstrating the effectiveness of leveraging RAW information.

Q6. Moreover, compared with RAOD and other RAW-based methods, we are not focused solely on achieving higher performance, which would be unrealistic because those approaches have full access to the entire RAW data. Our primary goal is to minimize the transmission and storage requirements of RAW-based pipelines so they can be deployed widely in edge-to-cloud scenarios. Hence, evaluating only downstream accuracy is an unfair comparison, given that RAW images typically require 48 bpp for storage and transmission.