GT-Mean Loss: A Simple Yet Effective Solution for Brightness Mismatch in Low-Light Image Enhancement

Thank you for reading, we respond to them one by one.

•  W (a):      We acknowledge that there are different opinions on the use of GT-Mean metrics in LLIE community. Some researchers oppose this technique, arguing that GT-Mean metrics involve post-processing that may lead to ground truth (GT) information leakage.

     However, our proposed GT-Mean Loss does not share the same concerns as GT-Mean metrics. Unlike GT-Mean metrics, GT-Mean Loss does not perform GT-Mean adjustment on the model’s initial input. Instead, it applies GT-Mean adjustment only during the computation of the loss function. We kindly ask you to consider the fundamental differences.

    Finally, we argue that GT-Mean metrics have merit because they allow comparisons of image quality at a same brightness level, eliminating the impact of brightness mismatch on evaluation metrics. One of the goals of our paper is to advocate for the LLIE community to report both standard metrics and GT-Mean metrics simultaneously to enable a more comprehensive evaluation of low-light image quality.

• W (b): About the Generalization of GT-Mean Loss to out-of-distribution scenes.     To address this concern, we now include supervised dataset metrics like LPIPS and QALIGN and unsupervised dataset metrics like MUSIQ and QALIGN in Appendix D (or see in our comments for reviewer 5 DCR). The results based on new metrics on unpaired datasets show that our improvements also apply to the out of distribution case, not due to the bias on the training dataset.

    Additionally, we provide detailed metrics for the images discussed in lines 1123-1129:

    - For LLFormer (baseline):

        - MUSIQ $\uparrow$: 64.073

        - QALIGN $\uparrow$: 4.078125 (IQA) / 2.885 (IAA)

    - For LLFormer with GT-Mean Loss (ours):

        - MUSIQ $\uparrow$: 65.889

        - QALIGN $\uparrow$: 4.145 (IQA) / 2.898 (IAA)

    Our method achieves better results than the baseline across all metrics (higher MUSIQ and QALIGN scores indicate better performance).

    From a visual perspective, the baseline output exhibits color distortion, with a slight greenish tint on the walls, while our method produces more natural results.

• W (C) and Q:

    Thank you for the suggestion. We have removed Algorithm 1 to make the paper more concise and added a more in-depth analysis of $W$ in Appendix A.

• W (D):

    We have now provided the code and checkpoints. https://anonymous.4open.science/r/Retinexformer-FEC9/

Looking for your feedback

Thanks for your comments and suggestions. We respond to them one by one.

• W 1:

it is unclear as to how brightness mismatch impacts the model training as mentioned

In fact, we demonstrated in the 'Impact on Training' section of the Introduction that the loss value due to brightness mismatch does not match the actual image quality, which affects the training process. Additionally, in Section 4.3 and Figures 4 (a) and 4 (b), we provided the actual training curves of the model, showing the influence of GTmeanloss on the training process.

• W 2:

Thank you for your suggestion. We have revised the expression to improve the understanding of the article.
In lines 99-101, we mentioned the definition of the GTmean metric to align with the reading sequence. Now, we have included the definition of the GTmean metric in the caption of Figure 2.

Specific changes:
For the scaled image, GT-mean PSNR is computed as $PSNR(\frac{E[GT]}{E[GT\times0.8]}GT\times0.8,GT)$, resulting in an infinite PSNR value (In practice, PSNR usually has a pre-defined upper bound. In this figure, we strictly follow the mathametical definition of PSNR for demonstration purposes).

• W 3:

It is worth noting that perceptual loss (or color fidelity losses) typically requires joint use with pixel-level loss (L1/MSE-like) to achieve pixel-level predictions for images. Using these perceptual losses alone is challenging for pixel-level predictions. However, pixel-level loss, due to its lack of perceptual capabilities, is affected by brightness mismatch.

In our experiments, we chose SNR and CIDnet as baselines, both of which use perceptual loss (as shown in Table 2). After extending their losses to GTmean loss, we observed a significant performance improvement. This demonstrates that even after using perceptual loss, it is difficult to eliminate the negative impacts caused by incorrect pixel-level loss. GTmean loss, however, eliminates this impact and achieves performance improvement.

• W 4:

Thank you for your suggestion. We have rephrased the sentence as follows: In this stage, the GT mean loss primarily compares image differences at the same mean brightness to avoid the negative effect of brightness mismatch, thereby maintaining effective model training.

• W 5:

The specific form of the GT-mean loss depends on the original loss $L(\cdot)$, as shown in Eq 1. We only require that the original loss is a supervised loss (with the input image and corresponding ground truth), and there are no additional constraints on $L(\cdot)$. For example, the L1 loss can be extended to the corresponding GT-mean L1 loss.

• W 7：

Since we did not modify any structures of the original baseline, there are no significant visual differences in supervised datasets, with improvements mainly observed in statistical metrics. Our method shows significant visual improvements in unsupervised datasets, demonstrating the good generalization of GT-mean loss. Additionally, we now provide more metrics to prove the effectiveness of our method (see in Appendix D or our comments for Reviewer 5DCR) and offer an anonymous code repository for quick reproduction of GT-mean loss. Welcome to try GT-mean loss in https://anonymous.4open.science/r/Retinexformer-FEC9/

Looking forward to your feedback

Thanks for your comments and suggestions. We respond to them one by one.

• W1:

The low-light enhancement (LLIE) community currently holds divided opinions on the GT-mean metric. Some researchers argue that it is merely a technique unworthy of broader adoption (a point of contention we share with reviewer wA78).

In this paper, we demonstrate the effectiveness of the GT-mean metric and highlight its value as a robust supplement to existing reference-based evaluation metrics. As part of our contribution, we advocate for the LLIE community to simultaneously report both conventional metrics and GT-mean metrics. Reporting results based on both metric types ensures a more comprehensive evaluation of low-light image quality and addresses the concerns surrounding fairness and reproducibility in LLIE benchmarking.

We hope this perspective aligns with the broader interests of the LLIE community. Thank you for raising this important discussion!

• W2 and Q 2:

The average brightness distribution is not derived from the statistics of multiple images but rather represents an assumption about the mean brightness of a single image . We propose that a single image may exhibit varying levels of average brightness, and the probabilities of these potential average brightness levels follow a Gaussian distribution.

• Q 1:

Thank you for your suggestion, which has significantly improved the quality of our paper. The design of $W$ was inspired by the construction of an intermediate distribution in JSD, which involves calculating KL divergence twice. However, since JSD does not have an analytical form, it cannot be computed efficiently for $W$.

To address this, we designed $W$ in a way that provides an analytical form, enabling faster computation. Additionally, we discovered a connection between $W$ and the Bhattacharyya distance, which we discuss in detail in Appendix A.

For convenience, the connection between $W$ and the Bhattacharyya distance is as follows:

Let $p (x)=\mathcal{N}(\mu_1, \sigma_1)$ and $q (x)= \mathcal{N}(\mu_2, \sigma_2)$. According Eq. 5, we have: $W = \frac{1}{2} \left[ \log \frac{\sigma_m}{\sigma_1} + \frac{\sigma_1^2 + (\mu_1 - \mu_m)^2}{2 \sigma_m^2} - \frac{1}{2} \right] + \frac{1}{2} \left[ \log \frac{\sigma_m}{\sigma_{2}} + \frac{\sigma_{2}^2 + (\mu_{2} - \mu_m)^2}{2 \sigma_m^2} - \frac{1}{2} \right]. (8)$ where $\mu_m = \frac{\mu_{1}+\mu_{2}}{2}$, $\sigma_m^2 = \frac{\sigma_{1}^2+\sigma_{2}^2}{2}$. We proof Eq. 8 can be written as $W=\frac{1}{4}\frac{(\mu_1-\mu_2)^2}{\sigma_1^2+\sigma_2^2}+\frac{1}{2}\log\left(\frac{\sigma_1^2+\sigma_2^2}{2\sigma_1\sigma_2}\right), (9)$ where Eq. 9 is also the closed form of the Bhattacharyya distance after two one-dimensional Gaussian distributions [1].

Proof. We can extend the first term in as:

\frac12\left[\log\frac{\sigma_m}{\sigma_{2}}+\frac{\sigma_{2}^2+(\mu_{2}-\mu_m)^2}{2\sigma_m^2}-\frac12\right] =\frac12\log\frac{\sigma_m}{\sigma_{2}}+\frac12\cdot\frac{\sigma_{2}^2+(\mu_{2}-\mu_m)^2}{2\sigma_m^2}-\frac14. (11)

\frac12\log\frac{\sigma_m^2}{\sigma_1\sigma_2}+\frac12\cdot\frac{\sigma_{1}^2+\sigma_{2}^2+(\mu_{1}-\mu_m)^2+(\mu_{2}-\mu_m)^2}{2\sigma_m^2}-\frac12.(12) Let $p (x)=\mathcal{N}(\mu_1, \sigma_1)$ and $q (x)= \mathcal{N}(\mu_2, \sigma_2)$. Due to $\mu_m = \frac{\mu_{1}+\mu_{2}}{2}$, $\sigma_m^2 = \frac{\sigma_{1}^2+\sigma_{2}^2}{2}$, Eq. 12 can be written as: \frac 12\log\frac{\sigma_m^2}{\sigma_1\sigma_2}+\frac 12\cdot\frac{\sigma_{1}^2+\sigma_{2}^2+ \frac{(\mu_{1}-\mu_{2})^2}{2}}{2\sigma_m^2}-\frac 12\ $$ =\frac 12\log\frac{\sigma_{1}^2+\sigma_{2}^2}{2\sigma_1\sigma_2}+\frac 12\cdot\frac{\sigma_{1}^2+\sigma_{2}^2+ \frac{(\mu_{1}-\mu_{2})^2}{2}}{\sigma_{1}^2+\sigma_{2}^2}-\frac12 $$=\frac 14\frac{ (\mu_{1}-\mu_{2})^2}{\sigma_{1}^2+\sigma_{2}^2}+\frac 12\log\frac{\sigma_{1}^2+\sigma_{2}^2}{2\sigma_1\sigma_2}$$ This concludes the proof.

[1] Ravi Kashyap,The perfect marriage and much more: Combining dimension reduction, distance measures and covariance, Physica A: Statistical Mechanics and its Applications, 2019.

• Q3:

Thank you for your valuable suggestions. We have provided additional quantitative results in Appendix D (or refer to our comments for reviewer 5DCR). For datasets such as LOLv1 and LOLv2, we report LPIPS and Q-ALIGN (IQA\IAA). For no-reference datasets like DICE, MEF, LIME, NPE, and VV, we provide MUSIQ and Q-ALIGN (IQA\IAA).

The newly added metrics demonstrate that our method consistently outperforms the baseline across various benchmarks, further validating its effectiveness in enhancing image quality.

Looking for your feedback

Thanks for your comments and suggestions. We respond to them one by one.

• W 1:

In Fig. 2, it seems more like a metric problem. If PSNR is not sensitive to this mismatch, why can the proposed loss function improve the PSNR performance?

The GT-Mean Loss addresses image degradation from two perspectives: The first term ensures that the output image $f(x)$ closely matches the ground truth (GT). The second term ensures that $\lambda f(x)$ (where $\lambda$ is a scalar) closely matches the GT after brightness adjustment.

Since $\lambda f(x)$ is a linear combination of $f(x)$ , the optimal values for both loss terms occur when $\lambda =1$ and $f(x)=GT$. Even though PSNR may not directly capture brightness mismatch, the minimization from these two aspects ensures a better overall match between the output and the ground truth, which ultimately improves PSNR.

When there is a resolution difference between the original GT and the scaled image, how do we compute the PSNR value of the scaled one?

The scaled image in our experiments refers to an image with its brightness scaled by 0.8, not its resolution. To avoid confusion, we have revised the caption for Fig. 2 as follows:

Specific changes: the original image's brightness scaled by a factor of 0.8

Moreover, how is the brightness mismatch problem illustrated in Fig. 2?

The PSNR of the noisy image is higher than that of the scaled image, but this does not align with the actual image quality. This discrepancy occurs because the brightness difference between the scaled image and the ground truth (GT) is too large (brightness mismatch), causing PSNR to fail to capture the true image quality.

• W 2:

The paper couples the original loss and the introduced one using W. How do we confirm that these two loss functions have any relationships?

The first term of the GT-Mean Loss is the original loss applied to the model's output. The second term calculates the original loss again after adjusting the model's output via GT-Mean. The weighting factor $W$ dynamically determines the importance of each term in training process.

Effectively, the GT-Mean Loss calculates the original loss twice (once for the raw output and once for its GT-Mean adjustment), ensuring the preservation of the original loss's functionality while mitigating the effects of brightness mismatch.

• W 3:

Tab. 1 shows that perceptual loss can also identify the scaled image as better. Can this loss make a more significant improvement than the proposed one when applied to algorithms that initially did not use this loss?

It is important to note that our method and perceptual loss can be used together to achieve even better performance. In our baseline experiments (e.g., SNR and CIDNET), perceptual loss is already included. By extending these baselines with GT-Mean Loss, we achieved further improvements. This demonstrates that GT-Mean Loss is complementary to perceptual loss and can enhance performance when used together.

• W 4:

Some typos. Lv1 -> LOLv1. The citation for ZeroDCE is missing.

Thank you for carefully reviewing the manuscript. We have corrected the typos and added the missing citation for ZeroDCE in the revised version.

Looking for your feedback

• Q2: Our method actually enables the original baseline to better handle complex image degradation factors. Specifically, the first term in Eq. 1 ensures that the baseline retains its original performance, while the second term reduces the impact of brightness mismatch, allowing the baseline to better address these complex degradation factors. This is because, after GT-mean adjustment, the model learns to repair degradations at the same brightness level.

• Q3: The computational cost of GT-Mean Loss is almost negligible:

  1. We provide a closed-form solution for $W$, enabling its fast computation.
  2. The second term in GT-Mean Loss is essentially the GT-mean-adjusted version of the original loss, with identical computational complexity. The original loss computation is minimal compared to the model’s forward and backward passes. To support this, we have provided an anonymized code repository: https://anonymous.4open.science/r/Retinexformer-FEC9/.
     For instance, training Retinexformer on LOLv 1 with a 4090 GPU takes 195 minutes for the baseline and only 199(+4) minutes when using GT-Mean Loss.

  GT-Mean Loss is widely applicable to various existing methods without altering the baseline model structure. Despite its minimal cost, it delivers notable improvements: on LOLv 1/v 2, the average PSNR increases by +0.617, and SSIM improves by +0.013. These results are substantial given the simplicity and generalizability of our approach.

• Q 4: Due to time constraints, we were unable to conduct a user study. However, we have provided Q-ALIGN metrics for all results to simulate human perception. The results indicate that our method is generally effective across all datasets.

• Q 5: Thank you for the suggestion. We have revised the captions in Appendix E for the visualized results to include more detailed descriptions. We hope this improves clarity.

  Specific changes:

  "The methods based on GT-Mean Loss achieve better exposure control in images 2-3, 8, and 10-13, enhancing the details in the dark areas to an appropriate level, while the baseline exhibits artifacts and color distortion due to overexposure in these images. In images 4 (the road in the lower left corner), 9 (the roof), and 14 (the palette), methods based on GT-Mean Loss provide more accurate colors. Additionally, methods based on GT-Mean Loss significantly suppress artifacts in images 1 and 5-8."

• Q 6 In fact, our motivation is clearly stated in the sixth paragraph of the Introduction:
  "The issue of model training under brightness mismatch has largely been ignored in existing supervised LLIE research, despite some indirect solutions..."

Looking forward to your feedback

GT-Mean Loss outperforms in all average metrics than the baseline on unpaired datasets.

GT-Mean Loss outperforms in almost all metrics on paired datasets (except for strikethrough).

Thank you for reading, your insights on image quality evaluation have enhanced the quality of our articles and now we are responding to your queries one by one.

• W and Q 1: Thank you for your valuable suggestion. In Appendix D, we provide additional quantitative results. For datasets like LOLv 1 and LOLv 2, we report LPIPS and Q-ALIGN (both IQA and IAA). For datasets without reference images, such as DICE, MEF, LIME, NPE, and VV, we report MUSIQ and Q-ALIGN (IQA and IAA). The new metrics consistently demonstrate that our method outperforms the baseline, further validating its effectiveness in improving image quality.

  For convenience, the results based on new metrics are as follows:

The authors have addressed some of my concerns.